[Python-Dev] mmap & munmap loop (Was: Compact ordered set
INADA Naoki
songofacandy at gmail.com
Wed Feb 27 03:59:57 EST 2019
>
> > Ah, another interesting point, this huge slowdown happens only when
bm_pickle.py
> > is executed through pyperformance. When run it directly, slowdown is
> > not so large.
>
> pyperformance runs benchmarks in a virtual environment. I don't know
> if it has any impact on bm_pickle.
>
> Most pyperformance can be run outside a virtual env if required
> modules are installed on the system. (bm_pickle only requires the
> stdlib and perf.)
>
Bingo!
Without venv:
unpickle: Mean +- std dev: 26.9 us +- 0.0 us
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
28.78 0.000438 0 1440 read
27.33 0.000416 1 440 25 stat
9.72 0.000148 1 144 mmap
...
0.79 0.000012 1 11 munmap
With venv:
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
57.12 0.099023 2 61471 munmap
41.87 0.072580 1 61618 mmap
0.23 0.000395 1 465 27 stat
unpickle and unpickle_list creates massive same-sized objects, then all
objects are
removed. If all pools in the arena is freed, munmap is called.
I think we should save some arenas to reuse. On recent Linux,
we may be able to use MADV_FREE instead of munmap.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190227/b0c5b6d7/attachment.html>
More information about the Python-Dev
mailing list