Sorry, I didn't get a coffee yet: more *often* in a venv.
Le mer. 27 févr. 2019 à 11:32, Victor Stinner vstinner@redhat.com a écrit :
Any idea why Python calls mmap+munmap more even in a venv?
Victor
Le mer. 27 févr. 2019 à 10:00, INADA Naoki songofacandy@gmail.com a écrit :
Ah, another interesting point, this huge slowdown happens only when bm_pickle.py is executed through pyperformance. When run it directly, slowdown is not so large.
pyperformance runs benchmarks in a virtual environment. I don't know if it has any impact on bm_pickle.
Most pyperformance can be run outside a virtual env if required modules are installed on the system. (bm_pickle only requires the stdlib and perf.)
Bingo!
Without venv:
unpickle: Mean +- std dev: 26.9 us +- 0.0 us % time seconds usecs/call calls errors syscall
28.78 0.000438 0 1440 read 27.33 0.000416 1 440 25 stat 9.72 0.000148 1 144 mmap ... 0.79 0.000012 1 11 munmap
With venv:
% time seconds usecs/call calls errors syscall
57.12 0.099023 2 61471 munmap 41.87 0.072580 1 61618 mmap 0.23 0.000395 1 465 27 stat
unpickle and unpickle_list creates massive same-sized objects, then all objects are removed. If all pools in the arena is freed, munmap is called.
I think we should save some arenas to reuse. On recent Linux, we may be able to use MADV_FREE instead of munmap.
-- Night gathers, and now my watch begins. It shall not end until my death.