Any idea why Python calls mmap+munmap more even in a venv?
Le mer. 27 févr. 2019 à 10:00, INADA Naoki firstname.lastname@example.org a écrit :
Ah, another interesting point, this huge slowdown happens only when bm_pickle.py is executed through pyperformance. When run it directly, slowdown is not so large.
pyperformance runs benchmarks in a virtual environment. I don't know if it has any impact on bm_pickle.
Most pyperformance can be run outside a virtual env if required modules are installed on the system. (bm_pickle only requires the stdlib and perf.)
unpickle: Mean +- std dev: 26.9 us +- 0.0 us % time seconds usecs/call calls errors syscall
28.78 0.000438 0 1440 read 27.33 0.000416 1 440 25 stat 9.72 0.000148 1 144 mmap ... 0.79 0.000012 1 11 munmap
% time seconds usecs/call calls errors syscall
57.12 0.099023 2 61471 munmap 41.87 0.072580 1 61618 mmap 0.23 0.000395 1 465 27 stat
unpickle and unpickle_list creates massive same-sized objects, then all objects are removed. If all pools in the arena is freed, munmap is called.
I think we should save some arenas to reuse. On recent Linux, we may be able to use MADV_FREE instead of munmap.