Le mar. 26 févr. 2019 à 17:33, INADA Naoki songofacandy@gmail.com a écrit :
My company gives me dedicated Linux machine with Core(TM) i7-6700. So I think it's not issue of my machine.
Oh great :-)
perf shows this line caused many page fault. https://github.com/python/cpython/blob/c606a9cbd48f69d3f4a09204c781dda986421...
This line is executed when pymalloc can't reuse existing pool and uses new pool. So I suspect there is some weak point about pymalloc and adding more hysteresis may help it. But I'm not sure yet. I'll investigate it later.
You might want to try PYTHONMALLOC=malloc to force the usage of system malloc() and so disable pymalloc.
You might also try jemalloc with LD_PRELOAD and PYTHONMALLOC=malloc.
Not sure if it helps :-)
Ah, another interesting point, this huge slowdown happens only when bm_pickle.py is executed through pyperformance. When run it directly, slowdown is not so large.
pyperformance runs benchmarks in a virtual environment. I don't know if it has any impact on bm_pickle.
Most pyperformance can be run outside a virtual env if required modules are installed on the system. (bm_pickle only requires the stdlib and perf.)
Victor