[Python-Dev] Compact ordered set

[Victor Stinner]

Le mar. 26 févr. 2019 à 17:33, INADA Naoki <songofacandy at gmail.com> a écrit :
> My company gives me dedicated Linux machine with Core(TM) i7-6700.
> So I think it's not issue of my machine.

Oh great :-)

> perf shows this line caused many page fault.
> https://github.com/python/cpython/blob/c606a9cbd48f69d3f4a09204c781dda9864218b7/Objects/obmalloc.c#L1513
>
> This line is executed when pymalloc can't reuse existing pool and uses new pool.
> So I suspect there is some weak point about pymalloc and adding more hysteresis
> may help it.  But I'm not sure yet.  I'll investigate it later.

You might want to try PYTHONMALLOC=malloc to force the usage of system
malloc() and so disable pymalloc.

You might also try jemalloc with LD_PRELOAD and PYTHONMALLOC=malloc.

Not sure if it helps :-)

> Ah, another interesting point, this huge slowdown happens only when bm_pickle.py
> is executed through pyperformance.  When run it directly, slowdown is
> not so large.

pyperformance runs benchmarks in a virtual environment. I don't know
if it has any impact on bm_pickle.

Most pyperformance can be run outside a virtual env if required
modules are installed on the system. (bm_pickle only requires the
stdlib and perf.)

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.