On 03.02.2016 22:03, Victor Stinner wrote:
There is an old discussion about the performance of PyMem_Malloc() memory allocator. CPython is stressing a lot memory allocators. Last time I made statistics, it was for the PEP 454: "For example, the Python test suites calls malloc() , realloc() or free() 270,000 times per second in average." https://www.python.org/dev/peps/pep-0454/#log-calls-to-the-memory-allocator
I proposed a simple change: modify PyMem_Malloc() to use the pymalloc allocator which is faster for allocation smaller than 512 bytes, or fallback to malloc() (which is the current internal allocator of PyMem_Malloc()).
This tiny change makes Python up to 6% faster on some specific (macro) benchmarks, and it doesn't seem to make Python slower on any benchmark: http://bugs.python.org/issue26249#msg259445
Do you see any drawback of using pymalloc for PyMem_Malloc()?
Yes: You cannot free memory allocated using pymalloc with the standard C lib free().
It would be better to go through the list of PyMem_*() calls in Python and replace them with PyObject_*() calls, where possible.
Does anyone recall the rationale to have two families to memory allocators?
The PyMem_*() APIs were needed to have a cross-platform malloc() implementation which returns standard C lib free()able memory, but also behaves well when passing 0 as size.