[Python-Dev] Modify PyMem_Malloc to use pymalloc for performance
M.-A. Lemburg
mal at egenix.com
Thu Feb 4 05:17:51 EST 2016
On 03.02.2016 22:03, Victor Stinner wrote:
> Hi,
>
> There is an old discussion about the performance of PyMem_Malloc()
> memory allocator. CPython is stressing a lot memory allocators. Last
> time I made statistics, it was for the PEP 454:
> "For example, the Python test suites calls malloc() , realloc() or
> free() 270,000 times per second in average."
> https://www.python.org/dev/peps/pep-0454/#log-calls-to-the-memory-allocator
>
> I proposed a simple change: modify PyMem_Malloc() to use the pymalloc
> allocator which is faster for allocation smaller than 512 bytes, or
> fallback to malloc() (which is the current internal allocator of
> PyMem_Malloc()).
>
> This tiny change makes Python up to 6% faster on some specific (macro)
> benchmarks, and it doesn't seem to make Python slower on any
> benchmark:
> http://bugs.python.org/issue26249#msg259445
>
> Do you see any drawback of using pymalloc for PyMem_Malloc()?
Yes: You cannot free memory allocated using pymalloc with the
standard C lib free().
It would be better to go through the list of PyMem_*() calls
in Python and replace them with PyObject_*() calls, where
possible.
> Does anyone recall the rationale to have two families to memory allocators?
The PyMem_*() APIs were needed to have a cross-platform malloc()
implementation which returns standard C lib free()able memory,
but also behaves well when passing 0 as size.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Experts (#1, Feb 04 2016)
>>> Python Projects, Coaching and Consulting ... http://www.egenix.com/
>>> Python Database Interfaces ... http://products.egenix.com/
>>> Plone/Zope Database Interfaces ... http://zope.egenix.com/
________________________________________________________________________
::: We implement business ideas - efficiently in both time and costs :::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/
http://www.malemburg.com/
More information about the Python-Dev
mailing list