[Python-Dev] Modify PyMem_Malloc to use pymalloc for performance
victor.stinner at gmail.com
Thu Feb 4 07:29:42 EST 2016
2016-02-04 11:17 GMT+01:00 M.-A. Lemburg <mal at egenix.com>:
>> Do you see any drawback of using pymalloc for PyMem_Malloc()?
> Yes: You cannot free memory allocated using pymalloc with the
> standard C lib free().
That's not completly new.
If Python is compiled in debug mode, you get a fatal error with a huge
error message if you free the memory allocated by PyMem_Malloc() using
PyObject_Free() or PyMem_RawFree().
But yes, technically it's possible to use free() when Python is *not*
compiled in debug mode.
> It would be better to go through the list of PyMem_*() calls
> in Python and replace them with PyObject_*() calls, where
There are 536 calls to the functions PyMem_Malloc(), PyMem_Realloc()
I would prefer to modify a single place having to replace 536 calls :-/
>> Does anyone recall the rationale to have two families to memory allocators?
> The PyMem_*() APIs were needed to have a cross-platform malloc()
> implementation which returns standard C lib free()able memory,
> but also behaves well when passing 0 as size.
Yeah, PyMem_Malloc() & PyMem_Free() help to have a portable behaviour.
But, why not PyObject_Malloc() & PObject_Free() were not used in the
An explanation can be that PyMem_Malloc() can be called without the
GIL held. But it wasn't true before Python 3.4, since PyMem_Malloc()
called (indirectly) PyObject_Malloc() when Python was compiled in
debug mode, and PyObject_Malloc() requires the GIL to be held.
When I wrote the PEP 445, there was a discussion about the GIL. It was
proposed to allow to call PyMem_xxx() without the GIL:
This option was rejected.
More information about the Python-Dev