[Python-Dev] Enhancement of Python memory allocators

[Victor Stinner]

2013/6/13 Nick Coghlan <ncoghlan at gmail.com>:
> Yes, that sounds better. One of the biggest problems with the current
> startup sequence is the way it relies on environment variables for
> configuration, which makes life hard for other applications that want to
> embed the CPython runtime.

I wrote a new patch (attached to issue #3329) adding a new
PyMem_SetupDebugHooks() function. So if an application replaces Python
memory allocator functions, it can still can PyMem_SetupDebugHooks()
to benefit of the Python debug hooks detecting memory bugs. The
function does nothing if hooks are already installed or if Python is
not compiled in debug mode. With this function, the API is now
complete for all use cases.

The PEP 432 helps to configure embedded Python, but the new "Set"
functions (ex: PyMem_SetAllocators) are still needed for my
pytracemalloc module which installs hooks at runtime, when Python is
fully initialized (the hooks can be installed anytime). pytracemalloc
is just an example, you may use PyMem_SetAllocators for other debug or
performance purpose.

With my patch, allocator functions like PyMem_Malloc() are no more
macro, and are always the same function. This helps the stable ABI: C
extension modules do not need to be recompiled to benefit of the debug
hooks ;-)

Victor