2013/6/15 Christian Heimes <christian@python.org>:
Am 15.06.2013 14:22, schrieb Nick Coghlan:
However, it's still desirable to be able to monitor those direct allocations in debug mode, thus it makes sense to have a GIL protected direct allocation API as well. You could try to hide the existence of the latter behaviour and treat it as a private API, but why? For custom allocators, it's useful to be able to *ensure* you can bypass CPython's small object allocator, rather than having to rely on it being bypassed for allocations above a certain size.
There is even more to it. We like to keep track of memory allocations in libraries that are wrapped by Python's extension modules, e.g. expat, openssl etc. Almost every library has a hook to set a custom memory allocator, either globally (CRYPTO_set_mem_functions) or for each object (XML_ParserCreate_MM's XML_Memory_Handling_Suite).
I just create the issue http://bugs.python.org/issue18227: "Use Python memory allocators in external libraries like zlib or OpenSSL". Is it possible to detect if Python is used as a standalone application (the classic "python" program) or if Python is embedded? If it is possible, we can modify the "global" memory allocators of a library. Otherwise, it is more tricky. Should Python sets its "own" memory allocators? Maybe only if PyMem_SetRawAllocators() was called?
Eventually I would like to ban direct usage of malloc() from Python's core and patch all memory management through our API.
I already create issue http://bugs.python.org/issue18203 for this part. Victor