Am 15.06.2013 14:22, schrieb Nick Coghlan:
However, it's still desirable to be able to monitor those direct allocations in debug mode, thus it makes sense to have a GIL protected direct allocation API as well. You could try to hide the existence of the latter behaviour and treat it as a private API, but why? For custom allocators, it's useful to be able to *ensure* you can bypass CPython's small object allocator, rather than having to rely on it being bypassed for allocations above a certain size.
There is even more to it. We like to keep track of memory allocations in libraries that are wrapped by Python's extension modules, e.g. expat, openssl etc. Almost every library has a hook to set a custom memory allocator, either globally (CRYPTO_set_mem_functions) or for each object (XML_ParserCreate_MM's XML_Memory_Handling_Suite). Python releases the GIL around IO or CPU critical sections of the library. But these sections may call the memory management functions. If these memory functions use the GIL, then some speed ups of releasing the GIL in the first place are lost. It might even be possible to walk into dead lock situations. For that reason it makes sense to have a set of low level memory management functions, that don't rely on the GIL for locking. These functions can still impose their own locking if they need to modify a global state (e.g. allocation statistics). In normal release mode and on most platforms the raw memory allocators should be thin wrappers around malloc(), realloc() and free() -- or perhaps just macros. Eventually I would like to ban direct usage of malloc() from Python's core and patch all memory management through our API. Christian