
Hi, I would like to remove the "GIL must be held" restriction from PyMem_Malloc(). In my opinion, the restriction was motived by a bug in Python, bug fixed by the issue #3329. Let me explain why. The PyMem_Malloc() function is a thin wrapper to malloc(). It returns NULL if the size is lager than PY_SSIZE_T_MAX and have a well defined behaviour for PyMem_Malloc(0) (don't return NULL). So it is surprising to read in Include/pymem.h: "The GIL must be held when using these APIs." The reason is more surprising: in debug mode, PyMem_Malloc() is no more a thin wrapper to malloc(), but it calls internally PyObject_Malloc(), the "Python allocator" (called pymalloc). (Many other checks are done in debug mode, but it's unrelated to my point.) The problem is that PyObject_Malloc() is not thread-safe, the GIL must be held. Short history: fb45791150d1 (Mar 23 2002) "gives Python a debug-mode pymalloc" f294fdd18b5b (Mar 28 2002) removes the "check API family" e16dbf875303 (Apr 22 2002) redirects indirectly PyMem_Malloc() to PyObject_Malloc() in debug mode b6aff7a59803 (Sep 28 2009) reintroduces API checks So the GIL issue is almost as old as the debug mode for Python memory allocators. My patch attached to http://bugs.python.org/issue3329 changes the design of the debug memory allocators: they are now wrapper (hooks) on the underlying memory allocator (PyMem: malloc, PyObject: pymalloc), instead of always redirecting to pymalloc (ex: PyObject_Malloc). Using my patch, PyMem_Malloc() now always calls malloc(), even in debug mode. Removing the "GIL must be held" restriction is now safe. Do you agree? May it cause backward compatibility issue? PyMem_Malloc() and PyMem_MALLOC() call malloc(), except if the Python source code was manually modified. Does this use case concern many developers? Removing the GIL restriction would help to replace direct calls to malloc() with PyMem_Malloc(). Using PyMem_SetAllocators(), an application would be able to replace memory allocators, and these allocators would be used "everywhere". => see http://bugs.python.org/issue18203 Victor