[Python-checkins] cpython: Issue #3329: Implement the PEP 445
victor.stinner
python-checkins at python.org
Sun Jul 7 02:25:37 CEST 2013
http://hg.python.org/cpython/rev/ca78c974e938
changeset: 84471:ca78c974e938
user: Victor Stinner <victor.stinner at gmail.com>
date: Sun Jul 07 02:05:46 2013 +0200
summary:
Issue #3329: Implement the PEP 445
Add new enum:
* PyMemAllocatorDomain
Add new structures:
* PyMemAllocator
* PyObjectArenaAllocator
Add new functions:
* PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree()
* PyMem_GetAllocator(), PyMem_SetAllocator()
* PyObject_GetArenaAllocator(), PyObject_SetArenaAllocator()
* PyMem_SetupDebugHooks()
Changes:
* PyMem_Malloc()/PyObject_Realloc() now always call malloc()/realloc(), instead
of calling PyObject_Malloc()/PyObject_Realloc() in debug mode.
* PyObject_Malloc()/PyObject_Realloc() now falls back to
PyMem_Malloc()/PyMem_Realloc() for allocations larger than 512 bytes.
* Redesign debug checks on memory block allocators as hooks, instead of using C
macros
files:
Doc/c-api/memory.rst | 171 ++++++++-
Doc/whatsnew/3.4.rst | 18 +-
Include/objimpl.h | 60 +-
Include/pymem.h | 94 +++-
Misc/NEWS | 3 +
Modules/_testcapimodule.c | 163 ++++++++
Objects/object.c | 20 -
Objects/obmalloc.c | 504 ++++++++++++++++++-------
8 files changed, 796 insertions(+), 237 deletions(-)
diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst
--- a/Doc/c-api/memory.rst
+++ b/Doc/c-api/memory.rst
@@ -84,6 +84,48 @@
for the I/O buffer escapes completely the Python memory manager.
+Raw Memory Interface
+====================
+
+The following function sets are wrappers to the system allocator. These
+functions are thread-safe, the :term:`GIL <global interpreter lock>` does not
+need to be held.
+
+The default raw memory block allocator uses the following functions:
+:c:func:`malloc`, :c:func:`realloc` and :c:func:`free`; call ``malloc(1)`` when
+requesting zero bytes.
+
+.. versionadded:: 3.4
+
+.. c:function:: void* PyMem_RawMalloc(size_t n)
+
+ Allocates *n* bytes and returns a pointer of type :c:type:`void\*` to the
+ allocated memory, or *NULL* if the request fails. Requesting zero bytes
+ returns a distinct non-*NULL* pointer if possible, as if
+ ``PyMem_RawMalloc(1)`` had been called instead. The memory will not have
+ been initialized in any way.
+
+
+.. c:function:: void* PyMem_RawRealloc(void *p, size_t n)
+
+ Resizes the memory block pointed to by *p* to *n* bytes. The contents will
+ be unchanged to the minimum of the old and the new sizes. If *p* is *NULL*,
+ the call is equivalent to ``PyMem_RawMalloc(n)``; else if *n* is equal to
+ zero, the memory block is resized but is not freed, and the returned pointer
+ is non-*NULL*. Unless *p* is *NULL*, it must have been returned by a
+ previous call to :c:func:`PyMem_RawMalloc` or :c:func:`PyMem_RawRealloc`. If
+ the request fails, :c:func:`PyMem_RawRealloc` returns *NULL* and *p* remains
+ a valid pointer to the previous memory area.
+
+
+.. c:function:: void PyMem_RawFree(void *p)
+
+ Frees the memory block pointed to by *p*, which must have been returned by a
+ previous call to :c:func:`PyMem_RawMalloc` or :c:func:`PyMem_RawRealloc`.
+ Otherwise, or if ``PyMem_Free(p)`` has been called before, undefined
+ behavior occurs. If *p* is *NULL*, no operation is performed.
+
+
.. _memoryinterface:
Memory Interface
@@ -91,8 +133,16 @@
The following function sets, modeled after the ANSI C standard, but specifying
behavior when requesting zero bytes, are available for allocating and releasing
-memory from the Python heap:
+memory from the Python heap.
+The default memory block allocator uses the following functions:
+:c:func:`malloc`, :c:func:`realloc` and :c:func:`free`; call ``malloc(1)`` when
+requesting zero bytes.
+
+.. warning::
+
+ The :term:`GIL <global interpreter lock>` must be held when using these
+ functions.
.. c:function:: void* PyMem_Malloc(size_t n)
@@ -155,6 +205,125 @@
:c:func:`PyMem_NEW`, :c:func:`PyMem_RESIZE`, :c:func:`PyMem_DEL`.
+Customize Memory Allocators
+===========================
+
+.. versionadded:: 3.4
+
+.. c:type:: PyMemAllocator
+
+ Structure used to describe a memory block allocator. The structure has
+ four fields:
+
+ +----------------------------------------------------------+---------------------------------------+
+ | Field | Meaning |
+ +==========================================================+=======================================+
+ | ``void *ctx`` | user context passed as first argument |
+ +----------------------------------------------------------+---------------------------------------+
+ | ``void* malloc(void *ctx, size_t size)`` | allocate a memory block |
+ +----------------------------------------------------------+---------------------------------------+
+ | ``void* realloc(void *ctx, void *ptr, size_t new_size)`` | allocate or resize a memory block |
+ +----------------------------------------------------------+---------------------------------------+
+ | ``void free(void *ctx, void *ptr)`` | free a memory block |
+ +----------------------------------------------------------+---------------------------------------+
+
+.. c:type:: PyMemAllocatorDomain
+
+ Enum used to identify an allocator domain. Domains:
+
+ * :c:data:`PYMEM_DOMAIN_RAW`: functions :c:func:`PyMem_RawMalloc`,
+ :c:func:`PyMem_RawRealloc` and :c:func:`PyMem_RawFree`
+ * :c:data:`PYMEM_DOMAIN_MEM`: functions :c:func:`PyMem_Malloc`,
+ :c:func:`PyMem_Realloc` and :c:func:`PyMem_Free`
+ * :c:data:`PYMEM_DOMAIN_OBJ`: functions :c:func:`PyObject_Malloc`,
+ :c:func:`PyObject_Realloc` and :c:func:`PyObject_Free`
+
+
+.. c:function:: void PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)
+
+ Get the memory block allocator of the specified domain.
+
+
+.. c:function:: void PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)
+
+ Set the memory block allocator of the specified domain.
+
+ The new allocator must return a distinct non-NULL pointer when requesting
+ zero bytes.
+
+ For the :c:data:`PYMEM_DOMAIN_RAW` domain, the allocator must be
+ thread-safe: the :term:`GIL <global interpreter lock>` is not held when the
+ allocator is called.
+
+ If the new allocator is not a hook (does not call the previous allocator),
+ the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the
+ debug hooks on top on the new allocator.
+
+
+.. c:function:: void PyMem_SetupDebugHooks(void)
+
+ Setup hooks to detect bugs in the following Python memory allocator
+ functions:
+
+ - :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc`,
+ :c:func:`PyMem_RawFree`
+ - :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc`, :c:func:`PyMem_Free`
+ - :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc`,
+ :c:func:`PyObject_Free`
+
+ Newly allocated memory is filled with the byte ``0xCB``, freed memory is
+ filled with the byte ``0xDB``. Additionnal checks:
+
+ - detect API violations, ex: :c:func:`PyObject_Free` called on a buffer
+ allocated by :c:func:`PyMem_Malloc`
+ - detect write before the start of the buffer (buffer underflow)
+ - detect write after the end of the buffer (buffer overflow)
+
+ The function does nothing if Python is not compiled is debug mode.
+
+
+Customize PyObject Arena Allocator
+==================================
+
+Python has a *pymalloc* allocator for allocations smaller than 512 bytes. This
+allocator is optimized for small objects with a short lifetime. It uses memory
+mappings called "arenas" with a fixed size of 256 KB. It falls back to
+:c:func:`PyMem_Malloc` and :c:func:`PyMem_Realloc` for allocations larger than
+512 bytes. *pymalloc* is the default allocator used by
+:c:func:`PyObject_Malloc`.
+
+The default arena allocator uses the following functions:
+
+* :c:func:`VirtualAlloc` and :c:func:`VirtualFree` on Windows,
+* :c:func:`mmap` and :c:func:`munmap` if available,
+* :c:func:`malloc` and :c:func:`free` otherwise.
+
+.. versionadded:: 3.4
+
+.. c:type:: PyObjectArenaAllocator
+
+ Structure used to describe an arena allocator. The structure has
+ three fields:
+
+ +--------------------------------------------------+---------------------------------------+
+ | Field | Meaning |
+ +==================================================+=======================================+
+ | ``void *ctx`` | user context passed as first argument |
+ +--------------------------------------------------+---------------------------------------+
+ | ``void* alloc(void *ctx, size_t size)`` | allocate an arena of size bytes |
+ +--------------------------------------------------+---------------------------------------+
+ | ``void free(void *ctx, size_t size, void *ptr)`` | free an arena |
+ +--------------------------------------------------+---------------------------------------+
+
+.. c:function:: PyObject_GetArenaAllocator(PyObjectArenaAllocator *allocator)
+
+ Get the arena allocator.
+
+.. c:function:: PyObject_SetArenaAllocator(PyObjectArenaAllocator *allocator)
+
+ Set the arena allocator.
+
+
.. _memoryexamples:
Examples
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -112,21 +112,11 @@
Please read on for a comprehensive list of user-facing changes.
-.. PEP-sized items next.
+PEP 445: Add new APIs to customize Python memory allocators
+===========================================================
-.. _pep-4XX:
-
-.. PEP 4XX: Example PEP
-.. ====================
-
-
-.. (Implemented by Foo Bar.)
-
-.. .. seealso::
-
- :pep:`4XX` - Example PEP
- PEP written by Example Author
-
+The :pep:`445` adds new Application Programming Interfaces (API) to customize
+Python memory allocators.
diff --git a/Include/objimpl.h b/Include/objimpl.h
--- a/Include/objimpl.h
+++ b/Include/objimpl.h
@@ -94,9 +94,9 @@
the object gets initialized via PyObject_{Init, InitVar} after obtaining
the raw memory.
*/
-PyAPI_FUNC(void *) PyObject_Malloc(size_t);
-PyAPI_FUNC(void *) PyObject_Realloc(void *, size_t);
-PyAPI_FUNC(void) PyObject_Free(void *);
+PyAPI_FUNC(void *) PyObject_Malloc(size_t size);
+PyAPI_FUNC(void *) PyObject_Realloc(void *ptr, size_t new_size);
+PyAPI_FUNC(void) PyObject_Free(void *ptr);
/* This function returns the number of allocated memory blocks, regardless of size */
PyAPI_FUNC(Py_ssize_t) _Py_GetAllocatedBlocks(void);
@@ -106,41 +106,15 @@
#ifndef Py_LIMITED_API
PyAPI_FUNC(void) _PyObject_DebugMallocStats(FILE *out);
#endif /* #ifndef Py_LIMITED_API */
-#ifdef PYMALLOC_DEBUG /* WITH_PYMALLOC && PYMALLOC_DEBUG */
-PyAPI_FUNC(void *) _PyObject_DebugMalloc(size_t nbytes);
-PyAPI_FUNC(void *) _PyObject_DebugRealloc(void *p, size_t nbytes);
-PyAPI_FUNC(void) _PyObject_DebugFree(void *p);
-PyAPI_FUNC(void) _PyObject_DebugDumpAddress(const void *p);
-PyAPI_FUNC(void) _PyObject_DebugCheckAddress(const void *p);
-PyAPI_FUNC(void *) _PyObject_DebugMallocApi(char api, size_t nbytes);
-PyAPI_FUNC(void *) _PyObject_DebugReallocApi(char api, void *p, size_t nbytes);
-PyAPI_FUNC(void) _PyObject_DebugFreeApi(char api, void *p);
-PyAPI_FUNC(void) _PyObject_DebugCheckAddressApi(char api, const void *p);
-PyAPI_FUNC(void *) _PyMem_DebugMalloc(size_t nbytes);
-PyAPI_FUNC(void *) _PyMem_DebugRealloc(void *p, size_t nbytes);
-PyAPI_FUNC(void) _PyMem_DebugFree(void *p);
-#define PyObject_MALLOC _PyObject_DebugMalloc
-#define PyObject_Malloc _PyObject_DebugMalloc
-#define PyObject_REALLOC _PyObject_DebugRealloc
-#define PyObject_Realloc _PyObject_DebugRealloc
-#define PyObject_FREE _PyObject_DebugFree
-#define PyObject_Free _PyObject_DebugFree
+#endif
-#else /* WITH_PYMALLOC && ! PYMALLOC_DEBUG */
+/* Macros */
#define PyObject_MALLOC PyObject_Malloc
#define PyObject_REALLOC PyObject_Realloc
#define PyObject_FREE PyObject_Free
-#endif
+#define PyObject_Del PyObject_Free
+#define PyObject_DEL PyObject_Free
-#else /* ! WITH_PYMALLOC */
-#define PyObject_MALLOC PyMem_MALLOC
-#define PyObject_REALLOC PyMem_REALLOC
-#define PyObject_FREE PyMem_FREE
-
-#endif /* WITH_PYMALLOC */
-
-#define PyObject_Del PyObject_Free
-#define PyObject_DEL PyObject_FREE
/*
* Generic object allocator interface
@@ -224,6 +198,26 @@
constructor you would start directly with PyObject_Init/InitVar
*/
+#ifndef Py_LIMITED_API
+typedef struct {
+ /* user context passed as the first argument to the 2 functions */
+ void *ctx;
+
+ /* allocate an arena of size bytes */
+ void* (*alloc) (void *ctx, size_t size);
+
+ /* free an arena */
+ void (*free) (void *ctx, void *ptr, size_t size);
+} PyObjectArenaAllocator;
+
+/* Get the arena allocator. */
+PyAPI_FUNC(void) PyObject_GetArenaAllocator(PyObjectArenaAllocator *allocator);
+
+/* Set the arena allocator. */
+PyAPI_FUNC(void) PyObject_SetArenaAllocator(PyObjectArenaAllocator *allocator);
+#endif
+
+
/*
* Garbage Collection Support
* ==========================
diff --git a/Include/pymem.h b/Include/pymem.h
--- a/Include/pymem.h
+++ b/Include/pymem.h
@@ -11,6 +11,11 @@
extern "C" {
#endif
+PyAPI_FUNC(void *) PyMem_RawMalloc(size_t size);
+PyAPI_FUNC(void *) PyMem_RawRealloc(void *ptr, size_t new_size);
+PyAPI_FUNC(void) PyMem_RawFree(void *ptr);
+
+
/* BEWARE:
Each interface exports both functions and macros. Extension modules should
@@ -49,21 +54,11 @@
performed on failure (no exception is set, no warning is printed, etc).
*/
-PyAPI_FUNC(void *) PyMem_Malloc(size_t);
-PyAPI_FUNC(void *) PyMem_Realloc(void *, size_t);
-PyAPI_FUNC(void) PyMem_Free(void *);
-
-/* Starting from Python 1.6, the wrappers Py_{Malloc,Realloc,Free} are
- no longer supported. They used to call PyErr_NoMemory() on failure. */
+PyAPI_FUNC(void *) PyMem_Malloc(size_t size);
+PyAPI_FUNC(void *) PyMem_Realloc(void *ptr, size_t new_size);
+PyAPI_FUNC(void) PyMem_Free(void *ptr);
/* Macros. */
-#ifdef PYMALLOC_DEBUG
-/* Redirect all memory operations to Python's debugging allocator. */
-#define PyMem_MALLOC _PyMem_DebugMalloc
-#define PyMem_REALLOC _PyMem_DebugRealloc
-#define PyMem_FREE _PyMem_DebugFree
-
-#else /* ! PYMALLOC_DEBUG */
/* PyMem_MALLOC(0) means malloc(1). Some systems would return NULL
for malloc(0), which would be treated as an error. Some platforms
@@ -71,13 +66,9 @@
pymalloc. To solve these problems, allocate an extra byte. */
/* Returns NULL to indicate error if a negative size or size larger than
Py_ssize_t can represent is supplied. Helps prevents security holes. */
-#define PyMem_MALLOC(n) ((size_t)(n) > (size_t)PY_SSIZE_T_MAX ? NULL \
- : malloc((n) ? (n) : 1))
-#define PyMem_REALLOC(p, n) ((size_t)(n) > (size_t)PY_SSIZE_T_MAX ? NULL \
- : realloc((p), (n) ? (n) : 1))
-#define PyMem_FREE free
-
-#endif /* PYMALLOC_DEBUG */
+#define PyMem_MALLOC(n) PyMem_Malloc(n)
+#define PyMem_REALLOC(p, n) PyMem_Realloc(p, n)
+#define PyMem_FREE(p) PyMem_Free(p)
/*
* Type-oriented memory interface
@@ -115,6 +106,69 @@
#define PyMem_Del PyMem_Free
#define PyMem_DEL PyMem_FREE
+#ifndef Py_LIMITED_API
+typedef enum {
+ /* PyMem_RawMalloc(), PyMem_RawRealloc() and PyMem_RawFree() */
+ PYMEM_DOMAIN_RAW,
+
+ /* PyMem_Malloc(), PyMem_Realloc() and PyMem_Free() */
+ PYMEM_DOMAIN_MEM,
+
+ /* PyObject_Malloc(), PyObject_Realloc() and PyObject_Free() */
+ PYMEM_DOMAIN_OBJ
+} PyMemAllocatorDomain;
+
+typedef struct {
+ /* user context passed as the first argument to the 3 functions */
+ void *ctx;
+
+ /* allocate a memory block */
+ void* (*malloc) (void *ctx, size_t size);
+
+ /* allocate or resize a memory block */
+ void* (*realloc) (void *ctx, void *ptr, size_t new_size);
+
+ /* release a memory block */
+ void (*free) (void *ctx, void *ptr);
+} PyMemAllocator;
+
+/* Get the memory block allocator of the specified domain. */
+PyAPI_FUNC(void) PyMem_GetAllocator(PyMemAllocatorDomain domain,
+ PyMemAllocator *allocator);
+
+/* Set the memory block allocator of the specified domain.
+
+ The new allocator must return a distinct non-NULL pointer when requesting
+ zero bytes.
+
+ For the PYMEM_DOMAIN_RAW domain, the allocator must be thread-safe: the GIL
+ is not held when the allocator is called.
+
+ If the new allocator is not a hook (don't call the previous allocator), the
+ PyMem_SetupDebugHooks() function must be called to reinstall the debug hooks
+ on top on the new allocator. */
+PyAPI_FUNC(void) PyMem_SetAllocator(PyMemAllocatorDomain domain,
+ PyMemAllocator *allocator);
+
+/* Setup hooks to detect bugs in the following Python memory allocator
+ functions:
+
+ - PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree()
+ - PyMem_Malloc(), PyMem_Realloc(), PyMem_Free()
+ - PyObject_Malloc(), PyObject_Realloc() and PyObject_Free()
+
+ Newly allocated memory is filled with the byte 0xCB, freed memory is filled
+ with the byte 0xDB. Additionnal checks:
+
+ - detect API violations, ex: PyObject_Free() called on a buffer allocated
+ by PyMem_Malloc()
+ - detect write before the start of the buffer (buffer underflow)
+ - detect write after the end of the buffer (buffer overflow)
+
+ The function does nothing if Python is not compiled is debug mode. */
+PyAPI_FUNC(void) PyMem_SetupDebugHooks(void);
+#endif
+
#ifdef __cplusplus
}
#endif
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,9 @@
Core and Builtins
-----------------
+- Issue #3329: Implement the PEP 445: Add new APIs to customize Python memory
+ allocators.
+
- Issue #18328: Reorder ops in PyThreadState_Delete*() functions. Now the
tstate is first removed from TLS and then deallocated.
diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c
--- a/Modules/_testcapimodule.c
+++ b/Modules/_testcapimodule.c
@@ -2511,6 +2511,161 @@
Py_RETURN_NONE;
}
+static PyObject *
+test_pymem_alloc0(PyObject *self)
+{
+ void *ptr;
+
+ ptr = PyMem_Malloc(0);
+ if (ptr == NULL) {
+ PyErr_SetString(PyExc_RuntimeError, "PyMem_Malloc(0) returns NULL");
+ return NULL;
+ }
+ PyMem_Free(ptr);
+
+ ptr = PyObject_Malloc(0);
+ if (ptr == NULL) {
+ PyErr_SetString(PyExc_RuntimeError, "PyObject_Malloc(0) returns NULL");
+ return NULL;
+ }
+ PyObject_Free(ptr);
+
+ Py_RETURN_NONE;
+}
+
+typedef struct {
+ PyMemAllocator alloc;
+
+ size_t malloc_size;
+ void *realloc_ptr;
+ size_t realloc_new_size;
+ void *free_ptr;
+} alloc_hook_t;
+
+static void* hook_malloc (void* ctx, size_t size)
+{
+ alloc_hook_t *hook = (alloc_hook_t *)ctx;
+ hook->malloc_size = size;
+ return hook->alloc.malloc(hook->alloc.ctx, size);
+}
+
+static void* hook_realloc (void* ctx, void* ptr, size_t new_size)
+{
+ alloc_hook_t *hook = (alloc_hook_t *)ctx;
+ hook->realloc_ptr = ptr;
+ hook->realloc_new_size = new_size;
+ return hook->alloc.realloc(hook->alloc.ctx, ptr, new_size);
+}
+
+static void hook_free (void *ctx, void *ptr)
+{
+ alloc_hook_t *hook = (alloc_hook_t *)ctx;
+ hook->free_ptr = ptr;
+ hook->alloc.free(hook->alloc.ctx, ptr);
+}
+
+static PyObject *
+test_setallocators(PyMemAllocatorDomain domain)
+{
+ PyObject *res = NULL;
+ const char *error_msg;
+ alloc_hook_t hook;
+ PyMemAllocator alloc;
+ size_t size, size2;
+ void *ptr, *ptr2;
+
+ hook.malloc_size = 0;
+ hook.realloc_ptr = NULL;
+ hook.realloc_new_size = 0;
+ hook.free_ptr = NULL;
+
+ alloc.ctx = &hook;
+ alloc.malloc = &hook_malloc;
+ alloc.realloc = &hook_realloc;
+ alloc.free = &hook_free;
+ PyMem_GetAllocator(domain, &hook.alloc);
+ PyMem_SetAllocator(domain, &alloc);
+
+ size = 42;
+ switch(domain)
+ {
+ case PYMEM_DOMAIN_RAW: ptr = PyMem_RawMalloc(size); break;
+ case PYMEM_DOMAIN_MEM: ptr = PyMem_Malloc(size); break;
+ case PYMEM_DOMAIN_OBJ: ptr = PyObject_Malloc(size); break;
+ default: ptr = NULL; break;
+ }
+
+ if (ptr == NULL) {
+ error_msg = "malloc failed";
+ goto fail;
+ }
+
+ if (hook.malloc_size != size) {
+ error_msg = "malloc invalid size";
+ goto fail;
+ }
+
+ size2 = 200;
+ switch(domain)
+ {
+ case PYMEM_DOMAIN_RAW: ptr2 = PyMem_RawRealloc(ptr, size2); break;
+ case PYMEM_DOMAIN_MEM: ptr2 = PyMem_Realloc(ptr, size2); break;
+ case PYMEM_DOMAIN_OBJ: ptr2 = PyObject_Realloc(ptr, size2); break;
+ }
+
+ if (ptr2 == NULL) {
+ error_msg = "realloc failed";
+ goto fail;
+ }
+
+ if (hook.realloc_ptr != ptr
+ || hook.realloc_new_size != size2) {
+ error_msg = "realloc invalid parameters";
+ goto fail;
+ }
+
+ switch(domain)
+ {
+ case PYMEM_DOMAIN_RAW: PyMem_RawFree(ptr2); break;
+ case PYMEM_DOMAIN_MEM: PyMem_Free(ptr2); break;
+ case PYMEM_DOMAIN_OBJ: PyObject_Free(ptr2); break;
+ }
+
+ if (hook.free_ptr != ptr2) {
+ error_msg = "free invalid pointer";
+ goto fail;
+ }
+
+ Py_INCREF(Py_None);
+ res = Py_None;
+ goto finally;
+
+fail:
+ PyErr_SetString(PyExc_RuntimeError, error_msg);
+
+finally:
+ PyMem_SetAllocator(domain, &hook.alloc);
+ return res;
+}
+
+static PyObject *
+test_pymem_setrawallocators(PyObject *self)
+{
+ return test_setallocators(PYMEM_DOMAIN_RAW);
+}
+
+static PyObject *
+test_pymem_setallocators(PyObject *self)
+{
+ return test_setallocators(PYMEM_DOMAIN_MEM);
+}
+
+static PyObject *
+test_pyobject_setallocators(PyObject *self)
+{
+ return test_setallocators(PYMEM_DOMAIN_OBJ);
+}
+
static PyMethodDef TestMethods[] = {
{"raise_exception", raise_exception, METH_VARARGS},
{"raise_memoryerror", (PyCFunction)raise_memoryerror, METH_NOARGS},
@@ -2611,6 +2766,14 @@
{"pytime_object_to_time_t", test_pytime_object_to_time_t, METH_VARARGS},
{"pytime_object_to_timeval", test_pytime_object_to_timeval, METH_VARARGS},
{"pytime_object_to_timespec", test_pytime_object_to_timespec, METH_VARARGS},
+ {"test_pymem",
+ (PyCFunction)test_pymem_alloc0, METH_NOARGS},
+ {"test_pymem_alloc0",
+ (PyCFunction)test_pymem_setrawallocators, METH_NOARGS},
+ {"test_pymem_setallocators",
+ (PyCFunction)test_pymem_setallocators, METH_NOARGS},
+ {"test_pyobject_setallocators",
+ (PyCFunction)test_pyobject_setallocators, METH_NOARGS},
{NULL, NULL} /* sentinel */
};
diff --git a/Objects/object.c b/Objects/object.c
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -1859,26 +1859,6 @@
Py_ssize_t (*_Py_abstract_hack)(PyObject *) = PyObject_Size;
-/* Python's malloc wrappers (see pymem.h) */
-
-void *
-PyMem_Malloc(size_t nbytes)
-{
- return PyMem_MALLOC(nbytes);
-}
-
-void *
-PyMem_Realloc(void *p, size_t nbytes)
-{
- return PyMem_REALLOC(p, nbytes);
-}
-
-void
-PyMem_Free(void *p)
-{
- PyMem_FREE(p);
-}
-
void
_PyObject_DebugTypeStats(FILE *out)
{
diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c
--- a/Objects/obmalloc.c
+++ b/Objects/obmalloc.c
@@ -1,18 +1,326 @@
#include "Python.h"
+/* Python's malloc wrappers (see pymem.h) */
+
+#ifdef PYMALLOC_DEBUG /* WITH_PYMALLOC && PYMALLOC_DEBUG */
+/* Forward declaration */
+static void* _PyMem_DebugMalloc(void *ctx, size_t size);
+static void _PyMem_DebugFree(void *ctx, void *p);
+static void* _PyMem_DebugRealloc(void *ctx, void *ptr, size_t size);
+
+static void _PyObject_DebugDumpAddress(const void *p);
+static void _PyMem_DebugCheckAddress(char api_id, const void *p);
+#endif
+
#ifdef WITH_PYMALLOC
-#ifdef HAVE_MMAP
- #include <sys/mman.h>
- #ifdef MAP_ANONYMOUS
- #define ARENAS_USE_MMAP
- #endif
+#ifdef MS_WINDOWS
+# include <windows.h>
+#elif defined(HAVE_MMAP)
+# include <sys/mman.h>
+# ifdef MAP_ANONYMOUS
+# define ARENAS_USE_MMAP
+# endif
#endif
+/* Forward declaration */
+static void* _PyObject_Malloc(void *ctx, size_t size);
+static void _PyObject_Free(void *ctx, void *p);
+static void* _PyObject_Realloc(void *ctx, void *ptr, size_t size);
+#endif
+
+
+static void *
+_PyMem_RawMalloc(void *ctx, size_t size)
+{
+ /* PyMem_Malloc(0) means malloc(1). Some systems would return NULL
+ for malloc(0), which would be treated as an error. Some platforms would
+ return a pointer with no memory behind it, which would break pymalloc.
+ To solve these problems, allocate an extra byte. */
+ if (size == 0)
+ size = 1;
+ return malloc(size);
+}
+
+static void *
+_PyMem_RawRealloc(void *ctx, void *ptr, size_t size)
+{
+ if (size == 0)
+ size = 1;
+ return realloc(ptr, size);
+}
+
+static void
+_PyMem_RawFree(void *ctx, void *ptr)
+{
+ free(ptr);
+}
+
+
#ifdef MS_WINDOWS
-#include <windows.h>
+static void *
+_PyObject_ArenaVirtualAlloc(void *ctx, size_t size)
+{
+ return VirtualAlloc(NULL, size,
+ MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
+}
+
+static void
+_PyObject_ArenaVirtualFree(void *ctx, void *ptr, size_t size)
+{
+ VirtualFree(ptr, size, MEM_RELEASE);
+}
+
+#elif defined(ARENAS_USE_MMAP)
+static void *
+_PyObject_ArenaMmap(void *ctx, size_t size)
+{
+ void *ptr;
+ ptr = mmap(NULL, size, PROT_READ|PROT_WRITE,
+ MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
+ if (ptr == MAP_FAILED)
+ return NULL;
+ assert(ptr != NULL);
+ return ptr;
+}
+
+static void
+_PyObject_ArenaMunmap(void *ctx, void *ptr, size_t size)
+{
+ munmap(ptr, size);
+}
+
+#else
+static void *
+_PyObject_ArenaMalloc(void *ctx, size_t size)
+{
+ return malloc(size);
+}
+
+static void
+_PyObject_ArenaFree(void *ctx, void *ptr, size_t size)
+{
+ free(ptr);
+}
#endif
+
+#define PYRAW_FUNCS _PyMem_RawMalloc, _PyMem_RawRealloc, _PyMem_RawFree
+#ifdef WITH_PYMALLOC
+#define PYOBJECT_FUNCS _PyObject_Malloc, _PyObject_Realloc, _PyObject_Free
+#else
+#define PYOBJECT_FUNCS PYRAW_FUNCS
+#endif
+
+#ifdef PYMALLOC_DEBUG
+typedef struct {
+ /* We tag each block with an API ID in order to tag API violations */
+ char api_id;
+ PyMemAllocator alloc;
+} debug_alloc_api_t;
+static struct {
+ debug_alloc_api_t raw;
+ debug_alloc_api_t mem;
+ debug_alloc_api_t obj;
+} _PyMem_Debug = {
+ {'r', {NULL, PYRAW_FUNCS}},
+ {'m', {NULL, PYRAW_FUNCS}},
+ {'o', {NULL, PYOBJECT_FUNCS}}
+ };
+
+#define PYDEBUG_FUNCS _PyMem_DebugMalloc, _PyMem_DebugRealloc, _PyMem_DebugFree
+#endif
+
+static PyMemAllocator _PyMem_Raw = {
+#ifdef PYMALLOC_DEBUG
+ &_PyMem_Debug.raw, PYDEBUG_FUNCS
+#else
+ NULL, PYRAW_FUNCS
+#endif
+ };
+
+static PyMemAllocator _PyMem = {
+#ifdef PYMALLOC_DEBUG
+ &_PyMem_Debug.mem, PYDEBUG_FUNCS
+#else
+ NULL, PYRAW_FUNCS
+#endif
+ };
+
+static PyMemAllocator _PyObject = {
+#ifdef PYMALLOC_DEBUG
+ &_PyMem_Debug.obj, PYDEBUG_FUNCS
+#else
+ NULL, PYOBJECT_FUNCS
+#endif
+ };
+
+#undef PYRAW_FUNCS
+#undef PYOBJECT_FUNCS
+#undef PYDEBUG_FUNCS
+
+static PyObjectArenaAllocator _PyObject_Arena = {NULL,
+#ifdef MS_WINDOWS
+ _PyObject_ArenaVirtualAlloc, _PyObject_ArenaVirtualFree
+#elif defined(ARENAS_USE_MMAP)
+ _PyObject_ArenaMmap, _PyObject_ArenaMunmap
+#else
+ _PyObject_ArenaMalloc, _PyObject_ArenaFree
+#endif
+ };
+
+void
+PyMem_SetupDebugHooks(void)
+{
+#ifdef PYMALLOC_DEBUG
+ PyMemAllocator alloc;
+
+ alloc.malloc = _PyMem_DebugMalloc;
+ alloc.realloc = _PyMem_DebugRealloc;
+ alloc.free = _PyMem_DebugFree;
+
+ if (_PyMem_Raw.malloc != _PyMem_DebugMalloc) {
+ alloc.ctx = &_PyMem_Debug.raw;
+ PyMem_GetAllocator(PYMEM_DOMAIN_RAW, &_PyMem_Debug.raw.alloc);
+ PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &alloc);
+ }
+
+ if (_PyMem.malloc != _PyMem_DebugMalloc) {
+ alloc.ctx = &_PyMem_Debug.mem;
+ PyMem_GetAllocator(PYMEM_DOMAIN_MEM, &_PyMem_Debug.mem.alloc);
+ PyMem_SetAllocator(PYMEM_DOMAIN_MEM, &alloc);
+ }
+
+ if (_PyObject.malloc != _PyMem_DebugMalloc) {
+ alloc.ctx = &_PyMem_Debug.obj;
+ PyMem_GetAllocator(PYMEM_DOMAIN_OBJ, &_PyMem_Debug.obj.alloc);
+ PyMem_SetAllocator(PYMEM_DOMAIN_OBJ, &alloc);
+ }
+#endif
+}
+
+void
+PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)
+{
+ switch(domain)
+ {
+ case PYMEM_DOMAIN_RAW: *allocator = _PyMem_Raw; break;
+ case PYMEM_DOMAIN_MEM: *allocator = _PyMem; break;
+ case PYMEM_DOMAIN_OBJ: *allocator = _PyObject; break;
+ default:
+ /* unknown domain */
+ allocator->ctx = NULL;
+ allocator->malloc = NULL;
+ allocator->realloc = NULL;
+ allocator->free = NULL;
+ }
+}
+
+void
+PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)
+{
+ switch(domain)
+ {
+ case PYMEM_DOMAIN_RAW: _PyMem_Raw = *allocator; break;
+ case PYMEM_DOMAIN_MEM: _PyMem = *allocator; break;
+ case PYMEM_DOMAIN_OBJ: _PyObject = *allocator; break;
+ /* ignore unknown domain */
+ }
+
+}
+
+void
+PyObject_GetArenaAllocator(PyObjectArenaAllocator *allocator)
+{
+ *allocator = _PyObject_Arena;
+}
+
+void
+PyObject_SetArenaAllocator(PyObjectArenaAllocator *allocator)
+{
+ _PyObject_Arena = *allocator;
+}
+
+void *
+PyMem_RawMalloc(size_t size)
+{
+ /*
+ * Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
+ * Most python internals blindly use a signed Py_ssize_t to track
+ * things without checking for overflows or negatives.
+ * As size_t is unsigned, checking for size < 0 is not required.
+ */
+ if (size > (size_t)PY_SSIZE_T_MAX)
+ return NULL;
+
+ return _PyMem_Raw.malloc(_PyMem_Raw.ctx, size);
+}
+
+void*
+PyMem_RawRealloc(void *ptr, size_t new_size)
+{
+ /* see PyMem_RawMalloc() */
+ if (new_size > (size_t)PY_SSIZE_T_MAX)
+ return NULL;
+ return _PyMem_Raw.realloc(_PyMem_Raw.ctx, ptr, new_size);
+}
+
+void PyMem_RawFree(void *ptr)
+{
+ _PyMem_Raw.free(_PyMem_Raw.ctx, ptr);
+}
+
+void *
+PyMem_Malloc(size_t size)
+{
+ /* see PyMem_RawMalloc() */
+ if (size > (size_t)PY_SSIZE_T_MAX)
+ return NULL;
+ return _PyMem.malloc(_PyMem.ctx, size);
+}
+
+void *
+PyMem_Realloc(void *ptr, size_t new_size)
+{
+ /* see PyMem_RawMalloc() */
+ if (new_size > (size_t)PY_SSIZE_T_MAX)
+ return NULL;
+ return _PyMem.realloc(_PyMem.ctx, ptr, new_size);
+}
+
+void
+PyMem_Free(void *ptr)
+{
+ _PyMem.free(_PyMem.ctx, ptr);
+}
+
+void *
+PyObject_Malloc(size_t size)
+{
+ /* see PyMem_RawMalloc() */
+ if (size > (size_t)PY_SSIZE_T_MAX)
+ return NULL;
+ return _PyObject.malloc(_PyObject.ctx, size);
+}
+
+void *
+PyObject_Realloc(void *ptr, size_t new_size)
+{
+ /* see PyMem_RawMalloc() */
+ if (new_size > (size_t)PY_SSIZE_T_MAX)
+ return NULL;
+ return _PyObject.realloc(_PyObject.ctx, ptr, new_size);
+}
+
+void
+PyObject_Free(void *ptr)
+{
+ _PyObject.free(_PyObject.ctx, ptr);
+}
+
+
+#ifdef WITH_PYMALLOC
+
#ifdef WITH_VALGRIND
#include <valgrind/valgrind.h>
@@ -549,7 +857,6 @@
struct arena_object* arenaobj;
uint excess; /* number of bytes above pool alignment */
void *address;
- int err;
#ifdef PYMALLOC_DEBUG
if (Py_GETENV("PYTHONMALLOCSTATS"))
@@ -571,7 +878,7 @@
return NULL; /* overflow */
#endif
nbytes = numarenas * sizeof(*arenas);
- arenaobj = (struct arena_object *)realloc(arenas, nbytes);
+ arenaobj = (struct arena_object *)PyMem_Realloc(arenas, nbytes);
if (arenaobj == NULL)
return NULL;
arenas = arenaobj;
@@ -602,19 +909,8 @@
arenaobj = unused_arena_objects;
unused_arena_objects = arenaobj->nextarena;
assert(arenaobj->address == 0);
-#ifdef MS_WINDOWS
- address = (void*)VirtualAlloc(NULL, ARENA_SIZE,
- MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
- err = (address == NULL);
-#elif defined(ARENAS_USE_MMAP)
- address = mmap(NULL, ARENA_SIZE, PROT_READ|PROT_WRITE,
- MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
- err = (address == MAP_FAILED);
-#else
- address = malloc(ARENA_SIZE);
- err = (address == 0);
-#endif
- if (err) {
+ address = _PyObject_Arena.alloc(_PyObject_Arena.ctx, ARENA_SIZE);
+ if (address == NULL) {
/* The allocation failed: return NULL after putting the
* arenaobj back.
*/
@@ -777,9 +1073,8 @@
* Unless the optimizer reorders everything, being too smart...
*/
-#undef PyObject_Malloc
-void *
-PyObject_Malloc(size_t nbytes)
+static void *
+_PyObject_Malloc(void *ctx, size_t nbytes)
{
block *bp;
poolp pool;
@@ -796,17 +1091,6 @@
#endif
/*
- * Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
- * Most python internals blindly use a signed Py_ssize_t to track
- * things without checking for overflows or negatives.
- * As size_t is unsigned, checking for nbytes < 0 is not required.
- */
- if (nbytes > PY_SSIZE_T_MAX) {
- _Py_AllocatedBlocks--;
- return NULL;
- }
-
- /*
* This implicitly redirects malloc(0).
*/
if ((nbytes - 1) < SMALL_REQUEST_THRESHOLD) {
@@ -978,10 +1262,8 @@
* last chance to serve the request) or when the max memory limit
* has been reached.
*/
- if (nbytes == 0)
- nbytes = 1;
{
- void *result = malloc(nbytes);
+ void *result = PyMem_Malloc(nbytes);
if (!result)
_Py_AllocatedBlocks--;
return result;
@@ -990,9 +1272,8 @@
/* free */
-#undef PyObject_Free
-void
-PyObject_Free(void *p)
+static void
+_PyObject_Free(void *ctx, void *p)
{
poolp pool;
block *lastfree;
@@ -1101,13 +1382,8 @@
unused_arena_objects = ao;
/* Free the entire arena. */
-#ifdef MS_WINDOWS
- VirtualFree((void *)ao->address, 0, MEM_RELEASE);
-#elif defined(ARENAS_USE_MMAP)
- munmap((void *)ao->address, ARENA_SIZE);
-#else
- free((void *)ao->address);
-#endif
+ _PyObject_Arena.free(_PyObject_Arena.ctx,
+ (void *)ao->address, ARENA_SIZE);
ao->address = 0; /* mark unassociated */
--narenas_currently_allocated;
@@ -1216,7 +1492,7 @@
redirect:
#endif
/* We didn't allocate this address. */
- free(p);
+ PyMem_Free(p);
}
/* realloc. If p is NULL, this acts like malloc(nbytes). Else if nbytes==0,
@@ -1224,9 +1500,8 @@
* return a non-NULL result.
*/
-#undef PyObject_Realloc
-void *
-PyObject_Realloc(void *p, size_t nbytes)
+static void *
+_PyObject_Realloc(void *ctx, void *p, size_t nbytes)
{
void *bp;
poolp pool;
@@ -1236,16 +1511,7 @@
#endif
if (p == NULL)
- return PyObject_Malloc(nbytes);
-
- /*
- * Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
- * Most python internals blindly use a signed Py_ssize_t to track
- * things without checking for overflows or negatives.
- * As size_t is unsigned, checking for nbytes < 0 is not required.
- */
- if (nbytes > PY_SSIZE_T_MAX)
- return NULL;
+ return _PyObject_Malloc(ctx, nbytes);
#ifdef WITH_VALGRIND
/* Treat running_on_valgrind == -1 the same as 0 */
@@ -1273,10 +1539,10 @@
}
size = nbytes;
}
- bp = PyObject_Malloc(nbytes);
+ bp = _PyObject_Malloc(ctx, nbytes);
if (bp != NULL) {
memcpy(bp, p, size);
- PyObject_Free(p);
+ _PyObject_Free(ctx, p);
}
return bp;
}
@@ -1294,14 +1560,14 @@
* at p. Instead we punt: let C continue to manage this block.
*/
if (nbytes)
- return realloc(p, nbytes);
+ return PyMem_Realloc(p, nbytes);
/* C doesn't define the result of realloc(p, 0) (it may or may not
* return NULL then), but Python's docs promise that nbytes==0 never
* returns NULL. We don't pass 0 to realloc(), to avoid that endcase
* to begin with. Even then, we can't be sure that realloc() won't
* return NULL.
*/
- bp = realloc(p, 1);
+ bp = PyMem_Realloc(p, 1);
return bp ? bp : p;
}
@@ -1311,24 +1577,6 @@
/* pymalloc not enabled: Redirect the entry points to malloc. These will
* only be used by extensions that are compiled with pymalloc enabled. */
-void *
-PyObject_Malloc(size_t n)
-{
- return PyMem_MALLOC(n);
-}
-
-void *
-PyObject_Realloc(void *p, size_t n)
-{
- return PyMem_REALLOC(p, n);
-}
-
-void
-PyObject_Free(void *p)
-{
- PyMem_FREE(p);
-}
-
Py_ssize_t
_Py_GetAllocatedBlocks(void)
{
@@ -1354,10 +1602,6 @@
#define DEADBYTE 0xDB /* dead (newly freed) memory */
#define FORBIDDENBYTE 0xFB /* untouchable bytes at each end of a block */
-/* We tag each block with an API ID in order to tag API violations */
-#define _PYMALLOC_MEM_ID 'm' /* the PyMem_Malloc() API */
-#define _PYMALLOC_OBJ_ID 'o' /* The PyObject_Malloc() API */
-
static size_t serialno = 0; /* incremented on each debug {m,re}alloc */
/* serialno is always incremented via calling this routine. The point is
@@ -1440,58 +1684,18 @@
p[2*S+n: 2*S+n+S]
Copies of FORBIDDENBYTE. Used to catch over- writes and reads.
p[2*S+n+S: 2*S+n+2*S]
- A serial number, incremented by 1 on each call to _PyObject_DebugMalloc
- and _PyObject_DebugRealloc.
+ A serial number, incremented by 1 on each call to _PyMem_DebugMalloc
+ and _PyMem_DebugRealloc.
This is a big-endian size_t.
If "bad memory" is detected later, the serial number gives an
excellent way to set a breakpoint on the next run, to capture the
instant at which this block was passed out.
*/
-/* debug replacements for the PyMem_* memory API */
-void *
-_PyMem_DebugMalloc(size_t nbytes)
+static void *
+_PyMem_DebugMalloc(void *ctx, size_t nbytes)
{
- return _PyObject_DebugMallocApi(_PYMALLOC_MEM_ID, nbytes);
-}
-void *
-_PyMem_DebugRealloc(void *p, size_t nbytes)
-{
- return _PyObject_DebugReallocApi(_PYMALLOC_MEM_ID, p, nbytes);
-}
-void
-_PyMem_DebugFree(void *p)
-{
- _PyObject_DebugFreeApi(_PYMALLOC_MEM_ID, p);
-}
-
-/* debug replacements for the PyObject_* memory API */
-void *
-_PyObject_DebugMalloc(size_t nbytes)
-{
- return _PyObject_DebugMallocApi(_PYMALLOC_OBJ_ID, nbytes);
-}
-void *
-_PyObject_DebugRealloc(void *p, size_t nbytes)
-{
- return _PyObject_DebugReallocApi(_PYMALLOC_OBJ_ID, p, nbytes);
-}
-void
-_PyObject_DebugFree(void *p)
-{
- _PyObject_DebugFreeApi(_PYMALLOC_OBJ_ID, p);
-}
-void
-_PyObject_DebugCheckAddress(const void *p)
-{
- _PyObject_DebugCheckAddressApi(_PYMALLOC_OBJ_ID, p);
-}
-
-
-/* generic debug memory api, with an "id" to identify the API in use */
-void *
-_PyObject_DebugMallocApi(char id, size_t nbytes)
-{
+ debug_alloc_api_t *api = (debug_alloc_api_t *)ctx;
uchar *p; /* base address of malloc'ed block */
uchar *tail; /* p + 2*SST + nbytes == pointer to tail pad bytes */
size_t total; /* nbytes + 4*SST */
@@ -1502,14 +1706,14 @@
/* overflow: can't represent total as a size_t */
return NULL;
- p = (uchar *)PyObject_Malloc(total);
+ p = (uchar *)api->alloc.malloc(api->alloc.ctx, total);
if (p == NULL)
return NULL;
/* at p, write size (SST bytes), id (1 byte), pad (SST-1 bytes) */
write_size_t(p, nbytes);
- p[SST] = (uchar)id;
- memset(p + SST + 1 , FORBIDDENBYTE, SST-1);
+ p[SST] = (uchar)api->api_id;
+ memset(p + SST + 1, FORBIDDENBYTE, SST-1);
if (nbytes > 0)
memset(p + 2*SST, CLEANBYTE, nbytes);
@@ -1527,25 +1731,27 @@
Then fills the original bytes with DEADBYTE.
Then calls the underlying free.
*/
-void
-_PyObject_DebugFreeApi(char api, void *p)
+static void
+_PyMem_DebugFree(void *ctx, void *p)
{
+ debug_alloc_api_t *api = (debug_alloc_api_t *)ctx;
uchar *q = (uchar *)p - 2*SST; /* address returned from malloc */
size_t nbytes;
if (p == NULL)
return;
- _PyObject_DebugCheckAddressApi(api, p);
+ _PyMem_DebugCheckAddress(api->api_id, p);
nbytes = read_size_t(q);
nbytes += 4*SST;
if (nbytes > 0)
memset(q, DEADBYTE, nbytes);
- PyObject_Free(q);
+ api->alloc.free(api->alloc.ctx, q);
}
-void *
-_PyObject_DebugReallocApi(char api, void *p, size_t nbytes)
+static void *
+_PyMem_DebugRealloc(void *ctx, void *p, size_t nbytes)
{
+ debug_alloc_api_t *api = (debug_alloc_api_t *)ctx;
uchar *q = (uchar *)p;
uchar *tail;
size_t total; /* nbytes + 4*SST */
@@ -1553,9 +1759,9 @@
int i;
if (p == NULL)
- return _PyObject_DebugMallocApi(api, nbytes);
+ return _PyMem_DebugMalloc(ctx, nbytes);
- _PyObject_DebugCheckAddressApi(api, p);
+ _PyMem_DebugCheckAddress(api->api_id, p);
bumpserialno();
original_nbytes = read_size_t(q - 2*SST);
total = nbytes + 4*SST;
@@ -1572,12 +1778,12 @@
* case we didn't get the chance to mark the old memory with DEADBYTE,
* but we live with that.
*/
- q = (uchar *)PyObject_Realloc(q - 2*SST, total);
+ q = (uchar *)api->alloc.realloc(api->alloc.ctx, q - 2*SST, total);
if (q == NULL)
return NULL;
write_size_t(q, nbytes);
- assert(q[SST] == (uchar)api);
+ assert(q[SST] == (uchar)api->api_id);
for (i = 1; i < SST; ++i)
assert(q[SST + i] == FORBIDDENBYTE);
q += 2*SST;
@@ -1599,8 +1805,8 @@
* and call Py_FatalError to kill the program.
* The API id, is also checked.
*/
- void
-_PyObject_DebugCheckAddressApi(char api, const void *p)
+static void
+_PyMem_DebugCheckAddress(char api, const void *p)
{
const uchar *q = (const uchar *)p;
char msgbuf[64];
@@ -1652,7 +1858,7 @@
}
/* Display info to stderr about the memory block at p. */
-void
+static void
_PyObject_DebugDumpAddress(const void *p)
{
const uchar *q = (const uchar *)p;
--
Repository URL: http://hg.python.org/cpython
More information about the Python-checkins
mailing list