[Python-Dev] Moving forward on the object memory API

[Neil Schemenauer]

It looks like Tim managed to work some magic and make pymalloc's free
handle memory allocated by either the system allocator or pymalloc
itself.  That means we can make PyMem_DEL use the pymalloc's 'free' and
make PyObject_NEW use pymalloc's 'malloc'.  Here's my plan for making
this happen:

    #define PyMem_MALLOC(n)         malloc(n)
    #define PyMem_REALLOC(p, n)     realloc((void *)(p), (n))
    #define PyMem_FREE(p)           free((void *)(p))

I think making PyMem_FREE call free() is safe.  I haven't seen any
extension modules allocate with PyObject_NEW and free with PyMem_FREE.
We deprecate the PyMem_* functions.  There's no need for them, IMHO:

    #define PyMem_Malloc PyMem_MALLOC
    #define PyMem_New PyMem_NEW
    #define PyMem_Resize PyMem_RESIZE
    #define PyMem_Free PyMem_DEL
    #define PyMem_Del PyMem_DEL
    /* See comment near MALLOC_ZERO_RETURNS_NULL in pyport.h. */
    #define PyMem_Realloc(p, n) \
            do { \
                    size_t _n = n; \
                    PyMem_REALLOC(p, _n ? _n : 1); \
            } while (0)

Next, we base PyObject_{MALLOC,REALLOC,FREE} on pymalloc.  Basically:

    #ifdef WITH_PYMALLOC
    #define PyObject_MALLOC(n)      _PyMalloc_Malloc(n)
    #define PyObject_REALLOC(op, n) _PyMalloc_Realloc((void *)(op), (n))
    #define PyObject_FREE(op)       _PyMalloc_Free((void *)(op))
    #else
    #define PyObject_MALLOC(n)      PyMem_MALLOC(n)
    #define PyObject_REALLOC(op, n) PyMem_REALLOC((void *)(op), (n))
    #define PyObject_FREE(op)       PyMem_FREE((void *)(op))
    #endif

PyMem_DEL and PyMem_Free need to call pymalloc:

    #define PyMem_DEL(op) PyObject_FREE(op)

We go through the source and replace all the PyMem_* function calls with
the equivalent PyMem_* macro calls and replace PyMem_DEL with
PyMem_FREE.

The recommended API for extension modules writers would be
PyMem_{MALLOC,REALLOC,NEW,RESIZE,FREE}, PyObject_{New,NewVar,Del}, and
PyObject_GC_{New,NewVar,Del}.


  Neil