[Python-Dev] Moving forward on the object memory API

[Guido van Rossum]

> It looks like Tim managed to work some magic and make pymalloc's free
> handle memory allocated by either the system allocator or pymalloc
> itself.  That means we can make PyMem_DEL use the pymalloc's 'free' and
> make PyObject_NEW use pymalloc's 'malloc'.  Here's my plan for making
> this happen:
> 
>     #define PyMem_MALLOC(n)         malloc(n)
>     #define PyMem_REALLOC(p, n)     realloc((void *)(p), (n))
>     #define PyMem_FREE(p)           free((void *)(p))

+1

> I think making PyMem_FREE call free() is safe.  I haven't seen any
> extension modules allocate with PyObject_NEW and free with PyMem_FREE.
> We deprecate the PyMem_* functions.  There's no need for them, IMHO:
> 
>     #define PyMem_Malloc PyMem_MALLOC
>     #define PyMem_New PyMem_NEW
>     #define PyMem_Resize PyMem_RESIZE
>     #define PyMem_Free PyMem_DEL
>     #define PyMem_Del PyMem_DEL
>     /* See comment near MALLOC_ZERO_RETURNS_NULL in pyport.h. */
>     #define PyMem_Realloc(p, n) \
>             do { \
>                     size_t _n = n; \
>                     PyMem_REALLOC(p, _n ? _n : 1); \
>             } while (0)

+1

> Next, we base PyObject_{MALLOC,REALLOC,FREE} on pymalloc.  Basically:
> 
>     #ifdef WITH_PYMALLOC
>     #define PyObject_MALLOC(n)      _PyMalloc_Malloc(n)
>     #define PyObject_REALLOC(op, n) _PyMalloc_Realloc((void *)(op), (n))
>     #define PyObject_FREE(op)       _PyMalloc_Free((void *)(op))
>     #else
>     #define PyObject_MALLOC(n)      PyMem_MALLOC(n)
>     #define PyObject_REALLOC(op, n) PyMem_REALLOC((void *)(op), (n))
>     #define PyObject_FREE(op)       PyMem_FREE((void *)(op))
>     #endif

Couldn't these always use the first set of definitions?  I suppose
that when configured --without-pymalloc, _PyMalloc_Malloc and friends
are aliases for malloc?

Also, do we need these at all?  (Hm, I must be missing something, but
I can't figure what.)

> PyMem_DEL and PyMem_Free need to call pymalloc:
> 
>     #define PyMem_DEL(op) PyObject_FREE(op)

Why not _PyMalloc_Malloc?

> We go through the source and replace all the PyMem_* function calls with
> the equivalent PyMem_* macro calls and replace PyMem_DEL with
> PyMem_FREE.

Or PyObject_Del (depending on the situation)?

> The recommended API for extension modules writers would be
> PyMem_{MALLOC,REALLOC,NEW,RESIZE,FREE}, PyObject_{New,NewVar,Del}, and
> PyObject_GC_{New,NewVar,Del}.

Is there a difference between PyMem_MALLOC and PyMem_NEW?  Between
PyMem_REALLOC and PyMem_RESIZE?  If there isn't, shouldn't we
recommend one set and deprecate the other?  I think I like the NEW
family best (it's got seniority).

I suppose the patches that changed PyObject_New into PyMalloc_New can
be reverted now?

Finally, there still seem to be too many macros and functions. But
it's better than before!

--Guido van Rossum (home page: http://www.python.org/~guido/)