[Python-Dev] Suggested memory API rules for 2.3

Guido van Rossum guido@python.org
Wed, 03 Apr 2002 11:56:56 -0500


> "Programs" does not include the Python core.  We can do anything we
> need to do in the core, based on our complete <heh> knowledge of
> each release's internal implementation details.

However, since many people use the core as an example, we may have to
be careful here.  Certainly xxmodule.c, xxobject.c and xxsubtype.c
should do the recommended thing; but if too many of the built-in
objects use the macros despite those being in danger of deprecation,
that might still perpetuate use of those macros.

>     Mixing and Matching
> 
>     After memory has been obtained via one of these functions, it
>     should be resized and freed only by a function from the same line,
>     except that PyMem_Free may be used freely in place of PyMem_Del,
>     and PyObject_Free in place of PyObject_Del.

I actually had a slightly different rule in mind: mixing and matching
between two adjacent lines that only differ by case is also permitted.
Except of course that the macro lines will be deprecated, so maybe it
doesn't matter to spell this out.

> Maintaining the Free/Del pseudo-distinction is pointless.
> 
>     For backward compatibility, memory obtained via the object memory
>     family can be freed by any of Py{Mem, Object}_{Free, FREE, Del, DEL}.
>     Mixing functions from the object family with the raw memory family
>     is likely to become deprecated,
> 
>     Memory obtained by PyMem_{Malloc, MALLOC} shall be resized only
>     by PyMem_{Realloc, REALLOC}.
> 
>     Memory obtained by PyMem_{New, NEW} shall be resized only
>     by PyMem_{Resize, RESIZE}.
> 
>     Memory obtained by PyObject_{Malloc, MALLOC} shall be resized
>     only by PyObject_{Realloc, REALLOC}.
> 
> Note that the eight ways to spell "free" all have to map to the
> pymalloc free when pymalloc is enabled in 2.3.  There is no way to
> spell "give me the raw platform free(), damn it", except for "free".
> If we think it's important to have a such a way in the core, it
> should be added to the private API.

IMO, PyMem_Malloc() and PyMem_Free() should be good enough.

>     Relationship to Platform Allocator
> 
>     All names in lines A, B, C and D ultimately invoke the platform
>     C malloc, realloc, or free.  However, programs shall not mix any
>     of these names with direct calls to the platform malloc, calloc,
>     realloc, or free referencing the same base memory addresses, as
>     Python may need to perform bookkeeping of its own around its calls
>     to the platform allocator.

Really?  Why not just say these are wrappers around malloc and free?
(On platforms where it matters, they will be guaranteed to use the
same heap as the core uses -- this is apparently an issue with Windows
DLLs.)

--Guido van Rossum (home page: http://www.python.org/~guido/)