[Python-Dev] Activating pymalloc

[Martin v. Loewis]

Tim Peters <tim.one@comcast.net> writes:

> but the crucial phrase "the same memory API family" is left undefined.  At a
> minimum, it needs to identify "the memory API families" by explicit,
> exhaustive enumeration.  The mess in practice is a fact, and proves people
> don't understand what the docs are *trying* to say here.  I'm not sure
> myself.

I always thought the notion of a "memory API family" is fairly obvious
- none of those families has more than three members. There is always
one for allocation and one for deallocation, and perhaps one for
reallocation. That's it.

> Like, are the functions and macros interchangeable?  

Of course not. You use the macros for efficiency, accepting that you
incorporate the implementation in your code, which results in a
depenency between the binary extension module and the binary Python
installation.

> For example, if you know something was allocated via PyMem_Malloc,
> is it OK to release it via PyMem_FREE?  I simply can't guess from
> what the docs say, again because what constitutes "a family" is left
> undefined.

If you buy the notion of memory API families, you cannot deallocate
the things this way. In Python, you will typically have the
deallocation code in the same translation unit as the allocation, so
this is an issue that rarely arises. If it does arise, whoever gives
you the memory should also tell you how to deallocate it.

Of course, you will get away with mixing the macro and non-macro
versions, atleast if you are not crossing Python version boundaries.
Do that and you are on your own, though (if desired, Python could
easily guarantee that you can mix APIs in this way, but I see no need
for that guarantee).

> So far we've covered more than two dozen spellings (even without plain
> "malloc" and "free"), spread over at least 4 manual pages.  One compact
> table listing all of them in "legal" pairs would be an enormous help.

If you think so ... If I can get the TeX to create a table for me,
I'll see whether I can produce a table. More likely, Fred will have to
fix it...

> Not well enough to use with confidence, no.  I've seen the docs before,
> although I did miss the Examples section previously.  I have a fancy editor
> with an integrated symbol database, and in practice I chase down the macros
> and functions until I see what they *do* at the leaves.  The endless layers
> of indirection macros make this quite a pain

I'm also uncomfortable with those. Fortunately, the family of the
"object allocator" does not deserve being documented - I doubt anybody
will ever need this. Also, the overriding hooks for replacing the
function names and signatures don't need to be documented.

> Which API?

PyObject_MALLOC/REALLOC/FREE, and PyCore_* (both object and raw).

> Well, to a first approximation, just searching for "free(" is valuable!  In
> the Python source, the thread implementations use malloc and free on
> purpose, but it also turns up a curious
> 
> 		    free((char *)Py_FileSystemDefaultEncoding);
[...]
> Does that mean that, on Windows, we may free() a static char*?!

Of course not. This is wrapped in

#if defined(HAVE_LANGINFO_H) && defined(CODESET)

neither of which is defined on Windows.

Regards,
Martin