[Python-Dev] Re: Activating pymalloc

[Vladimir Marangozov]

Hi,

[Neil]
> 
> > API Summary
> > -----------
> > 
> > Raw malloc API:       PyMem_{MALLOC, REALLOC, FREE}
> >                       PyMem_{NEW, DEL}
> > 
> >                       PyMem_{Malloc, Realloc, Free}
> >                       PyMem_{New, Del}
> > 
> > Object malloc API:    PyObject_{MALLOC, REALLOC, FREE}
> >                       PyObject_{Malloc, Realloc, Free}
> > 
> >                       PyObject_{New, NewVar, Del}
> >                       PyObject_GC_{New, NewVar, Del}
> > 
> > Python's internal
> > malloc API:           PyCore_{Malloc, Realloc, Free}
> 
> That doesn't solve the problem of broken extension modules 
> (modules that mix PyMem_{NEW, DEL, MALLOC, FREE}, PyObject_{New, Del}, 
> malloc and free or that call PyObject_* with the GIL).

Whatever you do, broken extensions will remain broken.
Currently they work because we don't use another malloc.

To save pain with pymalloc enabled, though, you can redirect
PyObject_{NEW, NEW_VAR, DEL} to PyObject_{New, NewVar, Del}
because the former would be deprecated under the 'new' scheme.
This would also have the side effect on 'fixing' extensions
that use the macros instead of the functions.

> 
> We can do two things about this problem.  First, we can say the hell
> with broken extension modules and keep the current API. Alternatively,
> we can expose a new object malloc API that is documented not to be
> thread safe and that may not use the system malloc() and free().

This doesn't repair broken extensions.
Under the second suggestion, all APIs would be malloc/free, and you'll
introduce yet another one, just to find out after some time that people
will mix this new API with the others just as well...

The doc issue is important, though, and a warning in NEWS is mandatory
if pymalloc is enabled.


Vladimir