[Python-Dev] Re: Activating pymalloc

[Martin v. Loewis]

Vladimir Marangozov <vladimir.marangozov@optimay.com> writes:

> Whatever you do, broken extensions will remain broken.
> Currently they work because we don't use another malloc.

Neil's point is that with his approach, we can activate pymalloc, and
the broken extensions continue to work. This is a desirable property.

> To save pain with pymalloc enabled, though, you can redirect
> PyObject_{NEW, NEW_VAR, DEL} to PyObject_{New, NewVar, Del}
> because the former would be deprecated under the 'new' scheme.
> This would also have the side effect on 'fixing' extensions
> that use the macros instead of the functions.

The counter argument here is that people even use PyObject_New
incorrectly, releasing the memory with some other API. Under Neil's
patch, module authors would have to actively change their extensions
to make use of pymalloc. They would also have a chance to test their
code, since pymalloc would be activated in the standard installation.

> > Alternatively,
> > we can expose a new object malloc API that is documented not to be
> > thread safe and that may not use the system malloc() and free().
> 
> This doesn't repair broken extensions.

Well, there are broken extensions, and extensions that don't work
properly. Merely activating pymalloc might have the effect that broken
extensions which currently work properly stop working. Under Neil's
change, they will continue to work.

Yes, this does mean that all PyObject_* families of memory API become
deprecated.

> Under the second suggestion, all APIs would be malloc/free, and you'll
> introduce yet another one, just to find out after some time that people
> will mix this new API with the others just as well...

People won't do that, because their code crashes when they test it. It
is the "it crashes but I changed NOTHING" cases that backwards
compatibility is concerned about.

Regards,
Martin