[Python-Dev] Activating pymalloc

[M.-A. Lemburg]

Tim Peters wrote:
> 
> [Michael Hudson]
> > I think one of the reasons lots of modules break is that back in the
> > 1.5.2 days, PyObject_Del didn't exist, and you were expected to use
> > PyMem_Del.  This now breaks.
> 
> You mean this breaks if pymalloc is enabled, right?  Else they're just more
> ways to spell "free()".

The problem is that if you have an extension which was written for
1.5.2 this extension will use PyMem_Del() for object deallocation
rather than PyObject_Del(). This will break as soon as PyObject_Del()
maps to something different than PyMem_Del() since the old code
will continue to call PyMem_Del().

> I wonder whether it's actually a good idea for PyMem_New (etc) to keep on
> calling malloc/free even when pymalloc is enabled.  Anyone think they know a
> reason (other than theoretical purity <wink>) for why pymalloc only targets
> the PyObject family? 

Depends on how you define PyMem_* ... 

If you are saying that
these macros/functions are only intended to allocate memory
blocks which Python will use, then it's probably OK to let
pymalloc handle these as well.

If, for some reason, you intend to share the memory
with some other process/DLL/whatever, then I'm not
sure if this continues to work with pymalloc -- could
be that the platform's malloc() uses flags/locks etc.
to manage sharing. Just a thought, probably just FUD.

ALAS, the only way to find out, is to try it.

> If Michael is right, the current distinction is indeed
> guaranteed to break just about every extension written for 1.5.2.  It's also
> plain irritating that if you have, e.g., a gazillion tiny lists, only the
> list headers will enjoy pymalloc's speed & memory savings (the guts still
> get allocated via PyMem_XXX).

Agreed.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/