[Patches] [ python-Patches-530556 ] Enable pymalloc

noreply@sourceforge.net noreply@sourceforge.net
Mon, 18 Mar 2002 15:23:37 -0800


Patches item #530556, was opened at 2002-03-16 00:01
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=530556&group_id=5470

Category: Core (C code)
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Neil Schemenauer (nascheme)
Assigned to: Neil Schemenauer (nascheme)
Summary: Enable pymalloc 

Initial Comment:
The attached patch removes the PyCore_* memory
management layer and gives up on the hope that
PyObject_DEL() will ever be anything but free().

pymalloc is given a visible API in the form of
PyMalloc_Malloc, PyMalloc_Realloc, PyMalloc_Free.  
A new object memory interface is implemented
on top of pymalloc in the form of
PyMalloc_{New,NewVar,Del}.  Those are ugly names.
Please suggest alternatives.

Some objects are changed to use pymalloc.  The
GC memory functions are changed to use pymalloc.

The configure support for enabling pymalloc was 
also removed.  Perhaps that should be left in so
people can disable pymalloc on low memory machines.

I left typeobject using the system allocator (new style
classes will not use pymalloc).  Fixing that is
probably a job for Guido. 


----------------------------------------------------------------------

>Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-18 23:23

Message:
Logged In: YES 
user_id=35752

Oops, forgot one important change in the last update.
PyObject_MALLOC needs to use PyMem_MALLOC not 
_PyMalloc_MALLOC. Clear as mud, no? :-)

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-18 23:08

Message:
Logged In: YES 
user_id=35752

Update patch to latest CVS.  It's now about 1/3 of its
original size.  We still need documentation for
PyMalloc_{New,NewVar,Del}.

Other than the docs, the only thing left to do is decide if we
want the new API.  The situation with extension modules is
not as bad as I originally thought.  The xxmodule.c example
has been correct since version 1.6.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-17 19:32

Message:
Logged In: YES 
user_id=31435

I certainly want, e.g., that our Unicode implementation can 
choose to use obmalloc.c for its raw string storage, 
despite that it isn't "object storage" (in the sense of 
Vladimir's level "+2" in the diagram at the top of 
obmalloc.c; the current CVS code restricts obmalloc use to 
level +2, while raw string storage is at level "+1").

Allowing to use pymalloc at level +1 changes Vladimir's 
original intent, and we have no experience with it, so I'm 
fine with restricting that ability to the core at the start.

About names, we've been calling this package "pymalloc" for 
years, and the general form of external name throughout 
Python is

    ["_"] "Py" Package "_" Function

_PyMalloc_{Malloc, Free, etc} fit that pattern perfectly.  
I don't see the attraction to giving functions from this 
package idiosyncratic names, and we've got so many ways to 
spell "get memory" that I expect it will be a genuine help 
to keep on making it clear, from the name alone, to 
which "family" a given variant of "new" (etc) belongs.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-17 17:11

Message:
Logged In: YES 
user_id=35752

I'm not sure exactly what Tim meant by that comment.  If we
want to make PyMalloc available to EXTENSION modules then,
yes, we need to remove the leading underscope and make a
wrapper for it.  I would prefer to keep it private for
now since it gives us more freedom on how PyMalloc_New
is implemented.  Tim?

Regarding the names, I have no problem with Py_Malloc.  If
we change should we keep PyMalloc_{New,NewVar,Del}?  Py_New
seems at little to short.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-03-17 10:12

Message:
Logged In: YES 
user_id=21627

The patch looks good, except that it does not meet one of
Tim's requirements: there is no way to spell "give me memory
from the allocator that PyMalloc_New uses". _PyMalloc_Malloc
is clearly not for general use, since it starts with an
underscore.

What about calling this allocator (which could be either
PyMalloc or malloc) Py_Malloc, Py_Realloc, Py_Free?

Also, it appears that there is no function wrapper around
this allocator: A module that uses the PyMalloc allocator
will break in a configuration where pymalloc is disabled.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-16 03:50

Message:
Logged In: YES 
user_id=35752

Okay, with-pymalloc is back but defaults to enabled.  The
functions PyMalloc_{Malloc,Realloc,Free} have been renamed
to _PyMalloc_{Malloc,Realloc,Free}.  Maybe their ugly names
will discourage their use.  People should use
PyMalloc_{New,NewVar,Del} if they want to allocate objects
using pymalloc.

There's no way we can reuse PyObject_{New,NewVar,Del}.
Memory can be allocated with PyObject_New and freed with
PyObject_DEL.  That would not work if PyObject_New used
pymalloc.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-03-16 00:54

Message:
Logged In: YES 
user_id=21627

-1. --with-pymalloc should remain an option; there is still
the heuristics in releasing memory that may people make
uncomfortable. Also, on systems with super-efficient malloc,
you may not want to use pymalloc.

I dislike the name PyMalloc_Malloc; it may be acceptable for
the allocation algorithm itself (although it sounds funny).
However, for the PyObject allocator, something else needs to
be found.

I can't really see the problem with calling it
PyObject_New/_NewVar/_Del. None of these where available in
Python 1.5.2, so I don't think 1.5.2 code could break.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=530556&group_id=5470