[Python-Dev] Changing pymalloc behaviour for long running
dan at sidhe.org
Tue Oct 19 18:29:15 CEST 2004
At 12:14 PM -0400 10/19/04, Tim Peters wrote:
> > The short version of the problem is that obmalloc.c never frees memory.
>True. That's one major problem for some apps. Another major problem
>for some apps is due to unbounded internal free lists outside of
>obmalloc. Another is that the platform OS+libc may not shrink VM at
>times even when memory is returned to the system free().
FWIW, at this point nearly all OSes have a means of allocating memory
from the system that can then later be returned to the system.
(malloc and free tend *not* to do this) Even on Unix platforms you
can play the "mmap a file with no filename" game to get returnable
chunks. Unfortunately there's often a limit to the number of these
chunks you can get from the OS so it's not safe to unconditionally
replace malloc and free. (The performance impact isn't worth it
If someone's going to do this, I'd suggest the place to start is
adding separate allocate and free API entry points for returnable
chunks and put in some criteria for getting memory from them
(allocation size, particular spots that allocate, or whatever) and
see where you go from there.
I'll point out that, from experience, this can be a non-trivial
thing, and with a non-moving GC system you'll probably find that
there are relatively few places where there's a win from it. It does,
though, tend to flush out dangling pointers. (Whether this is good or
bad is a separate issue, of course ;)
--------------------------------------it's like this-------------------
Dan Sugalski even samurai
dan at sidhe.org have teddy bears and even
teddy bears get drunk
More information about the Python-Dev