At 12:14 PM -0400 10/19/04, Tim Peters wrote:
The short version of the problem is that obmalloc.c never frees memory.
True. That's one major problem for some apps. Another major problem for some apps is due to unbounded internal free lists outside of obmalloc. Another is that the platform OS+libc may not shrink VM at times even when memory is returned to the system free().
FWIW, at this point nearly all OSes have a means of allocating memory from the system that can then later be returned to the system. (malloc and free tend *not* to do this) Even on Unix platforms you can play the "mmap a file with no filename" game to get returnable chunks. Unfortunately there's often a limit to the number of these chunks you can get from the OS so it's not safe to unconditionally replace malloc and free. (The performance impact isn't worth it either)
If someone's going to do this, I'd suggest the place to start is adding separate allocate and free API entry points for returnable chunks and put in some criteria for getting memory from them (allocation size, particular spots that allocate, or whatever) and see where you go from there.
I'll point out that, from experience, this can be a non-trivial thing, and with a non-moving GC system you'll probably find that there are relatively few places where there's a win from it. It does, though, tend to flush out dangling pointers. (Whether this is good or bad is a separate issue, of course ;)