[Python-ideas] cross platform memory usage high water mark for improved gc?

Adam Olsen rhamph at gmail.com
Mon Mar 10 17:30:56 CET 2008

On Mon, Mar 10, 2008 at 8:33 AM, Aaron Watters <aaron.watters at gmail.com> wrote:
> Hi.  Some months ago I complained on the python-list
> that python gc did too much work for apps that allocate
> and deallocate lots of structures.  In fact one of my apps
> was spending about 1/3 of its time garbage collecting
>  and not finding anything to collect (before i disabled gc).
> My proposal was that python
> should have some sort of a smarter strategy for garbage
> collection, perhaps involving watching the global
> high water mark for memory allocation or other tricks.
> The appropriate response was:
> "great idea! patch please!" :)
> Unfortunately dealing with cross platform
> memory management internals is beyond my
> C-level expertise, and I'm not having a lot of
>  luck finding good information sources.  Does anyone
> have any clues on this or other ideas for improving
> the gc heuristic?  For example, how do you find
> out the allocated heap size(s) in a cross platform
> way?
> This link provides some clues, but I don't really
> understand this code well enough to hope to
> patch gc.
> http://www.xfeedme.com/nucular/pydistro.py/go?FocusId=74&FREETEXT=high%20water%20mark

You can of course tweak gc.set_threshold() (and I would expect this to
be quite effective, once you find out what an appropriate threshold0
is for your app.)  I don't believe you'll find any existing counters
of the current heap size though (be it number of allocated objects or
total size consumed by those objects.)

Adam Olsen, aka Rhamphoryncus

More information about the Python-ideas mailing list