[Python-ideas] cross platform memory usage high water mark for improved gc?

[Adam Olsen]

On Mon, Mar 10, 2008 at 8:33 AM, Aaron Watters <aaron.watters at gmail.com> wrote:
> Hi.  Some months ago I complained on the python-list
> that python gc did too much work for apps that allocate
> and deallocate lots of structures.  In fact one of my apps
> was spending about 1/3 of its time garbage collecting
>  and not finding anything to collect (before i disabled gc).
>
> My proposal was that python
> should have some sort of a smarter strategy for garbage
> collection, perhaps involving watching the global
> high water mark for memory allocation or other tricks.
>
> The appropriate response was:
> "great idea! patch please!" :)
>
> Unfortunately dealing with cross platform
> memory management internals is beyond my
> C-level expertise, and I'm not having a lot of
>  luck finding good information sources.  Does anyone
> have any clues on this or other ideas for improving
> the gc heuristic?  For example, how do you find
> out the allocated heap size(s) in a cross platform
> way?
>
> This link provides some clues, but I don't really
> understand this code well enough to hope to
> patch gc.
>
> http://www.xfeedme.com/nucular/pydistro.py/go?FocusId=74&FREETEXT=high%20water%20mark

You can of course tweak gc.set_threshold() (and I would expect this to
be quite effective, once you find out what an appropriate threshold0
is for your app.)  I don't believe you'll find any existing counters
of the current heap size though (be it number of allocated objects or
total size consumed by those objects.)


-- 
Adam Olsen, aka Rhamphoryncus