[Python-ideas] cross platform memory usage high water mark for improved gc?

[Adam Olsen]

On Tue, Mar 11, 2008 at 7:28 AM, Aaron Watters <aaron.watters at gmail.com> wrote:
> On Mon, Mar 10, 2008 at 12:30 PM, Adam Olsen <rhamph at gmail.com> wrote:
> > You can of course tweak gc.set_threshold() (and I would expect this to
> > be quite effective, once you find out what an appropriate threshold0
> > is for your app.)  I don't believe you'll find any existing counters
> > of the current heap size though (be it number of allocated objects or
> > total size consumed by those objects.)...
>
>  It would be nice if the threshold would adjust based
>  on the performance characteristics of the app.
>  In particular it'd be nice if the garbage collector would
>  notice when it's never finding anything and wait longer
>  everytime it finds nothing for the next collection attempt.
>
>  How about this.
>  - The threshold slides between minimumThresh and maximumThresh
>  - At each collection the current number of objects
>    collected is compared to the last number collected (collectionTrend).
>  - If the collectionTrend is negative or zero the next threshold slides
>    towards the maximum.
>  - If the collectionTrend is a small increase, the threshold stays the same.
>  - If the collectionTrend is a large increase the next threshold slides
> towards
>    the minimum.
>  That way for apps that need no garbage collection
>  (outside of refcounting) the threshold would slide to the
>  maximum and stay there, but for apps that need a lot of
>  gc the threshold would bounce up and down near the minimum.
>
>  This is almost easy enough that I could implement it...
>      -- Aaron Watters

It sounds plausible to me.

But have you tried just tweaking the threshold?  Surely there's a
value at which it performs well, and that'd need to be within your
maximum anyway.

-- 
Adam Olsen, aka Rhamphoryncus