[Python-Dev] Lazily GC tracking tuples

[Tim Peters]

[Kevin Jacobs]
> I didn't count it as a strike against the patch -- I had just hoped that
> untracking tuples would result in faster execution than turning GC off and
> letting my heap swell obscenely.

Kevin, keep in mind that we haven't run your app:  the only things we know
about it are what you've told us.  That your heap swells obscenely when gc
is off is news to me. *Does* your heap swell obscenely when turning gc off,
but does not swell obscenely if you keep gc on?  If so, you should keep in
mind too that the best way to speed gc is not to create cyclic trash to
begin with.

> One extreme case could happend if I turn off GC, run my code, and let it
> fill all of my real memory with tuples, and start swapping to disk.
> Clearly, keeping GC enabled with the tuple untracking patch would result
> in huge performance gains.

Sorry, this isn't clear at all.  If your tuples aren't actually in cycles,
then whether GC is on or off is irrelevant to how long they live, and to how
much memory they consume.  It doesn't cost any extra memory (not even one
byte) for a tuple to live in a gc list; on the flip side, no memory is saved
by Neil's patch.

> This is not the situation I was dealing with, though I was hoping for a
> relatively smaller improvement from having a more compact and (hopefully)
> less fragmented heap.

Neil's patch should have no effect on memory use or fragmentation.  It's
only aiming at reducing the *time* spent uselessly scanning and rescanning
and rescanning and ... tuples in bad cases.  So long as your tuples stay
alive, they're going to consume the same amount of memory with or without
the patch, and whether or not you disable gc.