[Python-Dev] reference leaks, __del__, and annotations
Phillip J. Eby
pje at telecommunity.com
Mon Apr 3 23:14:12 CEST 2006
At 08:14 AM 3/31/2006, Tim Peters wrote:
>[Phillip J. Eby]
> > ...
> > As Tim suggested, it'd be better to have the code be generator-specific, at
> > least for now. That had actually been my original plan, to make it
> > generator-specific, but I was afraid of breaking encapsulation in the
> > garbage collector by having it know about generators.
>It sucks in a way, but so would adding yet another new slot just for
>(at present, and possibly forever) making gc and generators play nicer
>together. "Practicality beats purity" here.
I'm trying to figure out how to implement this now, and running into
a bit of a snag. It's easy enough for gcmodule.c to check if an
object is a generator, but I'm not sure how safe the dynamic check
actually is, since it depends on the generator's state. In
principle, running other finalizers could cause the generator's state
to change from a finalizer being required to not being required, or
vice versa. Could this mess up the GC process? It seems to me that
it's safe for a generator to say, "yes, I need finalization", because
if it later turns out not to, it's just a waste. But if the generator
says, "no, I don't need finalization", and then later turns out to
need it, doesn't that leave an opportunity to screw things up if the
GC does anything other than immediately clear the generator?
As best I can tell, the only things that could cause arbitrary Python
code to run, that could possibly result in generator state changes,
are structure traversal and weakref callback handling. It's probably
not sane for anybody to have structure traversal run arbitrary Python
code, so I'm going to ignore that for the sake of my own
sanity. :) Weakref callbacks are tougher, though; it seems possible
that you could have one of those cause a generator to be advanced to
a point where it now needs finalization.
OTOH, if such a generator could be advanced by the callback, then
wouldn't that mean the generator is reachable, and ergo, not
garbage? That is, since only reachable weakref callbacks are run,
they must by definition be unable to access any generator that
declared itself finalizer-free.
It does seem possible you could end up with a situation where an
object with a finalizer is called after a generator it references is
torn down, but that circumstance can occur in earlier versions of
Python anyway, and in fact this behavior would be consistent.
Okay, I *think* I've convinced myself that a dynamic state check is
OK, but I'm hoping somebody with more GC experience can check my
reasoning here for holes.
More information about the Python-Dev