[Python-Dev] Making weakref callbacks safe in cyclic gc
jim at zope.com
Mon Nov 17 15:33:19 EST 2003
Tim Peters wrote:
>>>If the callback itself is part of the garbage getting collected,
>>>then the weakref holding the callback must also be part of the
>>>garbage getting collected (else the weakref holding the callback
>>>would act as an external root, preventing the callback from being
>>>part of the garbage being collected too).
>>>My thought then was that a simpler scheme could simply call tp_clear
>>>on the trash weakrefs first. Calling tp_clear on a weakref just
>>>throws away the associated callbacks (if any) unexecuted, and if
>>>they don't get run then we have no reason to care what's reachable
>>>from them anymore.
> [Neil Schemenauer]
>>This I don't get. Don't people want the callbacks to be called?
As Tim pointed out, not if the weakref object dies before the object it
references. I agree with Tim that if both the weakref and the object
it references are in a cycle, it makes sense to remove the weakrefs first.
> We can certainly repair that by treating objects with callbacks the same as
> objects with __del__ methods when they're in cyclic trash, and that's an
> easy change to the implementation. Then the objects with callbacks, and
> everything reachable from them, leak unless/until the user snaps enough
> cycles in gc.garbage.
I think this would be really really bad.
> I don't have a feel for how much trouble it would be to avoid running afoul
> of that. Jim has so far presented it as an unacceptable burden.
There's a big difference between __del__ and weakref callbacks.
The __del__ method is "internal" to a design. When you design a
class with a del method, you know you have to avoid including the
class in cycles.
Now, suppose you have a design that makes has no __del__ methods but that
does use cyclic data structures. You reason about the design, run tests,
and convince yourself you don't have a leak.
Now, suppose some external code creates a weak ref to one of your objects.
All of a sudden, you start leaking. You can look at your code all
you want and you won't find a reason for the leak.
To protext yourself against this, you'd need a way of preventing wekrefs to
your class instances.
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
More information about the Python-Dev