[Python-Dev] Making weakref callbacks safe in cyclic gc
tim at zope.com
Mon Nov 17 16:59:25 EST 2003
> There's a big difference between __del__ and weakref callbacks.
> The __del__ method is "internal" to a design. When you design a
> class with a del method, you know you have to avoid including the
> class in cycles.
> Now, suppose you have a design that makes has no __del__ methods but
> that does use cyclic data structures. You reason about the design,
> run tests, and convince yourself you don't have a leak.
> Now, suppose some external code creates a weak ref to one of your
> objects. All of a sudden, you start leaking. You can look at your
> code all you want and you won't find a reason for the leak.
I think that's an excellent argument -- thanks.
> To protext yourself against this, you'd need a way of preventing
> wekrefs to your class instances.
Not just to them, but also to anything in a cycle with one of your class
instances. This may include the class itself, or instance bound method
objects I got hold of as "a callable" from somewhere else, and where I had
no idea that your class is involved. It becomes intractable then for both
the class designer and the weakref user.
The patch I posted seemed correct for the problem it was solving.
Unfortunately, that wasn't the real problem <wink>. However, instead of
identifying the transitive closure of objects reachable from trash objects
with a weakref callback, it could compute the transitive closure of
objects reachable from (all) the callbacks
associated with trash objects having a (at least one)
Don't call tp_clear on those objects, and everything callbacks see will be
wholly intact. Apart from a pile of new hair to compute that complicated
set instead, the rest of the patch is probably fine.
The other plausible idea is fixing the glitch with the simpler-at-first "do
tp_clear on trash weakref objects first" idea. The problem with that is
that doing tp_clear on a weakref (or proxy) object ends up decref'ing the
callback, and the callback may *itself* have a weak reference to it, so that
decref'ing the callback triggers a different callback, and again arbitrary
Python code starts running in the middle of gc.
More information about the Python-Dev