Pre-existing bug of the millennium (so far)
![](https://secure.gravatar.com/avatar/e79e2064f0ec68a0493ca5de758aeaa4.jpg?s=120&d=mm&r=g)
This one came up while working on ZODB: weakref callback vs. gc vs. threads http://www.python.org/sf/1055820 Short course: in the presence of weakrefs, cyclic gc is still hosed (it turns out that neither threads nor weakref callbacks are necessary to get hosed). temp2a.py there demonstrates there's a problem, but in an unclear way (hundreds of objects, hundreds of weakrefs and weakref callbacks (all via WeakValueDictionary internals), 3 threads). OTOH, there's nothing clever or tricky about it. Sooner or later, it just fails (an accessible instance of a user-defined class gets its __dict__ cleared "by magic"). temp2b.py reduces it to 2 objects and 1 thread. This is contrived, but is deterministic. temp2c.py boosts it to 3 objects, and is a nightmare: it shows that the problem can occur during a gc collection that doesn't see *any* objects having a weakref with a callback. There is a weakref with a callback here, but it's attached to an object in an older generation, and collection of a younger generation triggers that callback indirectly. Because this is such a nasty case (no amount of analysis of the objects in the generation being collected can deduce that it's possible for a weakref callback to run), there are extensive comments and an ASCII-art diagram in the file. Even worse, temp2d.py shows we can get in trouble even if there's never a weakref with a callback. It's enough to have one weakref (without a callback), and one object with a __del__ method. Offhand, I don't have a plausible solution that will work. The elegant <wink> analysis in gc_weakref.txt missed what should have been obvious even then: cyclic trash is still potentially reachable via "global" weakrefs, so if any Python code whatsoever can run while gc is breaking cycles (whether via __del__ or via wr callback), global weakrefs can resurrect cyclic trash. That suggests some Draconian approaches. Anyone have a bright idea? It's remarkable how long we've managed to go without noticing that everything is disastrously broken here <0.9 wink>.
participants (1)
-
Tim Peters