On 3/31/06, Jim Jewett <jimjjewett@gmail.com> wrote:
The checkins list has been struggling with generator reference leaks;
the latest conclusion was that some are unavoidable because of __del__
cycles.

That was Tim's conclusion, but I wasn't quite done thinking about it ;)
 
That sort of defeats the purpose of resource managers.  (Yes,
it can be worked around on a case-by-case basis.)

As I see it, part of the problem is that

(1)  When there is a cycle, python refuses to guess.
(2)  There is no way for a __del__ method to hint at ordering constraints.
(3)  There is no lightweight version of __del__ to say "I don't care
about ordering constraints."

An additional (and much more complicating) problem is that __del__ can (and is allowed to) revive 'self'. That means that halfway through your cleanup, however you decided to do it, you can suddenly find out that you shouldn't be cleaning up at all. And of course any given __del__ can rely on all of the parts of the cycle, even if one of those parts _claims_ it can safely break the cycle.

I think there are three different scenarios involving cycles and/or __del__ methods:

 - An object may conciously create a cycle, and know how to resolve it. A '__breakcycle__' method or such may be the right way to handle those cases. It would have to be pretty sure that no one outside itself can (or should) rely on the attribute or closure it breaks the cycle with, though. (Sensible exceptions ought to be fine, IMHO.)

 - An object may not care about cycles, but want to do some cleanup when it is deleted. The C types have 'tp_dealloc' for this, and that's what PyFile uses to close files. If you want to emulate this behaviour in Python, you are forced to use __del__, creating the unreclaimable cycle problem. Perhaps a __dealloc__ class/staticmethod makes sense; it would be passed a dictionary of instancedata, but not 'self', so it can never revive 'self' and can be sanely used in cycles -- that is, some of its instancedata may still suddenly be None, but that's a problem with __del__ methods and global variables, too.

 - An object may care about cycles, actually need a __del__ method that can revive the object, but not have a sane way to say 'break the cycle at this point'. Generators may be an example of that kind of object, although I'm not sure: they could throw() an exception to end the cycle. I don't think we can reclaim cycles where none of the objects can fairly break the cycle.

--
Thomas Wouters <thomas@python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!