[Guido]
Yes, the generator does clear its f_back when it's suspended.
[Phillip]
I realize this won't fix all your worries; I just want to rule out this one *particular* form of cycle as a possibility; i.e., to show that mere reference to a generator-iterator in a frame does not create a cycle:
callerframe ---------> traceback2 | ^ | | | | | | | +----------------+ | v v geniter -> genframe -> traceback1 ^ | | | +----------+
As you can see, the geniter itself doesn't have a reference to its calling frame, so as soon as the highest-level traceback object is released, the cycle collector should release the upper cycle, allowing the geniter to complete, and releasing the lower cycle.
The scenario assumes, by the way, that the traceback object referenced by a frame includes a pointer to that same frame, which I'm not sure is the case. I was under the impression that the current frame is only added to the traceback when the frame is exited, in which case the two cycles shown above wouldn't even exist; each traceback would be pointing to the *next* frame down, and there would be no cycles at all. It seems to me that this would almost have to be the design, since tracebacks existed before cyclic GC did.
Alas, your assumption is valid; this would indeed cause a cycle, much to the despair of early Python programmers. There used to be a whole body of literature about the best way to avoid this (never save a traceback, or if you do, clear it when you're done with it before exiting the frame). When you raise and immediately catch an exception, there is a single traceback object that references the current frame (the frame where it was raised *and* caught). So if you store sys.exc_info() in a local, you have a cycle already: try: raise Exception except: x = sys.exc_info()[2] # save the traceback Now we have the following cycle (with slightyl more detail than your diagram): tb_frame frame <------------- traceback | ^ | | v 'x' | f_locals ------------------+ BTW, note the repercussions this has for Ping's PEP 344 -- because the Exception instance references the traceback in that proposal, all code that catches an exception into a variable creates a cycle, like this: try: raise Exception except Exception, err: pass This would creates the following cycle: tb_frame frame <------------- traceback | ^ | | __traceback__ v | f_locals ---------------> err The good news in all this is that none of these objects has a __del__ method in the proposal; only the 'geniter' object would have one, and getting it involved in a cycle does seem like rather unlikely. I hereby declare my worries unwarranted and will happily add language to the revamped PEP 343 that a geniter object should have a tp_del slot and a corresponding __del__ attribute. This further complicates has_finalizer() in gcmodule.c, to the point where the latter might have to be turned into an internal slot.
Sure. But I still have some reservations, since cycles can pop up in the strangest of places (especially when tracebacks are involved -- tracebacks have been causing problems due to cycles and keeping variables alive almost since Python's inception). I posted about this a while ago, but don't recall seeing a response that took my fear away.
Well, I can't prove that it's not possible to create such cycles, certainly. But maybe we should finally get rid of the deprecated sys.exc_type/value/traceback variables, so that they can't root any cycles?
I sort of doubt that these are the main source of live cycles. After all, they are reset whenever a frame is popped (grep the sources for reset_exc_info). -- --Guido van Rossum (home page: http://www.python.org/~guido/)