[Python-Dev] __traceback__ and reference cycles

[Armin Rigo]

Hi all,

There are various proposals to add an attribute on exception instances
to store the traceback (see PEP 344).  A detail not discussed, which I
thought of historical interest only, is that today's exceptions try very
hard to avoid reference cycles, in particular the cycle

   'frame -> local variable -> traceback object -> frame'

which was important for pre-GC versions of Python.  A clause 'except
Exception, e' would not create a local reference to the traceback, only
to the exception instance.  If the latter grows a __traceback__
attribute, it is no longer true, and every such except clause typically
creates a cycle.

Of course, we don't care, we have a GC -- do we?  Well, there are cases
where we do: see the attached program...  In my opinion it should be
considered a bug of today's Python that this program leaks memory very
fast and takes longer and longer to run each loop (each loop takes half
a second longer than the previous one!).  (I don't know how this bug
could be fixed, though.)

Spoiling the fun of figuring out what is going on, the reason is that
'e_tb' creates a reference cycle involving the frame of __del__, which
keeps a reference to 'self' alive.  Python thinks 'self' was
resurrected.  The next time the GC runs, the cycle disappears, and the
refcount of 'self' drops to zero again, calling __del__ again -- which
gets resurrected again by a new cycle.  Etc...  Note that no cycle
actually contains 'self'; they just point to 'self'.  In summary, no X
instance gets ever freed, but they all have their destructors called
over and over again.

Attaching a __traceback__ will only make this "bug" show up more often,
as the 'except Exception, e' line in a __del__() method would be enough
to trigger it.

Not sure what to do about it.  I just thought I should share these
thoughts (I stumbled over almost this problem in PyPy).


A bientot,

Armin