[Python-Dev] Reference cycles in Exception.__traceback__

[Victor Stinner]

Hi,

Python 3 now stores the traceback object in Exception.__traceback__
and exceptions can be chained through Exception.__context__. It's
convinient but it introduced tricky reference cycles if the exception
object is used out of the except block.

Refrences: Exception.__traceback__ -> traceback -> frames -> local variables

Exception.__traceback__ keeps strong references to local variables
which will only be deleted after the exception is destroyed.

It becomes worse if the exception is stored in an object which is also
a local variables somewhere in the traceback:

DebugObject -> Exception -> ... frame -> DebugObject

For a concrete example of object storing an exception, see
Future.set_exception() of the ayncio module.

Python 3.4 introduced frame.clear(), but frame.clear() raises an
RuntimeError if the frame is still running. And it doesn't break all
reference cycles.

An obvious workaround is to store the traceback as text, but this
operation is "expensive" especially if the traceback is only needed in
rare cases.

I tried to write "views" of the traceback (and frames), but
Exception.__traceback__ rejects types other than traceback and
traceback instances cannot be created. It's possible to store the
traceback somewhere else and set Exception.__traceback__ to None, but
there is still the problem with chained exceptions.

Any idea for a generic fix to such problem?

Another workaround is to set Exception.__traceback__ to None in
release mode, and leaves it unchanged in debug mode. But it's not a
good solution because bugs also occur in production, and most tricky
bugs *only* occur in production :-)

For more info, see:

http://bugs.python.org/issue20032
http://code.google.com/p/tulip/issues/detail?id=42
http://code.google.com/p/tulip/issues/detail?id=155

Victor