[Python-Dev] Evil reference cycles caused Exception.__traceback__

[Martin (gzlist)]

Thanks for working on this and writing up the details Victor.

For those confused about why this matters, routinely having every
object in your application participating in one (or more) giant
reference cycles makes reasoning about and fixing resource issues very
difficult.

For instance, a while back I made some changes in Bazaar so that each
TestCase instance was dereferenced after being run:

<https://bugs.launchpad.net/bzr/+bug/613247>

Which mattered when running ~25000 tests, to keep peak memory usage
sane. Also there were various other object lifetime issues, and
tracking down which specific test failed to join a thread, or close a
file on Windows was much simpler after making sure cleanup actually
happened at the time a test was deleted.

Keeping more stuff alive for longer periods also makes peak memory
requirements higher, and the gc has to do more work each time.

On 18/09/2017, Victor Stinner <victor.stinner at gmail.com> wrote:
>
> Ideally, CPython 3.x should never create reference cycles. Removing
> Exception.__traceback__ is the obvious "fix" for the issue. But I
> expect that slowly, a lot of code started to rely on the attribute,
> maybe even for good reasons :-)
>
> A more practical solution would be to log a warning. Maybe the garbage
> collector can emit a warning if it detects an exception part of a
> reference cycle? Or maybe detect frames?

Logging a warning is unlikely to be practical. I had an optional test
flag that complained about reference cycles, and it produced a lot of
output.

Martin