[Python-Dev] Evil reference cycles caused Exception.__traceback__
njs at pobox.com
Mon Sep 18 13:53:26 EDT 2017
On Mon, Sep 18, 2017 at 9:50 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 18 Sep 2017 09:42:45 -0700
> Nathaniel Smith <njs at pobox.com> wrote:
>> Obviously it's nice when the refcount system is able to implicitly clean
>> things up in a prompt and deterministic way, but there are already tools to
>> handle the cases where it doesn't (ResourceWarning, context managers, ...),
>> and the more we encourage people to implicitly rely on refcounting, [...]
> The thing is, we don't need to encourage them. Having objects disposed
> of when the last visible reference vanishes is a pretty common
> expectation people have when using CPython.
>> Why are reference cycles a problem that needs solving?
> Because sometimes they are holding up costly resources in memory when
> people don't expect them to. Such as large Numpy arrays :-)
Do we have any reason to believe that this is actually happening on a
regular basis though?
If it is then it might make sense to look at the cycle collection
heuristics; IIRC they're based on a fairly naive count of how many
allocations have been made, without regard to their size.
> And, no, there are no obvious ways to fix for users. gc.collect()
> is much too costly to be invoked on a regular basis.
>> Because if so then that seems
>> like a bug in the warnings mechanism; there's no harm in a dead Thread
>> hanging around until collected, and Victor may have wasted a day debugging
>> an issue that wasn't a problem in the first place...
> Yes, I think Victor is getting a bit overboard with the so-called
> "dangling thread" issue. But the underlying issue (that heavyweight
> resources can be inadvertently held up in memory up just because some
> unrelated exception was caught and silenced along the way) is a real
Simply catching and silencing exceptions doesn't create any loops -- if you do
except ValueError as exc:
then there's no loop, because the 'exc' local gets cleared as soon as
you exit the except: block. The issue that Victor ran into with
socket.create_connection is a special case where that function saves
off the caught exception to use later.
If someone wanted to replace socket.create_connection's 'raise err' with
then I guess that would be pretty harmless...
Nathaniel J. Smith -- https://vorpus.org
More information about the Python-Dev