[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

[Neil Schemenauer]

Neil Schemenauer <nas-python at arctrix.com> added the comment:

> Fleshing out something I left implicit: if there's a trash object T
> with a finalizer but we don't KNOW it's trash, we won't force-run its
> finalizer before delete_garbage starts either.  Then, really the same
> thing: we may tp_clear some piece of trash T's finalizer needs before
> enough cycles are broken that T's finalizer gets run as a routine
> consequence of T's refcount falling to 0.

Definition: a 'valid' object is one that hasn't had tp_clear called

I think the difference is that non-weakref finalizers have strong
references to objects that they can access when they run.  So, if we
haven't found them, they will keep all the objects that they refer
to alive as well (subtract_refs() cannot account for those refs).
So those objects will all be valid.

There seems a hole though.  Non-weakref finalizers could have a
weakref (without callback) to something in the garbage set.  Then,
when the finalizer runs during delete_garbage(), that finalizer code
can see non-valid objects via the weakref.  I think this can only happen if there are missing/incomplete tp_traverse methods.

We can have finalizer code running during delete_garbage().  That
code should not have access to non-valid objects.  Weakrefs seem be
a way to violate that.  handle_weakrefs() take care of some of them
but it seems there are other issues.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38006>
_______________________________________