[Python-Dev] Provoking Jim's MRO segfault before shutdown

Tim Peters tim at zope.com
Thu Nov 13 10:12:14 EST 2003

>> Sorry, not so -- the "mini gc pass" of the same gc invocation would
>> collect all million of the other objects in vanilla trash cycles.
>> It's only weakref callbacks sick enough to install brand new weakref
>> callbacks on dead objects that would prevent the other trash from
>> getting collected in the same gc invocation.  There wasn't anything
>> like that in the segfaulting program.

[Barry Warsaw]
> When Python's shutting down, will there /be/ another GC invocation?

New in 2.3, gc is forced twice by Py_Finalize.  But it's quite possible for
a weakref callback that itself installs new weakref callbacks to objects in
unreachable (dead) cycles, and then resurrects those dead objects, to create
a situation where no number of gc collections can suffice, not under the
proposed scheme, nor under the current scheme, nor under any scheme -- the
programmer has then set things up so that, no matter how often we try to
clean up the trash, their code keeps resurrecting part of it, then pretends
to kill it off again, etc etc.  So it's always (under any scheme) possible
to write code that will leave a weakref callback uncalled at the time Python
does its C-level exit().  But at best, I think that's pathological code.
It's not a plausible use case, except to ensure that it's not a way to crash
the interpreter.

Under the proposed scheme, there's no issue here *except* for code that
(ab)uses weakref callbacks to install new weakref callbacks in their bodies,
and attaches the callbacks objects that are unreachable from outside a dead
clump of cyclic trash containing both the object running the original
weakref callback and the object that triggered the weakref callback.

BTW, I think Python should drop its second call of garbage collection in
Py_Finalize, and *possibly* its first call too.  The second call happens
after modules have been torn down, so callbacks or __del__ methods run then
are quite likely to suffer unexpected exceptions (module globals are None,
sys.stdout no longer exists, etc).  That second call is what triggered Jim's
original segfault; was the cause of the mysterious chain of information-free
messages when the Zope3 test suite finished (before we cleaned up forgotten
daemon threads); and is the cause of similar new shutdown irritations
reported on c.l.py.

The first call in Py_Finalize suffers a different problem:  because the
global C-level "initialized" flag has been set false by the time it's
called, any Python-level code run as a result of garbage collection that
tries to load a module gets a baffling (to the user) Py_FatalError
complaining that Python isn't initialized.  I stumbled into that one by
accident while trying to reproduce Jim's problem, and that's the only report
of it I know of  So I'm not excited <wink> about that one, but a
Py_FatalError at shutdown is sure going to attract attention when somebody
else stumbles into it.

More information about the Python-Dev mailing list