[Python-Dev] Boom

[Tim Peters]

While enduring dental implant surgery earlier today, I thought to myself
"oops -- I bet this program will crash Python".  Turns out it does, in
current CVS, and almost certainly in every version of Python since cyclic gc
was added:

"""
import gc

class C:
    def __getattr__(self, attr):
        del self.attr
        raise AttributeError

a = C()
b = C()
a.attr = b
b.attr = a

del a, b
gc.collect()
"""

Short course:  a and b are in a trash cycle.  gcmodule's move_finalizers()
finds one of them and calls has_finalizer() to see whether it's collectible.
Say it's b.  has_finalizer() calls (in effect) hasattr(b, "__del__"), and
b.__getattr__() deletes b.attr as a side effect before saying b.__del__
doesn't exist.  That drops the refcount on a to 0, which in turn drops the
refcount on a.__dict__ to 0.  Those two are the killers:  a and a.__dict__
become untracked (by gc) as part of cleaning them up, but the
move_finalizers() "next" local still points to one of them (to the __dict__,
in the run I happened to step thru).  As a result, the next trip around the
move_finalizer() loop calls has_finalizer() on memory that's already been
free()ed.  Hilarity ensues.

The anesthesia is wearing off and I won't speculate about solutions now.  I
suspect it's easy, or close to intractable.  PLabs folks, I'm unsure whether
this relates to the ZODB test failure we've been bashing away at.  All, ZODB
is a persistent database, and at one point in this test gc determines that
"a ghost" is unreachable.  When gc's has_finalizer() asks whether the ghost
has a __del__ method, the persistence machinery kicks in, sucking the
ghost's state off of disk, and executing a lot of Python code as a result.
Part of the Python code executed does appear (if hazy memory serves) to
delete some previously unreachable objects that were also in (or hanging off
of) the ghost's cycle, and so in the unreachable list gc's move_finalizers()
is crawling over.

The kind of blowup above could be one bad effect, and Jeremy was seeing
blowups with move_finalizers() in the traceback.  Unfortunately, the test
doesn't blow up under CVS Python, and 2.2.2 doesn't have the telltale
0xdbdbdbdb filler 2.3's debug PyMalloc sprays into free()ed memory.