[Python-Dev] Re: [Python-checkins]python/dist/src/Modules gcmodule.c,2.33.6.5,2.33.6.6

[Jeremy Hylton]

On Sat, 2003-04-05 at 14:34, Tim Peters wrote:
> While a __getattr__ side effect may resurrect an object in gc's unreachable
> list, gc has no way to know that an object has been resurrected short of
> starting over again.  In the absence of that, the object remains in gc's
> unreachable list, and its tp_clear slot eventually gets called.  The
> internal C stuff remains self-consistent, so this won't cause a segfault
> (etc), but it may (as above) be surprising.  I don't see a sane way to fix
> this so long as asking whether __del__ exists can execute arbitrary mounds
> of Python code.

I think I'll second the thought that there are no satisfactory answers
here.  We've made a big step forward by fixing the core dumps.

If we want to document the current behavior, we would say that garbage
collection may leave reachable objects in an "invalid state" in the
presence of "problematic objects."  A "problematic object" is an
instance of a classic class that defines a getattr hook (__getattr__)
but not a finalizer (__del__).  An object an in "invalid state" has had
its tp_clear slot executed; in the case of instances, this means the
__dict__ will be empty.  Specifically, if a problematic object is part
of unreachable cycle, the garbage collector will execute the code in its
getattr hook; if executing that code makes any object in the cycle
reachable again, it will be left in an invalid state.

If we document this for 2.2, it's more complicated because instances of
new-style classes are also affected.  What's worse, a new-style class
with a __getattribute__ hook is affected regardless of whether it has a
finalizer.

Here are a couple of thoughts about how to avoid leaving objects in an
invalid state.  It's pretty unlikely for it to happen, but speaking from
experience <wink> it's baffling when it does.

#1.  (I think this was Fred's suggestion on Friday.)  Don't do a
hasattr() check on the object, do it on the class.  This is what happens
with new-style classes in Python 2.3:  If a new-style class doesn't
define an __del__ method, then its instances don't have finalizer.  It
doesn't matter whether the specific instance has an __del__ attribute.

Limitations: This is a change in semantics, although it only covers a
nearly insane corner case.  The other limitation is that things could
still go wrong, although only in the presence of a classic metaclass!

#2.  If an object has a getattr hook and it's involved in a cycle, just
put it in gc.garbage.  Forget about checking for a finalizer.  That
seems fine for 2.3, since we're only talking about classic classes with
getattr hooks.  But it doesn't sound very pleasant for 2.2, since it
covers an class instance with a getattr hook.

I think #1 is pretty reasonable.  I'd like to see something fixed for
2.2.3, but I worry that the semantic change may be unacceptable for a
bug fix release.  (But maybe not, the semantics are pretty insane right
now :-).

Jeremy