[Python-Dev] Re: weakref gc semantics

[Tim Peters]

[Michael Hudson]
> As disclaimed above, I haven't thought about this a great deal, but I
> would expect it to be the lifetime of the weakref which matters -- in
> the not-all-that-unlikley case of the callback not being a lambda or
> local function, it is *always* going to be reachable, and indeed is
> never going to die until the process exits...

A difficulty remaining is that when a weakref and its referent both
become unreachable simultaneously, relative lifetime becomes fuzzy. 
Even when, in the "simple" examples, I write

    weakref = None
    referent = None # callback doesn't trigger

or

    referent = None # callback does trigger
    weakref = None

the idea that relative lifetime controls the outcome in a
*predictable* way depends on that CPython uses refcounting as its
primary gc strategy.  Indeed, Python the language (as opposed to
CPython the implementation) really can't guarantee that the callback
does or doesn't trigger in either simple example above.

These fuzzy issues have become important because the lack of clarity
about exact intended semantics has left us with segfault-producing
bugs in the interaction between cyclic gc and weakrefs, and we've been
thru more than one round of those now.

A complication is that, perhaps contrary to initial expectation,
outside of contrived tests I'm not sure callbacks that *aren't*
"lambdas or local functions" actually exist.  In the core, the two
flavors of weak dict use very different strategies, but both end up
using a unique function per dict instance as the callback, created in
the weak dict's __init__ and bound to an attribute of the dict.  So
these are guaranteed to create "fuzzy situations" when a weak dict
becomes part of cyclic trash (the dict, and the weakrefs it holds, and
their callbacks, generally all become unreachable at the same
instant).

> I take it the changes you're proposing aren't going to change the
> semantics of the quoted code?

I'm not proposing anything specific, and if it comes to it I bet I
could live with 2.4b2 forever.  Its rules seem ultimately arbitrary,
though, so aren't really attractive.  Other rules are possible, and
should be considered.  For the first time, I think we finally have a
deep understanding of what we can and cannot *get away* with now wrt
preventing segfault bugs (and other related nastiness).  But that
understanding doesn't reduce the universe of acceptable (bug-free and
defensible) policies to what 2.4b2 implements.

In any case, no change to gcmodule.c's weakref policy would change
anything about how CPython behaves in cases where refcounting does the
whole job.  So the answer to your literal <wink> question is "no". 
The behaviors of

> def callback(ignore):
>     print 'hi'
>
> class C:
>     pass
>
> c = C()
> w = weakref.ref(c, callback)
> w = None
> c = None

kinds of examples are wholly determined by refcount lifetime semantics
in CPython, and that won't change.