[Python-Dev] ob_refcnt access

[Tim Peters]

[Ka-Ping Yee, opines about GC]

Ping, I think you're not getting any responses because this has been beaten
to death on c.l.py over the last month (for the 53rd time, no less <wink>).

A hefty percentage of CPython users *like* the reliably timely destruction
refcounting yields, and some clearly rely on it.

Guido recently (10 June) posted the start of a "add GC on top of RC" scheme,
in a thread with the unlikely name "fork()".  The combination of cycles,
destructors and resurrection is quite difficult to handle in a way both
principled and useful (Java's way is principled but by most accounts
unhelpful to the point of uselessness).

Python experience with the Boehm collector can be found in the FAQ; note
that the Boehm collector deals with finalizers in cycles by letting cycles
with finalizers leak!

> ...
> While we're talking about refcounts and all, i've had the
> argument quite successfully made to me that a reasonably
> written garbage collector can be both (a) simple and (b) more
> efficient than refcounting.

That's a dubious claim.  Sophisticated mark-and-sweep (with or without
compaction) is almost universally acknowledged to beat RC, but simple M&S
has terrible cache behavior (you fill up the address space before reclaiming
anything, then leap all over the address space repeatedly cleaning it up).
Don't discount that, in Python unlike as in most other languages, the simple
loop

    for i in xrange(1000000):
        pass

creates a huge amount of trash at a furious pace.  Under RC it can happily
reuse the same little bit of storage each time around.

> Having spent a good number of work days doing nothing but debugging
> crashes by tracing refcounting bugs,

Yes, we can trade that for tracking down M&S bugs <0.5 wink> -- instead of
INCREF/DECREF macros, you end up with M&S macros marking regions where the
collector must not be run (because you're in a temporarily "inconsistent"
state).  That's under sophisticated M&S, though, but is an absolute
nightmare when you miss a pair (the bugs only show up "sometimes", and not
always the same ways -- depends on when M&S happens to run, and "how
inconsistent" you happen to be at the time).

> ...
> And so on; mix and match.  What are everyone's thoughts on this one?

I think Python probably needs to clean up cycles, but by some variant of
Guido's scheme on top of RC; I very much dislike the property of his scheme
that objects with destructors may be get destroyed without their destructors
getting invoked, but it seems hard to fix.

Alternatives include Java's scheme (which really has nothing going for it
other than that Java does it <0.3 wink>); Scheme's "guardian" scheme (which
would let the user "get at" cyclic trash with destructors, but refuses to do
anything with them on its own); following Boehm by saying that cycles with
destructors are immortal; following goofier historical precedent by e.g.
destroying such objects in reverse order of creation; or maybe just raising
an exception if a trash cycle containing a destructor is found.

All of those seem a comparative pain to implement, with Java's being the
most painful -- and quite possibly the least satisfying!

it's-a-whale-of-a-lot-easier-in-a-self-contained-universe-or-even-an-
    all-c-one-ly y'rs  - tim