[Python-Dev] RE: Evil Trashcan and GC interaction

[Tim Peters]

[Neil Schemenauer]
> That's the same fix Andrew and I had in mind.  My concern is that this
> is probably not the only bug of this type.  The trashcan mechanism
> changes the ordering of object deallocation.  What are the chances of
> other bugs like this lurking somewhere?

I'll first expand on what "bug of this type" means:  Any use of any C API
gimmick G is a bug (discovered or not) if (sufficient but not necessary):

1. G can (directly or indirectly) call back into Python code;

and

2A. The code using G leaves a reachable object in an inconsistent
    state; and/or

2B. The code using G assumes (and usually implicitly) that mutable
    portions of a reachable object won't change across its use of G.

I've lost track of how many bugs of this sort we've fixed over the years.
In an odd sense, it's a consequence of the GIL:  a "free threaded"
interpreter would get burned by "this kind of thing" all the time.  The GIL
restricts the possibilities for this kind of damage to specific call sites,
but exactly *which* sites changes across releases.  For a dramatic example,
before cyclic GC was added, PyTuple_New was as harmless as water (too
shallow to drown in, and not moving at lethal speed <wink>).  Now it can
lead to gc and destructor invocation, thread switches, and move list guts
around in memory.  As PyDict_Items learned the hard way, allocating a
2-tuple for a key/value pair can even resize the dict it's looking at.

I'm certain there are more vulnerabilities in the core, but I bet we've
fixed way more than half <wink>.  The SETLOCAL bug is a #2A bug that's been
there forever, yet even know that we know about it, I don't believe it's
been the cause of any mystery-failure I ever heard of.

So rather than asking what the odds are of other bugs like this (my guess is
99+% there's at least one more), I wonder what the odds are that anyone will
write a program that cares.  The odds of someone writing a program like Zope
are too small to measure <wink>.