[Python-Dev] PyObject_GC_UnTrack() no longer reliable in 2.7?

[Tim Peters]

Looks like 2.7 changes introduced to exempt dicts and tuples from
cyclic gc if they obviously can't be in cycles has some unintended
consequences.  Specifically, if an extension module calls
PyObject_GC_UnTrack() on a dict it _does not want tracked_, Python can
start tracking the dict again.

I assume this is unintended because (a) the docs weren't changed to
warn about this; and, (b) it's wrong ;-)

There are two main reasons an extension module may have been calling
PyObject_GC_UnTrack():

1. For correctness, if the extension is playing games with reference
counts Python isn't expecting.

2. For speed, if the extension is creating dicts (or tuples) it knows
cannot participate in cycles.

This came up when Jim Fulton asked me for advice about assertion
failures coming out of cyclic gc in a debug build when running ZODB's
tests under 2.7.  Turned out to be due to the "#1 reason" above:  ZODB
hand-rolled its own version of weak references long before Python had
them, and has a dict mapping strings ("object IDs") to complex objects
where the referenced objects' refcounts intentionally do _not_ account
for the references due to this dict.

This had been working fine for at least 8 years, thanks to calling
PyObject_GC_UnTrack() on this dict right after it's created.

But under 2.7, new code in Python apparently decides to track this
dict again the first time its __setitem__ is called.  Cyclic gc then
discovers object references due to this dict that aren't accounted for
in the referenced objects' refcounts, and dies early on with an
assertion failure (which it should do - the refcounts are nuts as far
as Python is concerned).

Jim wormed around that for now by calling PyObject_GC_UnTrack() again
every time the dict's content changes, but that was relatively easy in
this case because the dict is an internal implementation detail all
accesses to which are under ZODB's control.

Best if no changes had been needed.  "Better than nothing" if the docs
are changed to warn that the effect of calling PyObject_GC_UnTrack()
may be undone by Python a nanosecond later ;-)