[Python-Dev] PyObject_GC_UnTrack() no longer reliable in 2.7?

[Antoine Pitrou]

On Fri, 24 Sep 2010 15:14:32 -0400
Tim Peters <tim.peters at gmail.com> wrote:
> Looks like 2.7 changes introduced to exempt dicts and tuples from
> cyclic gc if they obviously can't be in cycles has some unintended
> consequences.  Specifically, if an extension module calls
> PyObject_GC_UnTrack() on a dict it _does not want tracked_, Python can
> start tracking the dict again.
> 
> I assume this is unintended because (a) the docs weren't changed to
> warn about this; and, (b) it's wrong ;-)

It was indeed unintended. I didn't know people were using
PyObject_GC_(Un)Track in other places than constructors and destructors.

> There are two main reasons an extension module may have been calling
> PyObject_GC_UnTrack():
> 
> 1. For correctness, if the extension is playing games with reference
> counts Python isn't expecting.

Yikes :)

> 2. For speed, if the extension is creating dicts (or tuples) it knows
> cannot participate in cycles.

The optimization is now automated in the simple cases (as you've found
out!).

> This came up when Jim Fulton asked me for advice about assertion
> failures coming out of cyclic gc in a debug build when running ZODB's
> tests under 2.7.  Turned out to be due to the "#1 reason" above:  ZODB
> hand-rolled its own version of weak references long before Python had
> them, and has a dict mapping strings ("object IDs") to complex objects
> where the referenced objects' refcounts intentionally do _not_ account
> for the references due to this dict.

Perhaps ZODB should switch to standard weak references these days?
(as a bonus, chances are it will be faster)

> Best if no changes had been needed.  "Better than nothing" if the docs
> are changed to warn that the effect of calling PyObject_GC_UnTrack()
> may be undone by Python a nanosecond later ;-)

A doc addition will be enough, hopefully.

Regards

Antoine.