[issue16195] Difficult or impossible to figure out how garbage collector and weak references should interact for user-defined extension types

New submission from Jean-Paul Calderone: There appears to be very little, if any, documentation about how to handle the list at tp_weaklistoffset for types supporting being weak referenced, particularly with respect to garbage collection. Who owns the list? Who owns the objects in the list? Should the list be traversed by the extension type's tp_traverse? Should it ever be INCREF'd or DECREF'd? Does tp_dealloc have any responsibilities with respect to it?
From looking at CPython's own source, it appears that tp_traverse should not touch it (Yhg1s confirmed this on #python). Cython, at least, appears to have come to an alternate conclusion though - it generates tp_traverse functions which visit the list (and provoke gc assertions to fail on Python 2.7 in debug mode).
Some documentation about what is correct to do would be excellent, and would make it easier to explain to Cython developers what they're doing wrong (or confirm that they're doing something right, in which case there are some more bugs to file against the CPython stdlib, since they do something different from Cython). ---------- assignee: docs@python components: Documentation messages: 172639 nosy: docs@python, exarkun priority: normal severity: normal status: open title: Difficult or impossible to figure out how garbage collector and weak references should interact for user-defined extension types _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue16195> _______________________________________

Changes by Andrew Svetlov <andrew.svetlov@gmail.com>: ---------- nosy: +asvetlov _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue16195> _______________________________________

Antoine Pitrou added the comment: I think Cython is wrong here. The only thing to do is to call PyObject_ClearWeakRefs() in the deallocator. Everything else is handled by the interpreter. Agreed improving the documentation would be good. ---------- nosy: +pitrou, scoder stage: -> needs patch type: -> behavior versions: +Python 2.7, Python 3.2, Python 3.3, Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue16195> _______________________________________

Changes by Jesús Cea Avión <jcea@jcea.es>: ---------- nosy: +jcea _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue16195> _______________________________________

Changes by Phil Connell <pconnell@gmail.com>: ---------- nosy: +pconnell _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue16195> _______________________________________

Stefan Behnel added the comment: Just as a quick update here: Cython has since then switched to only using PyObject_ClearWeakRefs() and otherwise leaves the handling of the weakref slot to CPython. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue16195> _______________________________________

Changes by Andrew Svetlov <andrew.svetlov@gmail.com>: ---------- nosy: +asvetlov _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue16195> _______________________________________

Antoine Pitrou added the comment: I think Cython is wrong here. The only thing to do is to call PyObject_ClearWeakRefs() in the deallocator. Everything else is handled by the interpreter. Agreed improving the documentation would be good. ---------- nosy: +pitrou, scoder stage: -> needs patch type: -> behavior versions: +Python 2.7, Python 3.2, Python 3.3, Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue16195> _______________________________________

Changes by Jesús Cea Avión <jcea@jcea.es>: ---------- nosy: +jcea _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue16195> _______________________________________

Changes by Phil Connell <pconnell@gmail.com>: ---------- nosy: +pconnell _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue16195> _______________________________________

Stefan Behnel added the comment: Just as a quick update here: Cython has since then switched to only using PyObject_ClearWeakRefs() and otherwise leaves the handling of the weakref slot to CPython. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue16195> _______________________________________
participants (6)
-
Andrew Svetlov
-
Antoine Pitrou
-
Jean-Paul Calderone
-
Jesús Cea Avión
-
Phil Connell
-
Stefan Behnel