On Oct 11, 2004, at 2:59 AM, Jason Evans wrote:
Working around this in Crux requires a lot of extra reference counting complexity, because there are three different cases for reference counts, depending on how many members there are in a ring (1, 2, or 3+ members).
Why is that? Just DECREF every PyObject* in "self", then set the PyObject* to NULL. No need for any additional reference counts.
The low level tree code is implemented completely separately from the Python object wrapper code. This means that, for example, the Python "Node" object does not actually store PyObject* pointers to Ring objects; instead it uses the low level tree code to find out what Ring objects are attached to it. Crux was designed this way in order to be able to implement various low level algorithms such that they never have to call interpreter-related code. As such, reference breaking code must actually tear down the low level tree in such a way that it is always "consistent" (simply setting pointers to NULL doesn't fit well with this way of doing things).
You've now reached the point where the Python-GC-ease-of-implementation breaks down. We have encountered the same problem - the GC is fine as long as you're solely extending Python, but if you're embedding it, chances are you'll encounter some interesting issues like this along the way. It's not that you can't make it work - you can, but need to do a lot more work yourself. Basically, if I understand you correctly (and I may not be), there are times when you do not want tp_dealloc to actually destroy the low-level data structure, because somebody else (not living in the Python reference-counted world) isn't done with it yet. The problem is in knowing when those times are, and when they aren't. In this case, you may need to implement basic reference counting in your own low-level data structures, so you know when you're really done with a ring and ready to destroy it. The other option is to attempt to make sure that you're not creating cycles (in python) and avoid interfacing with GC entirely, but that may not be possible in your case. -- Nick