On Sun, Aug 16, 2009 at 11:53 PM, "Martin v. Löwis"martin@v.loewis.de wrote:
Thanks for the pointer. I noticed that subtype_dealloc is only called for types that are allocated using type_new(). Does this mean that it is not safe to create types in C using just PyType_Ready() and set Py_TPFLAGS_HEAPTYPE on them? The documentation is not clear on this point.
As Benjamin says, this is getting off-topic - python-dev is not a place to ask for help in your project.
Please let me know where is a more suitable place to discuss the implementation of the cPython as it pertains to C extensions. I wrote to python-dev only because the other lists appeared to be more focused on Python-the-language.
I believe setting flags on a type is inherently unsafe.
Clearly this is not true in general. Take Py_TPFLAGS_BASETYPE, which C types are expected to set if they can be subclassed. Or Py_TPFLAGS_HAVE_GC, which C types set if they participate in cyclic reference collection.
The docs do not distinguish (AFAICS) between flags that C types may set directly and those that they may not. My reading of the docs left me with the impression that a type could set Py_TPFLAGS_HEAPTYPE if it had allocated that type on the heap and wanted it INCREF'd and DECREF'd by instances. I now know that there is much more to this flag than I anticipated (see http://thread.gmane.org/gmane.comp.python.devel/105648), I am just giving you feedback about why the docs led me to this incorrect conclusion.
In any case, I think I will experiment with a different approach, where instead of creating types in C dynamically at runtime, I will create a type whose instances "pretend" to be types (they will create instances when called). Still, I would appreciate knowing where I should direct further questions of this type, which are not questions about how to use Python but rather questions about how to properly implement extensions.
Here is what I would like to do when I create my types dynamically:
- implement tp_alloc and tp_dealloc() to INCREF and DECREF the type.
- not set Py_TPFLAGS_HEAPTYPE.
- set Py_TPFLAGS_HAVE_GC (because instances of my obj can create cycles)
Does this seem safe? I notice that subtype_dealloc() does some funky GC/trashcan stuff. Is it safe for me not to call subtype_dealloc? Can I safely implement my tp_dealloc function like this?
If you bypass documented API, you really need to study the code, understand its motivation, judge whether certain usage is "safe" wrt. to the current implementation, and judge the likelihood of this code not getting changed in future versions.
It was not my intention to bypass the documented API. Py_TPFLAGS_HEAPTYPE is documented here, with no note that the flag should not be set explicitly by C types:
http://docs.python.org/c-api/typeobj.html#Py_TPFLAGS_HEAPTYPE
Also, INCREF'ing and DECREF'ing my type from the tp_new and tp_dealloc functions doesn't seem outside of the documented API.
Thanks, Josh