[capi-sig] Creating type object dynamically in run-time

[Mateusz Loskot]

Hi,

This is my first post to this list, hello everyone!
I hope this post is not off-topic here.

I'm writing fairly complex structure of Python extensions directly using
Python C API (version 3+ only). I define some type objects with statically,
according to the canonical Noddy example presented in the docs.
Some type objects have to be defined/composed dynamically in run-time.

Possible 'brute force' approach is to 1) generate Python script in textual form,
2) compile it using Py_CompileStringExFlags and 3) import as a module using
PyImport_ExecCodeModule.

However, it seems there is a better way to do it, using C API only and without
intermediate textual representation imported as a separate module.

Now, I've been looking for correct, canonical or recommended way to generate
dynamically type object definitions, add methods, add attributes, etc.

Here is what I found three approaches discussed on the Web:


A)
SO: How to dynamically create a derived type in the Python C-API
http://stackoverflow.com/questions/8066438/how-to-dynamically-create-a-derived-type-in-the-python-c-api

where PyType_Type.tp_alloc is used to allocate type, then tp_* members
are filled
and PyType_Ready is called. I've copied the example here for easier discussion:

PyTypeObject *BrownNoddyType =  (PyTypeObject
*)PyType_Type.tp_alloc(&PyType_Type, 0);
BrownNoddyType->tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE;
BrownNoddyType->tp_name = "noddy.BrownNoddy";
BrownNoddyType->tp_doc = "BrownNoddy objects";
BrownNoddyType->tp_base = &NoddyType;
BrownNoddyType->tp_new = BrownNoddy_new;
PyType_Ready(BrownNoddyType);

This looks easy, but it is not clear to me

a) How to add methods and members, getters and setters?
Shall I simply call PyDescr_NewMethod, and other PyDescr_* functions?
The manual is quite short here:
http://docs.python.org/release/3.2.2/c-api/descriptor.html

b) How shall I perform clean-up of BrownNoddyType?
Is PyModuleDef .m_free function the right way to do it?
Or, I simply set Py_TPFLAGS_HEAPTYPE and the dynamically object types will
be reference-counted and cleaned automatically as discussed here
http://thread.gmane.org/gmane.comp.python.devel/105648


B)
SO: Python and Dynamically Extending C++ Classes
http://stackoverflow.com/questions/7759827/python-and-dynamically-extending-c-classes

This approach is similar to 1), but it uses base type object.
New sub-classed type objects are 1) created dynamically using
PyObject_New 2) and extended with new methods using
PyCFunction_New and PyInstanceMethod_New functions.

Here come questions:

a) Do I need to perform any clean-up? Where?

b) Is it possible/sensible to replace PyCFunction_New and
PyInstanceMethod_New with PyDescr_NewMethod?


C)
Following what's discussed in "How do I add method dynamically to
module using C API?"
http://www.velocityreviews.com/forums/t727676-how-do-i-add-method-dynamically-to-module-using-c-api.html
(Complete C/C++ code example is presented in the last post at the bottom.)
It seems to be similar to the B) approach.

This thread is about adding methods to modules and it uses
undocumented PyCFunction_NewEx. But, I sense this technique should work
for type objects too, once they are created.


The big question is, which of these approaches is recommended for Python 3.x,
or is there any other/better way to create dynamic type objects not
discussed above?

I'd appreciate any suggestions, hints or pointers in the Python docs
which discuss this
(perhaps I've missed it).

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net