Mateusz Loskot, 10.05.2012 13:47:
> On 10 May 2012 12:28, Stefan Behnel wrote:
>> Mateusz Loskot, 10.05.2012 13:24:
>>> This is my first post to this list, hello everyone!
>>> I hope this post is not off-topic here.
>>> I'm writing fairly complex structure of Python extensions directly using
>>> Python C API (version 3+ only). I define some type objects with statically,
>>> according to the canonical Noddy example presented in the docs.
>>> Some type objects have to be defined/composed dynamically in run-time.
>> Do you really need a type or is a Python class enough? The letter would be
>> much easier to create.
> I'm not sure I understand.
> Aren't terms "class" and "type" names of the same concept, since Python 2.2?
With "type" I meant "extension type", i.e. the one you define using C
structs. The alternative would be a simple Python class as you would get with
class MyClass(object): pass
> What is the noddy_NoddyType in the "Defining New Types" example?
That's an extension type.
> In my system, I'm embedding Python and I add custom Python extension
> too (let's call it 'emb')
> So, users of my embedded Python have access to 'emb' module.
> The 'emb' module defines number of types, some are defined statically,
> as the noddy_NoddyType, so users can instantiate it
> n = emb.Noddy()
> n.bar() # method defined statically in methods table of noddy_NoddyType
> Now, I'd like to add some types which are generated in run-time, way
> before the 'emb' module is appended to inittab and embedded Python is
Why would you want to do that before initialising the Python runtime?
> And, I'd like to enable users to instantiate them in the same way as
> Noddy above:
> d = emb.GeneratedNoddy()
> or allow users to use and access
> d = emb.foo()
> d.bar() # added dynamically in run-time during emb.GeneratedNoddy composition
> <class 'emb.GeneratedNoddy'>
> I hope it makes my intentions clear.
Not clear enough. The question is what your dynamically created types
should do and provide. Would they need to be implemented in C (not just
their methods but the types themselves!) or would a Python implementation
suffice, potentially with methods implemented in C?
You may get away with using normal Python classes that inherit from an
extension type, for example. In that case, I'd also advise you to take a
look at Cython, because that makes both the implementation of extension
types and the creation of Python classes inheriting from them as easy as Py.
Mateusz Loskot, 10.05.2012 13:24:
> This is my first post to this list, hello everyone!
> I hope this post is not off-topic here.
> I'm writing fairly complex structure of Python extensions directly using
> Python C API (version 3+ only). I define some type objects with statically,
> according to the canonical Noddy example presented in the docs.
> Some type objects have to be defined/composed dynamically in run-time.
Do you really need a type or is a Python class enough? The letter would be
much easier to create.
This is my first post to this list, hello everyone!
I hope this post is not off-topic here.
I'm writing fairly complex structure of Python extensions directly using
Python C API (version 3+ only). I define some type objects with statically,
according to the canonical Noddy example presented in the docs.
Some type objects have to be defined/composed dynamically in run-time.
Possible 'brute force' approach is to 1) generate Python script in textual form,
2) compile it using Py_CompileStringExFlags and 3) import as a module using
However, it seems there is a better way to do it, using C API only and without
intermediate textual representation imported as a separate module.
Now, I've been looking for correct, canonical or recommended way to generate
dynamically type object definitions, add methods, add attributes, etc.
Here is what I found three approaches discussed on the Web:
SO: How to dynamically create a derived type in the Python C-API
where PyType_Type.tp_alloc is used to allocate type, then tp_* members
and PyType_Ready is called. I've copied the example here for easier discussion:
PyTypeObject *BrownNoddyType = (PyTypeObject
BrownNoddyType->tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE;
BrownNoddyType->tp_name = "noddy.BrownNoddy";
BrownNoddyType->tp_doc = "BrownNoddy objects";
BrownNoddyType->tp_base = &NoddyType;
BrownNoddyType->tp_new = BrownNoddy_new;
This looks easy, but it is not clear to me
a) How to add methods and members, getters and setters?
Shall I simply call PyDescr_NewMethod, and other PyDescr_* functions?
The manual is quite short here:
b) How shall I perform clean-up of BrownNoddyType?
Is PyModuleDef .m_free function the right way to do it?
Or, I simply set Py_TPFLAGS_HEAPTYPE and the dynamically object types will
be reference-counted and cleaned automatically as discussed here
SO: Python and Dynamically Extending C++ Classes
This approach is similar to 1), but it uses base type object.
New sub-classed type objects are 1) created dynamically using
PyObject_New 2) and extended with new methods using
PyCFunction_New and PyInstanceMethod_New functions.
Here come questions:
a) Do I need to perform any clean-up? Where?
b) Is it possible/sensible to replace PyCFunction_New and
PyInstanceMethod_New with PyDescr_NewMethod?
Following what's discussed in "How do I add method dynamically to
module using C API?"
(Complete C/C++ code example is presented in the last post at the bottom.)
It seems to be similar to the B) approach.
This thread is about adding methods to modules and it uses
undocumented PyCFunction_NewEx. But, I sense this technique should work
for type objects too, once they are created.
The big question is, which of these approaches is recommended for Python 3.x,
or is there any other/better way to create dynamic type objects not
I'd appreciate any suggestions, hints or pointers in the Python docs
which discuss this
(perhaps I've missed it).
Mateusz Loskot, http://mateusz.loskot.net