[capi-sig] Creating type object dynamically in run-time

Stefan Behnel python_capi at behnel.de
Fri May 11 11:52:35 CEST 2012

Mateusz Loskot, 11.05.2012 11:38:
> On 10 May 2012 18:58, Stefan Behnel wrote:
>> Mateusz Loskot, 10.05.2012 18:58:
>>> Long story short, I have a software system implemented in C/C++ which
>>> provides public API. The API is not fixed, may change, so scripting
>>> engines use provided means of reflection to inspect this API and
>>> generate bindings on-fly.
>>> I embedded Python in this system and now I'm using the mentioned
>>> reflection mechanisms to discover API and generate Python extension
>>> modules and types dynamically during run-time.
>>> I simply use the reflection mechanisms provided to iterate over
>>> definitions (structures, functions, constants, etc.) of the public API
>>> and I compose Python extension types grouping methods, etc.
>> Interesting. So you're reflecting on a C/C++ API to dynamically export it
>> to Python space?
> Yes, roughly, that's the idea.
>> Then I guess that your internal binding code is also
>> rather generic, right?
> Let's describe it in simplified way:
> There are intermediate Python extensions defined which bridge
> communication between the C++ world and dynamically generated Python classes.
> For instance, there is a car and car engine.
> Now, there is 'base car engine' which defines common protocol for
> possible behaviours.
> The 'base car engine" is specified and known at compile time,
> Next, there is variety of kinds of engines where each provide
> subset of the well-specified behaviours. Polymorphism, indeed.
> So, those specific engines are dynamically generated in run-time
> and they communicate with car through the statically defined 'base car
> engine' protocol.

Hmm, that sounds more like you'd want to use a common compile time
interface for them statically instead, but I guess I'm just underestimating
the requirements here.

>> It won't be able to call the application code
>> directly, for example, if it doesn't know the API at compile time.
> Yes, so there are 'base' extension types defined statically.

Makes sense.

>>> I can not use Cython, nor I can use tools like SWIG, Boost.Python.
>>> The (C)Python is the only tool I can use.
>> Also interesting. I wonder who would notice with which tool you have
>> written your source code once it's compiled...
> I guess nobody. It's just I had not known Cython before I started.
> Now, I have most of my implementation ready.

Sure, fair enough. (Although you wouldn't be the first one who ends up
rewriting C code in Cython.)

> I also wouldn't want to introduce new dependencies
> and as long as Cython is not guaranteed to be 100% compatible
> drop-in replacement for Python 3.2+, then I'm afraid I can't use it.
> The FAQ does not confirm it is
> http://wiki.cython.org/FAQ#IsCythonaPythonimplementation.3F

Ah, that's the misunderstanding then. It's not meant as a replacement of
CPython, only as a replacement for writing C/C++ code using CPython's
C-API. It generates C code that runs on top of CPython and thus integrates
natively with both C/C++ code and Python code. So, basically, you get the
best of both worlds (C and Python) without having to write the C part of it.


More information about the capi-sig mailing list