[Python-ideas] ctypes and extensions
jeremy at jeremysanders.net
Sun Aug 28 11:58:24 CEST 2011
I was following the discussion on python-dev about ctypes not being suitable
for the standard library for linking to C libraries, and extension modules
not suitable for alternative implementations. Ctypes is fragile when it
comes linking to libraries whose API might change giving no kind of error.
Extension modules are too closely linked to cpython and reference counting.
Why not have some sort of intermediate C layer to produce a library which
can be immediatedly loaded by any python implementation?
You would write a short C program which would be linked to the library to be
wrapped. It would automatically expose what it is wrapping to python so no
python definitions are necessary. It would be compile-time linked to the
library so that ABI changes are apparent at compile-time. The interface
would be designed so that no python implementation features leak out. You'd
need some sort of loading code (perhaps based on ctypes initially) on each
Imagine something like this
/* wrap function int mylibstrlen(char*) and expose as strlen */
WRAP_FUNCTION(mylibstrlen, "strlen", WRAP_INT, WRAP_CHARPTR);
/* wrap constant M_PI and expose as PI */
WRAP_CONST(M_PI, "PI", WRAP_DOUBLE);
If these macros exposed symbols with some sort of C++-like mangled name they
could automatically be interpreted by python on loading the module.
Alternatively the macros could generate some sort of table which python
Now you could imagine that it might be possible to write the above code
definition in something like Python or XML and then converted to C to be
linked against the library in question e.g.
lib = wrap.Library("mlib", "mylib.h")
lib.definefunction("mylibstrlen", "strlen", wrap.int, wrap.charptr)
# compile, etc...
I quite like the direct C approach as it is an obvious place to write any
additional code to convert between the C library API and a nicer API usable
by Python, though you could have both. It would also be able to link to C++
symbols without python knowing about C++ name mangling, if the module was
compiled with a C++ compiler.
More information about the Python-ideas