[Python-Dev] Ctypes and the stdlib (was Re: LZMA compression support in 3.3)

[Guido van Rossum]

On Tue, Aug 30, 2011 at 9:49 AM, "Martin v. Löwis" <martin at v.loewis.de> wrote:
>>> I can understand how that works when building a CPython extension.
>>> But what about creating Jython/IronPython modules with Cython?
>>> At what point get the header files considered there?
>>
>> I had written a bit about this here:
>>
>> http://thread.gmane.org/gmane.comp.python.devel/126340/focus=126419
>
> I see. So there is potential for error there.

To elaborate, with CPython it looks pretty solid, at least for
functions and constants (does it do structs?). You must manually
declare the name and signature of a function, and Pyrex/Cython emits C
code that includes the header and calls the function with the
appropriate types. If the signature you declare doesn't match what's
in the .h file you'll get a compiler error when the C code is
compiled. If (perhaps on some platforms) the function is really a
macro, the macro in the .h file will be invoked and the right thing
will happen. So far so good.

The problem lies with the PyPy backend -- there it generates ctypes
code, which means that the signature you declare to Cython/Pyrex must
match the *linker* level API, not the C compiler level API. Thus, if
in a system header a certain function is really a macro that invokes
another function with a permuted or augmented argument list, you'd
have to know what that macro does. I also don't see how this would
work for #defined constants: where does Cython/Pyrex get their value?
ctypes doesn't have their values.

So, for PyPy, a solution based on Cython/Pyrex has many of the same
downsides as one based on ctypes where it comes to complying with an
API defined by a .h file.

-- 
--Guido van Rossum (python.org/~guido)