[Cython] [Python-Dev] C-level duck typing

Stefan Behnel stefan_ml at behnel.de
Wed May 16 21:15:58 CEST 2012

"Martin v. Löwis", 16.05.2012 20:33:
>> Does this use case make sense to everyone?
>> The reason why we are discussing this on python-dev is that we are looking
>> for a general way to expose these C level signatures within the Python
>> ecosystem. And Dag's idea was to expose them as part of the type object,
>> basically as an addition to the current Python level tp_call() slot.
> The use case makes sense, yet there is also a long-standing solution
> already to expose APIs and function pointers: the capsule objects.
> If you want to avoid dictionary lookups on the server side, implement
> tp_getattro, comparing addresses of interned strings.

I think Martin has a point there. Why not just use a custom attribute on
callables that hold a PyCapsule? Whenever we see inside of a Cython
implemented function that an object variable that was retrieved from the
outside, either as a function argument or as the result of a function call,
is being called, we try to unpack a C function pointer from it on all
assignments to the variable. If that works, we can scan for a suitable
signature (either right away or lazily on first access) and cache that. On
each subsequent call through that variable, the cached C function will be used.

That means we'd replace Python variables that are being called by multiple
local variables, one that holds the object and one for each C function with
a different signature that it is being called with. We set the C function
variables to NULL when the Python function variable is being assigned to.
When the C function variable is NULL on call, we scan for a matching
signature and assign it to the variable.  When no matching signature can be
found, we set it to (void*)-1.

Additionally, we allow explicit user casts of Python objects to C function
types, which would then try to unpack the C function, raising a TypeError
on mismatch.

Assignments to callable variables can be expected to occur much less
frequently than calls to them, so this will give us a good trade-off in
most cases. I don't see why this kind of caching would be any slower inside
of loops than what we were discussing so far.


More information about the cython-devel mailing list