[Python-compilers] Standard for C Functions backing Python Callables

Nathaniel Smith njs at pobox.com
Tue Jul 12 10:54:18 EDT 2016

On Mon, Jul 11, 2016 at 12:04 PM, Robert Bradshaw <robertwb at gmail.com> wrote:
> I started a doc for the first discussion bullet point at in
> https://python-compilers-workshop.github.io/ .

Here's the direct link:

I wrote some notes last night:

Use cases:
- {Numba,PyPy,Pyston,Pyjion} calling into {Numba,PyPy,Pyston,Pyjion}
or {Cython,SWIG,CFFI,...}
- scipy.integrate calling into any of the above
- Cython calling into ...?
- type-safe access to Cython capsules

CEP 1000: https://github.com/cython/cython/wiki/enhancements-cep1000
huge focus on speed, because the goal is for

  cdef func(cdef void(*)(void) callback):

to execute quickly and safely, where void(*)(void) is checked at function entry.

OTOH all of the other use cases above don't care about speed that much...

Need some data structure to represent C types
CFFI... doesn't really give us this!
Numba exporting cffi: casts to "void*", because they don't want to
deal with CFFI's pickiness about int != int32_t -- for a "fast call"
mechanism you care about actual runtime ABI compatibility, not
theoretical compatibility across machines

CFFI also doesn't seem to provide any public API for introspecting
types, and to the extent it does I'm not sure it's shared between
CPython and PyPy implementations?
And when Numba calls into CFFI, it has some weird thing where it needs
you to "register" cffi modules (but only "out-of-line" modules):
I have no idea what this is or why it exists

Simplest API:
- Someone (probably CFFI?) should define a standard API for
representing C types, and support basic operations like "are these API
compatible?" and "are these ABI compatible?"
- take this data structure for representing C types, attach it to
callables, and require that people use interned versions for fast
equality checking

Fancier add-ons:
- have a table of multiple type options
- have a callable where you show it a C type and ask if the JIT can
give you one of those
- have a callable where you show it the Python types and ask if the
JIT can give you one of those

Nathaniel J. Smith -- https://vorpus.org

More information about the Python-compilers mailing list