[Python-compilers] Standard for C Functions backing Python Callables

[Nathaniel Smith]

On Tue, Jul 12, 2016 at 9:54 AM, Nathaniel Smith <njs at pobox.com> wrote:
> On Mon, Jul 11, 2016 at 12:04 PM, Robert Bradshaw <robertwb at gmail.com> wrote:
>> I started a doc for the first discussion bullet point at in
>> https://python-compilers-workshop.github.io/ .
>
> Here's the direct link:
>   https://docs.google.com/document/d/1NQJTnZCGmB9Hc7KT6HRGsW_GzM18aP4SSpJZjFAo8wY/edit
>
> I wrote some notes last night:
>
> Use cases:
> - {Numba,PyPy,Pyston,Pyjion} calling into {Numba,PyPy,Pyston,Pyjion}
> or {Cython,SWIG,CFFI,...}
> - scipy.integrate calling into any of the above
> - Cython calling into ...?
> - type-safe access to Cython capsules
>
> CEP 1000: https://github.com/cython/cython/wiki/enhancements-cep1000
> huge focus on speed, because the goal is for
>
>   cdef func(cdef void(*)(void) callback):
>       callback()
>
> to execute quickly and safely, where void(*)(void) is checked at function entry.
>
> OTOH all of the other use cases above don't care about speed that much...
>
> Need some data structure to represent C types
> CFFI... doesn't really give us this!

Ah-hah, it turns out that I am wrong!

The magic is: ffi.typeof(function_object)

So I think the main things we might be missing are:
- a fast way to get at this type object pointer
- a way to do "abi level" comparison and interning of these objects

> Numba exporting cffi: casts to "void*", because they don't want to
> deal with CFFI's pickiness about int != int32_t -- for a "fast call"
> mechanism you care about actual runtime ABI compatibility, not
> theoretical compatibility across machines
>
> CFFI also doesn't seem to provide any public API for introspecting
> types, and to the extent it does I'm not sure it's shared between
> CPython and PyPy implementations?
> And when Numba calls into CFFI, it has some weird thing where it needs
> you to "register" cffi modules (but only "out-of-line" modules):
> http://numba.pydata.org/numba-doc/dev/reference/pysupported.html#cffi
> I have no idea what this is or why it exists
>
> Simplest API:
> - Someone (probably CFFI?) should define a standard API for
> representing C types, and support basic operations like "are these API
> compatible?" and "are these ABI compatible?"
> - take this data structure for representing C types, attach it to
> callables, and require that people use interned versions for fast
> equality checking
>
> Fancier add-ons:
> - have a table of multiple type options
> - have a callable where you show it a C type and ask if the JIT can
> give you one of those
> - have a callable where you show it the Python types and ask if the
> JIT can give you one of those
>
>
> --
> Nathaniel J. Smith -- https://vorpus.org



-- 
Nathaniel J. Smith -- https://vorpus.org