[Cython] CEP1000: Native dispatch through callables
robertwb at gmail.com
Fri Apr 13 01:38:33 CEST 2012
On Thu, Apr 12, 2012 at 3:34 PM, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> On 04/13/2012 12:11 AM, Dag Sverre Seljebotn wrote:
>> Travis Oliphant recently raised the issue on the NumPy list of what
>> mechanisms to use to box native functions produced by his Numba so that
>> SciPy functions can call it, e.g. (I'm making the numba part up):
>> @numba # Compiles function using LLVM
>> def f(x):
>> return 3 * x
>> print scipy.integrate.quad(f, 1, 2) # do many callbacks natively!
>> Obviously, we want something standard, so that Cython functions can also
>> be called in a fast way.
>> This is very similar to CEP 523
>> (http://wiki.cython.org/enhancements/nativecall), but rather than
>> Cython-to-Cython, we want something that both SciPy, NumPy, numba,
>> Cython, f2py, fwrap can implement.
>> Here's my proposal; Travis seems happy to implement something like it
>> for numba and parts of SciPy:
> I'm sorry. HERE is the CEP:
> Since writing that yesterday, I've moved more in the direction of wanting a
> zero-terminated list of overloads instead of providing a count, and have the
> fast protocol jump over the header (since version is available elsewhere),
> and just demand that the structure is sizeof(void*)-aligned in the first
> place rather than the complicated padding.
Great idea to coordinate with the many other projects here. Eventually
this could maybe even be a PEP.
Somewhat related, I'd like to add support for Go-style interfaces.
These would essentially be vtables of pre-fetched function pointers,
and could play very nicely with this interface.
Have you given any thought as to what happens if __call__ is
re-assigned for an object (or subclass of an object) supporting this
interface? Or is this out of scope?
Minor nit: I don't think should_dereference is worth branching on, if
one wants to save the allocation one can still use a variable-sized
type and point to oneself. Yes, that's an extra dereference, but the
memory is already likely close and it greatly simplifies the logic.
But I could be wrong here.
Also, I'm not sure the type registration will scale, especially if
every callable type wanted to get registered. (E.g. currently closures
and generators are new types...) Where to draw the line? (Perhaps
things could get registered lazily on the first __nativecall__ lookup,
as they're likely to be looked up again?)
More information about the cython-devel