[Cython] CEP1000: Native dispatch through callables
Dag Sverre Seljebotn
d.s.seljebotn at astro.uio.no
Fri Apr 13 11:13:15 CEST 2012
On 04/13/2012 07:24 AM, Stefan Behnel wrote:
> Dag Sverre Seljebotn, 13.04.2012 00:34:
>> On 04/13/2012 12:11 AM, Dag Sverre Seljebotn wrote:
>>> Travis Oliphant recently raised the issue on the NumPy list of what
>>> mechanisms to use to box native functions produced by his Numba so that
>>> SciPy functions can call it, e.g. (I'm making the numba part up):
>>> @numba # Compiles function using LLVM
>>> def f(x):
>>> return 3 * x
>>> print scipy.integrate.quad(f, 1, 2) # do many callbacks natively!
>>> Obviously, we want something standard, so that Cython functions can also
>>> be called in a fast way.
>>> This is very similar to CEP 523
>>> (http://wiki.cython.org/enhancements/nativecall), but rather than
>>> Cython-to-Cython, we want something that both SciPy, NumPy, numba,
>>> Cython, f2py, fwrap can implement.
>>> Here's my proposal; Travis seems happy to implement something like it
>>> for numba and parts of SciPy:
>> I'm sorry. HERE is the CEP:
> Some general remarks:
> I'm all for doing something in this direction and have been hinting at it
> on the PyPy mailing list for a while, without reaction so far. I'll trigger
> them again, with a pointer to this discussion and the CEP. PyPy should be
> totally interested in a generic way to do fast calls into wrapped C code in
> general and Cython implemented functions specifically. Their JIT would then
> look at the function at runtime and unwrap it.
> There's PEP 362 which proposes a Signature object. It seems to have
> attracted some interest lately and Guido seems to like it also. I think we
> should come up with a way to add a C level interface to that, instead of
> designing something entirely separate.
Well, provided that you still want an efficient representation that can
be strcmp-ed in dispatch codes, this seems to boil down to using a
Signature object rather than a capsule (with a C interface), and store
it in __signature__ rather than __fastcall__, and perhaps provide a slot
in the type object for a function returning it.
I really think the right approach is to prove the concept outside of the
standardization process first; a) by the time a PEP would be accepted it
will have been years since Travis had time to work on this, b) as far as
the slot in the type object goes, we're left with users on Python 2.4
today; a Python 3.4+ solution is not really a solution.
More information about the cython-devel