[Cython] CEP1000: Native dispatch through callables
Dag Sverre Seljebotn
d.s.seljebotn at astro.uio.no
Thu Apr 19 15:18:40 CEST 2012
On 04/19/2012 01:20 PM, Nathaniel Smith wrote:
> On Thu, Apr 19, 2012 at 11:56 AM, Dag Sverre Seljebotn
> <d.s.seljebotn at astro.uio.no> wrote:
>> I thought of some drawbacks of getfuncptr:
>> - Important: Doesn't allow you to actually inspect the supported
>> signatures, which is needed (or at least convenient) if you want to use an
>> FFI library or do some JIT-ing. So an iteration mechanism is still needed in
>> addition, meaning the number of things for the object to implement grows a
>> bit large. Default implementations help -- OTOH there really wasn't a major
>> drawback with the table approach as long as JIT's can just replace it?
> But this is orthogonal to the table vs. getfuncptr discussion. We're
> assuming that the table might be extended at runtime, which means you
> can't use it to determine which signatures are supported. So we need
> some sort of extra interface for the caller and callee to negotiate a
> type anyway. (I'm intentionally agnostic about whether it makes more
> sense for the caller or the callee to be doing the iterating... in
> general type negotiation could be quite complicated, and I don't think
> we know enough to get that interface right yet.)
Hmm. Right. Let's define an explicit goal for the CEP then.
What I care about at is getting the spec right enough such that, e.g.,
NumPy and SciPy, and other (mostly manually written) C extensions with
slow development pace, can be forward-compatible with whatever crazy
things Cython or Numba does.
There's 4 cases:
1) JIT calls JIT (ruled out straight away)
2) JIT calls static: Say that Numba wants to optimize calls to np.sin
etc. without special-casing; this seem to require reading a table of
3) Static calls JIT: This is the case when scipy.integrate routines
calls a Numba callback and Numba generates a specialization for the
dtype they explicitly needs. This calls for getfuncptr (but perhaps in a
form which we can't quite determine yet?).
4) Static calls static: Either table or getfuncptr works.
My gut feeling is go for 2) and 4) in this round => table.
The fact that the table can be extended at runtime is then not really
relevant -- perhaps there will be an API to trigger that in the future,
but it can't really be made use of today.
> The other other option would be to go to the far other end of
> simplicity, and just forget for now about allowing multiple signatures
> in the same object. Do signature selection by having the user select
> one explicitly:
> def square(x):
> return x * x
> # .specialize is an un-standardized Cython interface
> # square_double is an object implementing the standardized C-callable interface
> square_double = square.specialize("d->d")
> That'd be enough to get started, and doesn't rule out later extensions
> that do automatic type selection, once we have more experience.
Well, I want np.sin to replace "cdef extern from 'math.h'", and then
this seems to be needed... at least the possibility to have both "d->d"
More information about the cython-devel