[pypy-dev] Cython-CEP: Native dispatch through Python callables
stefan_ml at behnel.de
Fri Apr 13 21:43:17 CEST 2012
wlavrijsen at lbl.gov, 13.04.2012 21:15:
>> At least for Cython code that's not a problem - its functions raise Python
>> exceptions. Some C++ exceptions are mapped automatically and others can be
>> mapped explicitly.
>> Other wrapper generators could also generate an intermediate wrapper
>> function that does the error mapping and passes on Python exceptions.
> ah, that's what I referred to as "a problem" as it still leaves a layer,
> and hence a slowdown. :) Yes, in the "slow" path it works like that. I'm
> just hoping for something more elegant.
It's not necessarily slow because a) the intermediate function can do more
than just passing through data (especially in the case of Cython or Numba)
and b) the exception case is usually just that, an exceptional case.
>> Hmm, but that's RPython, isn't it? I thought that was compiled statically?
>> How would it adapt to a signature that it finds at runtime then?
> It is RPython, but compiled statically does not mean that it can not have
> behaviors at runtime: you specify at runtime the kind of low-level objects
> to expect, then map those objects at the time of the call. Like you say,
> just as with ctypes (which in PyPy has libffi underneath).
Ok, I just took a look at it and it seems like the right thing to use for
this. Then all that's left is an efficient runtime mapping from the
exported signature to a libffi call specification.
> The relevant classes are libffi.Func which receives the annotations from
> a selection of libffi.types, and libffi.ArgChain which receives the values
> just before the call. The return type is handed to Func.call, and then
> only needs boxing to be send back to python.
> In C++, there's for member functions a little bit of gymnastics going on as
> a naked function pointer can only be obtained after binding it to an object
> which again is slow (relatively, anyway). However, if the type is known, a
> single lookup suffices, and then the JIT can guard on that type. The 'this'
> pointer then becomes the first arg in the libffi.ArgChain and all is good.
> For gcc anyway, which has this behavior documented as an extension, so it
> can presumably be relied upon. :)
Ok, then the advantage is that you don't have to know the exact signature
in the calling code because it is documented in the called function. For
example, you could pass arbitrary context arguments through to the function
and the JIT would deal with the actual type mapping automatically.
>>> For cppyy, the current plan is to wrap python functions in generated C++
>>> functions for callbacks.
>> Yes, that would be the other direction.
> I'm more seeing it as only half the work, rather than a different direction.
> At least, as I understand Cython, the generated low level code is an actual
> identifiable function?
It's a direct mapping from the functions in your source code.
> That is not the case for JIT traces, and so where in
> Cython a function pointer can be given back to the C++ code performing the
> callback, I'm not aware of anything similar being available from the JIT.
Hmm, yes, this is pretty trivial in Cython. You declare your function with
the right C signature and that's it.
More information about the pypy-dev