[Cython] [GSoC] Python backend for Cython using PyPy's FFI
Dag Sverre Seljebotn
d.s.seljebotn at astro.uio.no
Thu Apr 7 18:06:31 CEST 2011
On 04/07/2011 05:01 PM, Romain Guillebert wrote:
> I proposed the Summer of Code project regarding the Python backend for
> As I said in my proposal this would translate Cython code to Python +
> FFI code (I don't know yet if it will use ctypes or something specific
> to PyPy). PyPy's ctypes is now really fast and this will allow people to
> port their Cython code to PyPy.
> For the moment I've been mostly in touch with the PyPy people and they
> seem happy with my proposal.
> Of course I'm available for questions.
Disclaimer: I haven't read the proposal (don't have access yet but will
soon). So perhaps the below is redundant.
This seems similar to Carl Witty's port of Cython to .NET/IronPython. An
important insight from that project is that Cython code does NOT specify
an ABI, only an API which requires a C compiler to make sense. That is;
many wrapped C libraries have plenty of macros, we only require partial
definition of struct, we only require approximate typedef's, and so on.
In the .NET port, the consequence was that rather than the original idea
of generating C# code (with FFI specifications) was dropped, and one
instead went with C++/CLR (which is a proper C++ compiler that really
understands the C side on an API level, in addition to giving access to
the .NET runtime).
There are two ways around this:
a) In addition to Python code, generate C code that can take (the
friendlest) APIs and probe for the ABIs (such as, for instance, getting
the offset of each struct field from the base pointer). Of course, this
must really be rerun for each platform/build of the wrapped library.
Essentially, you'd use Cython to generate C code that, in a target
build, would generate Python code...
b) Create a subset of the Cython language ("RCython" :-)), where you
require explicit ABIs (essentially this means either disallowing "cdef
extern from ...", or creating some new form of it). Most Cython
extensions I know about would not work with this though, so there would
need to be porting in each case. Ideally one should then have a similar
mode for Cython+CPython so that one can debug with CPython as well.
More information about the cython-devel