Hi! I have just built Cython but haven’t used it yet.
I want to check I understand what it does whilst examing the sources in the repository.
Please let me know if I have it wrong!
Given a Python file, Cython parses it, and translates it to the equivalent C, which can
then be compiled to binary, thereby bypassing the overhead of interpreting bytecode,
but still executing the same accesses to the CPython run time API that the interpreter would.
So there may be a small speed improvement, or maybe not (because it misses optimisations
the interpreter might be able to spot). The binary will typically be a C extension module
that can be loaded and operate the same way as the original Python.
Now, Cython is an extension of Python which allows some extra stuff, including
type annotations, and other directives related to integration with C. These can be
used to facilitate integration with external C libraries directly, and mapping into
Python, as if the code were written in C, only we’re using a Python like language
representing a subset of C instead of C.
Additionally, the compiler recognises the type annotations, and can reduce or
eliminate run time type checks, improving performance, or even replacing
common constructions in Python which much faster ones that do the same job
“closer to the metal”.
To make this work, the CPython API itself is represented in a set of *.pxd files
found in the repository in Includes/cpython, splitting the logic of the compiler
roughly into two parts: the front and back end. The front end groks Python
and Cython code whilst the back end generates the actual C.
===
Just FYI, I’m the developer of a programming language, Felix, which is C++ code
generator. You can think of it as a meta-programming language for C++ with a
proper type system. Felix binds C/C++ code with statements like:
type PyObject = “PyObject*”;
fun add: PyObject * PyObject -> PyObject = “Py_AddLong($1)”;
and can use the bindings like:
var a : PyObject = ….
var b: PyObject = ...
var sum = add (a,b);
so in some ways its doing the same kind of job as Cython, except it isn’t
specialised to bind to Python, it can bind to anything written in C or C++.
Including the Python API as illustrated.
Right now, I’ve writing a program to translate Include/python/*.pxd files to
Felix bindings, to save typing everything by hand. Its even possible I can teach
my compiler to read *.pxd files directly, which would make Cython a sublanguage
of Felix. Felix is very good at defining DSSLs, Domain Specific Sub Languages,
the grammar of the language is defined in the standard library.
In the process of doing this I may find some issues in Cython which I’ll report.
Hope that helps. As I learn more I may be able to contribute to the project
directly.
One possible future goal is to replace NumPy with something much better.
I also have code written in Python that I might translate to C using Cython.
It would be kind of interesting to use Cython to generate C, and then create
bindings to that C in Felix, so instead of calling the Python C API, we call
the Cython generated API instead, allowing people to write libraries for
Felix in Cython instead of C or Felix. However that’s a more major integration task.
You’d want Cython to generate the Felix bindings, or at least output meta-data
that would allow them to be generated easily .. such as .. a *.pxd file !!
—
John Skaller
skaller(a)internode.on.net
