Hi, 2012/2/18 Stefan Behnel <stefan_ml@behnel.de>
Stefan Behnel, 15.02.2012 12:32:
The current state of the discussion seems to be that PyPy provides ways to talk to C code, but nothing as complete as CPython's C-API in the sense that it allows efficient two-way communication between C code and Python objects. Thus, we need to either improve this or look for alternatives.
In order to get us more focussed on what can be done and what the implications are, so that we may eventually be able to decide what should be done, I started a Wiki page for a PyPy backend CEP (Cython Enhancement Proposal).
The discussion so far makes me rather certain that the most promising short-term solution is to make Cython generate C code that PyPy's cpyext can handle. This should get us a rather broad set of running code somewhat quickly, while requiring the least design-from-scratch type of work in a direction that does not yet allow us to see if it will really make existing code work or not.
On top of the basic cpyext interface, it should then be easy to implement obvious optimisations like native C level calls to Cython wrapped functions from PyPy (and potentially also the other direction) and otherwise avoid boxing/unboxing where unnecessary, e.g. for builtins. After all, it all boils down to native code at some point and I'm sure there are various ways to exploit that.
Also, going this route will help both projects to get to know each other better. I think that's a required basis if we really aim for designing a more high-level interface at some point.
The first steps I see are:
- get Cython's test suite to run on PyPy - analyse the failing tests and decide how to fix them - adapt the Cython generated C code accordingly, special casing for PyPy where required
Here is a "getting started" guide that tells you how testing works in Cython:
http://wiki.cython.org/HackerGuide
Once we have the test suite runnable, we can set up a PyPy instance on our CI server to get feed-back on any advances.
https://sage.math.washington.edu:8091/hudson/
So, any volunteers or otherwise interested parties to help in getting this to work? Anyone in for financial support?
Actually I spent several evenings on this. I made some modifications to pypy, cython and lxml, and now I can compile and install cython, lxml, and they seem to work! For example:: html = etree.Element("html") body = etree.SubElement(html, "body") body.text = "TEXT" br = etree.SubElement(body, "br") br.tail = "TAIL" html.xpath("//text()") Here are the changes I made, some parts are really hacks and should be polished: lxml: http://paste.pocoo.org/show/552903/ cython: http://paste.pocoo.org/show/552904/ pypy changes are already submitted. As expected, the example above is much slower on pypy, about 15x slower than with cpython2.6. And I still get crashes when running the lxml test suite. But the situation looks much better than before, support of all lxml features seems possible. Cheers, -- Amaury Forgeot d'Arc