[pypy-dev] practical experiences with PyPy's extension compiler

[Martijn Faassen]

Hey,

As someone who suggested to Brian he try something like this with PyPy, 
I just wanted to highlight this. To make sure the PyPy developer's 
brains don't turn off when they hear Zope, that's not important bit, so 
I changed the title. :) Please all read Brian's entry!

Brian Sutherland wrote:
[snip]

 > At EuroPython I investigated re-implementing a class from
 > zope.interface in PyPy's Restricted Python. I compiled it to C
 > and then compared it to the Pure Python implementation and
 > handcrafted C optimization.
 >
 > Not having touched PyPy before, I found it extremely interesting. The
 > report and code I wrote is available here for anyone else interested:
 > http://lentejasverdes.ath.cx/stuff/pypy/europython_2006/README.html

The point of Brian's experiment from PyPy's perspective is that you get 
feedback on PyPy's utility from a developer who is not primarily engaged 
with the development of programming language technology. I think that's 
good for the PyPy project.

I think the following strategy has a lot of potential getting PyPy 
technology to the "masses":

* write your module in RPython.

* use (R)CTypes if you want to bind to existing C libraries.

* promise performance as this module can compile to a CPython C extension.

* promise maintainability and testability as this module should be able 
to run in straight Python (+ CTypes). This is one of the main areas 
where this strategy would give practical benefits over technology like 
Pyrex.

* and actually all these people are making their extension modules ready 
for PyPy too.

I know that this is your strategy, I'm just repeating it to let the echo 
sink in. And feedback on the experience from people like Brian may give 
vital information on how to improve the experience... In this case, 
Brian's feedback (also personally on irc) indicates to me that the above 
points have not yet been reached entirely:

* writing RPython is rather difficult as you need to keep the various 
levels straight. This unfortunately hurts the maintainability point.

* in this case, the promised performance wasn't delivered, unfortunately.

* as far as I understood it, in this case, the code wasn't directly 
testable in CPython after the rewriting to RPython was done. This hurts 
the testability case.

(my apologies of any of this is wrong; Brian would know better of course)

I'm really excited about the potential here, so I hope this can be the 
start of a longer dialog between PyPy developers and application developers.

Now I'm done highlighting this. :)

Regards,

Martijn