[Python-Dev] PyPy: sprint and news

[Armin Rigo]

PyPy Sprint announcement & news from the project
================================================

We are coming close to a first experimental release of PyPy,
a more flexible Python implementation written in Python. 
The sprint to make this happen will take place in Amsterdam,
a city know to be reachable by cheap flights :-)

This is 1) the announcement for the sprint;
        2) news about the current state of PyPy; 
        3) some words about a proposal we recently submitted
           to the European Union.


Amsterdam Sprint Details
------------------------

The Sprint will take place from

    the 14th of December to the 21st of December at the

    "Vrije Universiteit in Amsterdam", 14th-21th Dec 2003.

thanks to Etienne Posthumus, who helps us to organize the event.  The
main goal will be a complete C translation of PyPy, probably still using
a hacked Pyrex-variant as an intermediate layer and using CPython's
runtime.  We also plan to work on some fun frontends to PyPy like one
based on pygame or a web browser to visualize interactions between
interpreter and objectspace.  

If you want to participate on the sprint, please subscribe here

    http://codespeak.net/mailman/listinfo/pypy-sprint

and list yourself on this wiki page

    http://codespeak.net/moin/pypy/moin.cgi/AmsterdamSprint

where you will also find more information as the sprint date
approaches.  If you are just interested but don't know if you
come then only subscribe to the mailing list. 


State of the PyPy project 
--------------------------

PyPy works pretty well but still on top of CPython. The double
interpretation penalty makes it - as expected - incredibly slow :-)  In
the Berlin sprint we have thus started to work on the "translation"
part, i.e. how this code should be translated into C.  We can now
translate simple functions to C-like code including some type
annotations.  For convenience, we are reusing a modified subset of Pyrex
to generate the low-level C code.  Thanks to Seo (who joined the project
from south-korea) we also have a lisp-backend to fuel the endless c.l.py
threads about python versus lisp :-)

The goal of the next sprint is to complete this work so that we can
translate the complete PyPy source into a huge Pyrex module, and then a
big CPython extension module.  True, the result is not independent from
CPython, as it runs reusing its runtime environment.  But it's probably
an interesting enough state to make a public release from.  

The translation is done by generating a control flow of functions by
means of abstract interpretation.  IOW, we run the PyPy interpreter with
a custom object space ("flowobjspace") which generates a control flow
graph (including the elementary operations) which is then used to
generate low-level code for backends.  We also have preliminary type
inference on the graphs, which can be used by the Pyrex generator to
emit some C type declarations.  

Writing transformations and analysis of these graphs and displaying them
with GraphViz's 'dot' is great fun!  We certainly have a greater need
than ever for graphical interactive tools to see, understand and debug
all these graph manipulations and run tests of them.  Currently it is a
bit difficult to write a test that checks that a transformed graph
"looks right"! 

What we expect from the Amsterdam sprint is thus:

- a big not-too-slow "cpypy.so" extension module for CPython, where at
  least integer arithmetic is done efficiently

- interactive tools to display and debug and test PyPy, visualizing 
  control flow, call-graphs and state models. 

- improving and rewriting our testing tools to give us more control over
  the testing process, and to allow more interactive testing sessions.


Other interesting News
----------------------

Before mid October, we also had a quite different Sprint.  It was an
approximately 10-day effort towards submitting a proposal to the EU.  If
it is accepted we will have resources to fund some developers working
full- or parttime on the project.  However, our "sprint driven
development" will continue to play the central role for development of
PyPy. 

There are especially two technical sections of the proposal which you 
might find interesting to read:

 "Scientific and technological objectives":
 http://codespeak.net/pypy/index.cgi?doc/funding/B1.0

 "Detailed implementation plan"
 http://codespeak.net/pypy/index.cgi?doc/funding/B6.0

Maybe you want to read the whole proposal for other reasons, too, like
making a EU project of your own or competing with us. Actually,
with our sprints there is usually a lot of room for cooperation :-)
Anyway, here is the PDF-url:

    http://codespeak.net/svn/pypy/trunk/doc/funding/proposal/part_b.pdf

Everybody who thinks that he/she could help on the project is
invited to join! Btw, the latest discussions about our sprint
goals usually take place on the pypy-dev list: 

    http://codespeak.net/mailman/listinfo/pypy-dev

have fun,

    Armin & Holger