[pypy-dev] day 6: breakthrough!
hpk at trillke.net
Sun Jul 31 10:05:19 CEST 2005
two and a half years later ... in the very same room where the
first PyPy sprint happened ... we got to the point where PyPy
basically runs on its own, after whole-program type inference and
translation to C!
The sixth day was a very productive day in various respects
and we still had a long and nice lunch break. If you want to
see the visual flow of events then revisit the pictures i took:
Here is a more detailed overview of what happended on the
- Carl, Samuele and holger finished the Heidelberg sprint
announcement, updated the website and the announcement
was sent out. See the home page for more details and links.
- Christian with initial help from Samuele went off to work
on the interpreter level marshal module which we will need
for better bootstrapping. Unfortunately, Christian had
a bad stomach and missed much of the fun of day 6.
- Armin and Holger as well as others continously restarted
the whole translation process to fix small bugs and issues
here and there. In between
- they extended testing to allow applevel tests to easily
access values setup_*'ed from interpreter level.
- they fixed the posix module which was completely broken
with respect to exception handling (you cannot let
escape interpreter-level exception to pypy application
level without carefully wrapping them in an OperationError).
Samuele helped with making os.strerror available
as an external (c-backend supported) function.
- analysed and fixed with Samuele and Carl segmentation faults ...
but note that GCC was behaving very fine and it turned
out that our c-code generator currently consumes more RAM
than GCC compiling the result at the moment :-)
- they also started refactoring the translation driving
- Carl Friedrich and Samuele used and extended/fixed
the ll-interpreter which was sometimes able to
discover possible segfaults and problems earlier
(and clearer) than just running the C-backend.
- Carl Friedrich started working on GC in PyPy
in constant consultation with Samuele and did first working
drafts in pypy/rpython/memory. Basically the whole memory
simulator is working and gives nices errors if you do
something strange like accessing freed or uninitialized.
Furthermore they implemented annotation for memory addresses.
- Richard communicated with Eric van Riet Paap (who has made
good progress on the llvm-backend off-sprint!) and
refactored, extended LLVM a bit driven by test suggestions
- the group discussed that there likely is a deeper exception
handling problem lurking: implicitely exceptions are
attached to the last (high level) operation in a block but
when the RTyper converts this operation into possibly many
low-level operations then it's unlikely that we can preserve
the implicit assumption that only the last (low-level)
operation possibly raises an exception.
solution: unknown at this point.
We should probably write down a few scenarios regarding
exception handling describing the interactions/processing
of exceptions in flowobjspace/annotation/rtyping
- Armin communicated with Niklaus Heidimann who meanwhile
has successfully implemented the _sre module as part of his
SoC project. We'd like to first integrate this as an
application level module but Niklaus wants to transform
it to an interpreter-level one soon.
- After we got the initial breakthrough run (goal/app_example.py
running on top of translated non-faking PyPy) we continued
a bit with various example and fixes. Running example code
and modules showed very good results (and no segfaults).
After having drunk champagne Samuele quickly mapped the good time
we were having into a better 'time' module for PyPy. With that
we could easily run the unmodified pystone.py on top of PyPy
and reached some 337 pystones (as opposed to 9 pystones
with PyPy on top of CPython). We are happy with this
because translated PyPy seems to run stable and we are not using
any optimizations yet. We quickly agreed that we only want to
tackle optimizations at this point that result in at least
factors-of-2 speedups rather than 5-10% speedups. Again please
note that PyPy - as opposed to other python implementations -
generally tackles compliancy, proper implementation and
tests first before diving into optimizations.
ok, so much so far. Today - on the last day - we are probably
going to discuss on how to continue from here and also tell
how you can actually run the translation and the translated PyPy
yourselves (the current pypy-dist is able to but it currently
requires a bit of manual tweaks at the very invocation-end,
maybe you can already figure it out looking at the pictures
and the entry_point code :-).
More information about the Pypy-dev