[pypy-dev] Question on the future of RPython

[Paolo Giarrusso]

On Thu, Sep 2, 2010 at 08:27, Douglas McNeil <mcneil at hku.hk> wrote:
>> No other python implementation can convert python programs to executables.
>
> There's shedskin, which is actually very good as these things go:
>
> http://code.google.com/p/shedskin/
>
> Like RPython, you have to write in a small subset of python which can
> be a little frustrating once you've gotten used to pythonic freedom.
> But I've found it very useful for some short numerical codes (putting
> on my OEIS associate editor hat).  And Cython is pretty powerful these
> days.

> ObPyPy: the other day I had cause to run a very short, unoptimized,
> mostly integer-arithmetic code.  With shedskin, it took between ~42s
> (with ints) and ~1m43 (with longs), as compared with only ~3m30 or so
> to run under pypy.  That's only a factor of two (if I'd needed longs).
>  Both could be much improved, and a lower-level version in C would
> beat them both, but I was very impressed by how little difference
> there was.  Major props!

> For numerics it'd be interesting to have a JIT option which didn't
> care about compilation times, and instead of generating assembly
> itself generated assembly-like C which was then delegated to an
> external compiler.

A more interesting road (which is mentioned somewhere in the PyPy
blog) is to use LLVM in place of this "external JIT compiler", so that
you generate "assembly-like LLVM Intermediate Representation". A bit
like UnladenSwallow is doing, with the difference of having a saner
runtime model to start with (say, no reference counting). Once you
start with LLVM, you are free to choose which optimization passes to
run, from very little to -O3 to even more ones.
The other C compilers incur huge startup costs for no good, and don't
usually allow being used as a library, if just for engineering
problems. LLVM is so much cooler anyway, especially now that say
_everybody_ is switching to it.

About the compilation times tradeoff, you can look for "tiered
compilation", which is a general strategy for doing it automatically,
possibly allowing different tunings (say, like java -server, which is
tuned for performance rather than responsiveness). My authoritative
reference is Cliff Click's blog [1], but you probably want to stop
reading it after the introduction, as I did in this case.

[1] http://www.azulsystems.com/blog/cliff-click/2010-07-16-tiered-compilation
-- 
Paolo Giarrusso - Ph.D. Student
http://www.informatik.uni-marburg.de/~pgiarrusso/