[pypy-dev] missing things for making PyPy "production" ready (for some value of production)

[Douglas McNeil]

cfb wrote:

> I thought I would start a new thread for discussing what PyPy needs to
> become "production ready" (whatever that is) and succeed as a Python
> implementation.
> [...]
>  - Speed. The JIT is still not in a state where it really speeds up
>    arbitrary Python code. I expect this to change sooner or later.
>    However, it's not an area were a lot of people can help.

</lurk>

Arbitrary code is less interesting to me than JIT-powered fast numerical 
code.  Moreover, we numerics types have much lower standards of 
"production ready" than the general public, and are willing to turn on 
options with names like --make-dangerous-assumptions-about-code-for-speed 
-do-not-use-this-flag-really-do-not-use-it-i-warned-you.

Currently there is no One Obvious Way to do heavy numerical programming in 
python.  To actually get things done requires a mix of numpy, boost, 
psyco, pyrex, pyinline, SWIG, some of the existing pypy tools -- even 
wrapped shedskin if you're feeling brave.  The toolset is unwieldy.

Yes, it's true that these often suffice -- I've run hundreds of 
semianalytic models over the past week myself using numpy/pygsl -- but I 
can't write my main production codes in python.  And it's frustrating when 
you write a nice piece of code and then bump up against speed limits you 
can't escape without ugly inline hacks I can't expect the people I 
encourage to use python for science to learn.

This is probably the most low-hanging fruit there could be for a (fully 
float-aware) JIT.  The functions tends to be embarrassingly simple, and 
seldom leave the int/float/list domain.  Most numerical code is borderline 
RPython as-is.

>From previous discussions, I suspect I'm not the only lurker-fan who would 
be willing to commit time to working on numericentric graph optimizations 
when that becomes a worthwhile investment.  There's no reason that the 
mostly-fortran bits of python code shouldn't run almost as fast as fortran 
after amortizing the JIT costs.

<relurk>


Doug

--
Queen Mary College, University of London      "Still creating worlds..
Mathematical Sciences, Astronomy Unit          .. but now with an accent!"