[pypy-dev] Questions on the pypy+numpy project

[Armin Rigo]

Hi David,

On Tue, Oct 18, 2011 at 18:29, David Cournapeau <cournape at gmail.com> wrote:
>>> (...) with the possibility to replace it in a PyPy specific way.
>>
>> I think you are disregarding what 8 years of the PyPy project should
>> have made obvious. (...)
>
> Ok. In that case, it is fair to say that you are talking about a full
> reimplementation of the whole scipy ecosystem, at least as much as
> pypy itself is a reimplementation of python  ?

I think the original topic of this discussion is numpy, not scipy.
The answer is that I don't know.  I am sure that people will
reimplement whatever module is needed, or design a generic but slower
way to interface with C a la cpyext, or write a different C API, or
rely on Cython versions of their libraries and have Cython support in
PyPy... or more likely all of these approaches and more.

The point is that right now we are focusing on numpy only, and we want
to make existing pure Python numpy programs run fast --- not just run
horribly slowly ---  both in the case of "standard" numpy programs,
and in the case of programs that do not strictly follow the mold of
"take your algorithm, then shuffle it and rewrite it and possibly
obfuscate it until it is expressed as matrix operations with no
computation left in pure Python".

This is the first step for us right now.  It will take some time
before we have to even consider running scipy programs.  By then I
imagine that either the approach works and delivers good performance
--- and then people (us and others) will have to consider the next
steps to build on top of that --- or it just doesn't (which looks
unlikely given the good preliminary results, which is why we can ask
for support via donations).

We did not draw precise plans for what comes next.  I think the above
would already be a very useful result for some users.  But to me, it
looks like a strong enough pull to motivate some more people to do the
next steps --- Cython, C API, rewrite of some modules, and so on,
including the perfectly fine opinion "in my case pypy is not giving
enough benefits for me to care".  Note that this is roughly the same
issues and same solution spaces as the ones that exist in any domain
with PyPy, not just numpy/scipy.


A bientôt,

Armin.