[pypy-dev] Questions on the pypy+numpy project

[Jacob Hallén]

Monday 17 October 2011 you wrote:
> On 17 October 2011 16:42, Ian Ozsvald <ian at ianozsvald.com> wrote:
> > > For pypy I can't see any better approach than the way they have taken.
> > 
> > Once
> > 
> > > people are using numpy on pypy the limitations and missing parts will
> > 
> > become
> > 
> > > clear, and not only will the way forward be more obvious but there will
> > 
> > be
> > 
> > > more people involved to do the work.
> > 
> > Michael - I agree that the PyPy community shouldn't do all the
> > legwork! I agree also that the proposed path may spur more work (and
> > maybe that's the best goal for now).
> > 
> > I've gone back to the donations page:
> > http://pypy.org/numpydonate.html
> > to re-read the spec. What I get now (but didn't get before the
> > discussion at Enthought Cambridge) is that "we don't plan to implement
> > NumPy's C API" is a big deal (and not taking it on is  entirely
> > reasonable for this project!).
> > 
> > In my mind (and maybe in the mind of some others who use scipy?) a
> > base pypy+numpy project would easily open the door to matplotlib and
> > all the other scipy goodies, it looks now like that isn't the case.
> > Hence my questions to try to understand what else might be involved.
> 
> Well, I think it definitely "opens the door" - certainly a lot more than
> not doing the work! You have to start somewhere.
> 
> It seems like other projects (like the pypy cython backend) will help make
> other parts of project easier down the line.
> 
> Back to Alex's question, how else would you *suggest* starting? Isn't a
> core port of the central parts the obvious way to begin?
> 
> Given the architecture of numpy it does seem that it opens up a whole bunch
> of questions around numpy on multiple implementations. Certainly pypy
> should be involved in the discussion here, but I don't think it is up to
> pypy to find (or implement) the answers...

I'd just like to note that the compelling reason for PyPy to develop numpy 
support is popular demand. We did a survey last spring, in which an 
overwhelming number of people asked for numpy support. This indicates that 
there is a large group of people who will be reap benefits from using PyPy 
plus Numpy, without specific support for scipy packages. Some of them may want 
to port their favourite scipy packages to work with PyPy.

If the PyPy community decided that it was more important to keep the integrity 
of the Numpy community, we would hold these people back in order to prevent a 
fragmentation. I think we have to accept that some people have needs that are 
better served with what PyPy can provide today, while others will have to wait 
for theirs to be dealt with. This is the natural succession of technologies.

From my perspective, PyPy based scientific computing makes sense. You get to 
write more of your code in a high level language, saving implementation time. 
If you agree with this, then the most sensible thing is to help make the 
transition as smooth as possible. Making it easy to integrate modules from 
FORTRAN, C++ and whatnot is part of such a task. Exactly what to do should be 
demand driven, and that is why PyPy doesn't have a plan or a timetable for 
these things. Like in all technology shifts, some of the old stuff will be 
easy to bring along, some will be hard and will be done anyway. The rest will 
fall to the wayside.

If you believe the researchers are better served writing more code in low 
level languages and dealing with the issues of integrating their low level 
stuff with Python (in order to not to have to modify existing packages), then 
you will probably hope that PyPy is a fad that will die. In the very short 
term you are probably right. In the very short term it will be quicker to wade 
across a river rather than build a bridge.

Jacob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20111018/a5923166/attachment-0001.pgp>