[pypy-dev] Questions on the pypy+numpy project

[Stefan Behnel]

Jacob Hallén, 18.10.2011 18:41:
> I'd just like to note that the compelling reason for PyPy to develop numpy
> support is popular demand. We did a survey last spring, in which an
> overwhelming number of people asked for numpy support. This indicates that
> there is a large group of people who will be reap benefits from using PyPy
> plus Numpy, without specific support for scipy packages.

Depends on what the question was. Many people say "NumPy", and when you ask 
back, you find out that they actually meant "SciPy" or at least "NumPy and 
parts x, y and z of its ecosystem that I commonly use, oh, and I forgot 
about abc as well, and ...". NumPy itself is just the most visible pile in 
a fairly vast landscape.


> From my perspective, PyPy based scientific computing makes sense. You get to
> write more of your code in a high level language, saving implementation time.
> If you agree with this, then the most sensible thing is to help make the
> transition as smooth as possible. Making it easy to integrate modules from
> FORTRAN, C++ and whatnot is part of such a task. Exactly what to do should be
> demand driven, and that is why PyPy doesn't have a plan or a timetable for
> these things. Like in all technology shifts, some of the old stuff will be
> easy to bring along, some will be hard and will be done anyway. The rest will
> fall to the wayside.

I don't think anyone here speaks against PyPy integrating with the 
scientific world and all of its existing achievements in terms of code. 
However, *that* is the right direction. It's PyPy that needs to integrate. 
Integration with (C)Python is already there, for tons of tools and in 
manyfold ways. Suggesting that people throw that away, that they restart 
from scratch and maintain a separate set of integration code in parallel, 
just to use yet another Python implementation, is asking for a huge waste 
of time.


> If you believe the researchers are better served writing more code in low
> level languages and dealing with the issues of integrating their low level
> stuff with Python (in order to not to have to modify existing packages), then
> you will probably hope that PyPy is a fad that will die. In the very short
> term you are probably right. In the very short term it will be quicker to wade
> across a river rather than build a bridge.

Sorry to get you wrong, but that smells a bit too much like a "PyPy will 
generate the fastest code on earth, so you won't need anything else" ad, 
which is not supported by any facts I know of. I'm yet to see PyPy compete 
with FFTW, just as an example.

Researchers *are* better served by integrating their own and other people's 
"low-level stuff", than by writing it all over again. They want to do 
research, not programming. And there will always be points where they 
resort to a low-level language, be it because of specific performance 
requirements or because they need to integrate it with something else than 
Python, be it in the form of PyPy or CPython. Remember that C is many times 
more ubiquitous than the entire set of Python implementations taken 
together. That won't change.

Stefan