[Numpy-discussion] Re: [SciPy-user] Current thoughts on future directions

[Fernando Perez]

Michiel Jan Laurens de Hoon wrote:
> Perry Greenfield wrote:

[weave & f2py in the core]

>>That they are perfectly fine to go into the core. In fact, if they are 
>>used by any of the extra packages, they should be in the core to 
>>eliminate the extra step in the installation of those packages.
>>
> 
> -0.
> 1) In der Beschraenkung zeigt sich der Meister. In other words, avoid 
> software bloat.
> 2) f2py is a Fortran-Python interface generator, once the interface is 
> created there is no need for the generator.
> 3) I'm sure f2py is useful, but I doubt that it has very general 
> usefulness. There are lots of other useful Python packages, but we're 
> not including them in scipy-core either.
> 4) f2py and weave don't fit in well with the rest of scipy-core, which 
> is mainly standard numerical algorithms.

I'd like to argue that these two tools are actually critically important in 
the core of a python for scientific computing toolkit, at its most basic 
layer.  The reason is that python's dynamic runtime type checking makes it 
impossible to write efficient loop-based code, as we all know.  And it is not 
always feasible to write all algorithms in terms of Numeric vector operations: 
sometimes you just need to write an indexed loop.

At this point, the standard python answer is 'go write an extension module'. 
While writing extension modules by hand, from scratch, is not all that hard, 
it certainly presents a significant barrier for less experienced programmers. 
  And yet both weave and f2py make it incredibly easy to get working compiled 
array code in no time at all.  I say this from direct experience, having 
pointed colleagues to weave and f2py for this very problem.  After handing 
them some notes I have to get started, they've come back saying "I can't 
believe it was that easy: in a few minutes I had sped up the loop I needed 
with a bit of C, and now I can continue working on the problem I'm interested 
in".  I know for a fact that if I'd told them to write a full extension module 
by hand, the result would have been quite different.

The reality is that, in scientific work, you are likely to run into this 
problem at a very early stage, much more so than for other kinds of python 
usage.  For this reason, it is important that the basic toolset provides a 
clean solution from the start.

At least that's been my experience.

Regards,

f