[SciPy-user] Low-level code integration discussion at scipy'03?

[Fernando Perez]

Hi everyone,

as I mentioned before, I think it would be great to have a bit of an open 
discussion on methods and techniques for plugging low-level code into python 
at Scipy'03.  I don't mean to teach people how to write an extension module, 
but rather what the best/easiest approaches are for writing time-critical code 
which manipulates Numpy/numarray arrays.

A brief summary of the alternatives:

* Scipy's weave relies on blitz, which is extremely nice and maps surprisingly 
well to python arrays.  Blitz arrays have slices, ranges, overloaded 
arithmetic operators and math functions, etc.  In fact, writing blitz C++ 
feels almost like writing python code, with the caveats of handling 
compile-time type information.

But blitz development seems to be rather slow these days, and the issue of 
access to Blas/lapack from blitz isn't fully settled (i.e., it's not 
out-of-the box easy).

* The ublas library 
(http://www.boost.org/libs/numeric/ublas/doc/overview.htm).  It has direct 
blas support, but it seems to focus strictly on vectors and matrices, and as 
far as I see it doesn't support higher rank objects (I could be wrong, though: 
  I've been using blitz for 'real' and only looked at the ublas webpage).

* Writing low-level code in fortran and using Pearu's f2py.  I love f2py, but 
I'd prefer my low-level code to have the ability to also use more information 
from the python side of things if needed.  I think f2py is a great tool for 
wrapping existing fortran codes, but not so much for extending a python 
program with a few low-level routines.   And there is always the potential 
need for transposition operations when going to Fortran because of the 
row/column memory ordering discrepancy between C and Fortran.

* Complete roll-your-own solutions using the Numarray C api.  This is doable, 
but after using blitz arrays, it feels really low-level.  Call me lazy, but 
after using python I've become fully enamored of writing code which reads as 
cleanly as possible.


What I'd like to know is if there is any interest in organizing a bit of a 
discussion on this.  If an hour can be squeezed in for this in the schedule, 
great (and I volunteer to moderate if nobody else wants to).  If not, perhaps 
some of people involved with these topics (Eric, Travis, Konrad, Todd and 
Perry at least, plus whoever else is interested) might be willing to commit to 
a lunch/dinner to be spent talking about this.

It would be nice to know something in advance, because one of blitz's current 
developers (Julian Cummings) is at Caltech.  Perhaps we could ask him to come 
in and answer some questions about Blitz's future.  Blitz feels like the 
lowest-impedance fit to python's semantics, but I'd like to know more about 
its long-term feasibility before committing to use it too much.  If we set any 
kind of schedule for this discussion, I can try to contact Julian.

Regards,

Fernando.