Behalf Of Fernando Perez Subject: [SciPy-dev] Blitz++, NumArray...
[...]
as some of you know, I'm a big fan of weave.inline() for the ease of development it offers. Essentially C++ becomes a scripting part of python, with only the occasional recompile overhead (but this is transparent to the users).
inline() makes heavy use of the blitz++ library, to represent numpy arrays as blitz arrays and therefore index them as Arr(i,j,k...). This seems to me like a very sound decision, as the blitz library was specifically written to provide simultaneously fortran-like performance with high-level object orientation.
With this in mind, I have a few questions:
- Is there anything in Numarray's design which prevents this same trick from being used? The core of getting a Numpy array into a blitz one is simply a call to:
Unfortunately, I'm neither knowledgeable (yet, anyway) about blitz++ nor am I a C++ expert. On the other hand, with some caveats, I would be very surprised if it isn't nearly as easy to do the same in numarray as in numpy. Most of the C-API is identical so it is possible that very few changes are necessary. On the other hand, we aren't yet at the point of trying to deal with weave so it would be a little while before we had time to tackle that issue (at the moment, we are porting the MA package). After our 1.0 release, getting scipy to work with numarray becomes one of our top priorities. As for caveats, the primary one is that numarray allows some flexibility in array storage that numpy doesn't. In particular it allows data that is byteswapped from the normal processor order, and arrays that are not aligned with the processor's address (both to facilitate accessing memory mapped data and arrays of records). For such arrays it is likely that weave will require a copy so that the array elements may be accessed in the usual way. Since such arrays are not permitted in numpy, this is in no way a narrowing of capabilities. [...]
- If any of this seems worthy of more detailed discussion, I told Eric Jones that I'd be happy to coordinate something along these lines at Scipy'03. If there's time in the schedule I'd be glad to do it officially, otherwise we can just try to keep it in mind for a lunch/dinner meeting.
Sure, but let us know what issues you think need to be worked out in light of what I've already said. This may be a good opportunity to get all those knowledgeable about the various components (Blitz++, weave, and numarray) together to discuss what is needed.
Documenting a 'Best Practices for low-level numerical work in python' would probably be a beneficial thing to do for the community. Right now there are so many options that I'm sure I'm not the only one to feel a bit confused.
That's assuming that we know them all already :-) For our part we have been concentrating on adding all the core numpy functionality and haven't yet been able to address all issue. Over the next few months we will be moving more into the scipy area and will be able to understand more of the wider issues. Perry