Hi, On 10/26/07, Timothy Hochberg <tim.hochberg@ieee.org> wrote:

On 10/26/07, Sebastian Haase <haase@msg.ucsf.edu> wrote:

On 10/26/07, David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:

P.S: IMHO, this is one of the main limitation of numpy (or any language using arrays for speed; and this is really difficult to optimize: you need compilation, JIT or similar to solve those efficiently).

This is where the scipy - sandbox numexpr project comes in - if I'm not misaken ....

http://www.scipy.org/SciPyPackages/NumExpr

Description The scipy.sandbox.numexpr package supplies routines for the fast evaluation of array expressions elementwise by using a vector-based virtual machine. It's comparable to scipy.weave.blitz (in Weave), but doesn't require a separate compile step of C or C++ code.

I hope that more noise around this will result in more interest and subsequentially result in more support. I think numexpr might be one of the most powerful ideas in numpy / scipy "recently". Did you know about numexpr - David ?

Sadly, I don't think numexpr will help here; It basically handles the same cases as numpy; only faster because it can avoid a lot of temporaries. I think he's going to need Psyco, Pyrex, Weave or similar.

This problem of efficient recursion turns up often. The only versions we support now are the reduce and accumulate methods for ufuncs. I wonder if we can think of some way of offering support that will speed it up? The problem space is big enough that there is probably no method that will solve all problems, but maybe we can toss around some thoughts. For 2-D sorts of things recursion based on functions on "half squares" preceding the current value could be useful. A reverse recursion would also be useful in evaluating a lot of series. The posted problem is sort of a matrix recursion, though not necessarily linear. Some sort of accumulate method using a vector of function calls would probably do the trick. Hmmm... I think that would not be too hard to do, although the calls to python functions would probably keep the speedup down to a factor of 2-3x if implemented in C. In a sense, I am thinking of accumulate and reduce type methods attached to vectors of functions with more than 2 arguments, iterated multidimensional maps, if you will. Chuck