[Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

[Francesc Alted]

A Friday 16 January 2009, jh at physics.ucf.edu escrigué:
> Hi Francesc,
>
> > Numexpr is a fast numerical expression evaluator for NumPy.  With
> > it, expressions that operate on arrays (like "3*a+4*b") are
> > accelerated and use less memory than doing the same calculation in
> > Python.
>
> Please pardon my ignorance as I know this project has been around for
> a while.  It this looks very exciting, but either it's cumbersome, or
> I'm not understanding exactly what's being fixed.  If you can
> accelerate evaluation, why not just integrate the faster math into
> numpy, rather than having two packages?  Or is this something that is
> only an advantage when the expression is given as a string (and why
> is that the case)?  It would be helpful if you could put the answer
> on your web page and in your standard release blurb in some compact
> form. I guess what I'm really looking for when I read one of those is
> a quick answer to the question "should I look into this?".

Well, there is a link in the project page to the "Overview" section of 
the wiki, but perhaps is a bit hidden.  I've added some blurb as you 
suggested in the main page an another link to the "Overview" wiki page.
Hope that, by reading the new blurb, you can see why it accelerates 
expression evaluation with regard to NumPy.  If not, tell me and will 
try to come with something more comprehensible.

> Right 
> now, I'm not quite sure whether the problem you are solving is merely
> the case of expressions-in-strings, and there is no advantage for
> expressions-in-code, or whether your expressions-in-strings are
> faster than numpy's expressions-in-code. In either case, it would 
> appear this would be a good addition to the numpy core, and it's past
> 1.0, so why keep it separate?  Even if there is value in having a
> non-numpy version, is there not also value in accelerating numpy by
> default?

Having the expression encapsulated in a string has the advantage that 
you exactly know the part of the code that you want to parse and 
accelerate.  Making NumPy to understand parts of the Python code that 
can be accelerated sounds more like a true JIT for Python, and this is 
something that is not trivial at all (although, with the advent of PyPy 
there are appearing some efforts in this direction [1]).

[1] http://www.enthought.com/~ischnell/paper.html

Cheers,

-- 
Francesc Alted