[Numpy-discussion] numexpr with the new iterator

[Francesc Alted]

A Monday 10 January 2011 17:54:16 Mark Wiebe escrigué: 
> > Apparently, you forgot to add the new_iterator_pywrap.h file.
> 
> Oops,  that's added now.

Excellent.  It works now.

> The aligned case should just be a matter of conditionally removing
> the NPY_ITER_ALIGNED flag in two places.

Wow, the support for unaligned in current `evaluate_iter()` seems pretty 
nice already:

$ python unaligned-simple.py 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Numexpr version:   1.5.dev
NumPy version:     2.0.0.dev-ebc963d
Python version:    2.6.1 (r261:67515, Feb  3 2009, 17:34:37) 
[GCC 4.3.2 [gcc-4_3-branch revision 141291]]
Platform:          linux2-x86_64
AMD/Intel CPU?     True
VML available?     False
Detected cores:    2
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
NumPy aligned:          0.658 s
NumPy unaligned:        1.597 s
Numexpr aligned:        0.59 s
Numexpr aligned (new iter):     0.59 s
Numexpr unaligned:      0.51 s
Numexpr unaligned (new_iter):   0.528 s

so, the new code is just < 5% slower.  I suppose that removing the 
NPY_ITER_ALIGNED flag would give us a bit more performance, but that's 
great as it is now.  How did you do that?  Your new_iter branch in NumPy 
already deals with unaligned data, right?

> The new code also needs support for the reduce operation.  I didn't
> look too closely at the code for that, but a nested iteration
> pattern is probably appropriate.  If the inner loop is just allowed
> to be one dimension, it could be done without actually creating the
> inner iterator.

Well, if you can support reduce operations with your patch that would be 
extremely good news as I'm afraid that the current reduce code is a bit 
broken in Numexpr (at least, I vaguely remember seeing it working badly 
in some cases).

-- 
Francesc Alted