[Numpy-discussion] numexpr with the new iterator
faltet at pytables.org
Mon Jan 10 12:47:02 EST 2011
A Monday 10 January 2011 17:54:16 Mark Wiebe escrigué:
> > Apparently, you forgot to add the new_iterator_pywrap.h file.
> Oops, that's added now.
Excellent. It works now.
> The aligned case should just be a matter of conditionally removing
> the NPY_ITER_ALIGNED flag in two places.
Wow, the support for unaligned in current `evaluate_iter()` seems pretty
$ python unaligned-simple.py
Numexpr version: 1.5.dev
NumPy version: 2.0.0.dev-ebc963d
Python version: 2.6.1 (r261:67515, Feb 3 2009, 17:34:37)
[GCC 4.3.2 [gcc-4_3-branch revision 141291]]
AMD/Intel CPU? True
VML available? False
Detected cores: 2
NumPy aligned: 0.658 s
NumPy unaligned: 1.597 s
Numexpr aligned: 0.59 s
Numexpr aligned (new iter): 0.59 s
Numexpr unaligned: 0.51 s
Numexpr unaligned (new_iter): 0.528 s
so, the new code is just < 5% slower. I suppose that removing the
NPY_ITER_ALIGNED flag would give us a bit more performance, but that's
great as it is now. How did you do that? Your new_iter branch in NumPy
already deals with unaligned data, right?
> The new code also needs support for the reduce operation. I didn't
> look too closely at the code for that, but a nested iteration
> pattern is probably appropriate. If the inner loop is just allowed
> to be one dimension, it could be done without actually creating the
> inner iterator.
Well, if you can support reduce operations with your patch that would be
extremely good news as I'm afraid that the current reduce code is a bit
broken in Numexpr (at least, I vaguely remember seeing it working badly
in some cases).
More information about the NumPy-Discussion