I applaud you on your vision. I only have one small suggestion: I suggest you put a table of contents at the beginning of your NEP so people may skip to the part that most interests them.<br><br><div class="gmail_quote">On Tue, Dec 21, 2010 at 4:59 PM, John Salvatier <span dir="ltr"><<a href="mailto:jsalvati@u.washington.edu">jsalvati@u.washington.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">That is an amazing christmas present.<br><br><div class="gmail_quote"><div><div></div><div class="h5">On Tue, Dec 21, 2010 at 4:53 PM, Mark Wiebe <span dir="ltr"><<a href="mailto:mwwiebe@gmail.com" target="_blank">mwwiebe@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div></div><div class="h5">
Hello NumPy-ers,<div><br></div><div>After some performance analysis, I've designed and implemented a new iterator designed to speed up ufuncs and allow for easier multi-dimensional iteration. The new code is fairly large, but works quite well already. If some people could read the NEP and give some feedback, that would be great! Here's a link:</div>
<div><br></div><div><a href="https://github.com/m-paradox/numpy/blob/mw_neps/doc/neps/new-iterator-ufunc.rst" target="_blank">https://github.com/m-paradox/numpy/blob/mw_neps/doc/neps/new-iterator-ufunc.rst</a></div>
<div><br></div><div>I would also love it if someone could try building the code and play around with it a bit. The github branch is here:</div><div><br></div><div><a href="https://github.com/m-paradox/numpy/tree/new_iterator" target="_blank">https://github.com/m-paradox/numpy/tree/new_iterator</a></div>
<div><br></div><div>To give a taste of the iterator's functionality, below is an example from the NEP for how to implement a "Lambda UFunc." With just a few lines of code, it's possible to replicate something similar to the numexpr library (numexpr still gets a bigger speedup, though). In the example expression I chose, execution time went from 138ms to 61ms.</div>
<div><div><br>Hopefully this is a good Christmas present for NumPy. :)</div><div><br></div><div>Cheers,</div><div>Mark</div>
</div><div><br></div><div><div>Here is the definition of the ``luf`` function.::</div><div><br></div><div><font face="'courier new', monospace"> def luf(lamdaexpr, *args, **kwargs):</font></div>
<div><font face="'courier new', monospace"> """Lambda UFunc</font></div><div><font face="'courier new', monospace"> </font></div>
<div><font face="'courier new', monospace"> e.g.</font></div><div><font face="'courier new', monospace"> c = luf(lambda i,j:i+j, a, b, order='K',</font></div>
<div><font face="'courier new', monospace"> casting='safe', buffersize=8192)</font></div><div><font face="'courier new', monospace"><br>
</font></div><div><font face="'courier new', monospace"> c = np.empty(...)</font></div><div><font face="'courier new', monospace"> luf(lambda i,j:i+j, a, b, out=c, order='K',</font></div>
<div><font face="'courier new', monospace"> casting='safe', buffersize=8192)</font></div><div><font face="'courier new', monospace"> """</font></div>
<div><font face="'courier new', monospace"><br></font></div><div><font face="'courier new', monospace"> nargs = len(args)</font></div><div><font face="'courier new', monospace"> op = args + (kwargs.get('out',None),)</font></div>
<div><font face="'courier new', monospace"> it = np.newiter(op, ['buffered','no_inner_iteration'],</font></div><div><font face="'courier new', monospace"> [['readonly','nbo_aligned']]*nargs +</font></div>
<div><font face="'courier new', monospace"> [['writeonly','allocate','no_broadcast']],</font></div><div><font face="'courier new', monospace"> order=kwargs.get('order','K'),</font></div>
<div><font face="'courier new', monospace"> casting=kwargs.get('casting','safe'),</font></div><div><font face="'courier new', monospace"> buffersize=kwargs.get('buffersize',0))</font></div>
<div><font face="'courier new', monospace"> while not it.finished:</font></div><div><font face="'courier new', monospace"> it[-1] = lamdaexpr(*it[:-1])</font></div>
<div><font face="'courier new', monospace"> it.iternext()</font></div><div><font face="'courier new', monospace"><br></font></div><div><font face="'courier new', monospace"> return it.operands[-1]</font></div>
<div><br></div><div>Then, by using ``luf`` instead of straight Python expressions, we</div><div>can gain some performance from better cache behavior.::</div><div><br></div><div><font face="'courier new', monospace"> In [2]: a = np.random.random((50,50,50,10))</font></div>
<div><font face="'courier new', monospace"> In [3]: b = np.random.random((50,50,1,10))</font></div><div><font face="'courier new', monospace"> In [4]: c = np.random.random((50,50,50,1))</font></div>
<div><font face="'courier new', monospace"><br></font></div><div><font face="'courier new', monospace"> In [5]: timeit 3*a+b-(a/c)</font></div><div><font face="'courier new', monospace"> 1 loops, best of 3: 138 ms per loop</font></div>
<div><font face="'courier new', monospace"><br></font></div><div><font face="'courier new', monospace"> In [6]: timeit luf(lambda a,b,c:3*a+b-(a/c), a, b, c)</font></div>
<div><font face="'courier new', monospace"> 10 loops, best of 3: 60.9 ms per loop</font></div><div><font face="'courier new', monospace"><br></font></div>
<div><font face="'courier new', monospace"> In [7]: np.all(3*a+b-(a/c) == luf(lambda a,b,c:3*a+b-(a/c), a, b, c))</font></div><div><font face="'courier new', monospace"> Out[7]: True</font></div>
</div><div><br></div>
<br></div></div>_______________________________________________<br>
NumPy-Discussion mailing list<br>
<a href="mailto:NumPy-Discussion@scipy.org" target="_blank">NumPy-Discussion@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">http://mail.scipy.org/mailman/listinfo/numpy-discussion</a><br>
<br></blockquote></div><br>
</blockquote></div><br>