<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<div><br class="webkit-block-placeholder"></div><div>Matthieu, linking in a multi-threading enabled library to numpy, like MKL, will provide speedup to only a limited set of operations and scale only on when you limit yourself to the cores in a single smp-like system. Star-P on the other hand, since it is MPI based, scales on cluster and covers a much larger set of operations that can be performed in parallel plus it provides a simple mechanism to do embarrassingly parallel operations.</div><div><br class="webkit-block-placeholder"></div><div>Ronnie</div><br><div><div>On Jun 29, 2007, at 1:46 PM, Matthieu Brucher wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">Hi,<br><br>Is there a comparison with parallel libraries that can be branched on numpy like MKL ? (and IPP for random number ?)<br><br>Matthieu<br><br><div><span class="gmail_quote">2007/6/29, Ronnie Hoogerwerf <<a href="mailto:rhoogerwerf@interactivesupercomputing.com"> rhoogerwerf@interactivesupercomputing.com</a>>:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div style=""><div style="margin: 0px;"> <font style="font-family: Times New Roman; font-style: normal; font-variant: normal; font-weight: normal; font-size: 16px; line-height: normal; font-size-adjust: none; font-stretch: normal;"><font face="Courier" size="3"> <span style="font-size: 12px;">I am an Application Engineer at Interactive Supercomputing and we are rolling out a beta version of our Star-P product for Python. We are actively looking for computationally intensive Python application to port to Star-P. Star-P is a parallel application development platform that allows users to tap into the power and memory of supercomputers from the comfort of the favorite desktop applications, in this case Python. </span></font></font></div><p style="margin: 0px;"><font style="font-family: Times New Roman; font-style: normal; font-variant: normal; font-weight: normal; font-size: 16px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> <font face="Courier" size="3"><span style="font-size: 12px;"> </span></font></font></p><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> Star-P is capable of both fine-grained parallel computation and embarrassingly parallel computation. The fine-grained mode of our Star-P Python implementation has been modeled on the Python NumPy package - for example:</font> </div><p style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> </font> </p><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;">x = starp.random.rand (20000,20000)</font></div><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> y = starp.linalg.inv(x)</font></div><p style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> </font></p><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> instead of </font></div><p style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> </font></p><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> x = numpy.random.rand(20000,20000)</font></div><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> y = numpy.linalg.inv(x)</font></div><p style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> </font></p><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> Where the first couple of lines are executed on the Star-P parallel server in full C/MPI mode and the last couple of lines are executed on the desktop using Python.</font></div><p style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> </font></p><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> The embarrassingly parallel mode is capable of executing any Python module, although input and output parameters are currently limited to NumPy arrays, scalars, and strings - for example:</font></div><p style="margin: 0px;"> <font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> </font></p><div style="margin: 0px;"> <font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;">y = starp.ppeval(mymodule.dosomething,x) </font></div><p style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> </font></p><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;">instead of </font></div><p style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> </font></p><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;">for i in range(0,n): </font></div><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> y[:,:,i] = mymodule.dosomething(x[:,i])</font></div><p style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> </font></p><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> Where again in the former example the iterations are spread out over the available CPUs (note the abstraction - user need not worry regarding the number of CPUs) on the Star-P server using Python and in the latter the looping is doing in serial on the client using Python. </font></div><p style="margin: 0px;"><font style="font-family: Times New Roman; font-style: normal; font-variant: normal; font-weight: normal; font-size: 16px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> <font face="Courier" size="3"><span style="font-size: 12px;"> </span></font></font></p><div style="margin: 0px;"><font style="font-family: Courier; font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> We are looking for real Python application that you would be willing to share with us that we can port to Star-P. We want to use this experience as a basis for further improvements and development of our Python client. </font> </div><p style="margin: 0px;"><font style="font-family: Times New Roman; font-style: normal; font-variant: normal; font-weight: normal; font-size: 16px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> <font face="Courier" size="3"><span style="font-size: 12px;"> </span></font></font></p><div style="margin: 0px;"><font style="font-family: Times New Roman; font-style: normal; font-variant: normal; font-weight: normal; font-size: 16px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> <font face="Courier" size="3"><span style="font-size: 12px;">Thanks,</span></font></font></div><span class="sg"><div style="margin: 0px;"><font style="font-family: Times New Roman; font-style: normal; font-variant: normal; font-weight: normal; font-size: 16px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> <font face="Courier" size="3"><span style="font-size: 12px;">Ronnie</span></font></font></div><p style="margin: 0px;"><font style="font-family: Times New Roman; font-style: normal; font-variant: normal; font-weight: normal; font-size: 16px; line-height: normal; font-size-adjust: none; font-stretch: normal;"> <font face="Courier" size="3"><span style="font-size: 12px;"> </span></font></font></p> </span></div><br>_______________________________________________<br>Numpy-discussion mailing list<br><a onclick="return top.js.OpenExtLink(window,event,this)" href="mailto:Numpy-discussion@scipy.org">Numpy-discussion@scipy.org </a><br><a onclick="return top.js.OpenExtLink(window,event,this)" href="http://projects.scipy.org/mailman/listinfo/numpy-discussion" target="_blank">http://projects.scipy.org/mailman/listinfo/numpy-discussion</a><br><br></blockquote> </div><br><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">_______________________________________________</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Numpy-discussion mailing list</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><a href="mailto:Numpy-discussion@scipy.org">Numpy-discussion@scipy.org</a></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><a href="http://projects.scipy.org/mailman/listinfo/numpy-discussion">http://projects.scipy.org/mailman/listinfo/numpy-discussion</a></div> </blockquote></div><br></body></html>