<div dir="ltr"><div style="font-family:arial,sans-serif;font-size:13px">===============================================</div><div style="font-family:arial,sans-serif;font-size:13px">DistArray 0.5 release</div><div style="font-family:arial,sans-serif;font-size:13px">
===============================================</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">**Mailing list:** <a href="mailto:distarray@googlegroups.com">distarray@googlegroups.com</a></div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">**Documentation:** <a href="http://distarray.readthedocs.org/" target="_blank">http://distarray.readthedocs.org</a></div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">**License:** Three-clause BSD</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div>
<div style="font-family:arial,sans-serif;font-size:13px">**Python versions:** 2.7, 3.3, and 3.4</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">
**OS support:** \*nix and Mac OS X</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">What is DistArray?</div><div style="font-family:arial,sans-serif;font-size:13px">
------------------</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">DistArray aims to bring the ease-of-use of NumPy to data-parallel</div><div style="font-family:arial,sans-serif;font-size:13px">
high-performance computing. It provides distributed multi-dimensional NumPy</div><div style="font-family:arial,sans-serif;font-size:13px">arrays, distributed ufuncs, and distributed IO capabilities. It can</div><div style="font-family:arial,sans-serif;font-size:13px">
efficiently interoperate with external distributed libraries like Trilinos.</div><div style="font-family:arial,sans-serif;font-size:13px">DistArray works with NumPy and builds on top of it in a flexible and natural</div><div style="font-family:arial,sans-serif;font-size:13px">
way.</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">0.5 Release</div><div style="font-family:arial,sans-serif;font-size:13px">-----------</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Noteworthy improvements in this release include:</div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">* closer alignment with NumPy's API,</div><div style="font-family:arial,sans-serif;font-size:13px">* support for Python 3.4 (existing support for Python 2.7 and 3.3),</div>
<div style="font-family:arial,sans-serif;font-size:13px">* a performance-oriented MPI-only mode for deployment on clusters and</div><div style="font-family:arial,sans-serif;font-size:13px"> supercomputers,</div><div style="font-family:arial,sans-serif;font-size:13px">
* a way to register user-defined functions to be callable locally on worker</div><div style="font-family:arial,sans-serif;font-size:13px"> processes,</div><div style="font-family:arial,sans-serif;font-size:13px">* more consistent naming of sub-packages,</div>
<div style="font-family:arial,sans-serif;font-size:13px">* testing with MPICH2 (already tested against OpenMPI),</div><div style="font-family:arial,sans-serif;font-size:13px">* improved and expanded examples,</div><div style="font-family:arial,sans-serif;font-size:13px">
* installed version testable via ``distarray.test()``, and</div><div style="font-family:arial,sans-serif;font-size:13px">* performance and scaling improvements.</div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">With this release, DistArray ready for real-world testing and deployment. The</div><div style="font-family:arial,sans-serif;font-size:13px">project is still evolving rapidly and we appreciate the continued input from</div>
<div style="font-family:arial,sans-serif;font-size:13px">the larger scientific-Python community.</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">
Existing features</div><div style="font-family:arial,sans-serif;font-size:13px">-----------------</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">
DistArray:</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">* supports NumPy-like slicing, reductions, and ufuncs on distributed</div><div style="font-family:arial,sans-serif;font-size:13px">
multidimensional arrays;</div><div style="font-family:arial,sans-serif;font-size:13px">* has a client-engine process design -- data resides on the worker processes,</div><div style="font-family:arial,sans-serif;font-size:13px">
commands are initiated from master;</div><div style="font-family:arial,sans-serif;font-size:13px">* allows full control over what is executed on the worker processes and</div><div style="font-family:arial,sans-serif;font-size:13px">
integrates transparently with the master process;</div><div style="font-family:arial,sans-serif;font-size:13px">* allows direct communication between workers, bypassing the master process</div><div style="font-family:arial,sans-serif;font-size:13px">
for scalability;</div><div style="font-family:arial,sans-serif;font-size:13px">* integrates with IPython.parallel for interactive creation and exploration of</div><div style="font-family:arial,sans-serif;font-size:13px">
distributed data;</div><div style="font-family:arial,sans-serif;font-size:13px">* supports distributed ufuncs (currently without broadcasting);</div><div style="font-family:arial,sans-serif;font-size:13px">* builds on and leverages MPI via MPI4Py in a transparent and user-friendly</div>
<div style="font-family:arial,sans-serif;font-size:13px"> way;</div><div style="font-family:arial,sans-serif;font-size:13px">* has basic support for unstructured arrays;</div><div style="font-family:arial,sans-serif;font-size:13px">
* supports user-controllable array distributions across workers (block,</div><div style="font-family:arial,sans-serif;font-size:13px"> cyclic, block-cyclic, and unstructured) on a per-axis basis;</div><div style="font-family:arial,sans-serif;font-size:13px">
* has a straightforward API to control how an array is distributed;</div><div style="font-family:arial,sans-serif;font-size:13px">* has basic plotting support for visualization of array distributions;</div><div style="font-family:arial,sans-serif;font-size:13px">
* separates the array’s distribution from the array’s data -- useful for</div><div style="font-family:arial,sans-serif;font-size:13px"> slicing, reductions, redistribution, broadcasting, and other operations;</div><div style="font-family:arial,sans-serif;font-size:13px">
* implements distributed random arrays;</div><div style="font-family:arial,sans-serif;font-size:13px">* supports ``.npy``-like flat-file IO and hdf5 parallel IO (via ``h5py``);</div><div style="font-family:arial,sans-serif;font-size:13px">
leverages MPI-based IO parallelism in an easy-to-use and transparent way;</div><div style="font-family:arial,sans-serif;font-size:13px"> and</div><div style="font-family:arial,sans-serif;font-size:13px">* supports the distributed array protocol [protocol]_, which allows</div>
<div style="font-family:arial,sans-serif;font-size:13px"> independently developed parallel libraries to share distributed arrays</div><div style="font-family:arial,sans-serif;font-size:13px"> without copying, analogous to the PEP-3118 new buffer protocol.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Planned features and roadmap</div><div style="font-family:arial,sans-serif;font-size:13px">----------------------------</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Near-term features and improvements include:</div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">* array re-distribution capabilities;</div><div style="font-family:arial,sans-serif;font-size:13px">* lazy evaluation and deferred computation for latency hiding;</div>
<div style="font-family:arial,sans-serif;font-size:13px">* interoperation with Trilinos [Trilinos]_; and</div><div style="font-family:arial,sans-serif;font-size:13px">* distributed broadcasting support.</div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">The longer-term roadmap includes:</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">
* Integration with other packages [petsc]_ that subscribe to the distributed</div><div style="font-family:arial,sans-serif;font-size:13px"> array protocol [protocol]_;</div><div style="font-family:arial,sans-serif;font-size:13px">
* Distributed fancy indexing;</div><div style="font-family:arial,sans-serif;font-size:13px">* Out-of-core computations;</div><div style="font-family:arial,sans-serif;font-size:13px">* Support for distributed sorting and other non-trivial distributed</div>
<div style="font-family:arial,sans-serif;font-size:13px"> algorithms; and</div><div style="font-family:arial,sans-serif;font-size:13px">* End-user control over communication and temporary array creation, and other</div><div style="font-family:arial,sans-serif;font-size:13px">
performance aspects of distributed computations.</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">History and funding</div><div style="font-family:arial,sans-serif;font-size:13px">
-------------------</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Brian Granger started DistArray as a NASA-funded SBIR project in 2008.</div>
<div style="font-family:arial,sans-serif;font-size:13px">Enthought picked it up as part of a DOE Phase II SBIR [SBIR]_ to provide a</div><div style="font-family:arial,sans-serif;font-size:13px">generally useful distributed array package. It builds on NumPy, MPI, MPI4Py,</div>
<div style="font-family:arial,sans-serif;font-size:13px">IPython, IPython.parallel, and interfaces with the Trilinos suite of</div><div style="font-family:arial,sans-serif;font-size:13px">distributed HPC solvers (via PyTrilinos [Trilinos]_).</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">This material is based upon work supported by the Department of Energy under</div><div style="font-family:arial,sans-serif;font-size:13px">
Award Number DE-SC0007699.</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">This report was prepared as an account of work sponsored by an agency of the</div>
<div style="font-family:arial,sans-serif;font-size:13px">United States Government. Neither the United States Government nor any agency</div><div style="font-family:arial,sans-serif;font-size:13px">thereof, nor any of their employees, makes any warranty, express or implied,</div>
<div style="font-family:arial,sans-serif;font-size:13px">or assumes any legal liability or responsibility for the accuracy,</div><div style="font-family:arial,sans-serif;font-size:13px">completeness, or usefulness of any information, apparatus, product, or process</div>
<div style="font-family:arial,sans-serif;font-size:13px">disclosed, or represents that its use would not infringe privately owned</div><div style="font-family:arial,sans-serif;font-size:13px">rights. Reference herein to any specific commercial product, process, or</div>
<div style="font-family:arial,sans-serif;font-size:13px">service by trade name, trademark, manufacturer, or otherwise does not</div><div style="font-family:arial,sans-serif;font-size:13px">necessarily constitute or imply its endorsement, recommendation, or favoring</div>
<div style="font-family:arial,sans-serif;font-size:13px">by the United States Government or any agency thereof. The views and opinions</div><div style="font-family:arial,sans-serif;font-size:13px">of authors expressed herein do not necessarily state or reflect those of the</div>
<div style="font-family:arial,sans-serif;font-size:13px">United States Government or any agency thereof.</div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px">.. [protocol] <a href="http://distributed-array-protocol.readthedocs.org/en/rel-0.10.0/" target="_blank">http://distributed-array-protocol.readthedocs.org/en/rel-0.10.0/</a></div>
<div style="font-family:arial,sans-serif;font-size:13px">.. [Trilinos] <a href="http://trilinos.org/" target="_blank">http://trilinos.org/</a></div><div style="font-family:arial,sans-serif;font-size:13px">.. [petsc] <a href="http://www.mcs.anl.gov/petsc/" target="_blank">http://www.mcs.anl.gov/petsc/</a></div>
<div style="font-family:arial,sans-serif;font-size:13px">.. [SBIR] <a href="http://www.sbir.gov/sbirsearch/detail/410257" target="_blank">http://www.sbir.gov/sbirsearch/detail/410257</a></div></div>