[Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

[Paul Anton Letnes]

On 21. apr. 2012, at 00:16, Drew Frank wrote:

> On Fri, Apr 20, 2012 at 11:45 AM, Chris Barker <chris.barker at noaa.gov> wrote:
>> 
>> On Fri, Apr 20, 2012 at 11:39 AM, Dag Sverre Seljebotn
>> <d.s.seljebotn at astro.uio.no> wrote:
>>> Oh, right. I was thinking "small" as in "fits in L2 cache", not small as
>>> in a few dozen entries.
> 
> Another example of a small array use-case: I've been using numpy for
> my research in multi-target tracking, which involves something like a
> bunch of entangled hidden markov models. I represent target states
> with small 2d arrays (e.g. 2x2, 4x4, ..) and observations with small
> 1d arrays (1 or 2 elements). It may be possible to combine a bunch of
> these small arrays into a single larger array and use indexing to
> extract views, but it is much cleaner and more intuitive to use
> separate, small arrays. It's also convenient to use numpy arrays
> rather than a custom class because I use the linear algebra
> functionality as well as integration with other libraries (e.g.
> matplotlib).
> 
> I also work with approximate probabilistic inference in graphical
> models (belief propagation, etc), which is another area where it can
> be nice to work with many small arrays.
> 
> In any case, I just wanted to chime in with my small bit of evidence
> for people wanting to use numpy for work with small arrays, even if
> they are currently pretty slow. If there were a special version of a
> numpy array that would be faster for cases like this, I would
> definitely make use of it.
> 
> Drew

Although performance hasn't been a killer for me, I've been using numpy arrays (or matrices) for Mueller matrices [0] and Stokes vectors [1]. These describe the polarization of light and are always 4x1 vectors or 4x4 matrices. It would be nice if my code ran in 1 night instead of one week, although this is still tolerable in my case. Again, just an example of how small-vector/matrix performance can be important in certain use cases.

Paul

[0] https://en.wikipedia.org/wiki/Mueller_calculus
[1] https://en.wikipedia.org/wiki/Stokes_vector