[Numpy-discussion] sparse array data

Travis Oliphant travis at continuum.io
Thu May 3 00:27:03 EDT 2012


On May 2, 2012, at 10:03 PM, Stéfan van der Walt wrote:

> On Wed, May 2, 2012 at 6:25 PM, Travis Oliphant <travis at continuum.io> wrote:
>> The only new principle (which is not strictly new --- but new to NumPy's world-view) is using one (or more) fields of a structured array as "synthetic dimensions" which replace 1 or more of the raw table dimensions.
> 
> Ah, thanks--that's the detail I was missing.  I wonder if the
> contiguity requirement will hamper us here, though.  E.g., I could
> imagine that some tree structure might be more suitable to storing and
> organizing indices, and for large arrays we wouldn't like to make a
> copy for each operation.  I guess we can't wait for discontiguous
> arrays to come along, though :)

Actually, it's better to keep the actual data together as much as possible, I think, and simulate a tree structure with a layer on top --- i.e. an index.    

Different algorithms will prefer different orderings of the underlying data just as today different algorithms prefer different striding patterns on the standard, strided view of a dense array. 

-Travis


> 
>> More to come....  If you are interested in this sort of thing please let me know....
> 
> Definitely--if we can optimize this machinery it will be beneficial to
> scipy.sparse as well.
> 
> Stéfan
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion




More information about the NumPy-Discussion mailing list