[Numpy-discussion] R: R: R: R: R: fast numpy.fromfile skipping data chunks

[Chris Barker - NOAA Federal]

On Thu, Mar 14, 2013 at 1:48 AM, Andrea Cimatoribus
<Andrea.Cimatoribus at nioz.nl> wrote:
> Thanks for all the feedback (on the SSD too). For what concerns "biggus" library, for working on larger-than-memory arrays, this is really interesting, but unfortunately I don't have time to test it at the moment, I will try to have a look at it in the future. I hope to see something like that implemented in numpy soon, though.

You may also want to look at carray:

https://github.com/FrancescAlted/carray

I"ve never used it, but it stores the contents of the array in a
compressed from in memory, so if you data compresses well, then it
could be a slick solution.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov