[Numpy-discussion] R: R: R: R: R: fast numpy.fromfile skipping data chunks

Chris Barker - NOAA Federal chris.barker at noaa.gov
Thu Mar 14 11:40:53 EDT 2013

On Thu, Mar 14, 2013 at 1:48 AM, Andrea Cimatoribus
<Andrea.Cimatoribus at nioz.nl> wrote:
> Thanks for all the feedback (on the SSD too). For what concerns "biggus" library, for working on larger-than-memory arrays, this is really interesting, but unfortunately I don't have time to test it at the moment, I will try to have a look at it in the future. I hope to see something like that implemented in numpy soon, though.

You may also want to look at carray:


I"ve never used it, but it stores the contents of the array in a
compressed from in memory, so if you data compresses well, then it
could be a slick solution.



Christopher Barker, Ph.D.

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

More information about the NumPy-Discussion mailing list