On Thu, Mar 14, 2013 at 1:48 AM, Andrea Cimatoribus <Andrea.Cimatoribus@nioz.nl> wrote:
Thanks for all the feedback (on the SSD too). For what concerns "biggus" library, for working on larger-than-memory arrays, this is really interesting, but unfortunately I don't have time to test it at the moment, I will try to have a look at it in the future. I hope to see something like that implemented in numpy soon, though.
You may also want to look at carray: https://github.com/FrancescAlted/carray I"ve never used it, but it stores the contents of the array in a compressed from in memory, so if you data compresses well, then it could be a slick solution. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov