[Numpy-discussion] fast numpy i/o

[Christopher Barker]

Neal Becker wrote:
> I'm wondering what are good choices for fast numpy array serialization?
> 
> mmap: fast, but I guess not self-describing?
> hdf5: ?

Should be pretty fast, and self describing -- advantage of being a 
standard. Disadvantage is that it requires an hdf5 library, which can b 
a pain to install on some systems.


> pickle: self-describing, but maybe not fast?
> others?

there is .tofile() and .fromfile() -- should be about as fast as you can 
get, not self-describing.

Then there is .save(), savez() and .load() (.npz format)? It should be 
pretty fast, and self-describing (but not a standard outside of numpy).

I doubt pickle will ever be your best bet.


-Chris




-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov