[Numpy-discussion] saving groups of numpy arrays to disk

[Paul Anton Letnes]

Hi!
On 21. aug. 2011, at 00.18, Chris Withers wrote:

> Hi All,
> 
> I've got a tree of nested dicts that at their leaves end in numpy arrays 
> of identical sizes.
> 
> What's the easiest way to persist these to disk so that I can pick up 
> with them where I left off?

Probably giving them names like
trunk_branch_leaf.txt
with numpy.savetxt, if you want it quick and dirty. Or possibly, use numpy.savez directly on your dict.

> What's the most "correct" way to do so?
> 
> I'm using IPython if that makes things easier...
> 
> I had wondered about PyTables, but that seems a bit too heavyweight for 
> this, unless I'm missing something?

In my (perhaps limited) experience, hdf5 is great for this. I personally use h5py, I believe it is a little lighter. You get the "tree structure" for free in something like a directory structure:
/branch/leaf
/trunk/branch/leaf
etc.

Cheers
Paul