[Numpy-discussion] Question about np.savez

[Francesc Alted]

A Wednesday 02 September 2009 05:50:57 Robert Kern escrigué:
> On Tue, Sep 1, 2009 at 21:11, Jorge Scandaliaris<jorgesmbox-ml at yahoo.es> 
wrote:
> > David Warde-Farley <dwf <at> cs.toronto.edu> writes:
> >> If you actually want to save multiple arrays, you can use
> >> savez('fname', *[a,b,c]) and they will be accessible under the names
> >> arr_0, arr_1, etc. and a list of these names is in the 'files'
> >> attribute on the NpzFile object. To retrieve your list of arrays when
> >> you load, you can just do
> >>
> >> mynewlist = [data[arrname] for arrname in data.files]
> >
> > Thanks for the tip. I have realized, though, that I might need some more
> > flexibility than just the ability to save ndarrays. The data I am dealing
> > with is best kept in a hierarchical way (I could represent the structure
> > with ndarrays also, but I think it would be messy and difficult). I am
> > having a look at h5py to see if it fulfill my needs. I know there is
> > pytables, too, but from having a quick look it seems h5py is simpler. Am
> > I right on this?. I also get a nice side-effect, the data would be
> > readable by the de-facto standard software used by most people in my
> > field.
>
> If there is a particular format that uses HDF5 that you are trying to
> replicate, h5py is the clear answer. However, PyTables will, by and
> large, make files that are entirely readable by other HDF5 libraries
> when you just use the subset of features that is supported by
> HDF5-proper. For example, tables and arrays work just fine. What won't
> be supported by non-PyTables libraries are things like dataset
> attributes which are pickled objects. Your non-PyTables HDF5 apps will
> see some extraneous attributes on the arrays and tables, but those are
> typically not necessary for interpretation.

Most of these 'extraneous' attributes are derived from the use of the high 
level HDF5 interface (http://www.hdfgroup.org/HDF5/doc/HL/).  If they bother 
you, you can get rid of them by setting the parameter ``PYTABLES_SYS_ATTRS`` 
to false (either in tables/parameters.py or passing it to `tables.openFile`).

-- 
Francesc Alted