[Numpy-discussion] Home for pyhdf5io?

[Francesc Alted]

A Monday 25 May 2009 20:29:25 Eric Firing escrigué:
> Francesc Alted wrote:
> > A Monday 25 May 2009 19:55:28 Eric Firing escrigué:
> >>>> If the aim is to come up with a method of saving numpy arrays that
> >>>> uses a standard protocol and does not introduce large dependencies,
> >>>> then could this be accomplished using netcdf instead of hdf5,
> >>>> specifically Roberto De Almeida's pupynere, which is already in
> >>>> scipy.io as netcdf.py?  Or does hdf5 have essential characteristics
> >>>> for this purpose that netcdf lacks?
> >
> > After looking a bit at the code of pupynere, there is the next line:
> >
> > assert magic == 'CDF', "Error: %s is not a valid NetCDF 3 file" %
> > self.filename
> >
> > So, the current version of pupynere is definitely for version 3 of
> > NetCDF, not version 4.
>
> Yes, and I presume it will stay that way--which is fine for the question
> I am asking above.  I should have said "netcdf3" explicitly.  Its
> simplicity compared to hdf5 and netcdf4 is potentially a virtue.
>
> The question is, is it *too* simple for the intended purpose?

I don't think the question is whether a format would be too simple or not, but 
rather about file compatibility.  In that sense HDF5 is emerging as a standard 
de facto, and many tools are acquiring the capability to read/write this 
format (e.g. Matlab, IDL, Octave, Mathematica, R, NetCDF4-based apps and many 
others).  Having this interchange capability is what should be seen as 
desirable, IMO.

>
> >>> >From what I understand, netCFD is based on on HDF5, at least as of the
> >>>
> >>> version 4 release.
> >>
> >> Netcdf4 is indeed built on hdf5, but netcdf3 is not, and netcdf3 format
> >> is likely to stick around for a *very* long time.  The netcdf4 library
> >> is backwards-compatible with netcdf3.
> >
> > NetCDF4 is backwards-compatible with NetCDF3 just at API level, not the
> > file format.  NetCDF3 has a much more simple format, and completely
> > different from NetCDF4, which is based on HDF5.
>
> Yes, but the netcdf4 *library* includes full netcdf3 compatibility; you
> can read and write netcdf3 using the netcdf4 library.  For example, you
> can build Jeff Whitaker's http://code.google.com/p/netcdf4-python/ with
> all the hdf5 bells and whistles, and it will still happily read and,
> upon request, write netcdf3 files.

Again, I think that the issue is compatibility with other tools, not just 
between  NetCDF3/NetCDF4 worlds.

-- 
Francesc Alted