Is there a way to store metadata for an array? For example, date the samples were collected, name of the operator, etc. Regards, -- Emmanuel
I believe there are no provisions made for that in ndarray. But you can subclass ndarray. Val On Wed, Jan 25, 2012 at 12:10 PM, Emmanuel Mayssat <emayssat@gmail.com>wrote:
Is there a way to store metadata for an array? For example, date the samples were collected, name of the operator, etc.
Regards, -- Emmanuel _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
If by "store" you mean "store on disk", I recommend h5py datasets and attributes. Reportedly pytables is also good but I don't have any first hand experience there. Both python modules use the hdf5 library, written in C/C++/Fortran. Paul On Wed, Jan 25, 2012 at 7:47 PM, Val Kalatsky <kalatsky@gmail.com> wrote:
I believe there are no provisions made for that in ndarray. But you can subclass ndarray. Val
On Wed, Jan 25, 2012 at 12:10 PM, Emmanuel Mayssat <emayssat@gmail.com> wrote:
Is there a way to store metadata for an array? For example, date the samples were collected, name of the operator, etc.
Regards, -- Emmanuel _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 26 Jan 2012, at 13:30, Paul Anton Letnes wrote:
If by "store" you mean "store on disk", I recommend h5py datasets and attributes. Reportedly pytables is also good but I don't have any first hand experience there. Both python modules use the hdf5 library, written in C/C++/Fortran.
Paul
On Wed, Jan 25, 2012 at 7:47 PM, Val Kalatsky <kalatsky@gmail.com> wrote:
I believe there are no provisions made for that in ndarray. But you can subclass ndarray.
You could probably use structured arrays with string and datetype fields for the metadata and multidimensional fields (i.e. effectively subarrays within the structured array) for the actual data. For file storage, they could probably be directly saved as .npy, if interoperability is not a concern. Otherwise I'd also highly recommend hdf5; with both h5py and pytables allowing quite transparent conversion of structured arrays to datasets in the HDF5, but you also have the option to store other objects, like dictionary elements, within the same data structure. Pytables is generally regarded as having a more database-oriented approach, while h5py appears more straightforward to use from a numerics background (at least in my experience). Cheers, Derek
subclassing is what I was looking for. Indeed the code is almost available at http://docs.scipy.org/doc/numpy/user/basics.subclassing.html#simple-example-... I just created a dictionary variable which I called 'metadata' I had to overload the __repr__ method to print my parameters in the python shell. As far as saving the data on the disk.... let me start a new thread ;-) -- Emmanuel On Thu, Jan 26, 2012 at 5:49 AM, Derek Homeier <derek@astro.physik.uni-goettingen.de> wrote:
On 26 Jan 2012, at 13:30, Paul Anton Letnes wrote:
If by "store" you mean "store on disk", I recommend h5py datasets andhttp://docs.scipy.org/doc/numpy/user/basics.subclassing.html attributes. Reportedly pytables is also good but I don't have any first hand experience there. Both python modules use the hdf5 library, written in C/C++/Fortran.
Paul
On Wed, Jan 25, 2012 at 7:47 PM, Val Kalatsky <kalatsky@gmail.com> wrote:
I believe there are no provisions made for that in ndarray. But you can subclass ndarray.
You could probably use structured arrays with string and datetype fields for the metadata and multidimensional fields (i.e. effectively subarrays within the structured array) for the actual data. For file storage, they could probably be directly saved as .npy, if interoperability is not a concern. Otherwise I'd also highly recommend hdf5; with both h5py and pytables allowing quite transparent conversion of structured arrays to datasets in the HDF5, but you also have the option to store other objects, like dictionary elements, within the same data structure. Pytables is generally regarded as having a more database-oriented approach, while h5py appears more straightforward to use from a numerics background (at least in my experience).
Cheers, Derek
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (4)
-
Derek Homeier -
Emmanuel Mayssat -
Paul Anton Letnes -
Val Kalatsky