Hi, I try to save the contents of a numpy recarray with PyTables into a file. That works well, however, if I then try to retrieve the data, I get back an array with matching dtypes rather than a recarray. What is the best way to get a recarray back, or if that's not possible what's the most efficient way to convert an array to a recarray? This is what I am doing at the moment: Python 2.5 (r25:51908, Oct 4 2006, 17:28:51) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import numpy as N import tables as t num = 2 a = N.recarray(num, formats='i4,f8,f8',names='id,x,y') a['id'] = [3,4] a['x'] = [3.4,4.5] a['y'] = [4.6,4.5] a recarray([(3, 3.3999999999999999, 4.5999999999999996), (4, 4.5, 4.5)], dtype=[('id', '<i4'), ('x', '<f8'), ('y', '<f8')]) f = t.openFile('test.h5', 'w') f.createTable('/', 'test', a) /test (Table(2L,)) '' description := { "id": Int32Col(shape=(), dflt=0, pos=0), "x": Float64Col(shape=(), dflt=0.0, pos=1), "y": Float64Col(shape=(), dflt=0.0, pos=2)} byteorder := 'little' chunkshape := (409,) f.close() f = t.openFile('test.h5', 'r') b = f.root.test[:] b array([(3, 3.3999999999999999, 4.5999999999999996), (4, 4.5, 4.5)], dtype=[('id', '<i4'), ('x', '<f8'), ('y', '<f8')]) type(b) <type 'numpy.ndarray'> type(a) <class 'numpy.core.records.recarray'>
Best regards, Hanno -- Hanno Klemm klemm@phys.ethz.ch
A Friday 30 May 2008, Hanno Klemm escrigué:
Hi,
I try to save the contents of a numpy recarray with PyTables into a file. That works well, however, if I then try to retrieve the data, I get back an array with matching dtypes rather than a recarray.
What is the best way to get a recarray back, or if that's not possible what's the most efficient way to convert an array to a recarray?
This is what I am doing at the moment:
Python 2.5 (r25:51908, Oct 4 2006, 17:28:51) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import numpy as N import tables as t num = 2 a = N.recarray(num, formats='i4,f8,f8',names='id,x,y') a['id'] = [3,4] a['x'] = [3.4,4.5] a['y'] = [4.6,4.5] a
recarray([(3, 3.3999999999999999, 4.5999999999999996), (4, 4.5, 4.5)], dtype=[('id', '<i4'), ('x', '<f8'), ('y', '<f8')])
f = t.openFile('test.h5', 'w') f.createTable('/', 'test', a)
/test (Table(2L,)) '' description := { "id": Int32Col(shape=(), dflt=0, pos=0), "x": Float64Col(shape=(), dflt=0.0, pos=1), "y": Float64Col(shape=(), dflt=0.0, pos=2)} byteorder := 'little' chunkshape := (409,)
f.close() f = t.openFile('test.h5', 'r') b = f.root.test[:] b
array([(3, 3.3999999999999999, 4.5999999999999996), (4, 4.5, 4.5)], dtype=[('id', '<i4'), ('x', '<f8'), ('y', '<f8')])
type(b)
<type 'numpy.ndarray'>
type(a)
<class 'numpy.core.records.recarray'>
Yeah, this is on purpose because ndarray classes are actually C extensions and they are generally more efficient than Python classes. If what you want is a recarray class (Python class), you can always create a view as Pierre suggested. However, this is rarely needed because you can access most of the recarray functionality out of the ndarray class. Cheers, -- Francesc Alted
Francesc, Pierre, thanks for your replies, the view works fine. The reason why I like recarrays is that you can address your fields simply with a.Name, when a is your array. I find that highly convenient when working with data interactively. As far as I know there is no similar possbility to do that with ndarrays, or is there? Cheers, Hanno Francesc Alted <falted@pytables.org> said:
A Friday 30 May 2008, Hanno Klemm escrigué:
Hi,
I try to save the contents of a numpy recarray with PyTables into a file. That works well, however, if I then try to retrieve the data, I get back an array with matching dtypes rather than a recarray.
What is the best way to get a recarray back, or if that's not possible what's the most efficient way to convert an array to a recarray?
This is what I am doing at the moment:
Python 2.5 (r25:51908, Oct 4 2006, 17:28:51) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import numpy as N import tables as t num = 2 a = N.recarray(num, formats='i4,f8,f8',names='id,x,y') a['id'] = [3,4] a['x'] = [3.4,4.5] a['y'] = [4.6,4.5] a
recarray([(3, 3.3999999999999999, 4.5999999999999996), (4, 4.5, 4.5)], dtype=[('id', '<i4'), ('x', '<f8'), ('y', '<f8')])
f = t.openFile('test.h5', 'w') f.createTable('/', 'test', a)
/test (Table(2L,)) '' description := { "id": Int32Col(shape=(), dflt=0, pos=0), "x": Float64Col(shape=(), dflt=0.0, pos=1), "y": Float64Col(shape=(), dflt=0.0, pos=2)} byteorder := 'little' chunkshape := (409,)
f.close() f = t.openFile('test.h5', 'r') b = f.root.test[:] b
array([(3, 3.3999999999999999, 4.5999999999999996), (4, 4.5, 4.5)], dtype=[('id', '<i4'), ('x', '<f8'), ('y', '<f8')])
type(b)
<type 'numpy.ndarray'>
type(a)
<class 'numpy.core.records.recarray'>
Yeah, this is on purpose because ndarray classes are actually C extensions and they are generally more efficient than Python classes.
If what you want is a recarray class (Python class), you can always create a view as Pierre suggested. However, this is rarely needed because you can access most of the recarray functionality out of the ndarray class.
Cheers,
-- Francesc Alted _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- Hanno Klemm klemm@phys.ethz.ch
On Monday 02 June 2008 08:03:15 Hanno Klemm wrote:
thanks for your replies, the view works fine. The reason why I like recarrays is that you can address your fields simply with a.Name, when a is your array. I find that highly convenient when working with data interactively.
As far as I know there is no similar possbility to do that with ndarrays, or is there?
No, but you can access the field Name of your ndarray a with a['Name'], which is just as convenient. An issue with recarrays is that the __getattribute__ and __setattr__ methods are overwritten, which tends to reduce performance.
participants (3)
-
Francesc Alted -
Hanno Klemm -
Pierre GM