[Numpy-discussion] Fastest way to save a dictionary of numpy record arrays

[Warren Weckesser]

Robert Kern wrote:
> On Mon, Jun 14, 2010 at 19:00, Vishal Rana <ranavishal at gmail.com> wrote:
>   
>> Hi,
>> I have dictionary of numpy record arrays, what could be fastest way to
>> save/load to/from a disk. I tried numpy.save() but my dictionary is lost and
>> cPickle seems to be slow.
>>     
>
> numpy.savez() will save a dictionary of arrays out to a .zip file.
> Each key/value pair will map to a file in the .zip file with a file
> name corresponding to the key.
>
>   

Hey Robert,

If I expand the dictionary to keyword arguments to savez, it works 
beautifully:

-----
In [4]: a = np.array([[1,2,3],[4,5,6]])

In [5]: b = np.array([('foo',1),('bar',2)], dtype=[('name', 'S8'), 
('code', int)])

In [6]: d = dict(a=a, b=b)

In [7]: np.savez('mydata.npz', **d)

In [8]: q = np.load('mydata.npz')

In [9]: q['a']
Out[9]:
array([[1, 2, 3],
       [4, 5, 6]])

In [10]: q['b']
Out[10]:
array([('foo', 1), ('bar', 2)],
      dtype=[('name', '|S8'), ('code', '<i4')])
-----


But if I just pass in the dictionary to savez:

-----
In [26]: np.savez('mydata2.npz', d)

In [27]: q2 = np.load('mydata2.npz')

In [28]: q2.files
Out[28]: ['arr_0']

In [29]: q2['arr_0']
Out[29]:
array({'a': array([[1, 2, 3],
       [4, 5, 6]]), 'b': array([('foo', 1), ('bar', 2)],
      dtype=[('name', '|S8'), ('code', '<i4')])}, dtype=object)
-----

What would be the canonical way to pull this apart to get the arrays?

Warren