[Numpy-discussion] Nasty bug with recarray and cPickle

[Bruce Southey]

James Philbin wrote:
> Anyone?
>
> James
>
> On Thu, Nov 6, 2008 at 2:53 PM, James Philbin <philbinj at gmail.com> wrote:
>   
>> Hi,
>>
>> I might be doing something stupid so I thought i'd check here before
>> filing a bug report.
>> Firstly:
>> In [8]: np.__version__
>> Out[8]: '1.3.0.dev5883'
>>
>> Basically, pickling an element from a recarray seems to break silently:
>> In [1]: import numpy as np
>>
>> In [2]: dtype = [('r','f4'),('g','f4'),('b','f4')]
>>
>> In [3]: arr = np.ones((10,), dtype=dtype)
>>
>> In [4]: arr
>> Out[4]:
>> array([(1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0),
>>       (1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0),
>>       (1.0, 1.0, 1.0), (1.0, 1.0, 1.0)],
>>      dtype=[('r', '<f4'), ('g', '<f4'), ('b', '<f4')])
>>
>> In [5]: arr[0]
>> Out[5]: (1.0, 1.0, 1.0)
>>
>> In [6]: import cPickle; cPickle.loads(cPickle.dumps(arr[0]))
>> Out[6]: (0.0, 0.0, 1.8643547392640242e-38)
>>
>>
>> Thanks,
>> James
>>
>>     
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>   
I also get the same on my 64-bit linux Fedora rawhide with
1) Python 2.5.2 and '1.2.0'
2) Python 2.4.5 and numpy '1.1.1'
3) Python 2.6 and numpy '1.3.0.dev5935'
Further, cPickle.dumps(arr) looks fine as does using 
cPickle.dumps(arr[0].tolist()).

Bruce

Using numpy 1.2.0
 >>> dtype = [('r','f4'),('g','f4'),('b','f4')]
 >>> dtype
[('r', 'f4'), ('g', 'f4'), ('b', 'f4')]
 >>> arr = np.ones((10,), dtype=dtype)
 >>> arr
array([(1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0),
       (1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0),
       (1.0, 1.0, 1.0), (1.0, 1.0, 1.0)],
      dtype=[('r', '<f4'), ('g', '<f4'), ('b', '<f4')])
 >>> arr[0]
(1.0, 1.0, 1.0)
 >>> import cPickle; cPickle.loads(cPickle.dumps(arr[0]))
(1.704050679100351e-37, 0.0, 1.704050679100351e-37)
 >>> import cPickle; cPickle.loads(cPickle.dumps(arr))
array([(1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0),
       (1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0), (1.0, 1.0, 1.0),
       (1.0, 1.0, 1.0), (1.0, 1.0, 1.0)],
      dtype=[('r', '<f4'), ('g', '<f4'), ('b', '<f4')])
 >>> import cPickle; cPickle.loads(cPickle.dumps(arr[0].tolist()))
(1.0, 1.0, 1.0)