[Numpy-discussion] object array binary pickling and nan

[David M. Cooke]

On Fri, 28 Jul 2006 12:17:39 -0500
"Tom Denniston" <tom.denniston at alum.dartmouth.org> wrote:

> It seems that object arrays will not pickle if they have a nan when
> the pickle protocol is set to binary.  From what I can tell the object
> array simply delegates to python (which makes sense) which in turn
> cannot pickle nans in binary format.  It is unfortunate because it is
> very useful to have heterogenous arrays that include nans.  What do
> other people do in this situation?  Does anyone know why python has
> this limitation?  Is there an intelligent workaround other than search
> and replace?  Would it be worth it to put an intelligent workaround
> into numpy so it is transparent to the user?  I was wondering what
> people thought.
> 
> Code that reproduces the problem:
> 
> This is regular python:
> pickle.dumps(numpy.nan, 2)
> SystemError: frexp() result out of range
> 
> This is fine in numpy:
> pickle.dumps(numpy.array([numpy.nan]), 2)
> 
> This breaks:
> pickle.dumps(numpy.array([numpy.nan], numpy.PyObject), 2)
> SystemError: frexp() result out of range

I believe this will be fixed in Python 2.5 (see
http://www.python.org/sf/445484). I guess your best alternative would be to
use a numpy scalar object (like double) instead of a Python float for NaN's.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|cookedm at physics.mcmaster.ca