[Numpy-discussion] object array binary pickling and nan

[Tom Denniston]

It seems that object arrays will not pickle if they have a nan when
the pickle protocol is set to binary.  From what I can tell the object
array simply delegates to python (which makes sense) which in turn
cannot pickle nans in binary format.  It is unfortunate because it is
very useful to have heterogenous arrays that include nans.  What do
other people do in this situation?  Does anyone know why python has
this limitation?  Is there an intelligent workaround other than search
and replace?  Would it be worth it to put an intelligent workaround
into numpy so it is transparent to the user?  I was wondering what
people thought.

Code that reproduces the problem:

This is regular python:
pickle.dumps(numpy.nan, 2)
SystemError: frexp() result out of range

This is fine in numpy:
pickle.dumps(numpy.array([numpy.nan]), 2)

This breaks:
pickle.dumps(numpy.array([numpy.nan], numpy.PyObject), 2)
SystemError: frexp() result out of range


--Tom