[Numpy-discussion] Pickling ufuncs?
James A. Bednar
jbednar at inf.ed.ac.uk
Mon Jan 29 16:57:33 EST 2007
Hi,
Does anyone know whether it is possible to pickle and unpickle numpy
ufuncs? I can't find anything about that on scipy.org or the mailing
list archives. I have several important pieces of code that accept a
numpy ufunc as an argument and later apply it to some data, while
keeping a copy of the ufunc in an attribute. So far I have been able
to pickle this code only by doing some very ugly hacks and
workarounds.
The ufuncs from Numeric 24 and earlier did not even deepcopy, which
caused us lots of other problems, but deepcopying them works now in
numpy 1.0.1. However, they still don't seem to pickle with the regular
picklers. Is this deliberately disabled for some reason, or is there
some workaround?
Here's an example that illustrates the problem:
I have a class Test defined in test.py:
class Test(object):
def __init__(self,a):
self.a = a
If I try this code at the commandline:
import numpy
from test import Test
t=Test(numpy.multiply)
import pickle
s=pickle.dumps(t)
I get "TypeError: can't pickle ufunc objects"
Also, I can't reproduce it in a small example yet, but pickling
completes without errors in our larger program that also stores
ufuncs, yet I then get a different error on unpickling:
File "/disk/home/lodestar1/jbednar/topographica/lib/python2.4/pickle.py", line 872, in load
dispatch[key](self)
File "/disk/home/lodestar1/jbednar/topographica/lib/python2.4/pickle.py", line 1097, in load_newobj
obj = cls.__new__(cls, *args)
TypeError: object.__new__(numpy.ufunc) is not safe, use numpy.ufunc.__new__()
Any ideas?
Thanks,
Jim
_______________________________________________________________________________
Unix> cat test.py
class Test(object):
def __init__(self,a):
self.a = a
Unix> bin/python
Python 2.4.4 (#5, Nov 9 2006, 22:58:03)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> from test import Test
>>> t=Test(numpy.multiply)
>>> import pickle
>>> s=pickle.dumps(t)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/home/jb/lib/python2.4/pickle.py", line 1386, in dumps
Pickler(file, protocol, bin).dump(obj)
File "/home/jb/lib/python2.4/pickle.py", line 231, in dump
self.save(obj)
File "/home/jb/lib/python2.4/pickle.py", line 338, in save
self.save_reduce(obj=obj, *rv)
File "/home/jb/lib/python2.4/pickle.py", line 433, in save_reduce
save(state)
File "/home/jb/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "/home/jb/lib/python2.4/pickle.py", line 663, in save_dict
self._batch_setitems(obj.iteritems())
File "/home/jb/lib/python2.4/pickle.py", line 677, in _batch_setitems
save(v)
File "/home/jb/lib/python2.4/pickle.py", line 313, in save
rv = reduce(self.proto)
File "/home/jb/lib/python2.4/copy_reg.py", line 69, in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle ufunc objects
>>>
More information about the NumPy-Discussion
mailing list