pickle/unpickle class which has changed

[Peter Otten]

Steven D'Aprano wrote:

> On Tue, 06 Mar 2012 07:34:34 -0500, Neal Becker wrote:
> 
>> What happens if I pickle a class, and later unpickle it where the class
>> now has added some new attributes?
> 
> Why don't you try it?
> 
> py> import pickle
> py> class C:
> ...     a = 23
> ...
> py> c = C()
> py> pickled = pickle.dumps(c)
> py> C.b = 42  # add a new class attribute
> py> d = pickle.loads(pickled)
> py> d.a
> 23
> py> d.b
> 42
> 
> 
> Unless you mean something different from this, adding attributes to the
> class is perfectly fine.
> 
> But... why are you dynamically adding attributes to the class? Isn't that
> rather unusual?

The way I understand the problem is that an apparently backwards-compatible 
change like adding a third dimension to a point with an obvious default 
breaks when you restore an "old" instance in a script with the "new" 
implementation:

>>> import pickle
>>> class P(object):
...     def __init__(self, x, y):
...             self.x = x
...             self.y = y
...     def r2(self):
...             return self.x*self.x + self.y*self.y
... 
>>> p = P(2, 3)
>>> p.r2()
13
>>> s = pickle.dumps(p)
>>> class P(object):
...     def __init__(self, x, y, z=0):
...             self.x = x
...             self.y = y
...             self.z = z
...     def r2(self):
...             return self.x*self.x + self.y*self.y + self.z*self.z
... 
>>> p = P(2, 3)
>>> p.r2()
13
>>> pickle.loads(s).r2()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 7, in r2
AttributeError: 'P' object has no attribute 'z'

By default pickle doesn't invoke __init__() and updates __dict__ directly.
As pointed out in my previous post one way to fix the problem is to 
implement a __setstate__() method:

>>> class P(object):
...     def __init__(self, x, y, z=0):
...             self.x = x
...             self.y = y
...             self.z = z
...     def r2(self):
...             return self.x*self.x + self.y*self.y + self.z*self.z
...     def __setstate__(self, state):
...             self.__dict__["z"] = 42 # stupid default
...             self.__dict__.update(state)
... 
>>> pickle.loads(s).r2()
1777

This keeps working with pickles of the new implementation of P:

>>> q = P(3, 4, 5)
>>> pickle.loads(pickle.dumps(q)).r2()
50