pickle/unpickle class which has changed

[Peter Otten]

Neal Becker wrote:

> Peter Otten wrote:
> 
>> Steven D'Aprano wrote:
>> 
>>> On Tue, 06 Mar 2012 07:34:34 -0500, Neal Becker wrote:
>>> 
>>>> What happens if I pickle a class, and later unpickle it where the class
>>>> now has added some new attributes?
>>> 
>>> Why don't you try it?
>>> 
>>> py> import pickle
>>> py> class C:
>>> ...     a = 23
>>> ...
>>> py> c = C()
>>> py> pickled = pickle.dumps(c)
>>> py> C.b = 42  # add a new class attribute
>>> py> d = pickle.loads(pickled)
>>> py> d.a
>>> 23
>>> py> d.b
>>> 42
>>> 
>>> 
>>> Unless you mean something different from this, adding attributes to the
>>> class is perfectly fine.
>>> 
>>> But... why are you dynamically adding attributes to the class? Isn't
>>> that rather unusual?
>> 
>> The way I understand the problem is that an apparently
>> backwards-compatible change like adding a third dimension to a point with
>> an obvious default breaks when you restore an "old" instance in a script
>> with the "new" implementation:
>> 
>>>>> import pickle
>>>>> class P(object):
>> ...     def __init__(self, x, y):
>> ...             self.x = x
>> ...             self.y = y
>> ...     def r2(self):
>> ...             return self.x*self.x + self.y*self.y
>> ...
>>>>> p = P(2, 3)
>>>>> p.r2()
>> 13
>>>>> s = pickle.dumps(p)
>>>>> class P(object):
>> ...     def __init__(self, x, y, z=0):
>> ...             self.x = x
>> ...             self.y = y
>> ...             self.z = z
>> ...     def r2(self):
>> ...             return self.x*self.x + self.y*self.y + self.z*self.z
>> ...
>>>>> p = P(2, 3)
>>>>> p.r2()
>> 13
>>>>> pickle.loads(s).r2()
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "<stdin>", line 7, in r2
>> AttributeError: 'P' object has no attribute 'z'
>> 
>> By default pickle doesn't invoke __init__() and updates __dict__
>> directly. As pointed out in my previous post one way to fix the problem
>> is to implement a __setstate__() method:
>> 
>>>>> class P(object):
>> ...     def __init__(self, x, y, z=0):
>> ...             self.x = x
>> ...             self.y = y
>> ...             self.z = z
>> ...     def r2(self):
>> ...             return self.x*self.x + self.y*self.y + self.z*self.z
>> ...     def __setstate__(self, state):
>> ...             self.__dict__["z"] = 42 # stupid default
>> ...             self.__dict__.update(state)
>> ...
>>>>> pickle.loads(s).r2()
>> 1777
>> 
>> This keeps working with pickles of the new implementation of P:
>> 
>>>>> q = P(3, 4, 5)
>>>>> pickle.loads(pickle.dumps(q)).r2()
>> 50
> 
> So if in my new class definition there are now some new attributes, and if
> I did not add a __setstate__ to set the new attributes, I guess then when
> unpickled the instance of the class will simply lack those attributes?

I don't know. If you don't trust the demo try it yourself with the actual 
code you have. Throwing in

obj = pickle.load(...)
print vars(obj)

should help.