[issue18068] pickle + weakref.proxy(self)

[Oleg Broytman]

New submission from Oleg Broytman:

Hello! I've found a problematic behaviour of pickle when it pickles
(direct or indirect) weak proxies to self. I suspect pickle cannot
detect reference loops with weak proxies. I managed to narrow the case
to the following example:

import pickle, weakref

class TestPickle(object):
    def __init__(self):
        self.recursive = weakref.proxy(self)

    def __getstate__(self):
        print "__getstate__", id(self)
        return self.__dict__.copy()

    def __setstate__(self, d):
        print "__setstate__", id(self)
        self.__dict__.update(d)

print "- 1 -"
test = TestPickle()
print "- 2 -"
data = pickle.dumps(test, pickle.HIGHEST_PROTOCOL)
print "- 3 -"
test2 = pickle.loads(data)
print "- 4 -"
print "Result:", id(test2)
print "- 5 -"

   It prints:

- 1 -
- 2 -
__getstate__ 3075348620
__getstate__ 3075348620
- 3 -
__setstate__ 3075348844
__setstate__ 3075349004
- 4 -
Result: 3075349004
- 5 -

   That is, __getstate__ is called twice for the same object. And what
is worse, __setstate__ is called twice for different objects. The
resulting unpickled object is the last one, but in the library that I
have been debugging creation of two different objects during unpickling
is a bug.

   I can fix it by avoiding pickling the proxy and recreating the proxy
on unpickling:

import pickle, weakref

class TestPickle(object):
    def __init__(self):
        self.recursive = weakref.proxy(self)

    def __getstate__(self):
        print "__getstate__", id(self)
        d = self.__dict__.copy()
        del d['recursive']
        return d

    def __setstate__(self, d):
        print "__setstate__", id(self)
        self.__dict__.update(d)
        self.recursive = weakref.proxy(self)

print "- 1 -"
test = TestPickle()
print "- 2 -"
data = pickle.dumps(test, pickle.HIGHEST_PROTOCOL)
print "- 3 -"
test2 = pickle.loads(data)
print "- 4 -"
print "Result:", id(test2)
print "- 5 -"

- 1 -
- 2 -
__getstate__ 3075070092
- 3 -
__setstate__ 3075070188
- 4 -
Result: 3075070188
- 5 -

   But I wonder if it's a bug that should be fixed? If it's an expected
behaviour it perhaps should be documented as a warning in docs for
pickle or weakref or both.

----------
components: Library (Lib)
messages: 190105
nosy: phd
priority: normal
severity: normal
status: open
title: pickle + weakref.proxy(self)
type: behavior
versions: Python 2.6, Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18068>
_______________________________________