[Python-Dev] Pickle implementation questions
tim.peters at gmail.com
Fri Jun 30 20:29:43 CEST 2006
> So just to be clear, is it something like this?
I hope you've read PEP 307:
That's where __reduce_ex__ was introduced (along with all the rest of
pickle protocol 2).
> class object:
> def __reduce__(self):
> return copy_reg._reduce_ex(self, -1)
> def __reduce_ex__(self, protocol):
> return copy_reg._reduce_ex(self, protocol)
The implementation is more like:
def __common_reduce__(self, proto=0):
if self.__class__.__reduce__ is not object.__reduce__:
# The class overrode __reduce__, so call the override.
# From PEP 307:
# The 'object' class implements both __reduce__ and
# __reduce_ex__; however, if a subclass overrides __reduce__
# but not __reduce_ex__, the __reduce_ex__ implementation
# detects this and calls __reduce__.
elif proto < 2:
return copy_reg._reduce_ex(self, proto)
# about 130 lines of C code exploiting proto 2
__reduce__ = __reduce_ex__ = __common_reduce__
> Does _reduce_ex's behavior actually change depending on the specified protocol
> version? The only difference that I can see or think of is that an assert causes it to
> fail if the protocol is >= 2.
That's right. As above, the object reduce methods never call
copy_reg._reduce_ex() when proto >= 2.
Note that __reduce_ex__ doesn't exist for the _benefit_ of object: it
was introduced in protocol 2 for the benefit of user classes that want
to exploit protocol-specific pickle opcodes in their own __reduce__
methods. They couldn't do that using the old-time __reduce__ because
__reduce__ wasn't passed the protocol version.
copy_reg._reduce_ex exists only because Guido got fatally weary of
writing mountains of C code, so left what "should be" a rarely-taken
path coded in Python.
>>> - What does copy_reg.constructor() do?
>> It does this:
>> def constructor(object):
>> if not callable(object):
>> raise TypeError("constructors must be callable")
> So it is part of the public interface? It's exported in __all__, but it appears
> that it's undocumented.
It's documented in the Library Reference Manual, in the `copy_reg` docs:
Declares object to be a valid constructor. If object is not callable
(and hence not valid as a constructor), raises TypeError.
Unfortunately, while all the "safe for unpickling?" gimmicks (of which
this is one -- see PEP 307 again) were abandoned in Python 2.3, the
docs and code comments still haven't entirely caught up.
copy_reg.constructor() exists now only for backward compatibility (old
code may still call it, but it no longer has any real use).
More information about the Python-Dev