Challenge supporting custom deepcopy with inheritance

[Michael H. Goldwasser]

Chris,

  Thanks for your well-written reply.  Your analogy to the
  complexities of other special methods is well noted.  I'll accept
  the "small price for flexibility" that you note, if necessary.
  However, I still desire a cleaner solution.

  I can examine the inherited slots to see which special methods are
  there, and to implement my own __deepcopy__ accordingly. But to do
  so well seems to essentially require reimplementing the complicated
  logic of the copy.deepcopy function.  That is, if my new class is
  the first to be implementing an explicit __deepcopy__ function, I
  seem to have no easy way to invoke the inherited version of
  "deepcopy(self)".

  I wonder if the logic inherent in the copy.deepcopy function could
  instead be implemented directly within object.__deepcopy__ (rather
  than the current model in which object does not have __deepcopy__).
  Then I would always have a means for simulating a call to
  deepcopy(self) based upon the super.__deepcopy__ logic.

  I wouldn't be surprised if I'm overlooking some undesirable
  consequence of such a major change in the model, but I don't see one
  upon first thought.
  
Michael

On Monday June 1, 2009, Gabriel Genellina wrote: 

>    In general, you have to know whether A implements __deepcopy__ or not.  
>    This is a small price for the flexibility (or anarchy) in the copy/pickle  
>    interfases: there are several ways to implement the same thing, and you  
>    have to know which one your base class has chosen in order to extend it.
>    
>    The following is a possible implementation that doesn't use __init__ at  
>    all, so their different signature is not an issue:
>    
>          # class A:
>          def __deepcopy__(self, memo={}):
>              dup = type(self).__new__(type(self))
>              dup.__aTag = self.__aTag
>              dup.__aList = copy.deepcopy(self.__aList, memo)
>              dup.__aList.reverse()
>              return dup
>    
>          # class B:
>          def __deepcopy__(self, memo={}):
>              dup = A.__deepcopy__(self, memo)
>              dup.__bTag = self.__bTag
>              dup.__bList = copy.deepcopy(self.__bList, memo)
>              return dup
>    
>    Note that A.__deepcopy__ does two things: a) create a new, empty,  
>    instance; and b) transfer state. B.__deepcopy__ handles *its* portion of  
>    state only. This can be written in a more generic way, relying on  
>    __getstate__/__setstate__ (the methods that should return the current  
>    state of the object):
>    
>          # class A:
>          def __deepcopy__(self, memo={}):
>              dup = type(self).__new__(type(self))
>              if hasattr(self, '__getstate__'): state = self.__getstate__()
>              else: state = self.__dict__
>              state = copy.deepcopy(state, memo)
>              if hasattr(dup, '__setstate__'): dup.__setstate__(state)
>              else: dup.__dict__.update(state)
>              dup.__aList.reverse()
>              return dup
>    
>          # remove __deepcopy__ definition from class B
>    
>    Now, B (and any other subclass) is concerned only with __getstate__ /  
>    __setstate__, and only when the default implementation isn't appropriate.
>    
>    > As another basic puzzle, consider a class definition for B where B has
>    > a registry list that it doesn't want cloned for the new instance (but
>    > it does want pickled when serialized).  This would seem to require
>    > that B implement its own __deepcopy__.   We want to somehow rely on
>    > the class definition for A to enact the cloning fo the state defined
>    > by A.   But without knowing about how A supports the deepcopy
>    > semantics, I don't see how to accomplish this goal.
>    
>    I don't understand such bizarre requirement, but anyway, you can override  
>    __deepcopy__ (make B fix only the part that the default implementation  
>    doesn't implement well)
>    
>          # A.__deepcopy__ as above
>    
>          # class B:
>          def __deepcopy__(self, memo={}):
>              dup = A.__deepcopy__(self, memo)
>              dup.__registry = self.__registry
>              return dup
>    
>    This [the need to know how certain feature is implemented in the base  
>    class] is not special or peculiar to pickle/copy, although the multiple  
>    (and confusing) ways in which a class can implement pickling doesn't help  
>    to understand the issue very well.
>    
>    Consider the + operator: when some subclass wants to implement addition,  
>    it must know which of the several special methods involved (__add__,  
>    __iadd__, __radd__) are implemented in the base class, in order to  
>    extend/override them. Same for __cmp__/__eq__/__hash__: you can't just  
>    ignore what your base class implements. All of this applies to other  
>    languages too, but in Python, there is an additional consideration: the  
>    mere *existence* of some methods/attributes can have consequences on how  
>    the object behaves. In short, you can't blindly write __special__ methods.
>    
>    -- 
>    Gabriel Genellina