[Python-Dev] The type of the result of the copy() method

[Guido van Rossum]

On Tue, Oct 31, 2017 at 3:12 AM, Serhiy Storchaka <storchaka at gmail.com>
wrote:

> 29.10.17 19:04, Guido van Rossum пише:
>
>> It's somewhat problematic. If I subclass dict with a different
>> constructor, but I don't overload copy(), how can the dict.copy() method
>> construct a correct instance of the subclass? Even if the constructor
>> signatures match, how can dict.copy() make sure it copies all attributes
>> properly? Without an answer to these questions I think it's better to admit
>> defeat and return a dict instance -- classes that want to do better should
>> overload copy().
>>
>> I notice that Counter.copy() has all the problems I indicate here -- it
>> works as long as you don't add attributes or change the constructor
>> signature. I bet this isn't documented anywhere.
>>
>
> I am familiar with these reasons, and agree with them. But I'm curious why
> some collections chose the way of creating an instance of the same class.
> For creating an instance of the same class we have the __copy__() method.
>
> An attempt to preserve a class in the returned value can cause problems.
> For example, the __add__() and __mul__() methods of deque first make a copy
> of the same type, and this can cause a crash [1]. Of course this is not
> occurred in real code, it is just yet one way of crashing the interpreter
> from Python code. list and tuple are free from this problem since their
> corresponding methods (as well as copy()) create an instance of the
> corresponding base type.
>
> I think there were reasons for copying the type in results. It would be
> nice to formalize the criteria, in what cases copy() and other methods
> should return an instance of the base class, and in what cases they should
> create an instance of the same type as the original object. This would help
> for new types. And maybe we need to change some existing type (the
> inconsistency between WeakKeyDictionary and WeakSet looks weird).
>
> [1] https://bugs.python.org/issue31608
>

I think it all depends on the use case. (Though in some cases I suspect the
class' author didn't think too hard about it.)

The more strict rule should be that a base class cannot know how to create
a subclass instance and hence it should not bother. (Or perhaps it should
use the __copy__ protocol.) But there are some cases where a useful pattern
of subclassing a stdlib class just to add some convenience methods to it,
without changing its essence. In those cases, it might be convenient that
by default you get something that preserves its type (and full contents)
when copying without having to explicitly implement copy() or __copy__().

Another useful rule is that if a class *does* have a copy() method, a
subclass *ought* to override it (or __copy__()) to make it work right.

IOW from the class author's POV, copy() should not attempt to copy the type
of a subclass. But from the user's POV copy() is more useful if it copies
the type. This places the burden on the subclass author to override copy()
or __copy__().

Traditionally we've done a terrible job at documenting what you should to
do subclass a class, and what you can expect from the base class (e.g.
which parts of the base class are part of the API for subclasses, and which
parts are truly private -- underscores aren't used consistently in many
class implementations).

For those classes that currently preserve the type in copy(), perhaps we
could document that if one overrides __init__() or __new__() one should
also override copy() or __copy__().

And for future classes we should recommend whether it's preferred to
preserve the type in copy() or not -- I'm not actually sure what to
recommend here. I guess it depends on what other methods of the class
return new instances. If there are a lot (like for int or str) then copy()
should follow those methods' lead.

Sorry about the rambling, this is hard to get consistent.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171031/329d3bda/attachment.html>