[Python-Dev] Return type of datetime subclasses added to timedelta

Alexander Belopolsky alexander.belopolsky at gmail.com
Mon Feb 4 14:38:00 EST 2019 

I'll merge it tonight.

On Mon, Feb 4, 2019 at 2:22 PM Guido van Rossum <guido at python.org> wrote:

> OK, I approved the PR. Can some other core dev ensure that it gets merged?
> No backports though!
>
> On Mon, Feb 4, 2019 at 8:46 AM Paul Ganssle <paul at ganssle.io> wrote:
>
>> There's already a PR, actually, #10902:
>> https://github.com/python/cpython/pull/10902
>>
>> Victor reviewed and approved it, I think before I started this thread, so
>> now it's just waiting on merge.
>> On 2/4/19 11:38 AM, Guido van Rossum wrote:
>>
>> I recommend that you submit a PR so we can get it into 3.8 alpha 2.
>>
>> On Mon, Feb 4, 2019 at 5:50 AM Paul Ganssle <paul at ganssle.io> wrote:
>>
>>> Hey all,
>>>
>>> This thread about the return type of datetime operations seems to have
>>> stopped without any explicit decision - I think I responded to everyone who
>>> had objections, but I think only Guido has given a +1 to whether or not we
>>> should go ahead.
>>>
>>> Have we got agreement to go ahead with this change? Are we still
>>> targeting Python 3.8 here?
>>>
>>> For those who don't want to dig through your old e-mails, here's the
>>> archive link for this thread:
>>> https://mail.python.org/pipermail/python-dev/2019-January/155984.html
>>>
>>> If you want to start commenting on the actual implementation, it's
>>> available here (though it's pretty simple):
>>> https://github.com/python/cpython/pull/10902
>>>
>>> Best,
>>>
>>> Paul
>>>
>>>
>>> On 1/6/19 7:17 PM, Guido van Rossum wrote:
>>>
>>> OK, I concede your point (and indeed I only tested this on 3.6). If we
>>> could break the backward compatibility for now() we presumably can break it
>>> for this purpose.
>>>
>>> On Sun, Jan 6, 2019 at 11:02 AM Paul Ganssle <paul at ganssle.io> wrote:
>>>
>>>> I did address this in the original post - the assumption that the
>>>> subclass constructor will have the same arguments as the base constructor
>>>> is baked into many alternate constructors of datetime. I acknowledge that
>>>> this is a breaking change, but it is a small one - anyone creating such a
>>>> subclass that *cannot* handled the class being created this way would
>>>> be broken in myriad ways.
>>>>
>>>> We have also in recent years changed several alternate constructors
>>>> (including `replace`) to retain the original subclass, which by your same
>>>> standard would be a breaking change. I believe there have been no
>>>> complaints. In fact, between Python 3.6 and 3.7, the very example you
>>>> showed broke:
>>>>
>>>> Python 3.6.6:
>>>>
>>>> >>> class D(datetime.datetime):
>>>> ...     def __new__(cls):
>>>> ...         return cls.now()
>>>> ...
>>>> >>> D()
>>>> D(2019, 1, 6, 13, 49, 38, 842033)
>>>>
>>>> Python 3.7.2:
>>>>
>>>> >>> class D(datetime.datetime):
>>>> ...     def __new__(cls):
>>>> ...         return cls.now()
>>>> ...
>>>> >>> D()
>>>> Traceback (most recent call last):
>>>>   File "<stdin>", line 1, in <module>
>>>>   File "<stdin>", line 3, in __new__
>>>> TypeError: __new__() takes 1 positional argument but 9 were given
>>>>
>>>>
>>>> We haven't seen any bug reports about this sort of thing; what we
>>>> *have* been getting is bug reports that subclassing datetime doesn't
>>>> retain the subclass in various ways (because people *are* using
>>>> datetime subclasses). This is likely to cause very little in the way of
>>>> problems, but it will improve convenience for people making datetime
>>>> subclasses and almost certainly performance for people using them (e.g.
>>>> pendulum and arrow, which now need to take a slow pure python route in many
>>>> situations to work around this problem).
>>>>
>>>> If we're *really* concerned with this backward compatibility breaking,
>>>> we could do the equivalent of:
>>>>
>>>> try:
>>>>     return new_behavior(...)
>>>> except TypeError:
>>>>     warnings.warn("The semantics of timedelta addition have "
>>>>                   "changed in a way that raises an error in "
>>>>                   "this subclass. Please implement __add__ "
>>>>                   "if you need the old behavior.", DeprecationWarning)
>>>>
>>>> Then after a suitable notice period drop the warning and turn it to a
>>>> hard error.
>>>>
>>>> Best,
>>>>
>>>> Paul
>>>> On 1/6/19 1:43 PM, Guido van Rossum wrote:
>>>>
>>>> I don't think datetime and builtins like int necessarily need to be
>>>> aligned. But I do see a problem -- the __new__ and __init__ methods defined
>>>> in the subclass (if any) should allow for being called with the same
>>>> signature as the base datetime class. Currently you can have a subclass of
>>>> datetime whose __new__ has no arguments (or, more realistically, interprets
>>>> its arguments differently). Instances of such a class can still be added to
>>>> a timedelta. The proposal would cause this to break (since such an addition
>>>> has to create a new instance, which calls __new__ and __init__). Since this
>>>> is a backwards incompatibility, I don't see how it can be done -- and I
>>>> also don't see many use cases, so I think it's not worth pursuing further.
>>>>
>>>> Note that the same problem already happens with the .fromordinal()
>>>> class method, though it doesn't happen with .fromdatetime() or .now():
>>>>
>>>> >>> class D(datetime.datetime):
>>>> ...   def __new__(cls): return cls.now()
>>>> ...
>>>> >>> D()
>>>> D(2019, 1, 6, 10, 33, 37, 161606)
>>>> >>> D.fromordinal(100)
>>>> Traceback (most recent call last):
>>>>   File "<stdin>", line 1, in <module>
>>>> TypeError: __new__() takes 1 positional argument but 4 were given
>>>> >>> D.fromtimestamp(123456789)
>>>> D(1973, 11, 29, 13, 33, 9)
>>>> >>>
>>>>
>>>> On Sun, Jan 6, 2019 at 9:05 AM Paul Ganssle <paul at ganssle.io> wrote:
>>>>
>>>>> I can think of many reasons why datetime is different from builtins,
>>>>> though to be honest I'm not sure that consistency for its own sake is
>>>>> really a strong argument for keeping a counter-intuitive behavior - and to
>>>>> be honest I'm open to the idea that *all* arithmetic types *should*
>>>>> have some form of this change.
>>>>>
>>>>> That said, I would say that the biggest difference between datetime
>>>>> and builtins (other than the fact that datetime is *not* a builtin,
>>>>> and as such doesn't necessarily need to be categorized in this group), is
>>>>> that unlike almost all other arithmetic types, *datetime* has a
>>>>> special, dedicated type for describing differences in datetimes. Using your
>>>>> example of a float subclass, consider that without the behavior of
>>>>> "addition of floats returns floats", it would be hard to predict what would
>>>>> happen in this situation:
>>>>>
>>>>> >>> F(1.2) + 3.4
>>>>>
>>>>> Would that always return a float, even though F(1.2) + F(3.4) returns
>>>>> an F? Would that return an F because F is the left-hand operand? Would it
>>>>> return a float because float is the right-hand operand? Would you walk the
>>>>> MROs and find the lowest type in common between the operands and return
>>>>> that? It's not entirely clear which subtype predominates. With datetime,
>>>>> you have:
>>>>>
>>>>> datetime - datetime -> timedelta
>>>>> datetime ± timedelta -> datetime
>>>>> timedelta ± timedelta -> timedelta
>>>>>
>>>>> There's no operation between two datetime objects that would return a
>>>>> datetime object, so it's always clear: operations between datetime
>>>>> subclasses return timedelta, operations between a datetime object and a
>>>>> timedelta return the subclass of the datetime that it was added to or
>>>>> subtracted from.
>>>>>
>>>>> Of course, the real way to resolve whether datetime should be
>>>>> different from int/float/string/etc is to look at why this choice was
>>>>> actually made for those types in the first place, and decide whether
>>>>> datetime is like them *in this respect*. The heterogeneous operations
>>>>> problem may be a reasonable justification for leaving the other builtins
>>>>> alone but changing datetime, but if someone knows of other fundamental
>>>>> reasons why the decision to have arithmetic operations always create the
>>>>> base class was chosen, please let me know.
>>>>>
>>>>> Best,
>>>>> Paul
>>>>> On 1/5/19 3:55 AM, Alexander Belopolsky wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jan 2, 2019 at 10:18 PM Paul Ganssle <paul at ganssle.io> wrote:
>>>>>
>>>>>> .. the original objection was that this implementation assumes that
>>>>>> the datetime subclass has a constructor with the same (or a sufficiently
>>>>>> similar) signature as datetime.
>>>>>>
>>>>> While this was used as a possible rationale for the way standard types
>>>>> behave, the main objection to changing datetime classes is that it will
>>>>> make them behave differently from builtins.  For example:
>>>>>
>>>>> >>> class F(float):
>>>>> ...     pass
>>>>> ...
>>>>> >>> type(F.fromhex('AA'))
>>>>> <class '__main__.F'>
>>>>> >>> type(F(1) + F(2))
>>>>> <class 'float'>
>>>>>
>>>>> This may be a legitimate gripe, but unfortunately that ship has sailed
>>>>>> long ago. All of datetime's alternate constructors make this assumption.
>>>>>> Any subclass that does not meet this requirement must have worked around it
>>>>>> long ago (or they don't care about alternate constructors).
>>>>>>
>>>>>
>>>>> This is right, but the same argument is equally applicable to int,
>>>>> float, etc. subclasses.  If you want to limit your change to datetime types
>>>>> you should explain what makes these types special.
>>>>>
>>>>> _______________________________________________
>>>>> Python-Dev mailing list
>>>>> Python-Dev at python.org
>>>>> https://mail.python.org/mailman/listinfo/python-dev
>>>>> Unsubscribe:
>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>>>
>>>>
>>>>
>>>> --
>>>> --Guido van Rossum (python.org/~guido)
>>>>
>>>> _______________________________________________
>>>> Python-Dev mailing list
>>>> Python-Dev at python.org
>>>> https://mail.python.org/mailman/listinfo/python-dev
>>>> Unsubscribe:
>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>>
>>>
>>>
>>> --
>>> --Guido van Rossum (python.org/~guido)
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/alexander.belopolsky%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190204/d76f0151/attachment-0001.html>

More information about the Python-Dev mailing list