[Python-Dev] Return type of datetime subclasses added to timedelta

Guido van Rossum guido at python.org
Mon Feb 4 11:38:34 EST 2019 

I recommend that you submit a PR so we can get it into 3.8 alpha 2.

On Mon, Feb 4, 2019 at 5:50 AM Paul Ganssle <paul at ganssle.io> wrote:

> Hey all,
>
> This thread about the return type of datetime operations seems to have
> stopped without any explicit decision - I think I responded to everyone who
> had objections, but I think only Guido has given a +1 to whether or not we
> should go ahead.
>
> Have we got agreement to go ahead with this change? Are we still targeting
> Python 3.8 here?
>
> For those who don't want to dig through your old e-mails, here's the
> archive link for this thread:
> https://mail.python.org/pipermail/python-dev/2019-January/155984.html
>
> If you want to start commenting on the actual implementation, it's
> available here (though it's pretty simple):
> https://github.com/python/cpython/pull/10902
>
> Best,
>
> Paul
>
>
> On 1/6/19 7:17 PM, Guido van Rossum wrote:
>
> OK, I concede your point (and indeed I only tested this on 3.6). If we
> could break the backward compatibility for now() we presumably can break it
> for this purpose.
>
> On Sun, Jan 6, 2019 at 11:02 AM Paul Ganssle <paul at ganssle.io> wrote:
>
>> I did address this in the original post - the assumption that the
>> subclass constructor will have the same arguments as the base constructor
>> is baked into many alternate constructors of datetime. I acknowledge that
>> this is a breaking change, but it is a small one - anyone creating such a
>> subclass that *cannot* handled the class being created this way would be
>> broken in myriad ways.
>>
>> We have also in recent years changed several alternate constructors
>> (including `replace`) to retain the original subclass, which by your same
>> standard would be a breaking change. I believe there have been no
>> complaints. In fact, between Python 3.6 and 3.7, the very example you
>> showed broke:
>>
>> Python 3.6.6:
>>
>> >>> class D(datetime.datetime):
>> ...     def __new__(cls):
>> ...         return cls.now()
>> ...
>> >>> D()
>> D(2019, 1, 6, 13, 49, 38, 842033)
>>
>> Python 3.7.2:
>>
>> >>> class D(datetime.datetime):
>> ...     def __new__(cls):
>> ...         return cls.now()
>> ...
>> >>> D()
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "<stdin>", line 3, in __new__
>> TypeError: __new__() takes 1 positional argument but 9 were given
>>
>>
>> We haven't seen any bug reports about this sort of thing; what we *have*
>> been getting is bug reports that subclassing datetime doesn't retain the
>> subclass in various ways (because people *are* using datetime
>> subclasses). This is likely to cause very little in the way of problems,
>> but it will improve convenience for people making datetime subclasses and
>> almost certainly performance for people using them (e.g. pendulum and
>> arrow, which now need to take a slow pure python route in many situations
>> to work around this problem).
>>
>> If we're *really* concerned with this backward compatibility breaking,
>> we could do the equivalent of:
>>
>> try:
>>     return new_behavior(...)
>> except TypeError:
>>     warnings.warn("The semantics of timedelta addition have "
>>                   "changed in a way that raises an error in "
>>                   "this subclass. Please implement __add__ "
>>                   "if you need the old behavior.", DeprecationWarning)
>>
>> Then after a suitable notice period drop the warning and turn it to a
>> hard error.
>>
>> Best,
>>
>> Paul
>> On 1/6/19 1:43 PM, Guido van Rossum wrote:
>>
>> I don't think datetime and builtins like int necessarily need to be
>> aligned. But I do see a problem -- the __new__ and __init__ methods defined
>> in the subclass (if any) should allow for being called with the same
>> signature as the base datetime class. Currently you can have a subclass of
>> datetime whose __new__ has no arguments (or, more realistically, interprets
>> its arguments differently). Instances of such a class can still be added to
>> a timedelta. The proposal would cause this to break (since such an addition
>> has to create a new instance, which calls __new__ and __init__). Since this
>> is a backwards incompatibility, I don't see how it can be done -- and I
>> also don't see many use cases, so I think it's not worth pursuing further.
>>
>> Note that the same problem already happens with the .fromordinal() class
>> method, though it doesn't happen with .fromdatetime() or .now():
>>
>> >>> class D(datetime.datetime):
>> ...   def __new__(cls): return cls.now()
>> ...
>> >>> D()
>> D(2019, 1, 6, 10, 33, 37, 161606)
>> >>> D.fromordinal(100)
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>> TypeError: __new__() takes 1 positional argument but 4 were given
>> >>> D.fromtimestamp(123456789)
>> D(1973, 11, 29, 13, 33, 9)
>> >>>
>>
>> On Sun, Jan 6, 2019 at 9:05 AM Paul Ganssle <paul at ganssle.io> wrote:
>>
>>> I can think of many reasons why datetime is different from builtins,
>>> though to be honest I'm not sure that consistency for its own sake is
>>> really a strong argument for keeping a counter-intuitive behavior - and to
>>> be honest I'm open to the idea that *all* arithmetic types *should*
>>> have some form of this change.
>>>
>>> That said, I would say that the biggest difference between datetime and
>>> builtins (other than the fact that datetime is *not* a builtin, and as
>>> such doesn't necessarily need to be categorized in this group), is that
>>> unlike almost all other arithmetic types, *datetime* has a special,
>>> dedicated type for describing differences in datetimes. Using your example
>>> of a float subclass, consider that without the behavior of "addition of
>>> floats returns floats", it would be hard to predict what would happen in
>>> this situation:
>>>
>>> >>> F(1.2) + 3.4
>>>
>>> Would that always return a float, even though F(1.2) + F(3.4) returns an
>>> F? Would that return an F because F is the left-hand operand? Would it
>>> return a float because float is the right-hand operand? Would you walk the
>>> MROs and find the lowest type in common between the operands and return
>>> that? It's not entirely clear which subtype predominates. With datetime,
>>> you have:
>>>
>>> datetime - datetime -> timedelta
>>> datetime ± timedelta -> datetime
>>> timedelta ± timedelta -> timedelta
>>>
>>> There's no operation between two datetime objects that would return a
>>> datetime object, so it's always clear: operations between datetime
>>> subclasses return timedelta, operations between a datetime object and a
>>> timedelta return the subclass of the datetime that it was added to or
>>> subtracted from.
>>>
>>> Of course, the real way to resolve whether datetime should be different
>>> from int/float/string/etc is to look at why this choice was actually made
>>> for those types in the first place, and decide whether datetime is like
>>> them *in this respect*. The heterogeneous operations problem may be a
>>> reasonable justification for leaving the other builtins alone but changing
>>> datetime, but if someone knows of other fundamental reasons why the
>>> decision to have arithmetic operations always create the base class was
>>> chosen, please let me know.
>>>
>>> Best,
>>> Paul
>>> On 1/5/19 3:55 AM, Alexander Belopolsky wrote:
>>>
>>>
>>>
>>> On Wed, Jan 2, 2019 at 10:18 PM Paul Ganssle <paul at ganssle.io> wrote:
>>>
>>>> .. the original objection was that this implementation assumes that the
>>>> datetime subclass has a constructor with the same (or a sufficiently
>>>> similar) signature as datetime.
>>>>
>>> While this was used as a possible rationale for the way standard types
>>> behave, the main objection to changing datetime classes is that it will
>>> make them behave differently from builtins.  For example:
>>>
>>> >>> class F(float):
>>> ...     pass
>>> ...
>>> >>> type(F.fromhex('AA'))
>>> <class '__main__.F'>
>>> >>> type(F(1) + F(2))
>>> <class 'float'>
>>>
>>> This may be a legitimate gripe, but unfortunately that ship has sailed
>>>> long ago. All of datetime's alternate constructors make this assumption.
>>>> Any subclass that does not meet this requirement must have worked around it
>>>> long ago (or they don't care about alternate constructors).
>>>>
>>>
>>> This is right, but the same argument is equally applicable to int,
>>> float, etc. subclasses.  If you want to limit your change to datetime types
>>> you should explain what makes these types special.
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190204/c0888c88/attachment-0001.html>

More information about the Python-Dev mailing list