[Python-Dev] Return type of datetime subclasses added to timedelta
Paul Ganssle
paul at ganssle.io
Sun Jan 6 15:50:32 EST 2019
Brett,
Thank you for bringing this up, but I think you /may/ have misunderstood
my position - though maybe you understood the thrust and wanted to
clarify for people coming in halfway, which I applaud.
I proposed this change /knowing/ that it was a breaking change - it's
why I brought it to the attention of datetime-SIG and now python-dev -
and I believe that there are several factors that lead this to being a
smaller compatibility problem than it seems.
One such factor is the fact that /many/ other features of `datetime`,
including the implementation of `datetime.now()` are /already broken/ in
the current implementation for anyone who would be broken by this
particular aspect of the semantic change. That is not saying that it's
impossible that there is code out there that will break if this change
goes through, it's just saying that the scope of the breakage is
necessarily very limited.
The reason I brought up the bug tracker is because between Python 3.6
and Python 3.7, we in fact made a similar breaking change to the one I'm
proposing here without thinking that anyone might be relying on the fact
that they could do something like:
class D(datetime.datetime):
def __new__(cls):
return cls.now()
My point was that there have been no bug reports about the /existing
change/ that Guido was bringing up (his example itself does not work on
Python 3.7!), which leads me to believe that few if any people are
relying on the fact that it is possible to define a datetime subclass
with a different default constructor.
As I mentioned, it is likely possible to have a transition period where
this would still work even if the subclassers have not created their own
__add__ method.
There is no way to create a similar deprecation/transition period for
people relying on the fact that `type(datetime_obj + timedelta_obj) ==
datetime.datetime`, but I think this is honestly a sufficiently minor
breakage that the good outweighs the harm. I will note that we have
already made several such changes with respect to alternate constructors
even though technically someone could have been relying on the fact that
`MyDateTime(*args).replace(month=3)` returns a `datetime` object.
This is not to say that we should lightly make the change (hence my
canvassing for opinions), it is just that there is a good amount of
evidence that, practically speaking, no one is relying on this, and in
fact it is likely that people are writing code that assumes that adding
`timedelta` to a datetime subclass returns the original subclass, either
directly or indirectly - I think we're likely to fix more people than we
break if we make this change.
Best,
Paul
On 1/6/19 3:24 PM, Brett Cannon wrote:
>
>
> On Sun, 6 Jan 2019 at 11:00, Paul Ganssle <paul at ganssle.io
> <mailto:paul at ganssle.io>> wrote:
>
> I did address this in the original post - the assumption that the
> subclass constructor will have the same arguments as the base
> constructor is baked into many alternate constructors of datetime.
> I acknowledge that this is a breaking change, but it is a small
> one - anyone creating such a subclass that /cannot/ handled the
> class being created this way would be broken in myriad ways.
>
> We have also in recent years changed several alternate
> constructors (including `replace`) to retain the original
> subclass, which by your same standard would be a breaking change.
> I believe there have been no complaints. In fact, between Python
> 3.6 and 3.7, the very example you showed broke:
>
> Python 3.6.6:
>
> >>> class D(datetime.datetime):
> ... def __new__(cls):
> ... return cls.now()
> ...
> >>> D()
> D(2019, 1, 6, 13, 49, 38, 842033)
>
> Python 3.7.2:
>
> >>> class D(datetime.datetime):
> ... def __new__(cls):
> ... return cls.now()
> ...
> >>> D()
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "<stdin>", line 3, in __new__
> TypeError: __new__() takes 1 positional argument but 9 were given
>
>
> We haven't seen any bug reports about this sort of thing; what we
> /have/ been getting is bug reports that subclassing datetime
> doesn't retain the subclass in various ways (because people /are/
> using datetime subclasses).
>
>
> To help set expectations, the current semantics are not a bug and so
> the proposal isn't fixing a bug but proposing a change in semantics.
>
>
> This is likely to cause very little in the way of problems, but it
> will improve convenience for people making datetime subclasses and
> almost certainly performance for people using them (e.g. pendulum
> and arrow, which now need to take a slow pure python route in many
> situations to work around this problem).
>
> If we're /really/ concerned with this backward compatibility
> breaking,
>
>
> We very much do care. Because this isn't a bug but a voluntary
> semantic change you're proposing to change we can't blindly break
> people who are relying on the current semantics. We need to have a
> justification for those people as to why we have decided to change the
> semantics now after all of these years as well as provide an upgrade path.
>
> -Brett
>
>
> we could do the equivalent of:
>
> try:
> return new_behavior(...)
> except TypeError:
> warnings.warn("The semantics of timedelta addition have "
> "changed in a way that raises an error in "
> "this subclass. Please implement __add__ "
> "if you need the old behavior.",
> DeprecationWarning)
>
>
> Then after a suitable notice period drop the warning and turn it
> to a hard error.
>
> Best,
>
> Paul
>
> On 1/6/19 1:43 PM, Guido van Rossum wrote:
>> I don't think datetime and builtins like int necessarily need to
>> be aligned. But I do see a problem -- the __new__ and __init__
>> methods defined in the subclass (if any) should allow for being
>> called with the same signature as the base datetime class.
>> Currently you can have a subclass of datetime whose __new__ has
>> no arguments (or, more realistically, interprets its arguments
>> differently). Instances of such a class can still be added to a
>> timedelta. The proposal would cause this to break (since such an
>> addition has to create a new instance, which calls __new__ and
>> __init__). Since this is a backwards incompatibility, I don't see
>> how it can be done -- and I also don't see many use cases, so I
>> think it's not worth pursuing further.
>>
>> Note that the same problem already happens with the
>> .fromordinal() class method, though it doesn't happen with
>> .fromdatetime() or .now():
>>
>> >>> class D(datetime.datetime):
>> ... def __new__(cls): return cls.now()
>> ...
>> >>> D()
>> D(2019, 1, 6, 10, 33, 37, 161606)
>> >>> D.fromordinal(100)
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in <module>
>> TypeError: __new__() takes 1 positional argument but 4 were given
>> >>> D.fromtimestamp(123456789)
>> D(1973, 11, 29, 13, 33, 9)
>> >>>
>>
>> On Sun, Jan 6, 2019 at 9:05 AM Paul Ganssle <paul at ganssle.io
>> <mailto:paul at ganssle.io>> wrote:
>>
>> I can think of many reasons why datetime is different from
>> builtins, though to be honest I'm not sure that consistency
>> for its own sake is really a strong argument for keeping a
>> counter-intuitive behavior - and to be honest I'm open to the
>> idea that /all/ arithmetic types /should/ have some form of
>> this change.
>>
>> That said, I would say that the biggest difference between
>> datetime and builtins (other than the fact that datetime is
>> /not/ a builtin, and as such doesn't necessarily need to be
>> categorized in this group), is that unlike almost all other
>> arithmetic types, /datetime/ has a special, dedicated type
>> for describing differences in datetimes. Using your example
>> of a float subclass, consider that without the behavior of
>> "addition of floats returns floats", it would be hard to
>> predict what would happen in this situation:
>>
>> >>> F(1.2) + 3.4
>>
>> Would that always return a float, even though F(1.2) + F(3.4)
>> returns an F? Would that return an F because F is the
>> left-hand operand? Would it return a float because float is
>> the right-hand operand? Would you walk the MROs and find the
>> lowest type in common between the operands and return that?
>> It's not entirely clear which subtype predominates. With
>> datetime, you have:
>>
>> datetime - datetime -> timedelta
>> datetime ± timedelta -> datetime
>> timedelta ± timedelta -> timedelta
>>
>> There's no operation between two datetime objects that would
>> return a datetime object, so it's always clear: operations
>> between datetime subclasses return timedelta, operations
>> between a datetime object and a timedelta return the subclass
>> of the datetime that it was added to or subtracted from.
>>
>> Of course, the real way to resolve whether datetime should be
>> different from int/float/string/etc is to look at why this
>> choice was actually made for those types in the first place,
>> and decide whether datetime is like them /in this respect/.
>> The heterogeneous operations problem may be a reasonable
>> justification for leaving the other builtins alone but
>> changing datetime, but if someone knows of other fundamental
>> reasons why the decision to have arithmetic operations always
>> create the base class was chosen, please let me know.
>>
>> Best,
>> Paul
>>
>> On 1/5/19 3:55 AM, Alexander Belopolsky wrote:
>>>
>>>
>>> On Wed, Jan 2, 2019 at 10:18 PM Paul Ganssle
>>> <paul at ganssle.io <mailto:paul at ganssle.io>> wrote:
>>>
>>> .. the original objection was that this implementation
>>> assumes that the datetime subclass has a constructor
>>> with the same (or a sufficiently similar) signature as
>>> datetime.
>>>
>>> While this was used as a possible rationale for the way
>>> standard types behave, the main objection to changing
>>> datetime classes is that it will make them behave
>>> differently from builtins. For example:
>>>
>>> >>> class F(float):
>>> ... pass
>>> ...
>>> >>> type(F.fromhex('AA'))
>>> <class '__main__.F'>
>>> >>> type(F(1) + F(2))
>>> <class 'float'>
>>>
>>> This may be a legitimate gripe, but unfortunately that
>>> ship has sailed long ago. All of datetime's alternate
>>> constructors make this assumption. Any subclass that
>>> does not meet this requirement must have worked around
>>> it long ago (or they don't care about alternate
>>> constructors).
>>>
>>>
>>> This is right, but the same argument is equally applicable
>>> to int, float, etc. subclasses. If you want to limit your
>>> change to datetime types you should explain what makes these
>>> types special.
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org <mailto:Python-Dev at python.org>
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <mailto:Python-Dev at python.org>
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190106/3463db7e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20190106/3463db7e/attachment-0001.sig>
More information about the Python-Dev
mailing list