Brett, Thank you for bringing this up, but I think you /may/ have misunderstood my position - though maybe you understood the thrust and wanted to clarify for people coming in halfway, which I applaud. I proposed this change /knowing/ that it was a breaking change - it's why I brought it to the attention of datetime-SIG and now python-dev - and I believe that there are several factors that lead this to being a smaller compatibility problem than it seems. One such factor is the fact that /many/ other features of `datetime`, including the implementation of `datetime.now()` are /already broken/ in the current implementation for anyone who would be broken by this particular aspect of the semantic change. That is not saying that it's impossible that there is code out there that will break if this change goes through, it's just saying that the scope of the breakage is necessarily very limited. The reason I brought up the bug tracker is because between Python 3.6 and Python 3.7, we in fact made a similar breaking change to the one I'm proposing here without thinking that anyone might be relying on the fact that they could do something like: class D(datetime.datetime): def __new__(cls): return cls.now() My point was that there have been no bug reports about the /existing change/ that Guido was bringing up (his example itself does not work on Python 3.7!), which leads me to believe that few if any people are relying on the fact that it is possible to define a datetime subclass with a different default constructor. As I mentioned, it is likely possible to have a transition period where this would still work even if the subclassers have not created their own __add__ method. There is no way to create a similar deprecation/transition period for people relying on the fact that `type(datetime_obj + timedelta_obj) == datetime.datetime`, but I think this is honestly a sufficiently minor breakage that the good outweighs the harm. I will note that we have already made several such changes with respect to alternate constructors even though technically someone could have been relying on the fact that `MyDateTime(*args).replace(month=3)` returns a `datetime` object. This is not to say that we should lightly make the change (hence my canvassing for opinions), it is just that there is a good amount of evidence that, practically speaking, no one is relying on this, and in fact it is likely that people are writing code that assumes that adding `timedelta` to a datetime subclass returns the original subclass, either directly or indirectly - I think we're likely to fix more people than we break if we make this change. Best, Paul On 1/6/19 3:24 PM, Brett Cannon wrote:
On Sun, 6 Jan 2019 at 11:00, Paul Ganssle <paul@ganssle.io <mailto:paul@ganssle.io>> wrote:
I did address this in the original post - the assumption that the subclass constructor will have the same arguments as the base constructor is baked into many alternate constructors of datetime. I acknowledge that this is a breaking change, but it is a small one - anyone creating such a subclass that /cannot/ handled the class being created this way would be broken in myriad ways.
We have also in recent years changed several alternate constructors (including `replace`) to retain the original subclass, which by your same standard would be a breaking change. I believe there have been no complaints. In fact, between Python 3.6 and 3.7, the very example you showed broke:
Python 3.6.6:
>>> class D(datetime.datetime): ... def __new__(cls): ... return cls.now() ... >>> D() D(2019, 1, 6, 13, 49, 38, 842033)
Python 3.7.2:
>>> class D(datetime.datetime): ... def __new__(cls): ... return cls.now() ... >>> D() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in __new__ TypeError: __new__() takes 1 positional argument but 9 were given
We haven't seen any bug reports about this sort of thing; what we /have/ been getting is bug reports that subclassing datetime doesn't retain the subclass in various ways (because people /are/ using datetime subclasses).
To help set expectations, the current semantics are not a bug and so the proposal isn't fixing a bug but proposing a change in semantics.
This is likely to cause very little in the way of problems, but it will improve convenience for people making datetime subclasses and almost certainly performance for people using them (e.g. pendulum and arrow, which now need to take a slow pure python route in many situations to work around this problem).
If we're /really/ concerned with this backward compatibility breaking,
We very much do care. Because this isn't a bug but a voluntary semantic change you're proposing to change we can't blindly break people who are relying on the current semantics. We need to have a justification for those people as to why we have decided to change the semantics now after all of these years as well as provide an upgrade path.
-Brett
we could do the equivalent of:
try: return new_behavior(...) except TypeError: warnings.warn("The semantics of timedelta addition have " "changed in a way that raises an error in " "this subclass. Please implement __add__ " "if you need the old behavior.", DeprecationWarning)
Then after a suitable notice period drop the warning and turn it to a hard error.
Best,
Paul
On 1/6/19 1:43 PM, Guido van Rossum wrote:
I don't think datetime and builtins like int necessarily need to be aligned. But I do see a problem -- the __new__ and __init__ methods defined in the subclass (if any) should allow for being called with the same signature as the base datetime class. Currently you can have a subclass of datetime whose __new__ has no arguments (or, more realistically, interprets its arguments differently). Instances of such a class can still be added to a timedelta. The proposal would cause this to break (since such an addition has to create a new instance, which calls __new__ and __init__). Since this is a backwards incompatibility, I don't see how it can be done -- and I also don't see many use cases, so I think it's not worth pursuing further.
Note that the same problem already happens with the .fromordinal() class method, though it doesn't happen with .fromdatetime() or .now():
>>> class D(datetime.datetime): ... def __new__(cls): return cls.now() ... >>> D() D(2019, 1, 6, 10, 33, 37, 161606) >>> D.fromordinal(100) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: __new__() takes 1 positional argument but 4 were given >>> D.fromtimestamp(123456789) D(1973, 11, 29, 13, 33, 9) >>>
On Sun, Jan 6, 2019 at 9:05 AM Paul Ganssle <paul@ganssle.io <mailto:paul@ganssle.io>> wrote:
I can think of many reasons why datetime is different from builtins, though to be honest I'm not sure that consistency for its own sake is really a strong argument for keeping a counter-intuitive behavior - and to be honest I'm open to the idea that /all/ arithmetic types /should/ have some form of this change.
That said, I would say that the biggest difference between datetime and builtins (other than the fact that datetime is /not/ a builtin, and as such doesn't necessarily need to be categorized in this group), is that unlike almost all other arithmetic types, /datetime/ has a special, dedicated type for describing differences in datetimes. Using your example of a float subclass, consider that without the behavior of "addition of floats returns floats", it would be hard to predict what would happen in this situation:
>>> F(1.2) + 3.4
Would that always return a float, even though F(1.2) + F(3.4) returns an F? Would that return an F because F is the left-hand operand? Would it return a float because float is the right-hand operand? Would you walk the MROs and find the lowest type in common between the operands and return that? It's not entirely clear which subtype predominates. With datetime, you have:
datetime - datetime -> timedelta datetime ± timedelta -> datetime timedelta ± timedelta -> timedelta
There's no operation between two datetime objects that would return a datetime object, so it's always clear: operations between datetime subclasses return timedelta, operations between a datetime object and a timedelta return the subclass of the datetime that it was added to or subtracted from.
Of course, the real way to resolve whether datetime should be different from int/float/string/etc is to look at why this choice was actually made for those types in the first place, and decide whether datetime is like them /in this respect/. The heterogeneous operations problem may be a reasonable justification for leaving the other builtins alone but changing datetime, but if someone knows of other fundamental reasons why the decision to have arithmetic operations always create the base class was chosen, please let me know.
Best, Paul
On 1/5/19 3:55 AM, Alexander Belopolsky wrote:
On Wed, Jan 2, 2019 at 10:18 PM Paul Ganssle <paul@ganssle.io <mailto:paul@ganssle.io>> wrote:
.. the original objection was that this implementation assumes that the datetime subclass has a constructor with the same (or a sufficiently similar) signature as datetime.
While this was used as a possible rationale for the way standard types behave, the main objection to changing datetime classes is that it will make them behave differently from builtins. For example:
>>> class F(float): ... pass ... >>> type(F.fromhex('AA')) <class '__main__.F'> >>> type(F(1) + F(2)) <class 'float'>
This may be a legitimate gripe, but unfortunately that ship has sailed long ago. All of datetime's alternate constructors make this assumption. Any subclass that does not meet this requirement must have worked around it long ago (or they don't care about alternate constructors).
This is right, but the same argument is equally applicable to int, float, etc. subclasses. If you want to limit your change to datetime types you should explain what makes these types special.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org <mailto:Python-Dev@python.org> https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
_______________________________________________ Python-Dev mailing list Python-Dev@python.org <mailto:Python-Dev@python.org> https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org