[Datetime-SIG] Timeline arithmetic?

[Tim Peters]

[Tim, on hats]
>> ...

[Carl]
> I don't expect consistency from humans, it's just that my hat-intuiter
> doesn't always work right :-)

Nor my hat-signaler!


[Carl]
>>> All else being equal, designing a green-field datetime library,
>>> "universally recognized best practice" does not provide any argument for
>>> naive arithmetic over aware arithmetic on aware datetimes. Making the
>>> choice to implement aware arithmetic is not "fighting" a best practice,
>>> it's just providing a reasonable and fully consistent convenience for
>>> simple cases.

>> It would create an "attractive nuisance", yes ;-)

> I think that either choice of arithmetic might be an attractive
> nuisance; what matters is consistency with the rest of the choices in
> the library.

I went on to explain why the specific case of default timeline
arithmetic is an "attractive nuisance":  making it dead easy to spell
a poor practice.  That remains poor practice forever after.  "Easy to
spell" makes it attractive.  "Poor practice forever after" makes it a
nuisance.

Classic arithmetic is equivalent to doing integer arithmetic on
integer POSIX timestamps (although with wider range the same across
all platforms, and extended to microsecond precision).  That's hardly
novel - there's a deep and long history of doing exactly that in the
Unix(tm) world.  Which is Guido's world.  There "shouldn't be"
anything controversial about that.  The direct predecessor was already
best practice in its world.  How that could be considered a nuisance
seems a real strain to me.

Where it gets muddy is extending classic arithmetic to aware datetimes
too.  Then compounding the conceptual confusion by adding timeline
interzone subtraction and comparison.


> If datetime did naive arithmetic on tz-annotated datetimes, and also
> refused to ever implicitly convert them to UTC for purposes of
> cross-timezone comparison or arithmetic, and included a `fold` parameter
> not on the datetime object itself but only as an additional input
> argument when you explicitly convert from some other timezone to UTC,
> that would be a consistent view of the meaning of a tz-annotated
> datetime, and I wouldn't have any problem with that.

I would.  Pure or not, it sounds unusable:  when I convert _from_ UTC
to a local zone, I have no idea whether I'll end up in a gap, a fold,
or neither.  And so I'll have no idea either what to pass _to_
.utcoffset() when I need to convert back to UTC.  It doesn't solve the
conversion problem.  It's a do-it-yourself kit missing the most
important piece.  "But .fromutc() could return the right flag to pass
back later" isn't attractive either.  Then the user ends up needing to
maintain their own (datetime, convert_back_flag) pairs.  In which
case, why not just store the flag _in_ the datetime?  Only tzinfo
methods would ever need to look at it.

But note it's still not theoretically ideal:  it would mean timezone
conversion is not a wholly order-preserving function in all cases..
I'd much rather be drinking that poison, though :-(


> It would be a view consistent with what Guido described a few days ago,
> that "noon Eastern on June 3 2020" is not necessarily equivalent to a
> UTC instant; it means nothing more than "noon Eastern on June 3 2020"

If it wasn't obvious, "noon Eastern on June 3 2020" _is_ a "naive
time" in Guido's head.  One that will eventually become a civil time,
but not before civil time gets close to 2020.


> until you choose to explicitly convert it to UTC, providing a full
> zoneinfo definition of "Eastern" (and possibly a `fold` argument too,
> though it's not needed for "noon Eastern June 3 2020" unless something
> changes) at that moment.
>
> But that isn't datetime's view, at least not consistently. The problem
> isn't datetime's choice of arithmetic; it's just that sometimes it wants
> to treat a tz-annotated datetime as one thing, and sometimes as another.

How many times do we need to agree on this? ;-)   Although the
conceptual fog has not really been an impediment to using the module
in my experience.

In yours?  Do you use datetime?  If so, do you trip over this?


> (The fact that a _person_ might also want to have one sometimes and
> another sometimes is not a reason for an implementation to try to guess
> when they want one and when they want another. It could be a reason for
> two different types.)

Or three, or four, or ... but, in practice, one type has worked OK for
me.  Guido's "noon Eastern on June 3 2020" won't actually create any
problems for him either.


>> There is no argument that can possibly succeed for changing arithmetic
>> on aware datetimes:  "Tim as Python developer hat" there.  That would
>> be massively backward-incompatible.  No chance whatsoever.

> Of course! That's abundantly clear, and I'd be every bit as opposed as
> you are to a backwards-incompatible change. Can we just assume that if I
> refer to "changing arithmetic" it's short-hand for "provide an option
> for full consistency in a way that only occurs with an opt-in choice by
> the user, leaving existing code behaving identically."
>
> The latter is the only thing I've ever proposed, so your choice to
> assume here that I meant the former feels a bit like an intentional
> misunderstanding so as to provide an opportunity for unnecessary
> hyperbole. Or maybe your intuiter is just fallible too ;-)

You missed that I had my jester hat on ;-)  That was intended to be
comic relief, a dogmatic & rigid over-the-top rant from "a Python
developer".  It's a shame that you chopped part of it, because the
fragment that remains doesn't do it full justice.  Next time I'll try
to sound even more insanely enraged ;-)


> ...
> "What is the root issue" and "is the root issue practically worth fixing
> today" are separable questions. I'm still trying to figure out the
> former (but I think we're finally getting there); I'm not at all sure
> what I think of the latter (and won't be until I try an implementation).

I think the root problem is that "civil time" is a frickin' mess.  If
you want purity on all counts, then you need an object that solely
represents civil time, even to the extent of _requiring_ a non-None,
fully functional tzinfo.  Else you're leaving "but _whose_ civil
time?" ambiguous, and your object no longer represents a single
instant in UTC, and you can only possibly support classic arithmetic
(if you support any arithmetic at all).  But so much baggage is
required to specify one of those, lots of apps will look elsewhere.
So types will multiply.  Maybe that's the best that can be done.

> ...
> Of course. But I don't believe at all that understanding the core issues
> clearly, and identifying what we'd ideally have chosen initially, is
> pointless. It can be very useful (even a precondition) for deciding
> _how_ to move on from what is the case.

Except PEPs yearn to get beyond this stage ;-)  That is, there's
always an early stage where everyone wants to debate every design
decision that was ever made leading up to the PEP (sometimes even just
vaguely related to something the PEP mentions).  That's fine, but the
PEP author(s) eventually tune out.  They're not free to redesign
anything, and are usually trying to solve a more-or-less specific
problem.  Like here, we're just trying to add one stinking bit ;-)

If that inspires someone else to create a grander solution, that's
great.  I'm not sure it's ever happened, but it _could_ be great :-)


>>> 2) Principle of least surprise for casual users. On this question, "you
>>> should use UTC for arithmetic" is equivalent to "you should use a period
>>> recurrence library for period arithmetic." Both arguments are true in
>>> principle, neither one is relevant to the question of casual users
>>> getting the results they expect.

>> That last wasn't ever really a _driving_ force in Python's design.
>> From the earlier example, a great many users have complained a great
>> many times that
>>
>>     1 + "123"
>>
>> _doesn't_ return 124.  That _is_ what most casual users expect.  Tough
>> luck - Python's not for the terminally lazy.

> This example is a false equivalence.

All equivalences are false, yes?  I remain happy enough with the
high-order bits of this one.

> Clearly, trying to guess what a casual user expects to result from
> an ambiguous operation is a bad idea.

We're not guessing at all:  we know darned well what most casual users
expect in this case.  They've been _screaming_ 124 from the start.
The high-order bit is, as I said earlier, that catering to what casual
users expect has never been a primary driver in Python's design.  It'
may be a consideration, but perhaps never at the top of the list.

> I don't think datetime arithmetic (even on non-UTC datetimes) is an
> ambiguous operation,

The analogy wasn't about ambiguity; it was intended to be about "what
a casual user expects" not being a strong argument in the context of
Python's design history.

> given an implementation that consistently treats all timezone-aware
> datetimes as unambiguous instants, or an implementation that
> consistently treats them as naive datetimes with a timezone annotation.
> Given an implementation like datetime that isn't sure what they are,
> _either_ choice of arithmetic is an attractive nuisance.

But only if I also assume the user is terminally dense.  It's like PEP 20 says:

     There should be one-- and preferably only one --obvious way to do it.
     Although that way may not be obvious at first unless you're Dutch.

datetime does Dutch arithmetic.  Once a user figures that out, it's
obvious _then_.  And then also the only obvious way to do classic
arithmetic.  Guido thought using UTC for timeline arithmetic was the
one obvious way to do that;

The first time a user encounters datetime, they may well _think_ "OK,
I'll add a tzinfo, and now I'll get timeline arithmetic!".  That's why
this general rule of Python design required two entire lines in PEP 20
- their thinking is flawed because they're not Dutch.  But they can
learn to be.  Then there is indeed one - and only one - obvious way to
do each flavor of arithmetic, and each way is consistent with best
practices appropriate for that way.

I couldn't care less whether they "get it" at once.  I would care if
they _never_ got it.  But Guido still wouldn't care - he will always
be more profoundly Dutch than me ;-)


>> Well, you can't see me, but I really do have a collection of 42 hats
>> on the table next to me, and every time I write a reply, sentence by
>> sentence I put on the hat most appropriate to what the current
>> sentence intends ;-)

> That's an excellent image, and I'll keep it in mind :-)

If you picture me wearing a nightcap now, you have it nailed ;-)