[Python-Dev] Status on PEP-431 Timezones

[Tim Peters]

[Tim]
>> However, the _body_ of the PEP said nothing whatsoever about altering
>> arithmetic.  The body of the PEP sounds like it's mainly just
>> proposing to fold the pytz package into the core.  Perhaps doing
>> _just_ that much would get this project unstuck?  Hope springs eternal :-)

[Lennart Regebro <regebro at gmail.com>]
> The pytz package has an API and a usage that is different from the
> datetime() module. One of the things you need to do is that after each
> time you do arithmetic, you have to normalize the result. This is done
> because the original API design did not realize the difficulties and
> complexities of timezone handling and therefore left out things like
> ambiguous times.

Oh, they were realized - indeed, the pytz docs point to Python's
tzinfo docs to explain the ambiguities, and the latter docs existed
before ;-) day 1.

The Python docs also are quite clear about that all arithmetic within
a single timezone is "naive".  That was intentional.  The _intended_
way to do "aware" arithmetic was always to convert to UTC, do the
arithmetic, then convert back.

You never _have_ to normalize() in pytz.  But it's needed if you
_don't_ follow pytz's explicit

    The preferred way of dealing with times is to always work in UTC,
    converting to localtime only when generating output to be read by
    humans

advice, and want to do "aware" arithmetic directly in a non-UTC time
zone.  Python's datetime never intended to support that directly.
Quite the contrary.  I know people who feel otherwise tend to think of
that as a lazy compromise (or some such), but naive arithmetic was
intended to be "a feature".  Fight the design every step of the way,
and, yup, you get problems every step of the way.

> The PEP attemps to improved the datetime modules API so that it can
> handle the ambiguous times.

No problem with that.  I always thought the lack of storing
is_dst-like info was datetime's biggest wart.

> It also says that the implementation will be based on pytz, because it
> was my assumption that this would be easy, since pytz already handles
> ambiguous times. During my attempt of implementing it I realized it
> wasn't easy at all, and it wasn't as easy as folding pytz into the core.

Is it the case that pytz also "fails" in the cases your attempts "fail"?

In any case, if you're trying to change how "aware" datetime
arithmetic works, that's a major and backward-incompatible change.
Does Guido realize it?  As before, it's not at all clear from the PEP.

> Yes, the PEP gives that impression, because that was the assumption
> when I wrote the draft. Just folding pytz into the core without
> modifying the API defeats the whole purpose of the PEP, since
> installing pytz is a trivial task.

"Batteries included" has some attractions all on its own.  On top of
that, adding is_dst-like flags to appropriate methods may have major
attractions.  Changing the semantics of datetime arithmetic has major
attractions to some people, but also major drawbacks - regardless,
since changing it turns Guido's original design on its head, he really
needs to Pronounce on that part.

>> Like what?  I'm still looking for a concrete example of what "the
>> problem" is (or even "a" problem).

> A problem is that you have a datetime, and add a timedelata to it, and
> it should then result in a datetime that is actually that timedelta
> later. And if you subtract the same timedelta from the result, it
> should return a datetime that is equal to the original datetime.
>
> This sounds ridiculously simple

Ah, but it already happens that way - because the builtin datetime
arithmetic is "naive".  The docs have always promised this:

"""
datetime2 = datetime1 + timedelta (1)
datetime2 = datetime1 - timedelta (2)

1) datetime2 is a duration of timedelta removed from datetime1, moving
forward in time if timedelta.days > 0, or backward if timedelta.days <
0. The result has the same tzinfo attribute as the input datetime, and
datetime2 - datetime1 == timedelta after. OverflowError is raised if
datetime2.year would be smaller than MINYEAR or larger than MAXYEAR.
Note that no time zone adjustments are done even if the input is an
aware object.

2) Computes the datetime2 such that datetime2 + timedelta ==
datetime1. As for addition, the result has the same tzinfo attribute
as the input datetime, and no time zone adjustments are done even if
the input is aware. This isn’t quite equivalent to datetime1 +
(-timedelta), because -timedelta in isolation can overflow in cases
where datetime1 - timedelta does not.
"""

>, and is ridiculously difficult to make happen in all cases that we want to
> support (Riyahd time zone and leap seconds not included).
> That IS the specific, concrete problem, and if you don't believe me, there
> is nothing I can do to convince you.

I apologize if I've come off as unduly critical - I truly have been
_only_ trying to find out what "the problem" is.  That helps!  Thank
you.  Note that I've had nothing to do with datetime (except to use
it) for about a decade.  I have no idea what you, or anyone else, has
said about it for years & years until this very thread caught my
attention this week.  Heck, for all I know, Guido _demanded_ that
datetime arithmetic be changed - although I doubt it ;-)

> Perhaps I am a complete moron and simply incompetent to do this, and
> in that case I'm sure you could implement this over a day, and then
> please do so, but for the love of the founders of computing I'm not
> going to spend more time repeating it on this mailing list, because
> then we would do better in having you implement this instead of
> reading emails. Me repeating this a waste of time for everyone
> involved, and I will now stop.

Then special thanks for repeating it one more time, since it's the
first time I've heard it :-)

FWIW, I'm sure Isaac and Alexander already know how to make this work,
although it requires (as you found out the hard way) more than just
copying in bits of pytz, and DST transitions aren't the only
non-insane potential problem.  But whether, specifically, the
semantics of datetime arithmetic _should_ change is a major question,
one not explicitly mentioned in the PEP, so this project may have to
move back to square 1 on that specific question.  Resolving
ambiguities is a different question.

> ...
> I was not involved in the discussion then, and even if I had been,
> that's still before I knew anything about the topic. I don't know what
> the arguments were, and I don't think it's constructive to try to
> figure out exactly why that decision was made. That is all to similar
> to assigning blame, which only makes people feel bad. Those who get
> blamed feel bad, and those who blame feel like dicks and onlookers get
> annoyed. Let us look forward instead.

No problem:  there's only Guido to blame, and he's so used to that he
doesn't even notice it anymore ;-)


> I am operating both without any need to defend that decision, as I was
> not involved in it, and I am operating with 20/20 hindsight as I am
> one of the few people having tried to implement a timezone
> implementation that supports ambiguous datetimes based on that
> decision. And then it is perfectly clear and obvious that the decision
> was a mistake and that we should rectify it.

There's more than one decision affecting this  In cases where a single
local time corresponds to more than one UTC time (typically at the end
of DST, when a local hour repeats), datetime never did give any clear
way to do "the intended" conversion from that local time _to_ UTC.
But resolving such ambiguities has nothing to do with how arithmetic
works:  it's utterly unsolvable by any means short of supplying new
info ("which UTC value is intended?" AKA is_dst).

> The only question for me is how and when.

Well, since there's more than one decision, I'm afraid there's also
more than one question ;-)