[Datetime-SIG] PEP 495 (Local Time Disambiguation) is ready for pronouncement

[Guido van Rossum]

On Sun, Aug 16, 2015 at 9:10 PM, Tim Peters <tim.peters at gmail.com> wrote:

> [Guido]
> > How did we end up bending over this far backwards for leap seconds?
>
> Eh - I don't see that we are.  The `first` flag is applicable to any
> source of folds and gaps, provided the folds aren't worse than 2-to-1.
> Leap seconds are just one more case of that.
>

That was in reference to PEP 500 more than to the `first` flag.


> If you want to build in gap-and-fold-aware arithmetic, then it seems
> only one kind can be supported.  If it's left to tzinfo implementers,
> then it's up to them.
>

Well, most tzinfo implementers have other priorities; leap-secondists,
while loud, are rare enough that the extra freedom just complexificates the
API for everyone else without reason.


> > To me, we're talking about a mapping to POSIX timestamps, which use a
> > straightforward algorithm to map compute the date and time -- in
> particular,
> > divmod(ts, 86400) will give the day number and the second within that
> day.
> > The day gets converted to a date using standard calendar math (assuming
> they
> > eventually fix the standard so that 2100 is not considered a leap year
> :-)
> > and the time gets converted to HH:MM:SS using even simpler calculations.
> > There's no room for leap seconds there.
>
> Nobody is proposing to change any of that.  A POSIX timestamp maps to
> the same UTC time in any case (including in cases where a POSIX
> timestamp is ambiguous, and including cases that produce a UTC time
> that never existed (hasn't happened yet, but will when a leap second
> gets removed someday)).
>
> None of that _precludes_ implementing an arithmetic that returns
> correct real-life deltas (between real-life UTC times) as SI-second
> durations.  That can be done in most (but not all) cases using POSIX
> timestamps by consulting a table of leap second adjustments.  It
> _could_ be done in all cases (not just most) using first-aware
> datetimes, because - unlike bare POSIX timestamps - the `first` flag
> gives datetimes a calendar notation that disambiguates the ambiguous
> POSIX timestamps.  Do note that POSIX supports a calendar notation for
> leap seconds too (allowing tm_sec to be 60).
>

But datetime does not. IMO a seconds value of 60 is more likely a bug than
an attempt to represent a leap second.


> _Given_ a `first`-like flag, nothing beyond that is really required
> from Python (although it would be more useful if a way were given to
> map a second value of "60" to/from first=False when applicable).
>

Using the `first` flag sounds like a good compromise then. (Unless leap
seconds may happen at a time when some jurisdiction also changes DST?)


> BTW, POSIX fixed the "2100 is not a leap year" problem in 2001 - but
> ancient Internet rants never die ;-)
>

Good to know. :-)


> > It's important to me that if two different Python implementations,
> running
> > on two different computers, convert a POSIX timestamp to a date+time they
> > get the same result.
>
> That's important to everyone.  It would remain true even if someone
> did write an implementation of leap-second-aware arithmetic.
>
>
> > This is *much* more important to me than the idea that if two computers
> > simultaneously call time.time() they get the same value -- there is
> simply
> > no such thing as "simultaneously" (imagine one of the computers is on
> > a rocket traveling to the moon).
>
> One factoid I learned recently:  the best atomic clocks now are so
> bloody sensitive that they run at detectably different rates if their
> altitude changes by an inch (due to gravitational time dilation).
>
> Luckily, nobody yet has demanded Python support relativistic datetime
> conversions ;-)
>
>
> > If you care about leap seconds you should use a different time source,
> and
> > you shouldn't be using either the time module or the datetime module.
> They
> > are inextricably linked.
>
> Eh - it's a shallow problem.  Just tedious.  Adding `first` is a
> crucial part of the battle for _all_ kinds of gap-and-fold-aware
> arithmetic.  And the code for all kinds of the latter is a tedious but
> conceptually trivial chore.  Accounting for clocks jumping around is
> no harder "theoretically" when the jump is caused by a leap second
> than when it's caused by daylight time starting or ending.
>
>
> > So there's my answer to #1. You may consider this a Pronouncement if you
> > wish. It should not come as any surprise.
>
> You don't want leap-aware arithmetic in the core.  Neither do I.
>
> The remaining question is whether you want to make it impossible for
> someone else to add it.
>

That depends on the cost for everyone else. If the cost is PEP 500, that's
too high a price.


> > And while I'm at it, I don't think PEP 500 is the answer. If you really
> want
> > the number of real-world seconds between two datetime values you can
> write
> > your own difftime() function that consults a leap seconds database.
>
> That's a start.  Related questions include "the nuclear reactor has to
> be vented exactly 3600 SI seconds from now - what will the local clock
> say then?".
>

They should probably use their own clock hardware and software rather than
use time.time().


> I agree such apps "should be" using TAI.  But the one actual scientist
> who has posted here most often says they don't get a choice about the
> time system used by the data they need to analyze.  I say they should
> convert all the data they're given to TAI.  They don't want to hear
> that ;-)
>

But what does their data look like? And what time source originally
generated it (and how does that time source handle leap seconds)? It also
makes a big difference what kind of intervals they are looking at.


> > As for how to request timeline arithmetic vs. the other kind ("human"? I
> > forget where our glossary is), that could be done by special-casing in
> the
> > datetime class using some property (yet to be determined) of the tzinfo
> > subclass or instance; or it could be done using different timedelta-ish
> > classes. PEP 500 seems overly general just so it can also support the
> leap
> > second case.
>
> I've mentioned this before:  I think it's insane to try to implement
> "human arithmetic" by overloading arithmetic operators.  See the
> iCalendar spec for the least people expect now.  Things like "the
> first Tuesday after the first Monday in November every 4 years" (US
> presidential election dates) are just the start.  Trying to spell all
> that with combinations of +-*/% would be a write-only nightmare.
>

We don't have to spell all of that using arithmetic operators. But we can
write it all using simple-minded functions on top of operators that behave
intuitively. E.g. the first Wednesday in November could be determined by
taking November 1 and then doing something like "while it's not Wednesday,
add a day".


> dateutil implements the full iCalendar RRULE spec, and sanely uses
> functions with lots of keyword arguments.  Nobody is _really_ going to
> improve on that.
>
> So I don't see PEP 500 as aiming at human arithmetic at all (although
> others may).  I do see it as a way to:
>
> 1. Keep _all_  gap-and-fold timeline arithmetics out of the core.  I
> don''t really want, e.g., DST-aware timeline arithmetic in the core
> either.
>
> 2. Let tzinfo implementers decide which kinds of gaps and folds they care
> about.
>
>
> > ("So how do I write a real-time stock trading system", you may ask. Good
> > question. Ask the stock exchanges. Their solution was not to trade near
> the
> > leap second. Given that they probably have to deal with a mix of
> languages
> > including at least Java, Cobol, Lisp, Python, and Smalltalk, I'm doubtful
> > that they'll do better during the lifetime of Python 3. Famous last words
> > perhaps.)
>
> Most shut down for at least an hour around the last leap second,
> because a leap second gets inserted just as the hour (on clocks
> running at a multiple-of-hour offset from UTC) is about to change.
> Experience taught them last time around that software confusions
> quickly propagate across fields :-(
>
> But that's a different problem.  Python isn't the _source_ of anyone's
> notion of time.  CPython inherits all it can know about "what time is
> it now?" from the OS and platform C libraries.
>

Sure. And?

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/datetime-sig/attachments/20150817/d2ab3a9a/attachment-0001.html>