Mailman 3 August 2015 - Datetime-SIG

Re: [Datetime-SIG] PEP-431/495
by Akira Li 26 Aug '15

26 Aug '15

Tim Peters <tim.peters(a)gmail.com> writes: > [Tim] >>> ... >>> Later you seem to say you'd prefer a 3-state flag instead, so not sure >>> you really mean "boolean" here. > > [Stuart Bishop <stuart(a)stuartbishop.net>] >> I write Python and SQL for a living. Booleans are 3 state to me ;) > > Got it! Python is sooooooo behind the times :-) > > >> In this case, I'm not fussed if the datetime instance has a 2 state or >> 3 state flag. This is different to the various constructors which I >> think need a 3 state flag in their arguments. True, False, None. > > As things seem to have progressed later, mapping pytz's explicit time > checking into a more magical scheme sprayed all over the datetime > internals is not straightforward, So, as I concluded elsewhere, that > may or may not be done someday, but it's out of scope for PEP 495. > I'm a fan of making progress ("now is better than never", where the > latter was PEP 431's fate waiting for perfection on all counts). > Even if datetime's or replace()'s *first* parameter would be 3-state None|True|False; the internal flag can still be 2-state True|False. first=None could cause a tzinfo callback (it implies that tzinfo must not be None in this case) that sets *first* to True|False appropriately. ... >> Do systems that rely on classic behavior actually exist? > > Of course. A more-or-less subtle example appears later. But we > already mentioned dead-obvious uses: things like "same time tomorrow" > and "same time two weeks from now" are common as mud, and classic > arithmetic implements them fine. So do functions building on those > primitives to implement more sophisticated calendar operations. You > might complain that naive time "same time tomorrow" makes no sense if > someone is starting from 24 hours before what turns out to be a gap > due to DST starting, but few in the real world schedule things at such > times (e.g., DST transitions never occur during normal "business work > hours" if, e.g;, some app postpones a business meeting a week, it's > not credible that they'll ever end up in a gap by adding > timedelta(weeks=1) - unless they're trying to account for leap seconds > too, and the Earth's rotation speeds up "a lot", and "same time next > week" ends up exactly in the missing second). > There is a $5 wifi button that can be used to track baby data. Python helps at various stages: https://medium.com/@edwardbenson/how-i-hacked-amazon-s-5-wifi-button-to-tra… Babies can poop at night and during DST transitions too. Sleep-deprived parents should be able to see the tracking data in local time in addition to UTC (doing timezone conversions is computer's job). On the internet, people may cooperate while being in different time zones i.e., even "business" software might have to work during DST transitions. MMORPGs are probably also not limited to a single time zone. Non-pytz timezones make mistake on the order of an hour regularly. It is *three orders of magnitude larger* than a second. It is a different class of errors. The code that can't handle ~1s errors over short period of time should use time.monotonic() anyway. ... >> It requires someone to have explicitly chosen to use daylight savings >> capable timezones, without using pytz, while at the same time relying on >> classic's surprising arithmetic. Maybe systems using dateutils without >> using dateutils' implementation of datetime arithmetic. > > ? dateutil doesn't implement arithmetic that I know of, apart from > "relative deltas". It inherits Python's classic arithmetic for > datetime - datetime, and datetime +/- timedelta, AFAICT. dateutil doesn't work during DST transitions but PEP 495 might allow to fix it. As I understand, outside of DST transitions if dates are unique valid local times; dateutil uses "same time tomorrow": (d_with_dateutil_tzinfo + DAY == d.tzinfo.localize(d.replace(tzinfo=None) + DAY, is_dst=None)) while pytz uses "+24 hours": dt_add(d_with_dateutil_tzinfo, DAY) == d + DAY where dt_add() is defined below. The equility works but (d + DAY) may have a wrong tzinfo object if the arithmetic crosses DST boundaries (but it has correct timestamp/utc time anyway). d.tzinfo.normalize(d + DAY) should be used to get the correct tzinfo e.g. for displaying the result. Both types of operations should be supported. >... > def dt_add(dt, td): > return dt.tzinfo.fromutc(dt + (td - dt.utcoffset())) > >... > Note: my dt_add 1-liner may fail in cases starting or landing on a > "problem time" (fold/gap). I've never cared, because DST transitions > are intentionally scheduled to occur "wee hours on a weekend", i.e. > when few people are both awake and sober enough _to_ care. But, after > 495 tzinfos are available, the dt_add 1-liner will always work > correctly. That this implementation of timeline arithmetic _can_ > screw up now has nothing to do with its code, it's inherited from the > inability of pure conversion to always work right now. > Such choices should make an application developer, not a library/language developer. library/language should avoid silent errors as much as possible. >> I think this is a bug worth fixing rather than entrenching, before >> adding any dst aware tzinfo implementations to stdlib (including >> 'local'). > > datetime was released a dozen years ago. There's nothing it does that > wasn't already thoroughly entrenched a decade ago. pytz is widely used. datetime objects with dateutil and pytz tzinfo behave differently as shown above. There are no non-fixed tzinfos in stdlib. dst-tzinfo in stdlib could adopt either pytz or dateutil behavior. If dateutil can be fixed to work correctly using the disambiguation flag then its behavior is preferable because it eliminates localize, normalize calls except localize() could be useful in __new__ if first parameter is None to raise an exception for invalid input otherwise it is equivalent to the default *first* value. >> ... >> However... this also means the new flag on the datetime instances is >> largely irrelevant to pytz. pytz' API will need to remain the same. > > My hope was that 495 alone would at least spare pytz's users from > needing to do a `.normalize()` dance after `.astimezone()` anymore. > Although I'm not clear on why it's needed even now. > As far as I know, normalize() is not necessary after astimezone() even now https://answers.launchpad.net/pytz/+question/249229 >> ... >> For pytz users, being able to write a function do tell if the data you >> were given is broken is a step backwards. When constructing a datetime >> instance with pytz, users have the choice of raising exceptions or >> having pytz normalize the input. They are never given broken data (by >> their definition), and there is no need to weed it out. > > Assuming they follow all "the rules", yes? For example, if they > forget to use .localize(), etc, it seems like anything could happen. > What if they use .replace()?: .combine()? Unpickle a datetime > representing a missing time? Etc. I don't see that pytz has anything > magical to check datetimes created by those. If people forget localize() then tzinfo is not attached and an exception is raised later. It is like mixing bytes and Unicode: if you forget decode() then an exception is raised later. replace() is just a shortcut for a constructor. combine() returns naive objects. You can unpickle non-normalized datetime. replace(first=None) may force normalization. Your program may avoid producing non-normalized values. You can choose to save utc+tzid (to use whatever tzdata is available) or utc+tzid+tzdata-version (if you need the same local time) to restore it later. >> ... >> I think all functions that can create datetime instances will need the >> new optional flag and the flag should be tri-state, defaulting to not >> whine. > > See the "PEP-495 - Strict Invalid Time Checking" thread for more. > There seems to be increasing "feature creep" here. Rewriting vast > swaths of datetime internals to cater to this is at best impractical, > especially compared to supplying a "check this datetime" function > users who care can call when they care. Nevertheless, it's a suitable > subject for a different PEP. I don't want to bog 495 down with it. > If it had _stopped_ with asking for an optional check in the datetime > constructor, it may have been implemented already ;-) It may be a subject for another PEP but here's a possible implementation: class datetime: def __new__(...): if first is None and hasattr(tzinfo, 'localize'): self = tzinfo.localize(naive, is_dst=None) # may raise InvalidTime note: self.first is never None i.e., utcoffset(), tzname(), dst() etc always see either first=True or first=False.

2 1

Re: [Datetime-SIG] Slightly better name for fold?
by Alexander Belopolsky 26 Aug '15

26 Aug '15

On Wed, Aug 26, 2015 at 6:02 PM, Alexander Belopolsky < alexander.belopolsky(a)gmail.com> wrote: > On Wed, Aug 26, 2015 at 4:54 PM, jon <jcronin(a)egh.com> wrote: > >> “fold” -> “folded”? >> >> “fold” strikes me as imperative, rather than descriptive. >> > > Note that "fold" is a noun, not a verb in the context of PEP 495. Think > of a bifold wallet as consisting of two "folds" rather than a piece of > leather folded once. Using a noun for the name of an attribute is in-line with the current naming scheme: time( hour=1, minute=30, second=0, fold=1)

1 0

Slightly better name for fold?
by jon 26 Aug '15

26 Aug '15

“fold” -> “folded”? “fold” strikes me as imperative, rather than descriptive. Jonathan Jonathan Cronin jcronin(a)egh.com 781 861 0670 EGH, Inc. 55 Waltham Street Lexington, MA 02421

2 1

Re: [Datetime-SIG] DST explained visually
by ISAAC J SCHWABACHER 26 Aug '15

26 Aug '15

Oh, yes I completely misinterpreted that sentence. Derf. I was led astray by the coincidence that CEST is +0200 and CET +0100. ijs ________________________________ From: Alexander Belopolsky <alexander.belopolsky(a)gmail.com> Sent: Wednesday, August 26, 2015 16:21 To: ISAAC J SCHWABACHER Cc: Guido van Rossum; datetime-sig Subject: Re: [Datetime-SIG] DST explained visually On Wed, Aug 26, 2015 at 5:04 PM, ISAAC J SCHWABACHER <ischwabacher(a)wisc.edu<mailto:ischwabacher@wisc.edu>> wrote: > [2]: https://de.wikipedia.org/wiki/Sommerzeit#Offizielle_Regelung_der_Zeitumstel… My favorite part of that Wikipedia article is this sentence: "Sie darf nicht verwechselt werden mit der international üblichen Abkürzung der Zeitzonen, nach der 2 Uhr MESZ als 0200B und 2 Uhr MEZ als 0200A bezeichnet werden." Which is to say (IIUC) that the international standard is that the first occurrence is B and the second is A, exactly the opposite of the German system. O Weh! I think they just caution you not to confuse the "A" and "B" times with the (military) letter zones A through Z. The fold disambiguation seems to be in the natural A first, then B order.

1 0

Re: [Datetime-SIG] DST explained visually
by Alexander Belopolsky 26 Aug '15

26 Aug '15

On Wed, Aug 26, 2015 at 5:04 PM, ISAAC J SCHWABACHER <ischwabacher(a)wisc.edu> wrote: > > [2]: > https://de.wikipedia.org/wiki/Sommerzeit#Offizielle_Regelung_der_Zeitumstel… > > My favorite part of that Wikipedia article is this sentence: > > "Sie darf nicht verwechselt werden mit der international üblichen > Abkürzung der Zeitzonen, nach der 2 Uhr MESZ als 0200B und 2 Uhr MEZ als > 0200A bezeichnet werden." > > Which is to say (IIUC) that the international standard is that the first > occurrence is B and the second is A, exactly the opposite of the German > system. O Weh! > I think they just caution you not to confuse the "A" and "B" times with the (military) letter zones A through Z. The fold disambiguation seems to be in the natural A first, then B order.

1 0

Re: [Datetime-SIG] DST explained visually
by ISAAC J SCHWABACHER 26 Aug '15

26 Aug '15

> However, apparently, the idea goes back to 1917 Germany. As Paul Eggert explained in private correspondence, > > """ > fold=0 and fold=1 is like the longstanding German standard for expressing times as strings, which uses "A" and "B" to distinguish time stamps that would otherwise be ambiguous. If I understand things correctly, normally fold=0, but you can have fold=1 when time stamps are repeated. For more about the German standard, you can start with the Wikipedia description here: [2] > """ > > [2]: https://de.wikipedia.org/wiki/Sommerzeit#Offizielle_Regelung_der_Zeitumstel… My favorite part of that Wikipedia article is this sentence: "Sie darf nicht verwechselt werden mit der international üblichen Abkürzung der Zeitzonen, nach der 2 Uhr MESZ als 0200B und 2 Uhr MEZ als 0200A bezeichnet werden." Which is to say (IIUC) that the international standard is that the first occurrence is B and the second is A, exactly the opposite of the German system. O Weh! ijs

1 0

Re: [Datetime-SIG] DST explained visually
by Tim Peters 26 Aug '15

26 Aug '15

[Tim] >> So we're switching to a boolean flag again, named solidbefore? [Alex] > Nah, now that we know the history nothing will beat "sommerzeitkaputt"! Bingo! No Google hits at all on "Python sommerzeitkaputt": Although Google is so annoyed by that it says "Showing results for python sommerzeit kaputt" instead, and then we're back to tens of thousands of hits. But good enough. sommerzeitkaputt has such a nice ring to it :-)

1 0

Re: [Datetime-SIG] DST explained visually
by Tim Peters 26 Aug '15

26 Aug '15

[Guido] > Which reminds me, there are some edge cases to consider. What's the > local time for UTC=A? And for UTC=C? I guess the rule is to use half-open > intervals on the X axis that are open on the right, so that A maps to Q and > C maps to R. This is clearer ;-) using the hyperreal number line: https://en.wikipedia.org/wiki/Hyperreal_number Transitions theoretically happen at A-h and C-h, where h is any infinitesimal > 0 (any hyperreal number strictly greater than 0 and strictly less than any real number). So your intuition is right. You may ask "but what are the local times corresponding to A-h and C-h?". That would just be making trouble for no good reason. There are far too few points on a real number line to display A-h or C-h, so "who cares?" is appropriate for _many_ reasons ;-)

1 0

Re: [Datetime-SIG] DST explained visually
by Alexander Belopolsky 26 Aug '15

26 Aug '15

On Wed, Aug 26, 2015 at 4:12 PM, Tim Peters <tim.peters(a)gmail.com> wrote: > So we're switching to a boolean flag again, named solidbefore? Nah, now that we know the history nothing will beat "sommerzeitkaputt"!

1 0

Re: [Datetime-SIG] DST explained visually
by Guido van Rossum 26 Aug '15

26 Aug '15

Here's a cleaned-up scan of my diagram: https://www.dropbox.com/s/q59g183ozahkk5n/DSTplot.pdf?dl=0 -- --Guido van Rossum (python.org/~guido)

1 0