Mailman 3 August 2015 - Datetime-SIG

Re: [Datetime-SIG] PEP-431/495
by Tim Peters Aug. 27, 2015

Aug. 27, 2015

[Alexander Belopolsky <alexander.belopolsky(a)gmail.com>] > Dealing with simple time strings like this should really be one step: > > >>> ts = '2004-10-31 01:15 EST-05:00' > >>> dt = datetime.strptime(ts.replace('EST', '').replace(':', ''), '%Y-%m-%d > >>> %H%M %z') > >>> print(dt) > 2004-10-31 01:15:00-05:00 > > It is unfortunate that we need to massage the input like this before passing > it to datetime.strptime(). Ideally, datetime.strptime() should have a way > to at least parse -05:00 as a fixed offset timezone. > > However, this problem has nothing to do with PEP 495. Once you have the UTC > offset - there is no ambiguity. The other 01:15 would be "2004-10-31 01:15 > -04:00." The EST part is redundant and is dropped explicitly in my solution > and not used in the solutions by Tim and Guido. You're solving a different problem, though. I keep guessing at what Stuart really wants in the end, but my _best_ guess is that he wants an aware datetime in US/Eastern, not an aware datetime in some anonymous eternally-fixed-offset-of-minus-5-hours timezone. If he really wanted a fixed-offset zone, then I can't imagine why he would pick a string so unlikely as to just happen to show an ambiguous time in US/Eastern. The relation to PEP 495 is the repeated claims that somehow or other it's easy to compute is_dst from the original string but very difficult (or something) to compute "fold". The points of the last few messages were: (1) mktime is a neither cheap nor reliable (in all cases) way to compute is_dst; and, (2) a user doesn't even have to know about "fold" to get a US/Eastern datetime in pytz from that string; and, (3) it's not expensive and is wholly reliable (in all cases) to compute "fold" in a post-PEP-495 pytz if for some unfathomable (to me ;-) ) reason Stewart really _does_ want to compute fold for its own sake from that string (do the dance I gave, then throw away everything except the fold bit). And I agree that extending .strptime() is irrelevant to the PEP ;-)

1 0

Re: [Datetime-SIG] PEP-431/495
by Alexander Belopolsky Aug. 27, 2015

Aug. 27, 2015

On Thu, Aug 27, 2015 at 2:32 PM, Tim Peters <tim.peters(a)gmail.com> wrote: > [Tim] > >> [...] > >> Stewart, I still don't grasp what your problem is. The only concrete > >> example I've seen is dealing with this string: > >> > >> 2004-10-31 01:15 EST-05:00 > .. > [Tim] > > That's fine - I'm trying to get across the idea, not suggest a > specific implementation, and 4 steps are one less than 5 ;-) > Dealing with simple time strings like this should really be one step: >>> ts = '2004-10-31 01:15 EST-05:00' >>> dt = datetime.strptime(ts.replace('EST', '').replace(':', ''), '%Y-%m-%d %H%M %z') >>> print(dt) 2004-10-31 01:15:00-05:00 It is unfortunate that we need to massage the input like this before passing it to datetime.strptime(). Ideally, datetime.strptime() should have a way to at least parse -05:00 as a fixed offset timezone. However, this problem has nothing to do with PEP 495. Once you have the UTC offset - there is no ambiguity. The other 01:15 would be "2004-10-31 01:15 -04:00." The EST part is redundant and is dropped explicitly in my solution and not used in the solutions by Tim and Guido.

1 0

Re: [Datetime-SIG] PEP-431/495
by Tim Peters Aug. 27, 2015

Aug. 27, 2015

[Tim] >> [...] >> Stewart, I still don't grasp what your problem is. The only concrete >> example I've seen is dealing with this string: >> >> 2004-10-31 01:15 EST-05:00 >> >> where you knew "EST" meant US/Eastern, but you haven't explained >> exactly what you're trying to _do_ with it. If you're trying to >> create an aware datetime out of it in a post-PEP-495 pytz, then "the >> obvious" way is: >> >> 1. Create a UTC datetime out of "2004-10-31 01:15" alone. >> 2. Create timedelta(hours=-5) out of "-05:00" alone. >> 3. Subtract the result of #2 from the result of #1, to convert to UTC for >> real. >> 4. Invoke .astimezone() on the result of #3, passing pytz's spelling >> of "US/Eastern". [Guido] > Funny, I would have done it this way (but the outcome should be the same): > > 1. Create a naive datetime out of "2004-10-31 01:15" alone. > 2. Create timedelta(hours=-5) out of "-05:00" alone. > 3. Subtract the result of #2 from the result of #1 to get the UTC time as a > naive datetime. > 4. Use .replace(tzinfo=datetime.timezone.utc) to mark the result as UTC. > 5. Invoke .astimezone() on the result of #3, passing pytz's spelling > of "US/Eastern". > > Only #5 requires pytz (though you could use pytz.utc in #4 -- it doesn't > matter). > > My reason for the extra step is philosophical: the datetime created in the > first step is just a (date, time) combo that *doesn't know* its timezone > yet. Only after step 3 do we have the designated point in time, so then we > can mark it as UTC. > > Of course, since the result is the same, if Tim's version is faster that's a > fine way to do it -- but IMO mine makes it clearer what's going on. That's fine - I'm trying to get across the idea, not suggest a specific implementation, and 4 steps are one less than 5 ;-) An actual speed-conscious implementation would create the datetime with the US/Eastern timezone in my step #1 already. Classic arithmetic still gets the right result in #3 (timedelta subtraction ignores the tzinfo, except just to copy it into the result). The _point_ is that then step #4 can call the cheaper .fromutc() instead (which requires that the datetime invoking it already has the destination tzinfo). You could do the same in yours by changing your #4 to attach US/Eastern, and changing #5 to just invoke .fromutc(). It's a little cheaper to attach the destination zone in #1, though (in all, requires creating one fewer datetime object).

1 0

Re: [Datetime-SIG] PEP-431/495
by Guido van Rossum Aug. 27, 2015

Aug. 27, 2015

On Thu, Aug 27, 2015 at 10:53 AM, Tim Peters <tim.peters(a)gmail.com> wrote: > [...] > Stewart, I still don't grasp what your problem is. The only concrete > example I've seen is dealing with this string: > > 2004-10-31 01:15 EST-05:00 > > where you knew "EST" meant US/Eastern, but you haven't explained > exactly what you're trying to _do_ with it. If you're trying to > create an aware datetime out of it in a post-PEP-495 pytz, then "the > obvious" way is: > > 1. Create a UTC datetime out of "2004-10-31 01:15" alone. > 2. Create timedelta(hours=-5) out of "-05:00" alone. > 3. Subtract the result of #2 from the result of #1, to convert to UTC for > real. > 4. Invoke .astimezone() on the result of #3, passing pytz's spelling > of "US/Eastern". Funny, I would have done it this way (but the outcome should be the same): 1. Create a naive datetime out of "2004-10-31 01:15" alone. 2. Create timedelta(hours=-5) out of "-05:00" alone. 3. Subtract the result of #2 from the result of #1 to get the UTC time as a naive datetime. 4. Use .replace(tzinfo=datetime.timezone.utc) to mark the result as UTC. 5. Invoke .astimezone() on the result of #3, passing pytz's spelling of "US/Eastern". Only #5 requires pytz (though you could use pytz.utc in #4 -- it doesn't matter). My reason for the extra step is philosophical: the datetime created in the first step is just a (date, time) combo that *doesn't know* its timezone yet. Only after step 3 do we have the designated point in time, so then we can mark it as UTC. Of course, since the result is the same, if Tim's version is faster that's a fine way to do it -- but IMO mine makes it clearer what's going on. -- --Guido van Rossum (python.org/~guido)

1 0

Re: [Datetime-SIG] PEP-431/495
by Tim Peters Aug. 27, 2015

Aug. 27, 2015

[Alexander] > Do you realize that a typical portable implementation of mktime calls > localtime up to four times to get tm_isdst? And to get it completely > "right", needs up to six calls? [1] I am sure I can do no worse than that > computing the fold value. Related: last night I was "amused" to see this in Gustavo's dateutil's tz.py's tzlocal implementation: def _isdst(self, dt): # We can't use mktime here. It is unstable when deciding if # the hour near to a change is DST or not. followed by examples from his native Brazilian timezone, then his own code to try to get it right. mktime is a frickin' cross-platform mess, and anyone determined to get transitions exactly right in all cases on all platforms should be scared to death of using it. Stewart, I still don't grasp what your problem is. The only concrete example I've seen is dealing with this string: 2004-10-31 01:15 EST-05:00 where you knew "EST" meant US/Eastern, but you haven't explained exactly what you're trying to _do_ with it. If you're trying to create an aware datetime out of it in a post-PEP-495 pytz, then "the obvious" way is: 1. Create a UTC datetime out of "2004-10-31 01:15" alone. 2. Create timedelta(hours=-5) out of "-05:00" alone. 3. Subtract the result of #2 from the result of #1, to convert to UTC for real. 4. Invoke .astimezone() on the result of #3, passing pytz's spelling of "US/Eastern". You (the user doing this) don't have to compute - or even know anything about - "fold" yourself. pytz's .fromutc() is the only part that has to know about "fold". The whole dance doesn't require invoking _any_ dodgy platform C library date functions. Absolutely everything known about transitions in this dance comes from pytz's wrapping of zoneinfo, so you're (Stuart) entirely in control of what happens, and all pytz users will get exactly the same result on all platforms. And none of it "should be" notably expensive. If that doesn't answer it, what - exactly - _are_ you trying to do with that string?

1 0

Re: [Datetime-SIG] PEP-431/495
by Alexander Belopolsky Aug. 27, 2015

Aug. 27, 2015

On Thu, Aug 27, 2015 at 12:54 PM, Tim Peters <tim.peters(a)gmail.com> wrote: > [Alexander] > > I am afraid this is a very US-centric view. > > If I'm not mistaken, Stewart is Australian. So it's a very AUS- > centric view ;-) > They are the lucky ones: "S" stands for "Summer" in their "EST", but in ours it stands for "Sommerzeitkaputt." Pop quiz: what is_dst value does CEST imply?

1 0

Re: [Datetime-SIG] PEP-431/495
by Carl Meyer Aug. 27, 2015

Aug. 27, 2015

On 08/27/2015 10:54 AM, Tim Peters wrote: > Heh. Try explaining the concept to 99.9% of humanity ;-) I live near > farm country. Every year, when DST ends in late fall, there are > "letters to the editor" complaining that the late crops really _need_ > more sunlight - why can't the damn fool city-slicker politicians just > leave the sun alone, as God created it? I spent some time in rural southern Mexico, where people referred to DST as "government time" and non-DST as "God's time." Most local affairs were scheduled on "God's time," even when DST was supposed to be in effect. Carl

2 1

Re: [Datetime-SIG] PEP-431/495
by Alexander Belopolsky Aug. 27, 2015

Aug. 27, 2015

On Thu, Aug 27, 2015 at 12:25 PM, Stuart Bishop <stuart(a)stuartbishop.net> wrote: > > I have billions of rows of data in my databases. If I'm dealing with > input from computer systems, such as my databases, I will have > billions of conversions to do and the overhead of calculating the fold > flag is billions of times worse than calculating the isdst flag from > the non-bulk human input. Do you realize that a typical portable implementation of mktime calls localtime up to four times to get tm_isdst? And to get it completely "right", needs up to six calls? [1] I am sure I can do no worse than that computing the fold value. [1]: "BUG in mktime" https://www.sourceware.org/ml/libc-hacker/1998-10/msg00027.html

1 0

Re: [Datetime-SIG] PEP-431/495
by Tim Peters Aug. 27, 2015

Aug. 27, 2015

[Stuart] >> I think people are more aware of daylight savings time than you >> imagine, given the nagging by the media on how to adjust their clocks. [Alexander] > I am afraid this is a very US-centric view. If I'm not mistaken, Stewart is Australian. So it's a very AUS- centric view ;-) > Try to explain the concept to someone from a country that > does not mess with the clocks. Heh. Try explaining the concept to 99.9% of humanity ;-) I live near farm country. Every year, when DST ends in late fall, there are "letters to the editor" complaining that the late crops really _need_ more sunlight - why can't the damn fool city-slicker politicians just leave the sun alone, as God created it?

1 0

Re: [Datetime-SIG] PEP-431/495
by Stuart Bishop Aug. 27, 2015

Aug. 27, 2015

On 27 August 2015 at 23:34, Alexander Belopolsky <alexander.belopolsky(a)gmail.com> wrote: > I am afraid this is a very US-centric view. Try to explain the concept to > someone from a country that does not mess with the clocks. People in countries that do not mess with the clocks will never be wondering about if the dst disambiguation flag should be set or not. This is about humans making a decision in the cases where it matters to them. (Currently at UTC+7 all year around, where people only experience DST changes in the form of satellite TV schedules). -- Stuart Bishop <stuart(a)stuartbishop.net> http://www.stuartbishop.net/

1 0