Mailman 3 September 2015 - Datetime-SIG

Re: [Datetime-SIG] IANA TZ database statistics
by Alexander Belopolsky 24 Sep '15

24 Sep '15

On Thu, Sep 24, 2015 at 2:42 PM, Random832 <random832(a)fastmail.com> wrote: > I still don't see the logic in doing any of this rather than parsing > zoneinfo files directly on systems that use it; > There is no portable way to even discover the location of the zoneinfo files. (The default location when installing from the source is the incredible /usr/local/etc/zoneinfo! ) If some location is guessed by searching some likely candidates, there is no guarantee that this is what system tzset() is using.

1 0

Re: [Datetime-SIG] IANA TZ database statistics
by Tim Peters 24 Sep '15

24 Sep '15

[Random832 <random832(a)fastmail.com>] > ... > I still don't see the logic in doing any of this rather than parsing > zoneinfo files directly on systems that use it; > Get[Dynamic]TimeZoneInformation[ForYear] on Windows, etc. Presumably Alex doesn't want to devote his life to fleshing out "etc" on endless platforms he doesn't use. If you think it's simple, _you_ write the code. Start by writing code to answer the question "well, _does_ this system use zoneinfo files?" on all possible Python platforms. Thanks ;-)

1 0

Re: [Datetime-SIG] IANA TZ database statistics
by Random832 24 Sep '15

24 Sep '15

On Thu, Sep 24, 2015, at 14:29, Alexander Belopolsky wrote: > Does Python support any systems with 32-bit time_t? Uh... Linux/i386 comes to mind. I still don't see the logic in doing any of this rather than parsing zoneinfo files directly on systems that use it; Get[Dynamic]TimeZoneInformation[ForYear] on Windows, etc.

1 0

Re: [Datetime-SIG] IANA TZ database statistics
by Tim Peters 24 Sep '15

24 Sep '15

[Alex] > Does Python support any systems with 32-bit time_t? Not that I use ;-) But I'm sure there are many - pick some random 32-bit box. The move to 64-bit time_t appears to be relatively recent even on Linux system. I don't know when it happened on Windows, but 32-bit Windows XP boxes definitely use 32 bits (and there are still flags to allow switching back to that). http://stackoverflow.com/questions/14361651/is-there-any-way-to-get-64-bit-…

1 0

Re: [Datetime-SIG] IANA TZ database statistics
by Alexander Belopolsky 24 Sep '15

24 Sep '15

On Thu, Sep 24, 2015 at 2:16 PM, Tim Peters <tim.peters(a)gmail.com> wrote: > > For the far in the past dates, I still think the earliest transition to > standard > > time should be used as the "big bang" transition. > > I'm not sure exactly what that means - I'm just trying to worm around > that time_t values less than 0 aren't supported on all systems. I should have said "earliest *discoverable* transition." For systems with non-negative time_t, that would be somewhere in 1970s. The key decision here is that regular DST transitions are extended into the future but not into the past. For the far past utcoffset will be fixed at the earliest standard time offset that can be fished out from localtime/mktime calls. Does Python support any systems with 32-bit time_t?

1 0

Re: [Datetime-SIG] IANA TZ database statistics
by Tim Peters 24 Sep '15

24 Sep '15

[Tim] >> Just one suggestion: force the year/timestamp into a 400-year span >> starting at 1971 first (via adding/subtracting multiples of 400 >> years). Then not even Windows will blow up ;-) [Alex] > This will work for the future dates (and I think I should use 2100 through > 2399 range to avoid extending not-regular rules into the far future). That's fine. > For the far in the past dates, I still think the earliest transition to standard > time should be used as the "big bang" transition. I'm not sure exactly what that means - I'm just trying to worm around that time_t values less than 0 aren't supported on all systems. > Note that the 400 year > hack does not work for systems with 32-bit time_t. I think it is ok to just > raise OverflowError on those whenever a timezone operation is requested on a > date outside of EPOCH ± 2**31 range. That's about 140 years. The 400-year hack is just mindlessly simple. It's possible to do far better, since there are only 14 possible yearly calendars (which day of the week is January first, and is it a leap year? 7*2 = 14). So a table with 14 entries, mapping (weekday_of_1_Jan, is_leap) -> fixed canonical year is sufficient. Nothing in that depends on the time zone - it can be precomputed as a static table equally applicable to all time zones (for years in which "normalization": is desired). In general, most (*) 28-year spans contain at least one of each possible yearly calendar. So a 32-bit time_t isn't a real problem here. For example, any system capable of representing the years from 1972 through 1996 inclusive covers all possible yearly calendars. (*) Exceptions can occur when the span crosses a century.

1 0

Re: [Datetime-SIG] IANA TZ database statistics
by Alexander Belopolsky 24 Sep '15

24 Sep '15

On Thu, Sep 24, 2015 at 1:06 PM, Tim Peters <tim.peters(a)gmail.com> wrote: > Just one suggestion: force the year/timestamp into a 400-year span > starting at 1971 first (via adding/subtracting multiples of 400 > years). Then not even Windows will blow up ;-) > This will work for the future dates (and I think I should use 2100 through 2399 range to avoid extending not-regular rules into the far future). For the far in the past dates, I still think the earliest transition to standard time should be used as the "big bang" transition. Note that the 400 year hack does not work for systems with 32-bit time_t. I think it is ok to just raise OverflowError on those whenever a timezone operation is requested on a date outside of EPOCH ± 2**31 range. That's about 140 years.

1 0

Re: [Datetime-SIG] IANA TZ database statistics
by Tim Peters 24 Sep '15

24 Sep '15

[Tim] >> ... >> Specifically which features? Do you just mean .astimezone() treating >> a naive datetime as being in the system zone, and the absence of any >> argument implying the system zone? Or more than just that? [Alex] > Also, .timestamp() respecting the fold attribute and datetime.now() and > datetime.fromtimestamp() setting the fold attribute appropriately. In all > these cases one needs to know how far the transition point is from a given > time. Got it. I should have known that the first time - sorry ;-) >> In that case, "best" is returning what the IANA database >> says should be returned in all cases. > The database itself does not say anything about what should be returned by > various tools, but I would interpret that as "whatever zdump returns." Gimme a break. > ... > I don't think Windows comes with any, but I know close to nothing about > Windows. Windows has minimal (compared to IANA) time zone info stored in the registry. You can look at dateutil's tzwin.py for code accessing it. Zones generally store no historical info, and assume a zone switches DST zero or two times per year.. In the latter case, the registry essentially stores a compiled version of the "n'th weekday of the month" flavor of POSIX TZ string rules, so code can compute when DST starts and ends each year. tzwin.py's tzwinlocal class implements a hybrid tzinfo appropriate for the current system zone (although it never worked for me :-( ). So, ironically enough, this could all be relatively straightforward on Windows: in return for sticking to regular rules, you get to know the rules up front. For portable code, think of Windows as implementing as little as POSIX requires of localtime() and mktime(). While it uses a 64-bit type for time_t, values must be >= 0 and are documented as working only through 31 December 3000 23:59:59 UTC. On my Windows 10 box, it actually goes 21 whole hours ;-) beyond that:" Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 64 bit (AMD64)] on win32 ... >>> datetime.utcfromtimestamp(32535215999) datetime.datetime(3000, 12, 31, 23, 59, 59) >>> datetime.utcfromtimestamp(32535215999 + 21 * 3600) datetime.datetime(3001, 1, 1, 20, 59, 59) >>> datetime.utcfromtimestamp(32535215999 + 21 * 3600 + 1) Traceback (most recent call last): File "<pyshell#36>", line 1, in <module> datetime.utcfromtimestamp(32535215999 + 21 * 3600 + 1) OSError: [Errno 22] Invalid argument >> I also agree figuring out the system zone's rules is a puzzle using >> POSIX. Note that Gustavo gave up on trying to use mktime() in >> dateutil's tzlocal class. > I think he was bitten by the flaky behavior of mktime() when tm_isdst > is passed as -1. Good point! > I intend calling mktime twice with tm_isdst=0 and tm_isdst=1 and detect > fold/gap by what mktime that does to the tm structure. If we discover > that some systems misbehave even in tm_isdst>=0 cases, we can roll > out our own mktime() that probes localtime() multiple times. Just one suggestion: force the year/timestamp into a 400-year span starting at 1971 first (via adding/subtracting multiples of 400 years). Then not even Windows will blow up ;-) > ...

1 0

Re: [Datetime-SIG] IANA TZ database statistics
by Alexander Belopolsky 24 Sep '15

24 Sep '15

On Thu, Sep 24, 2015 at 1:37 AM, Tim Peters <tim.peters(a)gmail.com> wrote: > I also agree figuring out the system zone's rules is a puzzle using > POSIX. Note that Gustavo gave up on trying to use mktime() in > dateutil's tzlocal class. > I think he was bitten by the flaky behavior of mktime() when tm_isdst is passed as -1. I intend calling mktime twice with tm_isdst=0 and tm_isdst=1 and detect fold/gap by what mktime that does to the tm structure. If we discover that some systems misbehave even in tm_isdst>=0 cases, we can roll out our own mktime() that probes localtime() multiple times. > You could say time.timezone and > time.altzone define the only two (or one, if time.daylight is 0) > possible total UTC offsets, and assume that's always been, and always > will be, the case. > Linux (glibc) updates timezone, altzone and tzname whenever localtime() is called. I think this is a horrible hack, but it does not seem to be in violation of POSIX. > But I don't think even `altzone` is actually > required by POSIX - it's of little help :-( > I don't want to rely on any of these variables. For offsets, I would just compute the timestamp on the output of localtime_r (the reentrant version does not mess with globals) and compare that to the input timestamp. For tzname, I would use strftime("%Z") which seems to be rather portable. Of course, on platforms where localtime and mktime fill in tm_gmtoff and tm_zone, I can just use those.

1 0

Re: [Datetime-SIG] IANA TZ database statistics
by Alexander Belopolsky 24 Sep '15

24 Sep '15

> [Alex] > This is not about wrapping IANA's tzdist. This is about implementing PEP > > 495 features using POSIX APIs. > [Tim] > Specifically which features? Do you just mean .astimezone() treating > a naive datetime as being in the system zone, and the absence of any > argument implying the system zone? Or more than just that? > Also, .timestamp() respecting the fold attribute and datetime.now() and datetime.fromtimestamp() setting the fold attribute appropriately. In all these cases one needs to know how far the transition point is from a given time. > > >> [Tim] > >> In that case, "best" is returning what the IANA database > >> says should be returned in all cases. > The database itself does not say anything about what should be returned by various tools, but I would interpret that as "whatever zdump returns." > > > [Alex] > Which version of IANA database? [Tim] > If it's still relevant, the only version any user cares about: the > one that happens to be installed on their machine ;-) I don't think Windows comes with any, but I know close to nothing about Windows.

1 0