Mailman 3 November 2016 - Datetime-SIG

Re: [Datetime-SIG] A tale of gaps, folds and leap seconds.
by Oren Tirosh Nov. 30, 2016

Nov. 30, 2016

I have considered exactly the same thing (using fold for leap seconds). My post was playful exploration and trying to find out what it would actually *mean*. Is it just a convenient place to cram that extra bit? Are these real folds? If they are, are they folds in exactly the same sense? I think I have reached some interesting conclusions from this thought experiment. PEP 495 defines folds for disambiguating “local” time. The “non local” time is presumably UTC and does not require such disambiguation. Except that it does. UTC has folds called leap seconds. So what I envision is extending this notion of localness to multiple levels. The “least local” time is TAI or time scales that differs from it by some constant. UTC is “more local” than TAI. It applies to our local planet Earth whose rate of rotation varies continuously. The real local Earth time is UT1 but UTC is a convenient approximation of UT1 that is quantized to 1 second steps. The “most local” time is civil time, as implemented by various national time zones. In the past, time was often reckoned from local sunrise (in Addis Ababa airport departure times still are, much to the confusion of non locals). Daylight savings could be considered a strange approximation of local sunrise, quantized to a 1 hour step. So we have a three tier hierarchy of localness. The definition is cumulative: civil time is defined by UTC and therefore inherits leap seconds. Do we need two separate fold bits to represent this unambiguously? Leap seconds occur in winter and summer, while DST switches on spring and autumn. The big switch in the Samoa time zone was close, but it was still a full day away from any potential leap second. I haven’t checked the entire IANA database but I think we are lucky and can safely disambiguate the two uses of this bit. Conversion between two local time zones may be done via UTC as an intermediate. Does it make any difference if this intermediate is taken all the way to TAI and back? The only difference is that the conversion will now preserve a leap second. This can be easily simulated without actually going to TAI. A more permissive implementation may verify that this is the last second of December or June without actually verifying that there is an official leap seconds at that time. All local civil time zones are defined by reference to UTC. Even Caesia Standard Time can be defined by reference to UTC. The three levels of localness can still maintain the use of the middle layer (UTC) as reference. There is no need for calculating an actual TAI count of seconds unless explicitly requested. An initial implementation may only support leap seconds for time arithmetic, not live clocks. The only way to get a UTC timestamp with fold=1 (other than setting it explicitly) would be to convert a TAI timestamp to UTC. Directly obtained UTC timestamps will keep doing whatever the local platform does during and around a leap second. A reliable source of TAI (or UTC+correct leap/fold flag) is not so easy to obtain and verify. What do you do if it is not consistent with time_t? CLOCK_TAI defaults to being equal to CLOCK_REALTIME. A comprehensive solution may actually be outside the scope of the Python library. The “right” timezones are a clever and compelling idea, but it means that *everything* must be converted through the timezone logic - including converting time_t to UTC. There is just too much code around that assumes time_t is already UTC - and that days have exactly 86400 SI seconds. These assumptions are obviously incompatible, but at least the one about time_t being UTC breaks only momentarily and infrequently, while using the “right” timezones requires it to be broken it all the time. Oren Oren On Monday, 28 November 2016, Alexander Belopolsky < alexander.belopolsky(a)gmail.com> wrote: > > On Sun, Nov 27, 2016 at 12:44 PM, Oren Tirosh <orent(a)hishome.net > <javascript:_e(%7B%7D,'cvml','orent(a)hishome.net');>> wrote: > >> The "gaps" in conversion from Caesia Standard Time to UTC all correspond >> to UTC Leap Seconds. It is well-known that gaps at one end of a timezone >> conversion correspond to "folds" at the other end - a period that occurs >> twice. Until recently, there was no support for folds in the datetime >> library of the Python language (Caesians are avid Pythonistas). Now that >> PEP495 has been implemented they are keen to check how well the datetime >> library interoperates with their unusual (yet completely well-formed and >> valid) time zone definition. >> >> >> Can you help them? >> > > I am not sure how serious your post is, but I've been toying with an idea > that UTC y-m-d:23:59:*60* should be represented in Python as datetime(y, > m, d, 23, 59, *59*, fold=*1*). This would allow one to implement Olson's > "right" timezones - a timekeeping scheme where time_t is at a fixed offset > of TAI and UTC has a second-wide "fold" on every positive leap second. >

1 0

Re: [Datetime-SIG] A tale of gaps, folds and leap seconds.
by Alexander Belopolsky Nov. 28, 2016

Nov. 28, 2016

On Sun, Nov 27, 2016 at 12:44 PM, Oren Tirosh <orent(a)hishome.net> wrote: > The "gaps" in conversion from Caesia Standard Time to UTC all correspond > to UTC Leap Seconds. It is well-known that gaps at one end of a timezone > conversion correspond to "folds" at the other end - a period that occurs > twice. Until recently, there was no support for folds in the datetime > library of the Python language (Caesians are avid Pythonistas). Now that > PEP495 has been implemented they are keen to check how well the datetime > library interoperates with their unusual (yet completely well-formed and > valid) time zone definition. > > > Can you help them? > I am not sure how serious your post is, but I've been toying with an idea that UTC y-m-d:23:59:*60* should be represented in Python as datetime(y, m, d, 23, 59, *59*, fold=*1*). This would allow one to implement Olson's "right" timezones - a timekeeping scheme where time_t is at a fixed offset of TAI and UTC has a second-wide "fold" on every positive leap second.

1 0

A tale of gaps, folds and leap seconds.
by Oren Tirosh Nov. 27, 2016

Nov. 27, 2016

High in the Pyrenees, on the border between Spain and France, lies the tiny principality of Caesia. Ancient Caesian statues depict the Horae as blindfolded and its people say that "Lady time is blind." The prime meridian happens to pass right through Caesia, which made Universal Time its official time. Caesia has never observed the barbaric practice of meddling with the clock known as Daylight Savings. When Leap Seconds were introduced, the people of Caesia devised an ingenious plan to thwart this new abomination: they have announced that their time zone UTC offset shall be adjusted by one second for every leap second added to UTC. By using the same mechanism that makes it possible for other countries to declare an entire hour as non-existent in local time at the beginning of Daylight Savings the people of Caesia have abolished the 61st second that is occasionally added to the last minute of the last day of June or December. This ensures that civilian time is still officially based on UTC, is well-defined as a proper time zone and can even be described in the tzfile(5) format - and yet never has a day that is not exactly 86400 SI seconds. Caesia had an atomic clock installed in its national labs in 1958. Their far-sighted metrologists had already suggested then to switch its time definition to this standard. Had they done so back in 1958, their clock would have been numerically equivalent to TAI, before the divorce of the second from the rotation of the Earth. However, the final ratification and implementation of this change took a while and only came into effect in 1980. By sheer coincidence, this makes Caesia Standard Time numerically equivalent to GPS time. The "gaps" in conversion from Caesia Standard Time to UTC all correspond to UTC Leap Seconds. It is well-known that gaps at one end of a timezone conversion correspond to "folds" at the other end - a period that occurs twice. Until recently, there was no support for folds in the datetime library of the Python language (Caesians are avid Pythonistas). Now that PEP495 has been implemented they are keen to check how well the datetime library interoperates with their unusual (yet completely well-formed and valid) time zone definition. Can you help them? Oren

1 0

Re: [Datetime-SIG] Add a strftime/strptime code for timezone ID
by Alexander Belopolsky Nov. 5, 2016

Nov. 5, 2016

On Sat, Nov 5, 2016 at 9:26 AM, Chris Angelico <rosuav(a)gmail.com> wrote: > > I believe there was a proposal at one point to have a pip-updateable > package, but a stdlib or OS-provided fallback. If it's possible to > query the version of the OS-provided tzdata, Python could easily check > all three and use the latest one available. Yes, that was PEP 431, [1] but it was withdrawn after its author got stuck on the issue that was later resolved in PEP 495. [2] With that hurdle behind, I plan to revive the proposal to include full timezone support in Python some time in the 3.7 cycle. I started a test project [3] and similar functionality is being incorporated by Paul Ganssle in the dateutil library. [4] We still have enough time for various ideas to get tested in PyPI packages before they can be solidified in a PEP. [1]: https://www.python.org/dev/peps/pep-0431 [2]: https://www.python.org/dev/peps/pep-0495 [3]: https://github.com/abalkin/tz [4]: https://dateutil.readthedocs.io/en/stable/tz.html

1 0

Re: [Datetime-SIG] Add a strftime/strptime code for timezone ID
by Paul G Nov. 5, 2016

Nov. 5, 2016

Does any other programming language actually support IANA zones out of the box? I feel like building in IANA zone support into the language itself is a bad idea, as I think I mentioned in my earlier post. This seems like something that is clearly best handled by third party libraries. Aside from favoring IANA-style zone specifications (which seems unnecessary, especially given that the way the compiled tzdata represents data somewhat inconsistently with Python's tzinfo approach), the distribution of the data seems like it would be a complete nightmare. On 11/03/2016 10:58 AM, Alexander Belopolsky wrote: > > On Thu, Nov 3, 2016 at 10:51 AM, Random832 <random832(a)fastmail.com <mailto:random832@fastmail.com>> wrote: > > Remember, Python doesn't actually support IANA timezones out of the box. > > > I plan to change this in 3.7, but this is a topic for a future PEP. > > > > > _______________________________________________ > Datetime-SIG mailing list > Datetime-SIG(a)python.org > https://mail.python.org/mailman/listinfo/datetime-sig > The PSF Code of Conduct applies to this mailing list: https://www.python.org/psf/codeofconduct/ >

2 1

Re: [Datetime-SIG] Equality of tzinfo objects
by Random832 Nov. 3, 2016

Nov. 3, 2016

On Thu, Nov 3, 2016, at 14:24, Alexander Belopolsky wrote: > On Thu, Nov 3, 2016 at 2:09 PM, Paul G <paul(a)ganssle.io> wrote: > > > I don't understand why it must be the case that something is considered an > > "inter-zone" comparison whenever `t.tzinfo is not s.tzinfo`. What is the > > objection to using `t.tzinfo != s.tzinfo` as the criterion? In general > > these will be equivalent, but using __eq__ allows time zone providers to > > determine what is considered an "inter-zone" comparison. > > > This was probably the case of premature optimization. I cannot think of > any valid reason, but datetime comparisons have always compared tzinfos > using "is" rather than "==" operator. We can probably still make a > change, but it should be implemented and thoroughly tested first. Please > open a bug report. I don't think this really goes far enough. Sure, it allows for times in America/New_York and US/Eastern to be compared, but doesn't allow for times from America/Indiana/Indianapolis - which should be considered the same time zone after 2007, and mechanisms should be provided for a sufficiently sophisticated pytz implementation to determine this - to be compared. So if I understand correctly the purpose of comparing the timezones at all is for ambiguous times, i.e. those near the DST fall-back transition - and other interzone comparisons are allowed? Wasn't resolving this supposed to be the purpose of the fold bit - how do ambiguous times still exist at all?

1 0

Equality of tzinfo objects
by Paul G Nov. 3, 2016

Nov. 3, 2016

When implementing PEP495 for dateutil, I ran across an issue that is related to this: https://www.python.org/dev/peps/pep-0495/#aware-datetime-equality-comparison I'm not sure I understand the reasoning for ambiguous times in inter-zone comparisons comparing as unequal, but I'll take as accepted that this is going to be the assumption, going forward. However, I don't understand why it must be the case that something is considered an "inter-zone" comparison whenever `t.tzinfo is not s.tzinfo`. What is the objection to using `t.tzinfo != s.tzinfo` as the criterion? In general these will be equivalent, but using __eq__ allows time zone providers to determine what is considered an "inter-zone" comparison. One issue is that this actually breaks backwards compatibility of the language, because as far as I can tell there's no way for me to provide a (backwards-compatible) tzinfo class that will preserve this behavior on python versions <= 3.6. Consider: ``` from datetime import datetime from dateutil import tz NYC = tz.gettz('America/New_York') ET = tz.gettz('US/Eastern') dt = datetime(2011, 11, 6, 5, 30, tzinfo=tz.tzutc()) # This is 2011-11-06 01:30 EDT-4 dt_edt = dt.astimezone(ET) dt_nyc = dt.astimezone(NYC) print(dt_nyc == dt_edt) ``` In Python 3.5 that will return True, in Python 3.6 it will return False, even though 'US/Eastern' and 'America/New_York' are the same zone. In this case, I might be able to enforce that these time zones are singletons so that `is` always returns True (though this may have other negative consequences for utility), but even that solution would fall apart for things like `tzrange` and `tzstr`, where you can know that the `dt.utcoffset()`s are going to be identical for ALL values of `dt`, but you can't force the objects to be identical.

2 1

Re: [Datetime-SIG] Add a strftime/strptime code for timezone ID
by M.-A. Lemburg Nov. 3, 2016

Nov. 3, 2016

On 03.11.2016 16:01, Alexander Belopolsky wrote: > On Thu, Nov 3, 2016 at 10:57 AM, M.-A. Lemburg <mal(a)egenix.com> wrote: > >> If anything, a new marker should be introduced to output or parse the TZID. >> > > This is exactly the proposal I started this thread with: add "%/" code to > output or parse the TZID. Right, but you also wrote there are discussions to change %Z to return the TZID. I wouldn't want Python to follow such a change. Perhaps I just misread your proposal :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Nov 03 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/

2 1

Re: [Datetime-SIG] Add a strftime/strptime code for timezone ID
by M.-A. Lemburg Nov. 3, 2016

Nov. 3, 2016

On 03.11.2016 15:20, Alexander Belopolsky wrote: > On Thu, Nov 3, 2016 at 12:38 AM, Random832 <random832(a)fastmail.com> wrote: > >> On the IANA side of things, the tzid is not intended to be displayed as >> a human-readable way to identify the timezone (and this is used to >> rebuff demands by users to create tzids for cities like Brasilia and >> Beijing that people want to see represented over Sao Paulo and Shanghai) >> >> What is the reason for wanting this? >> > > My proposal was indeed prompted by the recent discussion on the IANA > mailing list about human readable abbreviations being changed to numeric > codes. It is not uncommon for people to display timezones with '%Z %z' > format which with recent changes would result in "+10 +1000" for say > Asia/Vladivostok. With the proposed %/ code, "%/ %z" would print > "Asia/Vladivostok +1000" which looks like an improvement even over the > original "VLAT +1000". > > The results of strftime() are not necessarily intended for displaying > directly to end users. An application can have an extra layer that > translates tzids for display and the raw tzids may be useful for saving > times in text files or a database. I don't think Python should enter this mine field. Users will typically only write their own timezone abbreviation (if at all) and also expect this as output. Still, parsing is complicated: Users often get it wrong, e.g. write CET in summer or CEST in winter. Additionally, the timezone codes overlap, e.g. BST can refer to British Standard Time, but also to Bangladesh Standard Time: https://www.timeanddate.com/time/zones/ For machine interfacing, the timezone offset is more reliable and location shouldn't really be part of the time data anyway, but instead be handled in separate data fields. Could you provide a pointer to the proposal to change %Z to anything other than a timezone name abbreviation ? This sounds like a misguided change to me :-) If anything, a new marker should be introduced to output or parse the TZID. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Nov 03 2016) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> Python Database Interfaces ... http://products.egenix.com/ >>> Plone/Zope Database Interfaces ... http://zope.egenix.com/ ________________________________________________________________________ ::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/

2 1

Re: [Datetime-SIG] Add a strftime/strptime code for timezone ID
by Alexander Belopolsky Nov. 3, 2016

Nov. 3, 2016

On Thu, Nov 3, 2016 at 10:51 AM, Random832 <random832(a)fastmail.com> wrote: > Remember, Python doesn't actually support IANA timezones out of the box. > I plan to change this in 3.7, but this is a topic for a future PEP.

1 0