Mailman 3 August 2015 - Datetime-SIG

Re: [Datetime-SIG] Fwd: Calendar vs timespan calculations...
by M.-A. Lemburg Aug. 5, 2015

Aug. 5, 2015

I think you are mixing up a few things here, or I'm just misunderstanding you, which is just as possible (date/time is full of wonders, so no surprise there :-)) If you have a local date/time value which references a day and an time on that day, you can convert this into UTC without caring about leap seconds at all. The reason is that the conversion is done to, again, a date and a time on that day. The problems with leap seconds only matter when you care about time differences and then only if you need the elapsed time calculated in seconds when spanning multiple days. Most business applications only care about time differences calculated in number of days, or seconds between two times on a single day. In finance, you often even ignore complete days for the sake of simplicity (years having 360 days, all months having 30 days, etc. - there's a whole bunch of different systems to choose from). The few times where you really want to know the number of elapsed SI seconds between two points in time, you will most likely not use date/time objects for the calculation anyway, but instead revert to floats or integers counting nano seconds. For those cases, I think it's perfectly fine to have a package on PyPI which deals with the conversion from the point time to the value you are dealing with in your calculations and back again, so complicating the stdlib for this doesn't appear to me to be a good idea. BTW: In mxDateTime I have long resisted adding TZ code. The only TZ support currently in the system is for doing the one time conversion to UTC and back to local time again for display to the user when requested. IMHO, times should always be stored in "UTC", since that's the only half sane time standard that's understood by enough people to get decent interoperable work done (+/- 36 seconds that is, depending on who you talk to ;-)). Regarding terms: I started with "GMT" in mxDateTime, then added "UTC" as alias, and may well add "TAI" at some point as another alias. For most people, these are all the same, anyway :-) Purists will loudly disagree, of course, but I have the Zen of Python to my rescue: practicality beats purity. Cheers, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 05 2015) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/

2 1

Re: [Datetime-SIG] Fwd: Calendar vs timespan calculations...
by Chris Barker - NOAA Federal Aug. 5, 2015

Aug. 5, 2015

>> Exactly. So for now, we simply document that Naive time IS TAI time, > > But it's not. You're free to think of a naive datetime as > representing a TAI time if you like, but that's as far it goes. You > are, e.g., equally free to think of it as being a GPS time. That's up > to you (TAI and GPS time both fit Python's naive time model - but so > does "Tim's personal time", which has nothing to do with durations in > SI seconds). But naive time is not some arbitrary system unto itself. And while it is naive about time zones, and has no DST transitions, it is very much an implementation of a particular calendar system. And when you add a tzinfo object with methods like utcoffset(), then you are very strongly implying that it is now UTC. Oh, and datetime.datetime.utcnow() kind of implies that, too. The docs do clearly state "no leap seconds" in numerous places. But nudges like me bring it up enough that maybe a bit more in the docs would be helpful. I'll contemplate a doc patch. -Chris > > For the rest, this is the high-order bit: > >> ... >> This is uglier than it should be because a lot (most?) time libraries don't >> do leap seconds, > > Can you name one that does? Not a single one has been mentioned in > this entire mountain of messages so far (that I saw). Even Common > LISP, which can usually be counted on to be as anal as conceivable ;-) > , ignores leap seconds. > > Which is a primary reason Python will always ignore them too: it > makes Python's core calculations compatible with every other > programming language's on Earth. > > If you need surprise-free durations in SI seconds, use TAI. If your > colleagues create a Tower of Babel, each using their own definition of > time, that's far more a social problem than a technical one. Agree on > a standard, and - poof! - no more communication or conversion problems > in this area - and if you pick TAI, the intended meaning of the data > will remain clear until the last human dies ;-) > > There are ways to get at "real UTC" (and TAI, and UT1, and ...) from > Python now, but they're in specialized packages few people know about. > That's appropriate, because few people actually need them. They would > confuse people who don't need them. For example, just try to picture > 99.997% of users trying to make sense of the astropy docs: > > http://astropy.readthedocs.org/en/stable/time/index.html > > The astropy.time package provides functionality for manipulating > times and dates. Specific emphasis is placed on supporting > time scales (e.g. UTC, TAI, UT1, TDB) and time > representations (e.g. JD, MJD, ISO 8601) that are used in > astronomy and required to calculate, e.g., sidereal times and > barycentric corrections. > > It's wonderful that such packages exist, but they don't belong in the core.

2 1

Re: [Datetime-SIG] Local time disambiguation proposal
by ISAAC J SCHWABACHER Aug. 5, 2015

Aug. 5, 2015

Sounds like you understand everything I'm saying, and I understand some of what you're saying. :) Top-posted from Microsoft Outlook Web App; may its designers be consigned for eternity to that circle of hell in which their dog food is consumed. ________________________________________ From: Tim Peters <tim.peters(a)gmail.com> Sent: Tuesday, August 4, 2015 22:43 To: ISAAC J SCHWABACHER Cc: Alexander Belopolsky; datetime-sig Subject: Re: [Datetime-SIG] Local time disambiguation proposal [ijs] > Not so long ago I think I finally got a point that Tim has > been dancing around throughout this whole discussion, > without quite saying it outright. (Another possibility > would be Reading Comprehension Fail on my part...) I have to suggest another: that datetime is so far from what you would have designed that you just can't believe Guido didn't have your design in mind all along and was just too harried to implement it properly. See? I don't dance at all ;-) For example, I'm pretty sure you would have designed datetime to store a duration from an epoch. But that was never on the table. It was an explicit requirement to maintain year, month, day, hour ... attributes separately, in both the storage ("pickle") and in-memory formats. That's because conversions between that and a duration is expensive, and many use cases required quick access to the attributes. For example, business-oriented web apps typically do little (if any) time arithmetic, but are forever reading up datetimes from a database and needing to display them in human-readable form (show the attribute values). Sometimes with a time zone indicator attached. And sometimes converted to the viewer's timezone. > Classic arithmetic and `replace()` are low level operations > on datetimes, I can speak about .replace() definitively since I "invented" it: it'is just intended to be shorthand for calling a constructor in cases where "most of" the fields retain the same values. It was born of necessity, because I found an early datetime prototype unbearably tedious to use just for writing unit tests. Because datetime objects are immutable, "changing a single attribute" is horridly verbose without .replace(). > whereas duration The only intended support for durations was in "naive time". As Guido said earlier, he expected that people who needed more than that would continue using timestamps. It's not the first time his expectations were dashed. > and period arithmetic In naive time, period arithmetic with timedelta works fine for units <= weeks. There _were_ use cases for many more kinds of period arithmetic, although they were still all in naive time. But specifying and implementing all of that too is a major project of its own, and we ran out of time. For example, look at all the text it takes to explain RRULE in the iCalendar spec: http://www.ietf.org/rfc/rfc2445.txt Things like "the first Tuesday after a Monday in November, every 4 years" (which describes US presidential election dates) are just the start. Speaking of which, I think it would be insane to try to express such complexities by overloading binary arithmetic operators. Unless someone used that every day, they'd soon forget what all the magic meant: "write-only" code. So we punted on that, hoping someone else would take up the challenge. And, e.g., I believe "dateutil" does implement the whole RRULE spec. > and time zone conversion are high level ones. They're a world of pain unto themselves ;-) And another case where there wan't enough time to do a full-blown job, so only an abstract tzinfo base class was released at first. > The original design of datetime was to expose all of the low level > details It was primarily to implement "naive time" because that alone sufficed to meet the vast majority of the requirements. > so that people could implement their own high level stuff, because > we're programmers and that's what we do, ya know? We certainly did hope people could build on it, but this wasn't so much driven by philosophy as by that our employer was getting visibly (& understandably!) annoyed with continuing to pay for datetime development after it met every major use case identified in the requirements phase. We had to cut it off. > All of this is to say that I agree with Tim that `first=True` is not a > good default for `replace()`. While that method verifies that all of its > numerical arguments are within their valid ranges, it doesn't verify |> that the resulting time exists in its time zone, so for consistency I > would expect that it simply keep the value of the `first` flag without > validation or modification if that argument is not provided. I believe Alexander already agreed with this. > But this raises the question of how an unambiguous datetime with > first=False should be handled by other code. My preference is that > all high level operations should treat such datetimes as having > first=True. I agree this needs to be cleared up. The _actual_ "rules" now appear to be: 1. first==False means this is the later of two ambiguous times. 2. first==True means anything else (it's the earlier of two ambiguous times, or it's not an ambiguous time at all). 3. And, by the way, #1 was fibbing: first=False may also mean it's not an ambiguous time. However, so far #3 may only be in internal uses (like the .fromutc() implementation fiddling the flag to trick .utcoffset() into telling it something useful). I don't think it needs to be decided just yet. As things get implemented, the delights and drawbacks will get clearer. > Also, given this divide, it would be good to document in the datetime > module which methods are high level and which are low. Well, since that distinction doesn't exist in my head, you'll have to write the doc patch ;-) ... >> Are all known DST adjustments, and changes to standard UTC offsets -- >> in, say, zoneinfo -- exactly one hour? For example, I read this on >> the web, so it must be true: "Lord Howe Island (Australia) advances >> its clocks by half an hour in the summer" ;-) Since I expect we >> expect to support all the goofy timezones in zoneinfo, best to get >> that right from the start. > Also relevant is the fact that this rationale fails in such cases: > > """ > We chose the minute byte to store the the "first" bit because this choice preserves the natural ordering. > """ Sorry, I didn't grasp your meaning there. But I didn't grasp Alexander's intent either ;-) From "later ambiguous time" alone we have no idea _how_ much later, so the bit-fiddling comment there didn't make sense to me. Perhaps that was what you were getting at? The C-level datetime comparison code is much lower-level than in datetime.py, and the former actually compares raw bytestrings: if (GET_DT_TZINFO(self) == GET_DT_TZINFO(other)) { diff = memcmp(((PyDateTime_DateTime *)self)->data, ((PyDateTime_DateTime *)other)->data, _PyDateTime_DATETIME_DATASIZE); Stuffing a flag into any of the existing bytes is bound to break some case there. I'd add another byte to the pickle, but then I'm no longer paid to obsess over bytes ;-) > ... > I agree with dropping the part about comparisons, since that will > no longer be true for datetimes with tzstrict time zones if that part > of the discussion comes to fruition. Yup! It's a big step in the right direction if we can just get ambiguous local->utc conversions working correctly first.

1 0

Re: [Datetime-SIG] Fwd: Calendar vs timespan calculations...
by Chris Barker - NOAA Federal Aug. 5, 2015

Aug. 5, 2015

> Python isn't going to support leap seconds out of the box. Got it -- no problem there. > We > (primarily meaning Alexander) _are_ trying to make it possible for > someone who wants to supply an external implementation to do so, That's all I'm sayin' And thank you Alexander for putting so much effort into this. -Chris

1 0

Re: [Datetime-SIG] Local time disambiguation proposal
by Tim Peters Aug. 5, 2015

Aug. 5, 2015

[ijs] > Not so long ago I think I finally got a point that Tim has > been dancing around throughout this whole discussion, > without quite saying it outright. (Another possibility > would be Reading Comprehension Fail on my part...) I have to suggest another: that datetime is so far from what you would have designed that you just can't believe Guido didn't have your design in mind all along and was just too harried to implement it properly. See? I don't dance at all ;-) For example, I'm pretty sure you would have designed datetime to store a duration from an epoch. But that was never on the table. It was an explicit requirement to maintain year, month, day, hour ... attributes separately, in both the storage ("pickle") and in-memory formats. That's because conversions between that and a duration is expensive, and many use cases required quick access to the attributes. For example, business-oriented web apps typically do little (if any) time arithmetic, but are forever reading up datetimes from a database and needing to display them in human-readable form (show the attribute values). Sometimes with a time zone indicator attached. And sometimes converted to the viewer's timezone. > Classic arithmetic and `replace()` are low level operations > on datetimes, I can speak about .replace() definitively since I "invented" it: it'is just intended to be shorthand for calling a constructor in cases where "most of" the fields retain the same values. It was born of necessity, because I found an early datetime prototype unbearably tedious to use just for writing unit tests. Because datetime objects are immutable, "changing a single attribute" is horridly verbose without .replace(). > whereas duration The only intended support for durations was in "naive time". As Guido said earlier, he expected that people who needed more than that would continue using timestamps. It's not the first time his expectations were dashed. > and period arithmetic In naive time, period arithmetic with timedelta works fine for units <= weeks. There _were_ use cases for many more kinds of period arithmetic, although they were still all in naive time. But specifying and implementing all of that too is a major project of its own, and we ran out of time. For example, look at all the text it takes to explain RRULE in the iCalendar spec: http://www.ietf.org/rfc/rfc2445.txt Things like "the first Tuesday after a Monday in November, every 4 years" (which describes US presidential election dates) are just the start. Speaking of which, I think it would be insane to try to express such complexities by overloading binary arithmetic operators. Unless someone used that every day, they'd soon forget what all the magic meant: "write-only" code. So we punted on that, hoping someone else would take up the challenge. And, e.g., I believe "dateutil" does implement the whole RRULE spec. > and time zone conversion are high level ones. They're a world of pain unto themselves ;-) And another case where there wan't enough time to do a full-blown job, so only an abstract tzinfo base class was released at first. > The original design of datetime was to expose all of the low level > details It was primarily to implement "naive time" because that alone sufficed to meet the vast majority of the requirements. > so that people could implement their own high level stuff, because > we're programmers and that's what we do, ya know? We certainly did hope people could build on it, but this wasn't so much driven by philosophy as by that our employer was getting visibly (& understandably!) annoyed with continuing to pay for datetime development after it met every major use case identified in the requirements phase. We had to cut it off. > All of this is to say that I agree with Tim that `first=True` is not a > good default for `replace()`. While that method verifies that all of its > numerical arguments are within their valid ranges, it doesn't verify |> that the resulting time exists in its time zone, so for consistency I > would expect that it simply keep the value of the `first` flag without > validation or modification if that argument is not provided. I believe Alexander already agreed with this. > But this raises the question of how an unambiguous datetime with > first=False should be handled by other code. My preference is that > all high level operations should treat such datetimes as having > first=True. I agree this needs to be cleared up. The _actual_ "rules" now appear to be: 1. first==False means this is the later of two ambiguous times. 2. first==True means anything else (it's the earlier of two ambiguous times, or it's not an ambiguous time at all). 3. And, by the way, #1 was fibbing: first=False may also mean it's not an ambiguous time. However, so far #3 may only be in internal uses (like the .fromutc() implementation fiddling the flag to trick .utcoffset() into telling it something useful). I don't think it needs to be decided just yet. As things get implemented, the delights and drawbacks will get clearer. > Also, given this divide, it would be good to document in the datetime > module which methods are high level and which are low. Well, since that distinction doesn't exist in my head, you'll have to write the doc patch ;-) ... >> Are all known DST adjustments, and changes to standard UTC offsets -- >> in, say, zoneinfo -- exactly one hour? For example, I read this on >> the web, so it must be true: "Lord Howe Island (Australia) advances >> its clocks by half an hour in the summer" ;-) Since I expect we >> expect to support all the goofy timezones in zoneinfo, best to get >> that right from the start. > Also relevant is the fact that this rationale fails in such cases: > > """ > We chose the minute byte to store the the "first" bit because this choice preserves the natural ordering. > """ Sorry, I didn't grasp your meaning there. But I didn't grasp Alexander's intent either ;-) From "later ambiguous time" alone we have no idea _how_ much later, so the bit-fiddling comment there didn't make sense to me. Perhaps that was what you were getting at? The C-level datetime comparison code is much lower-level than in datetime.py, and the former actually compares raw bytestrings: if (GET_DT_TZINFO(self) == GET_DT_TZINFO(other)) { diff = memcmp(((PyDateTime_DateTime *)self)->data, ((PyDateTime_DateTime *)other)->data, _PyDateTime_DATETIME_DATASIZE); Stuffing a flag into any of the existing bytes is bound to break some case there. I'd add another byte to the pickle, but then I'm no longer paid to obsess over bytes ;-) > ... > I agree with dropping the part about comparisons, since that will > no longer be true for datetimes with tzstrict time zones if that part > of the discussion comes to fruition. Yup! It's a big step in the right direction if we can just get ambiguous local->utc conversions working correctly first.

1 0

Re: [Datetime-SIG] Fwd: Calendar vs timespan calculations...
by Tim Peters Aug. 5, 2015

Aug. 5, 2015

[Tim] >> Sorry about this. I don't want to talk about leap seconds anymore, [Chris Barker] > Nor do I, but ... Python isn't going to support leap seconds out of the box. We (primarily meaning Alexander) _are_ trying to make it possible for someone who wants to supply an external implementation to do so, but that's not going to drive any current decision. > ... > Exactly. So for now, we simply document that Naive time IS TAI time, But it's not. You're free to think of a naive datetime as representing a TAI time if you like, but that's as far it goes. You are, e.g., equally free to think of it as being a GPS time. That's up to you (TAI and GPS time both fit Python's naive time model - but so does "Tim's personal time", which has nothing to do with durations in SI seconds). For the rest, this is the high-order bit: > ... > This is uglier than it should be because a lot (most?) time libraries don't > do leap seconds, Can you name one that does? Not a single one has been mentioned in this entire mountain of messages so far (that I saw). Even Common LISP, which can usually be counted on to be as anal as conceivable ;-) , ignores leap seconds. Which is a primary reason Python will always ignore them too: it makes Python's core calculations compatible with every other programming language's on Earth. If you need surprise-free durations in SI seconds, use TAI. If your colleagues create a Tower of Babel, each using their own definition of time, that's far more a social problem than a technical one. Agree on a standard, and - poof! - no more communication or conversion problems in this area - and if you pick TAI, the intended meaning of the data will remain clear until the last human dies ;-) There are ways to get at "real UTC" (and TAI, and UT1, and ...) from Python now, but they're in specialized packages few people know about. That's appropriate, because few people actually need them. They would confuse people who don't need them. For example, just try to picture 99.997% of users trying to make sense of the astropy docs: http://astropy.readthedocs.org/en/stable/time/index.html The astropy.time package provides functionality for manipulating times and dates. Specific emphasis is placed on supporting time scales (e.g. UTC, TAI, UT1, TDB) and time representations (e.g. JD, MJD, ISO 8601) that are used in astronomy and required to calculate, e.g., sidereal times and barycentric corrections. It's wonderful that such packages exist, but they don't belong in the core.

1 0

Re: [Datetime-SIG] Local time disambiguation proposal
by Alexander Belopolsky Aug. 4, 2015

Aug. 4, 2015

On Tue, Aug 4, 2015 at 2:14 AM, Tim Peters <tim.peters(a)gmail.com> wrote: > """ > The value of "first" will be ignored in all operations except those > that involve conversion between timezones. > """ > > "Involve" is vague. Subtraction and comparison can "involve" > conversion to UTC, at least conceptually, when two datetimes don't > share a tzinfo member. I tried to address this in <https://github.com/abalkin/ltdf/commit/9cb251c7286cf0ad712fc911228c2af9cb95…>.

1 0

Re: [Datetime-SIG] Fwd: Calendar vs timespan calculations...
by Alexander Belopolsky Aug. 4, 2015

Aug. 4, 2015

[Tim Peters] >> TAI >> is an instance of naive time, that's the obvious choice. So add: >> >> 1.5. Use the frickin' leap second table to convert to TAI. >> >> and, in 2, s/UTC/TAI/, and >> >> 2.5 Use the frickin' leap second table to convert back to UTC. [Chris Barker] > > Exactly. So for now, we simply document that Naive time IS TAI time, > and that the to-from UTC methods on the tzinfo objects are actually > to-from TAI ( renaming them is probably out of the question, though) I this is not the POV that I've been advocating. Since the subject of this thread is "Calendar vs timespan calculations", let me discuss a use case that is hopefully more familiar: compute the number of days from 1900-02-01 to 1900-03-01: >>> date(1900,3,1) - date(1900,2,1) datetime.timedelta(28) Python's answer (28) is correct in the most of the Western world, but in Greece, there were 29 days from 1900-02-01 to 1900-03-01 because they did have 1900-02-29. (Greece did not switch to the Gregorian calendar until 1923.) Note that "1900-02-01" and "1900-03-01" are just names for two historical days which had different meanings in different parts of the world. The digits that appear in those names do not necessarily mean anything arithmetically. The first day of February does not have to be in a constant relationship to the first day of March any more than the first day of Hanukkah to the first day of Ramadan. While it is convenient to have a universal bijection between days and the (additive group of) integers, it is not a prerequisite for being able to compute the number of days between two dates. For example, the hypothetical >>> julian_days("1900-02-01", "1900-03-01") 29 >>> gregorian_days("1900-02-01", "1900-03-01") 28 can be implemented without any conversion to the common scale. (For example, we can have a large list of julian days from "1000-01-01" to "9000-01-01" and another large list of gregorian days from "0006-06-06" to "9999-09-09" and implement our functions using a binary search into these lists. Not an efficient algorithm, but universal enough to cover calendars that use names of the living Emperors instead of years.) My main point is that as long as we can spell two dates in a way that is understood in some part of the world, we can have a software module that can tell the number of days (or seconds) between these two dates as long as it has accurate enough information about timekeeping practices in that location. This software module may or may not operate by converting to any well-known integer scale. My specific proposal can be summarized in the following pseudocode: class datetime: ... def __add__(self, other): try: add = self.tzinfo.add except AttributeError: # old logic else: return add(self, other) def __sub__(self, other): try: sub = self.tzinfo.sub except AttributeError: # old logic else: return sub(self, other) If we do that, we can implement a HistoricalGreek timezone, so that >>> datetime(1900,3,1,tzinfo=HistoricalGreek) - date(1900,2,1,tzinfo=HistoricalGreek) datetime.timedelta(29) but we will face the problem datetime(1900,2,29,tzinfo=HistoricalGreek) will still raise "ValueError: ('day must be in 1..28', 29)". This problem can be solved by implementing HistoricalGreek.add so that >>> datetime(1900,2,28,tzinfo=HistoricalGreek) + timedelta(1) datetime .datetime(1900,2,29,tzinfo=HistoricalGreek,first=False) Now, if we also want "next day after 1900-02-28 in Greece" to print nicely as "1900-02-29", we should also arrange that isoformat() is also delegated to HistoricalGreek: def isoformat(self, other): try: fmt = self.tzinfo.isoformat except AttributeError: # old logic else: return fmt(self, other) and HistoricalGreek.isoformat will then know that what comes as the "repeated February 28" is the "February 29" in disguise. The only tricky issue here is the mixed timezone arithmetics. My solution is to disallow subtraction of datetime instances that both have tzinfo implement the "sub" method, but not the same: def __sub__(self, other): try: sub = self.tzinfo.sub try: other_sub = other.tzinfo.sub except AttributeError: other_sub = sub except AttributeError: # old logic else: if sub.__func__ is not other_sub.__func__: raise ValueError("Incompatible calendars") return sub(self, other)

1 0

Re: [Datetime-SIG] Fwd: Calendar vs timespan calculations...
by Alexander Belopolsky Aug. 4, 2015

Aug. 4, 2015

On Tue, Aug 4, 2015 at 12:13 PM, Chris Barker <chris.barker(a)noaa.gov> wrote: >> On Aug 3, 2015, at 9:11 PM, Tim Peters <tim.peters(a)gmail.com> wrote: >> >> Sorry about this. I don't want to talk about leap seconds anymore, > > Nor do I, but > > TL;DR -- leap seconds are in the definition of UTC. I have not reflected this in the PEP, but in the reference implementation, I am toying with the following bit of "leap seconds support": diff --git a/Lib/datetime.py b/Lib/datetime.py .. - ss = min(ss, 59) # clamp out leap seconds if the platform has them - return cls(y, m, d, hh, mm, ss, us) + ss, first = (ss, True) if ss < 60 else (59, False) + return cls(y, m, d, hh, mm, ss, us, first=first) .. See <https://github.com/abalkin/cpython/commit/a54ab4502b9b50ae87d7795930fd13e70…>. Note that even though many people claim that POSIX does not support leap seconds, this is only true with respect to the time_t timestamps and does not apply to the broken down tm structure which is more directly related to Python datetime than "seconds since EPOCH." To the contrary, leap seconds support is mandated by POSIX in the case of struct tm: """ The <time.h> header shall declare the structure tm, which shall include at least the following members: int tm_sec Seconds [0,60]. ... The range [0,60] for tm_sec allows for the occasional leap second. """ See <http://pubs.opengroup.org/onlinepubs/009695399/basedefs/time.h.html>.

1 0

Fwd: Calendar vs timespan calculations...
by Chris Barker Aug. 4, 2015

Aug. 4, 2015

> On Aug 3, 2015, at 9:11 PM, Tim Peters <tim.peters(a)gmail.com> wrote: > > Sorry about this. I don't want to talk about leap seconds anymore, Nor do I, but TL;DR -- leap seconds are in the definition of UTC. And there are use cases (see below) that python may want to support at some point. So it would be good to have infrastructure in place so that someone could implement it in the future (probably as a third party lib). Now, if you actually want to read more about this: > The basic template for doing timeline arithmetic is: > > 1. Convert the datetime(s) to UTC. > 2. Do straightforward classic arithmetic in UTC. > 3. Convert back to original timezone (except for datetime-datetime > subtraction, in which case we're done). > > This works because, in step #2, classic arithmetic is the same as > timeline arithmetic. However, that relies on that Python's current > implementation of UTC ignores leap seconds. The approximated UTC is an > instance of "naive time". If I have this right, the datetime object encodes time using the proleptic Gregorian calendar, with no leap seconds -- yes? Is that what TAI time is? From: http://www.timeanddate.com/time/international-atomic-time.html It appears so. > TAI > is an instance of naive time, that's the obvious choice. So add: > > 1.5. Use the frickin' leap second table to convert to TAI. > > and, in 2, s/UTC/TAI/, and > > 2.5 Use the frickin' leap second table to convert back to UTC. Exactly. So for now, we simply document that Naive time IS TAI time, and that the to-from UTC methods on the tzinfo objects are actually to-from TAI ( renaming them is probably out of the question, though) Then if anyone ever wants to implement leap seconds (I.e. a proper UTC), then they create a UTC tzinfo object that converts properly to-from TAI with that frikin' leap second table. ( and all the other time zones to match...). I still think that the base datetime object should be UTC, which would include leap seconds, but it's too late for that. (hmm -- could a user simply plug in a new datetime object, instead of a new tzinfo objects? but anyway, as long as the door is open one way or another, then we're good for now. Use case: In the netcdf CF metadata standard: http://cfconventions.org Datetimes are encoded as "some time span since an epoch", i.e. "seconds since 1970-01-01T0:0:0". The epoch can be anything, and the units can be any well-defined reasonable unit -- seconds, hours, days. Then elsewhere, you can specify the Calendar used. And this gets ugly because there are some weird ones -- for instance, some climate models use a 360 day year! There was recently a big huge discussion about how to specify the variations of the Gregorian calendar - whether leap seconds are in play or not, etc. This is uglier than it should be because a lot (most?) time libraries don't do leap seconds, but many people _think_ they are using UTC, when they are not. And for the most part, it's close enough. When using this encoding, it would be smart to use an epoch close to the time of your data / model results -- for instance, the zeroth time step of your model. And some people do this. In that case, you're likely to only cross maybe one or two leap-seconds. But in practice, a lot of folks encode their time using an epoch farther away from their data -- sometimes Unix epoch (that makes sense), and sometimes ones that seem totally arbitrary to me (maybe their time library uses it?). In this case, when the times are encoded back to calendar representation, they are often off by 26 seconds if a time lib that doesn't support leap-seconds is used. In practice, even 26 seconds probably doesn't matter in most cases, but when you are working with datetimes (like Python's) with a resolution of microseconds, errors that large are perceived to be significant, and can lead to actual errors (like getting time steps from different sources out of order), if the code is not written carefully to account for the limited precision of the inputs. OK -- I'm done with leap seconds. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

1 0