Mailman 3 August 2015 - Datetime-SIG

Re: [Datetime-SIG] Local time disambiguation proposal
by Alexander Belopolsky 04 Aug '15

04 Aug '15

On Tue, Aug 4, 2015 at 2:14 AM, Tim Peters <tim.peters(a)gmail.com> wrote: > [Alexander Belopolsky] >> ... make sure that you've read the latest version. >> >> [1]: https://github.com/abalkin/ltdf/blob/master/README.rst > > Since that appears to be obsolete now, here are some comments on: > > https://github.com/abalkin/ltdf/blob/master/pep-0495.txt > > instead. I made README.rst a symbolic link to pep-0495.txt, so the PEP will get rendered on the main project page: https://github.com/abalkin/ltdf Unfortunately I cannot make the old link work as before. > > """ > The ``replace()`` methods of the ``datetime.time`` and > ``datetime.datetime`` classes will get a new keyword-only argument > called ``first`` with the default value ``True`` > """ > > That's dubious: .replace() treats all other missing arguments as > meaning "copy this attribute's current value". I don't see why > `.first` should be different. Right. I updated the text: https://github.com/abalkin/ltdf/commit/2fa9e6bfeefec61bf642bec658149ba55a34… > """ > The ``timestamp()`` method of ``datetime.datetime`` will return value > advanced by 3600 if ``self`` represents an ambiguous hour and > ``first`` is False. > """ > > Are all known DST adjustments, and changes to standard UTC offsets -- > in, say, zoneinfo -- exactly one hour? For example, I read this on > the web, so it must be true: "Lord Howe Island (Australia) advances > its clocks by half an hour in the summer" ;-) Since I expect we > expect to support all the goofy timezones in zoneinfo, best to get > that right from the start. I agree. In addition to your example, I found some cases where standard time was rolled back: Date Time Zone Offset_before Offset_after 1942-01-28 16:30:00 Asia/Pontianak 9.0 7.5 1942-02-15 16:30:00 Asia/Kuala_Lumpur 9.0 7.5 1942-02-15 16:30:00 Asia/Singapore 9.0 7.5 1942-03-22 16:30:00 Asia/Jakarta 9.0 7.5 1942-04-30 17:30:00 Asia/Rangoon 9.0 6.5 1942-08-31 18:30:00 Asia/Dacca 6.5 6.0 1942-08-31 18:30:00 Asia/Dhaka 6.5 6.0 1944-08-31 15:00:00 Asia/Jayapura 9.5 9.0 1945-09-07 15:00:00 Asia/Seoul 9.0 8.5 1945-10-14 17:30:00 Asia/Karachi 5.5 5.0 1948-04-30 16:30:00 Asia/Jakarta 8.0 7.5 1948-04-30 16:30:00 Asia/Pontianak 8.0 7.5 1950-04-30 16:00:00 Asia/Jakarta 7.5 7.0 1965-01-01 04:30:00 America/Caracas -4.0 -4.5 1965-10-31 04:30:00 America/Goose_Bay -3.5 -4.0 1978-10-20 19:00:00 Asia/Tehran 4.0 3.5 1996-05-24 18:30:00 Asia/Colombo 6.5 6.0 1996-10-25 18:00:00 Asia/Colombo 6.0 5.5 I think, for practical purposes we will allow implementations without direct access to the timezone data to assume that "folds" can only happen on a 15 min boundary and be a multiple of 15 min in duration, but I rewrote that section now. https://github.com/abalkin/ltdf/commit/aa63f7d313ad245c7ca1e8713b07a8799112… I agree with your remaining points and will update the PEP accordingly. > > """ > The value of "first" will be ignored in all operations except those > that involve conversion between timezones. > """ > > "Involve" is vague. Subtraction and comparison can "involve" > conversion to UTC, at least conceptually, when two datetimes don't > share a tzinfo member. It's explicitly stated later that: > > """ > Comparison > ---------- > Instances of ``datetime.time`` and ``datetime.datetime`` classes that > differ only by the value of their ``first`` attribute will compare as > equal. > """ > > but it's not explicitly stated that subtraction of such instances will > return timedelta(0). If there's a reason to explicitly point out one, > then both should be mentioned, since comparison actually inherits its > behavior from subtraction. > > The real reason the cases named will always compare as equal is that > they share a common tzinfo member (or both have none). That > short-circuits the conceptional conversion to UTC before comparison. > But if you plug a workalike tzinfo member into one of them (same > timezone represented by a distinct tzinfo object), then the results of > comparison (and subtraction) _may_ change "just because" .first > differs between them. That all depends on what .utcoffset() returns. > > In the text about comparisons, I expect you're just trying to say that > .first isn't used as a tie breaker, as if datetime comparisons were > akin to tuple comparisons, comparing field by field. But that's not > how it's done, and there's no real point to explaining how a model > that didn't apply to begin with would be affected if it had applied to > begin with ;-) > > So I'd drop the text about comparisons. But I'd add some text > explaining that, while this PEP isn't aiming at timeline arithmetic, > some cases of subtraction and comparison, which have used timeline > arithmetic all along, will return different results now due to > .utcoffset() returning different results now. To make the reader > happy, you could even mention that these different results are also > correct now ;-)

1 0

Re: [Datetime-SIG] Local time disambiguation proposal
by ISAAC J SCHWABACHER 04 Aug '15

04 Aug '15

[Alexander Belopolsky] > > ... > > I am also working on converting my write-up [1] for publication as a > > PEP. Please note that I have made substantial changes since my > > initial post, so if you comment on the proposal please make sure that > > you've read the latest version. > > > > [1]: https://github.com/abalkin/ltdf/blob/master/README.rst [Tim Peters] > Since that appears to be obsolete now, here are some comments on: > > https://github.com/abalkin/ltdf/blob/master/pep-0495.txt > > instead. > > """ > The ``replace()`` methods of the ``datetime.time`` and > ``datetime.datetime`` classes will get a new keyword-only argument > called ``first`` with the default value ``True`` > """ > > That's dubious: .replace() treats all other missing arguments as > meaning "copy this attribute's current value". I don't see why > `.first` should be different. If there's a strong reason (I don't see > one) for > > dt2 = dt1.replace(minutes=2) > > to force dt2.first to True when dt1.first is False, it would be better > to raise an exception than change the value silently. [ijs] Not so long ago I think I finally got a point that Tim has been dancing around throughout this whole discussion, without quite saying it outright. (Another possibility would be Reading Comprehension Fail on my part...) Classic arithmetic and `replace()` are low level operations on datetimes, whereas duration and period arithmetic and time zone conversion are high level ones. The original design of datetime was to expose all of the low level details so that people could implement their own high level stuff, because we're programmers and that's what we do, ya know? All of this is to say that I agree with Tim that `first=True` is not a good default for `replace()`. While that method verifies that all of its numerical arguments are within their valid ranges, it doesn't verify that the resulting time exists in its time zone, so for consistency I would expect that it simply keep the value of the `first` flag without validation or modification if that argument is not provided. But this raises the question of how an unambiguous datetime with first=False should be handled by other code. My preference is that all high level operations should treat such datetimes as having first=True. Also, given this divide, it would be good to document in the datetime module which methods are high level and which are low. > """ > The ``timestamp()`` method of ``datetime.datetime`` will return value > advanced by 3600 if ``self`` represents an ambiguous hour and > ``first`` is False. > """ > > Are all known DST adjustments, and changes to standard UTC offsets -- > in, say, zoneinfo -- exactly one hour? For example, I read this on > the web, so it must be true: "Lord Howe Island (Australia) advances > its clocks by half an hour in the summer" ;-) Since I expect we > expect to support all the goofy timezones in zoneinfo, best to get > that right from the start. Also relevant is the fact that this rationale fails in such cases: """ We chose the minute byte to store the the "first" bit because this choice preserves the natural ordering. """ > """ > The value of "first" will be ignored in all operations except those > that involve conversion between timezones. > """ > > "Involve" is vague. Subtraction and comparison can "involve" > conversion to UTC, at least conceptually, when two datetimes don't > share a tzinfo member. It's explicitly stated later that: > > """ > Comparison > ---------- > Instances of ``datetime.time`` and ``datetime.datetime`` classes that > differ only by the value of their ``first`` attribute will compare as > equal. > """ > > but it's not explicitly stated that subtraction of such instances will > return timedelta(0). If there's a reason to explicitly point out one, > then both should be mentioned, since comparison actually inherits its > behavior from subtraction. > > The real reason the cases named will always compare as equal is that > they share a common tzinfo member (or both have none). That > short-circuits the conceptional conversion to UTC before comparison. > But if you plug a workalike tzinfo member into one of them (same > timezone represented by a distinct tzinfo object), then the results of > comparison (and subtraction) _may_ change "just because" .first > differs between them. That all depends on what .utcoffset() returns. > > In the text about comparisons, I expect you're just trying to say that > .first isn't used as a tie breaker, as if datetime comparisons were > akin to tuple comparisons, comparing field by field. But that's not > how it's done, and there's no real point to explaining how a model > that didn't apply to begin with would be affected if it had applied to > begin with ;-) > > So I'd drop the text about comparisons. But I'd add some text > explaining that, while this PEP isn't aiming at timeline arithmetic, > some cases of subtraction and comparison, which have used timeline > arithmetic all along, will return different results now due to > .utcoffset() returning different results now. To make the reader > happy, you could even mention that these different results are also > correct now ;-) I agree with dropping the part about comparisons, since that will no longer be true for datetimes with tzstrict time zones if that part of the discussion comes to fruition. ijs

1 0

Re: [Datetime-SIG] Local time disambiguation proposal
by Tim Peters 04 Aug '15

04 Aug '15

[Alexander Belopolsky] > ... > I am also working on converting my write-up [1] for publication as a > PEP. Please note that I have made substantial changes since my > initial post, so if you comment on the proposal please make sure that > you've read the latest version. > > [1]: https://github.com/abalkin/ltdf/blob/master/README.rst Since that appears to be obsolete now, here are some comments on: https://github.com/abalkin/ltdf/blob/master/pep-0495.txt instead. """ The ``replace()`` methods of the ``datetime.time`` and ``datetime.datetime`` classes will get a new keyword-only argument called ``first`` with the default value ``True`` """ That's dubious: .replace() treats all other missing arguments as meaning "copy this attribute's current value". I don't see why `.first` should be different. If there's a strong reason (I don't see one) for dt2 = dt1.replace(minutes=2) to force dt2.first to True when dt1.first is False, it would be better to raise an exception than change the value silently. """ The ``timestamp()`` method of ``datetime.datetime`` will return value advanced by 3600 if ``self`` represents an ambiguous hour and ``first`` is False. """ Are all known DST adjustments, and changes to standard UTC offsets -- in, say, zoneinfo -- exactly one hour? For example, I read this on the web, so it must be true: "Lord Howe Island (Australia) advances its clocks by half an hour in the summer" ;-) Since I expect we expect to support all the goofy timezones in zoneinfo, best to get that right from the start. """ The value of "first" will be ignored in all operations except those that involve conversion between timezones. """ "Involve" is vague. Subtraction and comparison can "involve" conversion to UTC, at least conceptually, when two datetimes don't share a tzinfo member. It's explicitly stated later that: """ Comparison ---------- Instances of ``datetime.time`` and ``datetime.datetime`` classes that differ only by the value of their ``first`` attribute will compare as equal. """ but it's not explicitly stated that subtraction of such instances will return timedelta(0). If there's a reason to explicitly point out one, then both should be mentioned, since comparison actually inherits its behavior from subtraction. The real reason the cases named will always compare as equal is that they share a common tzinfo member (or both have none). That short-circuits the conceptional conversion to UTC before comparison. But if you plug a workalike tzinfo member into one of them (same timezone represented by a distinct tzinfo object), then the results of comparison (and subtraction) _may_ change "just because" .first differs between them. That all depends on what .utcoffset() returns. In the text about comparisons, I expect you're just trying to say that .first isn't used as a tie breaker, as if datetime comparisons were akin to tuple comparisons, comparing field by field. But that's not how it's done, and there's no real point to explaining how a model that didn't apply to begin with would be affected if it had applied to begin with ;-) So I'd drop the text about comparisons. But I'd add some text explaining that, while this PEP isn't aiming at timeline arithmetic, some cases of subtraction and comparison, which have used timeline arithmetic all along, will return different results now due to .utcoffset() returning different results now. To make the reader happy, you could even mention that these different results are also correct now ;-)

1 0

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Tim Peters 04 Aug '15

04 Aug '15

Sorry about this. I don't want to talk about leap seconds anymore, but I left a bad impression that intersects with what we _are_ doing. [Guido] >>> ... >>> I like the idea of using a special tzinfo to reveal the leap seconds for >>> those who really want them. (And we won't have to provide such a tzinfo -- >>> it's enough that one could be written, given a table of leap seconds.) [Łukasz Rekucki] >> But if we don't write a one, how do we know it's possible to do? [Tim] > Because it's a shallow problem: not difficult, just tedious. I should expound on _how_ tedious ;-) The basic template for doing timeline arithmetic is: 1. Convert the datetime(s) to UTC. 2. Do straightforward classic arithmetic in UTC. 3. Convert back to original timezone (except for datetime-datetime subtraction, in which case we're done). This works because, in step #2, classic arithmetic is the same as timeline arithmetic. However, that relies on that Python's current implementation of UTC ignores leap seconds. The approximated UTC is an instance of "naive time". But in a world with UTC leap seconds, that's no longer true. In that world, UTC doesn't match naive time, and classic arithmetic can't work. More is needed then, to move to another kind of time bijectively related to UTC that _does_ match naive time (so that classic arithmetic is correct). Since real-life UTC is defined as an offset (well, a collection of pattern-less offsets) from TAI, and TAI is an instance of naive time, that's the obvious choice. So add: 1.5. Use the frickin' leap second table to convert to TAI. and, in 2, s/UTC/TAI/, and 2.5 Use the frickin' leap second table to convert back to UTC.

1 0

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Alexander Belopolsky 03 Aug '15

03 Aug '15

On Mon, Aug 3, 2015 at 4:47 PM, Guido van Rossum <guido(a)python.org> wrote: > The handling of leap seconds by the stdlib isn't going to change in the > forseeable future (let's say until 2038 :-). Until 2038-01-19T03:14:08 UTC+00:00 ± 26 leap seconds to be precise.

1 0

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Łukasz Rekucki 03 Aug '15

03 Aug '15

On Monday, August 3, 2015, Tim Peters <tim.peters(a)gmail.com> wrote: > [Guido] > > Is there a way to end the discussion of leap seconds by fiat? Or maybe > > someone can create leap-seconds-sig where those so inclined can continue > the > > discussion? The handling of leap seconds by the stdlib isn't going to > change > > in the forseeable future (let's say until 2038 :-). Thank you :) > > +1. That means it conveniently ends here with my last message on the > topic, overlapping yours by seconds. I said everything there that > needs to be said ;-) _______________________________________________ > Datetime-SIG mailing list > Datetime-SIG(a)python.org <javascript:;> > https://mail.python.org/mailman/listinfo/datetime-sig > The PSF Code of Conduct applies to this mailing list: > https://www.python.org/psf/codeofconduct/ > -- Łukasz Rekucki

1 0

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Tim Peters 03 Aug '15

03 Aug '15

[Guido] > Is there a way to end the discussion of leap seconds by fiat? Or maybe > someone can create leap-seconds-sig where those so inclined can continue the > discussion? The handling of leap seconds by the stdlib isn't going to change > in the forseeable future (let's say until 2038 :-). +1. That means it conveniently ends here with my last message on the topic, overlapping yours by seconds. I said everything there that needs to be said ;-)

1 0

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Tim Peters 03 Aug '15

03 Aug '15

[Guido] >> ... >> I like the idea of using a special tzinfo to reveal the leap seconds for >> those who really want them. (And we won't have to provide such a tzinfo -- >> it's enough that one could be written, given a table of leap seconds.) [Łukasz Rekucki] > But if we don't write a one, how do we know it's possible to do? Because it's a shallow problem: not difficult, just tedious. We _are_ proposing that tzstrict implement timeline arithmetic across DST transitions. Timeline arithmetic will work correctly for that if and only if it will also work correctly for leap seconds. They're exactly the same problem, it's just that one works naturally in units of minutes while the other works in units of seconds. None of the code cares about that, since - whatever it _looks_ like from the outside - all arithmetic in datetime is "really" working with microseconds. > IMHO, the same approach was taken for tzinfo and DST and it didn't > work out very well. Very different. It was known from the start - and documented from the start - that datetime supplied no way to distinguish between the ambiguous times at the end of DST. It wasn't anticipated that some people would care so much about this non-problem ;-) that they'd make heroic efforts to supply hacks to work around it. That the hacks _are_ so obviously hacks isn't their fault, because they're trying to accomplish something the design never intended to support. Alexander's "first" flag supplies the one bit of support that was missing from the start, and it's equally applicable to ambiguities due to leap seconds as to those due to DST transitions. > For example, I probably could implement a timezone which represents > TAI (assuming I have always up-to-date leap seconds table), but it is > not possible because tzinfo.utcoffset() requires me to return an > integer number of minutes. Yup. There's nothing in the code that really gives a rip about utcoffset() returns - any number of microseconds would work as well. Restricting it to a multiple of minutes with magnitude strictly less than 60*24 was simply intended to be an aid in catching programming errors. Relaxing that restriction is backward-compatible (except for code that's deliberately trying to provoke this exception). > Most times you need leap seconds, is to ignore them (as Tim described > in his trading stock example). Adding support to a single programming > language and fixing it every application, won't make the world better. > Instead, you can use UTC-SLS or do what Google did. > > [1]: http://www.cl.cam.ac.uk/~mgk25/time/utc-sls/ The people who clamor for leap seconds are of two kinds: 1. People who want "purity" because that's what they always want, and regardless of whether it makes any practical difference to them. 2. People who legitimately care about exactly how many SI seconds separate two datetimes. They ought to consider using TAI instead (where this is trivial to determine via code that's direct and obviously correct), but in principle it's possible to get that too from UTC datetimes.(although, if this were my concern, I'd always worry about whether a transformation so complex was correctly implemented by the library I was using! The more critical the application, the greater the value of "obviously correct" over "not obviously broken").

1 0

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Guido van Rossum 03 Aug '15

03 Aug '15

Is there a way to end the discussion of leap seconds by fiat? Or maybe someone can create leap-seconds-sig where those so inclined can continue the discussion? The handling of leap seconds by the stdlib isn't going to change in the forseeable future (let's say until 2038 :-). -- --Guido van Rossum (python.org/~guido)

1 0

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Ethan Furman 03 Aug '15

03 Aug '15

On 08/02/2015 10:32 AM, Łukasz Rekucki wrote: > I don't have much time to respond now, but please don't add leap seconds. People > are actively working to get rid of them entirely and they are never useful on the > business level which this module is aimed at. I don't know whether we should add support for leap-seconds to the new strict tzinfo, but I will mention that the proposal to get rid of them keeps being postponed, and leap seconds are actively harming the business community precisely because they are not being accounted for. -- ~Ethan~

3 2