Mailman 3 July 2015 - Datetime-SIG

Re: [Datetime-SIG] Local time disambiguation proposal
by Alexander Belopolsky July 30, 2015

July 30, 2015

On Thu, Jul 30, 2015 at 3:23 PM, Chris Barker <chris.barker(a)noaa.gov> wrote: > Whereas .is_first would mean nothing to me at a glance, so I'd go read the > docs, and hopefully get a nice explanation ;-) Did you read <https://github.com/abalkin/ltdf/blob/master/README.rst#questions-and-answers>?

1 0

Re: [Datetime-SIG] Local time disambiguation proposal
by Alexander Walters July 30, 2015

July 30, 2015

Just looking at the attribute '.first' does not tell you what it is intended to mean. '.is_dst' of one spelling or another is unambiguous; "This datetime object is in the 'special'[1] time." '.first' also makes the default 'True', and it might just be me, but I don't like that aesthetically. [1] Arguments over if DST is the new standard time or not in some parts of the world not withstanding On 7/30/2015 12:43, Alexander Belopolsky wrote: > I would like to make a specific proposal on a narrow topic of how to > handle ambiguities inherent in representing time instances in common > time zones. The following is a rough sketch where I attempted to > identify the areas affected by the proposal and outline the proposed > changes. > > ======= > Rationale > ======= > > In the most world locations there have been and will be times when > local clocks are moved back. In those times intervals are introduced > in which local clocks show the same time twice in the same day. In > these situations, the information displayed on a local clock (or > stored in a Python datetime instance) is insufficient to identify a > particular instance in time. The proposed solution is to add a > boolean flag to the datetime instances that will distinguish between > the two ambiguous times. > > ======= > Proposal > ======= > > The "first" flag > ------------------ > > We propose adding a boolean member called "first" to the instances of > datetime.time and datetime.time classes. This member will have the > value True for all instances except those that represent the second > (chronologically) moment in time in an ambiguous case. > > Affected APIs > ------------------ > > Attributes > ............... > > Instances of datetime.time and datetime.datetime will get a new > boolean attribute called "first." > > Constructors > .................... > > The __new__ methods of the datetime.time and datetime.datetime classes > will get a new keyword-only argument first=True that will control the > value of the "first" attribute in the returned instance. > > Methods > ............. > > The replace() methods methods of the datetime.time and > datetime.datetime classes will get a new keyword-only argument > first=True that will control the value of the "first" attribute in the > returned instance. > > Affected behaviors > ------------------------- > > Implementations of tzinfo > ....................................... > > Subclasses of datetime.tzinfo will read the values of "first" in > utcoffset() and dst() methods and set it appropriately in the > instances > returned by the fromutc() method. No change to the signatures of > these methods is proposed. > > Pickle size > -------------- > Pickle sizes for the datetime.datetime and datetime.time objects will > not change. The "first" flag will be encoded in the first bit of the > byte that currently stores the 4th byte of the datetime.datetime > pickle payload or the 1st byte of the datetime.time. In the current > implementation [1] these bytes are used to store hour value (0-23) and > the first bit is always 0. Note that first=True will be encoded as 0 > in the first bit and first=False as 1. (This change only affects > pickle format. In C implementation, the "first" member will get a > full byte to store the actual boolean value.) > > Temporal arithmetics > ---------------------------- > The value of "first" will be ignored in all operations except > utcoffset() and dst() methods of tzinfo implementations and __eq__ and > __hash__ methods of the datetime.datetime and datetime.time classes. > The only methods that will be able to produce nonzero values of > "first" are __new__ and replace() methods of the datetime.datetime and > datetime.time classes and fromutc() method of some tzinfo > implementations. > > Backward and forward compatibility > ----------------------------------------------- > > This proposal will have no effect on the programs that do not set the > "first" flag explicitly or use tzinfo implementations that do. > Pickles produced by older programs will remain fully forward > compatible. Only datetime/time instances with first=False pickled in > the new versions will become unreadable by the older Python versions. > Pickles of instances with first=False will remain unchanged. > > > ================== > Questions and Answers > ================== > > 1. Why not call the new flag "isdst"? > > Alice: Bob - let's have a stargazing party at 01:30 AM tomorrow! > Bob: Should I presume initially that summer time (for example, > Daylight Saving Time) is or is not (respectively) in effect for the > specified time? > Allice: Huh? > > Bob: Alice - let's have a stargazing party at 01:30 AM tomorrow! > Alice: You know, Bob, 01:30 AM will happen twice tomorrow. Which one > do you have in mind? > Bob: I did not think about it, but let's pick the first. > > > [1]: https://hg.python.org/cpython/file/d3b20bff9c5d/Include/datetime.h#l17 > _______________________________________________ > Datetime-SIG mailing list > Datetime-SIG(a)python.org > https://mail.python.org/mailman/listinfo/datetime-sig > The PSF Code of Conduct applies to this mailing list: https://www.python.org/psf/codeofconduct/

2 2

Re: [Datetime-SIG] Local time disambiguation proposal
by Chris Barker July 30, 2015

July 30, 2015

On Thu, Jul 30, 2015 at 11:33 AM, Alexander Belopolsky < alexander.belopolsky(a)gmail.com> wrote: > I have no emotional attachment to any particular name. The intent is for this new flag to be used for that ambiguous time period during the DST transitions. However, at least in theory, it COULD be used for any time -- i.e. I was US Eastern Standard Time, even though it's the middle of the summer. WHat a user should do is use a tzinfo object that is specifically the Easter Time zone offset, with no DST. (is this spelled "EST" or something in the Olsen database?). But we need to (a) be really clear in the docs whether the flag is consulted any any other time. and (b) probably be really careful in the code to make sure the flag is set appropriately at all times. and, in this case, using a name like "is_dst" would imply to me that you could turn DST on or off at any time. Whereas .is_first would mean nothing to me at a glance, so I'd go read the docs, and hopefully get a nice explanation ;-) -Chris If you like > "is_first" better than "first" on the grammatical grounds - I have no > problem changing the spelling. > > I do have two problems with calling it isdst: > > 1. Whether we want to implement the POSIX standard or not, but > tm_isdst member is the POSIX solution to the problem at hand and if we > give datetime objects a member variable called isdst, but change the > semantics, we will see no end of bug reports. (And since no one > understands POSIX semantics, we have no choice but to change them. :-) > > 2. It will be very confusing to have t.is_dst ≠ t.dst() which will > happen whenever you have an ambiguity due to a change in the standard > time. > _______________________________________________ > Datetime-SIG mailing list > Datetime-SIG(a)python.org > https://mail.python.org/mailman/listinfo/datetime-sig > The PSF Code of Conduct applies to this mailing list: > https://www.python.org/psf/codeofconduct/ > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

1 0

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Guido van Rossum July 30, 2015

July 30, 2015

On Thu, Jul 30, 2015 at 6:31 PM, Tim Peters <tim.peters(a)gmail.com> wrote: > [Guido] > > ... > > The road to improvement that I encourage everyone to explore: add an > is_dst > > flag (with a different name) > > There is no path whatsoever to "correct in all cases" in its absence, > so this should be the top priority for those to whom "correct in all > cases" is their top priority. > And that's why I listed it first. :) > > and create new timedelta-ish classes that implement "human" > > arithmetic (similar to what pytz offers) > > This part seems confused to me: beyond wrapping zoneinfo in a pile of > tzinfo objects, pytz's secondary thrust appears to be "correct in all > cases" involving DST transition times. Doing things like allowing to > correctly distinguish between the ambiguous times when DST ends is, if > I understand anything of your intent, a case of better implementing > "strict time" than of implementing "human time". IOW, to the extent > arithmetic in pytz acts like "human time", it inherits that behavior > from what Python already does. > Oops, I should have checked my facts. I misremembered what pytz does. Indeed "human time" is not one of its features. > > and "strict" arithmetic (similar to [NJ]odaTime, IIUC). > > That sounds right. > > > Separately, standardized access in the stdlib to a tz database (see > another thread). > > That's pytz's primary thrust. Also part of dateutil's thrust > (dateutil leaves Python's arithmetic results entirely alone in all > cases, but also supplies many "calendar operations" useful regardless > of whether you use _its_ wrapping of zoneinfo and/or Windows registry > databases). > Aha! I'd forgotten about dateutil -- it claims "computing of relative deltas (next month, next year, next monday, last week of month, etc)" which sounds more like my "human time" (at least the "next month, next year" part). > > [1] The *implementation* may change, but not in a way that changes > > observable behavior, except for the addition of new attributes, > parameters > > and/or methods. > > I'm afraid that may not be entirely possible. For example, if > datetime objects grow a new attribute, (like some spelling of .is_dst) > then repr() presumably needs to display it. But I know what you mean > ;-) > ;-) -- --Guido van Rossum (python.org/~guido)

1 0

Re: [Datetime-SIG] Clearing up terminology
by Alexander Walters July 30, 2015

July 30, 2015

On 7/30/2015 10:48, ISAAC J SCHWABACHER wrote: > Australia/Melbourne is a time zone, +1000 is an offset, and EST is an accident waiting to happen. > > ijs > Australia/Melbourne is the *Olsan name* for a time zone. +1000 is the current legal definition of that timezone in terms of a GMT offset, EST is the common name of one of many timezones, likely US Eastern Standard Time, but could be one of many. Timezones are political constructs that developers need to work around, not the other way around. Olsan's names are not written in law anywhere (that I know of).

3 2

Re: [Datetime-SIG] Local time disambiguation proposal
by Alexander Walters July 30, 2015

July 30, 2015

On 7/30/2015 14:08, Alexander Belopolsky wrote: > On Thu, Jul 30, 2015 at 1:46 PM, Alexander Walters > <tritium-list(a)sdamon.com> wrote: >> '.is_dst' of one spelling or another is unambiguous; > .. and is "clearly" defined by the POSIX standard as follows: > > "A positive or 0 value for tm_isdst shall cause mktime() to presume > initially that Daylight Savings Time, respectively, is or is not in > effect for the specified time. A negative value for tm_isdst shall > cause mktime() to attempt to determine whether Daylight Savings Time > is in effect for the specified time." [1] > > > Call me when you find two people who can agree on what this means. > > > [1]: http://pubs.opengroup.org/onlinepubs/009695399/functions/mktime.html We are not talking about implementing the POSiX argument to mktime, and I dont think anyone on the list ever was. .is_dst is a bool flag for 'if set, the time represented is in the DST time, if not set, it is in the non-DST time'. '.first' doesnt even imply a bool. "First what?" someone might ask.

2 1

Re: [Datetime-SIG] Clearing up terminology
by Skip Montanaro July 30, 2015

July 30, 2015

On Thu, Jul 30, 2015 at 1:22 PM, ISAAC J SCHWABACHER <ischwabacher(a)wisc.edu> wrote: > Arthur David Olson for supreme leader! As long as it's a position which reports to the BDFL, I'm down with that. :-) Skip

1 0

Re: [Datetime-SIG] Clearing up terminology
by Tim Peters July 30, 2015

July 30, 2015

[Tim] >> "Naive arithmetic" was my neologism, starting in this batch of >> messages. The docs don't call the arithmetic anything, or acknowledge >> that more than one kind of arithmetic is possible. [Chris Barker <chris.barker(a)noaa.gov>] > Which is really something that needs to addressed! Eh. It doesn't really scratch an itch I have, although I realize other people have already scratched down to their bones ;-) Because of the latter, Python should address it. In the few cases I've needed (what Guido now calls "strict" or "timeline") arithmetic, it's just a matter of converting to UTC, doing the arithmetic there, and converting back to the originat tzinfo. These are easy, straightforward functions to write. They don't always get the right answers in the "problem hours" involving DST transitions, but that's not a problem with arithmetic, it's due to that conversions alone can't always deliver the right answers (due to the absence of a correctly updated is_dst-like information in datetime objects). > Can we just use the Glossary borrowed from Noda? It doesn't define an > "arithmetic", but if we need that term, it could be "Period Arithmetic" and > "Duration Arithmetic". I haven't had made time to study the glossary hard, so have no opinion as to whether it's adequate (or even sensible) for Python's purposes. > And I had assumed that your "naive arithmetic" was Duration arithmetic -- > i.e. a day is 24 hours, but apparently not. I'm still confused as to how you > can: (paraphrasing my understanding of your earlier post): > > Convert the datetime to a timedelta from year 1 -- add the timedelta -- > convert back to a datetime. I suppose it's a failure of mine to understand > how datetime uses the tzinfo object, but this description sure sounds to me > like you'd get a datetime that really represented 24 hours later, not 23 or > 25 (or something else...) > > Anyway, my current understanding is that the current implementation of > datetime and timedelta is "Period Arithmetic", with only Periods defined in > units of days (fraction days, down to microsecond resolution). > > Is that correct? As I lack expert knowledge of what "Period Arithmetic" means _exactly_,. I can't answer your question. But, really, it's dead easy to understand what datetime actually does: its _model_ is "naive time". In naive time, a day _is_ exactly 24 hours, and not even the _concept_ of "time zone" exists. Everything (well, almost ;-) ) about how arithmetic works _follows_ from that view of the world. The only irregularities are due to the lumpy nature of the Gregorian calendar (leap years, and months with differing numbers of days). There's no such thing as "time zones" in the naive time model. What endlessly screws some people up is that (almost) all that remains strictly true for arithmetic on datetime objects that _do_ have a fully functinoal tzinfo member. They _still_ follow the "naive time" model. The presence or absence of a tzinfo member makes (almost) no difference at all to arithmetic. I personally would have preferred that "aware" datetime objects use strict/timeline arithmetic instead, but it's not a big deal to me either way. > ... > Sure -- but we really need the docs to clarify what the heck happens when > you have a tz-aware datetime. Nothing at all changes about the arithmetic, _except_ when subtracting two aware datetime objects. Then the subtraction (in effect) converts both to UTC before subtracting. Because when mixing datetimes from _different_ "time zones", there's no plausible justification for considering them to both be in the _same_ "naive time". So in that specific case, strict/timeline arithmetic is done. And that's the source of the repeated "(almost)" repetitions earlier. Ah, BTW, comparison of two aware dateimes also acts in strict/timeline fashion. In my head, that's a consequence of how subtraction works, but not everyone has the benefit of my mental afflictions ;-) > I'd offer to write a patch for the docs, but as you can see, I'm still > pretty confused! Then, by all means, wait until clarity erupts ;-) I think the only strong clues about this now are buried in footnotes on the docs covering datetime's arithmetic operators. They explain exactly _what_ happens, but with no explanation of _why_ it works that way. "That sucks" would be fair. > ... > Well, yes. I was ecstatic when datetime was introduced, and have found it > very, very useful since then. Me too. Honest, has it actually become _less_ useful to you just because this batch of messages started? As many in the Python world can tell you, I _can_ be as obnoxious about being "anally correct" as anyone, and far more so than most. I disliked "naive time" at first glance. I was forced ;-) to implement it (Guido was my boss in real life at the time). But as I used it, I grew to like it a whole lot. Provided you embrace the "naive time" model for what it is, datetime is great. Nevertheless, I would have preferred that aware datetimes use strict/timeline arithmetic in all cases, and would have _greatly_ preferred addressing is_dst from the start. > However, not for "calendar operations" (Period Arithmetic) , nor for tz-aware > datetimes. (because I don't have much need for these) Then you _live_ in a "naive time" world. > And unfortunately, I think the implementation of timedelta makes it > impossible to extend to richer Period Arithmetic -- it fundamentally uses > unit sof days, and has no way to encode any other "Period" unit -- i.e. > months or years. Without taking time now to try to learn exactly what all the new buzz phrases mean, mixed datetime/timedelta arithemtic was never intended to handle all possible, or even all common, use cases for Period Arithmetic. Just the ones with inarguably clear meanings _in_ the naive time model. There is no possible argument about what, e.g., "day" or "week" mean in naive time: in naive time, their meanings as microsecond durations and as deltas between datetimes are the same. But as microsecond durations, "month" and "year" have no clear meanings even in naive time, so timedelta doesn't support those units. > The implementation could fully support Duration Arithmetic, so it's a bit of > shame that that wasn't chosen in the first place. A timedelta represents a specific number of microseconds. That's all it does. When adding/subtracting to;from a datetime, the result is that many microseconds removed in "naive time". That timedelta's "day" is equivalent to naive time's "day" is a consequence of the naive time model. They work flawlessly in tandem. They may not work worth shit together in any other model of time. Note that Guido has already proposed adding new timedelta-ish classes, to "do the right things" for other models of time (well, _the_ other model: the messy ways local clocks actually work, although still ignoring the possibility of leap seconds) and/or fancier kinds of calendar operations (a commonly used phrase in these messages, which I won't presume to translate into the new vocabulary I haven't yet mastered).

1 0

Re: [Datetime-SIG] "Use cases" (was Re: [Python-Dev] Status on PEP-431 Timezones)
by Ethan Furman July 30, 2015

July 30, 2015

On 07/30/2015 09:19 AM, Chris Barker wrote: > Is there any disagreement that both Duration and Period (to use the > Noda terms) operations have important and common use-cases? Important, yes. Common? Not sure. Still worth supporting, though. > Granted, if not, that doesn't tell us anything about what should be > supported in the stdlib, but it seems odd to be arguing about something > this basic. The arguments, as far as I can tell, have been about what the stdlib actually supports. -- ~Ethan~

2 1

Re: [Datetime-SIG] Clearing up terminology
by Chris Barker July 30, 2015

July 30, 2015

Thanks Felipe -- nice and concise and clear! A couple notes: * "local/naive" datetime (no tzinfo) + timedelta is period arithmetic, > where timedelta is a restricted form of period that supports only days, > hours, minutes, seconds, and milliseconds. (Which is equivalent to > mixed-radix seconds, as Tim has pointed out) > With a naive datetime, doesn't this also mean timedelta supports Duration arithmetic? without any DST uglyness, they are the same thing, at least with well defined units like days. At least I've used it this way for years, with no noticed ill effects. Have I just been lucky? > * "aware" datetime (with tzinfo) + timedelta is sort of like period > arithmetic, though Noda doesn't define this operation. It is equivalent to: > 1) remove the tzinfo to get a local/naive dt 2) perform period arithmetic > 3) attach the tzinfo again to get an aware dt. > We need to add some text along these lines to the docs -- that would have helped a lot. > I'm fine extending the Noda definition of "period arithmetic" to cover > this case > Is it really different than Period Arithmetic that only support units of days? Anyway, not a distinction that matters. I am very much +1 for this idea, and will write up a Duration object in > the next couple of days > great -- would it: Convert to UTC Do the math Convert back to local time zone? Or is there another way to handle it? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

1 0