Mailman 3 July 2015 - Datetime-SIG

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Carl Meyer 02 Aug '15

02 Aug '15

On 07/30/2015 10:46 AM, Chris Barker wrote: > There is "correct" and incorrect, but I"m not arguing that anything is > incorrect about the current behavior -- I thought a timedelta was a > duration, but I was wrong, it is a Period in units of days (I think!), > and sure it apparently does that right. No, it isn't a Period, and it doesn't "do that right". The current behavior _is_ incorrect (or at least lacking in internal coherence), and I don't think we can get clarity on what we want unless we acknowledge that. The discussion keeps getting sidetracked on the red herring of whether there are use cases for period arithmetic with "one day" defined as "same local time next day" as a period unit. Of course there are such use cases (and timedelta(1) can satisfy those use cases), but the current timedelta (as it behaves in arithmetic with tz-aware datetimes) is a hybrid that is not coherent considered either as a Period or a Duration. "Satisfying some use cases sometimes" is not sufficient for correctness; conceptual coherency matters too. Timedelta can only be considered a "Period object in units of days" if one accepts that the things it calls "hours", "minutes", and "seconds" are not really hours, minutes, or seconds, but rather fractional units of the "day-as-same-time-next-day" period that often (but not always) correspond to real hours, minutes, and seconds. I don't think this is a tenable explanation (and no one has attempted it), but you've just shown that it's one possible conclusion from defending the current model. Tim's valiant efforts notwithstanding, I don't think there is any coherent conceptual model that justifies the current behavior of timedelta. The _implementation_ can easily be explained of course (and Tim has done so very clearly, many times - I'd summarize it as "all arithmetic temporarily pretends all datetimes are naive, and then blindly reattaches the original tzinfo member"), but in terms of the underlying concepts, it makes no sense. In order to defend the current model as coherent, one has to discard one of the following points, and (despite having read every message in all the related threads), I am still not clear precisely which one of these Tim et al consider untrue or expendable: 1) A datetime with a tzinfo member that carries both a timezone and a specific UTC offset within that timezone (e.g. a pytz timezone instance) corresponds precisely and unambiguously to a single instant in astronomical time (as well as carrying additional information). 2) A timedelta object is clearly a Duration, not a Period, because timedelta(days=1), timedelta(hours=24), and timedelta(seconds=86400) result in indistinguishable objects. I think this point is uncontroversial; Tim has said several times that a timedelta is just a complicated representation of an integer number of microseconds. That's a Duration. 3) If one has two datetime objects that each unambiguously correspond to an instant in real time, and one subtracts them and gets back an object which represents a Duration in real microseconds, the only reasonable content for that Duration is the elapsed microseconds in real time between the two instants. Much virtual ink has been spilled over whether the behavior of "datetime + timedelta(days=1)" is correct, but this is an intentionally muddying case to consider, because there _are_ two perfectly reasonable interpretations of "add one day to a datetime". It's just that one of those interpretations (the Period one), which has been used to justify the current model, is inconsistent with _everything else_ about the behavior and implementation of timedelta. To be clear, I'm not arguing that this behavior can now be changed in the existing library objects in a backwards-incompatible way. But accepting that it is lacking in internal coherence (rather than just being an "alternative and equally good model") would be useful in clarifying what kind of an implementation we actually want (IMO, something very much like JodaTime/NodaTime). And then can we figure out how to get there from here. Carl

4 3

Re: [Datetime-SIG] How open are we to re-thinking the whole thing?
by Ethan Furman 31 Jul '15

31 Jul '15

Personally I think what we have is pretty good. We can shore up the cracks with a new tzinfo object to handle actual durations across time skips. We can add a datedelta to handle things like weeks and months and years -- if we need to. I think the first step is the new tzinfo, and after that it seems there are already a bunch of third-party modules to handle "human time" calculations. -- ~Ethan~

2 1

Re: [Datetime-SIG] Local time disambiguation proposal
by Tim Peters 31 Jul '15

31 Jul '15

[Alexander Walters] >> 1. no one will expect the datetime module is_dst to map to the posix mktime >> function. [Alexander Belopolsky] > Please don't overgeneralize. The fact that *I* would expect "is_dst" > in local to UTC converstion to act as the familiar "tm_isdst" does > in local to "seconds since 1970-01-01T00:00 UTC" > conversion is enough to invalidate your claim about "no one". I also > suspect I am not alone. You're far from alone, but there's no need to argue the point: you already won ;-) That is, Guido already Pronounced on this: https://mail.python.org/pipermail/datetime-sig/2015-July/000094.html ... The road to improvement that I encourage everyone to explore: add an is_dst flag (with a different name) ... And you've already explained why at length: the intended meaning of tm_isdst is baffling in some cases, so we do want different semantics, but because CPython is intended to work nicely with C code written by C programmers it's always a mistake to use similar names in similar contexts when the semantics differ in fundamental ways. "first" is a good solution: not confusable with any familiar name in C's view of time, and its meaning is both sensible and utterly clear in all ambiguous cases. At least until the next C standard adds tm_first to mean "equal to tm_isdst except in rare cases of double (or greater) leap seconds, in which case the value is as specified by the Finely Interpolated Real Second Time (FIRST) standard" ;--)

1 0

How open are we to re-thinking the whole thing?
by Chris Barker 31 Jul '15

31 Jul '15

It seems we've all come to the conclusion that there are three types of arithmetic here -- we can discuss until the cows come home about how useful "classic" (what is implemented in datetime now) arithmetic is, and exactly how it behaves in various corner cases, but it is here to stay for backward compatibility, and if we want either of the other two ("Duration" and "Period"), then we'll need to write implementations of them. So the question I have is: do we want to re-use as much of the current API and existing objects as possible, or should we start "from scratch", and design what we want/need now, with all the benefits of hindsight? For example: Duration arithmetic is actually pretty easy (or am I just being "naive" ;-) ): - convert to UTC - do the arithmetic - convert back to the time zone desired. Done. Granted, that's only easy if the "convert" part is done, but once we work out the nuances of the "is_dst" flag, then that's done. Period arithmetic is theoretically easy, but full of corner cases that have to be resolved, one by one... I think the real challenge here is how to get these two while keeping maximum compatibility with the current objects and API. So maybe we shouldn't worry so much about that. "the right > thing" to do in Python's current state is to stick to UTC for every > operation apart from conversions for input or output. Exactly -- and "the right thing to do" to properly support duration arithmetic is for that to be what happens under the hood. Which, brings up a point: If new code is going to be written, and new APIs defined, then maybe a re-thinking of the datetime data structure is in order: As Tim points out, both datetime and timedelta are really just fancy ways to encode milliseconds. They make it easy to work with the calendar representation version of these quantities -- i.e. on I/O -- because that's what humans want to work with. Also, the method of attaching the tzinfo object makes it easy to create and read and write datetimes in a given time zone. But if one were to design a system optimized for a different purpose, i.e. to support Duration arithmetic, it makes more sense to: * store datetimes in a "time_unit_since an epoch" representation (e.g. milliseconds since year 1 in the proleptic gregorian calendar) * store timedeltas in a "time_unit" (e.g. milliseconds) and (somewhat independent): * store timezone aware datetimes in UTC, and do the conversion to/from calendar representation on I/O. [that makes Duration addition described above reduce to: "add two numbers"] On the other hand, if you want to support "Period Arithmetic" (aka Calendar Operations), then it's easier to use Calendar encoding internally -- after all, "moving to the next day", (or next month, or...) is natural and easy (except for the edge cases!) in Calendar encoding. All this points to having two independent systems for the two types of math: two datetime objects two timedelta objects That makes it easy to decide what "type" of delta to return when subtracting two dates, and easy for the user to know what type of arithmetic they are going to get. Also, it could be in a datetime2 package, and have fewer issues with backward compatibility and make it clear to users that they are getting something different than the existing datetime package. Downsides: * More code to implement and maintain -- always a bad thing * Harder for users that need to do both kinds of arithmetic in one application -- if that's a common use case, then one datetime object with multiple deltas might be better. And the datetime object would have to be optimised for one or the other use-case -- so be it. Another totally side note about API. Particularly for Period arithmetic, the timedelta API feels a bit "OO-heavy" to me: If I want "tomorrow", I need to do: DateTime.now() + PeriodDelta(days=1) I think I'd rather something like: DateTime.now().next_day() or DateTime.now().moveforward("1 day") And if the Period arithmetic is in methods, then we don't have to worry about what kind of delta to return when subtracting. Just a thought....... (side question: It seems to me that doing the Period operations one way is pretty straightforward e.g. "add a day", but is it well defined in the other direction? e.g. what is the Period between datetime1 and datetime2?) -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

1 0

Re: [Datetime-SIG] Local time disambiguation proposal
by Alexander Walters 31 Jul '15

31 Jul '15

1. no one will expect the datetime module is_dst to map to the posix mktime function. If they expect a mapping to the posix api, they are using the time module. I think you are the only one who thought that. Just because another api uses a clear term with one set of semantics, does not mean we cant use the clear term with a different set of semantics. 2. "Is this instance of time DST? Oh, it has a member attribute 'is_dst', let me print that and see..." "I need to tell python this time I am inputting is DST. ...I will pass True to the is_dst argument" "I am in the northern hemisphere and need to construct a date in the middle of july. My nation-state observes DST. ...what the hell do i do with a 'first' argument?" On 7/30/2015 14:33, Alexander Belopolsky wrote: > On Thu, Jul 30, 2015 at 2:15 PM, Alexander Walters > <tritium-list(a)sdamon.com> wrote: >> We are not talking about implementing the POSiX argument to mktime, and I >> dont think anyone on the list ever was. .is_dst is a bool flag for 'if set, >> the time represented is in the DST time, if not set, it is in the non-DST >> time'. >> >> '.first' doesnt even imply a bool. "First what?" someone might ask. > I have no emotional attachment to any particular name. If you like > "is_first" better than "first" on the grammatical grounds - I have no > problem changing the spelling. > > I do have two problems with calling it isdst: > > 1. Whether we want to implement the POSIX standard or not, but > tm_isdst member is the POSIX solution to the problem at hand and if we > give datetime objects a member variable called isdst, but change the > semantics, we will see no end of bug reports. (And since no one > understands POSIX semantics, we have no choice but to change them. :-) > > 2. It will be very confusing to have t.is_dst ≠ t.dst() which will > happen whenever you have an ambiguity due to a change in the standard > time.

4 4

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Guido van Rossum 31 Jul '15

31 Jul '15

[re-adding the list] On Fri, Jul 31, 2015 at 6:21 PM, Łukasz Rekucki <lrekucki(a)gmail.com> wrote: > On 31 July 2015 at 18:02, Guido van Rossum <guido(a)python.org> wrote: > > On Fri, Jul 31, 2015 at 5:15 AM, Chris Barker <chris.barker(a)noaa.gov> > wrote: > >> > >> On Thu, Jul 30, 2015 at 6:09 PM, Ethan Furman <ethan(a)stoneleaf.us> > wrote: > >>> > >>> On 07/30/2015 06:04 PM, Tim Peters wrote: > >>> > >>>> I like Alexander's new tzstrict (subclass of tzinfo) idea best so far. > > > > > > I like it too. > > What happens then when you substract a datetime with *strict* tzinfo > and a *naive* one? Would A - B == - (B - A) still be true ? That's for the authors of the new PEP to decide, really, but I think it could be made to follow the strict rules in both cases, since clearly the code isn't an old program requiring backward compatibility (how would such a program end up with a strict tzinfo?). -- --Guido van Rossum (python.org/~guido)

1 0

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Alexander Belopolsky 31 Jul '15

31 Jul '15

On Fri, Jul 31, 2015 at 12:02 PM, Guido van Rossum <guido(a)python.org> wrote: > On Fri, Jul 31, 2015 at 5:15 AM, Chris Barker <chris.barker(a)noaa.gov> wrote: >> >> On Thu, Jul 30, 2015 at 6:09 PM, Ethan Furman <ethan(a)stoneleaf.us> wrote: >>> >>> On 07/30/2015 06:04 PM, Tim Peters wrote: >>> >>>> I like Alexander's new tzstrict (subclass of tzinfo) idea best so far. > > > I like it too. Thanks for the kind words, Tim, Ethan and Guido! I've also wondered about a possibility to delegate arithmetics to the timezone class altogether. For example, we could have a rule like this: def __add__(self, delta): if hasattr(self.tzinfo, 'add'): return self.tzinfo.add(self, delta) This, in theory, is powerful enough to support leap seconds if we represent 23:59:60.x as time(23, 59, 59, x, first=False).

1 0

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Guido van Rossum 31 Jul '15

31 Jul '15

On Fri, Jul 31, 2015 at 5:15 AM, Chris Barker <chris.barker(a)noaa.gov> wrote: > On Thu, Jul 30, 2015 at 6:09 PM, Ethan Furman <ethan(a)stoneleaf.us> wrote: > >> On 07/30/2015 06:04 PM, Tim Peters wrote: >> >> I like Alexander's new tzstrict (subclass of tzinfo) idea best so far. >>> >> I like it too. > hmm -- somehow the tz object really feels like the wrong place for it. > > If you don't like flags, > If we're talking about a *global* flag, yes, that's a horrid idea. > why not a datetime subclass? -- is that any more ugly than another tzinfo > subclass? > Most code creates datetime instances through its constructor. But most code creates tzinfo instances through factory functions (usually in 3rd party libraries, since the stdlib has no tz database). I'm guessing there are way more places where datetime() is called than places where a tzinfo is created. So yes, a datetime subclass is uglier. > We should also think about how to handle the Period arithmetic case -- > would that be a special kind of tzinfo object? I would think not -- after > all, you'd certainly need a different kind of timedelta object. It would be > nice not to have too many new objects. > If needed, we could create as many alternative time-delta-ish classes as we wanted, and through the magic of Python's binary operator overloading (in particular __radd__ and __rsub__) we could have each time-delta-ish class implement a different kind of arithmetic on vanilla datetime objects. -- --Guido van Rossum (python.org/~guido)

1 0

Re: [Datetime-SIG] Clearing up terminology
by Tim Peters 31 Jul '15

31 Jul '15

Correcting an error, albeit an inconsequential one: [Tim] > - Since you start in Chicago standard time, I expect that (assuming > it's implemented) some spelling.of is_dst will record "no, it's not > DST" in your dt object. That will never change across the duration of > the program. There is no spelling of is_dst today. s/Chicago standard time/Chicago daylight time/ s/no, it's not/yes, it is/

1 0

Re: [Datetime-SIG] Clearing up terminology
by Tim Peters 31 Jul '15

31 Jul '15

[Guido] > .... > I assume you meant naive, not native. Always. In any context ;-) BTW, the only thing truly naive about datetime was how little time everyone expected its development to consume. 13 years later and a billion people are still arguing over how to finish implementing mathematically trivial timezone arithmetic ;-) > Yes, I meant to introduce human arithmetic as a third form, to support > adding e.g. months or years, while classic and strict arithmetic shouldn't. Got it. > (Since timedelta doesn't keep its days and seconds separated, it's not > really a Period, so I think it shouldn't be extended further. Absolutely not. It's just a funky way to spell a large integer (number of microseconds), and it's only in naive time that _any_ common named durations longer than an hour actually match a fixed number of microseconds. The only new argument I think we _could_ sanely add to the timedelta constructor is `fortnights=`. Well, that and `googoldays=`, except that would overflow for any non-zero value. So it goes: patch on the way! ;-) > And adding months is a requirement for human arithmetic -- e.g. I was > called a bad word on Twitter yesterday because timedelta doesn't support > months.) Thanks for telling me! It's another excellent reason to keep my life blissfully Twitter-free. >>>> (BTW, historically, early versions of Python datetime had a separate >>>> datetime subclass called datetimetz (IIRC), but the final design got >>>> rid of this unnecessary complexity.) >>> I don't believe that was ever released, but it's possible we >>> contemplated that and even implemented it in an early >>> pure-Python prototype. >> FWIW, IIRC it did exist in an early prototype but was discarded before >> it was even halfway implemented. > I stand corrected. Na. I could well be wrong about everything. Much of my few lucid moments are lost to bemoaning all I've forgotten. > My memory seems to be failing -- I also thought there was a datetime > PEP, Oops! It really is failing! There definitely was not. > but I guess we avoided the PEP process (even though datetime > was added in 2.3, well after PEP 1 was written) because we were > all working in the same office. (Or do I misremember that too? :-) Nobody could have made time to write a PEP. We were utterly drowning in "input" without one. Lots of discussion took place on a Zope-hosted public DateTime wiki. Yesterday I tried pretty hard to find it without success - I guess it's lost now. Also enormous wads of Python and Zope mailing list traffic. And what ended up being the text of the first version of the manual docs was written by me in a markup-free plain text file that was checked in and publicized frequently, kept roughly in synch with the ever-changing Python prototype and test suite. None of those three ever get "too far" ahead of the others. So everyone interested got plain-text docs, a prototype, and a test suite, all along the way, plus wiki and multiple mailing list access for questions, requests and discussions. Besides, it was your project. There wasn't much suspense over whether it would be accepted ;-)

1 0