Mailman 3 July 2015 - Datetime-SIG

Re: [Datetime-SIG] What's are the issues?
by Lennart Regebro 29 Jul '15

29 Jul '15

Tim Peters tim.peters at gmail.com: > But, as the docs have always pointed out, it (specifically .fromutc(), > which is at the base of all relevant calculations) doesn't even try to > handle transitions due to _other_ causes. The only other such cause > I'm aware of is when a timezone's _base_ offset from UTC changes (not > a temporary DST transition, the timezone's "standard" offset changes > permanently (meaning until someone decides to change it again)). Wait, what? This will be handled in the exact same way as DST changes, and if this is incorrect, then reasonably, so is the DST handling. What is "incorrect" here is of course a matter of opinion. I think it's incorrect because it doesn't agree with reality, you have claimed that it is correct, in "as per design". But then reasonably that goes for this case as well? The same thing happens, which is that depending on the transitions direction, you can either end up in a time that doesn't exist, or one that happens twice. //Lennart

1 0

Re: [Datetime-SIG] What's are the issues?
by Lennart Regebro 29 Jul '15

29 Jul '15

Chris Barker chris.barker at noaa.gov > What are the issues still to be resolved? [with the PEP] The is_dst flags added are not implementable, as they aim to make it possible to do correct timezone conversions, and Guido and Tim have declared that they don't want that, but that they want to keep the arithmetic what they call "naive". I'll send in a patch to withdraw PEP-431 it later today, I hope. //Lennart

1 0

Re: [Datetime-SIG] Calendar vs timespan calculations...
by Lennart Regebro 29 Jul '15

29 Jul '15

Sorry, catching up with the posts, here, as the mailing list was created when I was asleep. So I'm cutting and pasting from the archives... Chris Barker chris.barker at noaa.gov: > [that a day is always 24 hours] it's not an assumption, it's a definition. Right, but that's like defining Pi to be three. It's not in accordance to reality, and therefore it causes a lot of problems. > What is currently supported by the datetime module, and I think Tim is > referring to as "naive" time operations: adding, subtracting, units of time > along the theoretical time axis. Well, that is supported, yes, but only for datetimes that does not have a time zone, or that has a time zone with a fixed offset. It is NOT supported for datetimes with a time zone of the type that people typically live in: One whose offset changes from time to time. In practice, pretty much all of them except UTC. That means if you want to do these operations, you have to first convert the datetime to UTC, then do the arithmetic and then convert it back. > Then there are what I am calling "calendar" operations -- these are > operations that only make sense with a calendar (and, in fact a timezone, I > think). this is operations like: "the same time two days later" -- this is > not the same as moving two days (48 hrs) along the time axis -- it simply > is not. Right. These operations *are* in fact supported, but in a limited fashion. There is no support for months or years, notably. You can add days, hours, minutes and seconds, but you have to be aware then that you can end up with datetimes that does not actually exist, or are ambiguous, ie times that happens twice. I wish it was as you assumed it was, because that would have been much easier to fix, as you say, we can have add the "calendar" operations into a separate package, and yes, that is indeed what dateutil's "relativedelta" does, amongst some other things. It might be possible to add a real time delta in a separate package as well, I haven't looked into that. > Given all the ugliness of DST and changing time zones, and all that, UTC is > the lingua franca of time -- time zones are defined by how the are offset > from UTC, and in UTC (as in naive), math is relatively easy. So the best > way to handle time zones is to store and manipulate everything in UTC, and > then convert to/from the calendar representation using the time zone, when > the user needs (or provides). Yes, that would indeed have been the simple and obvious way to implement it, but datetime stores everything in the local zone, which makes it a lot more complex, as you get things like ambiguous times and non-existent times when you do that. //Lennart

1 0

Re: [Datetime-SIG] The BDFL's take
by Guido van Rossum 29 Jul '15

29 Jul '15

One more thing. The calendar module should probably be left alone, but its timegm() function should be moved to the time module. That was a very unfortunate API choice. -- --Guido van Rossum (python.org/~guido)

1 0

Re: [Datetime-SIG] [Python-ideas] Additional datetime-like module
by Guido van Rossum 29 Jul '15

29 Jul '15

Oops, let's move this to datetime-sig. I've reposted this there under the subject "The BDFL's take". On Wed, Jul 29, 2015 at 12:20 PM, Guido van Rossum <guido(a)python.org> wrote: > Honestly, I think some of these proposals are way too ambitious and will > drown in a sea of bike-shedding. > [...] > -- --Guido van Rossum (python.org/~guido)

1 0

The BDFL's take
by Guido van Rossum 29 Jul '15

29 Jul '15

[I posted this to python-ideas, but then found that this SIG was created. So let's discuss it here.] Honestly, I think some of these proposals are way too ambitious and will drown in a sea of bike-shedding. Here's what I think is reasonable: - Keep the time module; POSIX timestamps represented as floats are still very useful (and ubiquitous in other APIs), and it would take decades to deprecate. - Keep the datetime module; it has its uses (and even fans), and again, deprecating would take decades. - Add a few convenience APIs to one or both of the above for better conversion between various forms of datetime and POSIX timestamps (I think there are a few missing APIs in this area). - Design a new module with a separate API for "human time manipulation". This is where operations belong like "add 1 month" or "add 1 day" with heuristics for edge cases (e.g. months of different lengths due to calendar conventions, and the occasional short or long day due to DST). Possibly this can borrow some API and/or implementation from pytz. Possibly it could be made available as a 3rd party module on PyPI first, to get it user-tested in quick iterations (quicker than core Python releases, anyway). But that doesn't preclude also writing a PEP. This module needs to interface somehow with the datetime module (and possibly also with the time module) in ways that need some serious thinking -- that's where I expect the hard part of writing the PEP will be. (Maybe a "human timedelta" type would be useful here? Eventually this might even be added to the datetime module.) There are a few additional issues which I'm not sure are included in this discussion or not: - A unified API for accessing an up-to-date timezone database. This has been discussed in the past, and we couldn't find a solution that satisfied all requirements (the main problem being different platform expectations IIRC), but it's probably worth it trying again. - Nanosecond precision? I'm not against adding this to the datetime module, as long as backward compatibility is somehow preserved, especially for pickles (maybe nanoseconds default to zero unless explicitly requested or specified). - An "is_dst" flag for datetime objects. Again, this should default to "backwards compatible" (== "don't care", the -1 value in struct tm) so that pickles are two-way compatible with Python 3.5 and before. -- --Guido van Rossum (python.org/~guido)

1 0

Re: [Datetime-SIG] What's are the issues?
by Chris Angelico 29 Jul '15

29 Jul '15

On Wed, Jul 29, 2015 at 1:13 PM, Tim Peters <tim.peters(a)gmail.com> wrote: > [Chris Angelico <rosuav(a)gmail.com>] >> What if the existing timedelta is left as-is, > > That much isn't a "what if" - that much is a requirement ;-) Heh, sure. Vagaries of the English language sometimes result in an apparent suggestion like this :) >> and a new calendardelta type is introduced? Instead of folding >> everything down to a simple microsecond count, it would retain >> the exact values it was given, >> so calendardelta(days=1) != calendardelta(seconds=86400). >> When you add a calendardelta to an aware datetime, it performs >> what you're describing as calendar arithmetic > > That's what the _current_ timedelta already does (things like "same > time tomorrow", "same time in 25000 weeks", and "same time plus an > hour 57 days ago" are all trivially done already by datetime + > timedelta operations). Except that it isn't possible to distinguish "add 86400 seconds" from "add 1 day". It's fine for adding to a naive datetime, as they're equivalent, but they differ with aware datetimes - hence the notion of having a different way to add to them. The semantics would be that adding any unit to an aware datetime *will not change* any unit of finer resolution, unless the result would otherwise be impossible to represent. So when you add 1 day to 9AM today, you get 9AM tomorrow; but when you add 24 hours to 9AM today, you might get 10AM tomorrow, because one hour got snipped out of the intervening night. Similarly, adding 1 year to Feb 23rd 2012 will result in Feb 23rd 2013 (has added 366 days), but adding 1 year to Feb 23rd 2013 will add only 365 days; this is a concept that the current timedelta simply can't handle, as it doesn't work in any unit larger than the day. There is one edge case that I'm aware of, and that's where the simple calculation results in an invalid timestamp - either because the clock jumps forward (DST switch) or because you would land on a leap day. In each case, it's possible only when you're moving in a unit greater than the one that's being adjusted - you can't land on Feb 29th 2015 by moving X days, because it'll count days right past it, so you must have been moving months or years. When you move forward, the obvious thing is to jump forward by a distance exactly equal to the gap - one extra day to land on March 1st, or one extra hour to land at 3AM - but I'm not sure what the best solution is when moving backward. Are there any other nasty edge cases? ChrisA

1 0

Re: [Datetime-SIG] Datetime arithmetic proposal
by Alexander Belopolsky 28 Jul '15

28 Jul '15

On Tue, Jul 28, 2015 at 10:45 PM, Tim Peters <tim.peters(a)gmail.com> wrote: > But, > internally,datetimes, dates, times, and timedeltas are all sometimes > viewed as just complicated ways of spelling "some number of > microseconds". While obvious when you look from the inside of the implementation, this is something that datetime library users will never understand. That's why over my strong objections, timedelta grew its total_seconds() method which is nothing else but x / timedeta(seconds=1). After that, I gave up and committed datetime.timestamp() as a shortcut for (x - EPOCH) / timedeta(seconds=1).

1 0

Re: [Datetime-SIG] What's are the issues?
by Tim Peters 28 Jul '15

28 Jul '15

[Chris Angelico <rosuav(a)gmail.com>] > What if the existing timedelta is left as-is, That much isn't a "what if" - that much is a requirement ;-) > and a new calendardelta type is introduced? Instead of folding > everything down to a simple microsecond count, it would retain > the exact values it was given, > so calendardelta(days=1) != calendardelta(seconds=86400). > When you add a calendardelta to an aware datetime, it performs > what you're describing as calendar arithmetic That's what the _current_ timedelta already does (things like "same time tomorrow", "same time in 25000 weeks", and "same time plus an hour 57 days ago" are all trivially done already by datetime + timedelta operations). What Lennart is on about is that there's no direct way to spell "the other" kind of arithmetic now (operationally defined by: convert to UTC first, then do the "naive arithmetic", and finally convert back to the original timezone). But it's also straightforward to write a simple function to do that, and not every simple function has to be built in. > and it will be the most plausibly sane behaviour for the exact unit used; so, > for instance, adding calendardelta(days=1) would never affect the hour/minute/second > shown, unless it would otherwise create an illegal time. Would that serve? Note that timedelta doesn't accept any units for which there are any questions about what a "calendar operation" "should do" in Guido's "naive time" design. That's also an intentional part of the datetime design. So. e.g., you can ask a timedelta for some number of days or weeks, but _not_ for some number of months or years. "Why not years?" is because there's one case where what to do isn't 100% obvious to everyone: a year starting from a leap year's 29 Feb. "Why not a month?" has many more questionable cases. In any case, you should look at other Python date-&-time libraries (like mxDateTime and dateutil) because there are already approaches to richer "calendar options" fully thought out.

1 0

Re: [Datetime-SIG] What's are the issues?
by Chris Angelico 28 Jul '15

28 Jul '15

On Wed, Jul 29, 2015 at 12:02 PM, Chris Barker <chris.barker(a)noaa.gov> wrote: > On Tue, Jul 28, 2015 at 6:43 PM, Chris Angelico <rosuav(a)gmail.com> wrote: > >> What if the existing timedelta is left as-is, and a new calendardelta >> type is introduced? Instead of folding everything down to a simple >> microsecond count, it would retain the exact values it was given, > > > sure -- that would be one way to do it -- maybe good way. But it would be a > lot more complicated than just keeping days. Lots to hash out for API and > functionality at the corner cases (the old what's one year after Feb 29th? > kinds of questions) I'd say "one year after Feb 29th" is exactly the same question as "one day after 2:30AM before a DST switch", and has the same answer (add another day, or another hour). >> When you add a >> calendardelta to an aware datetime, it performs what you're describing >> as calendar arithmetic, > > actually, no reason you couldn't do calendar manipulations on naive > datetimes, too (they wouldn't get any ugly DST issues, but otherwise the > same). One reason I don't like calling the arithmetic naive or not.... Yes, but when you do them on naive datetimes, they'll be indistinguishable from the existing behaviour, and thus not a compelling argument for a new timedelta variant. I agree though - it wouldn't be an error to add a calendardelta to a naive datetime. >> adding >> calendardelta(days=1) would never affect the hour/minute/second shown, >> unless it would otherwise create an illegal time. > > > exactly -- all those corner cases would need to be worked out. Simplest definition would be what we want here. Jump forward by another unit of whatever one has disappeared. What if we're moving backward? If you subtract a year from Feb 29th 2012, do you get Feb 28th 2011 or Mar 1st 2011? But I think adding a year should produce Mar 1st 2013. ChrisA

1 0