The datetime module underwent a number of last-minute changes to make writing tzinfo classes a pleasure instead of an expert-level nightmare (which latter is what it turned to be -- it's not anymore). Realistic examples can be found in the Python CVS datetime sandbox, at http://tinyurl.com/3zr7 US.py Some US time zones EU.py Some European time zones PSF.py Essential if you're a PSF Director <wink> dateutil.py Examples of using the datetime module to compute other kinds of practical stuff (like the 2nd-last Tuesday in August) We still have two time zone conversion problems, and always will (they come with the territory -- they're inherent problems, not artifacts of the implementation). They only arise in a tzinfo class that's trying to model both standard and daylight time. In effect, that's one class trying to pretend it's two different time zones, and at the transition points there are "impossible problems". For concreteness, I'll use US Eastern here, UTC-0500. On some day in April, DST starts at the minute following the local wall clock's 1:59. On some day in October, DST ends at the minute following the local wall clock's 1:59. Here's a picture: UTC 3:MM 4:MM 5:MM 6:MM 7:MM 8:MM standard 22:MM 23:MM 0:MM 1:MM 2:MM 3:MM daylight 23:MM 0:MM 1:MM 2:MM 3:MM 4:MM wall start 22:MM 23:MM 0:MM 1:MM 3:MM 4:MM wall end 23:MM 0:MM 1:MM 1:MM 2:MM 3:MM UTC, EST and EDT are all self-consistent and trivial. It's only wall time that's a problem, and only at the transition points: 1. When DST starts (the "wall start" line), the wall clock leaps from 1:59 to 3:00. A wall time of the form 2:MM doesn't really make sense on that day. The example classes do what I believe is the best that can be done: since 2:MM is "after 2" on that day, it's taken as daylight time, and so as an alias for 1:MM standard == 1:MM wall on that day, which is a time that does make sense on the wall clock that day. The astimezone() function ensures that the "impossible hour" on that day is never the result of a conversion (you'll get the standard-time spelling instead). If you don't think that's the best that can be done, speak up now. 2. When DST ends (the "wall end" line), we have a worse problem: there'a an hour that can't be spelled *at all* in wall time. It's the hour beginning at the moment DST ends; in the example, that's times of the form 6:MM UTC on the day daylight time ends. The local wall clock leaps from 1:59 (daylight time) back to 1:00 again (but the second time as a standard time). The hour 6:MM UTC looks like 1:MM, but so does the hour 5:MM UTC on that day. A reasonable tzinfo class should take 1:MM as being daylight time on that day, since it's "before 2". As a consequence, the hour 6:MM UTC has no wall-clock spelling at all. This can't be glossed over. If you code a tzinfo class to take 1:MM as being standard time on that day instead, then the UTC hour 5:MM becomes unspellable in wall time instead. No matter how you cut it, the redundant spellings of an hour on the day DST starts means there's an hour that can't be spelled at all on the day DST ends (so in that sense, they're the two sides of a single problem). What to do? The current implementation of dt.astimezone(tz) raises ValueError if dt can't be expressed as a local time in tz. That's the "errors should never pass silently" school, which I briefly attended in college <wink>. If you don't like that, what would you rather see happen? Try to be precise, and remember that getting a "correct" time in tz is flatly impossible in this case. Two other debatable edge cases in the implementation of dt.astimezone(tz): 3. If dt.tzinfo.dst(dt) returns None, the current implementation takes that as a synonym for 0. Perhaps it should raise an exception instead. 4. If dt.tzinfo.utcoffset(dt) first returns an offset, and on a subsequent call (while still trying to figure out the same conversion)returns None, an exception is raised. Those who don't want unspellable hours to raise an exception may also want inconsistent tzinfo implementations to go without complaint. If so, what do you want it to do instead?
On Wed, Jan 01, 2003, Tim Peters wrote:
This can't be glossed over. If you code a tzinfo class to take 1:MM as being standard time on that day instead, then the UTC hour 5:MM becomes unspellable in wall time instead. No matter how you cut it, the redundant spellings of an hour on the day DST starts means there's an hour that can't be spelled at all on the day DST ends (so in that sense, they're the two sides of a single problem).
What to do? The current implementation of dt.astimezone(tz) raises ValueError if dt can't be expressed as a local time in tz. That's the "errors should never pass silently" school, which I briefly attended in college <wink>. If you don't like that, what would you rather see happen? Try to be precise, and remember that getting a "correct" time in tz is flatly impossible in this case.
What exactly do you mean by "wall time"? Is it supposed to be a timezone-less value? If yes, you can't convert between wall time and any timezone-based value without specifying the time zone the wall clock lives in, which has to be relative to UTC. This only becomes a serious problem for round-trip conversions. Does it make any sense to include a flag if a time value ever gets converted between wall time and any other kind of time? Every other datetime package seems to live with the wiggle involved in round-trip conversions, why not Python? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are three kinds of lies: Lies, Damn Lies, and Statistics." --Disraeli
[Aahz]
What exactly do you mean by "wall time"? Is it supposed to be a timezone-less value?
The diagram in the original gave agonizingly detailed answers to these questions. US Eastern as a tzinfo class tries to combine both EDT and EST; please read the original msg again, and note that the two "wall time" lines in the diagram show what US Eastern displays at all the relevant UTC times.
If yes, you can't convert between wall time and any timezone-based value without specifying the time zone the wall clock lives in, which has to be relative to UTC.
See the original; the relationship to UTC was explicit at all points there.
This only becomes a serious problem for round-trip conversions.
The major problem here was in one-way conversion; see the original; two-way conversion T1 -> T2 -> T1 is impossible if the time in T1 can't be represented at all in T2 (which is the case here).
Does it make any sense to include a flag if a time value ever gets converted between wall time and any other kind of time?
Every other datetime package seems to live with the wiggle involved in round-trip conversions, why not Python?
Round-trip conversions aren't at issue here, except to the extent that the "unspellable hour" at the end of DST makes some one-way conversions impossible.
On Wed, Jan 01, 2003, Tim Peters wrote:
[Aahz]
What exactly do you mean by "wall time"? Is it supposed to be a timezone-less value?
The diagram in the original gave agonizingly detailed answers to these questions. US Eastern as a tzinfo class tries to combine both EDT and EST; please read the original msg again, and note that the two "wall time" lines in the diagram show what US Eastern displays at all the relevant UTC times.
Ah! Okay, didn't read the code and didn't realize that "US Eastern" was a concrete representation.
Every other datetime package seems to live with the wiggle involved in round-trip conversions, why not Python?
Round-trip conversions aren't at issue here, except to the extent that the "unspellable hour" at the end of DST makes some one-way conversions impossible.
I'm not sure I agree. As I see it, "wall time" is for users. On the display side, I believe that users *expect* to see 1:59:57 am 1:59:58 am 1:59:59 am 1:00:00 am 1:00:01 am I therefore see no problem with the UTC->wall clock conversion. Going the other direction requires an explicit statement of which timezone you're in at the point of conversion (a real timezone, not a virtual one like "US Eastern"). Presumably that only occurs as a result of user input, and when you redisplay the input as a wall clock, it should be obvious to the user if the wrong time zone was selected because the time will be an hour (or whatever) off. The only way this is a problem seems to be if you want to do round-trip conversions purely programmatically. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are three kinds of lies: Lies, Damn Lies, and Statistics." --Disraeli
[Aahz]
I'm not sure I agree. As I see it, "wall time" is for users. On the display side, I believe that users *expect* to see
1:59:57 am 1:59:58 am 1:59:59 am 1:00:00 am 1:00:01 am
I therefore see no problem with the UTC->wall clock conversion.
That's defensible for some class of uses, although it may not be implementable. The tzinfo classes here are arbitrary pieces of user-written Python code, and all astimezone() can do is (1) build objects and pass them to its methods, to see what they return; and, (2) make assumptions. In particular, astimezone() has no knowledge of when DST begins or ends, or even of whether a tzinfo class believes there *is* such a thing as DST. In this case it can detect the unspellable hour, so I suppose it could add more code trying to infer what the local clock believes about it.
Going the other direction requires an explicit statement of which timezone you're in at the point of conversion (a real timezone, not a virtual one like "US Eastern").
As others have pointed out, C's struct tm tm_isdst flag serves this purpose for "US Eastern", and such a flag *could* be added to datetimetz objects too. The practical reality appears to be that people want hybrid classes like this, ill-defined or not.
Presumably that only occurs as a result of user input, and when you redisplay the input as a wall clock, it should be obvious to the user if the wrong time zone was selected because the time will be an hour (or whatever) off. The only way this is a problem seems to be if you want to do round-trip conversions purely programmatically.
If you view Guido's appointment calendar over the web, which he enters in US Eastern these days, and want it displayed in your local time, then (a) there's nothing roundtrip about it -- it's a one-way conversion; and (b) you'll have no idea whether some items are an hour off. For a web-based group scheduling application, this isn't even a stretch.
On Thu, Jan 02, 2003, Tim Peters wrote:
[Aahz]
I'm not sure I agree. As I see it, "wall time" is for users. On the display side, I believe that users *expect* to see
1:59:57 am 1:59:58 am 1:59:59 am 1:00:00 am 1:00:01 am
I therefore see no problem with the UTC->wall clock conversion.
That's defensible for some class of uses, although it may not be implementable. The tzinfo classes here are arbitrary pieces of user-written Python code, and all astimezone() can do is (1) build objects and pass them to its methods, to see what they return; and, (2) make assumptions. In particular, astimezone() has no knowledge of when DST begins or ends, or even of whether a tzinfo class believes there *is* such a thing as DST.
In this case it can detect the unspellable hour, so I suppose it could add more code trying to infer what the local clock believes about it.
It sure sounds like you're saying that the DateTime module can't handle timezone conversions at all. If it can, I don't understand what extra ambiguity (in human terms) is introduced by DST, as long as there are no round-trip conversions.
Presumably that only occurs as a result of user input, and when you redisplay the input as a wall clock, it should be obvious to the user if the wrong time zone was selected because the time will be an hour (or whatever) off. The only way this is a problem seems to be if you want to do round-trip conversions purely programmatically.
If you view Guido's appointment calendar over the web, which he enters in US Eastern these days, and want it displayed in your local time, then (a) there's nothing roundtrip about it -- it's a one-way conversion; and (b) you'll have no idea whether some items are an hour off. For a web-based group scheduling application, this isn't even a stretch.
I'm confused: are you saying that Guido's calendar doesn't get internally converted to UTC? If not, then the chart from your message starting this thread doesn't apply, because here you're talking about converting from wall clock to wall clock, which I think is a conversion that makes no sense -- that perhaps would be a suitable occasion for an exception. If Guido's appointment does get converted to UTC, then there's always a sensible (if possibly ambiguous) conversion to wall time, and if he looks at his calendar, he should be able to see if his appointment got converted to UTC correctly. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are three kinds of lies: Lies, Damn Lies, and Statistics." --Disraeli
I'm confused: are you saying that Guido's calendar doesn't get internally converted to UTC?
That's correct. I use a Palmpilot, which has no knowledge of timezones or DST at all. In fact, I write down appointments in whatever local time I will be in on a specific date, i.e. if I were to fly to California to meet with you at 1pm on some date, I'd put that down for 1pm on that date; on the plane I'll set the Palmpilot's clock back three hours. I've heard of a calendar application on Windows (Outlook?) that records all your appointments in UTC and displays them in local time. That means that when you are temporarily in a different timezone and you change your computer's clock to match local time, all your appointments are displayed in that local time -- which means that if you want to look up an appointment next week when you're back home, it is shown wrong! (E.g. a 1pm appointment in Washington DC would show up as 10am when the user is in San Francisco -- very confusing when you're in SF talking on the phone to a potential customer on the east coast!)
If not, then the chart from your message starting this thread doesn't apply, because here you're talking about converting from wall clock to wall clock, which I think is a conversion that makes no sense -- that perhaps would be a suitable occasion for an exception.
You can get from wall clock + tzinfo to UTC quite easily using the utcoffset() method of the tzinfo -- that's what it is for. So wall clock to wall clock conversion reduces to UTC to wall clock conversion anyway.
If Guido's appointment does get converted to UTC, then there's always a sensible (if possibly ambiguous) conversion to wall time, and if he looks at his calendar, he should be able to see if his appointment got converted to UTC correctly.
No -- why would I care about UTC or even know the UTC offset of my wall clock time? If I make an appointment for early next April, why should I have to know whether DST applies or not at that particular date? --Guido van Rossum (home page: http://www.python.org/~guido/)
On Thu, Jan 02, 2003, Guido van Rossum wrote:
Aahz:
I'm confused: are you saying that Guido's calendar doesn't get internally converted to UTC?
That's correct. I use a Palmpilot, which has no knowledge of timezones or DST at all. In fact, I write down appointments in whatever local time I will be in on a specific date, i.e. if I were to fly to California to meet with you at 1pm on some date, I'd put that down for 1pm on that date; on the plane I'll set the Palmpilot's clock back three hours.
I've heard of a calendar application on Windows (Outlook?) that records all your appointments in UTC and displays them in local time. That means that when you are temporarily in a different timezone and you change your computer's clock to match local time, all your appointments are displayed in that local time -- which means that if you want to look up an appointment next week when you're back home, it is shown wrong! (E.g. a 1pm appointment in Washington DC would show up as 10am when the user is in San Francisco -- very confusing when you're in SF talking on the phone to a potential customer on the east coast!)
All that is true and makes perfect sense -- but then Tim's comment about converting your calendar to someone else's wall clock makes no sense.
If not, then the chart from your message starting this thread doesn't apply, because here you're talking about converting from wall clock to wall clock, which I think is a conversion that makes no sense -- that perhaps would be a suitable occasion for an exception.
You can get from wall clock + tzinfo to UTC quite easily using the utcoffset() method of the tzinfo -- that's what it is for. So wall clock to wall clock conversion reduces to UTC to wall clock conversion anyway.
No, because there's no information about *which* wall clock is being used. As you yourself said above, you casually enter times for appointments in the local time that you expect to use them, without recording the extra information needed to make conversions.
If Guido's appointment does get converted to UTC, then there's always a sensible (if possibly ambiguous) conversion to wall time, and if he looks at his calendar, he should be able to see if his appointment got converted to UTC correctly.
No -- why would I care about UTC or even know the UTC offset of my wall clock time? If I make an appointment for early next April, why should I have to know whether DST applies or not at that particular date?
You don't -- but when your application displays wall time, you'd see a discrepency from the time you entered. This only applies when there's a consistent system of timezone information. You can't do date/time arithmetic and conversions with pure wall clock. Period. Python should raise exceptions if you try. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are three kinds of lies: Lies, Damn Lies, and Statistics." --Disraeli
[Aahz]
It sure sounds like you're saying that the DateTime module can't handle timezone conversions at all.
Nope, overall it does very well. Go back to the original msg: there are glitches during two hours per year, for tzinfo classes that try to model both daylight and standard time. That's it.
If it can, I don't understand what extra ambiguity (in human terms) is introduced by DST, as long as there are no round-trip conversions.
Sorry, I can't explain this any better than it's already been explained. There's no ambiguity in EST or EDT on their own, there is ambiguity twice a year in a single class that tries to combine both.
... I'm confused: are you saying that Guido's calendar doesn't get internally converted to UTC?
I'd say that whether it is is irrevelant to the example. Just as there are hours in UTC that can't be spelled unambiguously in US Eastern, there are hours in US Eastern than can't be spelled unambiguously in any other hybrid (daylight+standard) time zone. These problems have to do with the target time system, not with the originating time system.
If not, then the chart from your message starting this thread doesn't apply, because here you're talking about converting from wall clock to wall clock, which I think is a conversion that makes no sense
Of course it does: at any moment, you can call Guido, and if he answers the phone <wink> you can ask him what his wall clock says. He doesn't have to convert his local time to UTC to answer the question, and neither do you. There's "almost never" a problem with this, either.
-- that perhaps would be a suitable occasion for an exception.
It seems to depend on the app a person has in mind.
If Guido's appointment does get converted to UTC, then there's always a sensible (if possibly ambiguous) conversion to wall time, and if he looks at his calendar, he should be able to see if his appointment got converted to UTC correctly.
Fine, pretend Guido lives in Greenwich and UTC is his local time. The ambiguity remains, and it doesn't matter whether Guido can detect it: by hypothesis, *you're* the one looking at his calendar, and in your local time. If an appointment then shows up as starting at 1:00 in your local time on the day daylight time ends for you, you're most likely to believe that's your daylight time (assuming you live in the US and in an area that observes DST). It may or may not be in reality, and whether that matters depends on the use you make of the information.
On Thu, Jan 02, 2003, Tim Peters wrote:
[Aahz]
It sure sounds like you're saying that the DateTime module can't handle timezone conversions at all.
Nope, overall it does very well. Go back to the original msg: there are glitches during two hours per year, for tzinfo classes that try to model both daylight and standard time. That's it.
If it can, I don't understand what extra ambiguity (in human terms) is introduced by DST, as long as there are no round-trip conversions.
Sorry, I can't explain this any better than it's already been explained. There's no ambiguity in EST or EDT on their own, there is ambiguity twice a year in a single class that tries to combine both.
So there's ambiguity. So what? What I don't understand is why it's a problem. More precisely, I see these problems existing in the absence of computers, and I don't see where creating a Python DateTime class creates any more problems or makes the existing problems worse -- as long as you don't try to convert between timezones in the absence of sufficient information.
If not, then the chart from your message starting this thread doesn't apply, because here you're talking about converting from wall clock to wall clock, which I think is a conversion that makes no sense
Of course it does: at any moment, you can call Guido, and if he answers the phone <wink> you can ask him what his wall clock says. He doesn't have to convert his local time to UTC to answer the question, and neither do you. There's "almost never" a problem with this, either.
But that's not a conversion.
If Guido's appointment does get converted to UTC, then there's always a sensible (if possibly ambiguous) conversion to wall time, and if he looks at his calendar, he should be able to see if his appointment got converted to UTC correctly.
Fine, pretend Guido lives in Greenwich and UTC is his local time. The ambiguity remains, and it doesn't matter whether Guido can detect it: by hypothesis, *you're* the one looking at his calendar, and in your local time. If an appointment then shows up as starting at 1:00 in your local time on the day daylight time ends for you, you're most likely to believe that's your daylight time (assuming you live in the US and in an area that observes DST). It may or may not be in reality, and whether that matters depends on the use you make of the information.
[Aahz]
So there's ambiguity. So what?
That was my question in the orginal msg, and hasn't changed: there are problems at two points, they're inherent to the current design (so can't be wished away), but the current implementation could change what it does in these cases -- what do people want to see done?
What I don't understand is why it's a problem.
It depends on the app; like any other "surprise", depending on the app it may never be noticed, or may cost lives, or anything in between. Both you and Marc-Andre have been disturbed enough at the *possibilities* for bad consequences to claim that such cases should raise exceptions (although you seem to spend more energy arguing that there's no problem at all <wink>). Guido made a case on the other side, that once-a-year exceptions unlikely to be caught by testing carry a different kind of real risk.
More precisely, I see these problems existing in the absence of computers, and I don't see where creating a Python DateTime class creates any more problems or makes the existing problems worse --
I don't think any of that matters: an API needs to define what it does.
as long as you don't try to convert between timezones in the absence of sufficient information.
The current design doesn't allow the possibility for sufficient information in these cases. One plausible response to that is to insist that the current design is fatally flawed. Another is to shrug "so it goes", and define what it does do, as best as can be done.
... at any moment, you can call Guido, and ...
But that's not a conversion.
? It's clearly a way to convert your local time to Guido's local time. The only real difference is that when you call Guido at 6:30 UTC on the day daylight time ends for him, and he replies "oh, it's 1:30 here", you have the further possibility to ask him whether he meant 1:30 EDT or 1:30 EST. Apart from that, dt.astimezone(Eastern) will give you answers identical to his every time you try it (assuming you start with dt.tzinfo=Pacific (from US.py), that you still live in the Pacific time zone, and that you and Guido both scrupulously adjust your clocks at the politically defined DST transition times).
... From my POV, this problem exists regardless of whether a computer mediates the transaction.
Of course it does. but that doesn't excuse Python from defining its own behavior in these cases. Apart from that, the consequences of ambiguity in a program are often called "bugs", but the consequences of ambiguity in real life are merely called "real life" <0.9 wink>.
The most likely error (if one happens) is that someone shows up an hour early for the appointment, and presumably that person knows that the day is a DST transition.
This sounds like you don't want it to raise an exception, then.
Wow, this whole subject is still astonishing me. What an education. Two months ago, I would have thought that times were only complicated by UTC plus or minus a timezone and the international dateline. Also, I would have thought that dates were only complicated by 400 year cycles and the Julian/Gregorian/Mayan thing. Being a private pilot, ex-military, and having written perpetual calendar programs did not even suggest the complexity or depth of this subject. But, that was before watching the Timbot get ensnarled in 5600 lines of code development, testing, research, and documentation to create a little order out of chaos. Raymond
Raymond Hettinger wrote:
Wow, this whole subject is still astonishing me. What an education.
Here's a nice paper on the history of time http://www.naggum.no/lugm-time.html . Someone once said that time was invented by the gods to confuse man (roughly quoted). Neil
On Fri, Jan 03, 2003, Tim Peters wrote:
[Aahz]
So there's ambiguity. So what?
That was my question in the orginal msg, and hasn't changed: there are problems at two points, they're inherent to the current design (so can't be wished away), but the current implementation could change what it does in these cases -- what do people want to see done?
Hmmm.... Maybe we don't have an argument. What *does* the current implementation do when it hits the witching hour of DST->ST? So far, I've been saying that it should return 1:MM twice when converting from UTC; if it already does that, I'm fine.
What I don't understand is why it's a problem.
It depends on the app; like any other "surprise", depending on the app it may never be noticed, or may cost lives, or anything in between. Both you and Marc-Andre have been disturbed enough at the *possibilities* for bad consequences to claim that such cases should raise exceptions (although you seem to spend more energy arguing that there's no problem at all <wink>). Guido made a case on the other side, that once-a-year exceptions unlikely to be caught by testing carry a different kind of real risk.
The only case where I've advocated raising an exception was attempting to convert a pure wall clock time to any timezone-based time. (As in the case of Guido's calendar entries.) That would raise an exception for all times, not just DST change days.
as long as you don't try to convert between timezones in the absence of sufficient information.
The current design doesn't allow the possibility for sufficient information in these cases. One plausible response to that is to insist that the current design is fatally flawed. Another is to shrug "so it goes", and define what it does do, as best as can be done.
I'm starting to think that the current design is incomplete for tzinfo classes that model internal DST changes. If you say that the current design cannot be tweaked to return 1:MM twice at negative DST changes, then I'll respond to Guido's message asking for a better way with a proposed change to make that possible.
... at any moment, you can call Guido, and ...
But that's not a conversion.
? It's clearly a way to convert your local time to Guido's local time. The only real difference is that when you call Guido at 6:30 UTC on the day daylight time ends for him, and he replies "oh, it's 1:30 here", you have the further possibility to ask him whether he meant 1:30 EDT or 1:30 EST. Apart from that, dt.astimezone(Eastern) will give you answers identical to his every time you try it (assuming you start with dt.tzinfo=Pacific (from US.py), that you still live in the Pacific time zone, and that you and Guido both scrupulously adjust your clocks at the politically defined DST transition times).
Okay, I got confused by your later paragraph about Guido living in UTC time. In this case, I'd say that you're doing a heuristic conversion rather than an algebraic conversion -- and that's precisely what I'm advocating.
The most likely error (if one happens) is that someone shows up an hour early for the appointment, and presumably that person knows that the day is a DST transition.
This sounds like you don't want it to raise an exception, then.
Yup. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are three kinds of lies: Lies, Damn Lies, and Statistics." --Disraeli
[Aahz]
... Hmmm.... Maybe we don't have an argument.
We don't, unless you're claiming there isn't ambiguity at DST switch points.
What *does* the current implementation do when it hits the witching hour of DST->ST?
Please read the original msg again. Nothing that followed has been of much help, and I've really had nothing new to say since then -- it was all there at the start (including the answer to this question).
So far, I've been saying that it should return 1:MM twice when converting from UTC; if it already does that, I'm fine.
It doesn't now, but I believe it will be reasonably cheap to do so, and that's what Guido wants it to do too. Provided that's the defined and documented behavior, fine by me too. Thanks for pushing for it! (Right or wrong, it fosters the illusion of consensus <wink>.)
... The only case where I've advocated raising an exception was attempting to convert a pure wall clock time to any timezone-based time.
The datetime module doesn't have a class named "PureWallClockTime" <wink>. Seriously, the original msg posed focused questions about how a specific class method should act in its debatable endcases, and anything beyond that is out-of-scope for me here.
(As in the case of Guido's calendar entries.) That would raise an exception for all times, not just DST change days.
Under the agreement, Guido's calendar entries will display as 1:MM in this case, if a programmer uses datetimetz.astimezone() to convert them. It will never raise an exception merely for "I don't think you should do that" reasons, or even for "but the hour can't be spelled unambiguously in your time zone" reasons.
... I'm starting to think that the current design is incomplete for tzinfo classes that model internal DST changes.
Yes. Without a moral equivalent to struct tm's tm_isdst flag (which datetime does not have), it's necessarily incomplete, and that has consquences for two (and only two) hours per year in hybrid (standard+daylight) tzinfo classs. One consequence seems trivial (assigning "a meaning" to the non-extant 2:MM local hour at DST start); the importance of the other (an unspellable in local time hour at DST end) varies by app, and seemingly by what mood someone is in when they think about it.
On Fri, Jan 03, 2003, Tim Peters wrote:
[Aahz]
What *does* the current implementation do when it hits the witching hour of DST->ST?
Please read the original msg again. Nothing that followed has been of much help, and I've really had nothing new to say since then -- it was all there at the start (including the answer to this question).
Your love of mathematical precision sometimes gets in the way of clear answers to specific questions. ;-) (Yes, I saw the bit about ValueError earlier, but it was implied, not explicit.)
So far, I've been saying that it should return 1:MM twice when converting from UTC; if it already does that, I'm fine.
It doesn't now, but I believe it will be reasonably cheap to do so, and that's what Guido wants it to do too. Provided that's the defined and documented behavior, fine by me too. Thanks for pushing for it! (Right or wrong, it fosters the illusion of consensus <wink>.)
Cool!
The only case where I've advocated raising an exception was attempting to convert a pure wall clock time to any timezone-based time.
The datetime module doesn't have a class named "PureWallClockTime" <wink>. Seriously, the original msg posed focused questions about how a specific class method should act in its debatable endcases, and anything beyond that is out-of-scope for me here.
Uh, it sure looks to me like timetz defaults tzinfo to None, which I'd call "pure wall clock". But you're probably right that it's out-of-scope for this discussion -- I only brought it up because you mentioned Guido's calendar. (Nevertheless, I can't resist continuing the argument below.)
(As in the case of Guido's calendar entries.) That would raise an exception for all times, not just DST change days.
Under the agreement, Guido's calendar entries will display as 1:MM in this case, if a programmer uses datetimetz.astimezone() to convert them. ...
No, they shouldn't, assuming Guido's calendar entries are built out of timetz instances with tzinfo set to None. Remember that Guido sticks in entries for 1pm in SF while he's still in DC. If you're going to handle this use case, there needs to be a way to spell it, and someone trying to convert this to a timezone ought to get an exception. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "There are three kinds of lies: Lies, Damn Lies, and Statistics." --Disraeli
[Aahz]
Your love of mathematical precision sometimes gets in the way of clear answers to specific questions. ;-) (Yes, I saw the bit about ValueError earlier, but it was implied, not explicit.)
Given The current implementation of dt.astimezone(tz) raises ValueError if dt can't be expressed as a local time in tz. following an explict example of that case, that's a meaning for "implied" with which I was previously unacquainted <wink>.
... Uh, it sure looks to me like timetz defaults tzinfo to None, which I'd call "pure wall clock".
The docs call this "naive time", probably the same thing. ...
Under the agreement, Guido's calendar entries will display as 1:MM in this case, if a programmer uses datetimetz.astimezone() to convert them. ...
No, they shouldn't, assuming Guido's calendar entries are built out of timetz instances with tzinfo set to None. Remember that Guido sticks in entries for 1pm in SF while he's still in DC.
I can't deduce from that whether the generation of Palm that builds its apps with Python 2.3 will choose to create naive or aware times in its appointment app.
If you're going to handle this use case, there needs to be a way to spell it,
There are surely many ways to spell it, depending on all sorts of design criteria that haven't been made explicit here.
and someone trying to convert this to a timezone ought to get an exception.
You can't pass a time object or a timetz object (whether naive or aware makes no difference) to astimezone(), if that's what you're worried about. So, sure, you'll get a TypeError exception if you try.
The current datetime module class hierarchy looks like: object timedelta tzinfo time timetz date datetime datetimetz Several people have noted (on the wiki, and in email), with some bewilderment, that: 1. A timetz object with tzinfo=None acts like a time object. 2. A datetimetz object with tzinfo=None acts like a datetime object. A natural suggestion is to change the hierarchy to: object timedelta tzinfo time date datetime where the current time and datetime classes go away, the current timetz is renamed to time, and the current datetimetz to datetime. I tried that this evening for the Python implementation, and it was easy enough. No subtle problems popped up. The results are checked in to the Python datetime sandbox, as datetime2.py and test_datetime2.py; it appears to be fully functional, and allowed tossing about 10% of the code lines. IIRC, the original motivation for making a finer distinction among objects with and without tzinfo members was to save memory in the objects. Guido thinks that's a red herring, though, and I agree. The C implementation of datetime objects currently has unused "pad bytes" in the structs (due to compiler alignment of struct members), and adding a flag byte to record "does this have a non-None tzinfo member or not?" wouldn't actually consume any more memory. The C implementation would still need to have more than one way to allocate a time/datetime under the covers, in order to avoid allocating an additional pointer field (for the PyObject* tzinfo pointer) when it wasn't needed, so it wouldn't really simplify the C implementation much. It would simplify the user's mental model, and cut out a ton of near-redundant docs (those two are pretty much the same thing <wink>).
[Tim]
A natural suggestion is to change the hierarchy to:
object timedelta tzinfo time date datetime
[Shane Hathaway]
I think the proposal is so good and well-reasoned that no one had anything to say. ;-) +1
I think your vote carries the motion, then, Shane! Thank you. I know Guido is in favor of it (has been pushing for it, in fact), and that changing the Python implementation only took a few hours was a Good Sign. I've since played with the new implementation as a user, and found it more pleasant to use (e.g., fewer pointless choices to make, and "promoting" what was a datetime to what was a datetimetz has become trivial instead of an irritating puzzle).
1. When DST starts (the "wall start" line), the wall clock leaps from 1:59 to 3:00. A wall time of the form 2:MM doesn't really make sense on that day. The example classes do what I believe is the best that can be done: since 2:MM is "after 2" on that day, it's taken as daylight time, and so as an alias for 1:MM standard == 1:MM wall on that day, which is a time that does make sense on the wall clock that day. The astimezone() function ensures that the "impossible hour" on that day is never the result of a conversion (you'll get the standard-time spelling instead). If you don't think that's the best that can be done, speak up now.
I don't particularly care one way or the other, but when I'm *awake* during this hour, 2:MM more likely means that I forgot to move my clock forward, so it may make more sense to interpret it as standard time after all (meaning it should be switched to 3:MM DST rather than 1:MM STD).
2. When DST ends (the "wall end" line), we have a worse problem: there'a an hour that can't be spelled *at all* in wall time. It's the hour beginning at the moment DST ends; in the example, that's times of the form 6:MM UTC on the day daylight time ends. The local wall clock leaps from 1:59 (daylight time) back to 1:00 again (but the second time as a standard time). The hour 6:MM UTC looks like 1:MM, but so does the hour 5:MM UTC on that day. A reasonable tzinfo class should take 1:MM as being daylight time on that day, since it's "before 2". As a consequence, the hour 6:MM UTC has no wall-clock spelling at all.
This can't be glossed over. If you code a tzinfo class to take 1:MM as being standard time on that day instead, then the UTC hour 5:MM becomes unspellable in wall time instead. No matter how you cut it, the redundant spellings of an hour on the day DST starts means there's an hour that can't be spelled at all on the day DST ends (so in that sense, they're the two sides of a single problem).
What to do? The current implementation of dt.astimezone(tz) raises ValueError if dt can't be expressed as a local time in tz. That's the "errors should never pass silently" school, which I briefly attended in college <wink>. If you don't like that, what would you rather see happen? Try to be precise, and remember that getting a "correct" time in tz is flatly impossible in this case.
One (perhaps feeble) argument against raising ValueError here is that this introduces a case where a calculation that normally never raises an error (assuming sane timezones) can raise an exception for one hour a year. If you run a webserver that e.g. tries to render the current time at the server (which is represented in UTC of course) in the end user's local time, it would be embarrassing if this caused an error page when the end user's local time happens to be in the unrepresentable hour (i.e. one hour per year). You really want to render it as 1:MM, since the user should know whether his DST has already ended or not yet. While in an ideal world the programmer would have read the docs for .astimezone() and heeded the warning to catch ValueError (and then display what? UTC?), realistically if the programmer is a mere mortal and found no problems during testing, she will be embarrassed by the error page. (For worse effect, imagine the server running in a space probe :-) The counterargument is of course a use case where the time displayed is of real importance, and we would rather die than show the wrong time. The C standard library (following the original Unix treatment of timezones) has a solution for this: it adds a three-valued flag to the local time which indicates whether DST is in effect or not. Normally, you can set this flag to -1 ("don't know") in which case the proper value is calculated from the DST rules. But for ambiguous local times (the hour at the end of DST), setting it to 0 or 1 makes the times unambiguous again. This would mean that we'd have to add an "isdst" field to datetimetz objects and ways to set it; it would default to -1 (or perhaps to the proper value based on DST rules) but .astimezone() could set it explicitly to 0 or 1 to differentiate between the two ambiguous times.
Two other debatable edge cases in the implementation of dt.astimezone(tz):
3. If dt.tzinfo.dst(dt) returns None, the current implementation takes that as a synonym for 0. Perhaps it should raise an exception instead.
Why would dst() return None for a tzinfo object whose utcoffset() returns a definite value? I think that's a poor tzinfo implementation and an exception would be appropriate.
4. If dt.tzinfo.utcoffset(dt) first returns an offset, and on a subsequent call (while still trying to figure out the same conversion) returns None, an exception is raised. Those who don't want unspellable hours to raise an exception may also want inconsistent tzinfo implementations to go without complaint. If so, what do you want it to do instead?
I think this could only happen if a tzinfo's utcoffset() returns None for *some* times but not for others, right? I don't think such a tzinfo should be considered sane. Hmm, perhaps that's how a tzinfo object would signal that a particular local time is illegal (e.g. the hour at the start of DST). Then astimezone() would have to worm around that in some other way. --Guido van Rossum (home page: http://www.python.org/~guido/)
"GvR" == Guido van Rossum
writes:
GvR> The C standard library (following the original Unix treatment GvR> of timezones) has a solution for this: it adds a three-valued GvR> flag to the local time which indicates whether DST is in GvR> effect or not. Normally, you can set this flag to -1 ("don't GvR> know") in which case the proper value is calculated from the GvR> DST rules. But for ambiguous local times (the hour at the GvR> end of DST), setting it to 0 or 1 makes the times unambiguous GvR> again. This would mean that we'd have to add an "isdst" GvR> field to datetimetz objects and ways to set it; it would GvR> default to -1 (or perhaps to the proper value based on DST GvR> rules) but .astimezone() could set it explicitly to 0 or 1 to GvR> differentiate between the two ambiguous times. +1, but let's use True, False, and None. :) -Barry
On Wed, 1 Jan 2003, Barry A. Warsaw wrote:
"GvR" == Guido van Rossum
writes: GvR> The C standard library (following the original Unix treatment GvR> of timezones) has a solution for this: it adds a three-valued GvR> flag to the local time which indicates whether DST is in GvR> effect or not. Normally, you can set this flag to -1 ("don't GvR> know") in which case the proper value is calculated from the GvR> DST rules. But for ambiguous local times (the hour at the GvR> end of DST), setting it to 0 or 1 makes the times unambiguous GvR> again. This would mean that we'd have to add an "isdst" GvR> field to datetimetz objects and ways to set it; it would GvR> default to -1 (or perhaps to the proper value based on DST GvR> rules) but .astimezone() could set it explicitly to 0 or 1 to GvR> differentiate between the two ambiguous times.
+1, but let's use True, False, and None. :)
This is a typical example of where bool is wrong. Google for "double daylight savings", and you'll understand why. So it would make more sense to use: 3600, 0, and None. /Paul
"PS" == Paul Svensson
writes:
PS> This is a typical example of where bool is wrong. Google for PS> "double daylight savings", and you'll understand why. So it PS> would make more sense to use: 3600, 0, and None. Then you have to call it something other than "isdst()". -Barry
"PS" == Paul Svensson
writes: PS> This is a typical example of where bool is wrong. Google for PS> "double daylight savings", and you'll understand why. So it PS> would make more sense to use: 3600, 0, and None.
Then you have to call it something other than "isdst()". -Barry
Invert the test and have the best of both worlds: isStandardTime() --> bool Raymond Hettinger
Barry> Then you have to call it something other than "isdst()". Raymond> Invert the test and have the best of both worlds: Raymond> isStandardTime() --> bool Changing the sense of the test doesn't get rid of the fact that there are three possible answers, not two: "standard time", "daylight time", and "i don't know". Skip
+1, but let's use True, False, and None. :)
This is a typical example of where bool is wrong.
No it isn't.
Google for "double daylight savings", and you'll understand why. So it would make more sense to use: 3600, 0, and None.
Unless congressman Sherman proposes to move the clock forward and/or back more than once a year, this is not necessary: the flag only indicates *whether* DST is in effect or not; to know *how much* the DST adjustment is you should call the tzinfo's dst() method. The conventional value of one hour of is never implied in any of these calculations -- that's always up to the tzinfo object. (However, it's assumed that dst() returns a positive number -- otherwise it wouldn't be daylight *savings*. On the southern hemisphere, the seasons are reversed, but DST still moves the clock forward by some amount, not back.) --Guido van Rossum (home page: http://www.python.org/~guido/)
[Guido]
... The conventional value of one hour of is never implied in any of these calculations -- that's always up to the tzinfo object.
This is so.
(However, it's assumed that dst() returns a positive number -- otherwise it wouldn't be daylight *savings*. On the southern hemisphere, the seasons are reversed, but DST still moves the clock forward by some amount, not back.)
If you dig into the long new correctness proof, you'll find that the astimezone() implementation doesn't actually need the assumption that dst() returns a positive number. It does rely on the sanity requirement that dst() return a non-zero result if and only if daylight time is in effect, and on the subtler consistency requirement that dt.tzinfo.utcoffset(dt) - dt.tzinfo.dst(dt) is a fixed value across all datetimetz dt with the same dt.year member (hmm -- I think I'm assuming there too that a DST switch doesn't occur within a day or so of Jan 1). The flexibility to rely on only one-year-at-a-time consistency is important for people who need to model that Chile delayed its changeover date for the Pope's visit in 1987 <wink>.
If you dig into the long new correctness proof, you'll find that the astimezone() implementation doesn't actually need the assumption that dst() returns a positive number. It does rely on the sanity requirement that dst() return a non-zero result if and only if daylight time is in effect, and on the subtler consistency requirement that
dt.tzinfo.utcoffset(dt) - dt.tzinfo.dst(dt)
is a fixed value across all datetimetz dt with the same dt.year member (hmm -- I think I'm assuming there too that a DST switch doesn't occur within a day or so of Jan 1). The flexibility to rely on only one-year-at-a-time consistency is important for people who need to model that Chile delayed its changeover date for the Pope's visit in 1987 <wink>.
Unless a country changes timezones permanently, I'd assume that this expression is constant regardless of the year. And *if* a country switches timezones, it doesn't necessarily happen on Jan 1st. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, 1 Jan 2003, Guido van Rossum wrote:
+1, but let's use True, False, and None. :)
This is a typical example of where bool is wrong.
No it isn't.
Google for "double daylight savings", and you'll understand why. So it would make more sense to use: 3600, 0, and None.
Unless congressman Sherman proposes to move the clock forward and/or back more than once a year, this is not necessary: the flag only indicates *whether* DST is in effect or not; to know *how much* the DST adjustment is you should call the tzinfo's dst() method. The conventional value of one hour of is never implied in any of these calculations -- that's always up to the tzinfo object. (However, it's assumed that dst() returns a positive number -- otherwise it wouldn't be daylight *savings*. On the southern hemisphere, the seasons are reversed, but DST still moves the clock forward by some amount, not back.)
Congressman Sherman is totally irrelevant. In 1945, Britain used GMT+1 until April 2nd, then GMT+2 until July 15th, then went back GMT+1 until October 7th, and then GMT the rest of he year. Again in Britain, in 1947, DST extended from March 16th to November 2nd, with double DST from April 13th to August 10th. /Paul
Congressman Sherman is totally irrelevant.
Then why did you waste everybody's time by suggesting we Google for something that returned him as first and second hits, rather than just explaining the issue?
In 1945, Britain used GMT+1 until April 2nd, then GMT+2 until July 15th, then went back GMT+1 until October 7th, and then GMT the rest of he year. Again in Britain, in 1947, DST extended from March 16th to November 2nd, with double DST from April 13th to August 10th.
OK, I'm convinced. I believe the C99 standard also changes the tm_isdst flag to indicate the value of the DST adjustment; now I understand why. I wonder if this affects an assumption of Tim's correctness proof? --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Congressman Sherman is totally irrelevant.
Then why did you waste everybody's time by suggesting we Google for something that returned him as first and second hits, rather than just explaining the issue?
In 1945, Britain used GMT+1 until April 2nd, then GMT+2 until July 15th, then went back GMT+1 until October 7th, and then GMT the rest of he year. Again in Britain, in 1947, DST extended from March 16th to November 2nd, with double DST from April 13th to August 10th.
OK, I'm convinced. I believe the C99 standard also changes the tm_isdst flag to indicate the value of the DST adjustment; now I understand why.
I wonder if this affects an assumption of Tim's correctness proof?
Since we're on the subject, its always distressed me (in a purely intellectual way) that Unix time doesn't _really_ convert to wall-clock time properly, because it ignores leap seconds. The effect of this is that times in the past get converted incorrectly by several seconds (how many depending on exactly when in the past, of course). I don't suppose there's interest in fixing that? Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." - Robert Woodruff
Since we're on the subject, its always distressed me (in a purely intellectual way) that Unix time doesn't _really_ convert to wall-clock time properly, because it ignores leap seconds. The effect of this is that times in the past get converted incorrectly by several seconds (how many depending on exactly when in the past, of course).
I don't suppose there's interest in fixing that?
Given how timestamps in a typical Unix system are used, I think ignoring leap seconds is the only sensible thing. The POSIX standard agrees. If by "fixing" you mean taking leap seconds into account in any way, I would strongly object that. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Since we're on the subject, its always distressed me (in a purely intellectual way) that Unix time doesn't _really_ convert to wall-clock time properly, because it ignores leap seconds. The effect of this is that times in the past get converted incorrectly by several seconds (how many depending on exactly when in the past, of course).
I don't suppose there's interest in fixing that?
Given how timestamps in a typical Unix system are used, I think ignoring leap seconds is the only sensible thing. The POSIX standard agrees. If by "fixing" you mean taking leap seconds into account in any way, I would strongly object that.
Actually, now I think about it again, its not the human readable version that's wrong (though there is this silly issue that there are some seconds that you can't represent as Unix timestamps), its the interval between two timestamps that's wrong. Anyway, its not an issue I'm hugely passionate about - though I imagine it might matter to some scientists - I just thought I'd mention it, since we're on the subject. Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ "There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit." - Robert Woodruff
Actually, now I think about it again, its not the human readable version that's wrong (though there is this silly issue that there are some seconds that you can't represent as Unix timestamps), its the interval between two timestamps that's wrong.
Few clocks can represent those seconds. Few clocks need to.
Anyway, its not an issue I'm hugely passionate about - though I imagine it might matter to some scientists - I just thought I'd mention it, since we're on the subject.
Let's drop it please. I'm hugely passionate about this -- leap seconds have no reason to be accounted for in most people's lives. --Guido van Rossum (home page: http://www.python.org/~guido/)
[Guido, on Britain's pecularities in 1945 and 1947]
... I wonder if this affects an assumption of Tim's correctness proof?
I'd say it blows it all to hell. Is 1940's Britain an important use case <wink>? Here's the most accurate and complete astimezone() method that can be written (ignoring endcases due to naive objects and insane implementations of tzinfo methods): def astimezone(self, tz): selfasutc = self.replace(tzinfo=None) - self.utcoffset() intz = selfasutc.replace(tzinfo=tz) # Make no assumptions. Since the offset-returning # methods are restricted to minute granularity and # magnitude less than one day, the only possible equivalents # in tz can be enumerated exhaustively. results = [] # list of all equivalents in tz for offset in range(1-24*60, 24*60): candidate = intz + timedelta(minutes=offset) asutc = candidate - candidate.utcoffset() if selfasutc == asutc.replace(tzinfo=None): results.append(candidate) return results Skipping all the subtleties and optimizations, the current astimezone() is more like this: def astimezone(self, tz): selfasutc = self.replace(tzinfo=None) - self.utcoffset() other = selfasutc.replace(tzinfo=tz) # Convert other to tz's notion of standard time. other += other.utcoffset() - other.dst() if self == other: # comparison works in UTC time by magic return other # Else tz must want a daylight spelling. assert other.dst() != 0 other += other.dst() if self == other: return other raise ValueError("self has no spelling in the " "target time zone") That's more efficient than trying thousands of possibilities <wink>. The crucial subtlety in the correctness proof hinges on that other.utcoffset() - other.dst() returns the same value before and after this line: other += other.utcoffset() - other.dst() But if that isn't true, the implementation is wrong. Worse, it has no legs left to stand on. Here's a happy idea: punt. We could define a new tzinfo method to be implemented by the user: def fromutc(self, dt): """Convert UTC time to local time. dt is a plain datetime or a naive datetimetz whose date and time members represent a UTC time. Return a datetimetz y, with y.tzinfo == self, representing the equivalent in tz's local time. """ Then I can raise an exception for unspellable hours in my Eastern class, and you can make them return your birthday in your Eastern class, and the two surviving members of the United Kindgom Memorial Double Daylight Society can eat watercress sandwiches at the local pub in peace <wink>. astimezone becomes: def astimezone(self, dt): return dt.fromutc((self - self.utcoffset()).replace(tzinfo=None))
Here's a happy idea: punt.
I'd be happy to punt on the multiple DST switch points for astimezone().
We could define a new tzinfo method to be implemented by the user:
def fromutc(self, dt): """Convert UTC time to local time.
dt is a plain datetime or a naive datetimetz whose date and time members represent a UTC time. Return a datetimetz y, with y.tzinfo == self, representing the equivalent in tz's local time. """
Then I can raise an exception for unspellable hours in my Eastern class, and you can make them return your birthday in your Eastern class, and the two surviving members of the United Kindgom Memorial Double Daylight Society can eat watercress sandwiches at the local pub in peace <wink>.
astimezone becomes:
def astimezone(self, dt): return dt.fromutc((self - self.utcoffset()).replace(tzinfo=None))
But now even the simplest tzinfo implementations (those with a fixed offset) have to implement something a bit tricky: ZERO = timedelta() class fixed(tzinfo): def __init__(self, offset): self.offset = timedelta(minutes=offset) def utcoffset(self, dt): return self.offset def dst(self, dt): return ZERO def fromutc(self, dt): return datetimetz.combine(dt.date(), dt.time(), tzinfo=self) + self.offset I find that last line cumbersome. For variable-dst zones it goes way up: the DST transition boundaries have to be translated to UTC in order to be able to make sense out of them in fromutc(), but they're still needed in local time for dst(). But I agree that we could probably improve life for all involved if we changed the tzinfo implementation. Perhaps we should constrain DST-aware timezones to the most common model, where there are two offsets (standard and DST) and two transition datetime points in local standard time (DST on and DST off). A tzinfo would have methods that would let you inquire these things directly; the offsets would be simple attributes, and the DST on and off points would be returned by a method that only takes the year as input (astimezone() would have to be careful to expect DST on to be > DST off for the southern hemisphere). I suppose the Israeli implementation would have to use a table of transition points for past years and always return standard time for future years (since the Knesseth decides on the switch points each year). The implementation for British time during WWII would have to tell a little lie -- big deal. --Guido van Rossum (home page: http://www.python.org/~guido/)
[Tim suggests a tzinfo.fromutc() method] [Guido]
But now even the simplest tzinfo implementations (those with a fixed offset) have to implement something a bit tricky:
ZERO = timedelta() class fixed(tzinfo): def __init__(self, offset): self.offset = timedelta(minutes=offset) def utcoffset(self, dt): return self.offset def dst(self, dt): return ZERO def fromutc(self, dt): return datetimetz.combine(dt.date(), dt.time(), tzinfo=self) + self.offset
I find that last line cumbersome.
It is. Since astimezone() will only call it with a datetimetz argument, it could be simplified to return dt.replace(tzinfo=self) + self.offset For that matter, astimezone could attach self to dt before calling this, so that the user implementation becomes return dt + self.offset
For variable-dst zones it goes way up: the DST transition boundaries have to be translated to UTC in order to be able to make sense out of them in fromutc(), but they're still needed in local time for dst().
Some time zones have to translate them to UTC in dst() anyway; your EU.py does this, for example.
But I agree that we could probably improve life for all involved if we changed the tzinfo implementation. Perhaps we should constrain DST-aware timezones to the most common model, where there are two offsets (standard and DST) and two transition datetime points in local standard time (DST on and DST off). A tzinfo would have methods that would let you inquire these things directly; the offsets would be simple attributes, and the DST on and off points would be returned by a method that only takes the year as input (astimezone() would have to be careful to expect DST on to be > DST off for the southern hemisphere). I suppose the Israeli implementation would have to use a table of transition points for past years and always return standard time for future years (since the Knesseth decides on the switch points each year). The implementation for British time during WWII would have to tell a little lie -- big deal.
Well, I'm not sure that adding a bunch of new methods is going to make life easier for users. Another idea: fromutc() is just a defined hook. If a tzinfo object has it, astimezone() will use it and believe whatever it returns. If it's not defined, then the current implementation is used -- and it should work for any tz where tz.utcoffset(dt)-tz.dst(dt) is invariant wrt dt. That covers the only worlds I live in <wink>. If we constrain what a tzinfo class *can* do to fit a one-size-fits-all implementation, then we may as well steal Java's SimpleTimeZone API and let users get away without coding anything (in return, they get to spend days trying to guess what all the constructor arguments really mean <wink>).
[Guido]
But now even the simplest tzinfo implementations (those with a fixed offset) have to implement something a bit tricky:
ZERO = timedelta() class fixed(tzinfo): def __init__(self, offset): self.offset = timedelta(minutes=offset) def utcoffset(self, dt): return self.offset def dst(self, dt): return ZERO def fromutc(self, dt): return datetimetz.combine(dt.date(), dt.time(), tzinfo=self) + self.offset
I find that last line cumbersome.
It is. Since astimezone() will only call it with a datetimetz argument,
That's not how I read your previous mail, but it's certainly more sensible, given that astimezone() is primarily a method on a datetimetz. :-)
it could be simplified to
return dt.replace(tzinfo=self) + self.offset
For that matter, astimezone could attach self to dt before calling this, so that the user implementation becomes
return dt + self.offset
For variable-dst zones it goes way up: the DST transition boundaries have to be translated to UTC in order to be able to make sense out of them in fromutc(), but they're still needed in local time for dst().
Some time zones have to translate them to UTC in dst() anyway; your EU.py does this, for example.
Agreed. DST-aware tzinfo classes will always be complex.
But I agree that we could probably improve life for all involved if we changed the tzinfo implementation. Perhaps we should constrain DST-aware timezones to the most common model, where there are two offsets (standard and DST) and two transition datetime points in local standard time (DST on and DST off). A tzinfo would have methods that would let you inquire these things directly; the offsets would be simple attributes, and the DST on and off points would be returned by a method that only takes the year as input (astimezone() would have to be careful to expect DST on to be > DST off for the southern hemisphere). I suppose the Israeli implementation would have to use a table of transition points for past years and always return standard time for future years (since the Knesseth decides on the switch points each year). The implementation for British time during WWII would have to tell a little lie -- big deal.
Well, I'm not sure that adding a bunch of new methods is going to make life easier for users.
I meant to use these *only* and throw away the other methods. (tzname() would have to be replaced by two string attributes, e.g. stdname and dstname, or perhaps a sequence of two strings, like the time module does in its tzname variable.)
Another idea: fromutc() is just a defined hook. If a tzinfo object has it, astimezone() will use it and believe whatever it returns. If it's not defined, then the current implementation is used -- and it should work for any tz where tz.utcoffset(dt)-tz.dst(dt) is invariant wrt dt. That covers the only worlds I live in <wink>.
If we constrain what a tzinfo class *can* do to fit a one-size-fits-all implementation, then we may as well steal Java's SimpleTimeZone API and let users get away without coding anything (in return, they get to spend days trying to guess what all the constructor arguments really mean <wink>).
I want to be able to model at least table-based historic DST transitions, which SimpleTimeZone doesn't do. Its abstract base class, TimeZone, allows you to write a concrete subclass that *does* support historic DST transitions. But the mess Java made here is obvious in the number of deprecated methods and constructors, and I'd like to stay away from it as far as possible. (January==0? Gimme a break! :-). --Guido van Rossum (home page: http://www.python.org/~guido/)
[Guido, on the DST start case: in US Eastern, 2:MM doesn't exist on the wall clock, which jumps from 1:MM to 3:MM; the example tzinfo classes take 2:MM as being daylight then, since it's "after 2", and astimezone() delivers the equivalent standard-time 1:MM spelling when it has to deliver a result in this hour]
I don't particularly care one way or the other, but when I'm *awake* during this hour, 2:MM more likely means that I forgot to move my clock forward, so it may make more sense to interpret it as standard time after all (meaning it should be switched to 3:MM DST rather than 1:MM STD).
Note that this issue is almost entirely about how users code *their* tzinfo classes: the interpretion of 2:MM is up to them. If you would like to call 2:MM standard time instead, then write your dst() method accordingly. Note that the example time zone classes in US.py call 2:MM daylight time "because it's after 2:00", and the time zone classes in EU.py do an equivalent thing for the time zones they implement; it's the most natural thing to code. If this hour happens to be the result of an astimezone() conversion, it picks the unambiguous standard-time spelling (which is 1:MM whether you moved your clock forward at exactly the right moment, or waited until the next day; the astimezone() part of this case is simply that it doesn't deliver an ambiguous result in this case -- an Eastern tzinfo subclass would be insane if it called 1:MM daylight time on this day). [on the unspellable hour at the end of DST]
One (perhaps feeble) argument against raising ValueError here is that this introduces a case where a calculation that normally never raises an error (assuming sane timezones) can raise an exception for one hour a year. If you run a webserver that e.g. tries to render the current time at the server (which is represented in UTC of course) in the end user's local time, it would be embarrassing if this caused an error page when the end user's local time happens to be in the unrepresentable hour (i.e. one hour per year). You really want to render it as 1:MM, since the user should know whether his DST has already ended or not yet. While in an ideal world the programmer would have read the docs for .astimezone() and heeded the warning to catch ValueError (and then display what? UTC?),
The parenthetical questions have to be answered even if the implementation changes: if astimezone() doesn't raise an exception here, it has to set *some* time values in the result object, and the server will display them. Probably 1:MM in the Eastern case -- it would take some thought to figure out whether it could at least promise to deliver the ambiguous wall-clock spelling in this case (so that it's never worse than an off-by-one (hour) error).
realistically if the programmer is a mere mortal and found no problems during testing, she will be embarrassed by the error page. (For worse effect, imagine the server running in a space probe :-)
The counterargument is of course a use case where the time displayed is of real importance, and we would rather die than show the wrong time.
Displaying times across time zones is a pretty harmless case. The reason I worry more about this here is that one-zone (intrazone, as opposed to your interzone use case) DST-aware arithmetic can't be done in the datetime module without converting to UTC (or some other fixed reference), doing the arithmetic there, and converting back again. In that set of use cases, getting an incorrect result seems much more likely to have bad consequences. If the Russian space probe is programmed in some hybrid Russian std+daylight time zone, you tell it to turn away from the sun 6 hours from now, and it doesn't actually turn away for 7 hours and fries itself as a result, it will be a lead story on comp.risks. Although I hope they have sense enough to pick on it more for programming a space probe in some hybrid Russian std+daylight time zone <wink>. Perhaps astimezone() could grow an optional "error return" value, a datetimetz to be used if the time is unrepresentable in the target zone.
The C standard library (following the original Unix treatment of timezones) has a solution for this: it adds a three-valued flag to the local time which indicates whether DST is in effect or not. Normally, you can set this flag to -1 ("don't know") in which case the proper value is calculated from the DST rules. But for ambiguous local times (the hour at the end of DST), setting it to 0 or 1 makes the times unambiguous again. This would mean that we'd have to add an "isdst" field to datetimetz objects and ways to set it; it would default to -1 (or perhaps to the proper value based on DST rules) but .astimezone() could set it explicitly to 0 or 1 to differentiate between the two ambiguous times.
This could work too.
3. If dt.tzinfo.dst(dt) returns None, the current implementation takes that as a synonym for 0. Perhaps it should raise an exception instead.
Why would dst() return None for a tzinfo object whose utcoffset() returns a definite value? I think that's a poor tzinfo implementation and an exception would be appropriate.
Only because the first three times I wrote a tzinfo class, I didn't give a rip about DST so implemented dst() to return None <wink>. I'm afraid you still have to care *enough* to read the docs, and heed the advice that dst() should return 0 if DST isn't in effect. I'll change this.
4. If dt.tzinfo.utcoffset(dt) first returns an offset, and on a subsequent call (while still trying to figure out the same conversion) returns None, an exception is raised. Those who don't want unspellable hours to raise an exception may also want inconsistent tzinfo implementations to go without complaint. If so, what do you want it to do instead?
I think this could only happen if a tzinfo's utcoffset() returns None for *some* times but not for others, right?
Correct. When this happens, the implementation currently raises ValueError, with a msg complaining that the utcoffset() method is inconsistent.
I don't think such a tzinfo should be considered sane.
Then you don't hate the current implementation on this count <wink>.
Hmm, perhaps that's how a tzinfo object would signal that a particular local time is illegal (e.g. the hour at the start of DST).
I'm not worried about that case: astimezone() never *produces* that spelling (not even internally -- see the long new correctness proof for the gory details).
Then astimezone() would have to worm around that in some other way.
If the input hour to astimezone() is of that ambiguous form, astimezone() already takes a None return from utcoffset() as meaning the input datetimetz is naive, and simply attaches the new timezone to the input date and time fields without altering the latter. On the other end (when DST ends), since there's no way to spell the "impossible hour" in local time, a tzinfo class knows nothing about that hour (in particular, it can't detect it -- astimezone() can, based on the otherwise impossible sequence of results it gets from calling utcoffset() more than once).
Tim Peters wrote:
We still have two time zone conversion problems, and always will (they come with the territory -- they're inherent problems, not artifacts of the implementation). They only arise in a tzinfo class that's trying to model both standard and daylight time. In effect, that's one class trying to pretend it's two different time zones, and at the transition points there are "impossible problems". ... UTC, EST and EDT are all self-consistent and trivial. It's only wall time that's a problem, and only at the transition points:
1. When DST starts (the "wall start" line), the wall clock leaps from 1:59 to 3:00. A wall time of the form 2:MM doesn't really make sense on that day. The example classes do what I believe is the best that can be done: since 2:MM is "after 2" on that day, it's taken as daylight time, and so as an alias for 1:MM standard == 1:MM wall on that day, which is a time that does make sense on the wall clock that day. The astimezone() function ensures that the "impossible hour" on that day is never the result of a conversion (you'll get the standard-time spelling instead). If you don't think that's the best that can be done, speak up now.
2. When DST ends (the "wall end" line), we have a worse problem: there'a an hour that can't be spelled *at all* in wall time. It's the hour beginning at the moment DST ends; in the example, that's times of the form 6:MM UTC on the day daylight time ends. The local wall clock leaps from 1:59 (daylight time) back to 1:00 again (but the second time as a standard time). The hour 6:MM UTC looks like 1:MM, but so does the hour 5:MM UTC on that day. A reasonable tzinfo class should take 1:MM as being daylight time on that day, since it's "before 2". As a consequence, the hour 6:MM UTC has no wall-clock spelling at all.
The only right way to deal with these problems is to raise ValueErrors. Calculations resulting in these local times are simply doomed and should be done in UTC instead. DST and local times are not mathematical properties, so you shouldn't expect them to behave in that way. For some fun reading have a look at the tzarchive package docs at: ftp://elsie.nci.nih.gov/pub/ -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/
[M.-A. Lemburg]
The only right way to deal with these problems is to raise ValueErrors.
That was one of the presented alternatives, yes <wink>.
Calculations resulting in these local times are simply doomed and should be done in UTC instead.
That one isn't the issue here: DST-aware calculations under the datetime module indeed *must* convert to UTC (or some other fixed reference). Arithmetic within a single time zone is "naive" under datetime, doing no adjustments whatsoever to the date and time fields you'd expect if you had no notion of time zone. The question is what happens when you get a final result in UTC (or other fixed reference), try to convert it back to your local time zone, and the astimezone() method deduces it's the hour at the end of DST that can't be spelled in local time.
DST and local times are not mathematical properties, so you shouldn't expect them to behave in that way.
They *have* mathemetical properties, though, and very simple ones at that. The difficulty is that they're not complicated enough to give the illusion of meeting naive expectations <wink>.
For some fun reading have a look at the tzarchive package docs at: ftp://elsie.nci.nih.gov/pub/
time zones are compressed dicts.
participants (13)
-
Aahz
-
barry@python.org
-
barry@zope.com
-
Ben Laurie
-
Guido van Rossum
-
M.-A. Lemburg
-
Neil Schemenauer
-
Paul Svensson
-
Raymond Hettinger
-
Shane Hathaway
-
Skip Montanaro
-
Tim Peters
-
Tim Peters