Mailman 3 August 2015 - Datetime-SIG

Re: [Datetime-SIG] PEP-495 - Strict Invalid Time Checking
by Carl Meyer Aug. 25, 2015

Aug. 25, 2015

On 08/25/2015 03:01 PM, Alexander Belopolsky wrote: > On Tue, Aug 25, 2015 at 4:48 PM, Carl Meyer <carl(a)oddbird.net > <mailto:carl@oddbird.net>> wrote: > > If you accept this, and still > think that the best API datetime can provide is "localize twice and > figure it out yourself", I'm not interested in arguing the point. > > > Trust me, I considered many other options, but I have 4000+ lines of > datetime unit tests that I cannot break. If you can come up with a > patch that does what you want and passes python -m test test_datetime - > I would certainly like to see it. I don't understand why it seems like you continue to interpret all requests for an opt-in way to raise an exception as requests to change the default, which nobody is contemplating. I can't imagine how raising an exception on invalid times only if a non-default sentinel value is given for a flag that is _new in PEP 495_ could possibly break 4000 lines of existing datetime tests. Carl

2 1

Re: [Datetime-SIG] PEP-495 - Strict Invalid Time Checking
by ISAAC J SCHWABACHER Aug. 25, 2015

Aug. 25, 2015

[Alexander Belopolsky] > Whatever your requirements are, the difference between "strict checking" and PEP 495 code will be the same as between > dt = get_naive_time() > try: > dt = dt.astimezone() > except AmbiguousTimeError: > # do something > except MissingTimeError: > # do something else > else: > # use dt It's obvious to me what this code means. > and > dt = get_naive_time() > dt0 = dt.replace(fold=0).astimezone() > dt1 = dt.replace(fold=1).astimezone() > if dt1 > dt0: > # do something > elif dt1 < dt0: > # do something else > else: > # use dt0 which is the same as dt1 The only thing that's obvious to me about this code is that it's hacking around something. ijs

1 0

Re: [Datetime-SIG] PEP-495 - Strict Invalid Time Checking
by Carl Meyer Aug. 25, 2015

Aug. 25, 2015

On 08/25/2015 02:42 PM, Alexander Belopolsky wrote: > > On Tue, Aug 25, 2015 at 4:30 PM, Carl Meyer <carl(a)oddbird.net > <mailto:carl@oddbird.net>> wrote: > > > start = datetime.combine(date, time(0)).astimezone() > > while True: > > end = (start + timedelta(hours=0.5)).astimezone() > > print(start, end) > > start = end > > if end.time() == time(0): > > break > > On DST transition days, this code does not generate exactly 48 slots, > displayable to the user in a schedule that includes all hours from 0 to > 23 labeled exactly once. > > That introduces an unacceptable level of additional display-layer > complexity, and remains inferior to the "strict checking" solution I > chose. > > OK. So you just want [datetime.combine(date, time(0)) + > timedelta(hours=0.5*i)) for i in range(48)] but detect those that are > either in the fold or in the gap? Just check for dt.astimezone() != > dt.replace(fold=1).astimezone() instead of catching the strict exceptions. Yes, I know :-) If you'd read my email to the end, you'd have seen that I already know that that's the PEP 495 solution to my problem, and that it's probably fine if wrapped up in a utility function, though the ugliness of having to do all timezone localizations twice might have caused me to spend another couple hours looking for a better solution, whereas the convenience, availability, and discoverability of pytz's `is_dst=None` option made things simple and easy for me. I'm glad you've at least reached the point of acknowledging that valid use cases for strict checking exist; this seems to be a new development :-) That was really my only intent here. If you accept this, and still think that the best API datetime can provide is "localize twice and figure it out yourself", I'm not interested in arguing the point. Carl

2 1

Re: [Datetime-SIG] PEP-495 - Strict Invalid Time Checking
by Carl Meyer Aug. 25, 2015

Aug. 25, 2015

On 08/25/2015 02:56 PM, Alexander Belopolsky wrote: ... > > The PEP 495 code may be slightly more involved, but most people can get > away with just > > dt = get_time().astimezone() > > and not see any crashes. Since nobody has argued (that I can recall) for "raise an exception" to be the _default_ behavior, this last point will be true regardless. Carl

1 0

Re: [Datetime-SIG] PEP-495 - Strict Invalid Time Checking
by Carl Meyer Aug. 25, 2015

Aug. 25, 2015

On 08/25/2015 02:24 PM, Alexander Belopolsky wrote: > > On Tue, Aug 25, 2015 at 4:13 PM, Carl Meyer <carl(a)oddbird.net > <mailto:carl@oddbird.net>> wrote: > > You are missing the crux of my use case, which is that I need to > generate a calendar to display to the user, with all the half-hour slots > from midnight to midnight for one day in it > > > Got it. How is this then > > start = datetime.combine(date, time(0)).astimezone() > while True: > end = (start + timedelta(hours=0.5)).astimezone() > print(start, end) > start = end > if end.time() == time(0): > break On DST transition days, this code does not generate exactly 48 slots, displayable to the user in a schedule that includes all hours from 0 to 23 labeled exactly once. That introduces an unacceptable level of additional display-layer complexity, and remains inferior to the "strict checking" solution I chose. Carl

2 3

Re: [Datetime-SIG] PEP 495 Q & A
by Alexander Belopolsky Aug. 25, 2015

Aug. 25, 2015

On Tue, Aug 25, 2015 at 3:56 PM, ISAAC J SCHWABACHER <ischwabacher(a)wisc.edu> wrote: > I like it. It's obvious from the field name what problem it solves, and > which value of the flag corresponds to which instant in time. Which problem "which" solves? Obvious! It tells you who is on first. (Answer: not "who" - "which"!)

1 0

Re: [Datetime-SIG] PEP-495 - Strict Invalid Time Checking
by Carl Meyer Aug. 25, 2015

Aug. 25, 2015

On 08/25/2015 02:10 PM, Alexander Belopolsky wrote: > > On Tue, Aug 25, 2015 at 3:36 PM, Carl Meyer <carl(a)oddbird.net > <mailto:carl@oddbird.net>> wrote: > > Given the unlikelihood of someone trying to schedule an appointment at > 2am, I didn't care much about the specific choice of user experience for > the DST gaps and folds; my overriding concerns were to a) not crash the > application when displaying a DST-transition day, and b) minimize the > harm done to my codebase (in fragility or added complexity) for the sake > of handling this rare edge case. > > > Here is my recommendation for your case. Assuming that taking user > entry is equivalent to parsing date and time string in some format and > scheduling is equivalent to writing a seconds since epoch timestamp into > some database, do the following post-PEP 495: > > dt = datetime.strptime(input, format) > s = dt.timestamp() > schedule(s) > # optionally > sdt = datetime.fromtimestamp(s) > if dt != sdt: > warn("You specified an invalid time %s, we scheduled your > appointment for %s", dt, sdt) > > Instead of a warning, your program may highlight the changed hour in read. > > This program will have no means of scheduling in the "fold=1" hour and > will correct you if you enter time that falls in a gap. The PEP 495 > design is motivated by precisely this type of application. You are missing the crux of my use case, which is that I need to generate a calendar to display to the user, with all the half-hour slots from midnight to midnight for one day in it (and I need the actual timezone-aware instants of the start and end time for each slot so that I can properly populate its availability status based on a database table of existing appointments, with start and end times stored in UTC). So your solution, which is all about "taking user entry," is not relevant to the use case. Carl

2 1

Re: [Datetime-SIG] PEP-495 - Strict Invalid Time Checking
by Carl Meyer Aug. 25, 2015

Aug. 25, 2015

Hi Alexander, On 08/25/2015 08:51 AM, Alexander Belopolsky wrote: > On Tue, Aug 25, 2015 at 9:30 AM, Stuart Bishop <stuart(a)stuartbishop.net > <mailto:stuart@stuartbishop.net>> wrote: > > As mentioned elsewhere, pytz requires strict checking to remain > backwards compatible. > > > Can you provide the specific examples where strict checking is > required? I mentioned my most recent use case for pytz's strict checking in another thread, but I did not describe it thoroughly and you may have missed it. I'll try to do better here, in case it's helpful. The system is a calendaring application that allows users to request an appointment in any open slot, where slots are fixed half-hour blocks. A day is displayed as a column of half-hour slots (48 per day). Some slots are open and available for the user to click and request an appointment; some are not. The system is multi-user and timezone-aware; different users may be in different timezones and must be able to schedule appointments with one another. To generate a day's calendar to display to the user, I `combine()` the day's date and the start/end time for each slot into a naive datetime, and `pytz.localize()` that naive datetime into the timezone of the user viewing the calendar. Those start/end datetimes, and information about the availability status of the slot, are included in a small data structure for each slot. This data structure is made available to the client-side (Javascript) code so it can display the slots with appropriate styling, and make the right appointment-request to the server if a user clicks an available slot. (Now, I'm sure there are various ways this design could be critiqued and explored; I'd personally find that interesting and enlightening, but I don't think it'd be useful to this thread. For the purposes of the thread, the question is whether the design is at least reasonable and functional. It is currently working well in production, and I believe it is reasonable.) Given the unlikelihood of someone trying to schedule an appointment at 2am, I didn't care much about the specific choice of user experience for the DST gaps and folds; my overriding concerns were to a) not crash the application when displaying a DST-transition day, and b) minimize the harm done to my codebase (in fragility or added complexity) for the sake of handling this rare edge case. Options I considered: 1) Just set the `is_dst` flag to either True or False always. In a fold, this inevitably results in at least one slot which ends before it starts. My slot data structure was built to raise an exception in that scenario. I didn't want to remove this useful sanity check just for this rare case. Also in a gap, this could result in apparently hour-and-a-half long slots, which introduced complexity on the display side. 2) Avoid "ends before it starts" by figuring out which value of `is_dst` meant "later" or "earlier" in each case and always using the earlier choice for "start" and the later choice for "end." Translating between `is_dst` and "earlier/later" isn't trivial to do generically, so I'd have to localize all datetimes twice. This solution also results in overlapping slots, which (without further changes to the design) could in some cases result in a single existing appointment being displayed to the user twice on the calendar. 3) Consider any slot with an invalid or ambiguous start or end time to be an invalid slot and always display it to the user as "unavailable", thus preventing any appointment from ever being requested in that slot. I implemented this by setting `is_dst=None` and catching `InvalidTimeError`, and it turned out to be a very simple solution to the problem. The effect is an always-unavailable couple of slots right around 2am twice a year, which is totally fine, and better than e.g. double-displaying appointments. I went through this exact progression of options and settled on the third one, and was quite happy with its simplicity and lack of cascading effects on the rest of the code. I'm sure I could have found other workable options, but it seems to me that at least this indicates that there are reasonable solutions to some problems where "strict checking" is useful. I'm aware that PEP 495 as it stands would let me write a function to do the strict checking myself (at the cost of localizing all my datetimes twice, I think). I could probably live with that, though the ugliness of it might have pushed me in a different direction, where the simplicity of `is_dst=None` in pytz made option 3 attractive. In any case, I don't feel strongly enough to argue any more about this (especially since I trust Stuart to maintain a backwards-compatible pytz API regardless -- thanks Stuart!); I'm just providing a real-world use case for strict checking, since you asked for one. Carl

2 2

Re: [Datetime-SIG] PEP 495 Q & A
by Carl Meyer Aug. 25, 2015

Aug. 25, 2015

On 08/25/2015 01:46 PM, Alexander Belopolsky wrote: > > On Tue, Aug 25, 2015 at 3:39 PM, Carl Meyer <carl(a)oddbird.net > <mailto:carl@oddbird.net>> wrote: >> >> Another possible name for the flag/index just occurred to me: what about >> `which`? > > That was in my very first proposal: > > """ > In other words, instead of localtime(dt, isdst=-1), we may want > localtime(dt, which=0) where "which" is used to resolve the ambiguity: > "which=0" means return the first (in UTC order) of the two times and > "which=1" means return the second. (In the non-ambiguous cases "which" is > ignored.) > """ -- https://mail.python.org/pipermail/python-dev/2015-April/139099.html > > The name did not catch up. Ha! Well in that case, consider this a vote of confidence in your intuition -- I think it's the best of the options that have been discussed. Carl

2 1

Re: [Datetime-SIG] PEP 495 Q & A
by Carl Meyer Aug. 25, 2015

Aug. 25, 2015

On 08/22/2015 08:20 PM, Tim Peters wrote: > [Alexander Belopolsky] >> ... >> Note that I did not include all suggestions for the name of the flag, but I >> thank everyone who made their suggestions. I think we are really left with >> two contenders: "fold" and "later." The only additional variant I would >> like to consider is "fold" with the integer values of 0 and 1. I think >> time(1, 30, fold=1) is short and sweet and looks better than time(1, 30, >> later=True). >> >> Note that neither spelling is self-explanatory, particularly if you see >> something like if dt.replace(later=True) < dt.replace(later=False) in >> someone's code, but the word "fold" points you in the right direction and is >> more Google-friendly than "later". >> >> The reason I think fold=0 and fold=1 may work better than booleans, is that >> you can think of the local time line as consisting of two "folds" one - the >> main timeline and the other a discontinuous line covering the fall-back >> hours. > > I'm on board with fold=0 and fold=1. I only hated "fold" when it was > False and True. Now we're indexing a theoretically unbounded sequence > of folds by an ordinal, which makes perfect sense - the later the > time, the larger the ordinal ;-) That's ok by me. It seems wrong to use `fold` to decide which side of a gap to choose, too -- but it's even more wrong to use `first` or `later` when they actually mean the reverse when disambiguating a gap. So I'll go with your explanation that a gap is a "negative fold" and be happy :-) Carl

2 2