Mailman 3 September 2015 - Datetime-SIG

Re: [Datetime-SIG] Another round on error-checking
by Tim Peters Sept. 1, 2015

Sept. 1, 2015

[Alex] > I think the main difference between Tim's current proposal and what was > previously discussed is that all older proposals somehow required a third > value for fold. Note that there is a third variant suggested by Guido > off-list and discussed in the PEP: have fold=-1 by default, ignore it > unless it is nonnegative and design whatever you want for fold=0/1 without > concerns for backward compatibility. This effectively will give two > different datetime classes: classic and new. Both are perfectly consistent, > but if you think interoperation between naive and aware is confusing, try to > explain how new naive instances will interoperate with classic aware! It's worth some thought. I don't think interoperation between naive and aware now is confusing at all. It's usually just plain forbidden; e.g., >>> import datetime >>> x = datetime.datetime.now() # naive >>> y = x.replace(tzinfo=datetime.timezone.utc) # aware >>> x < y Traceback (most recent call last): File "<pyshell#5>", line 1, in <module> x < y TypeError: can't compare offset-naive and offset-aware datetimes >>> x - y Traceback (most recent call last): File "<pyshell#6>", line 1, in <module> x - y TypeError: can't subtract offset-naive and offset-aware datetimes >>> x == y False Only that last one may be surprising, but it's really just another way of saying "naive and aware don't mix, period". Do you have other kinds of interoperation in mind? Presumably a similarly high wall would be erected between fold < 0 and fold >= 0 instances. If this were pursued then, e.g., the seemingly intractable problem with __hash__() would go away (no more reason to _try_ to ignore fold >= 0), and, e.g., for an aware dt then dt.replace(fold=1) - dt.replace(fold=0) could return the expected result when dt specified an ambiguous time (ditto: no more reason to try to ignore fold==1), and likewise for comparing those values. I can see one kind of annoyance that would remain: dt2 = dt1 + a_timedelta is currently specified to force dt2.fold==0 even if dt1.fold==1. But that may not make good sense. There's no way to know whether adding `a_timedelta` takes dt1 out of a fold without doing timeline arithmetic. The conceptual mess in my head is that "fold=1" screams "I'm no longer in naive time", but "fold=0" does not (where "in naive time" means classic arithmetic is appropriate, and "not in naive time" means timeline arithmetic is appropriate - while fold < 0 would be an explicit way to say "in naive time", it's unclear that "fold >= 0" should always mean "not in naive time", despite that fold=1 makes no sense in naive time). At least it's all clear now ;-)

1 0

Re: [Datetime-SIG] Another round on error-checking
by Chris Angelico Sept. 1, 2015

Sept. 1, 2015

On Tue, Sep 1, 2015 at 12:59 PM, Tim Peters <tim.peters(a)gmail.com> wrote: >> I was under the impression that such cases are to be considered >> bugs. I know it was a driving concern in the implementation of the enum >> module. > > Sorry, I just had to laugh at the notion that enums _could_ be > implemented in such a convoluted way that there'd ever be the > slightest possibility of transitivity failing ;-) > Easy: you just declare that different enumerations are not comparable,, but that all are comparable to their base type. class Color(IntEnum): RED=1 GREEN=2 BLUE=3 class Permission(IntEnum): READ=1 WRITE=2 EXECUTE=3 What should Color.RED==Permission.READ give? True, because they're both 1? False, because they're completely different things? TypeError, because you can't logically compare colors and permissions? If you allow that Color.RED==1 (which you need to if it's going to be possible to backwardly-compatibly change raw numbers into an enum), then transitivity demands that the otherwise-illogical comparison above succeed, and be True. As a design decision, it could viably be taken either way, but once taken, it has to be maintained forever. ChrisA

1 0

Re: [Datetime-SIG] Another round on error-checking
by Ethan Furman Sept. 1, 2015

Sept. 1, 2015

On 08/31/2015 04:55 PM, Tim Peters wrote: [...] > At this point we have: > > u1 == dt1 == dt2 == u2 and u1 < u2 > hash(dt1) == hash(dt2) == hash(u1) > > (Parenthetically, note that despite the chain of equalities in the > first of those lines, we do _not_ have u1 == u2 - transitivity fails, > which is a bit of a wart by itself.) At this point are there any other cases in the stdlib where transitivity fails? I was under the impression that such cases are to be considered bugs. I know it was a driving concern in the implementation of the enum module. -- ~Ethan~

2 1

Re: [Datetime-SIG] Another round on error-checking
by Tim Peters Sept. 1, 2015

Sept. 1, 2015

[Alex] > It's getting late in my TZ, but what you are saying below sounds like a > complaint that if you put t=second 01:30 as a key in the dictionary, you > cannot later retrieve it by looking up t.astimezone(timezone.utc). I don't grasp that. What I am saying should be very clear: under all schemes so far, there are datetimes x and y such that x == y but hash(x) != hash(y). You see that or you don't. If you don't, I'll keep trying until you do ;-) So do you see that? > Sorry, but PEP 495 has never promised you that: "instances that differ > only by the value of fold will compare as equal. Applications that need to > differentiate between such instances should check the value of fold or > convert them to a timezone that does not have ambiguous times." Oh, come on. That's in the "Temporal Arithmetic" section: > <https://www.python.org/dev/peps/pep-0495/#temporal-arithmetic> There isn't a single instance of any kind of arithmetic in the example I gave, except for comparison, where I assumed only that comparison would behave the way the PEP _said_ it behaves. I'm not fighting the PEP here - I'm trying to illustrate a _consequence_ of what the PEP says. It's simply impossible to deduce from the paragraph above.that the fundamental invariant required for dict key types may fail. Here from the __hash__ docs: https://docs.python.org/3/reference/datamodel.html#object.__hash__ ... The only required property is that objects which compare equal have the same hash value; It's a violation of __hash__'s _only_ requirement, so even if there's no intent to fix it, the PEP needs to spell that out clearly. Code slinging dicts can fail in bizarre ways when the invariant is violated. > Maybe if we decide to do something with the arithmetic, we will be able to > fix this wart as well. Doubt it - this has nothing to do with arithmetic I can see. It's a consequence of wanting to ignore `fold` in contexts where it really does make a difference. __hash__() is one such place. Like I said at the start, it's a puzzle.

1 0

Re: [Datetime-SIG] Another round on error-checking
by Alexander Belopolsky Sept. 1, 2015

Sept. 1, 2015

On Mon, Aug 31, 2015 at 8:16 PM, Alexander Belopolsky < alexander.belopolsky(a)gmail.com> wrote: > Sorry, but PEP 495 has never promised you that: "instances that differ > only by the value of fold will compare as equal. Applications that need to > differentiate between such instances should check the value of fold or > convert them to a timezone that does not have ambiguous times." When I was writing some early drafts, I thought about advising users to use (local_datetime, local_datetime.fold) pairs as dictionary keys, but decided not to because using local_datetime.astimezone(timezone.utc) is a much better option. Now I think such advise may be relevant if a user truly has a need to sort out timestamps that come from many different timezones and for some reason wants to avoid conversion to UTC, but I don't think it will belong to the PEP.

1 0

Re: [Datetime-SIG] Another round on error-checking
by Alexander Belopolsky Sept. 1, 2015

Sept. 1, 2015

On Mon, Aug 31, 2015 at 7:55 PM, Tim Peters <tim.peters(a)gmail.com> wrote: > > [Alex] > >> After some thought, I believe the way to fix the implementation is what I > >> suggested at first: reset fold to 0 before calling utcoffset() in __hash__. > >> A rare hash collision is a small price to pay for having datetimes with > >> different timezones in the same dictionary. > > [Tim] > > Ya, I can live with that. In effect, we give up on converting to UTC > > correctly for purposes of computing hash(), but only in rare cases. > > hash() doesn't really care, and it remains true that datetime equality > > (which does care) still implies hash equality. The later and earlier > > of ambiguous times will simply land on the same hash chain. > > Nope, you wore me out prematurely ;-) > It's getting late in my TZ, but what you are saying below sounds like a complaint that if you put t=second 01:30 as a key in the dictionary, you cannot later retrieve it by looking up t.astimezone(timezone.utc). Sorry, but PEP 495 has never promised you that: "instances that differ only by the value of fold will compare as equal. Applications that need to differentiate between such instances should check the value of fold or convert them to a timezone that does not have ambiguous times." <https://www.python.org/dev/peps/pep-0495/#temporal-arithmetic> Maybe if we decide to do something with the arithmetic, we will be able to fix this wart as well. > > Consider datetimes dt1 and dt2 representing the earlier & later of an > ambiguous time in their common zone (whatever it may be - doesn't > matter). Then all fields are identical except for `fold`. Assume > __hash__ forces `fold` to 0 before obtaining the UTC offset. Then we > have: > > dt1 == dt2 > hash(dt1) == hash(dt2) > > Fine so far as it goes. Now do: > > u1 = dt1.astimezone(timezone.utc) > u2 = dt2.astimezone(timezone.utc) > > At this point we have: > > u1 == dt1 == dt2 == u2 and u1 < u2 > hash(dt1) == hash(dt2) == hash(u1) > > (Parenthetically, note that despite the chain of equalities in the > first of those lines, we do _not_ have u1 == u2 - transitivity fails, > which is a bit of a wart by itself.) > > Since u1 == dt1, and hash(u1) == hash(dt1), no problem there either. > > But u1 isn't at all the same as u2, so hash(u2) can be the same as > hash(u1) only by (unlikely) accident. hash(u2) is off in a world of > its own. Therefore hash(dt2) can be the same as hash(u2) only by (the > same unlikely) accident, despite that dt2 == u2. > > So, in all, __hash__ forcing fold=0 at the start hides the problem for > ambiguous times in the same zone, but doesn't really touch the problem > for cross-zone equivalent spellings of such times (not even if one of > the zones is UTC, which is likely the most important case). > > One way to fix that is to have datetime.__hash__() _always_ return, say, 0 ;-)

1 0