4kir4.1i at gmail.com
Thu Aug 27 13:33:13 CEST 2015
Tim Peters <tim.peters at gmail.com> writes:
> [Akira Li <4kir4.1i at gmail.com>]
> Nobody has said some apps don't need reliable conversions (to the
> contrary, that's the primary _point_ of PEP 495). Nobody has said
> some apps don't need timeline arithmetic - although I have said it's
> poor practice to even _try_ to do timeline arithmetic if an app isn't
> working in UTC or with naive datetimes. If an app is following best
> practice (UTC or naive datetimes), then timeline arithmetic is what
> they _always_ get (it's the same thing as classic arithmetic in those
I agree on the best practices here. I would prefer that __add__ would be
forbidden for local timezones unless they have a fixed utc offset. But
it might be too late for that now.
If __add__ is allowed for timezone-aware datetime objects then
arithmetic "as though via conversion to utc time" is *equally valid* as
the arithmetic "as though it is a timezone-naive datetime object".
>> Non-pytz timezones make mistake on the order of an hour regularly.
>> It is *three orders of magnitude larger* than a second. It is a different
>> class of errors. The code that can't handle ~1s errors over short period
>> of time should use time.monotonic() anyway.
> Apps that care about leap seconds _should_ be using TAI. Apps that
> want timeline arithmetic _should_ be using UTC. Unfortunately, people
> shoot themselves in the feet all the time. Python can't stop that.
> But it doesn't have to _cater_ to poor practices either.
By your logic: Apps that care about timezone-naive arithmetic _should_
be using naive datetime objects.
I agree it is a poor practice to perform arithmetic on localized time.
But as long as such arithmetic *is* allowed then it *is* ambiguous what
type of arithmetic should be used. There is no *one obvious* way here.
>> dateutil doesn't work during DST transitions but PEP 495 might allow to
>> fix it.
> I don't know what "doesn't work" means, precisely. There are certain
> behaviors that do and don't work as you might hope. For example, even
> the stupidest possible tzinfo implementation that follows the docs
> today has no problem converting from UTC to local time across DST
> transitions - the default .fromutc() was designed to ensure that
> conversion in _that_ direction mimics the local clock in all cases
> (including skipping a local hour at DST start, and repeating a local
> hour at DST end - where "hour" really means "whole number of
> minutes"). What's impossible now (outside of pytz) is converting
> ambiguous local times _back_ to UTC in all cases. PEP 495 will repair
> that - that's its primary point. There's no "might" about it. But,
> for that to be of use to dateutil users, dateutil will need to change
> its tzinfo implementation to meet 495's new tzinfo requirements.
I've linked to a couple of dateutil bugs previously in PEP-431/495
I was surprised as you that dateutil .fromutc() appears to be broken.
I use "might" because I haven't read dateutil code. I can't be sure
e.g., what backward-compatibility concerns might prevent PEP 495 fix its
issues with an ambigous local time. Timezones is a very complicated
topic -- no solution works in the general case.
>> As I understand, outside of DST transitions if dates are unique valid
>> local times; dateutil uses "same time tomorrow":
>> (d_with_dateutil_tzinfo + DAY ==
>> d.tzinfo.localize(d.replace(tzinfo=None) + DAY, is_dst=None))
>> while pytz uses "+24 hours":
>> dt_add(d_with_dateutil_tzinfo, DAY) == d + DAY
>> where dt_add() is defined below. The equility works but (d + DAY) may
>> have a wrong tzinfo object if the arithmetic crosses DST boundaries (but
>> it has correct timestamp/utc time anyway). d.tzinfo.normalize(d + DAY)
>> should be used to get the correct tzinfo e.g. for displaying the result.
>> Both types of operations should be supported.
> If you're saying that classic and timeline arithmetic both have
> legitimate uses, sure. Nobody has said otherwise. If you're trying
> to say more than just that, sorry, I missed the point.
Yes, it is exactly my point.
The only my objection that timezone-naive arithmetic is somehow superior
for localized times. Though I don't mind it as long as timezone
conversions would work.
>> If dateutil can be fixed to work correctly using the disambiguation flag
>> then its behavior is preferable because it eliminates localize,
>> normalize calls
> Then you get classic arithmetic. Which is not only fine by me, I
> believe it's the only realistic outcome for the reasons explained just
The key word here is "If". *If* it works; great. It is still possible to
perform both types of arithmetic as the examples above demonstrate.
>> If people forget localize() then tzinfo is not attached and an exception
>> is raised later. It is like mixing bytes and Unicode: if you forget
>> decode() then an exception is raised later.
> AFAICT, pytz can't enforce anything. You don't _need_ to call
> localize() to get _a_ datetime. From scanning message boards, e.g., I
> see it's a common mistake for new pytz users to use
> datetime.datetime(..., tzinfo=...;) directly, not using localize() at
> all, despite the very clear instructions in the docs that they must
> _not_ do that...
I had this (incorrect as I've now realized) picture in mind:
naive = datetime.strptime(...)
aware = tz.localize(naive, is_dst=None)
If localize() is forgotten then we would get a naive object that would
raise an exception in tzinfo-related methods later.
Yes. datetime(..., tzinfo=...) is a common error and I don't see how
*without a performance hit* pytz can't fix it with PEP 495 *alone*. But
it *can* be fixed if the performance is not a concern (perhaps with a
slight change in the pickle semantics if relevant tzdata has changed).
> ...That can be a real problem for modules fighting basic
> design warts: newcomers are lost at first, and even experts can have
> trouble inter-operating with code _outside_ what typically becomes an
> increasingly self-contained world (e.g., Isaac cheerfully complained
> earlier about his pains trying to get pytz and dateutil to work
I agree, the usability is a real issue (for newcomers and experts).
But dateutil doesn't work  in cases where pytz does work and
therefore if people use dateutil now; the correctness is not their
primary concern and it should be much easier to combine the two
libraries if you don't actually need correct answers.
More information about the Datetime-SIG