Nice overview! In my opinion section 5. Datetimes could benefit from some clarifications, and contains a couple of typos. Let me start with the typos. I read "As in pure python, np.datetime64 is accompained by np.timedelta64 (stored as a single np.uint64) with the expectable arithmetic operators.” but it should be “np.int64” since time delta values are signed. "Leap seconds are supported: <code snippet> Leap seconds are not (but proposed) <code snippet>" Of course the first sentence should be “leap years”, which leads to my main point. It makes no sense to claim “leap year support” without specifying the relevant calendar. Thus I would suggest to clearly state, from the very begin of this section, that - calendrical calculations are performed using a proleptic Gregorian calendar <https://en.wikipedia.org/wiki/Proleptic_Gregorian_calendar <https://en.wikipedia.org/wiki/Proleptic_Gregorian_calendar>>, - Posix semantics is followed, i.e. each day comprises exactly 86400 SI seconds, thus ignoring the existence of leap seconds. I would also point out that this choice is consistent with python datetime. As what regards the promised future support for leap seconds, I would not mention it, for now. In fact leap second support requires a leap second table, which is not available on all platforms supported by numpy. Therefore the leap second table should be bundled and updated with every numpy release with the very undesirable effect that older version (with outdated tables) would behave differently from newer ones. Stefano
On Tue, Dec 28, 2021 at 9:12 PM Stefano Miccoli <stefano.miccoli@polimi.it> wrote: Hi Stefano,
Nice overview!
Thanks!
Of course the first sentence should be “leap years”, which leads to my main point.
Sure, I've already fixed this one, it just hasn't found its way online yet.
It is not a matter of formal definitions. Leap seconds are uncompromisingly practical. If you look at the wall clock on 1 Jan 1970 00:00 and then look at the same clock today and measure the difference with atomic clock you won't get the time delta that np.timedelta64 reports. There will be a difference of ~37 seconds. One would expect that a library claiming to work with attoseconds would at least count the seconds correctly ) Astropy library calculates <https://het.as.utexas.edu/HET/Software/Astropy-1.0/api/astropy.time.TimeGPS....> them properly: "GPS Time. Seconds from 1980-01-06 00:00:00 UTC For example, 630720013.0 is midnight on January 1, 2000."
np.datetime64('2000-01-01', 's') - np.datetime64('1980-01-06', 's') numpy.timedelta64(630720000,'s')
Everything should be made as simple as possible but not simpler. Leap seconds are an inherent part of the world we live in. Eg this is how people deal with them currently: they have to parse times like 23:59:60.209215 manually https://stackoverflow.com/questions/21027639/python-datetime-not-accounting-... - calendrical calculations are performed using a proleptic Gregorian
But not consistent with python time ;) "Unlike the time module, the datetime module does not support leap seconds." • time.CLOCK_TAI International Atomic Time The system must have a current leap second table in order for this to give the correct answer. PTP or NTP software can maintain a leap second table. Availability: Linux. New in version 3.9.
The olson database is much larger yet it is updated on millions of computers, phones and what not without causing extra difficulties (except when the government unexpectedly decides to shift a region from one TZ to another). This way developers have a choice whether to work with naive datetimes (ok in a single timezone without daylight-saving) or with timezone-aware ones (and take care about updating the pytz). This is how astropy deals with updating the table: https://docs.astropy.org/en/stable/api/astropy.utils.iers.LeapSeconds.html Pytz also has this table both inside the binary tz files and in a text file: https://github.com/stub42/pytz/blob/master/tz/leap-seconds.list which it in turn downloads from NIST ftp://ftp.nist.gov/pub/time/leap-seconds.list It is in the public domain, NIST updates this file regularly and it even has an expiration date (presently it is 28 June 2022). Activation of the 'leap-second-aware mode' could be made dependent on the presence of the pytz mode and/or this expiration date. I don't think having a non-default leap-second-aware mode would hurt anyone, but I also wouldn't consider it a priority. I think when someone needs them he'll make a patch and until that moment it is safe to have them as 'proposed' ) I feel that leap seconds should be mentioned somewhere—in the article or in the docs, because it limits practical precise usage of timedelta64 to a period between 2021 and 2016 (last time when a leap second was injected). A modest timespan for a library claiming to work with years upto 9.2e18 BC ;) Thank you for your suggestions! I've included them into the article, plz have a look at the updated version. Best regards, Lev Stefano
On Tue, Dec 28, 2021 at 9:12 PM Stefano Miccoli <stefano.miccoli@polimi.it> wrote: Hi Stefano,
Nice overview!
Thanks!
Of course the first sentence should be “leap years”, which leads to my main point.
Sure, I've already fixed this one, it just hasn't found its way online yet.
It is not a matter of formal definitions. Leap seconds are uncompromisingly practical. If you look at the wall clock on 1 Jan 1970 00:00 and then look at the same clock today and measure the difference with atomic clock you won't get the time delta that np.timedelta64 reports. There will be a difference of ~37 seconds. One would expect that a library claiming to work with attoseconds would at least count the seconds correctly ) Astropy library calculates <https://het.as.utexas.edu/HET/Software/Astropy-1.0/api/astropy.time.TimeGPS....> them properly: "GPS Time. Seconds from 1980-01-06 00:00:00 UTC For example, 630720013.0 is midnight on January 1, 2000."
np.datetime64('2000-01-01', 's') - np.datetime64('1980-01-06', 's') numpy.timedelta64(630720000,'s')
Everything should be made as simple as possible but not simpler. Leap seconds are an inherent part of the world we live in. Eg this is how people deal with them currently: they have to parse times like 23:59:60.209215 manually https://stackoverflow.com/questions/21027639/python-datetime-not-accounting-... - calendrical calculations are performed using a proleptic Gregorian
But not consistent with python time ;) "Unlike the time module, the datetime module does not support leap seconds." • time.CLOCK_TAI International Atomic Time The system must have a current leap second table in order for this to give the correct answer. PTP or NTP software can maintain a leap second table. Availability: Linux. New in version 3.9.
The olson database is much larger yet it is updated on millions of computers, phones and what not without causing extra difficulties (except when the government unexpectedly decides to shift a region from one TZ to another). This way developers have a choice whether to work with naive datetimes (ok in a single timezone without daylight-saving) or with timezone-aware ones (and take care about updating the pytz). This is how astropy deals with updating the table: https://docs.astropy.org/en/stable/api/astropy.utils.iers.LeapSeconds.html Pytz also has this table both inside the binary tz files and in a text file: https://github.com/stub42/pytz/blob/master/tz/leap-seconds.list which it in turn downloads from NIST ftp://ftp.nist.gov/pub/time/leap-seconds.list It is in the public domain, NIST updates this file regularly and it even has an expiration date (presently it is 28 June 2022). Activation of the 'leap-second-aware mode' could be made dependent on the presence of the pytz mode and/or this expiration date. I don't think having a non-default leap-second-aware mode would hurt anyone, but I also wouldn't consider it a priority. I think when someone needs them he'll make a patch and until that moment it is safe to have them as 'proposed' ) I feel that leap seconds should be mentioned somewhere—in the article or in the docs, because it limits practical precise usage of timedelta64 to a period between 2021 and 2016 (last time when a leap second was injected). A modest timespan for a library claiming to work with years upto 9.2e18 BC ;) Thank you for your suggestions! I've included them into the article, plz have a look at the updated version. Best regards, Lev Stefano
participants (2)
-
Lev Maximov -
Stefano Miccoli