[Numpy-discussion] Proposal: add the timestamp64 type (Noam Yorav-Raphael)

[Noam Yorav-Raphael]

Hi Matti and Stefano,

My understanding is that datetime64 was decided to be neither TAI nor posix
time, but rather represent an abstract calendar point, like
datetime.datetime without a specified timezone. This can usually be
converted into posix time given a timezone (although in the "repeated" hour
between DST and winter time there will be ambiguity!) If it is agreed by
all users that a datetime64 represents the time in UTC, it is the same as
posix time.

I would like to have a type that is defined to be equivalent to posix time.
I don't agree with Stefano, I think that posix time is very useful (as I
think its ubiquity shows that), and I think that a type that is defined to
be posix time would also be very useful. I think that posix time is well
suited for the vast majority of use cases. Indeed, there are use cases
where you should take into account leap seconds, but those are rare. In
practice, a leap second would be presented by the OS as a second that
actually takes more than a second. This actually happens all the time
without leap seconds - when your computer automatically syncs with ntp, it
adjusts the time continuously, so applications will not experience "time
bumps". If you want to make sure that the intervals you measure are
correct, you should use something like time.monotonic().

So, most users are not interested in very precise time measurements, but
rather in knowing what happened before what, and roughly when. For this,
posix time is great - it's very simple, and does the job. In some cases you
need to take into account leap seconds, but in those cases, just using the
computer clock will not give you the precision you need no matter what - so
you'll need specialized software anyway.

I think that posix time is great, and since it's very easy to make wrong
decisions that seem to work until you discover they don't (such as
discovering too late that local time won't work when you are not sure of
the time zone, or when you switch from DST to winter time), a sane and
simple default is important.

Cheers,
Noam





On Thu, Nov 12, 2020 at 6:41 PM Matti Picus <matti.picus at gmail.com> wrote:

>
> On 11/12/20 6:04 PM, Stefano Miccoli wrote:
> >
> >
> >> On 11 Nov 2020, at 18:00, numpy-discussion-request at python.org
> >> <mailto:numpy-discussion-request at python.org> wrote:
> >>
> >> I propose to add a new type called "timestamp64". It will be a pure
> >> timestamp, meaning that it represents a moment in time (as
> >> seconds/ms/us/ns since the epoch), without any timezone information.
> >
> > Sorry, but I really don see the usefulness for another time stamping
> > format based on POSIX time. Indeed POSIX time is based on a naive
> > approximation of UTC and is ambiguous across leap seconds. Quoting
> > from Wikipedia <https://en.wikipedia.org/wiki/Unix_time#Leap_seconds>
> >
> > ...
>
>
> In a one-on-one discussion with Noam in a pre-community call (that, how
> ironically, we had time for since we both messed up the meeting
> time-zone change) we reached the conclusion that the request is to
> clarify whether NumPy's datetime64 represents TAI time [0] or POSIX
> time, with a preferecne for TAI time. The documentation mentions POSIX
> time[1]. As Stefano points out, there is a couple of seconds difference
> between POSIX (or Unix) time and TAI time. In practice numpy simply
> stores a int64 value to represent the datetime64, and relies on others
> to convert it. The leap-second might be getting lost in the conversions.
> So it might make sense to clarify exactly how those conversions deal
> with the leap-seconds and choose which one we mean when we use
> datetime64. Noam please correct me if I am mistaken.
>
>
> Matti
>
>
> [0] https://en.wikipedia.org/wiki/International_Atomic_Time
>
> [1]
> https://numpy.org/doc/stable/reference/arrays.datetime.html#datetime-units
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/numpy-discussion/attachments/20201112/ec358c2f/attachment.html>