[Numpy-discussion] Datetime again

[Stephan Hoyer]

On Wed, Jan 28, 2015 at 5:13 PM, Chris Barker <chris.barker at noaa.gov> wrote:

> I tend to agree with Nathaniel that a ndarray subclass is less than ideal
> -- they tend to get ugly fast. But maybe that is the only way to do
> anything in Python, short of a major refactor to be able to write a dtype
> in Python -- which would be great, but sure sounds like a major project to
> me.
>

My vote would be for using composition rather than inheritance. So
DatetimeArray should contain but not be an ndarray, making use of
appropriate APIs like __array__, __array_wrap__ and __numpy_ufunc__.

And as for " The 64 bits of long long really isn't enough and leads to all
> sorts of compromises". not long enough for what? I've always thought that
> what we need is the ability to set the epoch. Does anyone ever need
> picoseconds since 100 years ago? And if they did, we'd be in a heck of a
> mess with leap seconds and all that anyway.
>

I agree pretty strongly with the Blaze docs with respect to time units. I
think fixed precision int64 is probably OK (simplifying things quite a
bit), but the ns precision chosen by pandas was probably a mistake (not a
big enough range). The main advantage of using a single array for the
underlying data is that it's very straightforward to drop in a Cython or
Numba or whatever for performance critical steps.

In my mind, the main advantage of using floating point math is that NaT
(not a time) becomes much easier to represent and work with -- you can
share map it to NaN. Handling NaT is a major source of complexity for the
datetime operations in pandas.

The other thing to consider is how much progress has been made on the
datetime dype in DyND, which is where the "numpy replacement" part of Blaze
has ended up. I know some sort of datetime object *has* been implemented,
though from my tests it does not really appear to be in fully working
condition at this point (e.g., there does not appear to be a corresponding
timedelta time):
https://github.com/libdynd/dynd-python

Stephan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20150128/40247adc/attachment.html>