[Pandas-dev] tslibs 2.0 and non-nanosecond datetime64/timedelta64

[Brock Mendel]

This is a discussion of what it would take to support non-nanosecond
datetime64/timedelta64 dtypes and what decisions would need to be made
along the way.

The implementation would probably consist of:
- add a NPY_DATETIMEUNIT attribute to Timestamp and Datetime64TZDtype
- for timezone-related methods:
    - short-term: cast to nanosecond, use existing code, cast back to other
unit
    - longer-term: update existing code to support non-nano units directly
- comb through the code for all the places where we implicitly assume nano
units and update
- tests, so, so many tests

We could then consider de-duplication. Tick is already redundant with
Timedelta, and Timestamp[H] would render Period[H] redundant.  With
appropriate deprecation cycle, we could rip out a bunch of code.

Another possibility is to try to upstream some code to numpy, which they
have recently been receptive to (#16266
<https://github.com/numpy/numpy/pull/16266>, #16363
<https://github.com/numpy/numpy/pull/16363>, #16364
<https://github.com/numpy/numpy/pull/16364>, #16352
<https://github.com/numpy/numpy/issues/16352>,
<https://github.com/numpy/numpy/issues/16195>#16195
<https://github.com/numpy/numpy/issues/16195>).  @rgommers tells me that
trying to implement a tz-aware datetime64 dtype in numpy would be "folly,
that way madness lies", but that it might be more feasible once @seberg's
dtype refactor lands.  More realistically short-term, if we convinced numpy
to update NPY_DATETIMEUNIT to include the anchored quarter/year/week units
we use for Period, we could condense a lot of confusing enum-like code.

Tangentially related: with zoneinfo (PEP 615) we should consider making
those our canonical tzinfos and converting any dateutil/pytz tzinfos we
encounter to those.  They are implemented in C, so I'm _hopeful_ we can
make some of our vectorized tzconversion code unnecessary.  @pganssle has
suggested we implement our own tzinfos, but I'm holding out hope we can
keep that upstream.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pandas-dev/attachments/20200529/52fcb388/attachment.html>