Hi Sebastian,
On Wed, Sep 18, 2019 at 4:35 PM Sebastian Berg
Hi all,
to try and make some progress towards a decision since the broad design is pretty much settling from my side. I am thinking about making a meeting, and suggest Monday at 11am Pacific Time (I am open to other times though).
My hope is to get everyone interested on board, so that we can make an informed decision about the general direction very soon. So just reach out, or discuss on the mailing list as well.
The current draft for an NEP is here: https://hackmd.io/kxuh15QGSjueEKft5SaMug?both
There are some design goals that I would like to clear up.
The design itself seems very sensible to me insofar as I understand it.
After having read your document again, I think you're still missing the
actual goals though. "structure of class layout" and "type hierarchy" are
important, but they're not the goals. You're touching on the real goals in
places, but it may be valuable to be much more explicit there.
Here are some example goals:
1. Make creating new dtypes via the NumPy C API take >4x less lines of code
on average (in practice: for rational/quaternion, hard to measure
otherwise).
2. Make it possible to create new dypes with full functionality via the
NumPy Python API. Performance may be up to 1-2 orders of magnitude worse
than when creating the same dtype via the C API; the main purpose is to
allow easier prototyping of new dtypes.
3. Make the NumPy codebase more maintainable by removing special-casing of
datetime dtypes in many places.
4. Enable creation of a units library whose arrays *are* numpy arrays
rather than a subclass or duck array. This will make such a library work
much better with SciPy and other existing libraries that use np.asarray
extensively.
5. Hide currently exposed implementation details in the C API so long-term
.... (you have this one, but it would be nice to work it out a little more
- after all we recently considered reverting the deprecation for direct
field access, so how important is this?)
6. Improve casting behavior for external dtypes
7. Make np.char behavior better
import datetime
import pandas as pd
import datetime
dti = pd.to_datetime(['1/1/2018', np.datetime64('2018-01-01'), ... datetime.datetime(2018, 1, 1)])
dti.values
array(['2018-01-01T00:00:00.000000000', '2018-01-01T00:00:00.000000000', '2018-01-01T00:00:00.000000000'], dtype='datetime64[ns]')
dti.values.dtype
dtype(' isinstance(dti.values.dtype, np.dtype) True dti.dtype == dti.values.dtype # okay, that's nice True start = pd.to_datetime('2015-02-24') rng = pd.date_range(start, periods=3) t = pd.Series(rng) t_withzone = t.dt.tz_localize('UTC').dt.tz_convert('Asia/Kolkata') t_withzone 0 2015-02-24 05:30:00+05:30
1 2015-02-25 05:30:00+05:30
2 2015-02-26 05:30:00+05:30
dtype: datetime64[ns, Asia/Kolkata] t_withzone.dtype datetime64[ns, Asia/Kolkata] t_withzone.values.dtype t_withzone.dtype == t_withzone.values.dtype # could this be True in dtype('