On Thu, Nov 2, 2017 at 12:42 PM Nathan Goldbaum email@example.com wrote:
Would this issue be ameliorated given Nathaniel's proposal to try to move away from subclasses and towards storing data in dtypes? Or would that just mean that xarray would need to ban dtypes it doesn't know about?
Yes, I think custom dtypes would definitely help. Custom dtypes have a well contained interface, so lots of operations (e.g., concatenate, reshaping, indexing) are guaranteed to work in a dtype independent way. If you try to do an unsupported operation for such a dtype (e.g., np.datetime64), you will generally get a good error message about an invalid dtype.
In contrast, you can overload a subclass with totally arbitrary semantics (e.g., np.matrix) and of course for duck-types as well.
This makes a big difference for libraries like dask or xarray, which need a standard interface to guarantee they do the right thing. I'm pretty sure we can wrap a custom dtype ndarray with units, but there's no way we're going to support np.matrix without significant work. It's hard to know which is which without well defined interfaces.