[Numpy-discussion] is __array_ufunc__ ready for prime-time?

Nathan Goldbaum nathan12343 at gmail.com
Thu Nov 2 18:33:16 EDT 2017

On Thu, Nov 2, 2017 at 5:21 PM, Stephan Hoyer <shoyer at gmail.com> wrote:

> On Thu, Nov 2, 2017 at 12:42 PM Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
>> Would this issue be ameliorated given Nathaniel's proposal to try to move
>> away from subclasses and towards storing data in dtypes? Or would that just
>> mean that xarray would need to ban dtypes it doesn't know about?
> Yes, I think custom dtypes would definitely help. Custom dtypes have a
> well contained interface, so lots of operations (e.g., concatenate,
> reshaping, indexing) are guaranteed to work in a dtype independent way. If
> you try to do an unsupported operation for such a dtype (e.g.,
> np.datetime64), you will generally get a good error message about an
> invalid dtype.
> In contrast, you can overload a subclass with totally arbitrary semantics
> (e.g., np.matrix) and of course for duck-types as well.
> This makes a big difference for libraries like dask or xarray, which need
> a standard interface to guarantee they do the right thing. I'm pretty sure
> we can wrap a custom dtype ndarray with units, but there's no way we're
> going to support np.matrix without significant work. It's hard to know
> which is which without well defined interfaces.

Ah, but what if the dtype modifies the interface? That might sound evil,
but it's something that's been proposed. For example, if I wanted to
replace yt's YTArray in a backward compatibile way with a dtype and just
use plain ndarrays everywhere, the dtype would need to *at least* modify
ndarray's API, adding e.g. to(), convert_to_unit(), a units attribute, and
several other things.

Of course if I don't care about backward compatibility I can just do all of
these operations on the dtype object itself. However, I suspect whatever
implementation of custom dtypes gets added to numpy, it will have the
property that it can act like an arbitrary ndarray subclass otherwise
libraries like yt, Pint, metpy, and astropy won't be able to switch to it.


> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20171102/40bcce4e/attachment-0001.html>

More information about the NumPy-Discussion mailing list