is __array_ufunc__ ready for prime-time?
![](https://secure.gravatar.com/avatar/41bd6094402273d78b018e646323f18c.jpg?s=120&d=mm&r=g)
Right before 1.12, I arranged an API around an np.ndarray subclass, making use of __array_ufunc__ to customize behavior based on structured dtype (we come from c++ and really like operator overloading). Having seen __array_ufunc__ featured in Travis Oliphant's Guide to NumPy: 2nd Edition, I assumed this was the way to go. But it was removed in 1.12. Now that 1.13 has reintroduced __array_ufunc__, can I now rely on its continued availability? I am considering using it as a base-level component in several libraries... is this a dangerous idea? Thanks! Will -- William H. Sheffler Ph.D. Principal Engineer Institute for Protein Design University of Washington
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
Hi Will, We spent a *long time* sorting out the messy details of __array_ufunc__ [1], especially for handling interactions between different types, e.g., between numpy arrays, non-numpy array-like objects, builtin Python objects, objects that override arithmetic to act in non-numpy-like ways, and of course subclasses of all the above. We hope that we have it right this time, but as we wrote in the NumPy 1.13 release notes "The API is provisional, we do not yet guarantee backward compatibility as modifications may be made pending feedback." That said, let's give it a try! If any changes are necessary, I expect it would likely relate to how we handle interactions between different types. That's where we spent the majority of the design effort, but debate is a poor substitute for experience. I would be very surprised if the basic cases (one argument or two arguments of the same type) need any changes. Best, Stephan [1] https://docs.scipy.org/doc/numpy-1.13.0/neps/ufunc-overrides.html On Fri, Oct 27, 2017 at 12:39 PM William Sheffler <willsheffler@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/851ff10fbb1363b7d6111ac60194cc1c.jpg?s=120&d=mm&r=g)
Just to second Stephan's comment: do try it! I've moved astropy's Quantity over to it, and am certainly counting on the basic interface staying put... -- Marten
![](https://secure.gravatar.com/avatar/7857f26c1ef2e9bdbfa843f9087710f7.jpg?s=120&d=mm&r=g)
I’m using it in yt. If we were able to drop support for all old numpy versions, switching would allow me to delete hundreds of lines of code. As-is since we need to simultaneously support old and new versions, it adds some additional complexity. If you’re ok with only supporting numpy >= 1.13, array_ufunc will make you life a lot easier. On Fri, Oct 27, 2017 at 6:55 PM Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/851ff10fbb1363b7d6111ac60194cc1c.jpg?s=120&d=mm&r=g)
Hi Nathan, Happy to hear that it works well for yt! In astropy's Quantity as well, it greatly simplifies the code, and has made many operations about two times faster (which is why I pushed so hard to get __array_ufunc__ done...). But for now we're stuck with supporting __array_prepare__ and __array_wrap__ as well. Of course, do let us know if there are issues with the current design. Some further cleanup, especially with the use of `super` for ndarray subclasses, is still foreseen (but none of that should impact the API). All the best, Marten
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
Hi Will, We spent a *long time* sorting out the messy details of __array_ufunc__ [1], especially for handling interactions between different types, e.g., between numpy arrays, non-numpy array-like objects, builtin Python objects, objects that override arithmetic to act in non-numpy-like ways, and of course subclasses of all the above. We hope that we have it right this time, but as we wrote in the NumPy 1.13 release notes "The API is provisional, we do not yet guarantee backward compatibility as modifications may be made pending feedback." That said, let's give it a try! If any changes are necessary, I expect it would likely relate to how we handle interactions between different types. That's where we spent the majority of the design effort, but debate is a poor substitute for experience. I would be very surprised if the basic cases (one argument or two arguments of the same type) need any changes. Best, Stephan [1] https://docs.scipy.org/doc/numpy-1.13.0/neps/ufunc-overrides.html On Fri, Oct 27, 2017 at 12:39 PM William Sheffler <willsheffler@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/851ff10fbb1363b7d6111ac60194cc1c.jpg?s=120&d=mm&r=g)
Just to second Stephan's comment: do try it! I've moved astropy's Quantity over to it, and am certainly counting on the basic interface staying put... -- Marten
![](https://secure.gravatar.com/avatar/7857f26c1ef2e9bdbfa843f9087710f7.jpg?s=120&d=mm&r=g)
I’m using it in yt. If we were able to drop support for all old numpy versions, switching would allow me to delete hundreds of lines of code. As-is since we need to simultaneously support old and new versions, it adds some additional complexity. If you’re ok with only supporting numpy >= 1.13, array_ufunc will make you life a lot easier. On Fri, Oct 27, 2017 at 6:55 PM Marten van Kerkwijk < m.h.vankerkwijk@gmail.com> wrote:
![](https://secure.gravatar.com/avatar/851ff10fbb1363b7d6111ac60194cc1c.jpg?s=120&d=mm&r=g)
Hi Nathan, Happy to hear that it works well for yt! In astropy's Quantity as well, it greatly simplifies the code, and has made many operations about two times faster (which is why I pushed so hard to get __array_ufunc__ done...). But for now we're stuck with supporting __array_prepare__ and __array_wrap__ as well. Of course, do let us know if there are issues with the current design. Some further cleanup, especially with the use of `super` for ndarray subclasses, is still foreseen (but none of that should impact the API). All the best, Marten
participants (4)
-
Marten van Kerkwijk
-
Nathan Goldbaum
-
Stephan Hoyer
-
William Sheffler