[Numpy-discussion] NEP-18 comment

[Frederic Bastien]

That is a good point. In the worst case, if some library doesn’t check the env variable what would happen? Crash I suppose? If so, this isn’t too bad.

I think that the script that care about the small array case are mostly disjoint from the one that care about integration with other lib. But I could be wrong on that. I’m too close to machine learning/deep learning to evaluation well that point.

From: NumPy-Discussion <numpy-discussion-bounces+fbastien=nvidia.com at python.org> On Behalf Of Stephan Hoyer
Sent: Wednesday, March 6, 2019 3:41 PM
To: Discussion of Numerical Python <numpy-discussion at python.org>
Subject: Re: [Numpy-discussion] NEP-18 comment

On Wed, Mar 6, 2019 at 10:10 AM Frederic Bastien <fbastien at nvidia.com<mailto:fbastien at nvidia.com>> wrote:
Hi,

I was told recently about the NEP-18. I like it, but I have a comment.

At first, it is enabled in a release by setting an environment variable.
Then in the following release, it is enabled by default.

Is it possible to allow for the second release to disable it by an environment variable? This would allow to disable it for people that would be inconvenienced by this change.

Who could be negatively impacted by this? Small array operation.

I recall many years ago of much effort to speed up small array, including at least one GSoC. I didn’t do timing, but I’m pretty sure the change needed for NEP-19 raise the default overhead for all operation that it impact.
I also recall people in the mailing list asking how to speed up the small array case.
So giving people a way to not pay for that overhead would make sure that for people that care about the small array cases won’t pay that extra overhead price. So they won’t see a slowdown by updating NumPy.

Probably less than 1% of user will enable the new functionality in the current release as it is not enabled by default. Making it easy to try is great, but doesn’t guaranty that it will be tested in all corner cases. So it isn’t sure we would heard before the second release about regression this change does.

What do you think about that small changes?

Hi Frederic,

Thanks for raising these concerns (and thanks for your work on Theano!).

I agree, this seems like an easy and worthwhile change. The NEP-18 implementation has been rewritten in C, so I expect that the typical overhead will be quite minimal (about 1 us per function call), but I agree that there may be some important edge cases that we missed.

The only downside I can think of is that libraries using NEP-18 won't be able to simply rely upon the NumPy version for checking if it's supported -- they will also have to check an environment variable.

Best,
Stephan


Thanks for the great work on such important library.

Frédéric Bastien
________________________________
This email message is for the sole use of the intended recipient(s) and may contain confidential information.  Any unauthorized review, use, disclosure or distribution is prohibited.  If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
________________________________
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at python.org<mailto:NumPy-Discussion at python.org>
https://mail.python.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190306/82319eb9/attachment.html>