[Numpy-discussion] Short-circuiting equivalent of np.any or np.all?

[Nathan Goldbaum]

On Thu, Apr 26, 2018 at 12:03 PM Joseph Fox-Rabinovitz <
jfoxrabinovitz at gmail.com> wrote:

> Would it be useful to have a short-circuited version of the function that
> is not a ufunc?
>

Yes definitely. I could use numba as suggested by Hameer but I'd rather not
add a new runtime dependency. I could use cython or C but I'd need to deal
with the packaging headaches of including C code in your package.

I guess I could also create a new project that just implements the
functions I need in cython, deal with the packaging headaches there, and
then depend on that package. At least that way others won't need to deal
with the pain :)


> - Joe
>
> On Thu, Apr 26, 2018 at 12:51 PM, Hameer Abbasi <einstein.edison at gmail.com
> > wrote:
>
>> Hi Nathan,
>>
>> np.any and np.all call np.or.reduce and np.and.reduce respectively, and
>> unfortunately the underlying function (ufunc.reduce) has no way of
>> detecting that the value isn’t going to change anymore. It’s also used for
>> (for example) np.sum (np.add.reduce), np.prod (np.multiply.reduce),
>> np.min(np.minimum.reduce), np.max(np.maximum.reduce).
>>
>> You can find more information about this on the ufunc doc page
>> <https://docs.scipy.org/doc/numpy/reference/ufuncs.html>. I don’t think
>> it’s worth it to break this machinery for any and all, as it has numerous
>> other advantages (such as being able to override in duck arrays, etc)
>>
>> Best regards,
>> Hameer Abbasi
>> Sent from Astro <https://www.helloastro.com> for Mac
>>
>> On Apr 26, 2018 at 18:45, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
>>
>>
>> Hi all,
>>
>> I was surprised recently to discover that both np.any and np.all() do not
>> have a way to exit early:
>>
>> In [1]: import numpy as np
>>
>> In [2]: data = np.arange(1e6)
>>
>> In [3]: print(data[:10])
>> [0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]
>>
>> In [4]: %timeit np.any(data)
>> 724 us +- 42.4 us per loop (mean +- std. dev. of 7 runs, 1000 loops each)
>>
>> In [5]: data = np.zeros(int(1e6))
>>
>> In [6]: %timeit np.any(data)
>> 732 us +- 52.9 us per loop (mean +- std. dev. of 7 runs, 1000 loops each)
>>
>> I don't see any discussions about this on the NumPy issue tracker but
>> perhaps I'm missing something.
>>
>> I'm curious if there's a way to get a fast early-terminating search in
>> NumPy? Perhaps there's another package I can depend on that does this? I
>> guess I could also write a bit of cython code that does this but so far
>> this project is pure python and I don't want to deal with the packaging
>> headache of getting wheels built and conda-forge packages set up on all
>> platforms.
>>
>> Thanks for your help!
>>
>> -Nathan
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180426/fdedb116/attachment-0001.html>