[Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

Robert Kern robert.kern at gmail.com
Wed Sep 4 06:21:09 EDT 2013


On Wed, Sep 4, 2013 at 11:05 AM, Graeme B. Bell <grb at skogoglandskap.no>
wrote:
>
> In my current GIS raster work I often have a situation where I generate
code something like this:
>
>          np.any([A>4, A==2, B==5, ...])
>
> However, np.any() is quite slow.
>
> It's possible to use np.logical_or to solve the problem, but then you get
nested logical_or's, since logical_or combines only two parameters.
> It's also possible to use integer maths e.g. (A>4)+(A==2)+(B==5)>0.
>
> The question is: which is best (syntactically, in terms of performance,
etc)?
>
> I've written a little helper function to provide a faster version of
any() and all(). It's embarrassingly simple - just a for loop. However, I
think there's a syntactic advantage to using a helper function for this
situation rather than writing it idiomatically each time; and it reduces
the chance of a bug in idiomatic implementation. However, the code does not
cover all the use cases currently addressed by np.any() and np.all().
>
> I benchmarked to pick the fastest underlying implementation (logical_or
rather than integer maths).
>
> The result is 14 to 17x faster than np.any() for this use case.*
>
> Code & benchmark here:
>
>       https://github.com/gbb/numpy-fast-any-all
>
> Please feel welcome to use it or improve it :-)

Try the following:

  any(map(np.any, inputs))
  all(map(np.all, inputs))

--
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130904/ff9dc635/attachment.html>


More information about the NumPy-Discussion mailing list