[Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs
Hameer Abbasi
einstein.edison at gmail.com
Thu May 31 09:58:44 EDT 2018
While "on average" short-circuiting only gives a speedup of 2x, in many
situations you can arrange your algorithm so short circuiting will
happen early, eg usually in the first 10 elements of a 10^6 element
array, giving enormous speedups.
Also, I do not imagine these as free-floating ufuncs, I think we can
arrange them in a logical way in a gufunc ecosystem. There would be some
"core ufuncs", with "associated gufuncs" accessible as attributes. For
instance, any_less_than will be accessible as less.any
binary "comparison" ufuncs would have attributes
less.any
less.all
less.first # returns first matching index
less.count # counts matches without intermediate bool array
This adds on to the existing attributes, for instance
ufuncs already have:
add.reduce
add.accumulate
add.reduceat
add.outer
add.at
It is unfortunate that all ufuncs currently have these attributes even
if they are unimplemented/inappropriate (eg, np.sin.reduce), I would
like to remove the inappropriate ones, so each core ufunc will only
have the appropriate attribute "associated gufuncs".
I’m definitely in favour of all this. It’d be great to have this, and it’d
be an excellent ecosystem. I’ll add that composing ufuncs is something I’ve
wanted, and that has come up from time to time.
Incidentally, once we make reduce/accumuate/... into "associated
gufuncs", I propose completely removing the "method" argument of
__array_ufunc__, since it is no longer needed and adds a lot
of complexity which implementors of an __array_ufunc__ are forced to
account for.
While removing ‘method’ is okay in my book, there should at least be a way
to detect if something is e.g., a reduction, or an element-wise ufunc (this
one is obvious, all shapes involved will be ()). We, for example, use this
in pydata/sparse. As you can imagine, for sparse arrays, element-wise
operations behave a certain way and there turns out to be a general way to
do reductions that have certain properties as well. See my paper’s draft[1]
for details. I don’t mind the __array_ufunc__ api changing, but I’d like it
if there was a way to still access the information that was previously
available.
[1] https://github.com/scipy-conference/scipy_proceedings/pull/388
Regards,
Hameer Abbasi
Sent from Astro <https://www.helloastro.com> for Mac
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180531/efada236/attachment.html>
More information about the NumPy-Discussion
mailing list