Yesterday I created a GitHub issue proposing adding an axis argument to numpy's gufuncs: https://github.com/numpy/numpy/issues/5197 I was told I should repost this on the mailing list, so here's the recap: I would like to write generalized ufuncs (probably using numba), to create fast functions such as nanmean (signature '(n)->()') or rolling_mean (signature '(n),()->(n)') that take the axis along which to aggregate as a keyword argument, e.g., nanmean(x, axis=0) or rolling_mean(x, window=5, axis=0). Of course, I could write my own wrapper for this that reorders dimensions using swapaxes or transpose. But I also think that an "axis" argument to allow for specifying the core dimensions of gufuncs would be more generally useful, and we should consider adding it to numpy. Nathaniel and Jaime added some good points, noting that such an axis argument should cleanly handle multiple input and output arguments and have a plan for handling optional dimensions (e.g., (m?,n),(n,p?)->(m?,p?) for the new dot). Here are my initial thoughts on the syntax: (1) Generally speaking, I think the "nested tuple" syntax (e.g., axis=[(0, 1), (2, 3)]) would be most congruous with the axis arguments numpy already supports. (2) For gufuncs with simpler signatures, we should support supplying an integer or an unnested tuple, e.g., - axis=0 for (n)->() - axis=(0, 1) for (n)(m)->() or (n,m)->() - axis=[(0, 1), 2] for (n,m),(o)->(). (3) If we require a full axis specification for core dimensions, we could use the axis argument for unambiguous control of optional core dimensions: e.g., axis=(0, 1) would indicate that you want the "vectorized inner product" version of the new dot operator, rather than matrix multiplication, and axis=[(-2, -1), -1] would mean that you want the "vectorized matrix-vector product". This seems relatively tidy, although I admit I am not convinced that optional core dimensions are necessary. (4) We can either include the output axis as part of the signature, or add another argument "axis_out" or "out_axis". I think prefer the separate argument, particularly if we require "axis" to specify all core dimensions, which may be a good idea even if we don't use "axis" for controlling optional core dimensions. Cheers, Stephan
On Fri, Oct 17, 2014 at 10:56 PM, Stephan Hoyer <shoyer@gmail.com> wrote:
Yesterday I created a GitHub issue proposing adding an axis argument to numpy's gufuncs: https://github.com/numpy/numpy/issues/5197
I was told I should repost this on the mailing list, so here's the recap:
I would like to write generalized ufuncs (probably using numba), to create fast functions such as nanmean (signature '(n)->()') or rolling_mean (signature '(n),()->(n)') that take the axis along which to aggregate as a keyword argument, e.g., nanmean(x, axis=0) or rolling_mean(x, window=5, axis=0).
Of course, I could write my own wrapper for this that reorders dimensions using swapaxes or transpose. But I also think that an "axis" argument to allow for specifying the core dimensions of gufuncs would be more generally useful, and we should consider adding it to numpy.
Nathaniel and Jaime added some good points, noting that such an axis argument should cleanly handle multiple input and output arguments and have a plan for handling optional dimensions (e.g., (m?,n),(n,p?)->(m?,p?) for the new dot).
Here are my initial thoughts on the syntax:
(1) Generally speaking, I think the "nested tuple" syntax (e.g., axis=[(0, 1), (2, 3)]) would be most congruous with the axis arguments numpy already supports.
(2) For gufuncs with simpler signatures, we should support supplying an integer or an unnested tuple, e.g., - axis=0 for (n)->() - axis=(0, 1) for (n)(m)->() or (n,m)->() - axis=[(0, 1), 2] for (n,m),(o)->().
(3) If we require a full axis specification for core dimensions, we could use the axis argument for unambiguous control of optional core dimensions: e.g., axis=(0, 1) would indicate that you want the "vectorized inner product" version of the new dot operator, rather than matrix multiplication, and axis=[(-2, -1), -1] would mean that you want the "vectorized matrix-vector product". This seems relatively tidy, although I admit I am not convinced that optional core dimensions are necessary.
(4) We can either include the output axis as part of the signature, or add another argument "axis_out" or "out_axis". I think prefer the separate argument, particularly if we require "axis" to specify all core dimensions, which may be a good idea even if we don't use "axis" for controlling optional core dimensions.
Might want to contact continuum analytics also. They recently created a gufunc <https://github.com/ContinuumIO/blaze> repository. Chuck
On Sat, Oct 18, 2014 at 5:56 AM, Stephan Hoyer <shoyer@gmail.com> wrote:
Here are my initial thoughts on the syntax:
(1) Generally speaking, I think the "nested tuple" syntax (e.g., axis=[(0, 1), (2, 3)]) would be most congruous with the axis arguments numpy already supports.
(2) For gufuncs with simpler signatures, we should support supplying an integer or an unnested tuple, e.g., - axis=0 for (n)->() - axis=(0, 1) for (n)(m)->() or (n,m)->() - axis=[(0, 1), 2] for (n,m),(o)->().
One thing we'll have to watch out for is that for reduction operations (which are basically gufuncs with (n)->() signatures), we already allow axis=(0,1) to mean "reshape axes 0 and 1 together into one big axis, and then use that as the gufunc core axis". I don't know if we'll ever want to support this functionality for gufuncs in general, but we shouldn't rule it out with the syntax. One option would be to add a new argument axes=... for gufunc core specification, and say that axis=foo is an alias for axes=[[foo]]. -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org
On Sat, Oct 18, 2014 at 6:46 PM, Nathaniel Smith <njs@pobox.com> wrote:
One thing we'll have to watch out for is that for reduction operations (which are basically gufuncs with (n)->() signatures), we already allow axis=(0,1) to mean "reshape axes 0 and 1 together into one big axis, and then use that as the gufunc core axis". I don't know if we'll ever want to support this functionality for gufuncs in general, but we shouldn't rule it out with the syntax.
This is a great point. In fact, I think supporting this sort of functionality for gufuncs would be quite valuable, since there are a plenty of reduction operations that can't fit into the model provided by ufunc.reduce. An excellent example is np.median, which currently can only act on either one axis or an entire flattened array. If the syntax (m?,n),(n,p?)->(m?,p?) is accepted, then I think the natural extension to reduction operators that can act on one or more axes would be (n+)->() (this is regex syntax). Actually, adding using an axis keyword seems like the only elegant way to handle disambiguating cases like this.
One option would be to add a new argument axes=... for gufunc core specification, and say that axis=foo is an alias for axes=[[foo]].
Indeed, this is exactly what I was thinking. The "canonical form" for the axis argument would be doubly nested tuples, but if an integer or unnested tuple is encountered, additional nesting should be added until reaching canoncial form, e.g., axis=0 -> axis=(0,) -> axis=((0,),). The only particularly tricky case will be scenarios like my second one, axis=(0, 1) for (n)(m)->() or (n,m)->(). To deal with cases like this, the parsing will need to take the gufunc signature into consideration, and start by asking whether or not tuple is of the right size to match each function argument separately. To make it clear that this proposal covers all the bases, I would be happy to write some prototype code (and test cases) to demonstrate such a transformation to canonical form, including all these edge cases. Cheers, Stephan
On Sun, Oct 19, 2014 at 8:25 AM, Stephan Hoyer <shoyer@gmail.com> wrote:
On Sat, Oct 18, 2014 at 6:46 PM, Nathaniel Smith <njs@pobox.com> wrote:
One thing we'll have to watch out for is that for reduction operations (which are basically gufuncs with (n)->() signatures), we already allow axis=(0,1) to mean "reshape axes 0 and 1 together into one big axis, and then use that as the gufunc core axis". I don't know if we'll ever want to support this functionality for gufuncs in general, but we shouldn't rule it out with the syntax.
This is a great point.
In fact, I think supporting this sort of functionality for gufuncs would be quite valuable, since there are a plenty of reduction operations that can't fit into the model provided by ufunc.reduce. An excellent example is np.median, which currently can only act on either one axis or an entire flattened array.
If the syntax (m?,n),(n,p?)->(m?,p?) is accepted, then I think the natural extension to reduction operators that can act on one or more axes would be (n+)->() (this is regex syntax).
It's not clear we even need to alter the signature here -- the reduction operations don't bother distinguishing between reductions that make sense in this case (the commutative ones) and the ones that don't (everything else), they just trust that no-one will try doing something like np.subtract.reduce(arr, axis=(0, 1)) because it's meaningless. Providing some basic checks here might be useful though given that gufunc signatures can be much more complicated than just (n)->().
Actually, adding using an axis keyword seems like the only elegant way to handle disambiguating cases like this.
One option would be to add a new argument axes=... for gufunc core specification, and say that axis=foo is an alias for axes=[[foo]].
Indeed, this is exactly what I was thinking. The "canonical form" for the axis argument would be doubly nested tuples, but if an integer or unnested tuple is encountered, additional nesting should be added until reaching canoncial form, e.g., axis=0 -> axis=(0,) -> axis=((0,),).
The only particularly tricky case will be scenarios like my second one, axis=(0, 1) for (n)(m)->() or (n,m)->(). To deal with cases like this, the parsing will need to take the gufunc signature into consideration, and start by asking whether or not tuple is of the right size to match each function argument separately.
Right, the problem with (0, 1) in this system is that you can either read it as being a single reshaping axis description and expand it to ((0, 1),), or you can read it as being two non-reshaping axis descriptions and expand it to ((0,), (1,)). I feel strongly that we should come up with a syntax that is unambiguous even *without* looking at the gufunc signature. It's easy for the computer to disambiguate stuff like this, but it'd be cruel to ask people trying to skim through code to work out the signature and then simulate the disambiguation algorithm in their head. Notice in my suggestion above there are two different kwargs, "axis" and "axes". -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org
On Sun, Oct 19, 2014 at 6:43 AM, Nathaniel Smith <njs@pobox.com> wrote:
I feel strongly that we should come up with a syntax that is
unambiguous even *without* looking at the gufunc signature. It's easy
for the computer to disambiguate stuff like this, but it'd be cruel to ask people trying to skim through code to work out the signature and then simulate the disambiguation algorithm in their head.
Since code speaks stronger than mere words, here is a notebook showing my disambiguation algorithm: http://nbviewer.ipython.org/gist/shoyer/7740d32850084261d870 I don't think this is so cruel, but I agree that the logic is more complex than ideal: "If the axis argument is a sequence with length equal to the number of variables with axis specifications in the gufunc signature, then each element is taken to specify the axis for each corresponding variable. Otherwise, if the gufunc has only one variable with a core dimension, the entire axis argument is taken to refer to only that variable."
Notice in my suggestion above there are two different kwargs, "axis" and "axes".
Ah, I missed that. That's actually pretty elegant, so +1 from me. My only ask then would be that we allow for "axis" to also be a sequence of integers, in which case they are also used to specify the axis for the single variable, e.g., axis=(1, 2) translates to axes=[(1, 2)]. This would allow for using the axis argument in the same way as it works on ufunc.reduce already. I don't think distinguishing cases for "integer" vs "tuple of integers" is too complex.
participants (3)
-
Charles R Harris -
Nathaniel Smith -
Stephan Hoyer