NumPy has the handy np.vectorize for turning Python code that operates on scalars into a function that vectorizes works like a ufunc, but no helper function for creating generalized ufuncs ( http://docs.scipy.org/doc/numpy/reference/c-api.generalized-ufuncs.html). np.apply_along_axis accomplishes some of this, but it only allows a single core dimension on a single argument. So I propose adding a new object, np.guvectorize(pyfunc, signature, otypes, ...), where pyfunc is defined over the core dimensions only of any inputs and signature is any valid gufunc signature (a string). Calling this object would apply the gufunc. This is inspired by the similar numba.guvectorize, which is currently the easiest way to write a gufunc in Python. In addition to be handy like vectorize, such functionality would be especially useful for with working libraries that build upon NumPy to extend the capabilities of generalized ufuncs (e.g., xarray after https://github.com/pydata/xarray/pull/964). Cheers, Stephan
On Tue, Sep 13, 2016 at 11:47 AM, Stephan Hoyer <shoyer@gmail.com> wrote:
NumPy has the handy np.vectorize for turning Python code that operates on scalars into a function that vectorizes works like a ufunc, but no helper function for creating generalized ufuncs (http://docs.scipy.org/doc/ numpy/reference/c-api.generalized-ufuncs.html).
np.apply_along_axis accomplishes some of this, but it only allows a single core dimension on a single argument.
So I propose adding a new object, np.guvectorize(pyfunc, signature, otypes, ...), where pyfunc is defined over the core dimensions only of any inputs and signature is any valid gufunc signature (a string). Calling this object would apply the gufunc. This is inspired by the similar numba.guvectorize, which is currently the easiest way to write a gufunc in Python.
In addition to be handy like vectorize, such functionality would be especially useful for with working libraries that build upon NumPy to extend the capabilities of generalized ufuncs (e.g., xarray after https://github.com/pydata/xarray/pull/964).
First, this seems really cool. I hope it goes somewhere. I'm curious whether you have a plan to deal with the python functional call overhead. Numba gets around this by JIT-compiling python functions - is there something analogous you can do in NumPy or will this always be limited by the overhead of repeatedly calling a Python implementation of the "core" operation? -Nathan
Cheers, Stephan
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Sep 13, 2016 at 10:39 AM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
I'm curious whether you have a plan to deal with the python functional call overhead. Numba gets around this by JIT-compiling python functions - is there something analogous you can do in NumPy or will this always be limited by the overhead of repeatedly calling a Python implementation of the "core" operation?
I don't think there is any way to avoid this in NumPy proper, but that's OK (it's similar to the existing overhead of vectorize). Numba already has guvectorize (and it's own version of vectorize as well), which already does exactly this.
There has been some discussion on the Numba mailing list as well about a version of guvectorize that doesn't compile for testing and flexibility. Having this be inside NumPy itself seems ideal. -Travis On Tue, Sep 13, 2016 at 12:59 PM, Stephan Hoyer <shoyer@gmail.com> wrote:
On Tue, Sep 13, 2016 at 10:39 AM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
I'm curious whether you have a plan to deal with the python functional call overhead. Numba gets around this by JIT-compiling python functions - is there something analogous you can do in NumPy or will this always be limited by the overhead of repeatedly calling a Python implementation of the "core" operation?
I don't think there is any way to avoid this in NumPy proper, but that's OK (it's similar to the existing overhead of vectorize).
Numba already has guvectorize (and it's own version of vectorize as well), which already does exactly this.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
-- *Travis Oliphant, PhD* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
I have put a pull request implementing numpy.guvectorize up for review: https://github.com/numpy/numpy/pull/8054 Cheers, Stephan On Tue, Sep 13, 2016 at 10:54 PM, Travis Oliphant <travis@continuum.io> wrote:
There has been some discussion on the Numba mailing list as well about a version of guvectorize that doesn't compile for testing and flexibility.
Having this be inside NumPy itself seems ideal.
-Travis
On Tue, Sep 13, 2016 at 12:59 PM, Stephan Hoyer <shoyer@gmail.com> wrote:
On Tue, Sep 13, 2016 at 10:39 AM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
I'm curious whether you have a plan to deal with the python functional call overhead. Numba gets around this by JIT-compiling python functions - is there something analogous you can do in NumPy or will this always be limited by the overhead of repeatedly calling a Python implementation of the "core" operation?
I don't think there is any way to avoid this in NumPy proper, but that's OK (it's similar to the existing overhead of vectorize).
Numba already has guvectorize (and it's own version of vectorize as well), which already does exactly this.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
--
*Travis Oliphant, PhD* *Co-founder and CEO*
@teoliphant 512-222-5440 http://www.continuum.io
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (3)
-
Nathan Goldbaum -
Stephan Hoyer -
Travis Oliphant