First, my apologies if this isn't the right forum for this question- I looked for a dev list, but couldn't really find it. I have a small method I'd like to contribute to numpy, ideally as a method on ndarrays and a general function in the numpy namespace. I found it on a stackoverflow thread, and it is a generator that yields slices of a multidimensional array over a specified axis, which is convenient for use in list comprehensions and loops. https://gist.github.com/andrewgiessel/5400659 I've forked the numpy source and am familar with git/pull requests/etc but the code base is a bit overwhelming. I have 2 questions: 1) Is there a document which gives an overview of the numpy source and perhaps a tutorial on the best way to add methods/functions? 2) is there a current way to do this in numpy? The only iterator related stuff I found in a brief search last night was for essentially looping over all elements of an array, one by one. best, Andrew ps: please feel free to contribute to the gist! -- Andrew Giessel, PhD Department of Neurobiology, Harvard Medical School 220 Longwood Ave Boston, MA 02115 ph: 617.432.7971 email: andrew_giessel@hms.harvard.edu
On Wed, Apr 17, 2013 at 6:29 AM, andrew giessel <andrew.giessel@gmail.com>wrote:
First, my apologies if this isn't the right forum for this question- I looked for a dev list, but couldn't really find it.
I have a small method I'd like to contribute to numpy, ideally as a method on ndarrays and a general function in the numpy namespace. I found it on a stackoverflow thread, and it is a generator that yields slices of a multidimensional array over a specified axis, which is convenient for use in list comprehensions and loops.
https://gist.github.com/andrewgiessel/5400659
I've forked the numpy source and am familar with git/pull requests/etc but the code base is a bit overwhelming.
I have 2 questions:
1) Is there a document which gives an overview of the numpy source and perhaps a tutorial on the best way to add methods/functions?
The closest thing is probably Contributing to Numpy <http://docs.scipy.org/doc/numpy/dev/index.html?highlight=development>, which I suspect is not what you are looking for. At this point you pretty much need to dig through the source to see how it is organized. To see how things like tests/documentation are organized look for existing examples and also the relevant docs in doc/.
2) is there a current way to do this in numpy? The only iterator related stuff I found in a brief search last night was for essentially looping over all elements of an array, one by one.
I think a function like this would be useful. There are ad-hoc ways to get the same result but they aren't quite as flexible. A few comments 1) The function would probably best go in numpy/core/numeric.py 2) It will need a docstring 3) It will need tests in numpy/core/tests/test_numeric.py 4) xrange isn't Python 3 compatible, use range instead. The name isn't very descriptive, maybe iter_over_axis? One possible generalization would to let axis (axes) be either a number, or a list of axes. The first being the number of leading axes, the second letting one choose arbitrary axes. Chuck
Chuck- Thank you for the very helpful and encouraging email! I will first try to just add a function, rather than a method on ndarray (which looks to be lower level ie: C). The pointer to numeric.py / test_numeric.py is exactly what I needed. I will of course figure out a good test and document it well. For starters, I'll make the function simply take an integer corresponding to an axis. I'm not exactly sure what you mean by generalizing to take multiple axes -- would the idea be to return slices of an array w/o one of the dimensions? I'll try to tackle this over the next week and hopefully the conversation on the PR will be the place to talk about these issues. I'll need to figure out the best way to have a dev branch of numpy side-by-side with a stock version, and how to build the module, first. Lastly, I'll also try to write up something re: my experience so others can have something to take a look at. best+thanks, ag On Wed, Apr 17, 2013 at 11:43 AM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Wed, Apr 17, 2013 at 6:29 AM, andrew giessel <andrew.giessel@gmail.com>wrote:
First, my apologies if this isn't the right forum for this question- I looked for a dev list, but couldn't really find it.
I have a small method I'd like to contribute to numpy, ideally as a method on ndarrays and a general function in the numpy namespace. I found it on a stackoverflow thread, and it is a generator that yields slices of a multidimensional array over a specified axis, which is convenient for use in list comprehensions and loops.
https://gist.github.com/andrewgiessel/5400659
I've forked the numpy source and am familar with git/pull requests/etc but the code base is a bit overwhelming.
I have 2 questions:
1) Is there a document which gives an overview of the numpy source and perhaps a tutorial on the best way to add methods/functions?
The closest thing is probably Contributing to Numpy <http://docs.scipy.org/doc/numpy/dev/index.html?highlight=development>, which I suspect is not what you are looking for. At this point you pretty much need to dig through the source to see how it is organized. To see how things like tests/documentation are organized look for existing examples and also the relevant docs in doc/.
2) is there a current way to do this in numpy? The only iterator related stuff I found in a brief search last night was for essentially looping over all elements of an array, one by one.
I think a function like this would be useful. There are ad-hoc ways to get the same result but they aren't quite as flexible. A few comments
1) The function would probably best go in numpy/core/numeric.py 2) It will need a docstring 3) It will need tests in numpy/core/tests/test_numeric.py 4) xrange isn't Python 3 compatible, use range instead.
The name isn't very descriptive, maybe iter_over_axis? One possible generalization would to let axis (axes) be either a number, or a list of axes. The first being the number of leading axes, the second letting one choose arbitrary axes.
Chuck
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Andrew Giessel, PhD Department of Neurobiology, Harvard Medical School 220 Longwood Ave Boston, MA 02115 ph: 617.432.7971 email: andrew_giessel@hms.harvard.edu
An update--- I submitted a PR if anyone is interested: https://github.com/numpy/numpy/pull/3262 Secondly, it was pointed out to my by Stefan van der Walt that one could use np.rollaxis() to reorder an array such that the default iterator behavior would yield the same slices of the array: a = np.ones((100,10,3)) for i in np.rollaxis(a, 2): print i.shape gives you 3 100x10 slices. My PR is more of a pure iterator, so I sent it in all the best, ag On Wed, Apr 17, 2013 at 2:18 PM, Andrew Giessel < andrew_giessel@hms.harvard.edu> wrote:
Chuck-
Thank you for the very helpful and encouraging email! I will first try to just add a function, rather than a method on ndarray (which looks to be lower level ie: C). The pointer to numeric.py / test_numeric.py is exactly what I needed.
I will of course figure out a good test and document it well. For starters, I'll make the function simply take an integer corresponding to an axis. I'm not exactly sure what you mean by generalizing to take multiple axes -- would the idea be to return slices of an array w/o one of the dimensions?
I'll try to tackle this over the next week and hopefully the conversation on the PR will be the place to talk about these issues. I'll need to figure out the best way to have a dev branch of numpy side-by-side with a stock version, and how to build the module, first.
Lastly, I'll also try to write up something re: my experience so others can have something to take a look at.
best+thanks,
ag
On Wed, Apr 17, 2013 at 11:43 AM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Wed, Apr 17, 2013 at 6:29 AM, andrew giessel <andrew.giessel@gmail.com
wrote:
First, my apologies if this isn't the right forum for this question- I looked for a dev list, but couldn't really find it.
I have a small method I'd like to contribute to numpy, ideally as a method on ndarrays and a general function in the numpy namespace. I found it on a stackoverflow thread, and it is a generator that yields slices of a multidimensional array over a specified axis, which is convenient for use in list comprehensions and loops.
https://gist.github.com/andrewgiessel/5400659
I've forked the numpy source and am familar with git/pull requests/etc but the code base is a bit overwhelming.
I have 2 questions:
1) Is there a document which gives an overview of the numpy source and perhaps a tutorial on the best way to add methods/functions?
The closest thing is probably Contributing to Numpy <http://docs.scipy.org/doc/numpy/dev/index.html?highlight=development>, which I suspect is not what you are looking for. At this point you pretty much need to dig through the source to see how it is organized. To see how things like tests/documentation are organized look for existing examples and also the relevant docs in doc/.
2) is there a current way to do this in numpy? The only iterator related stuff I found in a brief search last night was for essentially looping over all elements of an array, one by one.
I think a function like this would be useful. There are ad-hoc ways to get the same result but they aren't quite as flexible. A few comments
1) The function would probably best go in numpy/core/numeric.py 2) It will need a docstring 3) It will need tests in numpy/core/tests/test_numeric.py 4) xrange isn't Python 3 compatible, use range instead.
The name isn't very descriptive, maybe iter_over_axis? One possible generalization would to let axis (axes) be either a number, or a list of axes. The first being the number of leading axes, the second letting one choose arbitrary axes.
Chuck
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Andrew Giessel, PhD
Department of Neurobiology, Harvard Medical School 220 Longwood Ave Boston, MA 02115 ph: 617.432.7971 email: andrew_giessel@hms.harvard.edu
-- Andrew Giessel, PhD Department of Neurobiology, Harvard Medical School 220 Longwood Ave Boston, MA 02115 ph: 617.432.7971 email: andrew_giessel@hms.harvard.edu
participants (3)
-
andrew giessel -
Andrew Giessel -
Charles R Harris