ndrange, like range but multidimensiontal
Hi All, I've been using numpy array objects to store collections of 2D (and soon ND) variables. When iterating through these collections, I often found it useful to use `ndindex`, which for `for loops` behaves much like `range` with only a `stop` parameter. That said, it lacks a few features that are now present in `range` are missing from `ndindex`, most notably the ability to iterate over a subset of the ndindex. I found myself often writing `itertools.product(range(1, data.shapep[0]), range(3, data.shape[2]))` for custom iterations. While it does flatten out the for loop, it is arguable less readable than having 1 or 2 levels of nested for loops. It is quite possible that `nditer` would solve my problems, but unfortunately I am still not able to make sense of then numerous options it has. I propose an `ndrange` class that can be used to iterate over nd-collections mimicking the API of `range` as much as possible and adapting it to the ND case (i.e. returning tuples instead of singletons). Since this is an enhancement proposal, I am bringing the idea to the mailing list for reactions. The implementation in this PR https://github.com/numpy/numpy/pull/12094 is based on keeping track of a tuple of python `range` range objects. The `__iter__` method returns the result of `itertools.product(*self._ranges)` By leveraging python's `range` implementation, operations like `containement` `index`, `reversed`, `equality` and most importantly slicing of the ndrange object are possible to offer to the general numpy audiance. For example, iterating through a 2D collection but avoiding indexing the first and last column used to look like this: ``` c = np.empty((4, 4), dtype=object) # ... compute on c for j in range(c.shape[0]): for i in range(1, c.shape[1]-1): c[j, i] # = compute on c[j, i] that depends on the index i, j ``` With `np.ndrange` it can look something like this: ``` c = np.empty((4, 4), dtype=object) # ... compute on c for i in np.ndrange(c.shape)[:, 1:-1]: c[i] # = some operation on c[i] that depends on the index i ``` very pythonic, very familiar to numpy users Thank you for the feedback, Mark References: An issue requesting expansion to the ndindex API on github: https://github.com/numpy/numpy/issues/6393
On 10/07/2018 10:32 AM, Mark Harfouche wrote:
So if I understand, this does the same as `np.ndindex` but allows numpy-like slicing of the returned generator object, as requested in #6393. I don't like the duplication in functionality between ndindex and ndrange here. Better rather to add the slicing functionality to ndindex, than create a whole new nearly-identical function. np.ndindex is already a somewhat obscure and discouraged method since it is usually better to find a vectorized numpy operation instead of a for loop, and I don't like adding more obscure functions. But as an improvement to np.ndindex, I think adding this functionality seems good if it can be nicely implemented. Maybe there is a way to use the same optimization tricks as in the current implementation of ndindex but allow different stop/step? A simple wrapper of ndindex? Cheers, Allan
On 10/07/2018 10:32 AM, Mark Harfouche wrote:
So if I understand, this does the same as `np.ndindex` but allows numpy-like slicing of the returned generator object, as requested in #6393. I don't like the duplication in functionality between ndindex and ndrange here. Better rather to add the slicing functionality to ndindex, than create a whole new nearly-identical function. np.ndindex is already a somewhat obscure and discouraged method since it is usually better to find a vectorized numpy operation instead of a for loop, and I don't like adding more obscure functions. But as an improvement to np.ndindex, I think adding this functionality seems good if it can be nicely implemented. Maybe there is a way to use the same optimization tricks as in the current implementation of ndindex but allow different stop/step? A simple wrapper of ndindex? Cheers, Allan
participants (2)
-
Allan Haldane -
Mark Harfouche