Add a function to broadcast arrays to a given shape to numpy's stride_tricks?
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
I recently wrote function to manually broadcast an ndarray to a given shape according to numpy's broadcasting rules (using strides): https://github.com/xray/xray/commit/7aee4a3ed2dfd3b9aff7f3c5c6c68d51df2e3ff3 The same functionality can be done pretty straightforwardly with np.broadcast_arrays, but that function does both too much (I don't actually have a second array that needs to be broadcast) and not enough (I need to create a dummy array to broadcast against it). This approach is simpler, and also, according to my benchmarks, about 3x faster than np.broadcast_arrays: In [1]: import xray In [2]: import numpy as np In [3]: x = np.random.randn(4) In [4]: y = np.empty((2, 3, 4)) In [5]: %timeit xray.core.utils.as_shape(x, y.shape) 100000 loops, best of 3: 17 µs per loop In [6]: %timeit np.broadcast_arrays(x, y)[0] 10000 loops, best of 3: 47.4 µs per loop Would this be a welcome addition to numpy's lib.stride_tricks? If so, I will put together a PR. In my search, I turned up a Stack Overflow post looking for similar functionality: https://stackoverflow.com/questions/11622692/is-there-a-better-way-to-broadc... Cheers, Stephan
![](https://secure.gravatar.com/avatar/f14373168edcc55c5f2598a40de55c0d.jpg?s=120&d=mm&r=g)
Hi, Le 07/12/2014 08:10, Stephan Hoyer a écrit :
Instead of putting this function in stride_tricks (which is quite hidden), could it be added instead as a boolean flag to the existing `reshape` method ? Something like: x.reshape(y.shape, broadcast=True) What other people think ? best, Pierre
![](https://secure.gravatar.com/avatar/09939f25b639512a537ce2c90f77f958.jpg?s=120&d=mm&r=g)
I like the idea of the broadcast argument to reshape. It certainly makes sense there, and it avoids adding a new function. Probably should add a note to the docstring of broadcast_arrays, too. Ben Root On Mon, Dec 8, 2014 at 2:31 AM, Pierre Haessig <pierre.haessig@crans.org> wrote:
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Sun, Dec 7, 2014 at 11:31 PM, Pierre Haessig <pierre.haessig@crans.org> wrote:
I agree that it would be nice to expose this more directly, but I see two (small) downsides to putting this in reshape: 1. This would one of those flags that changes a method to an entirely different mode -- there's not much the way of shared logic with reshape. 2. reshape is written in C (like all ndarray methods, I believe), so implementing this there will be a little trickier than adding a new function. Cheers, Stephan
![](https://secure.gravatar.com/avatar/b4929294417e9ac44c17967baae75a36.jpg?s=120&d=mm&r=g)
Hi, On Monday, December 8, 2014, Pierre Haessig <pierre.haessig@crans.org> wrote:
That might be a bit odd, because the non-broadcast version would allow entirely different parameters for shape than the broadcast version. For example, what would these do? a = np.zeros((2, 3, 4)) a.reshape((6, 4), broadcast=True) a.reshape((2, -1), broadcast=True) So I think 'reshape' is doing something different enough that this should be a separate function. Cheers, Matthew
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On Sun, Dec 7, 2014 at 7:10 AM, Stephan Hoyer <shoyer@gmail.com> wrote:
Seems like a useful addition to me -- I've definitely wanted this in the past. I agree with Stephan that reshape() might not be the best place, though; I wouldn't think to look for it there. Two API ideas, which are not mutually exclusive: 1) Give broadcast_arrays an extra_shapes=[shape1, shape2, ...] argument. Each entry in that list is a tuple of integers; broadcast_arrays chooses the final output shape "as if" additional arrays with the given shapes had been passed in, in addition to the ones in *args. 2) Add a broadcast_to(arr, shape) function, which broadcasts the array to exactly the shape given, or else errors out if this is not possible. Given (1), (2) could just be: def broadcast_to(arr, shape): output = broadcast_arrays(arr, extra_shapes=[shape]) if output.shape != shape: raise ... return output -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Wed, Dec 10, 2014 at 4:00 PM, Nathaniel Smith <njs@pobox.com> wrote:
I like np.broadcast_to as a new function. We can document it alongside broadcast and broadcast_arrays under array manipulation routines, which would make it at least as discoverable as the standard broadcasting functions. I'm not opposed to adding extra_shapes as a keyword argument to broadcast_arrays, but it seems unnecessarily complex, Implementation wise, I think it would actual make more sense to make broadcast_arrays depend on broadcast_to (e.g., by composing a function to calculate the broadcast shape with broadcast_to). Stephan
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On 11 Dec 2014 14:31, "Pierre Haessig" <pierre.haessig@crans.org> wrote:
It could, but then there wouldn't be much to distinguish it from broadcast_arrays. Broadcasting is generally a symmetric operation - see broadcast_arrays or arr1 + arr2. So the 'to' is there to give a clue that this function is not symmetric, and rather has a specific goal in mind. -n
![](https://secure.gravatar.com/avatar/764323a14e554c97ab74177e0bce51d4.jpg?s=120&d=mm&r=g)
On Thu, Dec 11, 2014 at 2:47 PM, Nathaniel Smith <njs@pobox.com> wrote:
docstring. this function is not symmetric, and rather has a specific goal in mind. And we already have a numpy.broadcast() function. http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast.html -- Robert Kern
![](https://secure.gravatar.com/avatar/f14373168edcc55c5f2598a40de55c0d.jpg?s=120&d=mm&r=g)
Le 11/12/2014 16:52, Robert Kern a écrit :
And we already have a numpy.broadcast() function.
http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast.html
True, I once read the docstring of this function. but never used it though. Pierre
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
On Do, 2014-12-11 at 16:56 +0100, Pierre Haessig wrote:
I am not sure it is really the right thing for most things since it returns an old style iterator. On the other hand arrays with 0-strides need a bit more care (if we add this top level, one might have copy=True as a default or so?). Also because of that it is currently limited to NPY_MAXARGS (32). Personally, I would like to see this type of functionality implemented in C, and may be willing to help with it. This kind of code exists in enough places in numpy so it can be stolen pretty readily. One option would also be to have something like: np.common_shape(*arrays) np.broadcast_to(array, shape) # (though I would like many arrays too) and then broadcast_ar rays could be implemented in terms of these two. Just some thoughts, Sebastian
![](https://secure.gravatar.com/avatar/764323a14e554c97ab74177e0bce51d4.jpg?s=120&d=mm&r=g)
On Thu, Dec 11, 2014 at 4:17 PM, Sebastian Berg <sebastian@sipsolutions.net> wrote:
True, I once read the docstring of this function. but never used it
http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast.html though.
I am not sure it is really the right thing for most things since it returns an old style iterator.
Indeed. That's why I wrote broadcast_arrays().
Why? What benefit does broadcast_arrays() get from being reimplemented in C? -- Robert Kern
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
On Do, 2014-12-11 at 16:20 +0000, Robert Kern wrote:
To be honest, maybe it is not. I remember that I had some function where broadcast_arrays was the largest part of the runtime for smaller arrays and I thought it should be easy since such code exists elsewhere. - Sebastian
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Thu, Dec 11, 2014 at 8:17 AM, Sebastian Berg <sebastian@sipsolutions.net> wrote:
It looks like np.broadcast let's us write the common_shape function very easily; def common_shape(*args): return np.broadcast(*args).shape And it's also very fast: 1000000 loops, best of 3: 1.04 µs per loop So that does seem like a feasible refactor/simplification for np.broadcast_arrays. Sebastian -- if you're up for writing np.broadcast_to in C, that's great! If you're not sure if you'll be able to get around to that in the near future, I'll submit my PR with a Python implementation (which will have tests that will be useful in any case).
![](https://secure.gravatar.com/avatar/dce2259ff9b547103d54acf1ea622314.jpg?s=120&d=mm&r=g)
On Thu, Dec 11, 2014 at 10:53 AM, Stephan Hoyer <shoyer@gmail.com> wrote:
np.broadcast is the Python object of the old iterator. It may be a better idea to write all of these functions using the new one, np.nditer: def common_shape(*args): return np.nditer(args).shape[::-1] # Yes, you do need to reverse it! And in writing 'broadcast_to', rather than rewriting the broadcasting logic, you could check the compatibility of the shape with something like: np.nditer((arr,), itershape=shape) # will raise ValueError if shapes incompatible After that, all that would be left is some prepending of zero strides, and some zeroing of strides of shape 1 dimensions before calling as_strided Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
On Fr, 2014-12-12 at 05:48 -0800, Jaime Fernández del Río wrote:
Hahaha, right there is the 32 limitation, but you can also (ab)use it: np.nditer(np.arange(10), itershape=(5, 10)).itviews[0] - Sebastian
![](https://secure.gravatar.com/avatar/dce2259ff9b547103d54acf1ea622314.jpg?s=120&d=mm&r=g)
On Fri, Dec 12, 2014 at 5:57 AM, Sebastian Berg <sebastian@sipsolutions.net> wrote:
That's neat! But itviews is not even listed in the attributes of nditer in the docs, we should fix that. Is the 32 argument limitation really a concern? Because that aside, it seems that all the functionality that has been discussed are one-liners using nditer: do we need new functions, or better documentation? Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
On Fr, 2014-12-12 at 06:25 -0800, Jaime Fernández del Río wrote:
Maybe we could say it isn't a large concern, more something you can fix later on if we find it is, but you would have to check the types, I think that subclasses are probably lost here.
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Fri, Dec 12, 2014 at 6:25 AM, Jaime Fernández del Río < jaime.frio@gmail.com> wrote:
it seems that all the functionality that has been discussed are one-liners using nditer: do we need new functions, or better documentation?
I think there is utility to adding a new function or two (my inclination is to expose broadcast_to in the public API, but leave common_shape in strick_tricks). NumPy provides all the cools to write these in a few lines, but you need to know some very deep details of the NumPy API (nditer and strides). I don't think more documentation would make this obvious -- certainly nditer does not need a longer docstring! The best sort of documentation would be more examples. If this is a recipe that many NumPy users would use, including it in stride_tricks would also serve such an educational purpose (reading stride_tricks is how I figured out how strides work).
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Fri, Dec 12, 2014 at 5:48 AM, Jaime Fernández del Río < jaime.frio@gmail.com> wrote:
Unfortunately, that version does not seem to do what I'm looking for: def common_shape(*args): return np.nditer(args).shape[::-1] x = np.empty((4,)) y = np.empty((2, 3, 4)) print(common_shape(x, y)) Outputs: (6, 4) And in writing 'broadcast_to', rather than rewriting the broadcasting
Yes, that is a good idea. Here is a gist with the latest version of this code (shortly to be turned into a PR): https://gist.github.com/shoyer/3e36af0a8196c82d4b42
![](https://secure.gravatar.com/avatar/dce2259ff9b547103d54acf1ea622314.jpg?s=120&d=mm&r=g)
On Fri, Dec 12, 2014 at 11:28 AM, Stephan Hoyer <shoyer@gmail.com> wrote:
Yes, the iterator is a smart beast. I think this is what you need then, with no reversing involved:
np.nditer((x,y), flags=['multi_index']).shape (2, 3, 4)
-- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On 12 Dec 2014 19:29, "Stephan Hoyer" <shoyer@gmail.com> wrote:
def common_shape(*args):
Nitpick: let's call this broadcast_shape, not common_shape; it's as-or-more clear and clearly groups the related functions together. -n
![](https://secure.gravatar.com/avatar/b4929294417e9ac44c17967baae75a36.jpg?s=120&d=mm&r=g)
On Sunday, December 7, 2014, Stephan Hoyer <shoyer@gmail.com> wrote:
That would be excellent - I ran into exactly the same problem, with the same conclusions, but I was lazier than you were and I did write a routine for making a dummy array in order to use broadcast_arrays: https://github.com/nipy/nibabel/blob/master/nibabel/fileslice.py#L722 https://github.com/nipy/nibabel/blob/master/nibabel/parrec.py#L577 Having a function to do this would be much clearer, thanks for doing that. Cheers, Matthew
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
Here is an update on a new function for broadcasting arrays to a given shape (now named np.broadcast_to). I have a pull request up for review, which has received some feedback now: https://github.com/numpy/numpy/pull/5371 There is still at least one design decision to settle: should we expose "broadcast_shape" in the public API? In the current implementation, it is exposed as a public function in numpy.lib.tride_tricks (like as_strided), but it is not exported into the main numpy namespace. The alternatives would be to either make it a private function (_broadcast_shape) or expose it publicly (np.broadcast_shape). Please do speak if you have any thoughts to share on the implementation, either here or in the pull request. Best, Stephan
![](https://secure.gravatar.com/avatar/f14373168edcc55c5f2598a40de55c0d.jpg?s=120&d=mm&r=g)
Hi, Le 07/12/2014 08:10, Stephan Hoyer a écrit :
Instead of putting this function in stride_tricks (which is quite hidden), could it be added instead as a boolean flag to the existing `reshape` method ? Something like: x.reshape(y.shape, broadcast=True) What other people think ? best, Pierre
![](https://secure.gravatar.com/avatar/09939f25b639512a537ce2c90f77f958.jpg?s=120&d=mm&r=g)
I like the idea of the broadcast argument to reshape. It certainly makes sense there, and it avoids adding a new function. Probably should add a note to the docstring of broadcast_arrays, too. Ben Root On Mon, Dec 8, 2014 at 2:31 AM, Pierre Haessig <pierre.haessig@crans.org> wrote:
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Sun, Dec 7, 2014 at 11:31 PM, Pierre Haessig <pierre.haessig@crans.org> wrote:
I agree that it would be nice to expose this more directly, but I see two (small) downsides to putting this in reshape: 1. This would one of those flags that changes a method to an entirely different mode -- there's not much the way of shared logic with reshape. 2. reshape is written in C (like all ndarray methods, I believe), so implementing this there will be a little trickier than adding a new function. Cheers, Stephan
![](https://secure.gravatar.com/avatar/b4929294417e9ac44c17967baae75a36.jpg?s=120&d=mm&r=g)
Hi, On Monday, December 8, 2014, Pierre Haessig <pierre.haessig@crans.org> wrote:
That might be a bit odd, because the non-broadcast version would allow entirely different parameters for shape than the broadcast version. For example, what would these do? a = np.zeros((2, 3, 4)) a.reshape((6, 4), broadcast=True) a.reshape((2, -1), broadcast=True) So I think 'reshape' is doing something different enough that this should be a separate function. Cheers, Matthew
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On Sun, Dec 7, 2014 at 7:10 AM, Stephan Hoyer <shoyer@gmail.com> wrote:
Seems like a useful addition to me -- I've definitely wanted this in the past. I agree with Stephan that reshape() might not be the best place, though; I wouldn't think to look for it there. Two API ideas, which are not mutually exclusive: 1) Give broadcast_arrays an extra_shapes=[shape1, shape2, ...] argument. Each entry in that list is a tuple of integers; broadcast_arrays chooses the final output shape "as if" additional arrays with the given shapes had been passed in, in addition to the ones in *args. 2) Add a broadcast_to(arr, shape) function, which broadcasts the array to exactly the shape given, or else errors out if this is not possible. Given (1), (2) could just be: def broadcast_to(arr, shape): output = broadcast_arrays(arr, extra_shapes=[shape]) if output.shape != shape: raise ... return output -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Wed, Dec 10, 2014 at 4:00 PM, Nathaniel Smith <njs@pobox.com> wrote:
I like np.broadcast_to as a new function. We can document it alongside broadcast and broadcast_arrays under array manipulation routines, which would make it at least as discoverable as the standard broadcasting functions. I'm not opposed to adding extra_shapes as a keyword argument to broadcast_arrays, but it seems unnecessarily complex, Implementation wise, I think it would actual make more sense to make broadcast_arrays depend on broadcast_to (e.g., by composing a function to calculate the broadcast shape with broadcast_to). Stephan
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On 11 Dec 2014 14:31, "Pierre Haessig" <pierre.haessig@crans.org> wrote:
It could, but then there wouldn't be much to distinguish it from broadcast_arrays. Broadcasting is generally a symmetric operation - see broadcast_arrays or arr1 + arr2. So the 'to' is there to give a clue that this function is not symmetric, and rather has a specific goal in mind. -n
![](https://secure.gravatar.com/avatar/764323a14e554c97ab74177e0bce51d4.jpg?s=120&d=mm&r=g)
On Thu, Dec 11, 2014 at 2:47 PM, Nathaniel Smith <njs@pobox.com> wrote:
docstring. this function is not symmetric, and rather has a specific goal in mind. And we already have a numpy.broadcast() function. http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast.html -- Robert Kern
![](https://secure.gravatar.com/avatar/f14373168edcc55c5f2598a40de55c0d.jpg?s=120&d=mm&r=g)
Le 11/12/2014 16:52, Robert Kern a écrit :
And we already have a numpy.broadcast() function.
http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast.html
True, I once read the docstring of this function. but never used it though. Pierre
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
On Do, 2014-12-11 at 16:56 +0100, Pierre Haessig wrote:
I am not sure it is really the right thing for most things since it returns an old style iterator. On the other hand arrays with 0-strides need a bit more care (if we add this top level, one might have copy=True as a default or so?). Also because of that it is currently limited to NPY_MAXARGS (32). Personally, I would like to see this type of functionality implemented in C, and may be willing to help with it. This kind of code exists in enough places in numpy so it can be stolen pretty readily. One option would also be to have something like: np.common_shape(*arrays) np.broadcast_to(array, shape) # (though I would like many arrays too) and then broadcast_ar rays could be implemented in terms of these two. Just some thoughts, Sebastian
![](https://secure.gravatar.com/avatar/764323a14e554c97ab74177e0bce51d4.jpg?s=120&d=mm&r=g)
On Thu, Dec 11, 2014 at 4:17 PM, Sebastian Berg <sebastian@sipsolutions.net> wrote:
True, I once read the docstring of this function. but never used it
http://docs.scipy.org/doc/numpy/reference/generated/numpy.broadcast.html though.
I am not sure it is really the right thing for most things since it returns an old style iterator.
Indeed. That's why I wrote broadcast_arrays().
Why? What benefit does broadcast_arrays() get from being reimplemented in C? -- Robert Kern
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
On Do, 2014-12-11 at 16:20 +0000, Robert Kern wrote:
To be honest, maybe it is not. I remember that I had some function where broadcast_arrays was the largest part of the runtime for smaller arrays and I thought it should be easy since such code exists elsewhere. - Sebastian
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Thu, Dec 11, 2014 at 8:17 AM, Sebastian Berg <sebastian@sipsolutions.net> wrote:
It looks like np.broadcast let's us write the common_shape function very easily; def common_shape(*args): return np.broadcast(*args).shape And it's also very fast: 1000000 loops, best of 3: 1.04 µs per loop So that does seem like a feasible refactor/simplification for np.broadcast_arrays. Sebastian -- if you're up for writing np.broadcast_to in C, that's great! If you're not sure if you'll be able to get around to that in the near future, I'll submit my PR with a Python implementation (which will have tests that will be useful in any case).
![](https://secure.gravatar.com/avatar/dce2259ff9b547103d54acf1ea622314.jpg?s=120&d=mm&r=g)
On Thu, Dec 11, 2014 at 10:53 AM, Stephan Hoyer <shoyer@gmail.com> wrote:
np.broadcast is the Python object of the old iterator. It may be a better idea to write all of these functions using the new one, np.nditer: def common_shape(*args): return np.nditer(args).shape[::-1] # Yes, you do need to reverse it! And in writing 'broadcast_to', rather than rewriting the broadcasting logic, you could check the compatibility of the shape with something like: np.nditer((arr,), itershape=shape) # will raise ValueError if shapes incompatible After that, all that would be left is some prepending of zero strides, and some zeroing of strides of shape 1 dimensions before calling as_strided Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
On Fr, 2014-12-12 at 05:48 -0800, Jaime Fernández del Río wrote:
Hahaha, right there is the 32 limitation, but you can also (ab)use it: np.nditer(np.arange(10), itershape=(5, 10)).itviews[0] - Sebastian
![](https://secure.gravatar.com/avatar/dce2259ff9b547103d54acf1ea622314.jpg?s=120&d=mm&r=g)
On Fri, Dec 12, 2014 at 5:57 AM, Sebastian Berg <sebastian@sipsolutions.net> wrote:
That's neat! But itviews is not even listed in the attributes of nditer in the docs, we should fix that. Is the 32 argument limitation really a concern? Because that aside, it seems that all the functionality that has been discussed are one-liners using nditer: do we need new functions, or better documentation? Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.
![](https://secure.gravatar.com/avatar/b4f6d4f8b501cb05fd054944a166a121.jpg?s=120&d=mm&r=g)
On Fr, 2014-12-12 at 06:25 -0800, Jaime Fernández del Río wrote:
Maybe we could say it isn't a large concern, more something you can fix later on if we find it is, but you would have to check the types, I think that subclasses are probably lost here.
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Fri, Dec 12, 2014 at 6:25 AM, Jaime Fernández del Río < jaime.frio@gmail.com> wrote:
it seems that all the functionality that has been discussed are one-liners using nditer: do we need new functions, or better documentation?
I think there is utility to adding a new function or two (my inclination is to expose broadcast_to in the public API, but leave common_shape in strick_tricks). NumPy provides all the cools to write these in a few lines, but you need to know some very deep details of the NumPy API (nditer and strides). I don't think more documentation would make this obvious -- certainly nditer does not need a longer docstring! The best sort of documentation would be more examples. If this is a recipe that many NumPy users would use, including it in stride_tricks would also serve such an educational purpose (reading stride_tricks is how I figured out how strides work).
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
On Fri, Dec 12, 2014 at 5:48 AM, Jaime Fernández del Río < jaime.frio@gmail.com> wrote:
Unfortunately, that version does not seem to do what I'm looking for: def common_shape(*args): return np.nditer(args).shape[::-1] x = np.empty((4,)) y = np.empty((2, 3, 4)) print(common_shape(x, y)) Outputs: (6, 4) And in writing 'broadcast_to', rather than rewriting the broadcasting
Yes, that is a good idea. Here is a gist with the latest version of this code (shortly to be turned into a PR): https://gist.github.com/shoyer/3e36af0a8196c82d4b42
![](https://secure.gravatar.com/avatar/dce2259ff9b547103d54acf1ea622314.jpg?s=120&d=mm&r=g)
On Fri, Dec 12, 2014 at 11:28 AM, Stephan Hoyer <shoyer@gmail.com> wrote:
Yes, the iterator is a smart beast. I think this is what you need then, with no reversing involved:
np.nditer((x,y), flags=['multi_index']).shape (2, 3, 4)
-- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.
![](https://secure.gravatar.com/avatar/97c543aca1ac7bbcfb5279d0300c8330.jpg?s=120&d=mm&r=g)
On 12 Dec 2014 19:29, "Stephan Hoyer" <shoyer@gmail.com> wrote:
def common_shape(*args):
Nitpick: let's call this broadcast_shape, not common_shape; it's as-or-more clear and clearly groups the related functions together. -n
![](https://secure.gravatar.com/avatar/b4929294417e9ac44c17967baae75a36.jpg?s=120&d=mm&r=g)
On Sunday, December 7, 2014, Stephan Hoyer <shoyer@gmail.com> wrote:
That would be excellent - I ran into exactly the same problem, with the same conclusions, but I was lazier than you were and I did write a routine for making a dummy array in order to use broadcast_arrays: https://github.com/nipy/nibabel/blob/master/nibabel/fileslice.py#L722 https://github.com/nipy/nibabel/blob/master/nibabel/parrec.py#L577 Having a function to do this would be much clearer, thanks for doing that. Cheers, Matthew
![](https://secure.gravatar.com/avatar/93a76a800ef6c5919baa8ba91120ee98.jpg?s=120&d=mm&r=g)
Here is an update on a new function for broadcasting arrays to a given shape (now named np.broadcast_to). I have a pull request up for review, which has received some feedback now: https://github.com/numpy/numpy/pull/5371 There is still at least one design decision to settle: should we expose "broadcast_shape" in the public API? In the current implementation, it is exposed as a public function in numpy.lib.tride_tricks (like as_strided), but it is not exported into the main numpy namespace. The alternatives would be to either make it a private function (_broadcast_shape) or expose it publicly (np.broadcast_shape). Please do speak if you have any thoughts to share on the implementation, either here or in the pull request. Best, Stephan
participants (8)
-
Benjamin Root
-
Jaime Fernández del Río
-
Matthew Brett
-
Nathaniel Smith
-
Pierre Haessig
-
Robert Kern
-
Sebastian Berg
-
Stephan Hoyer