In the following piece of code:
import numpy as N R = N.arange(9).reshape(3,3) ax = [1,2] R array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) R[ax,:][:,ax] = 100 R array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
Why R is not updated? I was expecting:
R array([[0, 1, 2], [3, 100, 100], [6, 100, 100]])
I am using numpy 1.04 on gentoo-linux/amd64 Nadav
A Wednesday 30 January 2008, Nadav Horesh escrigué:
In the following piece of code:
import numpy as N R = N.arange(9).reshape(3,3) ax = [1,2] R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
R[ax,:][:,ax] = 100 R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
Why R is not updated?
Because R[ax] is not a view of R, but another copy of the original object (fancy indexing does return references to different objects). In order to get views, you must specify only a slice of the original array. For example: In [50]: S = R[::2] In [51]: S[:] = 2 In [52]: R Out[52]: array([[2, 2, 2], [3, 4, 5], [2, 2, 2]]) So, what you need is something like: In [68]: R = N.arange(9).reshape(3,3) In [69]: S = R[1:3,:][:,1:3] In [70]: S[:] = 2 In [71]: R Out[71]: array([[0, 1, 2], [3, 2, 2], [6, 2, 2]]) Cheers, --
0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"
But:
R[ax,:] = 100 R array([[ 0, 1, 2], [100, 100, 100], [100, 100, 100]]) R[:,ax] = 200 R array([[ 0, 200, 200], [100, 200, 200], [100, 200, 200]])
Do I get an array view only if the array is contiguous? Nadav. On Wed, 2008-01-30 at 16:08 +0100, Francesc Altet wrote:
A Wednesday 30 January 2008, Nadav Horesh escrigué:
In the following piece of code:
import numpy as N R = N.arange(9).reshape(3,3) ax = [1,2] R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
R[ax,:][:,ax] = 100 R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
Why R is not updated?
Because R[ax] is not a view of R, but another copy of the original object (fancy indexing does return references to different objects). In order to get views, you must specify only a slice of the original array. For example:
In [50]: S = R[::2] In [51]: S[:] = 2 In [52]: R Out[52]: array([[2, 2, 2], [3, 4, 5], [2, 2, 2]])
So, what you need is something like:
In [68]: R = N.arange(9).reshape(3,3) In [69]: S = R[1:3,:][:,1:3] In [70]: S[:] = 2 In [71]: R Out[71]: array([[0, 1, 2], [3, 2, 2], [6, 2, 2]])
Cheers,
Am Mittwoch, 30. Januar 2008 16:21:40 schrieb Nadav Horesh:
But:
R[ax,:] = 100
This is calling __setitem__, i.e. does not create either a view or a copy. Non-contiguous views (e.g. using [::2]) are also possible AFAIK, but fancy indexing is something different. -- Ciao, / / /--/ / / ANS
On Jan 30, 2008 8:21 AM, Nadav Horesh <nadavh@visionsense.com> wrote:
But:
R[ax,:] = 100 R array([[ 0, 1, 2], [100, 100, 100], [100, 100, 100]]) R[:,ax] = 200 R array([[ 0, 200, 200], [100, 200, 200], [100, 200, 200]])
Do I get an array view only if the array is contiguous?
You get an array view if the elements can be addressed with offsets, strides, and counts. Think nested for loops. In [1]: a = zeros((4,4)) In [2]: a[::2,::2] = 1 In [3]: a Out[3]: array([[ 1., 0., 1., 0.], [ 0., 0., 0., 0.], [ 1., 0., 1., 0.], [ 0., 0., 0., 0.]]) Chuck
you simply need to change the definition of ax: ax = slice(1,3) and all works fine. L. On 1/30/08, Francesc Altet <faltet@carabos.com> wrote:
A Wednesday 30 January 2008, Nadav Horesh escrigué:
In the following piece of code:
import numpy as N R = N.arange(9).reshape(3,3) ax = [1,2] R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
R[ax,:][:,ax] = 100 R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
Why R is not updated?
Because R[ax] is not a view of R, but another copy of the original object (fancy indexing does return references to different objects). In order to get views, you must specify only a slice of the original array. For example:
In [50]: S = R[::2] In [51]: S[:] = 2 In [52]: R Out[52]: array([[2, 2, 2], [3, 4, 5], [2, 2, 2]])
So, what you need is something like:
In [68]: R = N.arange(9).reshape(3,3) In [69]: S = R[1:3,:][:,1:3] In [70]: S[:] = 2 In [71]: R Out[71]: array([[0, 1, 2], [3, 2, 2], [6, 2, 2]])
Cheers,
--
0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"
Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- Lorenzo Bolla lbolla@gmail.com http://lorenzobolla.emurse.com/
or you can maybe use numpy.ix_: ax = [1,2] R[numpy.ix_(ax,ax)] = 100 hth, L. On 1/30/08, lorenzo bolla <lbolla@gmail.com> wrote:
you simply need to change the definition of ax: ax = slice(1,3)
and all works fine. L.
On 1/30/08, Francesc Altet <faltet@carabos.com> wrote:
A Wednesday 30 January 2008, Nadav Horesh escrigué:
In the following piece of code:
import numpy as N R = N.arange(9).reshape(3,3) ax = [1,2] R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
R[ax,:][:,ax] = 100 R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
Why R is not updated?
Because R[ax] is not a view of R, but another copy of the original object (fancy indexing does return references to different objects). In order to get views, you must specify only a slice of the original array. For example:
In [50]: S = R[::2] In [51]: S[:] = 2 In [52]: R Out[52]: array([[2, 2, 2], [3, 4, 5], [2, 2, 2]])
So, what you need is something like:
In [68]: R = N.arange(9).reshape(3,3) In [69]: S = R[1:3,:][:,1:3] In [70]: S[:] = 2 In [71]: R Out[71]: array([[0, 1, 2], [3, 2, 2], [6, 2, 2]])
Cheers,
--
0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"
Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- Lorenzo Bolla lbolla@gmail.com http://lorenzobolla.emurse.com/
-- Lorenzo Bolla lbolla@gmail.com http://lorenzobolla.emurse.com/
Thank you very much, that what I was looking for. Charles made a good point about offsets, counts and strides --- I really should go and reread the documentation. Nadav. -----Original Message----- From: numpy-discussion-bounces@scipy.org on behalf of lorenzo bolla Sent: Wed 30-Jan-08 17:31 To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] Can not update a submatrix or you can maybe use numpy.ix_: ax = [1,2] R[numpy.ix_(ax,ax)] = 100 hth, L. On 1/30/08, lorenzo bolla <lbolla@gmail.com> wrote:
you simply need to change the definition of ax: ax = slice(1,3)
and all works fine. L.
On 1/30/08, Francesc Altet <faltet@carabos.com> wrote:
A Wednesday 30 January 2008, Nadav Horesh escrigué:
In the following piece of code:
import numpy as N R = N.arange(9).reshape(3,3) ax = [1,2] R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
R[ax,:][:,ax] = 100 R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
Why R is not updated?
Because R[ax] is not a view of R, but another copy of the original object (fancy indexing does return references to different objects). In order to get views, you must specify only a slice of the original array. For example:
In [50]: S = R[::2] In [51]: S[:] = 2 In [52]: R Out[52]: array([[2, 2, 2], [3, 4, 5], [2, 2, 2]])
So, what you need is something like:
In [68]: R = N.arange(9).reshape(3,3) In [69]: S = R[1:3,:][:,1:3] In [70]: S[:] = 2 In [71]: R Out[71]: array([[0, 1, 2], [3, 2, 2], [6, 2, 2]])
Cheers,
--
0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"
Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- Lorenzo Bolla lbolla@gmail.com http://lorenzobolla.emurse.com/
-- Lorenzo Bolla lbolla@gmail.com http://lorenzobolla.emurse.com/
On 30/01/2008, Francesc Altet <faltet@carabos.com> wrote:
A Wednesday 30 January 2008, Nadav Horesh escrigué:
In the following piece of code:
import numpy as N R = N.arange(9).reshape(3,3) ax = [1,2] R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
R[ax,:][:,ax] = 100 R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
Why R is not updated?
Because R[ax] is not a view of R, but another copy of the original object (fancy indexing does return references to different objects). In order to get views, you must specify only a slice of the original array. For example:
This is not exactly correct. There are two kinds of fancy indexing in numpy, which behave similarly. There is normal fancy indexing, which produces a new array, not sharing data with the old array: a = N.random.normal(size=10) b = a[a>0] There is also fancy indexing of lvalues (to use a C term): a = N.random.normal(size=10) a[a<0] *= -1 Here no copy is made; instead, the data is modified in-place. This requires some clever hacks under the hood, since a[a<0] *can't* be a view of a. The problem the OP had is that they are going beyond what the clever hacks can deal with. That's why R[ax,:] = 100 works but R[ax,:][:,ax] = 100 doesn't. The problem is that you have two indexing operations in the second situation, and the inplace operators can't deal with that. Solutions include working on a flattened version of the array (for which a single indexing operation suffices), using the (not in-place) where() construct, or using the ix_ function: R[N.ix_(ax,ax)] = 100 N.ix_ is necessary because R[ax,ax] doesn't do what I, for one, would have expected: R[[1,2],[3,4]] yields [R[1,3],R[2,4]], not [[R[1,3],R[1,4]],[R[2,3],R[2,4]]]. But in any case, once you reduce it to a single fancy-indexing operation, you can successfully use it as an lvalue. Striding and views are not really the issue. Anne
On Jan 30, 2008 12:43 PM, Anne Archibald <peridot.faceted@gmail.com> wrote:
On 30/01/2008, Francesc Altet <faltet@carabos.com> wrote:
A Wednesday 30 January 2008, Nadav Horesh escrigué:
In the following piece of code:
import numpy as N R = N.arange(9).reshape(3,3) ax = [1,2] R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
R[ax,:][:,ax] = 100 R
array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
Why R is not updated?
Because R[ax] is not a view of R, but another copy of the original object (fancy indexing does return references to different objects). In order to get views, you must specify only a slice of the original array. For example:
This is not exactly correct.
[...a fine explanation by Anne...] Another way to look at this is note that: R[ax,:][:,ax] = 100 is translated to: R.__getitem__((ax, :)).__setitem__((:,ax), 100) '__setitem__' is capable of setting values using fancy indexing, but as has been noted, '__getitem__' makes a copy of R when using fancy indexing, and therefore the original does not get modified. [Technically, ':' gets translated to slice(None,None,None), but I didn't want to write that out]. -- . __ . |-\ . . tim.hochberg@ieee.org
Thanks for your explanation. It explains also the following:
R = N.arange(9).reshape(3,3) ax = [0,2] U = R[ax,:] U array([[0, 1, 2], [6, 7, 8]]) U[:] = 100 U array([[100, 100, 100], [100, 100, 100]]) R # No change since U is a copy array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) R[ax,:] = 100 # Works through __setitem__ R array([[100, 100, 100], [ 3, 4, 5], [100, 100, 100]])
Nadav On Wed, 2008-01-30 at 13:35 -0700, Timothy Hochberg wrote:
On Jan 30, 2008 12:43 PM, Anne Archibald <peridot.faceted@gmail.com> wrote:
On 30/01/2008, Francesc Altet <faltet@carabos.com> wrote: > A Wednesday 30 January 2008, Nadav Horesh escrigué: > > In the following piece of code: > > >>> import numpy as N > > >>> R = N.arange(9).reshape(3,3) > > >>> ax = [1,2] > > >>> R > > > > array([[0, 1, 2], > > [3, 4, 5], > > [6, 7, 8]]) > > > > >>> R[ax,:][:,ax] = 100 > > >>> R > > > > array([[0, 1, 2], > > [3, 4, 5], > > [6, 7, 8]]) > > > > Why R is not updated? > > Because R[ax] is not a view of R, but another copy of the original > object (fancy indexing does return references to different objects). > In order to get views, you must specify only a slice of the original > array. For example:
This is not exactly correct.
[...a fine explanation by Anne...]
Another way to look at this is note that:
R[ax,:][:,ax] = 100
is translated to:
R.__getitem__((ax, :)).__setitem__((:,ax), 100)
'__setitem__' is capable of setting values using fancy indexing, but as has been noted, '__getitem__' makes a copy of R when using fancy indexing, and therefore the original does not get modified.
[Technically, ':' gets translated to slice(None,None,None), but I didn't want to write that out].
-- . __ . |-\ . . tim.hochberg@ieee.org
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
A Wednesday 30 January 2008, Timothy Hochberg escrigué:
[...a fine explanation by Anne and Timothy...]
Ok. As it seems that this subject has interest enough, I went ahead and created a small document about views vs copies at: http://www.scipy.org/Cookbook/ViewsVsCopies I think it resumes what has been said in this thread, but feel free to update it if you find some inaccuracy or want to complete it more. Although perhaps this is more a FAQ than a recipe, I've added it to the SciPy cookbook because I find it a bit long for a FAQ entry (but I may be wrong). In fact, I feel that the cookbook would need some reorganization, because there are too many different subjects in NumPy/SciPy section of http://www.scipy.org/Cookbook. One first action, IMHO, would be to create separate NumPy/SciPy FAQs and Cookbooks. It's my impression that NumPy is a much more general tool than SciPy with much less requeriments to install, test and use it, and having such a monolithic merge of NumPy and SciPy cookbok could discourage many potential NumPy users that might see NumPy as 'too scientific'. After that, I'd introduce a new subsection called 'Advanced topics' in both NumPy & SciPy FAQ/Cookbook. For example, currently there exists things like "SphericalBesselZeros", "Savitzky Golay filtering of data" in the first section of the cookbook, which is clearly a bit overhelming for the naive user. I'd like to hear some comments about this proposal from the community. At any rate, I've placed the "ViewsVsCopies" entry under a new section that I created in the cookbook, that I called "Advanced topics", but I'm open to suggestions on a better name/place. Cheers, --
0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"
A Thursday 31 January 2008, Francesc Altet escrigué:
A Wednesday 30 January 2008, Timothy Hochberg escrigué:
[...a fine explanation by Anne and Timothy...]
Ok. As it seems that this subject has interest enough, I went ahead and created a small document about views vs copies at:
Ooops, I think I've missed the NumPy tutorial: http://www.scipy.org/Tentative_NumPy_Tutorial which already talked about copies vs views :-/. Well, I think my small document can complement some parts of the tutorial. I'll do that as soon as I can and remove the recipe from the cookbook. Sorry for the noise. Cheers, --
0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"
participants (7)
-
Anne Archibald -
Charles R Harris -
Francesc Altet -
Hans Meine -
lorenzo bolla -
Nadav Horesh -
Timothy Hochberg