Re: [Numpy-discussion] Numpy-discussion Digest, Vol 22, Issue 109
Thanks so much for your help on the '*' confusion. It makes sense now. Thanks, Frank On Fri, Jul 25, 2008 at 3:57 PM, <numpy-discussion-request@scipy.org> wrote:
Send Numpy-discussion mailing list submissions to numpy-discussion@scipy.org
To subscribe or unsubscribe via the World Wide Web, visit http://projects.scipy.org/mailman/listinfo/numpy-discussion or, via email, send a message with subject or body 'help' to numpy-discussion-request@scipy.org
You can reach the person managing the list at numpy-discussion-owner@scipy.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Numpy-discussion digest..."
Today's Topics:
1. Re: [Cdat-discussion] Arrays containing NaNs (Charles Doutriaux) 2. numpy-1.1.1rc2 Mac binary - Please Test. (Christopher Burns) 3. Re: numpy-1.1.1rc2 Mac binary - Please Test. (Alan McIntyre) 4. curious problem with SVD (Frank Lagor) 5. Re: curious problem with SVD (Keith Goodman) 6. Re: curious problem with SVD (Keith Goodman) 7. Re: curious problem with SVD (Robert Kern) 8. Re: curious problem with SVD (Arnar Flatberg)
---------- Forwarded message ---------- From: Charles Doutriaux <doutriaux1@llnl.gov> To: Pierre GM <pgmdevlist@gmail.com> Date: Fri, 25 Jul 2008 11:43:41 -0700 Subject: Re: [Numpy-discussion] [Cdat-discussion] Arrays containing NaNs Hi Pierre,
Thanks for the answer, I'm ccing cdat's discussion list.
It makes sense, that's also the way we develop things here NEVER assume what the user is going to do with the data BUT give the user the necessary tools to do what you're assuming he/she wants to do (as simple as possible)
Thanks again for the answer.
C.
Pierre GM wrote:
Oh, I guess this one's for me...
On Thursday 01 January 1970 04:21:03 Charles Doutriaux wrote:
Basically it was suggested to automarically mask NaN (and Inf ?) when creating ma. I'm sure you already thought of this on this list and was curious to know why you decided not to do it.
Because it's always best to let the user decide what to do with his/her data and not impose anything ?
Masking a point doesn't necessarily mean that the point is invalid (in the sense of NaNs/Infs), just that it doesn't satisfy some particular condition. In that sense, masks act as selecting tools.
By forcing invalid data to be masked at the creation of an array, you run the risk to tamper with the (potential) physical meaning of the mask you have given as input, and/or miss the fact that some data are actually invalid when you don't expect it to be.
Let's take an example: I want to analyze sea surface temperatures at the world scale. The data comes as a regular 2D ndarray, with NaNs for missing or invalid data. In a first step, I create a masked array of this data, filtering out the land masses by a predefined geographical mask. The remaining NaNs in the masked array indicate areas where the sensor failed... It's an important information I would probably have missed by masking all the NaNs at first...
As Eric F. suggested, you can use numpy.ma.masked_invalid to create a masked array with NaNs/Infs filtered out:
import numpy as np,. numpy.ma as ma
x = np.array([1,2,None,4], dtype=float) x
array([ 1., 2., NaN, 4.])
mx = ma.masked_invalid(x)
mx
masked_array(data = [1.0 2.0 -- 4.0], mask = [False False True False], fill_value=1e+20)
Note that the underlying data still has NaNs/Infs:
mx._data
array([ 1., 2., NaN, 4.])
You can also use the ma.fix_invalid function: it creates a mask where the data is not finite (NaNs/Infs), and set the corresponding points to fill_value.
mx = ma.fix_invalid(x, fill_value=999)
mx
masked_array(data = [1.0 2.0 -- 4.0], mask = [False False True False], fill_value=1e+20)
mx._data
array([ 1., 2., 999., 4.])
The advantage of the second approach is that you no longer have NaNs/Infs in the underlying data, which speeds things up during computation. The obvious disadvantage is that you no longer know where the data was invalid...
---------- Forwarded message ---------- From: "Christopher Burns" <cburns@berkeley.edu> To: "Discussion of Numerical Python" <numpy-discussion@scipy.org> Date: Fri, 25 Jul 2008 11:48:13 -0700 Subject: [Numpy-discussion] numpy-1.1.1rc2 Mac binary - Please Test. Reminder, please test the Mac installer for rc2 so we have time to fix any bugs before the release next week.
Also, I committed my build script to the trunk/tools/osxbuild. bdist_mpkg 0.4.3 is required.
Thank you, Chris
On Thu, Jul 24, 2008 at 11:03 AM, Jarrod Millman <millman@berkeley.edu> wrote:
Hello,
The 1.1.1rc2 is now available: http://svn.scipy.org/svn/numpy/tags/1.1.1rc2
The source tarball is here: http://cirl.berkeley.edu/numpy/numpy-1.1.1rc2.tar.gz
Here is the universal Mac binary: http://cirl.berkeley.edu/numpy/numpy-1.1.1rc2-py2.5-macosx10.5.dmg
David Cournapeau will be creating a 1.1.1rc2 Windows binary in next few days.
Please test this release ASAP and let us know if there are any problems. If there are no show stoppers, this will likely become the 1.1.1 release.
Thanks,
-- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/ _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
---------- Forwarded message ---------- From: "Alan McIntyre" <alan.mcintyre@gmail.com> To: "Discussion of Numerical Python" <numpy-discussion@scipy.org> Date: Fri, 25 Jul 2008 15:00:42 -0400 Subject: Re: [Numpy-discussion] numpy-1.1.1rc2 Mac binary - Please Test. On Fri, Jul 25, 2008 at 2:48 PM, Christopher Burns <cburns@berkeley.edu> wrote:
Reminder, please test the Mac installer for rc2 so we have time to fix any bugs before the release next week.
I just tried it; it installs with no problems and tests run with no failures.
---------- Forwarded message ---------- From: "Frank Lagor" <dfranci@seas.upenn.edu> To: numpy-discussion@scipy.org Date: Fri, 25 Jul 2008 15:32:56 -0400 Subject: [Numpy-discussion] curious problem with SVD Perhaps I do not understand something properly, if so could someone please explain the behavior I notice with numpy.linalg.svd when acting on arrays. It gives the incorrect answer, but works fine with matrices. My numpy is 1.1.0.
R = n.array([[3.6,.35],[.35,1.8]]) V,D,W = n.linalg.svd(R) V*n.diag(D)*W.transpose() array([[ 3.5410365 , 0. ], [ 0. , 1.67537611]]) R = n.matrix([[3.6,.35],[.35,1.8]]) V,D,W = n.linalg.svd(R) V*n.diag(D)*W.transpose() matrix([[ 3.6 , 0.35], [ 0.35, 1.8 ]])
Thanks in advance, Frank -- Frank Lagor Ph.D. Candidate Mechanical Engineering and Applied Mechanics University of Pennsylvania
Perhaps I do not understand something properly, if so could someone
---------- Forwarded message ---------- From: "Keith Goodman" <kwgoodman@gmail.com> To: "Discussion of Numerical Python" <numpy-discussion@scipy.org> Date: Fri, 25 Jul 2008 12:36:28 -0700 Subject: Re: [Numpy-discussion] curious problem with SVD On Fri, Jul 25, 2008 at 12:32 PM, Frank Lagor <dfranci@seas.upenn.edu> wrote: please
explain the behavior I notice with numpy.linalg.svd when acting on arrays. It gives the incorrect answer, but works fine with matrices. My numpy is 1.1.0.
R = n.array([[3.6,.35],[.35,1.8]]) V,D,W = n.linalg.svd(R) V*n.diag(D)*W.transpose() array([[ 3.5410365 , 0. ], [ 0. , 1.67537611]]) R = n.matrix([[3.6,.35],[.35,1.8]]) V,D,W = n.linalg.svd(R) V*n.diag(D)*W.transpose() matrix([[ 3.6 , 0.35], [ 0.35, 1.8 ]])
'*' does element-by-element multiplication for arrays but matrix multiplication for matrices.
On Fri, Jul 25, 2008 at 12:32 PM, Frank Lagor <dfranci@seas.upenn.edu> wrote:
Perhaps I do not understand something properly, if so could someone
---------- Forwarded message ---------- From: "Keith Goodman" <kwgoodman@gmail.com> To: "Discussion of Numerical Python" <numpy-discussion@scipy.org> Date: Fri, 25 Jul 2008 12:39:23 -0700 Subject: Re: [Numpy-discussion] curious problem with SVD On Fri, Jul 25, 2008 at 12:36 PM, Keith Goodman <kwgoodman@gmail.com> wrote: please
explain the behavior I notice with numpy.linalg.svd when acting on arrays. It gives the incorrect answer, but works fine with matrices. My numpy is 1.1.0.
R = n.array([[3.6,.35],[.35,1.8]]) V,D,W = n.linalg.svd(R) V*n.diag(D)*W.transpose() array([[ 3.5410365 , 0. ], [ 0. , 1.67537611]]) R = n.matrix([[3.6,.35],[.35,1.8]]) V,D,W = n.linalg.svd(R) V*n.diag(D)*W.transpose() matrix([[ 3.6 , 0.35], [ 0.35, 1.8 ]])
'*' does element-by-element multiplication for arrays but matrix multiplication for matrices.
As a check (for the array case):
n.dot(V, n.dot(n.diag(D), W.transpose())) # That's hard to read!
array([[ 3.6 , 0.35], [ 0.35, 1.8 ]])
Perhaps I do not understand something properly, if so could someone
---------- Forwarded message ---------- From: "Robert Kern" <robert.kern@gmail.com> To: "Discussion of Numerical Python" <numpy-discussion@scipy.org> Date: Fri, 25 Jul 2008 14:50:24 -0500 Subject: Re: [Numpy-discussion] curious problem with SVD On Fri, Jul 25, 2008 at 14:32, Frank Lagor <dfranci@seas.upenn.edu> wrote: please
explain the behavior I notice with numpy.linalg.svd when acting on arrays. It gives the incorrect answer, but works fine with matrices. My numpy is 1.1.0.
R = n.array([[3.6,.35],[.35,1.8]]) V,D,W = n.linalg.svd(R) V*n.diag(D)*W.transpose()
For regular arrays, * is element-wise multiplication, not matrix multiplication. For matrix objects, * is matrix multiplication.
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
---------- Forwarded message ---------- From: "Arnar Flatberg" <arnar.flatberg@gmail.com> To: "Discussion of Numerical Python" <numpy-discussion@scipy.org> Date: Fri, 25 Jul 2008 21:53:09 +0200 Subject: Re: [Numpy-discussion] curious problem with SVD
On Fri, Jul 25, 2008 at 9:39 PM, Keith Goodman <kwgoodman@gmail.com> wrote:
On Fri, Jul 25, 2008 at 12:32 PM, Frank Lagor <dfranci@seas.upenn.edu> wrote:
Perhaps I do not understand something properly, if so could someone
On Fri, Jul 25, 2008 at 12:36 PM, Keith Goodman <kwgoodman@gmail.com> wrote: please
explain the behavior I notice with numpy.linalg.svd when acting on arrays. It gives the incorrect answer, but works fine with matrices. My numpy is
'*' does element-by-element multiplication for arrays but matrix multiplication for mat
n.dot(V, n.dot(n.diag(D), W.transpose())) # That's hard to read!
Just two small points:
1.) Broadcasting may be easier on the eye ... well, atleast when you have gotten used to it Then the above is np.dot(V*D, W)
2.) Also, note that the right hand side eigenvectors in numpy's svd routine is ordered by rows! Yes, I know this is confusing as it is different from just about any other linear algebra software out there, but the documentation is clear. It is also a little inconsistent with eig and eigh, some more experienced user can probably answer on why it is like that?
Arnar
<http://projects.scipy.org/mailman/listinfo/numpy-discussion>
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- Frank Lagor Ph.D. Candidate Mechanical Engineering and Applied Mechanics University of Pennsylvania
participants (1)
-
Frank Lagor