Mailman 3 August 2001 - NumPy-Discussion

simple array help?
by Pete Shinners Sept. 17, 2001

Sept. 17, 2001

i have a simple array with floating values >>> print a array([ 0. , 0.2, 0.4, 0.6, 0.8]) what i need to do is change each index into another dimension with 3 indices. hmm, kind of confusing, an example will explain better >>> b = array(zip(a,a,a)) >>> print b array([[ 0. , 0. , 0. ], [ 0.2, 0.2, 0.2], [ 0.4, 0.4, 0.4], [ 0.6, 0.6, 0.6], [ 0.8, 0.8, 0.8]]) ok, this does the job, but once i start using large arrays, the back-n-forth conversion between arrays and python lists is costing me quite a bit. is there a way i can reshape the array this way without the call to python's 'zip'? (no, 'map' doesn't count either) i've tried much fooling with NewAxis, resize, and friends. but either i haven't stumbled upon the correct combination or i'm not trying the correct tools. i'd like to keep this as quick as possible, so hopefully it can be done without anything too elaborate. thanks (doh, sorry for the previous distutils related post... hit the wrong mailing list)

6 19

matrix invert() problems
by Rob Aug. 31, 2001

Aug. 31, 2001

I found a slight discrepancy in my Python EM code output when I use the invert() function. It may be that the C routine that I am comparing it to actually has the bug. For now I don't know. It would be cool to have a large two dimensional array that has a known inverse. I think NIST keeps an array collection, but I don't know if they have the inverses. Rob. -- The Numeric Python EM Project www.members.home.net/europax

1 1

Re:RE: [Numpy-discussion] Cholesky factorization
by Jeff Whitaker Aug. 31, 2001

Aug. 31, 2001

I've uploaded a new version of my LinearAlgebra patch that adds a Cholesky decomposition routine (cholesky_decomposition), as per Frederik's request. -Jeff On Fri, 31 Aug 2001, Fredrik Stenberg wrote: > You read my mind...... > > ;)) > > /fredriks > > > ------- Ursprungligt meddelande ------- > Fr�n: jsw(a)cdc.noaa.gov > Datum: Fri, 31 Aug 2001 08:32:01 -0700 > > This message was sent from Geocrawler.com by "jeff whitaker" > <jsw(a)cdc.noaa.gov> > > > Fredrik: I could easily add this to the > LinearAlgebra patches I've just submitted. Are > you thinking of an interface to the lapack routine > dpotrf (cholesky factorization of a real symmteric > pos definite matrix), or something else? > > -Jeff > > --------------------------------------- > Hi.. > > Has anyone written a Cholesky factorization for > general > matrices? > > Any package out there? > > The only one i know of is the python > wrappers around lapack (pylapack). > > I could write it myself, but i prefer to be lazy ;)) > > /fredriks > > > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion(a)lists.sourceforge.net > http://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > > Geocrawler.com - The Knowledge Archive > > > > -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/CDC R/CDC1 Email : jsw(a)cdc.noaa.gov 325 Broadway Web : www.cdc.noaa.gov/~jsw Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124

1 0

RE: [Numpy-discussion] one-offset arrays
by Rodgers, Kevin Aug. 31, 2001

Aug. 31, 2001

Indices in Fortran can be defined at the time an array is declared, so Fortran is not necessarily "one-indexed". Kevin Rodgers Northrop Grumman Ryan Aeronautical -----Original Message----- From: Eric Nodwell [mailto:nodwell@physics.ubc.ca] Sent: Tuesday, August 28, 2001 10:01 PM To: Numpy-discussion(a)lists.sourceforge.net Subject: Re: [Numpy-discussion] one-offset arrays >> Does anyone have a simple method of using one-offset arrays? >> Specifically, can I define an array "a" so that a[1] refers to the >> first element? >> > Inherit from the UserArray and redefine slicing to your hearts content. >> Without one-offset indexing, it seems to me that Python is minimally >> useful for numerical computations. Many, perhaps the majority, of >> numerical algorithms are one-indexed. Matlab for example is one-based >> for this reason. In fact it seems strange to me that a "high-level" >> language like Python should use zero-offset lists. >> > Wow, that is a pretty condescending-sounding statement, though I'm sure you > didn't mean it that way. Python is indeed being used for quite serious > numerical computations. I have been using Python for quite some time for > Numerical work and find it's zero-based indexing combined with the > leave-last-one-out slicing notation to be much more convenient. Oops, what I really need is a wet-ware (i.e. brain) macro which enforces the correct order of the pair (think,write,send)! The above unconsidered comments arose from the sequence (frustration,write,send,think,regret). ;) Obviously there is a lot of numerical work being done with python and people are very happy with it. But for me I still think it would be "minimally useful" without 1-indexed arrays. Here's why: In my view, the most important reason to prefer 1-based indexing versus 0-based indexing is compatibility. For numerical work, some of the languages which I use or have used are Matlab, Mathematica, Maple and Fortran. These are all 1-indexed. (C is by nature 0-indexed because it is so close to machine architecture, but with a little bit of not-entirely-clean pointer manipulation, you can easily make 1-indexed arrays and matrices.) Obviously Python can't be compatible with these languages in a strict sense, but like most people who do some programming work, I've built up a library of my own commonly used routines specific to my work; in general it's a trivial matter to translate numerical routines from one language to the another if translation is just a matter of substituting of one set of syntactical symbols and function names for anther. However it can be damn tricky to convert 1-indexed code to 0-indexed code or visa versa without introducing any errors- believe me! (Yes it's possible to call nearly any language from nearly any other language these days so in theory you don't have to recode, but there are lots of reasons why often recoding is the preferable route.) The other reason for choosing 1-based indexing is to keep the code as near to the standard notation as possible. This is one of the advantages of using a high level language - it approximates the way you think about things instead of the way the computer organizes them. Of course, this can go either way depending on the quantity in question: as a simple example a spatial vector (x,y,z) is conventionally labelled 1,2,3 (1-indexed), but a relativistic four-vector with time included (t,x,y,z) is conventionally labelled 0,1,2,3 (0-indexed). So ideally one would be able to choose the indexing-type case-by-case. I'm sure that computer programmers will argue vehemently that code which mixes both 0-indexed and 1-indexed arrays is hard to understand and maintain, but for experts in a particular field who are accustomed to certain ingrained notations, it is the code which breaks the conventional notation which is most error-prone. In my case, I'm dealing at the moment with crystal structures with which are associated certain conventional sets of vectors and tensors - all 1-indexed by convention. I find it a delicate enough task to always get the correct vector or tensor without having to remember that d[2] is actually d3. Defining d1,d2,d3 is not convenient because often the index itself needs to be calculated. I guess if I understood the reason for 0-indexed lists and tuples in Python I would be happier. In normal, everyday usage, sets, collections and lists are 1-indexed (the first item, the second item, the third item, and so on). Python is otherwise such an elegant and natural language. Why the ugly exception of making the user conform to the underlying mechanism of an array being an address plus an offset? All this is really neither here nor there, since this debate, at least as far as Python is concerned, was probably settled 10 years ago and I'm sure nobody wants to hear anything more about it at this point. As you point out, I can define my own array type with inheritance. I will also need my own range command and several other functions which haven't occured to me yet. I was hoping that there would be a standard module to implement this. By the way, what is leave-last-one-out slicing? Is it a[:-1] or is it a[0,...] or is it something else? Eric _______________________________________________ Numpy-discussion mailing list Numpy-discussion(a)lists.sourceforge.net http://lists.sourceforge.net/lists/listinfo/numpy-discussion

1 0

Cholesky factorization
by Fredrik Stenberg Aug. 31, 2001

Aug. 31, 2001

Hi.. Has anyone written a Cholesky factorization for general matrices? Any package out there? The only one i know of is the python wrappers around lapack (pylapack). I could write it myself, but i prefer to be lazy ;)) /fredriks

2 1

MLab.py status: do you use it?
by Chris Barker Aug. 31, 2001

Aug. 31, 2001

Hi all, Paul recently requested help with testing and debugging MLab.py, and I offered to help out. He then sent me this note: "Paul F. Dubois" wrote: > MLab.py started out as a package that was supposed to give numpy a > Matlab look and feel. So useful questions are: are there really useful > things that should be added? I can't answer these questions myself. Interestingly enough, despite the fact that I came to NumPy having used MATLAB heavily for five years (and one dissertation), I have made very little use of MLab.py. I'm wondering if anyone is, indeed, using it. As for trying to give numpy a MATLAB look and feel, I question the usefulness of that. NumPy looks and feels a little different than MATLAB, and, frankly, I like NumPy's feel better for the most part (now that rich comparisons have been added, anyway). The one thing that MATLAB has that NumPy doesn't, that I really miss, is list indexing. Having to use put and take is so much less elegant! I'm sure this isn't the least bit trivial to add, however. (is it even possible, given Python's idea of slicing?) Anyway, what I do think MATLAB does provide, and MLab.py should, is a whole bunch of utility functions for various common manipulations. > Do we really have what we have correct? > Most of all, these is no test routine for it. If you could make one > following the model of the test.py in directory Test, it would be great. > The idea is to have something that does not require the runner of the > test to know anything. It just runs silently unless something is wrong. I certainly could write up a test routine. I will start work on that. In the meantime, before I put too much effort into it, I'd really like some feedback on what people want MLab.py to be, if they see using it at all. As I said above, I'm not sure trying to give NumPy a MATLAB feel is a useful goal, so what is left is having all those handy functions. Perhaps we could shift away from MLab.py, and turn it into Utilities.py, which could use MATLAB (and other user experience) as guidance as to what utilities to include. I'd really like folks' feedback on this before I put too much work into something no one uses, or will use. -Chris -- Christopher Barker, Ph.D. ChrisHBarker(a)home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------

4 4

Subarray with with arbitrary index?
by Huaiyu Zhu Aug. 31, 2001

Aug. 31, 2001

Hi, Is it possible to assign to a subarray with arbitrary index? Suppose I have three arrays a = arange(8) b = array([2, 3, 5]) c = arange(3)+100 I want a function f, such that calling f(a, b, c) would change a to [0 1 100 101 4 102 6 7] This would be useful in certain applications that would otherwise require sparse matrices. Huaiyu Zhu

3 3

invert() function works just fine
by Rob Aug. 31, 2001

Aug. 31, 2001

My problems with the invert() function has been resolved. I found that somewhere in my program it was trying to invert [0]. So I just added a conditional statement ahead of it. Evidently the C program did the same sort of thing. So now I've shaved another 20% of execution time off of my program! Rob. -- The Numeric Python EM Project www.members.home.net/europax

1 0

LinearAlgebra patch
by Jeff Whitaker Aug. 31, 2001

Aug. 31, 2001

Has anyone tried my patch to LinearAlgebra.py and lapack_litemodule.c? I've found major speedups (by a factor of 3-5) in all operations involving an svd or a eigenanalysis of a symmetric matrix. Paul's initial comment was that since it was not self-contained (it required linking external blas and lapack libs) he could not consider putting it in. I have rectified that now by including new versions of f2c_lite.c, blas_lite.c and d(z)lapack_lite.c that support the newer, faster lapack routines. I would appreciate any reports on how it works (or doesn't) on other platforms - I've tried it in on MacOS X and Solaris. Grab it from the 'Patches' section of the sourceforge project page. -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/CDC R/CDC1 Email : jsw(a)cdc.noaa.gov 325 Broadway Web : www.cdc.noaa.gov/~jsw Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124

3 3

Re: [Numpy-discussion] one-offset arrays
by Eric Nodwell Aug. 30, 2001

Aug. 30, 2001

Thanks to everyone who responded to my comments about one-offset arrays in Python. I understand much better now why zero-offset arrays are a good choice for Python. For the reasons already discussed (mostly ease of translating one-offset algorithms), one-offset arrays would nevertheless be useful in certain situations, so I went ahead and wrote a class for a one-offset array. It ended up being a somewhat bigger job than I expected, and since it is rather too much material for a mailing-list submission I created a web site for it. Anyone who is interested in this topic please have a look: http://www.physics.ubc.ca/~mbelab/python/arrayone/index.html I'd love some feedback along the following lines: 1. There are 4 outstanding problems/bugs which I could identify but was unable to fix. Most of these seem to be due to limitations of the UserArray class, but they could also be due to limitations of the programmer. None of these problems make the class unusable. One of them is the issue which Travis identified about functions taking UserArrays but returning standard arrays. It would be nice if the resulting discussion led to something be done... Any suggestions for fixing the other 3 issues would be most welcome. 2. I would like this module to be as accessible as possible to others, particularly those who are just getting started with Python and may therefore be especially encumbered with one-offset baggage. How can I do this? Submit it to a Python resource such as the Vaults of Parnassus? Get people to link to my web page? Does someone with a more prominant website want to volunteer to host it? Is there any possibility of having it included eventually with Numerical Python or Scientific Python? Probably some people will consider it too trivial to include, but I found it less trivial than I expected once all the cases were covered - and there are still the 4 outstanding issues. (Yes the code sucks! Suggestions for improvement are most welcome!) Why make people reinvent the wheel? People coming from MatLab for example might be inclined to give up and go back to MatLab. Eric P.S. In case anyone is interested in the outstanding problems but for some reason doesn't want to or can't visit the web site, here are the summaries: Problem 1 --------- Description for loops don't work with variables of type arrayone Example X=arrayone((1,2,3)) for item in X: print item This just generates an error. Reason In Python, a for loop works by starting at x(0) and incrementing until an out-of-bounds error occurs. arrayone types have no 0 element. Work-around: Cast to type array in for loops. For example X=arrayone((1,2,3)) for item in array(X): print item Possible solutions: Is it possible to "overload" "for" so that it behaves differently for arrayone type? Problem 2 --------- Description Slicing an arrayone from 0 doesn't generate an error but it should. Example: X=arrayone((1,2,3)) X[0:3] This returns ArrayOne.arrayone([1, 2, 3]) instead of an error. Reason X[:3] results in a call to __getslice__ with low=0. This cannot be distinguished from X[0:3]. Therefore in order to deal correctly with the X[:] case, we have to assume that low=0 means an unspecified low. Work-around If you don't trust your code, you have to implement specific checking for this condition before slicing. Possible solution If it was possible to get access to the unparsed input (literally '0:3' for example), this could be fixed. Problem 3 --------- Description Negative subscripts return a slice offset by -1 from the expected result. Example X=arrayone((1,2,3,4)) X[-3:-2] This returns ArrayOne.arrayone([1, 2]) instead of ArrayOne.arrayone([2, 3]). Reason X[-3:-2] in the above example results in a call to __getslice__ with low=1, high=2, which cannot be distinguished from X[1:2]. Work-around Don't use negative index slicing with arrayone types. Possible solution: If had access to unparsed input, could fix this. Problem 4 --------- Description Functions which take arrayone arguments return type array. Work-around Have to cast back into arrayone, for example: Y=arrayone(somefunc(X)) Possible solution See http://www.geocrawler.com/lists/3/SourceForge/1329/0/6505926

2 2