Mailman 3 October 2005 - NumPy-Discussion

Re: [Numpy-discussion] Question: why is this different?
by meng＠are.berkeley.edu 18 Oct '05

18 Oct '05

Todd and Paul- Thank you so much for your help! Any suggestions on my next step? This difference doesn't make my result qualitatively different but my final result differs in the 3rd decimal point across these two machines, which could be troublesome when you want something that's robust to these computing errors. Best, Xiangyi At 18:10 2005-10-17 -0400, Todd Miller wrote: >meng(a)are.berkeley.edu wrote: > >>Hi there- >> >>I have a question on how this difference arises when I run the following >>simple code on windows and Debian: >> >>> from numarray import * >> >>> a=reshape(arange(81.0), (9,9))+identity(9) >> >>> ca = matrixmultiply(transpose(a),a) >> >>> import numarray.linear_algebra as la >> >>> ica = la.inverse(ca) >> >>> ica.min() >>-0.30951621414374736 >> >>> ica.max() >>0.8888918586585135 >> >>The above is the result in Debian and below is that in Windows: >> >>> ica.min() >>-0.30951621414449404 >> >>> ica.max() >>0.88889185865875686 > >I'm able to reproduce your "Windows results" in Fedora Core 4 x86_64... >exactly. That says that a variant of gcc and VC.NET are producing >identical results to 18 digits... which is encouraging. Numeric agrees as >well. > >When I build numarray with LAPACK and ATLAS rather than the builit-in >lapack_lite on RHEL3 i386 I get different results from any of the above: > >ica.min: -0.30951621414375136 >ica.max: 0.88889185865854037 > >They are closest to Debian I think, differing in the 15th decimal >place. My suspicion is that your Debian version is being built against >different numerical libraries... but I'm not privy to exactly how. A key >point is that numarray on Windows is *not* built using LAPACK and ATLAS. > >>So, what caused this difference (both machines have python2.4 and >>numarray 1.3.2)? The difference happens in the 10th/11th decimal point, >>which should be still in the floating point precision, right? > >I would expect so. > Best, Xiangyi Xiangyi Meng Department of Agricultural and Resource Economics Room 303, Giannini Hall #3310 University of California, Berkeley Berkeley, CA 94720-3310 Tel: (510) 643-4124 Fax: (510) 643-8911 Email: meng(a)are.berkeley.edu

1 0

Question: why is this different?
by meng＠are.berkeley.edu 17 Oct '05

17 Oct '05

Hi there- I have a question on how this difference arises when I run the following simple code on windows and Debian: >>> from numarray import * >>> a=reshape(arange(81.0), (9,9))+identity(9) >>> ca = matrixmultiply(transpose(a),a) >>> import numarray.linear_algebra as la >>> ica = la.inverse(ca) >>> ica.min() -0.30951621414374736 >>> ica.max() 0.8888918586585135 The above is the result in Debian and below is that in Windows: >>> ica.min() -0.30951621414449404 >>> ica.max() 0.88889185865875686 So, what caused this difference (both machines have python2.4 and numarray 1.3.2)? The difference happens in the 10th/11th decimal point, which should be still in the floating point precision, right? This simple difference caused big differences in my later calculation. Best, Xiangyi

1 0

Numeric precision
by Stephen Walton 14 Oct '05

14 Oct '05

Over on the scipy-dev mailing list, Travis Oliphant raised a question which is of interest to the folks over here as well. To summarize very briefly, Travis wondered whether the rule of "safest conversion" might require that an integer to float computation conversion on a 64-bit platform would require promotion of, for example, sqrt(2) to a long double (128 bits). I suggested that it would make the most sense for an M-bit integer to be converted to an N-bit real, where N>=M, on all platforms. For example, sqrt(2) would become a 32-bit real if 2 was a 32-bit integer on the plaform. I am not sure if this is a change from current Numeric and numarray practice, but wanted to give a heads-up over here. You can read the entire original thread beginning at http://www.scipy.org/mailinglists/mailman?fn=scipy-dev/2005-October/003403.…

2 1

Error involving import_libnumarray in packaged program
by Russell E. Owen 13 Oct '05

13 Oct '05

If I convert my python code to an application (Windows via py2exe or Mac via bundlebuilder) it fails with the following error: Fatal Python error: Call to API function without first calling import_libnumarray() in Src/_convmodule.c I can force *all* of numarray into the application, which avoids the issue. But that is overkill. Is there some simpler way to solve this, e.g. some an import or command in my code that will prevent this from happening? -- Russell

3 3

Re: asarray behaviour
by Chris Barker 13 Oct '05

13 Oct '05

Travis Oliphant wrote: > Chris Barker wrote: >> Travis Oliphant wrote: >>> 2) new behavior >>> >>> array (and asarray) always return an ndarray base-class object (or a >>> big-nd array if the object is already one). >> >> +1 ...I think. What is a big-nd array? > > A big-nd arrary is the parent-class of the ndarray object. It doesn't > have the sequence protocols or the buffer protocols and should not > suffer from the 32-bit limitations of those protocols. An ndarray > inherits from the bigarray. Eventually, when Python cleans up its > 32-bit limitations, the bigarray will disappear. > >>> asanyarray --- returns an nd-array or a sub-class if the object is >>> already a sub-class. >> >> What about having each subclass define it's own function-- asmatrix, >> etc? Maybe that's not general enough, but I know I use asarray because >> I know I want a NumPy Array, and nothing else. > > I guess the problem is that we are not used to coding to interfaces. I'm not sure I get this. If we were coding to an interface, then I'd write my functions to jsut expect the type I want, and be done iowth it, I wouldn't need asarray(). > I'm going to make the change suggested by the second point, just because > I think it's more explicit and will make porting scipy a lot easier. um, I'm not sure I follow...what change are you making? > The fact that multiplication could be redefined by the matrix which > still passes as an array, means that lots of code can choke on matrices. Exactly. What I like about asarray is that I can write a function that really does require a NumPy array, and let the user code pass in anything that can be turned into one, without any performance penalty if a NumPy array is passed in. However, it's very important that I know EXACTLY what type asarray will return. The same thing applies to matrixes, etc. If I write a function that manipulates a matrix, Iwant to be darn sure that's what I've got...I'd need an asmatrix() function to give it to me. > Of course, this will have negative consequences. It will make matrices > much less pervasive through function calls "automatically", but it will > be safer. People who believe their code is safe for matrices, can use > asanyarray. I can see that there may well be sone code that could take any subclass of ndarray..perhaps if it's only accessing the data, for instance, so it's nice to have that option. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception

2 2

Problem with name numeric
by Colin J. Williams 12 Oct '05

12 Oct '05

With the following sequence I had expected "scipy.base.numeric" to be a module, instead it is reported as a type. Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\>python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import scipy.base.numeric >>> dir() ['__builtins__', '__doc__', '__name__', 'scipy'] >>> scipy <module 'scipy' from 'C:\Python24\lib\site-packages\scipy\__init__.pyc'> >>> scipy.base <module 'scipy.base' from 'C:\Python24\lib\site-packages\scipy\base\__init__.pyc'> >>> scipy.base.numeric <type 'numeric_arrtype'> >>> I need to be able to access ndarray to create a subclass. Colin W.

2 1

Re: [SciPy-dev] matrix and ravel()
by Travis Oliphant 12 Oct '05

12 Oct '05

Alan G Isaac wrote: >>Travis Oliphant wrote: >> >> >>>Not really a side-effect. An intended feature. Matrices are supposed >>>to be matrices (i.e. 2-d arrays). They cannot be 1-d arrays no matter >>>how hard you try.... If you want to ravel a matrix and get a 1-d array >>>out, you are not using matrices right.... Use a 1-d array (i.e. the .A >>>attribute). >>> >>> >1. Better support for matrices is very welcome; I use them > a lot. >2. I expect ravel to return a 1-d array, not a matrix. My > background is a matrix programming language (GAUSS from > before its N-dimensional array support), and yet the > > > behavior of ravel never surprised me. > > I'm less concerned about this issue because it is so easy to get an array out of a matrix. >3. I expect a row and column vectorization functions (named > e.g., vecr and vecc) to do row and column vectorization. > I do not have a strong opinion on whether what is really > wanted is a vec function that takes an axis argument, > although this is probably right. In particular, both row > and column vectorization of matrices is common, and this > is just a choice of axis. > > These are m.T.ravel() and m.ravel() right now. To get a 1-d array it is m.A.ravel() >The same goes double for asarray: it will be a really >unwanted surprise if it does not return an array. > > I'm much more concerned about this issue. I would like to hear more people way in on what should be the behavior here. In particular, here are two proposals. 1) current behavior array (and asarray) --- returns the object itself if it is a sub-class of the array (or an array scalar). asndarray --- returns an actual nd array object 2) new behavior array (and asarray) always return an ndarray base-class object (or a big-nd array if the object is already one). asanyarray --- returns an nd-array or a sub-class if the object is already a sub-class. Please weigh in... --Travis >fwiw, >Alan Isaac > > >_______________________________________________ >Scipy-dev mailing list >Scipy-dev(a)scipy.net >http://www.scipy.net/mailman/listinfo/scipy-dev > >

2 1

New 0.4.2 beta release of scipy core
by Travis Oliphant 12 Oct '05

12 Oct '05

I made another beta release of scipy core last night. There are windows binaries for Python 2.4 and Python 2.3. If you are already a user of scipy, the new __init__ file installed for newcore will break your current installation of scipy (but the problem with linalg, fftpack, and stats is no longer there). There have been many improvements: - bug fixes (including 64-bit fixes) - threading support fixes - optimizations - more random numbers (thanks Robert Kern). - more distutils fixes (thanks Pearu Peterson). More tests are welcome. We are moving towards a release (but still need to get Masked Arrays working and all of scipy to build on top of the new scipy core before a full release). -Travis

2 1

question on differential results in windows and debian
by meng＠are.berkeley.edu 12 Oct '05

12 Oct '05

>Hi there- > >I have a matrix X, calculate cX=matrixmultiply(transpose(X), X) and >finally invert it by icX = linear_algebra.inverse(cX) >cX is kind of ill-conditioned, which caused the icX differs a lot between >windows and debian, after a tiny difference on cX across platforms. I >don't understand why a multiplication of a matrix and its transpose will >create a tiny difference across windows and debian, which could be >disastrous when inverting it. Here are the results: >In windows: > >>> cX = matrixmultiply(transpose(x), x) > >>> cX.min() >-666.96857541904126 > >>> cX.max() >1073.3945530725441 > >>> import numarray.linear_algebra as la > >>> icX = la.inverse(cX) > >>> icX.min() >-3287030277.3580685 > >>> icX.max() >0.0 > >In debian: > >>> cX = matrixmultiply(transpose(x), x) > >>> cX.min() >-666.96857541898294 > >>> cX.max() >1073.3945530726264 > >>> import numarray.linear_algebra as la > >>> icX = la.inverse(cX) > >>> icX.min() >0.0 > >>> icX.max() >84778028554.337051 > >Thanks a lot for your time! BTW, I am using Python2.4 and numarray 1.3.2 >in both machines. > >Best, >Xiangyi > >Xiangyi Meng >Department of Agricultural and Resource Economics >Room 303, Giannini Hall #3310 >University of California, Berkeley >Berkeley, CA 94720-3310 >Tel: (510) 643-4124 >Fax: (510) 643-8911 >Email: meng(a)are.berkeley.edu > Best, Xiangyi Xiangyi Meng Department of Agricultural and Resource Economics Room 303, Giannini Hall #3310 University of California, Berkeley Berkeley, CA 94720-3310 Tel: (510) 643-4124 Fax: (510) 643-8911 Email: meng(a)are.berkeley.edu

1 0

Re: [Numpy-discussion] Re: Purchasing Documentation
by Tim Churches 12 Oct '05

12 Oct '05

Paul F. Dubois wrote: > The original sources were in Framemaker. I am not positive where they > are. In any case they are copyrighted by the Regents of the University > of California. I am not a lawyer and don't know what the consequences of > that are. LLNL granted free distribution of the printed document with > the Numeric source code but I don't know what their position would be on > using their copyrighted text in a new document or on giving away the > sources. OK, thanks Paul. That may have implications for you, Travis, if you are planning to base your SciPy Core book on the existing NumPy documentation. Given the circumstance which Paul describes, perhaps the only way forward is to create a freely available addendum to the freely available, existing NumPy documentation which describes how and where SciPy Core differs or extends NumPy? Or to write entirely new documentation from scratch. > I believe that under U.S. copyright law, a person may lend their copy of > a book to anyone but not reproduce it and give them a copy. Yes, likewise here in Oz, and in most countries. Of course, the publisher of a book can impose additional restrictions to which the purchaser of the book must agree, such as 'You may not show or lend your copy of this book to your colleagues, or to anyone else.' However, who in their right mind would buy a book which was sold with such restrictions attached to it? > Given the way open source capitalism works, I would not be surprised if > someone produces a 'quick reference guide' that they give away, or > writes a book on 'Using scipy.core in biology' that they sell for $88. > Let a thousand flowers bloom. Yes, I agree entirely. Travis, you are at perfect liberty to create commercial documentation for SciPy Core, but please don't object if others try to organise to create free open source documentation as well. Tim C > Tim Churches wrote: > >> Travis Oliphant wrote: >> >>> Tim Churches wrote: >>> >>> >>>> Travis Oliphant wrote: >>>> >>>> >>>> >>>>> Tim Churches wrote: >>>>> >>>>> >>>> >>>> >>>> Travis, are you saying that this agreement only allows a single person >>>> to read the single printed copy? If so, I think you need a formally >>>> worded legal license to make that stick - certainly Australian >>>> copyright >>>> law (nor copyright law in other countries, I suspect) does not provide >>>> any support for such severe restrictions in the use of a printed >>>> document. Under copyright law, you may not make unauthorised copies >>>> of a >>>> printed document, but you can certainly lend that copy to others, or >>>> sell or give it to them, or share it as a bench manual. >>>> >>>> >>> >>> I'm just stating what I think is fair. I'm not going to try and use the >>> power of the Australian state or any other to try and enforce it. >> >> >> >> My point was that you won't get any help from the State if what you >> think is fair extends to restricting who can read a single printed copy >> of your documentation. You will need to resort to tort law to enforce >> such restrictions, and thus you really need a formal documentation >> license, if that is your aim. >> >> >>>> Many of those downloads are for "t[y|i]re-kicking" - potential users >>>> determining whether it meets their needs. If those potential users have >>>> to pay for documentation up-front, then a large proportion will go >>>> elsewhere. The rest are probably existing NumPy users upgrading (or >>>> testing the upgrade waters). The latter group may well pay for >>>> documentation, but how large is that group? For example, how many >>>> people >>>> are subscribed to the numpy-discussion mailing list? >>>> >>>> >>> >>> Don't know. But, there is enough free documentation to get started, I >>> think. Lack of documentation has hampered more people than cost of >>> existing documentation, I think. >> >> >> >> I have to disagree on this point. Granted the existing documentation for >> NumPy is not fantastic - some aspects of it are positively obscure - but >> overall it is better than good-enough, we have found, and was not a >> barrier to our decision to use NumPy. Having to cough up $50 to even >> peek at the documentation would be a far, far greater barrier to uptake >> than the occasional obscure wording of some of the function >> descriptions, IMHO. >> >> >>>> Travis, although many of us are grateful for your efforts on SciPy >>>> Core, >>>> no-one made you do it. If you wanted to earn (more) money by doing >>>> other >>>> things, you should have done them. >>>> >>> >>> I'm really thinking about the future here. If I can succesfully raise >>> money to support work on this using a simple time-delay documentation >>> encouragement, then perhaps others will do the same, and we can all have >>> more. Waiting for people to "donate" their time to develop scientific >>> python is rather slow.... I've been freely donating time for a long >>> time, and there are few who help. So, we'll try a slightly different >>> route and see where that goes. >> >> >> >> Yes, fair enough. My point was that 7 years is far too long a time >> delay. Given that SciPy Core will probably take another year to become >> rock-solid, I would happily buy several copies of your documentation at >> $50 per copy if I knew that in a year the documentation could be freely >> distributed to all God's children. But in 7 years? Nope, too long to >> wait, not interested in supporting that. >> >> >>>> I think there needs to be some community debate about this. Is there >>>> sufficient interest for people other than Travis to start with the >>>> Numeric documentation and update it as necessary to become a free SciPy >>>> Core documentation? The NumPy documentation is available in HTML format >>>> as the basis of this - perhaps the original source (LaTeX?) for the >>>> Numeric docs is also available? >>> >>> >>> I don't know why you would want to undermine my efforts in this way by >>> duplicating effort. >> >> >> >> Travis, I don't want to undermine your efforts, but you must understand >> that one of the attractions of NumPy and thus Scipy Core is that they >> are free, open source software, which implies that there exists adequate >> free, open source documentation for them, or if such free, open source >> documentation does not exist, then taht users are collectively at >> liberty to create it. That's a basic FOSS tenet. What you are are >> proposing is the creation of proprietary documentation for SciPy Core. >> No-one objects to this, but only if it is supplementary to free, open >> source "core" documentation, not instead of it. >> >> >>> Perhaps, instead you could have people donate $$ >>> instead of time to releasing the documentation. I give away copies of >>> the documentation to people who participate in the development process >>> all the time (and that comes off the total price --- though I haven't >>> advertised this as of yet). So, why don't you encourage people who >>> don't have the money to contribute to the project instead. >> >> >> >> I understand that open source software and its documentation don't just >> appear out of thin air, and I fully support the idea of funded or >> collectively-commissioned open source development. But only if the >> results of that funded or commissioned development is, in fact, free and >> open sourced. By preventing free distribution of the documentation you >> are creating for 7 years, the end product cannot even remotely be >> described as open source, thus personally I am disinterested in funding >> it. If the delay until open sourcing was just one year, then that's a >> different matter, but you do not seem to be proposing that. >> >> Hence I think the NumPy/SciPy Core user community should establish a >> SciPy Core documentation project, with the aim of updating and enhancing >> the existing NumPy documentation so that it covers the new and changed >> features of SciPy Core, and making that documentation available on teh >> same free, open source basis as NumPy/SciPy Core itself. >> >> To that end, are the original LaTeX or whatever sources for the currenet >> NumPy documenattion available somewhere (apart from the HTML and PDF >> versions, that is). >> >> Tim C >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by: >> Power Architecture Resource Center: Free content, downloads, discussions, >> and more. http://solutions.newsforge.com/ibmarch.tmpl >> _______________________________________________ >> Numpy-discussion mailing list >> Numpy-discussion(a)lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >

1 0

NumPy-Discussion October 2005