Mailman 3 March 2002 - NumPy-Discussion

How the a**T trick works
by Paul F Dubois March 8, 2002

March 8, 2002

I confess I admire the a**T suggestion for a notation for transpose(a). The question was raised as to whether this really works and an empirical proof was offered that it does. Here is how it works. In a**T the first thing tried is to ask a.__pow__ if it can manage with T as an argument. The array says, hmm, T is an instance of a class that I never heard of, and it doesn't have an __array__ attribute to call in order to make it an array. I pass. Then T's class' __rpow__ is given a turn, and asked if it can do the job with a as an argument, which it can. This is how 2**a works, or 2 - a, too. The int type disavows any knowledge of what to do, so the other operand gets a chance to save the day. So this is a "trick" we use every day.

1 0

Re: [Numpy-discussion] big picture? One proposal
by Pearu Peterson March 8, 2002

March 8, 2002

On Fri, 8 Mar 2002, Konrad Hinsen wrote: <snip> > Unfortunately, I have the impression that there are two schools of > thought in collision here (and not just when it comes to programming). > There is the "mathematical" school that defines matrices and arrays > as abstract entities with certain properties and associated operations. > And there is the "engineering" school that sees arrays as a convenient > data structure to express certain operations, of which "matrix operations" > are a subset. I see arrays as a convenient data structure (being implemented in computer programs) to hold matrices (being members of a mathematical concept). I guess that my views are narrow-minded (but willing to widen it) regarding to consider arrays as a mathematical concept too. Just in mathematics I never (need to) use arrays in that way (my fields are mathematical analysis, integrable systems, and not computer science nor engineering). So, I also belong to the school of "mathematics", but may be into a different one. <snip> > That's another point where I disagree. I use Python for many different > uses, numerics is only one of them (though the most important one). > Uniformity of style is an important value for me. Me too. Just I am not too crazy about the constant style but more of if something can be accomplished efficiently. To be honest, I don't like programming in Python because it has a nice style, but because I can accomplish a lot with it in a very efficient way (and not only by using efficient algorithms). Writing, for example, Numeric.transpose(a) instead of a**T, a.T, a`, or whatever just reduces this efficiency. I also realize and respect that for computer scientists (that I presume the developers of Python are) it is crucial to have consistent style for their reasons. Sometimes this style makes some site-specific simple tasks too verbose to follow. > Moreover, I claim that Python *does* provide a good solution, it is > merely a very different one. So, what is it? <snip> > Does that work? I'd expect that a**T would first call .__pow__(T) > which quite probably crashes... (Not that it matters to me, I find > this almost as abusive as the matrix attributes.) Yes, it works: >>> from Numeric import * >>> class T: __rpow__ = lambda s,o: transpose(o) ... >>> print array([[1,2],[3,4]]) ** T() [[1 3] [2 4]] And I don't understand why it is abusive (because it is a different approach?). It's just an idea. Pearu

3 2

Re: [Numpy-discussion] big picture? One proposal
by Konrad Hinsen March 8, 2002

March 8, 2002

> situation. My general point is that a _good_ solution is simple, > if a solution is not simple, then it is probably a bad solution. Agreed. So we just need to agree on what is "simple". > I find separating array and matrix instances (in a sense of raising > exception when doing <matrix> <op> <array>) not a very simple solution: > New concepts are introduced that actually do not solve the simplicity > problem of representing matrix operations. As I see it, they I disagree there, separating the concepts of matrix and array *does* solve the simplicity problem in my opinion. Matrices use operators for common matrix operations, and arrays use the same operators for common array operations. > only introduce restrictions and the main assumption behind the rationale > is that "users are dumb and they don't know what is best for them". No, not at all. On the contrary, the rationale is "users are smart and know that arrays and matrices are different" ;-) Unfortunately, I have the impression that there are two schools of thought in collision here (and not just when it comes to programming). There is the "mathematical" school that defines matrices and arrays as abstract entities with certain properties and associated operations. And there is the "engineering" school that sees arrays as a convenient data structure to express certain operations, of which "matrix operations" are a subset. As a student, I had a friend who studied mechanical engineering, and his math exercises made me go mad more than once. When I read "...the vector of the masses...", I just had to scream ;-) Many engineering textbooks have the same effect on me. Now obviously I belong to the "mathematical" school, but I don't expect to convert everyone else to it. So my arguments will remain pythonic and pragmatic: the "mathematical" approach solves the problem without asking for new operators, and thus has a better chance of getting realized. > This is how I interpret the raised exception as behind the scenes matrix > and array are the same (in the sense of data representation). But data representation and data semantics are two different things. Readibility of code depends on semantics, not on internal representations or even implementation. Using the same representation merely implies that conversion should be efficient, but not necessarily implicit. > The main objection to this proposal seems to be that it deviates from a > good pythonic style (ie don't mess with attributes in this way). > I'd say that if python does not provide a good solution to our problem, > then we are entitled to deviate from a general style. After all, in doing That's another point where I disagree. I use Python for many different uses, numerics is only one of them (though the most important one). Uniformity of style is an important value for me. Moreover, I claim that Python *does* provide a good solution, it is merely a very different one. > numerics the efficiency issue has a rather high weight. And a generally > good style of Python cannot always support that. Computational efficiency is not the issue here. If that's all you want, call a BLAS routine for matrix multiplication with two array arguments - doable today, without any modification of whatsoever. Even Fortran programmers do that, instead of suggesting that Fortran 2002 should add a "multiply-by-calling-BLAS" operator. > define > > T = TransposeOp() > H = TransposeOp(conjugate=1) Does that work? I'd expect that a**T would first call .__pow__(T) which quite probably crashes... (Not that it matters to me, I find this almost as abusive as the matrix attributes.) Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen(a)cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------

1 0

Adding a flag to allow integer array access and masking
by Travis Oliphant March 7, 2002

March 7, 2002

I have not heard any feedback back on my proposal to add a final object to the extended slice syntax to current Numeric to allow for unambiguous index and mask-array access. As a modification to the proposal, suppose we just check to see if the last argument (of at least two) is a 0d array of type signed byte (currently this is illegal and will raise an error). This number would be a flag indicating how to interpret the previous objects. Of course these numbers would be hidden from the user who would write: a[index_array, _I] = <values> b = a[index_array, _I] or a[mask_array, _M] = <values> b = a[mask_array, _M] where _M is a 0d signed byte array indicating that the mask_array should be interpreted as a mask while _I is a 0d signed byte array indicating that the index_array should be interpreted as a integers into the flattened version of a. Other indexing schemes could be envisioned as well a[a1,a2,a3,_X] could be the cross product of the integer arrays a1, a2, and a3 for example. or a[a1, a2, a3, _Z] could select elements from a by "zipping" the sequences a1, a2, and a3 together to form a list of tuples to grab from a. Comments?

4 5

Numerical Python and LAPACK on 64-bit machines
by Roman Geus March 7, 2002

March 7, 2002

Dear Numerical Python user and developers I ran into the following problem: The python application I'm developing uses Numerical Python and other C modules that call LAPACK. My application runs well on 32bit architectures: When I tried to run the application on a HP-UX 64bit machine the application produced bus errors. After a long debugging session I found out that Fortran integers are still 32bit wide on this machine. Therefore also the HP LAPACK library has to called using 32bit integers. Numerical Python however codes Fortran integers as C 'long int' variables, which are 64bit wide on this machine. To make my application run on the HP-UX 64bit machine, I had to change all 'long int' to 'int' variables in Src/lapack_litemodule.c, which is a rather painful hack (see end of message for an example). My question is: Should Fortran integers not be coded as 'int' instead of 'long int' in Numerical Python? This way, it would still work on all 32 bit machines and also on the 64-bit machines I know. Would this work on all 64-bit machines? Thanks for your comments/help. -- Roman Geus E.g. the lapack_lite_dgetrf() function now looks like this: static PyObject *lapack_lite_dgetrf(PyObject *self, PyObject *args) { int lapack_lite_status__; int m; int n; PyObject *a; int lda; PyObject *ipiv; int info; int i; int *ipiv_int; int ipiv_len; TRY(PyArg_ParseTuple(args,"iiOiOi",&m,&n,&a,&lda,&ipiv,&info)); TRY(lapack_lite_CheckObject(a,PyArray_DOUBLE,"a","PyArray_DOUBLE","dgetrf")); TRY(lapack_lite_CheckObject(ipiv,PyArray_LONG,"ipiv","PyArray_LONG","dgetrf")); ipiv_len = m < n ? m : n; ipiv_int = (int *)malloc(ipiv_len * sizeof(int)); assert(ipiv_int); for (i = 0; i < ipiv_len; i ++) ipiv_int[i] = LDATA(ipiv)[i]; #if defined(NO_APPEND_FORTRAN) lapack_lite_status__ = dgetrf(&m,&n,DDATA(a),&lda,ipiv_int,&info); #else lapack_lite_status__ = dgetrf_(&m,&n,DDATA(a),&lda,ipiv_int,&info); #endif for (i = 0; i < ipiv_len; i ++) LDATA(ipiv)[i] = ipiv_int[i]; free(ipiv); return Py_BuildValue("{s:l,s:l,s:l,s:l,s:l}","dgetrf_",(long)lapack_lite_status__,"m",(long)m,"n",(long)n,"lda",(long)lda,"info",(long)info); }

5 6

Re: Numpy-discussion digest, Vol 1 #406 - 13 msgs
by Greg Kochanski March 7, 2002

March 7, 2002

> Are you opposed to anything going inside the [] operator to help indicate > how the objects inside should be interpreted I'm opposed to overloading any operator so that it becomes confusing. Confusion is generated when a look at a small part of code (a line or two) does not tell you what the code is doing. This is a real problem for languages like Python and Perl, where any variable can contain any data, IF the code behaves in different ways, dependent on the data type. "Different" is defined by the user, and it roughly translates into the amount of coffee you have to drink to fix the code, if the input data changes it's type. > > Besides, you know some idiot is going to eventually write > > code that looks like this: > > > > def access(a, b, x): > > return a[b, x] # I think that a must be a 2-D array... > > > > # 1000 lines later... > > access(a, _I) # Whoops all my assumptions were wrong... > > I have no idea, what your concern is here. This would result in an error > currently and under the scheme I suggested. Then, perhaps you should explain again. I assumed you were proposing a magic second argument to [] that would cause the first argument to be interpreted differently. If so, I think that's a bad idea from a human-interface point of view, because (a) To a person who only uses Numpy occasionally, it is not obvious that an argument is "magic". That makes the code less readable. (b) It is possible to write code where one can pass in the "magic" value in a variable, and no simple inspection of the code will tell if it is magic or not. Using an explicit function or method call fixes (a) by telling the naive user that "this is not normal array access here." It also fixes (b) by making it more obvious that fancy stuff is going on.

4 4

OverflowError: math range error
by Nils Wagner March 7, 2002

March 7, 2002

Hi, >gdb /usr/bin/python GNU gdb 20010316 Copyright 2001 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-suse-linux"...(no debugging symbols found)... (gdb) run fredholm.py Starting program: /usr/bin/python fredholm.py (no debugging symbols found)...[New Thread 1024 (LWP 13596)] (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)... (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)... (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)... Traceback (most recent call last): File "fredholm.py", line 2, in ? from LinearAlgebra import * File "/usr/lib/python2.1/site-packages/Numeric/LinearAlgebra.py", line 10, in ? import MLab File "/usr/lib/python2.1/site-packages/Numeric/MLab.py", line 17, in ? import RandomArray File "/usr/lib/python2.1/site-packages/Numeric/RandomArray.py", line 30, in ? seed() File "/usr/lib/python2.1/site-packages/Numeric/RandomArray.py", line 24, in seed ndigits = int(math.log10(t)) OverflowError: math range error (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)... Program exited with code 01. (gdb) Any idea ? Thanks in advance. Nils

1 0

Final prep for 21.0
by Paul F Dubois March 6, 2002

March 6, 2002

I have committed to CVS that which I expect to become 21.0. The final set of changes is a revision to the way we use distutils to make RPMs. I am unqualified to test these but the submitter (Vermeulen) did. Please update your CVS and test this version. Developers, please make no further commits until I make the release. I will make the release this weekend unless I receive advice to the contrary from testers.

1 0

Re: [Numpy-discussion] Adding a flag to allow integer array access and masking
by Greg Kochanski March 6, 2002

March 6, 2002

Please, no. There are many short names that you could use that would avoid overloading the [] operator. Especially in Python, where one cannot trivially decide the type of a variable, the behavior should change as little as possible as the type of each variable changes. Here, the indexing operation changes completely if you change the last index from an int to an array. That means you have to execute the code to understand it -- one can't just look and assume from local syntax. Besides, you know some idiot is going to eventually write code that looks like this: def access(a, b, x): return a[b, x] # I think that a must be a 2-D array... # 1000 lines later... access(a, _I) # Whoops all my assumptions were wrong... > From: Travis Oliphant <oliphant.travis(a)ieee.org> > Subject: [Numpy-discussion] Adding a flag to allow integer array access and masking > > > I have not heard any feedback back on my proposal to add a final object to > the extended slice syntax to current Numeric to allow for unambiguous index > and mask-array access. > ...hidden from the user who would write: > > a[index_array, _I] = <values> > b = a[index_array, _I] > > or > > a[mask_array, _M] = <values> > b = a[mask_array, _M] > > where _M is a 0d signed byte array indicating that the mask_array should be > interpreted as a mask while _I is a 0d signed byte array indicating that the > index_array should be interpreted as a integers into the flattened version of > a. > > Other indexing schemes could be envisioned as well...

2 1

problem with integer type array object
by Jin Woo Park March 6, 2002

March 6, 2002

I was working on an external module build in C where a function returns a 2-D integer array. However, when I imported module, I found a strange behavior concerning the type of the element of the array. This is what basically happens: static PyObject* foo(PyObject* arg, PyObject* args) { PyObject* a; int dims[2]={2,2}; a = PyArray_FromDims(2,dims,PyArray_INT); return a; } in python, >>> a=foo() >>> type(a[0,0]) <type 'array'> I expected the type 'int'. If you make a Numpy array in python, then you get the expected type of 'int'. >>> b=array([[0,0],[0,0]]) >>> type(b[0,0]) <type 'int'> Is this somehow intended? Thanks for any info, -- +----------------------------------------+ | Jin Woo Park (jwpark(a)aeroguy.snu.ac.kr)| | Research Assistant,Dept.Aerospace Eng. | | Seoul National University, Korea | +----------------------------------------+

1 1