Mailman 3 March 2003 - NumPy-Discussion

Matrix element assignment
by Blair Hall 16 Mar '03

16 Mar '03

I have upgraded from NumPy 21 to NumPy 23 and am surprised to find that my matrix assignment doesn't work like it used to. For example: >>> from Matrix import * >>> m = Matrix( [[1,2],[3,4]]) >>> m[1,1] 4 >>> m[1,1] = 2 Traceback (most recent call last): File "<interactive input>", line 1, in ? File "C:\PYTHON22\lib\site-packages\Numeric\Matrix.py", line 183, in __setitem__ value = value[0,0] IndexError: invalid index >>> Apparently, I should have written >>> m[1,1] = Matrix( 2 ) Is this really indended, or am I confused? The change appears to have been introduced to NumPy in release 21.3 I note that similar code using an array does work: >>> from Numeric import * >>> m = array( [[1,2],[3,4]]) >>> m[1,1] 4 >>> m[1,1] = 3 >>> m[1,1] 3

1 0

Numerics 21 -> Numerics 22/23
by Mathieu Malaterre 12 Mar '03

12 Mar '03

Hi all I have been using some scripts with Numeric 21 that worked great but after an upgrade to (22 or 23), I can't find the equivalent. Could someone point me out some documentation, please. Here is a script that used to work with Numeric-21: #---- from Scientific.IO.NetCDF import * file = NetCDFFile('test.nc', 'w') var = file.createVariable('zspace','l',()) var.dummy = 'foo' #everything is fine ! var.bug = 8 Traceback (most recent call last): File "<stdin>", line 1, in ? ValueError: Invalid type for array var.bug = 18.0 Traceback (most recent call last): File "<stdin>", line 1, in ? ValueError: Invalid type for array var.bug = '213.7' #everything is fine ! #---- Numerics don't want real type anymore ? Thanks for any help mathieu -- Mathieu Malaterre CREATIS 28 Avenue du Doyen LEPINE B.P. Lyon-Montchat 69394 Lyon Cedex 03 http://www.creatis.insa-lyon.fr/~malaterre/

3 2

Re: [Numpy-discussion] Please help - pointer to slice
by Sebastian Haase 12 Mar '03

12 Mar '03

> > You are using Numeric not numarray , right? > > Correct. I need this stuff for production use, so I just can't quite move to > numarray yet. > > > In any case : > > My thinking goes as follows: > > I was looking for a wrapping-solution that would be most transparent to the > > people that would write the C/C++ code (or even Fortran, BTW) > > Since I already had some experience using SWIG, that's what I wanted to use > > to handle the "numarray binding". > > SWIG has a quite strong (meaning: flexible, general) way of handling data > > types. They call it "type maps": > > Once you have that in place (later more) the whole "binding" looks like > > this: (an example) > > [Snip] > > Does your stuff handle non-contiguity transparently? I couldn't quite find a > solution for that, which means that a **pointer I get can't be blindly passed > to -say- a Numerical Recipes routine which expects to treat it like a matrix. > If it's contiguous all is fine, but otherwise one needs to either make a > copy or handle the 'skipping' by hand in the column-index loops (as shown in > my examples). > > I tried to come up with a solution in C, but managed to convince myself that > it just can't be done. I was thinking of some typecasting black magic where > the pointers were recast for the pointer arithmetic to hide the column > offsets, with the proper type still being returned at dereferencing time. I > finally decided this was just impossible in C because the compiler wants to > know the pointer sizes at compile time. But maybe I'm wrong and just didn't > think about the problem hard enough. > > At any rate, I'd be interested in seeing your typemap stuff. Especially if it > either works with Numeric or it can be modified for Numeric use with > reasonably limited effort. > Fernando, Sorry, I forgot to comment on the "non-contiguity " question... First: My last post contained practically everything there is to typemapping needed when you use SWIG. ("practically" means that I have many versions of that for handling other types (like short, int, complex, ...) and fro handling 2d arrays.) Do YOU know SWIG ? "handle non-contiguity transparently" What are exactly the possible scenarios of "non-contiguous" ? If, for example having a 3d array, only going along the first axis is non-c. but along the other ones still is, I could imagine that it would not be to problematic to "forward" that scheme into the C side, that is: in a non-transparent way. The only alternative I see would be introducing some macros to translate each memory pointer. But that would also not be transparent ( since the C side would have to use these "wired" macros) and I is like to be slower, because of all the new pointer arithmetic (inside those macros) I would favor the the first scenario. Regards, Sebastian

1 0

C extensions to Numpy
by christiaan.kok＠philips.com 12 Mar '03

12 Mar '03

> Well, in case you (or others) find it useful, I'm including here a little > library I wrote for accessing general Numeric 2-d arrays (contiguous or not) > in an easy manner. > > Here's a snippet of a simple (included) example of a function to print an > integer array: > > static PyObject *idisp(PyObject *self, PyObject *args) > { > PyArrayObject *array; > int **arr; // for the data area > int i,j,cs; > > if (!PyArg_ParseTuple(args, "O!",&PyArray_Type,&array ) ) > return NULL; > > arr = imatrix_data(array,&cs); > for (i=0;i<array->dimensions[0];++i) { > for (j=0;j<cs*array->dimensions[1];j+=cs) > printf("%5d ",arr[i][j]); > printf("\n"); > } > free(arr); > > Py_INCREF(Py_None); > return Py_None; > } > > You get the **arr pointer and you can then manipulate it as a[i][j] > conveniently. The supplied example file may be enough for many to write their > Numpy C extensions without much trouble. Could someone also give an example of a small C code which has as input a numeric array and returns an other numeric array? My code either coredumps or leaks memory. wbr Christiaan Kok

1 0

Numeric 23.0 released
by Paul Dubois 11 Mar '03

11 Mar '03

Numeric-23.0.tar.gz is available for download at http://prdownloads.sourceforge.net/numpy/Numeric-23.0.tar.gz?download The Windows zip and exe are ready to upload but I am having trouble with ftp. I will try later today if another developer doesn't beat me to it. A special thanks to all those who contributed bug fixes, especially Travis Oliphant, and all of you bug reporters out there. This is my last release as Head Nummie. The developers will now choose a new victim, er, I mean leader, as set out in our charter. I will still be an active developer and user. I want to thank all of you for your help and civility over the years I have done this job. And thank you to my supervisors over the years at Lawrence Livermore National Laboratory for encouraging me to do it. -- Paul Dubois Here comes the news: Version 23 March 2003 Important notice: Two packages have been removed from optional ones: PropertiedClasses, kinds. MA has been rewritten to use standard property and will not work for ancient Pythons. (Pre 2.1, I think). Use the MA / Propertied Classes from Numeric 22 if you can't use this one. The kinds package (subject of PEP-0242) will be released as a separate package shortly. PEP-0242 was withdrawn because this facility did not seem to be worth putting in the standard library, but kinds is correct as is. [ 695200 ] Richard Everson (R.M.Everson(a)exeter.ac.uk) has donated a dotblas Package that gets Numeric to use optimized BLAS libraries for dot innerproduct, and vdot --- a conjugate vector dot product he introduced. setup.py must be altered by the user in a manner similar to the alterations to use optimized BLAS for LinearAlgebra [675777] new-style classes as objects in a sequence were not being detected correctly by array_objecttype. Corrected array_objecttype to handle them. (Oliphant) [contribution] Fernando Perez has donated a revised version of the tutorial file view.py that seems to be less likely to hang the interpreter. [68392923] dimensions+scalar -> crash (jneb) Found divergent value of MAX_DIMS in ufuncobject.c; The value was 20 there and 40 in two other places. But 40 is ludicrous, we would never have that much memory available. Changed them all to 30. [ unreported ] Changed PY_VERSION_HEX check for version 2.2 to 0x0202000 as it should have been so that true_division numeric ops can be supported [ 614808 ] Inconsistent use of tabs and spaces Fixed as suggested by Jimmy Retzlaff LinearAlgebra.py Matrix.py RNG/__init__.py RNG/Statistics.py [ 621032 ] needless work in multiarraymodule.c Fixes suggested by Greg Smith applied. Also recoded OBJECT_DotProduct to eliminate a warning error. [ 630584 ] generalized_inverse of complex array Fix suggested by Greg Smith applied. [ 652061 ] PyArray_As2D doesn't check pointer. Fix suggested by Andrea Riciputi applied. [ 655512 ] inverse_real_fft incorrect many sizes Fix given by mbriest applied. [unreported] a.real increased reference count of a and raised error when a is not complex. Fixed to apparent intended behavior of returning an array with the same data. (Oliphant) Patch for 64-bit machines applied. Appears to work ok on 32 bit but don't have the machine to test the patch.(Dubois) [ 627771 ] matrixmultiply broken for non-contig (fixed, test case added) (Greg Smith) [unreported] Fixed ArrayPrinter when NaN's show up in Float32 precision. [ 545336 ] Bug in RandomArray.randint Changed the function ranf to type double (Chuck Harris) Harris is probably right that all floats should be double in this module but it may be this has performance or storage consequences that would bite somebody. I support RNG, not this one. -- Dubois

2 1

eigenvalues fails on cygwin
by Michiel Jan Laurens de Hoon 10 Mar '03

10 Mar '03

Dear pythoneers, I noticed recently that a call to eigenvalues in numpy's LinearAlgebra does not return with Python run from Cygwin. The eigenvalues functions runs fine on Linux and Windows. The problem seems to occur in the routine int dlamc4_(integer *emin, doublereal *start, integer *base); in Src/dlapack_lite.c. To my surprise, by adding some printf statements, this routine can be made to work. So this seems to be related to optimization issues. Indeed, if I compile dlapack_lite.c without optimization, then eigenvalues works correctly without having to add any printf's. Now in the LAPACK FAQ, I found the following statement: > 1.26) Problems compiling dlamch.f? > > The routine dlamch.f (and its dependent subroutines dlamc1, dlamc2, > dlamc3, dlamc4, dlamc5) MUST be compiled without optimization. If > you downloaded the entire lapack distribution this will be taken > care of by the LAPACK/SRC/Makefile. However, if you downloaded a > specific LAPACK routine plus dependencies, you need to take care > that slamch.f (if you downloaded a single precision real or single > precision complex routine) or dlamch.f (if you downloaded a double > precision real or double precision complex routine) has been > included. When browsing through Src/dlapack_lite.c, it seems that some tricks are being used (such as calling DLAMC3) to avoid optimization to take place. So I can see two ways to resolve this bug: 1) Come up with more tricks to avoid optimization to take place when compiling dlamc4_. 2) Keep the dlamc* routines in a separate file, and let setup.py compile it without optimization. Any ideas / preferences on which way to go? Unfortunately I don't know much about optimization issues, so all suggestions are welcome. Michiel de Hoon University of Tokyo, Human Genome Center. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

1 0

Please help - pointer to slice
by Sebastian Haase 10 Mar '03

10 Mar '03

Hi, I am using SWIG to wrap some C++ function operating on numarray arrays. Everything works quite well -- up until yesterday... I have some 3d data and was thinking if the actual operation I want to do is done "slice by slice", I could just loop over Z in python and have my C function defined like this: void DoSomething(float * array2d, int nx, int ny) The python loop looks like this: arr = numarray.array(shape=(100,100,100), type=numarray.Float32) for z in range(nz): DoSomething( arr[z] ) SOMEHOW the pointer that get transfered to the C++ side is always the same as for the full (3d) array. if I try to debug this by doing: for z in range(nz): print repr(arr[z]._data) that also tells be that python / numarray thinks all slices are located at the same address, that is: every slice looks just like the full array. Please help, I don't know what to do... Sebastian Haase

4 5

numarray.fromfunction and Float32
by Sebastian Haase 08 Mar '03

08 Mar '03

Hi, I am just looking into numarray.fromfuntion. I have e.g. this: def func(x,y,z): return 1.2*(x+y+z) a = na.fromfunction(func, (nz,ny,nx)) b = a.astype(na.Float32) Is there a way to tell fromfunction to _directly_ generate a Float32 typed array. As I see it only "standard" Python type are possible now. Or: Is there a scalar conversion function like numarray.Float32( 3.141 ) ? Thanks, Sebastian Haase

2 1

cvs help?
by Paul Dubois 06 Mar '03

06 Mar '03

I tried to put the kinds module at cvsroot/numpy/Extras/kinds. I didn't succeed. We now have a cvsroot/numpy/kinds (and an Extras from which I have already removed the contents of kinds after it got installed without the 'kinds' part.) CVS just never has mapped to my brain. I don't get something about it. If another developer knows how to fix this, great. Otherwise that is where it is going to sit.

1 0

Integrating cephes with numpy
by Michiel Jan Laurens de Hoon 06 Mar '03

06 Mar '03

Hi everybody, You probably all know Travis Oliphant's SpecialFuncs (previously called cephes) module, which contains a large number of special functions. Currently it is not easy to install this module, since SpecialFuncs became a part of SciPy, which is great in itself but cannot be installed easily as other parts of SciPy depend on a host of other packages. As Scipy will get extended in the future, this problem will only get worse. Recently I have been asked several times for such a module for special functions, but I don't know where I can refer people to without qualms, especially for newbies. Since the 'cephes' part of SpecialFuncs is basically an extension of umathmodule.c in numpy, it seems that numpy would be the natural place for cephes. So I would suggest to integrate cephes with numpy, either by adding cephes' special functions to umathmodule.c or as a separate module (similar to the LinearAlgebra or RandomArray parts of numpy). In the process, we can solve some installation problems in cephes which seem to be recurring frequently (see the numpy mailing list for some examples). For the moment, I slapped together a version of the cephes module that can be installed more easily; however, I would think it is better to do this the right way and to avoid multiple variations of basically the same package floating around in cyberspace. Any opinions on this? If this seems like a good idea, I'd be happy to do some additional coding if needed to set this up, though Travis Oliphant has basically done everything already so I wouldn't think much further coding is needed. --Michiel de Hoon, University of Tokyo. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

4 3