Mailman 3 July 2001 - NumPy-Discussion

simple array help?
by Pete Shinners Sept. 17, 2001

Sept. 17, 2001

i have a simple array with floating values >>> print a array([ 0. , 0.2, 0.4, 0.6, 0.8]) what i need to do is change each index into another dimension with 3 indices. hmm, kind of confusing, an example will explain better >>> b = array(zip(a,a,a)) >>> print b array([[ 0. , 0. , 0. ], [ 0.2, 0.2, 0.2], [ 0.4, 0.4, 0.4], [ 0.6, 0.6, 0.6], [ 0.8, 0.8, 0.8]]) ok, this does the job, but once i start using large arrays, the back-n-forth conversion between arrays and python lists is costing me quite a bit. is there a way i can reshape the array this way without the call to python's 'zip'? (no, 'map' doesn't count either) i've tried much fooling with NewAxis, resize, and friends. but either i haven't stumbled upon the correct combination or i'm not trying the correct tools. i'd like to keep this as quick as possible, so hopefully it can be done without anything too elaborate. thanks (doh, sorry for the previous distutils related post... hit the wrong mailing list)

6 19

Help: need info on Fortran/C mixing on Windows
by Paul F. Dubois July 31, 2001

July 31, 2001

Any fellow nummies and Pythonistas out there with experience using Digital Fortran on Windows? I'm trying to get Pyfort to work with Visual Fortran / Visual C++ on Windows. I am not usually a Windows developer so I do not know how to do things like look at an object file to see what names are in it (as I could do with nm on Unix). I have discovered that by including c$DEC ATTRIBUTES C:: foo in my Fortran, where foo is the name of my Fortran function, that my missing external for foo disappears. This is not really ok as I want to be able to link to unmodified Fortran files that have simply been compiled into a foostuff.lib file. I need information such as: How to get the names to match up, or what name to generate in my C to correspond to the Fortran; Which library to load with: I've tried dfor.lib and dformd.lib, in the context of running a python extension link through distutils, and both complain vociferously about library conflicts. If this was my working life I could afford to put in the time to figure all this out, but this is pro bono publico and I would appreciate your tolerance of my asking for help. Even just your Makefile or setup.py from a successful project of this type would be a big help. Thanks, Paul Dubois

1 0

RE: [Numpy-developers] Error behavior.
by Paul F. Dubois July 31, 2001

July 31, 2001

Numpys, This is Travis' original message to the developers list, discussing his ideas for dealing with floating point errors. When I answered him a minute ago I inadvertently posted to this list instead. But, it is a good idea to have everyone's thoughts. I only ask that you keep the noise down and not have this degenerate into the numerous IEEE discussions we have had before. Paul P.S. I still am looking for help on solving the Numeric bug on Alphas. -----Original Message----- From: numpy-developers-admin(a)lists.sourceforge.net [mailto:numpy-developers-admin@lists.sourceforge.net]On Behalf Of Travis Oliphant Sent: Monday, July 23, 2001 12:46 PM To: numpy-developers(a)lists.sourceforge.net Subject: [Numpy-developers] Error behavior. Many have noticed that with one of the newer Python releases, domain and range errors on one of the computations in an element-by-element umath function now always raise Python errors. It used to be that one could say: >>> x = arange(1,10) >>> y = arange(0,9) >>> print x/y and get an output with reasonable values in positions 1..9 of the output array but get an inf in the first position (how inf printed was platform specific). Many, including me, prefer returning an array with some undefined entries rather than raising an error and ruining the rest of the perfectly valid computations. Proposed solutions: 1) Make a new module (bring back the old fastumathmodule), which does not set the "check" flag when the ufunc is created so that error are not raised --- quick and fast fix. 2) Use the warnings framework so that rather than raise a Python error, a warning is issued. This allows the user to do what they want with the warning (from raise an error to ignore it). --- this feels like a better solution. Please let me know your position on this, so we can take action if warranted. Sincerely, Travis Oliphant _______________________________________________ Numpy-developers mailing list Numpy-developers(a)lists.sourceforge.net http://lists.sourceforge.net/lists/listinfo/numpy-developers

1 0

RE: [Numpy-developers] Error behavior. (2)
by Paul F. Dubois July 31, 2001

July 31, 2001

I understand why some people might want the behavior (return with inf or NaN etc in arrays rather than get an exception). Let's ask some basic questions. Having done such an operation, how do I tell if I have any errant values in the result? For example, what can I compare x[i] to, in order to know if it is invalid? We have had several requests for full support of IEEE but nobody knows how to do it portably, as far as I know. I resist bringing back fastumath because I wonder if it really would work portably, I don't know the answers to the simpleest questions like the above, and if it would I know that it made people reluctant to fix bugs or add new functions because they would have to do it twice. Adding a new function already requires something like five different places to fix. Using optional package MA, a user *can* achieve the goal you desire. For example, if x and y are numpy arrays, and I want to divide them without possibility of error, I can do: z = MA.masked_array(x) / MA.masked_array(y) if MA.getmask(z) is not None: print "Your operation failed in one or more components" The masked_array constructor does not copy its argument. (MAs mix with Numerics and scalars, so one of the masked_array calls above could be omitted.) >From a Windows session: >>> print x/y [ 1.#INF0000e+000, 2.00000000e+000, 1.50000000e+000, 1.33333333e+000,] >>> print MA.masked_array(x)/y [-- ,2.0 ,1.5 ,1.33333333333 ,] >>> >>> print MA.sqrt(y-2.) [-- ,-- ,0.0 ,1.0 ,] I realize this isn't exactly what you had in mind, and it is more expensive than just letting it rip, but it is reliable and portable. With the infs and Nans, I'm not sure what you can do with the answer other than print it. It does place the burden on the programmer to distinguish cases where he wants to survive a divide by zero from those where he does not, but in my mind that is a good thing. Paul

1 0

Re: [Numpy-discussion] Troubles with arrays, proposal for Solution
by Phil Austin July 30, 2001

July 30, 2001

Vanroose Wim writes: > Dear Numerical Python Users, > > It is now clear to me, after the comments of Travis Oliphant, > Paul Dubois and Phil Austin, that in EVERY C-file that uses NumPy > extensions, the "import_array" statement must be present. Actually, what I was trying to say (in a message that didn't make it to the list) is the opposite -- import_array must be present in only one file. So below I would do something like: (my additions marked by PA>) > > ///////////////////////// > //file: arraytest.h > ////////////////////// > > #include "Python.h" PA> #define PY_ARRAY_UNIQUE_SYMBOL paArray > #include "arrayobject.h" > > double *myTest(void); > > ////////////////////// > // file: arraytest.c > //////////////////// > PA> #define NO_IMPORT_ARRAY > #include "arraytest.h" > > double * myTest(void ){ > double *result; > ... > > return result; > } > > //////////////////// > //file: test.c > //////////////// > #include "arraytest.h" > > static PyObject *function(PyObject *self, PyObject *args){ > > ... > int dimensions[2]; > > dimensions[0] = N; > dimensions[1] = N; > > PyArrayObject *result ; > result = (PyArrayObject *)PyArray_FromDims(2,dimensions,PyArray_DOUBLE); > > double *data; > data = myTest(); > memcpy(result->data,data,N*N*sizeof(double)); > return PyArray_Return(result); > } > > static PyMethodDef testMethods[] = { > {"test",function,1}, > {NULL,NULL} > }; > > extern "C" { > void inittest(){ > PyObject *m,*d; > m = Py_InitModule("test", testMethods); > import_array(); > d = PyModule_GetDict(m); > } > You should see the following when you run /lib/cpp -E on these files: in arraytest.c #define NO_IMPORT_ARRAY #include "arraytest.h" will produce a line like: extern void **paArray; but in test.c #include "arraytest.h" will produce: void **paArray; and import_array(); will expand to { PyObject *numpy = PyImport_ImportModule("_numpy"); if (numpy != 0) { PyObject *module_dict = PyModule_GetDict(numpy); PyObject *c_api_object = PyDict_GetItemString(module_dict, "_ARRAY_API"); if (((c_api_object)->ob_type == &PyCObject_Type)) { paArray = (void **)PyCObject_AsVoidPtr(c_api_object); } } }; The result is that there is a single global function pointer array called paArray that has its space reserved in test.c, and is referenced externally in arraytest.c. This allows you to use any numeric api call in either file. If you compile the module to, say, test.so on unix you should see something like this when you do a nm -D: ~phil% nm -D test.so | grep paArray 000a3d00 B paArray i.e. one instance of paArray in the data segment of the so file. Scanning the output of /lib/cpp -E output is a good way to get a feel for this (on unix anyway). Regards, Phil

1 0

Troubles with arrays, proposal for Solution
by Vanroose Wim July 30, 2001

July 30, 2001

Dear Numerical Python Users, It is now clear to me, after the comments of Travis Oliphant, Paul Dubois and Phil Austin, that in EVERY C-file that uses NumPy extensions, the "import_array" statement must be present. However, after some try-outs with parts my code, it seems not a good idea to allow a diffusion of the "import_array" statement and "PyArrayObject" throughout the "C" code. The code becomes unclear by the mixing of Python concepts and C concepts. Therefore it is, in my opinion, a good rule to limit the use of NumPy to the interface between python and "C" code; i.e. the function where the python parameters are read and the PyArrayObject is created. Do more experienced NumPy users agree??? The code below illustrates the limited use of NumPy parts. Wim Vanroose ///////////////////////// //file: arraytest.h ////////////////////// #include "Python.h" #include "arrayobject.h" double *myTest(void); ////////////////////// // file: arraytest.c //////////////////// #include "arraytest.h" double * myTest(void ){ double *result; ... return result; } //////////////////// //file: test.c //////////////// #include "arraytest.h" static PyObject *function(PyObject *self, PyObject *args){ ... int dimensions[2]; dimensions[0] = N; dimensions[1] = N; PyArrayObject *result ; result = (PyArrayObject *)PyArray_FromDims(2,dimensions,PyArray_DOUBLE); double *data; data = myTest(); memcpy(result->data,data,N*N*sizeof(double)); return PyArray_Return(result); } static PyMethodDef testMethods[] = { {"test",function,1}, {NULL,NULL} }; extern "C" { void inittest(){ PyObject *m,*d; m = Py_InitModule("test", testMethods); import_array(); d = PyModule_GetDict(m); }

1 0

Re: Numeric.transpose (incorrect documentation)
by Tim Hochberg July 27, 2001

July 27, 2001

"Curtis Jensen" <cjensen(a)bioeng.ucsd.edu> wrote in message news:3B6076D4.68B6840C@bioeng.ucsd.edu... > the documenation for the "transpose" function for the "Numeric" module > seems to be incoorect, or at least missleading. The correct place for this is probably the numpy-discussion list, so I'm cc'ing it there. Perhaps the Paul Dubois will see it and have some ideas for clearing of the docs. > It says that transpose is suppose to return a "new" array. In fact it > does not. It returns a pointer to the same array, only with transposed > indicies. It's been a while since I looked at the NumPy docs, so I don't recall what conventions they use here. However, transpose does return a new array, it just points to the same data as the original. If a=transpose(b), "a is b" is false and "a.shape == b.shape" is false, so clearly they are different objects (i.e., different arrays). You know most of this, as you demonstrate down below, I just wanted to make the point that there is a difference between a new array and new data. Note that nearly every operation that can return a reference rather than a copy does so. Even slicing, if b=a[3:5], then 'b' holds a reference to the same data as 'a'. Some functions go so far to return a reference when they can, but otherwise copy. See ravel for example -- it copies the data if its argument is not contiguous, otherwise it uses a reference. Now _that_ can be confusing! Useful though. -tim > consider the example: > > Python 1.5.2 (#4, Sep 5 2000, 10:29:12) [C] on irix646 > Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam > >>> from Numeric import * > >>> foo = zeros([5,10]) > >>> bar = transpose( foo ) > >>> foo > array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]) > >>> bar > array([[0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0]]) > >>> foo[0,2] = 1 > >>> foo > array([[0, 0, 1, 0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], > [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]) > >>> bar > array([[0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [1, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0], > [0, 0, 0, 0, 0]]) > >>> > > > if bar was truely a new array, then changes to foo would not affect > bar. In the example above, it does. This way actualy works better for > my purposes, however, the documentation is missleading. > > -- > Curtis Jensen > cjensen(a)bioeng.ucsd.edu > http://www-bioeng.ucsd.edu/~cjensen/ > FAX (425) 740-1451

2 2

Numpy install problems
by Michael D. Risser July 27, 2001

July 27, 2001

When attempting to install Numeric 17.0 with Python 1.5.2 and Distutils 1.0 on Red Hat 7.0, I get the following error after issuing the command 'python setup.py install' Traceback (innermost last): File "setup.py", line 15, in ? vs = map(string.atoi, v.groups()) AttributeError: 'None' object has no attribute 'groups' This error occurs regardless of whether or not I am root. I have also tried uninstalling the Red Hat Python packages and installing all from source, but I get the same error. Any help would be appreciated. Thanks.

4 5

Troubles with arrays
by Wim Vanroose July 27, 2001

July 27, 2001

Dear Numerical Python Users, I have a small program that produces array's that I want to import in Python. The code is organised in three files. 1) arraytest.h, 2) arraytest.c 3) testmodule.c. and the code is shown below. The function myTest() produces the array and is called from the test module. However, the program does not work! it crashes. In my opinion, I do not have yet enough insight in Numerical Python. Is there an essential part that I do not understand. Remarkable is that the pure Python equivalent of the program does not crash. Any ideas? Wim Vanroose ///////////////////////// //file: arraytest.h ////////////////////// #include "Python.h" #include "arrayobject.h" PyObject *myTest(void); ////////////////////// // file: arraytest.c //////////////////// #include "arraytest.h" PyObject * myTest(void ){ PyArrayObject *result; double *datainput; int dimensions[1]; int M=10; int MM; dimensions[0]=M; result = (PyArrayObject *)PyArray_FromDims(1,dimensions,PyArray_DOUBLE); datainput =(double *)result->data; for(MM=0;MM < M ; MM++){ datainput[MM] = MM*0.1; } return PyArray_Return(result); } //////////////////// //file: test.c CRASHES!!!! //////////////// #include "arraytest.h" static PyObject *function(PyObject *self, PyObject *args){ return myTest(); } static PyMethodDef testMethods[] = { {"test",function,1}, {NULL,NULL} }; extern "C" { void inittest(){ PyObject *m,*d; m = Py_InitModule("test", testMethods); import_array(); d = PyModule_GetDict(m); } %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% The Python Equivalent: DOES NOT CRASH %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ///////////// // file: simpletest.h ///////////////// #include "simpletest.h" PyObject * myTest(void); //////////////////// // file: test.c ////////////////////// #include "simpletest.h" PyObject * myTest(void ){ return Py_BuildValue("i",123); } //////////////// //file: test.c ////////////////// #include "simpletest.h" static PyObject *function(PyObject *self, PyObject *args){ return myTest(); } static PyMethodDef testMethods[] = { {"test",function,1}, {NULL,NULL} }; extern "C" { void inittest(){ PyObject *m,*d; m = Py_InitModule("test", testMethods); d = PyModule_GetDict(m); }

3 2

vec-operator
by Nils Wagner July 26, 2001

July 26, 2001

Hi, A matrix operation is that of stacking the columns of a matrix one under the other to form a single column. This operation is called "vec" or "cs" (c)olumn (s)tring Example A is a m * n matrix vec(A) = reshape(transpose(A),(m*n,1)) How can I copy the result of vec(A) into the i-th column of another matrix called B ? Thanks in advance. Nils

2 1