Mailman 3 February 2001 - NumPy-Discussion

configuration ideas
by Christopher Lee 16 Feb '01

16 Feb '01

I am preparing a patch to Numeric 17.3.0 that allows for easier integration of native BLAS/Lapack libraries with Numeric's dot() function and with the LAPACK package. What I would like to know is how/where should I specify build preferences. The current situation is that I have added a config.py to the top directory. Inside this file, python variables like HAVE_CBLAS and/or HAVE_FBLAS control linking and preprocessor flags. By default, the distribution would build w/o the native libraries. Necessary info like library directories, includes and link flags would be listed as well and available to distutils for Numeric and any of it's sub-packages. How does this approach sound? -chris

2 1

RE: [Numpy-discussion] PEP 209: Multi-dimensional Arrays
by Harrison, Robert J 14 Feb '01

14 Feb '01

Paul Barrett writes: > Rob W. W. Hooft writes: > > Being a scientist, I have learned that when you multiply a > very accurate > > number with a very approximate number, your result is > going to be very > > approximate, not very accurate! It would thus be more > logical to have > > Float32*Float64 return a Float32! > > If numeric precision was all that mattered, then you would be correct. > But numeric range is also important. I would hate to take the chance > of overflowing the above multiplication because I stored the > result as > a Float32, instead of a Float64, even though the Float64 is overkill > in terms of precision. FORTRAN has made an attempt to address this > issue in FORTRAN 9X by allowing the user to indicate the range and > precision of the calculation. > A number in a floating point representation is not necessarily represented inexactly. The discussion of Barrett and Hooft is confusing the distinct concepts of precision and accuracy. Well worth reading is Kahan's scathing critcism of Java's floating-point model, at least some of which relates directly to that of Python or proposals in PEPs 209 and 228. http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf See p18 for "definitions" of precision and accuracy. There's a lot more material in the literature, on Kahan's web-site, and the following is an excellent discussion of floating point arithmetic and the IEEE standards. http://cch.loria.fr/documentation/IEEE754/ACM/goldberg.pdf With regard to the treatment of errors: Correct and detailed handling of floating-point exceptions need not impact speed, provided that a mechanism is provided to (en/dis)able each exception. Users not interested in exceptions can simply mask them. I recall relevant prior discussion including constructive comments from Tim Peters. Many modern and efficient numerical algorithms, and also effective debugging of numerical programs that use large datasets, *require* accurate and prompt identification of exceptions. Accurate meaning that the arrays, their indices, the operation, traceback and type of exception must be reported. Delayed reporting of errors is not satisfactory since operations performed in the interim may destroy valuable data, or take a very long time (esp. if many exceptions are being generated). It is probably unreasonable to ask for more than the capabilities provided by some subset of the still platform dependent optimizing compilers used to implement Python/Numpy, but I don't see why we should have much less. I would encourage the developers of PEPs 209 and 228 to submit their designs for review by a panel of professional numerical analysts (not just numerically literate programmers or scientists). While full IEEE 754 within Python or NumPy may still be just a pipe-dream (for some at least), we can at least take a step closer. Robert Robert Harrison Pacific Northwest National Laboratory Richland, Washington 99352 (509) 375-2037 robert.harrison(a)pnl.gov

1 0

problem with numeric array on window.
by Alexandre Gillet 09 Feb '01

09 Feb '01

Hi all, We are having problems when creating numeric arrays in C extensions under windows. We narrowed the problem down to a very simple example (shown below) where we simply allocate some memory, create a Numeric array using PyArrayFromDimsAndData. If we call that function as soon as we delete the returned array the python interpreter crashes as it tries to free self->data. If we do not the the OWN_DATA flag in the array it works fine but we have a memory leak. We tried this using both release and Debug versions of Python1.5.2, and Numeric 17.1.1. I have been using this mechanism under Unix for a long time and have not had this problem before ! Using the same extension and test under unix works fine. Has something changed ? Any help is welcome ... Thank you I join the two files we are using. bug.c : a C module that create an array numeric with the flags set to own_data, so when the array is garbage collected, the memory is free. ################# bug.c##################################################### #ifdef WIN32 #include <stdlib.h> #include <malloc.h> #endif #include "Python.h" #include "arrayobject.h" static PyObject* createArray(PyObject* self, PyObject* args) { int *dims; float *data; PyArrayObject *out; dims = (int *)malloc(2 * sizeof(int)); dims[0] = 500; dims[1] = 2; data = (float *)malloc(100000 * sizeof(float)); out = (PyArrayObject *)PyArray_FromDimsAndData(2, dims, PyArray_FLOAT, (char *)data); if (!out) { PyErr_SetString(PyExc_RuntimeError, "Failed to allocate memory for normals"); return NULL; } out->flags |= OWN_DATA; /*so we'll free this memory when this array will be garbage collected */ return (PyObject *)out; } static PyMethodDef bug_methods[] = { {"createArray", createArray, 1}, {NULL, NULL} /* Sentinel */ }; static char bug_documentation[] = "No Doc"; #ifdef WIN32 extern __declspec(dllexport) #endif void initbug() { PyObject *m, *d; m = Py_InitModule4("bug", bug_methods, bug_documentation, (PyObject *)NULL, PYTHON_API_VERSION); d = PyModule_GetDict(m); import_array(); if (PyErr_Occurred()) Py_FatalError("can't initialize module bug"); } ##################################################################### ###############testbug.py############################################ import bug ar = bug.createArray() del ar # fatal if ar owns the data member ##################################################################### -- ********************************** Alexandre Gillet The Scripps Research Institute, tel: (858) 784-9557 Dept. Molecular Biology, MB-5, fax: (858) 784-2860 10550 North Torrey Pines Road, email: gillet(a)scripps.edu La Jolla, CA 92037-1000, USA.

1 0

A Numerical Python BoF at Python 9
by Paul Barrett 09 Feb '01

09 Feb '01

I've been encouraged to set-up a BoF at Python 9 to discuss Numerical Python issues, specifically the design and implemenation of Numeric 2. I'd like to get a head count of those interested in attending such a BoF. So far there are 3 of us at STScI who are interested. -- Dr. Paul Barrett Space Telescope Science Institute Phone: 410-338-4475 ESS/Science Software Group FAX: 410-338-4767 Baltimore, MD 21218

3 2

Jordan's normal form of matrices
by Nils Wagner 09 Feb '01

09 Feb '01

Hi, I am looking for a program to calculate the Jordan normal form of a real or complex matrix. Thanks in advance. Nils Wagner

1 0

Is this a wheel?
by Jon Saenz 08 Feb '01

08 Feb '01

Hello, there. I needed last Saturday a function which returns the greatest/smallest element of a NumPy array. I seeked through the documentation and found the argmax/argmin functions. However, they must be called recursively to find the greatest(smallest) element of a multidimendional array. As I needed to process a BIG dataset of multidimensional arrays, I wrote a function in C which returns as a NumPy array shaped (2,) the [smallest one,biggest one] elements in an arbitrarily shaped NumPy array. It is pure C and works for multidimensional arrays. The return typecode is the same of the input array (except with complex numbers, which compare numbers through their modules). I can make this function available to general public by means of my WEB page or my starship account as a module. However, I wonder: a) Is this a wheel already invented some 40,000 years ago? May be I missed something in the manual? b) If the answer to the previous question is NO, would you (main developers) be interested in making it available as one of the "general purpose" NumPy functions? It is quite general-purpose, indeed. I have needed it five times or so in the last two years... Looking after your comments. Jon Saenz. | Tfno: +34 946012470 Depto. Fisica Aplicada II | Fax: +34 944648500 Facultad de Ciencias. \\ Universidad del Pais Vasco \\ Apdo. 644 \\ 48080 - Bilbao \\ SPAIN

5 7

New packages
by Michel Sanner 07 Feb '01

07 Feb '01

Hello, Is there any reason why Numeric never became a "New" Python package with a __init__.py ? -Michel PS: I have been adding __init__.py to my installation for a long time now and it works just fine. For those who want to be able to import directly we could extend the python path in __init__ so that after an import Numeric all .so would be directly loadable -- ----------------------------------------------------------------------- >>>>>>>>>> AREA CODE CHANGE <<<<<<<<< we are now 858 !!!!!!! Michel F. Sanner Ph.D. The Scripps Research Institute Assistant Professor Department of Molecular Biology 10550 North Torrey Pines Road Tel. (858) 784-2341 La Jolla, CA 92037 Fax. (858) 784-2860 sanner(a)scripps.edu http://www.scripps.edu/sanner -----------------------------------------------------------------------

3 2

Re: [Numpy-discussion] more general LAPACK support for NumPy
by hoel＠germanlloyd.org 06 Feb '01

06 Feb '01

"Paul F. Dubois" <paul(a)pfdubois.com> writes: > Two notes in regard to this thread: > 1. I am in progress making Pyfort support Digital Visual Fortran on > Windows. GREAT > 2. Pyfort does have a facility for automatic allocation of work > space, at least in the case that the size can be computed using > ordinary arithmetic from the sizes of other arguments or other > integer arguments. I know of that, but the optimal workspace size for LAPACK routines is for optimal efficiency. The size can be returned by the routine or by calling the FORTRAN function ILAENV. It would be great, if workspacesize could be made depending on the result of functions. Thanks Berthold -- email: hoel(a)GermanLloyd.org ) tel. : +49 (40) 3 61 49 - 73 74 ( C[_] These opinions might be mine, but never those of my employer.

1 0

more general LAPACK support for NumPy
by hoel＠germanlloyd.org 06 Feb '01

06 Feb '01

Hello, >From time to time we need an additional linear algebra routine to be avaible in Python. I find myself wrapping these functions then. As these are FORTRAN routines, doing this for Python version on Solaris (Sun CC/Sun F77), linux (gcc/g77) and Windows (VC++/Digital VF) becomes nontrivial. Neither f2py nor pyfort provide Win support and I doubt that automatic generation is usefull for many LAPACK routines, especially those that need workspace, because usually we want LWORK to be the optimal size. So my Question is, are there other users wrapping LAPACK routines for NumPy. If so, how are you doing it. For the C/FORTRAN wrapping I somehow like the approach used for cfortran.h (see http://www-zeus.desy.de/~burow/cfortran/index.html), but I'm afraid, the license is not acceptable. Is anyone aware of a C version of LAPACK besides the f2c version on netlib. I do like the approach used in the ATLAS clapack part, but there are only a very few LAPACK routines handeled there. If there is greater need for additional LAPACK routines for NumPy, should we bundle the efforts in (a) developing guidelines for how to write Python wrappers for LAPACK routines and (b) collecting routines provided by different users to provide a hopefully growing support for LAPACK in Python. Greetings Berthold -- email: hoel(a)GermanLloyd.org ) tel. : +49 (40) 3 61 49 - 73 74 ( C[_] These opinions might be mine, but never those of my employer.

3 2

Re: [Numpy-discussion] more general LAPACK support for NumPy
by hoel＠germanlloyd.org 06 Feb '01

06 Feb '01

OK, I got, compiled and installed PyLapack.tar.gz. It looks quite complete, for LAPACK 1 or 2, but, of course those routines I need are new in LAPACK 3 :-(. It seemes, Doug Heisterkamp used some kind of a script or program to generate the wrapper. Is there anyone who knows, whether this program is avaible anywhere, so it could be extended for LAPACK 3 and Win? Thanks Berthold -- email: hoel(a)GermanLloyd.org ) tel. : +49 (40) 3 61 49 - 73 74 ( C[_] These opinions might be mine, but never those of my employer.

1 0