Mailman 3 August 2001 - NumPy-Discussion

For the next round on the manual
by Eric Hagemann 23 Aug '01

23 Aug '01

Type-o in the manual Check the definition for the 'convolve' function. Manual tells of mode=0 as default when code and experiment point to mode=2 being default No biggie -- but it might be nice to correct if anybody is editing the manual any time soon Cheers Eric

3 2

patch to speed up LinearAlgebra module
by Jeff Whitaker 23 Aug '01

23 Aug '01

I've uploaded a patch to LinearAlgebra.py and lapack_litemodule.c that speeds up some computations (svd, linear least-squares, and hermitian eigenanalysis) by 3-5 times when linking with external (preferably optimized) blas and lapack libs. The patch replaces calls to dgesvd, dgelss and dsyev with the newer, faster lapack routines dgesdd, dgelsd and dsyevd. Here are my timing results before and after (G4 350mhz powermac, optimized ATLAS blas lib and lapack 3.0 from netlib.org): Before: svd of 1000X1000 matrix takes 609.19 generalized inverse of 1000X1000 matrix takes 744.36 linear least-squares solution of 1000X1000 matrix takes 451.68 eigenvectors of 1000X1000 symmetric matrix takes 210.08 After: svd of 1000X1000 matrix takes 142.55 generalized inverse of 1000X1000 matrix takes 237.08 linear least-squares solution of 1000X1000 matrix takes 81.91 eigenvectors of 1000X1000 symmetric matrix takes 56.23 Note that since these newer lapack routines are not in lapack_lite, you must link external blas and lapack3 libs. -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/CDC R/CDC1 Email : jsw(a)cdc.noaa.gov 325 Broadway Web : www.cdc.noaa.gov/~jsw Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124

1 0

Matlab_NumPy_Array?
by Karshi Hasanov 22 Aug '01

22 Aug '01

Hi all, I don't understand how to conver arrays from Matlab to Numpy. Here what I did: 1) Converted Matlab A(i,j,k) to 'A.dat' binary file 2) file_open=open('~/A.dat', 'rb') 3) read_file=file_open.read() 4) A_string=fromstring(read_file, 'f') 5) A=reshape(A_string, (i,j,k)) where i,j,k are size of A[i,j,k] I am not getting the right. What is the best/right way of convering arrays? Thanks

2 1

MA patch submitter, please upload file
by Paul F. Dubois 18 Aug '01

18 Aug '01

Would the person who submitted the patch concerning MA binary ops, please upload the file to the patch, or email it to me? You have identified a real problem whose answer is not obvious so I'd like to see your idea of how to fix it. If you log in to SourceForge before submitting bugs and patches then I can know who you are to thank you or make followup inquiries like this one.

1 0

SWIG + Numeric 20.1.0 = core dump: the fix
by Edward C. Jones 11 Aug '01

11 Aug '01

If Numeric and SWIG are used together in creating a Python extension and there are C/C++ files as part of the extension then Numeric 20.1.0 will probably break your code. This happens because changes were made to arrayobject.h between Numeric 20.0.0 and 20.1.0. The changes that need to be made in your code are explained by Konrad Hinsen: (http://www.geocrawler.com/archives/3/1329/2001/5/0/5718231/) FROM: Konrad Hinsen DATE: 05/09/2001 09:12:25 SUBJECT: [Numpy-discussion] Modified header files Recently we had a discussion about how to use NumPy arrays from extension modules with multiple source files, on various platforms. The modified header files that are attached to this message provide a (hopefully!) general solution. In fact, I propose to make them part of the official distribution, unless there are objections. If used like before, these header files give exactly the same result as the ones in NumPy 20.0.0. However, they permit to define the name of the C API pointer array and make it globally visible. Under the condition that the chosen name is unique, this should not create problems under any platform, no matter if static or dynamic linking is used. To use NumPy features in multiple source file extension modules, you have to write #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX #include "Numeric/arrayobject.h" in the main source file (the one that contains the init function) and #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX #define NO_IMPORT_ARRAY #include "Numeric/arrayobject.h" in all the others. The symbol you choose instead of PyArrayXXX should contain both the name of the imported module (array) and the name of the importing module (whatever your module is called) in order to be unique with a reasonably high probability. The same applies to the Ufunc module, just replace "array" by "ufunc" in the example. I have also applied the "static" correction to the Ufunc header file, there is no reason not to do it. Konrad. -- ----------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen(a)NO-SPAM.cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais ----------------------------------------------------------------------- I do this by modifying one header file that is included in all the ".c" and ".i" files. In the following example the header file is "spam.h". ------------------------------------------------------------ doit.py: #! /usr/bin/python import Numeric, spam spam.funct() print 'spam.funct completed' ------------------------------------------------------------ spam.h: #include "Python.h" #define PY_ARRAY_UNIQUE_SYMBOL Py_Array_API_spam #ifndef SWIG_FILE_WITH_INIT #define NO_IMPORT_ARRAY #endif #include "/usr/include/python2.1/Numeric/arrayobject.h" void funct(void); ------------------------------------------------------------ spam.i: %module spam %{ #define SWIG_FILE_WITH_INIT #include "spam.h" %} %init %{ import_array(); printf("called import_array\n"); %} void funct(void); ------------------------------------------------------------ spammodule.c #include "spam.h" void funct(void) { PyArrayObject *pao; int dims[2]; dims[0] = 100; dims[1] = 200; printf("Calling PyArray_FromDims\n"); pao = (PyArrayObject*) PyArray_FromDims(2, dims, PyArray_UBYTE); printf("Completed PyArray_FromDims\n"); Py_DECREF(pao); } ------------------------------------------------------------ compile script: swig -python spam.i gcc -c -Wall spammodule.c spam_wrap.c -I/usr/include/python2.1 ld -shared spammodule.o spam_wrap.o -o spam.so ------------------------------------------------------------ I use RedHat 7.1 Linux on a PC, Python 2.1, Numeric 20.1.0, SWIG 1.1 build 883, and gcc 2.96. Anyone is welcome to use the above example in documentation.

1 0

SWIG / Numeric core dump
by Edward C. Jones 10 Aug '01

10 Aug '01

The following Python / SWIG / Numeric program dumps core. I use RedHat Linux 7.1 on a PC. Python 2.1, Numeric 20.1.0, SWIG 1.1 build 883, gcc 2.96. What is the problem? The output looks like: > doit.py called import_array Calling PyArray_FromDims Segmentation fault (core dumped) > The program files are: ---------------------------------------------- doit.py: #! /usr/bin/python import Numeric, spam spam.funct() print 'spam.funct completed' ---------------------------------------------- spam.h: #include "Python.h" #include "/usr/include/python2.1/Numeric/arrayobject.h" void funct(void); ---------------------------------------------- spam.i %module spam %{ #include "spam.h" %} %init %{ import_array(); printf("called import_array\n"); %} void funct(void); ---------------------------------------------- spammodule.c #include "spam.h" void funct(void) { PyArrayObject *pao; int dims[2]; dims[0] = 100; dims[1] = 200; printf("Calling PyArray_FromDims\n"); pao = (PyArrayObject*) PyArray_FromDims(2, dims, PyArray_UBYTE); printf("Completed PyArray_FromDims\n"); Py_DECREF(pao); } ---------------------------------------------- swigit: swig -python spam.i gcc -c -Wall spammodule.c spam_wrap.c -I/usr/include/python2.1 ld -shared spammodule.o spam_wrap.o -o spam.so

2 1

NumPy installation instructions for Windows
by Just van Rossum 10 Aug '01

10 Aug '01

I develop a package that depends on NumPy, and I would like to add a line or two to my documentation about how to install NumPy under Windows -- just the basic set of modules, not the extensions. I understand users should grab the .exe archive. The FAQ only mentions "The .exe files are Windows installers of the sort that you are used to". Does this mean everything is explained and done by the installer itself, or is there some additional action the user should perform? Thanks! (I have no Windows box here so I can't see for myself...) Just

2 1

swig and C/C++
by Tariq Rashid 08 Aug '01

08 Aug '01

in fact ... even simpler ... can anyone point me to an example of passing Numeric arrays to and from C ... the swig examples are fine for integers / floats ... but as i'm a new to this stuff I need a bit if guidance with Numeric arrays ... any ideas / pointers would be very gratefully receieved... i'm working on wavelet stuff which i want to give back to the Numeric community.... thanks tariq (tariq_rashid at lineone.net) ---------------------------- Today's Topics: 1. NumTut (=?ks_c_5601-1987?B?wMyx1LrA?=) 2. swig and Numeric (Paul F. Dubois)

1 0

fast constructor for arrays from byte data (unpickling?)
by Hung Jung Lu 08 Aug '01

08 Aug '01

Hi, Arrays have a method called tostring() which generates the binary data. Is there an inverse function to that? That is, generating an array from the binary data string? For large matrices, pickling/unpickling is a bit too much overhead (data stored as ASCII character strings instead of binary data strings.) I know, I am talking about a factor 4 here. But there is a big difference between 1 minute loading time and 4 minute loading time. I would imagine this is a very common problem/request for people working with large matrices. And I am sure hacking the C code to provide another fast constructor for arrays from binary strings won't be too hard. The questions is: has anyone already tried it? Is it already there? (For the kludge masters: one kludge is of course to store the binary data on disk, then use cStringIO to build the pickled file and then unpickle from the cStringIO. Speed is probably OK since the pickled file lives on RAM. But that's a kludge. :) ) regards, Hung Jung __________________________________________________ Do You Yahoo!? Make international calls for as low as $.04/minute with Yahoo! Messenger http://phonecard.yahoo.com/

4 11

mmap (was: fast constructor for arrays from byte data)
by Phil Austin 08 Aug '01

08 Aug '01

Chris Barker writes: > I don't understand mmap at all. From the name, it sounds like the entire > contents of the file is mapped into memory, so the memory would get used > as soon as you set it up. If anyone knows, I'd like to hear... This might be more than you need to know, but there's a copy of the article: "Why aren't you using mmap() yet?" by Kevin Sheehan at: http://thrush.eos.ubc.ca/users/phil/mmap/mmap.pdf (note reversed page order) and a Solaris/SGI email debate in mhonarc format: http://thrush.eos.ubc.ca/users/phil/mmap/threads.html bottom line: you really need a vm implementation that supports madvise() to get the maximum benefit from mmap. (Linux 2.4 might have madvise, I haven't tried it out yet -- 2.2 didn't). The way I used to use mmap was through a numpy array class that was initiated from a pointer to a mmap'd file. This made reading/writing to the file as simple as indexing into the numpy array. I haven't been back to reimplement that class on linux since I left solaris, though. You can also get some of the benefits of mmap via netcdf, which will use mmap if it exists. Regards, Phil

2 1