Mailman 3 March 2007 - NumPy-Discussion

Multiplying Each Column of a Matrix by a Vector (Element-Wise)
by Alexander Michael 14 Mar '07

14 Mar '07

Is there a clever way to multiply each column of a matrix by a vector *element-wise*? That is, the equivalent of (from some 1D v and 2D m): r = numpy.empty_like(m) for i in range(m.shape[-1]): r[...,i] = v*m[...,i] Thanks, Alex

2 1

cannot pickle large numpy objects when memory resources are already stressed
by Glen W. Mabey 14 Mar '07

14 Mar '07

Hello, After running a simulation that took 6 days to complete, my script proceeded to attempt to write the results out to a file, pickled. The operation failed even though there was 1G of RAM free (4G machine). I've since reconsidered using the pickle format for storing data sets that include large numpy arrays. However, somehow I assumed that one would be able to pickle anything that you already had in memory, but I see now that this was a rash assumption. Ought there to be a way to do this, or should I forget about being able to bundle large numpy arrays and other objects in a single pickle? Thanks, Glen (these commands performed on a different machine with 1G RAM) In [10]: za = numpy.zeros( (100000000,), dtype=numpy.float32 ) In [11]: import cPickle In [12]: zfile = file( '/tmp/zfile', 'w' ) In [13]: cPickle.dump( za, zfile ) --------------------------------------------------------------------------- exceptions.MemoryError Traceback (most recent call last)

3 3

Questions regarding migrating from Numeric to numpy
by vinjvinj 14 Mar '07

14 Mar '07

I'm in the process of migrating from Numeric to numpy. In some of my code I have the following: a = zeros(num_elements, PyObject) b = zeros(num_elements, PyObject) a is an array of python string objects and b is an array holding mx.DateTime objects. What do I have to do to migrate this over to numpy? PS: I tried to register with the scipy mailing list but got no email confirmation. I tried with two different email addresses. Thanks, VJ

2 1

Migrating from numeric to numpy
by vinjvinj 14 Mar '07

14 Mar '07

So far my migration seems to be going well. I have one problem: I've been using the scipy_base.insert and scipy_base.extract functions and the behavior in numpy is not the same. a = [0, 0, 0, 0] mask = [0, 0, 0, 1] c = [10] numpy.insert(a, mask, c) would change a so that a = [0, 0, 0, 10] This is not what I'm seeing in numpy. Thanks, VJ

2 1

compiling numarrray on Cygwin
by Duhaime Johanne 14 Mar '07

14 Mar '07

I have problem to install numarray on cygwin. I have seen emails on that topic but I finally do not have a solution. Below is the error message for numarray-1.5.2 on python 2.4.3. I am using numarray instead of numpy because I need the module to be compatible with a software : MAT (Model-based Analysis of Tiling-array). I have seen some solutions but I do not know exactly how to do since I am not familiar with python and cygwin. Someone suggests to disable the package that contains fetestexcept and fetestexcept. I do not know if this is important for my software but I am willing to try. The problem is that I do not know how to disable this package... The same person mention that it would put a "solution" in the csv repository. If I am right there is no csv repository for numarray. Should I look to the python csv. In fact I did that, but have not seen any comment related to my problem. As you can see my questions are very basic. Sorry about that. But I would really appreciate help to solve that problem. Thank you in advance for any help. -----------------Compile: Building many extension but then with 'numarray.libnumarray' I have an error. (...) building 'numarray._ufuncComplex32' extension gcc -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -IInclude/nu array -I/usr/include/python2.4 -c Src/_ufuncComplex32module.c -o build/temp.cyg in-1.5.24-i686-2.4/Src/_ufuncComplex32module.o gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-1.5.24-i686-2.4/Src/ ufuncComplex32module.o -L/usr/lib/python2.4/config -lpython2.4 -o build/lib.cyg in-1.5.24-i686-2.4/numarray/_ufuncComplex32.dll -lm -L/lib -lm -lc -lgcc -L/lib mingw -lmingwex building 'numarray._ufuncComplex64' extension gcc -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -IInclude/nu array -I/usr/include/python2.4 -c Src/_ufuncComplex64module.c -o build/temp.cyg in-1.5.24-i686-2.4/Src/_ufuncComplex64module.o gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-1.5.24-i686-2.4/Src/ ufuncComplex64module.o -L/usr/lib/python2.4/config -lpython2.4 -o build/lib.cyg in-1.5.24-i686-2.4/numarray/_ufuncComplex64.dll -lm -L/lib -lm -lc -lgcc -L/lib mingw -lmingwex building 'numarray.libnumarray' extension gcc -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -IInclude/num array -I/usr/include/python2.4 -c Src/libnumarraymodule.c -o build/temp.cygwin-1 .5.24-i686-2.4/Src/libnumarraymodule.o gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-1.5.24-i686-2.4/Src/l ibnumarraymodule.o -L/usr/lib/python2.4/config -lpython2.4 -o build/lib.cygwin-1 .5.24-i686-2.4/numarray/libnumarray.dll -lm -L/lib -lm -lc -lgcc -L/lib/mingw -l mingwex /lib/mingw/libmingwex.a(feclearexcept.o):feclearexcept.c:(.text+0x21): undefined reference to `___cpu_features' /lib/mingw/libmingwex.a(fetestexcept.o):fetestexcept.c:(.text+0x7): undefined re ference to `___cpu_features' collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1 duhaimj(a)ircm.qc.ca

2 1

problem with installing scipy
by nevin 14 Mar '07

14 Mar '07

I am trying to use scipy optimize module but I am having problem when I try to import it: >>> import numpy >>> import scipy >>> from scipy import optimize Traceback (most recent call last): File "<stdin>", line 1, in ? File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/scipy/optimize/__init__.py", line 7, in ? from optimize import * File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/scipy/optimize/optimize.py", line 25, in ? import linesearch File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/scipy/optimize/linesearch.py", line 3, in ? import minpack2 ImportError: Inappropriate file type for dynamic loading How can I fix this problem? My system is Mac OSX Tiger- Pentium. Thanks.

2 1

A surprising result from benchmarking
by Dan Becker 14 Mar '07

14 Mar '07

Hi Everyone, I'm new to numpy, and I'm finding it hard to predict what is fast in python/numpy and what is slow. The following seems puzzling: I am doing the same thing an ugly way and a cleaner way. But the ugly map/lambda/filter expression is 15x faster than using numpy's internals. Can anyone explain why? For now, this makes me nervous about incorporating basic numpy functionality into real programs. ---Code starts here--- import scipy import time import psyco from numpy import matrix print("New run") myMat=scipy.randn(500,500) t1=time.time() highEnough=myMat>0.6 greaterPerLine=[sum(x) for x in highEnough] elapsed1=time.time()-t1 print("method 1 took %f seconds"%elapsed1) t2=time.time() greaterPerLine2=map(lambda(x):len(filter(lambda(y):y>0.6,x)),myMat) elapsed2=time.time()-t2 print("method 2 took %f seconds"%elapsed2) ---Output starts here--- New run method 1 took 3.566760 seconds method 2 took 0.232356 seconds --- Thanks so much! Dan

4 5

Vectorized ufuncs with OS X frameworks
by Eric Brown 14 Mar '07

14 Mar '07

Hi All, I have a set of large arrays to which I have to do exp functions over and over again. I am wondering if there is any benefit to teaching numpy how to use the vForce frameworks to do this function. Does anyone have experience hacking the source to get numpy to use system libraries to do vectorized versions of sqrt, exp, log, addition, etc.? I am also interested in doing this for other operating systems. Best regards, Eric

2 1

[PATCH] a fix for compiling numexpr with MSVC Toolkit 2003
by faltet＠carabos.com 14 Mar '07

14 Mar '07

Hi, I've done a patch for allowing compiling the last version of numexpr with the MSVC Toolkit 2003 compiler on Windows platforms. You can fetch it from: http://www.pytables.org/trac/changeset/2514/trunk BTW, I understand now why Tim Hochberg was so worried about the time that it takes to compile numexpr on Win platforms. On my Pentium4 @ 2 GHz, and using the MSVC Toolkit 2003, compiling numexpr takes 20 minutes aprox. (!). With the same machine and using GCC under Linux it takes no more than 1 minute. Mmmm, I think it's time to look at the MINGW compiler (GCC based). Cheers, Francesc

4 3

efficient norm of a vector
by lorenzo bolla 14 Mar '07

14 Mar '07

Hi all, just a quick (and easy?) question. what is the best (fastest) way to implement the euclidean norm of a vector, i.e. the function: import scipy as S def norm(x): """normalize a vector.""" return S.sqrt(S.sum(S.absolute(x)**2)) ? thanks in advance, Lorenzo.

2 2