Mailman 3 December 2006 - NumPy-Discussion

numpy in debian
by Keith Goodman Dec. 5, 2006

Dec. 5, 2006

I'm impressed with how easy it is to compile numpy. I'm even more impressed with how easy it is let someone else compile it and just apt-get it. Does anyone know the numpy plans for debian? It is currently at 1.0rc1. I'm afraid to ask the debian numpy maintainers since they have already done me such a big favor by packaging numpy and I haven't even thanked them yet. Oh, yeah. Then there's that other project I should thank. Thank you. Numpy is a joy to use.

1 0

dot operations on multidimensional arrays
by Carsten Rostgaard Dec. 5, 2006

Dec. 5, 2006

Hi! I am trying to use the "dot" method on multi-(more than 2)-dimensional arrays. Specifically I do >> y = dot(a, b) where a is a 2D array and b is a 3D array. using numpy I get the the help: " dot(...) dot(a,v) returns matrix-multiplication between a and b. The product-sum is over the last dimension of a and the second-to-last dimension of b. " I then expect that >> y[i, j, k] = sum(a[i, :] * b[j, :, k]) which is actually what I get. The question is then: 1) Is there any way to change the axis for which the product-sum is performed. This can of course be done by a swapaxis before and after the operation, but this makes the array non-contiguous, in which case the dot operation often makes bugs (at least in Numeric). 2) For complicated reasons we still use Numeric in our software package, and in this, "dot" behaves very strangely. According to the Numeric help: " dot(a, b) dot(a,b) returns matrix-multiplication between a and b. The product-sum is over the last dimension of a and the second-to-last dimension of b. " so I would have expected again that y[i, j, k] = sum(a[i, :] * b[j, :, k]), and the dimensions actually fit, i.e. y.shape = (a.shape[0], b.shape[0], b.shape[2]), but only some rows of the result has these values!! Does anyone know what Numeric.dot(a, b) actually does when b has more than two dimensions? I use the following test script: ---------------------BEGIN SCRIPT----------------------- import Numeric as num # import numpy as num # make 'random' input arrays a = num.zeros((2, 5)) b = num.zeros((3, 5, 4)) a.flat[:] = num.arange(len(a.flat)) - 3 b.flat[:] = num.arange(len(b.flat)) + 5 # built-in dot product y1 = num.dot(a, b) # manual dot product y2 = num.zeros((a.shape[0], b.shape[0], b.shape[2])) for i in range(a.shape[0]): for j in range(b.shape[0]): for k in range(b.shape[2]): y2[i, j, k] = num.sum(a[i,:] * b[j, :, k]) # test for consistency print y1 == y2 ---------------------END SCRIPT------------------------ with the result: [[[1 1 1 1] [0 0 0 0] [0 0 0 0]] [[1 1 1 1] [0 0 0 0] [0 0 0 0]]] thanks a lot, Carsten Rostgaard Carsten.Rostgaard(a)fysik.dtu.dk

3 2

Precision in Python
by Elton Mendes Dec. 5, 2006

Dec. 5, 2006

Hi. I'm having a precision problem in python Example: >>> a = 5.14343434 >>> b = round(a,1) >>> b 5.0999999999999996 >>> It´s possible to round the number exactly to 5.1

3 2

How to speed up this function?
by fsenkel＠verizon.net Dec. 5, 2006

Dec. 5, 2006

Hello, I'm taking a CFD class, one of the codes I wrote runs very slow. When I look at hotshot is says the function below is the problem. Since this is an explicit step, the for loops are only traversed once, so I think it's caused by memory usage, but I'm not sure if it's the local variables or the loop? I can vectorize the inner loop, would declaring the data structures in the calling routine and passing them in be a better idea than using local storage? I'm new at python and numpy, I need to look at how to get profiling information for the lines within a function. Thank you, Frank PS I tried to post this via google groups, but it didn't seem to go through, sorry if it ends up as multiple postings def findw(wnext,wprior,phiprior,uprior,vprior): #format here is x[i,j] where i's are rows, j's columns, use flipud() to get the #print out consistent with the spacial up-down directions #assign local names that are more #inline with the class notation w = wprior phi = phiprior u = uprior v = vprior #three of the BC are known so just set them #symetry plane wnext[0,0:gcols] = 0.0 #upper wall wnext[gN,0:gcols] = 2.0/gdy**2 * (phi[gN,0:gcols] - phi[gN-1,0:gcols]) #inlet, off the walls wnext[1:grows-1,0] = 0.0 upos = where(u>0) vpos = where(v>0) Sx = ones_like(u) Sx[upos] = 0.0 Sy = ones_like(v) Sy[vpos] = 0.0 uw = u*w vw = v*w #interior nodes for j in range(1,gasizej-1): for i in range(1,gasizei-1): wnext[i,j] =( w[i,j] + gnu*gdt/gdx**2 * (w[i,j-1] - 2.0*w[i,j] + w[i,j+1]) + gnu*gdt/gdy**2 * (w[i-1,j] - 2.0*w[i,j] + w[i+1,j]) - (1.0 - Sx[i,j]) * gdt/gdx * (uw[i,j] - uw[i,j-1]) - Sx[i,j] * gdt/gdx * (uw[i,j+1] - uw[i,j]) - (1.0 - Sy[i,j]) * gdt/gdy * (vw[i,j] - vw[i-1,j]) - Sy[i,j] * gdt/gdy * (vw[i+1,j] - vw[i,j]) ) ## print "***wnext****" ## print "i: ", i, "j: ", j, "wnext[i,j]: ", wnext[i,j] #final BC at outlet, off walls wnext[1:grows-1,gM] = wnext[1:grows-1,gM-1]

4 4

ValueError: dimensions too large.
by koara＠atlas.cz Dec. 4, 2006

Dec. 4, 2006

Hello, i tried to create a 2d array, but encountered: ValueError: dimensions too large. Does this refer to insufficient memory, or is there really a limit on dimension sizes? Cheers.

1 0

Forward from PIL list...
by Christopher Barker Dec. 4, 2006

Dec. 4, 2006

Hi all, PIL 1.1.6 final was just released, which includes support for the numpy array interface. However, this just came out on the list: Zachary Pincus wrote: > - The 'fromarray' command is a bit broken in Image.py: > Specifically, the following stanza is incorrect -- > if mode is None: > typestr = arr['typestr'] > if not (typestr[0] == '|' or typestr[0] == _ENDIAN or > typestr[1:] not in ['u1', 'b1', 'i4', 'f4']): > raise TypeError("cannot handle data-type") > typestr = typestr[:2] > if typestr == 'i4': > ... > > The error is that 'typestr = typestr[:2]' should instead be 'typestr > = typestr[1:]' that code was contributed by the numpy developers, but it sure looks broken. hmm. </F> Maybe someone who knows better than me can send a not to the PIL list. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

1 0

problems installing NumPy on OSX
by arildna＠stud.ntnu.no Dec. 3, 2006

Dec. 3, 2006

Hi, I'm sorry if this might seem like a stupid question to some of you, but I have been struggling for the better part of the afternoon trying to install NumPy on my G4 iBook, so I hope somebody can take the time to lend me a helping hand. I have searched a little in the mail archives and tried to follow the instructions on this page: http://www.scipy.org/Installing_SciPy/Mac_OS_X Python 2.5 is installed, and I have installed Apple's Developer's Tools (i.e. the package "December 2002 Mac OS X Developer Tools"). The GCC/G77-installation seemed to go ok. When I try to install the FFW libraries however, I get an error message telling me that my "C compiler cannot create executables" (full output below). After that I didn't really expect the installation of NumPy to work, but I got a "permission denied"-error message that I'm not so sure is connected to the missing FFW. So I'm at loss... Anyone? Many thanks, Arild Næss =============================================================== %pwd /Users/arildnss/Desktop/fftw-3.1.2 %./configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for gawk... no checking for mawk... no checking for nawk... no checking for awk... awk checking whether make sets $(MAKE)... yes checking whether to enable maintainer-specific portions of Makefiles... no checking build system type... powerpc-apple-darwin8.8.0 checking host system type... powerpc-apple-darwin8.8.0 checking for gcc... gcc checking for C compiler default output file name... configure: error: C compiler cannot create executables See `config.log' for more details. %pwd /Users/arildnss/Desktop/numpy-1.0 %python setup.py build Running from numpy source directory. F2PY Version 2_3396 blas_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers'] lapack_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec'] running build running config_fc running build_src building py_modules sources error: build/src.macosx-10.3-fat-2.5/numpy/distutils/__config__.py: Permission denied ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

9 21

Re: [Numpy-discussion] nan functions convert matrix to array
by Pierre GM Dec. 3, 2006

Dec. 3, 2006

On Friday 01 December 2006 17:56, Keith Goodman wrote: ... > Would it break anything to change the first line of the nan functions from > a = array(a) > to > a = asanyarray(a) > ? Seeing what the nan functions do, I don't think that would be a problem. An expception would be raised if the operation could not be performed anyway (like a N.sum on a record array). But I'm no judge, so I'll let the powers in place decide of that. In the same order of idea, I'm bumping a post of mine: would it be possible to get 'asnayarray' in 'apply_along_axis', 'apply_over_axes', 'vectorize' ?

1 0

numpy.fft.rfftn -> non-contiguous
by Pierre Thibault Dec. 1, 2006

Dec. 1, 2006

Hello! I'm a little confused about what rfftn is doing: It seems to me that the best would be for it to return a C-contiguous array with the first element reduced by a half (plus one), so that one can easily obtain the non-repeated slices. What I get is the following: In [1]: from numpy import * In [2]: a = random.rand(8,8,8) In [3]: fa = fft.rfftn(a) In [4]: fa.shape Out[4]: (8, 8, 5) In [5]: fa.flags Out[5]: CONTIGUOUS : False FORTRAN : False OWNDATA : False WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False In [6]: fa.swapaxes(-1,-2).flags Out[6]: CONTIGUOUS : True FORTRAN : False OWNDATA : False WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False What I would like to have is fa.shape: (5,8,8) fa.flags: CONTIGUOUS = True So, now fa[0] is a contiguous block containing the data that is not supposed to appear twice in the complex fft (that is, it would be nice if fft.rfftn(a)[0] == fft.fftn(a)[0]). I tried playing with the axes argument in rfftn, to no avail. I can do without, but at least it looks kind of ugly to me that the array returned by a built-in function is not contiguous. By the way, I am using the official debian unstable numpy package. It doesn't look like it is using fftw, and I don't know if any of this behaviour would be different if I had compiled numpy by myself - I am not even sure using fftw is an option with numpy. Thanks for any answer/comments on that! Pierre -- Pierre Thibault 616 Clark Hall, Cornell University (607) 255-5522

1 0

nan functions convert matrix to array
by Keith Goodman Dec. 1, 2006

Dec. 1, 2006

The first line of the nan functions (such as nansum, nanmin, nanmax) is y = array(a) That leads to matrix in, array out. Is there some way to make it matrix in, matrix out? Here, for example, is nansum: def nansum(a, axis=None): """Sum the array over the given axis, treating NaNs as 0. """ y = array(a) if not issubclass(y.dtype.type, _nx.integer): y[isnan(a)] = 0 return y.sum(axis)

2 1

efficient way to get first index of first masked/non-masked value in a masked array
by Matt Knox Dec. 1, 2006

Dec. 1, 2006

all I can come up with is dumb brute force methods by iterating through all the values. Anyone got any tricks I can use? Thanks, - Matt Knox

2 1

seting the dtype for where...
by Chris Barker Dec. 1, 2006

Dec. 1, 2006

Hi all, I'd like to set the data type for what numpy.where creates. For example: import numpy as N N.where(a >= 5, 5, 0) creates an integer array, which makes sense. N.where(a >= 5, 5.0, 0) creates a float64 array, which also makes sense, but I'd like a float32 array, so I tried: N.where(a >= 5, array(5.0, dtype=N.float32), 0) but I got a float64 array again. How can I get a float32 array? where doesn't take a dtype argument -- maybe it should? numpy version 1.0 thanks, -Chris

4 3

subclass
by Lionel Roubeyrie Dec. 1, 2006

Dec. 1, 2006

Hi all, is it possible to subclass numpy.array to set extras functionalities and change the behavior of others? I can't find any docs on that. Thanks -- Lionel Roubeyrie - lroubeyrie(a)limair.asso.fr LIMAIR http://www.limair.asso.fr

3 4

ScipySuperpack for Mac (PowerPC)
by Tommy Grav Dec. 1, 2006

Dec. 1, 2006

I installed the Mac ScipySuperpack (from http://www.scipy.org/Download). However it seems that the version of matplotlib in there is not compatible with their version of numpy [tgrav@******] ch2/pbcd -> python ActivePython 2.4.3 Build 11 (ActiveState Software Inc.) based on Python 2.4.3 (#1, Apr 3 2006, 18:07:18) [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> import pylab >>> x = range(1,10) >>> y = range(1,10) >>> pylab.plot(x,y) [<matplotlib.lines.Line2D instance at 0x2b20210>] >>> pylab.show() alloc: invalid block: 0xa08bcd8: a 68 0 Abort [tgrav@*****] ch2/pbcd -> Anyone know how to fix this? Cheers Tommy

1 0

bare bones numpy extension code
by John Hunter Dec. 1, 2006

Dec. 1, 2006

A colleague of mine wants to write some numpy extension code. I pointed him to lots of examples in the matplotlib src dir, but the build environment is more complicated than he needs with all the numpy/numeric/numarray switches, etc. Does someone have the basic "hello world" of numpy extensions that includes src code and a basic setup.py that I can pass on to him. It might be nice to include something like that in a numpy "examples" directory. JDH

2 1

save a matrix
by Keith Goodman Dec. 1, 2006

Dec. 1, 2006

What's a good way to save matrix objects to file for later use? I just need something quick for debugging. I saw two suggestions on this list from Francesc Altet (2006-05-22): 1. Use tofile and fromfile and save the meta data yourself. 2. pytables Any suggestions for #3? If I do #1, how would I put multiple matrices in a single file? Make everything a 1-d array, first element is number of matrices, followed by number of rows and columns for each matrix?

2 6