Mailman 3 March 2010 - SciPy-User

Compiling scipy.sparse.sparsetools extension throws g++ internal error
by Diederik van Liere 09 Mar '10

09 Mar '10

Dear scipy-user members, I am trying to build Scipy for my 32bit Redhat server and I am running into serious trouble, both with scipy 0.7.1 and the latest SVN version. I get the following error: building 'scipy.sparse.sparsetools._csr' extension compiling C++ sources C compiler: g++ -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC creating build/temp.linux-i686-2.6/scipy/sparse/sparsetools compile options: '-I/usr/local/lib/python2.6/site-packages/numpy/core/include -I/usr/local/include/python2.6 -c' g++: scipy/sparse/sparsetools/csr_wrap.cxx g++: Internal error: Terminated (program cc1plus) Please submit a full bug report. See <URL:http://gcc.gnu.org/bugs.html> for instructions. g++: Internal error: Terminated (program cc1plus) Please submit a full bug report. See <URL:http://gcc.gnu.org/bugs.html> for instructions. error: Command "g++ -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/lib/python2.6/site-packages/numpy/core/include -I/usr/local/include/python2.6 -c scipy/sparse/sparsetools/csr_wrap.cxx -o build/temp.linux-i686-2.6/scipy/sparse/sparsetools/csr_wrap.o" failed with exit status 1 and I really don't know how to fix this. I have tried both Numpy 1.3.0 and the latest Numpy SVN version but to no avail. I hope someone can point me in the right direction. Thanks so much, Best regards, Diederik

2 1

operations on large arrays
by Vincent Davis 09 Mar '10

09 Mar '10

I have arrays of 8-20 rows and 230,000 column, all the data is float64 I what to be able to find the difference in the correlation matrix between arrays let A and B be of size (10, 230000) np.corrcoef(a)-np.corrcoef(b) I can't seem to do this with more than 10000 columns at a time because of memory limitations. (about 9GB usable to python) Is there a better way? I also have problem finding the column means which is surprising to me, I was not able to get the column means for 10000 columns, but I can computer the corrcoeff ? np.mean(a, axis=0) Do I just need to divide up the job or is there a better approach? Thanks *Vincent Davis 720-301-3003 * vincent(a)vincentdavis.net my blog <http://vincentdavis.net> | LinkedIn<http://www.linkedin.com/in/vincentdavis>

2 2

flat / nonflat array index conversion continued
by Christoph Deil 09 Mar '10

09 Mar '10

Sorry that I don't reply to the thread I started today (http://mail.scipy.org/pipermail/scipy-user/2010-March/024565.html), but I had the list in digest mode. Anyways, Ryan suggested I use numpy.unravel_index, and indeed this does exactly what I wanted. My question now is how I can vectorize numpy.unravel_index? Here is what I tried: idx = array([1,5]) # really I have 1e6 indices I want to unravel unravel_index(idx,(2, 3)) # Doesn't work. Raises a ValueError v_u_i = vectorize(unravel_index) # Try to vectorize it v_u_i(idx,(2, 3)) # Doesn't work. Raises a TypeError Thanks for you help! --Christoph

1 0

Finding coefficient
by mudit sharma 09 Mar '10

09 Mar '10

I have 2d array of observations x1, x2, x3, x4 which I am looking to solve in such a way that z have a minimum value. ax1 + bx2 + cx3 + dx4 = s z = np.where(s < 0).shape[0] I am currently using a brute force method by putting a constraint 0 > (a,b,c,d) < 20. I am looking for ideas to acheive this more efficiently. Cheers, M

1 0

flat / nonflat array index conversion
by Christoph Deil 09 Mar '10

09 Mar '10

Is there a numpy function to convert corresponding array indices in flattened / nonflat multidimensional arrays for a given shape? E.g. for a = array([0,1,2,3,4,5]).reshape(2,3) I want some function that converts e.g. 1 to [0,1] and 5 to [1,2] if I tell it a.shape. For 2D it's of course easy to do it by hand, but I need something that is fast and works for arrays of any dimension.

4 3

scipy+lapack array copy
by Paul Anton Letnes 09 Mar '10

09 Mar '10

Hi everyone. I have a numerical project where I link a fortran module to python using f2py. The fortran module basically sets up the left hand side coefficient matrix of a linear equation system. The right hand side is small, so I set that up in python. So far, everything works great. Now, when I call scipy-lapack, I am using: from scipy.linalg.lapack import flapack as lapack lu, piv, x, info = lapack.zgesv(A, b) A is created in the fortran part of the program. b is created from numpy.zeros(). Both report as F_CONTIGUOUS by printing A.flags and b.flags. The problem is that I'd like A to be large (>1GB), but the lapack wrapper (or lapack itself?) copies the array, doubling the memory use. Aside from being unnecessary, this also reduces the practical problem size I can solve by a factor 2, which of course is very annoying. How can I get around this problem? Regards, Paul.

2 2

Google Summer of Code...
by Christopher Barker 08 Mar '10

08 Mar '10

Hi folks, Are we going to try to get some sciPy project in GSoC under the Python umbrella this year? Or might it be worth applying to be a mentoring organization? Has anyone started thinking about it? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

7 8

[OT] Anyone here know Audacity and WAV?
by David Goldsmith 08 Mar '10

08 Mar '10

If so, please reply off list. Thanks! DG

1 0

structured array question? (slow learner :)
by Vincent Davis 08 Mar '10

08 Mar '10

Ok not sure what I am doing wrong here, In fact I am sure this was working and now I updated python and it doesn't now. I am skeptical that that is true. My question is what is happening to the strings? >>> import numpy as np >>> a = [(1,1),(1,2),(1,3),(2,4),(2,5),(2,6)] >>> b = [(1,1,'apple', 'pie'),(2,5,'boys','play')] >>> aa = np.array(a, [('x',int),('y',int)]) >>> bb = np.array(b, [('x',int),('y',int), ('xb',str), ('yb',str)]) >>> bb array([(1, 1, '', ''), (2, 5, '', '')], dtype=[('x', '<i8'), ('y', '<i8'), ('xb', '|S0'), ('yb', '|S0')]) >>> *Vincent Davis 720-301-3003 * vincent(a)vincentdavis.net my blog <http://vincentdavis.net> | LinkedIn<http://www.linkedin.com/in/vincentdavis>

5 7

Combining arrays based on (x,y) index values
by Vincent Davis 07 Mar '10

07 Mar '10

I have 3 arrays, each have an x and y column and I need to extend each row with the additional rows from the other arrays. Only one of the arrays has all the x,y values. I am also not sure how to deal with the missing values. I was thinking about making an array full of nan. So for example if I have >>> bb array([['1', '1', 'apple', 'pie'], ['2', '5', 'boys', 'play']], dtype='|S8') >>> aa array([[1, 1,free], [1, 2,upgrade], [1, 3, down], [2, 4, left], [2, 5, right], [2, 6, now]]) and I what to end up with magicfunc(aa,bb) array([[1, 1, free, 'apple', 'pie'], [1, 2, upgrade, '', ''], [1, 3, down, '', ''], [2, 4, left, '', ''], [2, 5, right, 'boys', 'play'], [2, 6, now, '', '']]) Whats the "right" way to do this, I have several ideas but I can't be the first to do this. I know the dimensions of the final array. but don't really have a plan of where to go from there. *Vincent Davis 720-301-3003 * vincent(a)vincentdavis.net my blog <http://vincentdavis.net> | LinkedIn<http://www.linkedin.com/in/vincentdavis>

2 2