Mailman 3 September 2010 - NumPy-Discussion

multiple 2d vs n-dimensional arrays
by Adam Sept. 11, 2010

Sept. 11, 2010

I'm keeping a large number of data points in multiple 2d arrays, for example: class c(object): def __init__(self): self.a = np.zeros((24, 60)) self.b = np.zeros((24, 60)) ... After processing the data, I'm serializing these to disk for future reference/post-processing. It's a largish amount of data and is only going to get larger. Would it be more efficient (in terms of memory/disk storage to use a single n-dimensional array: self.a = np.zeros((24, 60, 5)) What other advantages (if any) would I gain from storing the data in a single array rather than multiple? The deeper into this project I get, the more I am probably going to need to correlate data points from one or more of the arrays to one or more of the other arrays. I think I just answered my own question... -- THE SuperBitBucket.com You know you're a geek when you can hold an entire conversation with friends in linux CLI talk, i.e. 'cat /proc/lookattheassonthat!' or 'cd /pub/beer'

2 1

adding two pseudoinverse matrixes
by Jose Borreguero Sept. 10, 2010

Sept. 10, 2010

Dear Numpy users, I have to solve for Z in the following equation Z^(-1) = A^(-1) - B^(-1), where A and B are covariance matrices with zero determinant. I have never used pseudoinverse matrixes, could anybody please point to me any cautions I have to take when solving this equation for Z? The brute force approach linalg.pinv( linalg.pinv(A) - lingal.pinv(B) ) gives me a matrix with all entries equal to 'infinity'. -Jose Borreguero

2 2

f2py: generic,public/private procedure
by David Froger Sept. 10, 2010

Sept. 10, 2010

Hy all, As a test case before writing something bigger, I'm trying to write a little Fortran module to compute the average of a array in these 4 cases: avg2d_float, avg2d_double avg3d_float, avg3d_double I want this module to be callable from both Fortran and Python, using f2py. 4 Fortran functions have to be written, and a generic Fortran function 'avg' overloads its. The Fortran module 'stat.f90' (containing the functions), a Fortran program 'stat_example.f90' and a Python script 'stat_example.py' are at the end of the email. (everything works fine: the compilation, f2py, the execution in Python and in Fortran.) My questions are: - Assumed shape array are not supported by f2py. However, it is much more handy to write in Fortran 'avg(array)' than 'avg(array,n1,n2,n3)'. Would it be possible to supported it by modifying the signature files? - Is there a way to avoid to write by hand the 'avg' Python function (in example_stat.py) ? - Isn't there no way to declare private the functions: avg2d_float, avg2d_double, avg3d_float,avg3d_double? It's embarassing its are visible from stat_example.f90. Uncommenting the lines in the top of the file stat.f90: !private !public:: avg and running: f2py -m f90stat -c stat.f90 give the error: use stat, only : avg2d_float Error: Symbol 'avg2d_float' referenced at (1) not found in module 'stat' Thanks for any suggestion, David * to create the Fortran executable: gfortran -c stat.f90 gfortran -c stat_example.f90 gfortran -o stat_example.x stat.o stat_example.o * to create the Python module: f2py -m f90stat -c stat.f90 * The Fortran module is: """ !file stat.f90 module stat implicit none !private !public:: avg interface avg module procedure avg2d_float, avg2d_double, avg3d_float, avg3d_double end interface contains function avg2d_float(array,n1,n2) result(average) implicit none real(kind=4):: average integer:: n1,n2 real(kind=4),dimension(n1,n2),intent(in):: array average = sum(array) average = average / n1 average = average / n2 end function avg2d_float function avg2d_double(array,n1,n2) result(average) implicit none real(kind=8):: average integer:: n1,n2 real(kind=8),dimension(n1,n2),intent(in):: array average = sum(array) average = average / n1 average = average / n2 end function avg2d_double function avg3d_float(array,n1,n2,n3) result(average) implicit none real(kind=4):: average integer:: n1,n2,n3 real(kind=4),dimension(n1,n2,n3),intent(in):: array average = sum(array) average = average / n1 average = average / n2 average = average / n3 end function avg3d_float function avg3d_double(array,n1,n2,n3) result(average) implicit none real(kind=8):: average integer:: n1,n2,n3 real(kind=8),dimension(n1,n2,n3),intent(in):: array average = sum(array) average = average / n1 average = average / n2 average = average / n3 end function avg3d_double end module stat """ * The Fortran program is: """ !file: stat_example.f90 program stat_example use stat implicit none integer,parameter:: n1=10,n2=10,n3=10 real(kind=4),dimension(n1,n2):: array2d_float real(kind=8),dimension(n1,n2):: array2d_double real(kind=4),dimension(n1,n2,n3):: array3d_float real(kind=8),dimension(n1,n2,n3):: array3d_double array2d_float = 4. array2d_double = 4. array3d_float = 4. array3d_double = 4. write(*,*) avg(array2d_float,n1,n2) write(*,*) avg(array2d_double,n1,n2) write(*,*) avg(array3d_float,n1,n2,n3) write(*,*) avg(array3d_double,n1,n2,n3) write(*,*) avg2d_float(array2d_float,n1,n2) end program stat_example """ * The Python script is: """ #!/usr/bin/env python #file stat_example.py import numpy as np import f90stat def avg(a): if a.ndim == 2: if a.dtype == 'float32': return f90stat.stat.avg2d_float(a) elif a.dtype == 'float64' : return f90stat.stat.avg2d_double(a) else: raise ValueError, 'dtype = %r unsupported.' % (a.dtype) elif a.ndim == 3: if a.dtype == 'float32': return f90stat.stat.avg3d_float(a) elif a.dtype == 'float64' : return f90stat.stat.avg3d_double(a) else: raise ValueError, 'dtype = %r unsupported.' % (a.dtype) else: raise ValueError, 'ndim = %r unsupported.' % (a.ndim) if __name__ == '__main__': a = np.arange(6.) a = a.reshape(2,3) print avg(a) """

1 0

Non-Zero Sub-Matrix
by Radek Machulka Sept. 10, 2010

Sept. 10, 2010

Hi Folks, I have array (numpy.ndarray object) with non-zero elements cumulated 'somewhere' (like a array([[0,0,0,0],[0,1,1,0],[0,0,1,0],[0,0,0,0]])) and I need sub-array with just non-zero elements (array([[1,1],[0,1]])). I can do this with iterating throught an array, but I also found some magic tricks with boolen operators in indexing and nonzero function, but it is still not clear to me, how to use it. I am sure that this will be pretty easy for those who are familiar with this kind of array indexing and I hope I will be one of them soon :-). Thanks a lot Radek

2 3

interrupting large matrix operations
by Tom K. Sept. 10, 2010

Sept. 10, 2010

OK, I know it's my problem if I try to form a 15000x15000 array and take the cosine of each element, but the result is that my python session completely hangs - that is, the operation is not interruptible. t=np.arange(15360)/15.36e6 t.shape=(-1,1) X=np.cos(2*np.pi*750*(t-t.T)) <hangs indefinitely> I'd like to hit "control-c" and get out of this hung state. What would it take to support this? (I'm running ancient numpy and python at work, so if this is already supported in later versions, my apologies) - Tom K. -- View this message in context: http://old.nabble.com/interrupting-large-matrix-operations-tp29676118p29676… Sent from the Numpy-discussion mailing list archive at Nabble.com.

2 3

distutils
by Charles Doutriaux Sept. 9, 2010

Sept. 9, 2010

Hi, I'm using distutils to build extensions written in C. I noticed that lately (it seems to be python 2.7 related) whenever I touch 1 C file, ALL the C files are rebuilt. Since I have a lot of C code, it takes a lot of time for nothing. Any idea why this is happening? Do I need to set something new in my setup.py ? Thx, C.

2 1

Fwd: distutils
by Charles Doutriaux Sept. 9, 2010

Sept. 9, 2010

-------- Original Message -------- Subject: [Numpy-discussion] distutils Date: Tue, 7 Sep 2010 12:12:58 -0700 From: Charles Doutriaux <doutriaux1(a)llnl.gov> Reply-To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> To: Discussion of Numerical Python <numpy-discussion(a)scipy.org> Hi, I'm using distutils to build extensions written in C. I noticed that lately (it seems to be python 2.7 related) whenever I touch 1 C file, ALL the C files are rebuilt. Since I have a lot of C code, it takes a lot of time for nothing. Any idea why this is happening? Do I need to set something new in my setup.py ? Thx, C. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion(a)scipy.org http://*mail.scipy.org/mailman/listinfo/numpy-discussion

1 0

clip() with None as argument
by Friedrich Romstedt Sept. 9, 2010

Sept. 9, 2010

I just came across a problem with the intention to specify unset boundaries given to numpy.clip() or array.clip(): a.clip(1e-10, None) a.clip(None, -1e-10) When doing this, the returned array is dtype=numpy.object, seemingly None gets converted to a numpy.asarray(None, dtype=numpy.object), and this is then used for the comparison. Since (*any number* > None) is true, and (*any number* < None) is false, it works or not, depending on the usecase, despite one is still left with a dtype=numpy.object array. Precisely, using None as the lower boundary semi-works, while using None as the upper boundary doesn't match expectations at all. One can work around this, sure, but in my opinion it would be handy anyway to have clip() interpret None as argument as \pm \infty. Maybe I'm just missing the appropriate function, though. Friedrich

3 6

IEEE 754-2008 decimal floating point support
by Michael Gilbert Sept. 8, 2010

Sept. 8, 2010

Hi, Are there any plans to add support for decimal floating point arithmetic, as defined in the 2008 revision of the IEEE 754 standard [0], in numpy? Thanks for any info. Best wishes, Mike [0] http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4610935&tag=1

6 14

indexing with booleans without making a copy?
by Ernest Adrogué Sept. 8, 2010

Sept. 8, 2010

Hello, I have a sorted, flat array: In [139]: a =np.array([0,1,2,2,2,3]) Basically, I want views of the areas where there are repeated numbers (since the array is sorted, they will be contiguous). But, of course, to find the numbers that are repeated I have to use comparison operations that return boolean arrays, so I suppose the problem is converting the boolean array into a slice. This is what I have come up with: In [146]: np.flatnonzero(a==2) Out[146]: array([2, 3, 4]) In [147]: b=np.flatnonzero(a==2) In [148]: b.min() Out[148]: 2 In [149]: b.max() Out[149]: 4 In [150]: a[b.min():b.max()+1] Out[150]: array([2, 2, 2]) If you know a more straightforward way of doing this, I'll be glad to know... Cheers :) Ernest

3 3