Mailman 3 July 2010 - NumPy-Discussion

Can't get ufunc to work for integers
by John Salvatier July 22, 2010

July 22, 2010

Hello, I am trying to learn how to create ufuncs, and I got a ufunc to compile correctly with the signature int -> double, but I can't get it to accept any arguments. My function is testfunc and I used NPY_INT as the first signature and NPY_DOUBLE as the second signature. What should I look at to figure this out? In [10]: testfunc.types Out[10]: ['i->d', '?->?'] In [11]: testfunc(x) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /Users/johnsalvatier/Documents/workspace/pymc_numexprtest/src/test/<ipython console> in <module>() TypeError: function not supported for these types, and can't coerce safely to supported types In [12]: x Out[12]: array([1, 2, 3]) In [13]: x.dtype Out[13]: dtype('int32')

2 3

irfft, odd size input and truncation
by David Cournapeau July 22, 2010

July 22, 2010

Hi, While looking at improving numpy.fft, I encountered some issue with the definition of irfft for odd-size input. The docstring says that irfft(x, n) when n < x.size truncate the high frequencies of x, but it seems this is ambiguous for odd-size x and even n ? In particular, I can't manage to emulate irfft with ifft in that case. cheers, David

1 0

lstsq functionality
by Charles R Harris July 22, 2010

July 22, 2010

Hi All, I'm thinking about adding some functionality to lstsq because I find myself doing the same fixes over and over. List follows. 1. Add weights so data points can be weighted. 2. Use column scaling so condition numbers make more sense. 3. Compute covariance approximation? Unfortunately, the last will require using svd since there no linear least squares routines in LAPACK that also compute the covariance, at least that google knows about. Thoughts? Chuck

5 19

UMFPACK interface is unexpectedly slow
by Ioan-Alexandru Lazar July 22, 2010

July 22, 2010

Hello everyone, First of all, let me apologize for my earlier message; I made the mistake of trying to indent my code using SquirrelMail's horrible interface -- and pressing Tab and Space resulted in sending my (incomplete) e-mail to the list. Cursed be Opera's keyboard shortcuts now :-). I'm currently planning to use a Python-based infrastructure for our HPC project. I've previously used NumPy and SciPy for basic scientific computing tasks, so performance hasn't been quite an issue for me until now. At the moment I'm not too sure as to what to do next though, and I was hoping that someone with more experience in performance-related issues could point me to a way out of this. The trouble lays in the following piece of code: === w = 2 * math.pi * f M = A - (1j*w*E) n = M.shape[1] B1 = numpy.zeros(n) B2 = numpy.zeros(n) B1[n-2] = 1.0 B2[n-1] = 1.0 -> slow part starts here umfpack.numeric(M) x1 = umfpack.solve( um.UMFPACK_A, M, B1, autoTranspose = False) x2 = umfpack.solve( um.UMFPACK_A, M, B2, autoTranspose = False) solution = scipy.array([ [ x1[n-2], x2[n-2] ], [ x1[n-1], x2[n-1] ]]) return solution ==== This isn't really too much -- it's generating a system matrix via operations that take little time, as I was expecting. Trouble is, the solve part takes significantly more time than Octave -- about 4 times. I'm using the stock version of UMFPACK in Ubuntu's repository; it's compiled against standard BLAS, so it's fairly slow, but so is Octave -- so the problem isn't there. I'm obviously doing something wrong related to memory management here, because the memory consumption is also rocketing, but I'm not sure what exactly it is that I'm doing wrong. Could you point me towards some relevant documentation describing what I could do in order to improve the performance, or give me some hint related to that? Best regards, Alexandru Lazar

1 1

(no subject)
by Ioan-Alexandru Lazar July 22, 2010

July 22, 2010

Hello everyone, I'm currently planning to use a Python-based infrastructure for our HPC project. I've previously used NumPy and SciPy for basic scientific computing tasks, but performance hasn't been quite an issue for me until now. At the moment I'm not too sure as to what to do next though, and I was hoping that someone with more experience in performance-related issues could point me to a way out of this. The trouble lays in the following piece of code: === w = 2 * math.pi * f M = A - (1j*w*E) n = M.shape[1] B1 = numpy.zeros(n) B2 = numpy.zeros(n) B1[n-2] = 1.0 B2[n-1] = 1.0 -> slow part starts here umfpack.numeric(M) x1 = umfpack.solve( um.UMFPACK_A, M, B1, autoTranspose = False) x2 = umfpack.solve( um.UMFPACK_A, M, B2, autoTranspose = False) solution = scipy.array([ [ x1[n-2], x2[n-2] ], [ x1[n-1], x2[n-1] ]]) return solution ==== This isn't really too much -- it's generating a small

1 0

Datarray BoF, part2
by Keith Goodman July 21, 2010

July 21, 2010

About a dozen people attended what was billed as a continuation of the SciPy 2010 datarray BoF. We met at UC Berkeley on July 19 as part of the py4science series. A datarray is a subclass of a Numpy array that adds the ability to label the axes and to label the elements along each axis. We spent most of the time discussing how to index with tick labels. The main issue is with integers: is an integer index a tick name or a position index? At the top level, datarrays always use regular Numpy indexing: an int is a position, never a label. So darr[0] always returns the first element of the datarray. The ambiguity occurs in specialized indexing methods that allow indexing by tick label name (because the name could be an int). To break the ambiguity, the proposal was to provide several tick indexing methods[1]: 1. Integers are always labels 2. Integers are never treated as labels 3. Try 1, then 2 We also discussed allowing axis labels to be any hashable object (currently only strings are allowed). The main problem: integers. Currently if an axis is labeled, say, "time", you can do darr.sum(axis="time"). What happens when an axis is labeled with an int? What does the 2 in darr.sum(axis=2) refer to? A position or a label? The same problem exists for floats since a float is (currently) a valid axis for Numpy arrays. References: [1] http://github.com/fperez/datarray/commit/3c5151baa233675b355058eb3ba028d262…

8 14

Re: [Numpy-discussion] doc string linalg.solve --> works for b is matrix
by Mark Bakker July 21, 2010

July 21, 2010

I am using 1.3.0. Glad to hear it is correct in 1.4.0 Sorry for bothering you with an old version, but I am very happy with this feature! Mark > What version of numpy are you using? That docstring was updated in that > fashion about 8 mo. ago (at least in the Wiki; I'm not sure exactly when it > was merged, but it does appear that way in version 1.4.0). > > DG > > I am using linalg.solve to solve a system of linear equations. As I have > to > > solve multiple systems with the same matrix, but different right-hand > sides, > > I tried to make the right-hand side a matrix and that works fine. > > So the docstring should say: > > > > Solve the equation ``a x = b`` for ``x``. > > > > Parameters > > ---------- > > a : array_like, shape (M, M) > > Input equation coefficients. > > b : array_like, shape (M,) or array_like, shape (M,N) > > N can be arbitrary size > > Equation target values. > > > > Returns > > ------- > > x : array, shape (M,) or array, shape (M,N) > > > > Thanks, > > > > Mark > >

2 1

recarray slow?
by wheres pythonmonks July 21, 2010

July 21, 2010

I have an recarray -- the first column is date. I have the following function to compute the number of unique dates in my data set: def byName(): return(len(list(set(d['Date'])) )) Question: is the string 'Date' looked up at each iteration? If so, this is dumb, but explains my horrible performance. Or, is there a better way to code the above? Can I convert this to something indexed by column number and convert 'Date' to column number "0" upfront? Would this help with speed? W

4 9

doc string linalg.solve --> works for b is matrix
by Mark Bakker July 21, 2010

July 21, 2010

Hello list, I am using linalg.solve to solve a system of linear equations. As I have to solve multiple systems with the same matrix, but different right-hand sides, I tried to make the right-hand side a matrix and that works fine. So the docstring should say: Solve the equation ``a x = b`` for ``x``. Parameters ---------- a : array_like, shape (M, M) Input equation coefficients. b : array_like, shape (M,) or array_like, shape (M,N) N can be arbitrary size Equation target values. Returns ------- x : array, shape (M,) or array, shape (M,N) Thanks, Mark

2 1

Complete and build the example from the "Using Numpy C-API" documentation.
by Warren Weckesser July 21, 2010

July 21, 2010

I would like to take the example shown here: http://docs.scipy.org/doc/numpy/user/c-info.how-to-extend.html#example and complete the code so I can build and test the extension module. Is that example currently correct? It is slightly different from the older example in the numpy book, so someone must have had a reason to change it. I've added the C array of PyMethodDefs and the module init function, but I'm still searching for the correct #includes for the numpy stuff. Any suggestions (or pointers to similar simple but complete examples) would be appreciated. Warren

2 3