Mailman 3 September 2008 - NumPy-Discussion

Interesting bug with numpy.ma
by Pierre GM Sept. 27, 2008

Sept. 27, 2008

All, I've just been pointed to a strange bug in numpy.ma import numpy as np import numpy.ma as ma assert(ma.masked*float(1) is ma.masked) assert(float(1)*ma.masked is ma.masked) assert(ma.masked*np.float(1) is ma.masked) assert(np.float(1)*ma.masked is ma.masked) assert(ma.masked*np.float_(1) is ma.masked) assert(np.float_(1)*ma.masked is ma.masked) The last assertion fails, and I have no idea why. What's the difference between assertions 3 and 5 ? FYI, ma.masked=ma.MaskedArray(0.,dtype=np.float_,mask=True), and MaskedArray has a __array_priority__ of 15. Implementing a __rmul__ doesn't work either. Any idea welcome.

1 0

Bug fix or behavior change?
by Charles R Harris Sept. 27, 2008

Sept. 27, 2008

Hi All, Currently subtract for boolean arrays is defined in /**begin repeat * Arithmetic operators * * # OP = ||, ^, &&# * #kind = add, subtract, multiply# */ static void BOOL_@kind@(char **args, intp *dimensions, intp *steps, void *func) { register intp i; intp is1=steps[0],is2=steps[1],os=steps[2], n=dimensions[0]; char *i1=args[0], *i2=args[1], *op=args[2]; for(i=0; i<n; i++, i1+=is1, i2+=is2, op+=os) { *((Bool *)op)=*((Bool *)i1) @OP@ *((Bool *)i2); } } /**end repeat**/ Note that this might yield unexpected results if the boolean value is not 0 or 1, which is possible using views. Note also that bitwise_xor converts the boolean to 0 or 1 before using ^. I think subtract should work the same way, but that would change current behavior. So... do I make the change and call it a bug fix or leave it as it is? Chuck

2 3

ANNOUNCE: EPD (Enthought Python Distribution) 4.0.300 Beta 3 available
by Dave Peterson Sept. 27, 2008

Sept. 27, 2008

Hello, We've recently posted the third beta release of EPD (the Enthought Python Distribution) with Python 2.5 version 4.0.300. You may download the beta from here: http://www.enthought.com/products/epdbeta.php Please help us test it out and provide feedback on the EPD Trac instance: https://svn.enthought.com/epd You can check out the release notes here: http://www.enthought.com/products/epdbetareleasenotes.php About EPD --------- The Enthought Python Distribution (EPD) is a "kitchen-sink-included" distribution of the Python™ Programming Language, including over 60 additional tools and libraries. The EPD bundle includes NumPy, SciPy, IPython, 2D and 3D visualization, database adapters, and a lot of other tools right out of the box. http://www.enthought.com/products/epd.php It is currently available as a single-click installer for Windows XP (x86), Mac OS X (a universal binary for OS X 10.4 and above), and RedHat 3 and 4 (x86 and amd64). EPD is free for academic use. An annual subscription and installation support are available for individual commercial use. An enterprise subscription with support for particular deployment environments is also available for commercial purchase. The beta versions of EPD are available for indefinite free trial. -- Dave

1 0

bug in fromiter - with patch
by Russel Howe Sept. 27, 2008

Sept. 27, 2008

As of SVN from this morning, fromiter cannot create an array whose dtype is a byte type In [1]: np.fromiter(range(10), dtype='b') --------------------------------------------------------------------------- MemoryError Traceback (most recent call last) /Users/baxter/<ipython console> in <module>() MemoryError: cannot allocate array memory I am not sure how to include a test whose failure condition is the raising of an exception, but the patch includes the above test. Russel Index: numpy/core/src/multiarraymodule.c =================================================================== --- numpy/core/src/multiarraymodule.c (revision 5866) +++ numpy/core/src/multiarraymodule.c (working copy) @@ -6503,7 +6503,8 @@ PyObject *value; PyObject *iter = PyObject_GetIter(obj); PyArrayObject *ret = NULL; - intp i, elsize, elcount; + intp i; + uintp elsize, elcount; char *item, *new_data; if (iter == NULL) goto done; Index: numpy/core/tests/test_regression.py =================================================================== --- numpy/core/tests/test_regression.py (revision 5866) +++ numpy/core/tests/test_regression.py (working copy) @@ -1208,5 +1208,11 @@ a = np.array(1) self.failUnlessRaises(ValueError, lambda x: x.choose([]), a) + def test_fromiter_comparison(self, level=rlevel): + a = np.fromiter(np.range(10), dtype='b') + b = np.fromiter(np.range(10), dtype='B') + assert np.alltrue(a == np.array([0,1,2,3,4,5,6,7,8,9])) + assert np.alltrue(b == np.array([0,1,2,3,4,5,6,7,8,9])) + if __name__ == "__main__": run_module_suite()

1 1

Proper NaN handling in max and co: a redux
by David Cournapeau Sept. 26, 2008

Sept. 26, 2008

Hi, We started a small document to gather all information given in previous threads about NaN handling in max and co in numpy. Thanks to Anne (Archibald) for useful comments/additions/typo corrections: http://projects.scipy.org/scipy/numpy/wiki/ProperNanHandling We describe approaches taken by R and matlab for the relevant functions, and possible approaches to take in numpy. I have an patch almost ready (with tests) for the approach "Returning NaN" (Chuck said he would be willing to implement the sort/argsort part, which is the part missing in my current implementation). I think it would be nice for 1.3.0, cheers, David

4 6

vectorize in new.instancemethod and *args
by joep Sept. 26, 2008

Sept. 26, 2008

I have a question about the use of vectorize in new.instancemethod: Using vectorize with *args requires adjustment to the number of arguments, nin = nargs. This works when it is used with a function. However, I don't manage to set nin when using vectorize with a method created with new.instancemethod. I would like to do: def funcm(self,x,*args): ... class D(object): def __init__(self): vecfunc = vectorize(funcm) vecfunc.nin = 2 self.funcm = new.instancemethod(vecfunc,self,A) But, calling D().funcm still causes a value error. I managed to do it by setting vecfunc.nin = None. What is the correct or best way to do this? I needed it when I wanted to correct scipy.stats.distributions. Below (and as attachment) is a script that summarizes what I figured out about the use of vectorize with *args. I did not find any documentation for this case, I got the information by trial and error and hints in the bug reports. Josef ''' Nin adjustment in np.vectorize with *args ========================================= * Problem: simple use of vectorize with *args raises exception:: File "C:\Programs\Python24\Lib\site-packages\numpy\lib \function_base.py", line 1636, in __call__ raise ValueError, "mismatch between python function inputs"\ ValueError: mismatch between python function inputs and received arguments * solution adjust nin directly, either set nin to the correct number of arguments or set nin to None * with functions both ways of adjusting nin work, (case 2 and 3) * with new.instance method, the only way, I managed to get it to work is by setting nin = None (case E). Setting nin = nargs did not work (case C and D) with functions -------------- * case 0: function without vectorize * case 1: function with vectorize -> broken * case 2: function with vectorize, with nin adjustment -> works * case 3: function with vectorize, with nin adjustment, nin = None -> works with class and new.instancemethod --------------------------------- * case A: class without vectorize -> works * case B: class with vectorize -> broken: argument miss match * case C: class with vectorize with nin adjustment -> broken * case D: class with vectorize with nin adjustment -> broken * case E: class with vectorize with nin adjustment, nin = None -> works Motivation: ----------- vectorize (sgf) in scipy.stats.distribution does not work in cases where there is no correct nin adjustment. Example for new.instancemethod, that looks broken, is self._ppf in class rv_discrete. ''' import new import numpy as np from numpy import vectorize def func1(x,*args): print 'args = ', args print 'x = ', x return np.sum(x) def funcm(self,x,*args): ''' function for use as instance method''' print 'args = ', args print 'x = ', x return np.sum(x) # case 0: function without vectorize print 'func1(1,*(2,))' func1(1,*(2,)) # case 1: function with vectorize -> broken vecfunc = vectorize(func1) print 'vecfunc(1):' vecfunc(1) print 'vecfunc(1,3):' try: vecfunc(1,3) except ValueError,e: print e print 'vecfunc(1,*(2,))' try: vecfunc(1,*(2,)) except ValueError,e: print e # case 2: function with vectorize, with nin adjustment -> works vecfunc2 = vectorize(func1) vecfunc2.nin = 2 print 'vecfunc2(1):' vecfunc2(1) print 'vecfunc2(1,3):' vecfunc2(1,3) print 'vecfunc2(1,*(2,))' vecfunc2(1,*(2,)) # case 3: function with vectorize, with nin adjustment, nin = None -> works vecfunc3 = vectorize(func1) vecfunc3.nin = None print 'vecfunc3(1):' vecfunc3(1) print 'vecfunc3(1,3):' vecfunc3(1,3) print 'vecfunc3(1,*(2,))' vecfunc3(1,*(2,)) print 'vecfunc3(1,*(2,5))' vecfunc3(1,*(2,5)) # with class and new.instancemethod # case A: class without vectorize -> works class A(object): def __init__(self): self.funcm = new.instancemethod(funcm,self,A) aa = A() #print dir(aav) print 'A: aav.funcm(5)' aa.funcm(5) print 'A: aav.funcm(5,2)' aa.funcm(5,2) print 'A: aav.funcm(5,*(2,))' aa.funcm(5,*(2,)) # case B: class with vectorize -> broken: argument miss match class B(object): def __init__(self): self.funcm = new.instancemethod(vectorize(funcm),self,A) aav = B() #print dir(aav) print 'B: aav.funcm(5)' aav.funcm(5) print 'B: aav.funcm(5,2)' try: aav.funcm(5,2) except ValueError,e: print e print 'B: aav.funcm(5,*(2,))' try: aav.funcm(5,*(2,)) except ValueError,e: print e # case C: class with vectorize with nin adjustment -> broken # AttributeError: 'instancemethod' object has no attribute 'nin' class C(object): def __init__(self): self.funcm = new.instancemethod(vectorize(funcm),self,A) #self.funcm.nin = 2 try: self.funcm.nin = 2 except AttributeError,e: print e aav = C() #print dir(aav) print 'C: aav.funcm(5)' aav.funcm(5) print 'C: aav.funcm(5,2)' try: aav.funcm(5,2) except ValueError,e: print e print 'C: aav.funcm(5,*(2,))' try: aav.funcm(5,*(2,)) except ValueError,e: print e # case D: class with vectorize with nin adjustment -> broken # nin is not correctly used by vectorize class D(object): def __init__(self): # define the vectorized function vecfunc = vectorize(funcm) vecfunc.nin = 2 self.funcm = new.instancemethod(vecfunc,self,A) #self.funcm.nin = 2 try: self.funcm.nin = 2 except AttributeError,e: print e aav = D() #print dir(aav) print 'D: aav.funcm(5)' aav.funcm(5) print 'D: aav.funcm(5,2)' try: aav.funcm(5,2) except ValueError,e: print e print 'D: aav.funcm(5,*(2,))' try: aav.funcm(5,*(2,)) except ValueError,e: print e # case E: class with vectorize with nin adjustment, nin = None -> works # nin is calculated by vectorize class E(object): def __init__(self): vecfunc = vectorize(funcm,otypes='d') vecfunc.nin = None # remove nin at let vectorize do the work self.funcm = new.instancemethod(vecfunc,self,A) #self.funcm.nin = 2 try: self.funcm.nin = 2 except AttributeError,e: print e aav = E() #print dir(aav) print 'E: aav.funcm(5)' aav.funcm(5) print 'E: aav.funcm(5,2)' try: aav.funcm(5,2) except ValueError,e: print e print 'E: aav.funcm(5,*(2,))' try: aav.funcm(5,*(2,)) except ValueError,e: print e aav.funcm([1,2,5.2]) aav.funcm([1,2,5.2],*(2,)) res = aav.funcm(np.array([1,2,5.2]),*(2,)) print res

1 1

maskedarray: how to force mask to expand
by Vincent Schut Sept. 26, 2008

Sept. 26, 2008

Probably I'm just overlooking something obvious, but I'm having problems with maskedarrays (numpy.ma from svn: '1.3.0.dev5861'), the mask by default being a single bool value ('False') instead of a properly sized bool array. If I then try to mask one value by assigning values to certain mask positions (a.mask[0,0]=True) I get an error, logically. I know I can use mask_where, but I like the mask[...] idiom. And I have to expand the mask anyway, as I'm gonna write it to a file at the end. 1) Is there a way to have ma always use properly expanded masks (bool arrays instead of single bool values)? I tried the shrink=False keyword, but that does not do what I want, and is not available for numpy.ma.zeros, which I conveniently use a lot. 2) Is there a method/function to request the mask, be it a single bool value or an array, as a properly sized array? I found shrink_mask but no opposite method, and shrink_mask seems to do something subtly different even. Regards, Vincent.

2 5

Cython: Passing numpy array into C via buffer?
by Richard Shaw Sept. 26, 2008

Sept. 26, 2008

Hello, Forgive me if this is a stupid question, I've been looking around all the Cython documentation and I can't find out if this is possible. What I would like to do is generally is wrap a C function that takes a double array, and be able to pass in a numpy array, I was wondering if it's possible to do this using the buffer interface? I understand I could do it using this method (http://wiki.cython.org/WrappingNumpy), but the buffer interface seems much neater. Specifically, I have a C function that I would like to wrap with signature: #### cfile.h double do_something(double * array, int len); I was hoping with the buffer interface, I could do something like this: #### pyfile.pyx import numpy cimport numpy cdef extern from "cfile.h" double do_something(double * array, int len) cdef do_something_python(numpy.ndarray[numpy.float64, ndim = 1] numarr): cdef double x cdef double * data cdef int l data = ... # Not sure what to do here l = numarr.shape[0] x = do_something(data, l) The above probably doesn't work for many reasons (I've just jotted it down), but the think I can't figure out is what to do at the "data = ..." line. Any help would be greatly appreciated! Thanks, Richard

2 3

Help speeding up element-wise operations for video processing
by brendan simons Sept. 26, 2008

Sept. 26, 2008

Hi everyone I have a quick problem. I'm trying to write a routine to "demosaic" video data extremely quickly. I thought my algorithm would make good use of my processor's vector math units, but it isn't fast enough (it currently takes about 300ms for each 8 megapixel frame) Maybe you can help me speed it up? Lets just look at one piece of the algorithm - interpolating the green pixels: My camera hands me a frame as an int* c array of 8 bit integers. Forget the problem of turning that into a numpy array for now and assume I have the image as a numpy uint8 array "ain" . Also assume that Greenmask is a precomputed array of 1s and 0s of the same size as ain. Now, I do the following: #interpolate the green pixels from the bayer filter image ain g = greenMask * ain gi = g[:-2, 1:-1].astype('uint16') gi += g[2:, 1:-1] gi += g[1:-1, :-2] gi += g[1:-1, 2:] gi /= 4 gi += g[1:-1, 1:-1] return gi I do something similar for red and blue, then stack the interpolated red, green and blue integers into an array of 24 bit integers and blit to the screen. I was hoping that none of the lines would have to iterate over pixels, and would instead do the adds and multiplies as single operations. Perhaps numpy has to iterate when copying a subset of an array? Is there a faster array "crop" ? Any hints as to how I might code this part up using ctypes? Thanks in advance, Brendan __________________________________________________________________ Ask a question on any topic and get answers from real people. Go to Yahoo! Answers and share what you know at http://ca.answers.yahoo.com

8 10

lsame_ undefined ...
by thorsten＠partita.dartmouth.edu Sept. 26, 2008

Sept. 26, 2008

I've been trying to install numpy on my linux system and get the following error message whey I try to import it: Python 2.4.3 (#1, Apr 22 2006, 18:02:44) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-54)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/local/lib/python2.4/site-packages/numpy/__init__.py", line 103, in ? import linalg File "/usr/local/lib/python2.4/site-packages/numpy/linalg/__init__.py", line 4, in ? from linalg import * File "/usr/local/lib/python2.4/site-packages/numpy/linalg/linalg.py", line 29, in ? from numpy.linalg import lapack_lite ImportError: /usr/local/lib/python2.4/site-packages/numpy/linalg/lapack_lite.so: undefined symbol: lsame_ >>> Is there a way to fix this? Thanks, John Thorstensen -- ---------------------------------------------------------------------------- John R. Thorstensen Department of Physics and Astronomy Dartmouth College 6127 Wilder Laboratory Hanover, NH 03755 Office: 239 Wilder Hall Phone: voice 603-646-2869 FAX 603-646-1446 Hinman box 6127 ----------------------------------------------------------------------------

3 9