Mailman 3 February 2003 - NumPy-Discussion

Tentative fix for Numtut's view.py
by Fernando Perez 28 Feb '03

28 Feb '03

Hi all, > Subject: [Numpy-discussion] Last call for v. 23 > > I am going to make a release of Numeric, 23.0. Fellow developers who are inspired > to fix a bug are urged to do so immediately. > > This will be a bug fix release. in the scipy mailing list there were some discussions about view.py as included in NumTut. It seems that many folks (including myself) have had problems with it, and they seem to be threading-related. The symptom is that once view is imported, the interactive interpreter essentially locks up, and typing becomes nearly impossible. I know next to nothing about threading, but in an attempt to fix the problem I stripped view.py bare of everything I didn't understand, until it worked :) Basically I removed all PIL and threading-related code, and left only the bare Tk code in place. Naive as this approach was, it seems to have worked. Some folks reported success, and David Ascher (the original author of view.py) suggested I submit this to the Numpy team as an update to the tutorial. There's a good chance the current view is just broken and nobody has bothered to use it in a long time. I'm attaching the new view here as a file, but if there is a different protocol I should follow, please let me know (patch, etc). As I said, this was the most simple-minded thing I could do to make it work. So if you are interested in accepting this, it might be wise to have a look at it first. On the upside, pretty much all I did was to _remove_ code, not to add anything. So the analysis should be easy (the new code is far simpler and shorter than the original). I've tested it personally under python 2.2.1 (the stock Redhat 8.0 install). Best, Fernando Perez. """ Displaying image files in a separate thread on Tk+thread, w/ xv in forked & execv'ed processes otherwise. view(array): will spawn a displaying program for arrays which are either NxM or NxMx3. does the 'min/max' and conversion to char. array2ppm(array): given an NxM or NxMx3 array, returns a ppm string which is a valid thing to put in a PPM file. (or PGM file if NxM file). TODO: - automatic scaling for small images - accept rank-1 arrays NOTE: This is a modified version which removes all threading and PIL support. It should work on any system with Tkinter alone. """ DEFAULT_HEIGHT = 255 MINSIZE = 150 import os import Tkinter from Numeric import * import tempfile, time def save_ppm(ppm, fname=None): if fname == None: fname = tempfile.mktemp('.ppm') f = open(fname, 'wb') f.write(ppm) f.close() return fname def array2ppm(image): # scaling if len(image.shape) == 2: # B&W: image = transpose(image) return "P5\n#PPM version of array\n%d %d\n255\n%s" % \ (image.shape[1], image.shape[0], ravel(image).tostring()) else: # color image = transpose(image, (1, 0, 2)) return "P6\n%d %d\n255\n%s" % \ (image.shape[1], image.shape[0], ravel(image).tostring()) def preprocess(image, (scalex,scaley)): assert len(image.shape) in (1, 2) or \ len(image.shape) == 3 and image.shape[2] == 3, \ "image not correct format" themin = float(minimum.reduce(ravel(image))) themax = float(maximum.reduce(ravel(image))) if len(image.shape) == 1: len_x = image.shape[0] ys = ((image - themin)/(themax-themin)*(DEFAULT_HEIGHT-1)).astype('b') image = (zeros((DEFAULT_HEIGHT, len_x))+255).astype('b') for x in range(len_x): image[DEFAULT_HEIGHT-1-ys[x],len_x-x-1] = 0 image = transpose(image) elif image.typecode() != 'b': image = (image - themin) / (themax-themin) * 255 image = image.astype('b') len_x, len_y = image.shape[:2] if scalex is None: if len_x < MINSIZE: scalex = int(float(MINSIZE) / len_x) + 1 else: scalex = 1 if scaley is None: if len_y < MINSIZE: scaley = int(float(MINSIZE) / len_y) + 1 else: scaley = 1 return image, (scalex, scaley) class PPMImage(Tkinter.Label): def __init__(self, master, ppm, (scalex, scaley)): self.image = Tkinter.PhotoImage(file=save_ppm(ppm)) w, h = self.image.width(), self.image.height() self.image = self.image.zoom(scalex, scaley) self.image.configure(width=w*scalex, height=h*scaley) Tkinter.Label.__init__(self, master, image=self.image, bg="black", bd=0) self.pack() # Start the Tk process from which all subsequent windows will be opened. def tk_root(): if Tkinter._default_root is None: root = Tkinter.Tk() Tkinter._default_root.withdraw() else: root = Tkinter._default_root return root _root = tk_root() def view(image, scale=None): """Display an image, optionally rescaling it. scale can be either an integer or a tuple of 2 integers (for separate x/y rescaling. """ if scale is None: scale=(None,None) else: try: len(scale) except TypeError: scale = (scale,scale) image, scales = preprocess(image, scale) tl = Tkinter.Toplevel() u = PPMImage(tl, array2ppm(image), scales) u.pack(fill='both', expand=1) u.tkraise()

1 0

RE: [Numpy-discussion] BLAS
by Paul F Dubois 28 Feb '03

28 Feb '03

I had forgotten about this case. I think when these were done it was thought that it would be better if the Numeric core did not require use of LAPACK/BLAS. We were thinking back then of a core with other packages, and the blas we use by default is probably the same speed so it didn't seem important. I would have no problem with a patch to change this. > -----Original Message----- > From: Jens Jorgen Mortensen [mailto:jensj@fysik.dtu.dk] > Sent: Thursday, February 20, 2003 9:00 AM > To: Paul F Dubois > Subject: Re: [Numpy-discussion] BLAS > > > On Torsdag den 20. februar 2003 17:49, Paul F Dubois wrote: > > > > When doing matrix-matrix multiplications with large > matrices, using > > > the BLAS library (Basic Linear Algebra Subprograms) can speed up > > > things a lot. I don't > > > think Numeric takes advantage of this (is this correct?). > > > > No. You can configure it at installation to use the BLAS of choice. > > > > I know that the stuff in LinearAlgebra can be configured to > use a BLAS of > choice, but what about the Numeric.dot function? > > Can I configure Numeric so that this: > > >>> a = Numeric.dot(b, c) > > will use BLAS? > > Jens > >

2 1

Re: [Numpy-discussion] speedy remove mean of rows
by Eric Jones 27 Feb '03

27 Feb '03

Hey John, I think broadcasting is your best bet. Here is a snippet using scipy (Numeric will be pretty much the same). >>> from scipy import * >>> a = stats.random((4,3)) a array([[ 0.94058263, 0.24342623, 0.74673623], [ 0.53151542, 0.07523929, 0.49730805], [ 0.5161854 , 0.51049614, 0.70360875], [ 0.09470515, 0.60604334, 0.64941102]]) >>> stats.mean(a) # axis=-1 by default in scipy array([ 0.6435817 , 0.36802092, 0.57676343, 0.45005317]) >>> a-stats.mean(a)[:,NewAxis] array([[ 0.29700093, -0.40015546, 0.10315453], [ 0.1634945 , -0.29278163, 0.12928713], [-0.06057803, -0.06626729, 0.12684532], [-0.35534802, 0.15599017, 0.19935785]]) eric John Hunter <jdhunter(a)ace.bsd.uchicago.edu> wrote .. > > I have a large (40,000 x 128) Numeric array, X, with typecode Float. > In some cases the number of rows may be approx 10x greater. > > I want to create an array Y with the same dimensions as X, where each > element of Y is the corresponding element of X with the mean of the > row on which it occurs subtracted away. Ie, > > Y = X - transpose(resize(mean(X,1), (X.shape[1],X.shape[0]))) > > I am wondering if this is the most efficient way (speed and memory). > > Thanks for any suggestions, > John Hunter > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion

1 0

speedy remove mean of rows
by John Hunter 27 Feb '03

27 Feb '03

I have a large (40,000 x 128) Numeric array, X, with typecode Float. In some cases the number of rows may be approx 10x greater. I want to create an array Y with the same dimensions as X, where each element of Y is the corresponding element of X with the mean of the row on which it occurs subtracted away. Ie, Y = X - transpose(resize(mean(X,1), (X.shape[1],X.shape[0]))) I am wondering if this is the most efficient way (speed and memory). Thanks for any suggestions, John Hunter

1 0

Last call for v. 23
by Paul F. Dubois 27 Feb '03

27 Feb '03

I am going to make a release of Numeric, 23.0. Fellow developers who are inspired to fix a bug are urged to do so immediately. This will be a bug fix release.

1 0

Re: filtering numeric arrays
by Gordon Williams 27 Feb '03

27 Feb '03

Thanks to Tim and Chris. Just what I was looking for! I tested both along with some other tries that I had made. For 10000 points - G:\GPS\Python\GUI_Test\Filter>speed.py time for <function listComp at 0x007AE4D8> is 0.022809 time for <function arraykludge at 0x0092AFF0> is 0.060303 time for <function arrayComp at 0x0092AFB0> is 0.055692 time for <function arrayTimH at 0x0092B030> is 0.003652 time for <function arrayChrisB at 0x0092B070> is 0.003561 For 100 points - G:\GPS\Python\GUI_Test\Filter>speed.py time for <function listComp at 0x007AE4D8> is 0.000238 time for <function arraykludge at 0x00804BD8> is 0.000784 time for <function arrayComp at 0x00804B98> is 0.000678 time for <function arrayTimH at 0x00804C18> is 0.000376 time for <function arrayChrisB at 0x00804C58> is 0.000153 They scale slightly differently between Tim's and Chris' methods. Thanks again, Gordon Williams Here is the code (since someone will ask): '''Test the speed of different methods of getting points out of a list''' import time import Numeric as n size= 100 maxNum=size/10. #data a= n.array(n.arange(0,maxNum,.05)) a.shape= (size,2) #list l= a.tolist() (xMin,yMin)= (3,2) (xMax,yMax)= (4,6) def listComp(seq): '''using list comprehension''' return [(x,y) for x,y in seq if xMin < x <xMax and yMin< y <yMax] def arrayComp(seq): '''using list comprehension''' return n.array([(x,y) for x,y in seq if xMin < x <xMax and yMin< y <yMax]) def arraykludge(seq): '''using numeric stuff''' res= n.copy.deepcopy(seq) i=0 #index for x,y in seq: if xMin < x <xMax and yMin< y <yMax: res[i]= [x,y] i+=1 return res[:i] def arrayTimH(seq): '''using Tim Hochberg suggestions''' cond = (xMin < seq[:,0]) & (seq[:,0] < xMax) & (yMin < seq[:,1]) & (seq[:,1] < yMax) return n.compress(cond, seq, 0) def arrayChrisB(seq): '''using Chris Barker suggestions''' valid = (seq[:,0] > xMin) & (seq[:,0] < xMax) & (seq[:,1] > yMin) & (seq[:,1] < yMax) return n.take(a,n.nonzero(valid)) #Tests tests= [(listComp,l), (arraykludge,a),(arrayComp,a), (arrayTimH,a), (arrayChrisB,a)] for fun,seq in tests: t=time.clock() apply(fun,(seq,)) dt= time.clock()-t print "time for %s is %f" %(str(fun),dt) ----- Original Message ----- From: "Gordon Williams" <g_will(a)cyberus.ca> To: <numpy-discussion(a)lists.sourceforge.net> Sent: Thursday, February 27, 2003 2:05 PM Subject: filtering numeric arrays > Hi All, > > I have an 2D numeric array of x,y points eg [(1,3),(2,4),(5,6)] and I would > like to remove all the points in the array that don't meet the min/max point > criteria. I will have several thousand points. With lists I can do it like > > [(x,y) for x,y in seq if xMin < x <xMax and yMin< y <yMax] > > How do I get the same functionality and better speed using numeric. I have > tried a bunch of things using compress and take but I am running up against > a brick wall. > > > Any ideas? > > Thanks > > Gordon Williams > > > >

1 0

filtering numeric arrays
by Gordon Williams 27 Feb '03

27 Feb '03

Hi All, I have an 2D numeric array of x,y points eg [(1,3),(2,4),(5,6)] and I would like to remove all the points in the array that don't meet the min/max point criteria. I will have several thousand points. With lists I can do it like [(x,y) for x,y in seq if xMin < x <xMax and yMin< y <yMax] How do I get the same functionality and better speed using numeric. I have tried a bunch of things using compress and take but I am running up against a brick wall. Any ideas? Thanks Gordon Williams

3 2

Re: [Numpy-discussion] make C array accessible to python without copy
by Sebastian Haase 21 Feb '03

21 Feb '03

Short follow up: 1) Is it planned to support this more directly? 2) How much does it cost to create a buffer object if it uses my already allocated memory ? 3) Can I change the pointer so that it points to a different memory space WITHOUT having to recreate any python objects? Or would that "confuse" the buffer or numarray? (We are hoping to aquire 30 images per second - the images should get written into a circular buffer so that the data can be written to disk in larger chunks - but the python array should always refer to the current image ) Thanks for all the nice toys (tools) ;-) Sebastian Haase On Fri, 17 Jan 2003 18:16:01 -0500 Todd Miller <jmiller(a)stsci.edu> wrote: >Sebastian Haase wrote: > >>Hi, >>What is the C API to make an array that got allocated, >>let's say, by a = new short[512*512], >>accessible to python as numarray. >> >What you want to do is not currently supported well in C. > The way to do what you want is: > >1. Create a buffer object from your C++ array. The >buffer object can be built such that it refers to the >original copy of the data. > >2. Call back into Python (numarray.NumArray) with your >buffer object as the buffer parameter. > >You can scavenge the code in NA_newAll (Src/newarray.ch) >for most of the callback. > >>I tried NA_New - but that seems to make a copy. >>I would need it to use the original memory space >>so that I can "observe" the array from Python WHILE >>the underlying C array changes (it's actually a camera >>image) >> >That sounds cool! > >> >>Thanks, >>Sebastian Haase

1 0

Non-regular lists in numarray
by Francesc Alted 21 Feb '03

21 Feb '03

Hi, I've found that numarray.array doesn't check enough the input for non-regular objects. For example: In [95]: numarray.array([3., [4, 5.2]]) Out[95]: array([ 3. , 5.7096262]) but, In [96]: Numeric.array([3., [4, 5.2]]) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) ? TypeError: bad argument type for built-in operation I find Numeric behaviour more appropriate. Regards, -- Francesc Alted

2 1

BLAS
by Jens Jorgen Mortensen 20 Feb '03

20 Feb '03

Hi, When doing matrix-matrix multiplications with large matrices, using the BLAS library (Basic Linear Algebra Subprograms) can speed up things a lot. I don't think Numeric takes advantage of this (is this correct?). Will numarray be able to do that? Jens

4 3