Mailman 3 August 2006 - NumPy-Discussion

Fastest binary threshold?
by Mark Heslep Aug. 3, 2006

Aug. 3, 2006

I need a binary threshold and numpy.where() seems very slow on numpy 0.9.9.2800: python -m timeit -n 10 -s "import numpy as n;a=n.ones((512,512), n.uint8)*129" "a_bin=n.where( a>128, 128,0)" 10 loops, best of 3: 37.9 msec per loop I'm thinking the conversion of the min, max constants from python ints to n.uint8 might be slowing it down? Is there a better way? Scipy is also an option. Ive search up list quickly and nothing jumps out. For comparison Ive got some ctypes wrapped OpenCv code (that I'd like to avoid) doing the same thing in < 1 msec: Cv images here are unsigned 8 bit as above: python -m timeit -n 50 -s "import cv;sz=cv.cvSize(512,512);a=cv.cvCreateImage(sz, 8, 1); a_bin=cv.cvCreateImage(sz,8,1)" "cv.cvThreshold(a, a_bin, float(128), float(255), cv.CV_THRESH_BINARY )" 50 loops, best of 3: 348 usec per loop And with the Intel IPP optimizations turned on < 0.1msec: python -m timeit -n 50 -s "import cv; sz=cv.cvSize(512,512); a=cv.cvCreateImage(sz, 8, 1); a_bin=cv.cvCreateImage(sz,8,1)" "cv.cvThreshold(a, a_bin, float(128), float(255), cv.CV_THRESH_BINARY )" 50 loops, best of 3: 59.5 usec per loop Regards, Mark

2 3

help! type 'float64scalar' is not type 'float'
by Sebastian Haase Aug. 3, 2006

Aug. 3, 2006

Hi! I just finished maybe a total of 5 hours tracking down a nasty bug. So I thought I would share this: I'm keeping a version of (old) SciPy's 'plt' module around. (I know about matplotlib - anyway - ...) I changed the code some time ago from Numeric to numarray - no problem. Now I switched to numpy ... and suddenly the zooming does not work anymore: it always zooms to "full view". Finally I traced the problem down to a utility function: "is_number" - it is simply implemented as def is_number(val): return (type(val) in [type(0.0),type(0)]) As I said - now I finally saw that I always got False since the type of my number (0.025) is <type 'float64scalar'> and that's neither <type 'float'> nor <type 'int'> OK - how should this have been done right ? Anyway, I'm excited about the new numpy and am looking forward to it's progress Thanks, Sebastian Haase

4 6

Mean of n values within an array
by Phil Ruggera Aug. 3, 2006

Aug. 3, 2006

A variation of the proposed convolve routine is very fast: regular python took: 1.150214 sec. numpy mean slice took: 2.427513 sec. numpy convolve took: 0.546854 sec. numpy convolve noloop took: 0.058611 sec. Code: # mean of n values within an array import numpy, time def nmean(list,n): a = [] for i in range(1,len(list)+1): start = i-n divisor = n if start < 0: start = 0 divisor = i a.append(sum(list[start:i])/divisor) return a t = [1.0*i for i in range(1400)] start = time.clock() for x in range(100): reg = nmean(t,50) print "regular python took: %f sec."%(time.clock() - start) def numpy_nmean(list,n): a = numpy.empty(len(list),dtype=float) for i in range(1,len(list)+1): start = i-n if start < 0: start = 0 a[i-1] = list[start:i].mean(0) return a t = numpy.arange(0,1400,dtype=float) start = time.clock() for x in range(100): npm = numpy_nmean(t,50) print "numpy mean slice took: %f sec."%(time.clock() - start) def numpy_nmean_conv(list,n): b = numpy.ones(n,dtype=float) a = numpy.convolve(list,b,mode="full") for i in range(0,len(list)): if i < n : a[i] /= i + 1 else : a[i] /= n return a[:len(list)] t = numpy.arange(0,1400,dtype=float) start = time.clock() for x in range(100): npc = numpy_nmean_conv(t,50) print "numpy convolve took: %f sec."%(time.clock() - start) def numpy_nmean_conv_nl(list,n): b = numpy.ones(n,dtype=float) a = numpy.convolve(list,b,mode="full") for i in range(n): a[i] /= i + 1 a[n:] /= n return a[:len(list)] t = numpy.arange(0,1400,dtype=float) start = time.clock() for x in range(100): npn = numpy_nmean_conv_nl(t,50) print "numpy convolve noloop took: %f sec."%(time.clock() - start) numpy.testing.assert_equal(reg,npm) numpy.testing.assert_equal(reg,npc) numpy.testing.assert_equal(reg,npn) On 7/29/06, David Grant <davidgrant(a)gmail.com> wrote: > > > > On 7/29/06, Charles R Harris <charlesr.harris(a)gmail.com> wrote: > > > > Hmmm, > > > > I rewrote the subroutine a bit. > > > > > > def numpy_nmean(list,n): > > a = numpy.empty(len(list),dtype=float) > > > > b = numpy.cumsum(list) > > for i in range(0,len(list)): > > if i < n : > > a[i] = b[i]/(i+1) > > else : > > a[i] = (b[i] - b[i-n])/(i+1) > > return a > > > > and got > > > > regular python took: 0.750000 sec. > > numpy took: 0.380000 sec. > > > I got rid of the for loop entirely. Usually this is the thing to do, at > least this will always give speedups in Matlab and also in my limited > experience with Numpy/Numeric: > > def numpy_nmean2(list,n): > > a = numpy.empty(len(list),dtype=float) > b = numpy.cumsum(list) > c = concatenate((b[n:],b[:n])) > a[:n] = b[:n]/(i+1) > a[n:] = (b[n:] - c[n:])/(i+1) > return a > > I got no noticeable speedup from doing this which I thought was pretty > amazing. I even profiled all the functions, the original, the one written by > Charles, and mine, using hotspot just to make sure nothing funny was going > on. I guess plain old Python can be better than you'd expect in certain > situtations. > > -- > David Grant

2 1

Confusion re. version numbers
by Rudolph van der Merwe Aug. 3, 2006

Aug. 3, 2006

Is the current 1.0b1 version of Numpy a maintenace release of the stable 1.0 release, or is it a BETA release for the UPCOMMING 1.0 release of Numpy? -- Rudolph van der Merwe

2 1

unexpected behaviour of numpy.var
by Hanno Klemm Aug. 3, 2006

Aug. 3, 2006

Hello, numpy.var exhibits a rather dangereous behviour, as I have just noticed. In some cases, numpy.var calculates the variance, and in some cases the standard deviation (=square root of variance). Is this intended? I have to admit that I use numpy 0.9.6 at the moment. Has this been changed in more recent versions? Below a sample session Python 2.4.3 (#1, May 8 2006, 18:35:42) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> a = [1,2,3,4,5] >>> numpy.var(a) 2.5 >>> numpy.std(a) 1.5811388300841898 >>> numpy.sqrt(2.5) 1.5811388300841898 >>> a1 = numpy.array([[1],[2],[3],[4],[5]]) >>> a1 array([[1], [2], [3], [4], [5]]) >>> numpy.var(a1) array([ 1.58113883]) >>> numpy.std(a1) array([ 1.58113883]) >>> a =numpy.array([1,2,3,4,5]) >>> numpy.std(a) 1.5811388300841898 >>> numpy.var(a) 1.5811388300841898 >>> numpy.__version__ '0.9.6' Hanno -- Hanno Klemm klemm(a)phys.ethz.ch

9 10

Reverting changes on Wiki, contacting users
by Stephan Tolksdorf Aug. 2, 2006

Aug. 2, 2006

Hi A user named jlc46 is misusing the wiki page "Installing SciPy/Windows" to ask for help on his installation problems. How can I a) contact him in order to ask him to post his questions on the mailing lists, and b) most easily revert changes to wiki-pages? Any hint would be appreciated. Regards, Stephan

5 8

Graph class
by David Grant Aug. 2, 2006

Aug. 2, 2006

I have written my own graph class, it doesn't really do much, just has a few methods, it might do more later. Up until now it has just had one piece of data, an adjacency matrix, so it looks something like this: class Graph: def __init__(self, Adj): self.Adj = Adj I had the idea of changing Graph to inherit numpy.ndarray instead, so then I can just access itself directly rather than having to type self.Adj. Is this the right way to go about it? To inherit from numpy.ndarray? The reason I'm using a numpy array to store the graph by the way is the following: -Memory is not a concern (yet) so I don't need to use a sparse structure like a sparse array or a dictionary -I run a lot of sums on it, argmin, blanking out of certain rows and columns using fancy indexing, grabbing subgraphs using vector indexing -- David Grant http://www.davidgrant.ca

5 7

Int64 and string support for numexpr
by Ivan Vilata i Balaguer Aug. 2, 2006

Aug. 2, 2006

Hi all, I'm attaching some patches that enable the current version of numexpr (r2142) to: 1. Handle int64 integers in addition to int32 (constants, variables and arrays). Python int objects are considered int32 if they fit in 32 bits. Python long objects and int objects that don't fit in 32 bits (for 64-bit platforms) are considered int64. 2. Handle string constants, variables and arrays (not Unicode), with support for comparison operators (==, !=, <, <=, >=, >). (This brings the old ``memsizes`` variable back.) String temporaries (necessary for other kinds of operations) are not supported. The patches also include test cases and some minor corrections (e.g. removing odd carriage returns in some lines in compile.py). There are three patches to ease their individual review: * numexpr-int64.diff only contains the changes for int64 support. * numexpr-str.diff only contains the changes for string support. * numexpr-int64str.diff contains all changes. The task has been somehow difficult, but I think the result fits quite well in numexpr. So, what's your opinion about the patches? Are they worth integrating into the main branch? Thanks! :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ Cárabos Coop. V. V V Enjoy Data ""

2 1

Re: [Numpy-discussion] Release Notes for 1.0 posted
by Tom Loredo Aug. 2, 2006

Aug. 2, 2006

> http://www.scipy.org/ReleaseNotes/NumPy_1.0 > > Please correct problems and add to them as needed. This is incredibly helpful---quite a few things I wasn't aware of. Many, many thanks! -Tom Loredo ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/

1 0

Re: [Numpy-discussion] Release Notes for 1.0 posted
by Nick Fotopoulos Aug. 2, 2006

Aug. 2, 2006

> Message: 2 > Date: Wed, 02 Aug 2006 12:06:45 -0600 > From: Travis Oliphant <oliphant.travis(a)ieee.org> > Subject: [Numpy-discussion] Release Notes for 1.0 posted > To: numpy-discussion <numpy-discussion(a)lists.sourceforge.net> > Message-ID: <44D0E9B5.3080001(a)ieee.org> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > http://www.scipy.org/ReleaseNotes/NumPy_1.0 > > Please correct problems and add to them as needed. > > -Travis > What's not clear to me upon reading this page is what diff set this is describing. Are these the changes between 0.9.8 and 1.0b1? Especially if this page is to be updated with each release, we should be explicit about what changed when. This is a helpful document. Thanks. Take care, Nick

1 0