Mailman 3 December 2006 - NumPy-Discussion

Unexpected output using numpy.ndarray and __radd__
by Mark Hoffmann 15 Dec '06

15 Dec '06

Hi, The following issue has puzzled me for a while. I want to add a numpy.ndarray and an instance of my own class. I define this operation by implementing the methods __add__ and __radd__. My programme (including output) looks like: #!/usr/local/bin/python import numpy class Cyclehist: def __init__(self,vals): self.valuearray = numpy.array(vals) def __str__(self): return 'Cyclehist object: valuearray = '+str(self.valuearray) def __add__(self,other): print "__add__ : ",self,other return self.valuearray + other def __radd__(self,other): print "__radd__ : ",self,other return other + self.valuearray c = Cyclehist([1.0,-21.2,3.2]) a = numpy.array([-1.0,2.2,-2.2]) print c + a print a + c # ---------- OUTPUT ---------- # # addprob $ addprob.py # __add__ : Cyclehist object: valuearray = [ 1. -21.2 3.2] [-1. 2.2 -2.2] # [ 0. -19. 1.] # __radd__ : Cyclehist object: valuearray = [ 1. -21.2 3.2] -1.0 # __radd__ : Cyclehist object: valuearray = [ 1. -21.2 3.2] 2.2 # __radd__ : Cyclehist object: valuearray = [ 1. -21.2 3.2] -2.2 # [[ 0. -22.2 2.2] [ 3.2 -19. 5.4] [ -1.2 -23.4 1. ]] # addprob $ # # ---------------------------- I expected the output of "c+a" and "a+c" to be identical, however, the output of "a+c" gets nested in an elementwise fashion. Can anybody explain this? Is it a bug or a feature? I'm using Python 2.4.4c1 and numpy 1.0. I tried the programme using an older version of Python and numpy and there the result of "c+a" and "a+c" are identical. Regards, Mark Hoffmann

1 0

migrating to numpy
by Christian Meesters 15 Dec '06

15 Dec '06

Hi I was looking for a guide how to migrate old numeric code to numpy and couldn't find anything on the web. Any pointers for me? TIA and sorry for bothering, since this surely was discussed already, Christian

2 1

Histograms of extremely large data sets
by Cameron Walsh 15 Dec '06

15 Dec '06

Hi all, I'm trying to generate histograms of extremely large datasets. I've tried a few methods, listed below, all with their own shortcomings. Mailing-list archive and google searches have not revealed any solutions. Method 1: import numpy import matplotlib data=numpy.empty((489,1000,1000),dtype="uint8") # Replace this line with actual data samples, but the size and types are correct. histogram = pylab.hist(data, bins=range(0,256)) pylab.xlim(0,256) pylab.show() The problem with this method is it appears to never finish. It is however, extremely fast for smaller data sets, like 5x1000x1000 (1-2 seconds) instead of 500x1000x1000. Method 2: import numpy import matplotlib data=numpy.empty((489,1000,1000),dtype="uint8") # Replace this line with actual data samples, but the size and types are correct. bins=numpy.zeros((256),dtype="uint32") for val in data.flat: bins[val]+=1 barchart = pylab.bar(xrange(256),bins,align="center") pylab.xlim(0,256) pylab.show() The problem with this method is it is incredibly slow, taking up to 30 seconds for a 1x1000x1000 sample, I have neither the patience nor the inclination to time a 500x1000x1000 sample. Method 3: import numpy data=numpy.empty((489,1000,1000),dtype="uint8") # Replace this line with actual data samples, but the size and types are correct. a=numpy.histogram(data,256) The problem with this one is: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.5/site-packages/numpy/lib/function_base.py", line 96, in histogram n = sort(a).searchsorted(bins) ValueError: dimensions too large. It seems that iterating over the entire array and doing it manually is the slowest possible method, but that the rest are not much better. Is there a faster method available, or do I have to implement method 2 in C and submit the change as a patch? Thanks and best regards, Cameron.

8 21

Can we change how fortran compiler version strings are handled?!
by Brian Granger 14 Dec '06

14 Dec '06

Hi, I have been doing quite a bit of numpy evangelism here at my work and slowly people are starting to use it. One of the main things people are interested in is f2py. But, I am finding that there is one persistent problem that keeps coming up when people try to install numpy on various systems: In three cases I have found that numpy failed to find and use a fortran compiler because the version string didn't match what was hardcoded into numpy.distutils. The reality is that version strings are in no way "standardized". In the most recent cases, we had a version of the lahey compiler that had the extra word "Express" and in another case, the xlf version string on a supercomputer was completely different. What is crazy to me is that this simple mismatch prevents numpy from even trying the compiler. Can we please change how Numpy handles the version string of fortran compilers? My suggestion would be to simply print the version string, but to attempt to use the compiler no matter what the version string is. That way, the success or failure of using the fortran compiler will be determined by the actual compiler, not its version string. There could be some other smart way of handling this, but I think it should be dealt with to make the installation process easier. I am willing to work up a patch if there is agreement on what should be done. Oh, the other difficult thing is that in the current arrangement, numpy.distutils doesn't print an error message that is easy to debug. It just silently does find the compiler rather than saying why. Thanks Brian

2 1

Pyrex and numpy
by Tim Hochberg 14 Dec '06

14 Dec '06

I was just going to try pyrex out with numpy to see how it compares with weave (which is cool but quirky). My first attempt ended in failure: I tried to compile the demo in in numpy/doc/pyrex and got this error: c_numpy.pxd:99:22: Array element cannot be a Python object Does anyone who uses pyrex see this? Does anyone know what it's from? Not that I deleted numpyx.c, since otherwise pyrex isn't invoked at all? -tim

2 1

linalg.lstsq for complex
by Bill Baxter 13 Dec '06

13 Dec '06

Is this code from linalg.lstsq for the complex case correct? lapack_routine = lapack_lite.zgelsd lwork = 1 rwork = zeros((lwork,), real_t) work = zeros((lwork,),t) results = lapack_routine(m, n, n_rhs, a, m, bstar, ldb, s, rcond, 0, work, -1, rwork, iwork, 0) lwork = int(abs(work[0])) rwork = zeros((lwork,),real_t) a_real = zeros((m,n),real_t) bstar_real = zeros((ldb,n_rhs,),real_t) results = lapack_lite.dgelsd(m, n, n_rhs, a_real, m, bstar_real, ldb, s, rcond, 0, rwork, -1, iwork, 0) lrwork = int(rwork[0]) work = zeros((lwork,), t) rwork = zeros((lrwork,), real_t) results = lapack_routine(m, n, n_rhs, a, m, bstar, ldb, s, rcond, The middle call to dgelsd looks unnecessary to me. At the very least, allocating astar_real and bstar_real shouldn't be necessary since they aren't referenced anywhere else in the lstsq function. The lapack documentation for zgelsd also doesn't mention any need to call dgelsd to compute the size of the work array. --bb

1 0

Re: [Numpy-discussion] rollaxis
by Pierre GM 13 Dec '06

13 Dec '06

> Generate a column-permutation tuple and use fancy indexing: Works like a charm, thanks a lot !

1 0

rollaxis
by Pierre GM 13 Dec '06

13 Dec '06

All, I have a ND array whose axes I want to reorganize, so that axis "i" is at the end while the others stay in their relative position. What's the easiest ?

2 2

fromfile and tofile access with a tempfile.TemporaryFile()
by Tim Hirzel 13 Dec '06

13 Dec '06

Hi, Does anyone know how to get fromfile and tofile to work from a tempfile.TemporaryFile? Or if its not possible? I am getting this: >>> import tempfile >>> f = tempfile.TemporaryFile() >>> f <open file '<fdopen>', mode 'w+b' at 0x01EE1728> >>> a = numpy.arange(10) >>> a.tofile(f) Traceback (most recent call last): File "<input>", line 1, in ? IOError: first argument must be a string or open file thanks! tim

5 10

.byteswap() and copy/view dilemma
by Francesc Altet 13 Dec '06

13 Dec '06

Hi, I'm a bit confused about which cases the .byteswap() method in NumPy would return a copy or a view. From the docstrings: """ a.byteswap(False) -> View or copy. Swap the bytes in the array. Swap the bytes in the array. Return the byteswapped array. If the first argument is TRUE, byteswap in-place and return a reference to self. """ >From the above description, it is not clear to me whether the .byteswap(False) would return a copy or a view. However, the official NumPy book seems to explain this clearer: """ Byteswap the elements of the array and return the byteswapped array. If the argument is True, then byteswap in-place and return a reference to self. Otherwise, return a copy of the array with the elements byteswapped. The data-type descriptor is not changed so the array will have changed numbers. """ My experiments also show that .byteswap(False) does do a copy: In [154]:a=numpy.array([1,2,3]) In [155]:b=a.byteswap() In [156]:b Out[156]:array([16777216, 33554432, 50331648]) In [157]:a[1]=0 In [158]:b Out[158]:array([16777216, 33554432, 50331648]) I'm wondering if I'm the only one that finds this docstring confusing. Regards, -- Francesc Altet | Be careful about using the following code -- Carabos Coop. V. | I've only proven that it works, www.carabos.com | I haven't tested it. -- Donald Knuth

1 0