Mailman 3 September 2010 - NumPy-Discussion

Improvements to SciPy.org Server
by Aaron River 01 Oct '10

01 Oct '10

Greetings Gentle Folk, I'm happy to announce that the server hosting SciPy.org, as well as other OSS projects, has been migrated into a cluster assigned to our community support efforts. The base instance itself remains relatively unchanged, but is now running atop a more capable set of servers. (Improvements to the services themselves are next in line.) Yesterday afternoon, I allowed the system to run with the configurations carried over from the old hardware, while watching the services for warning signs of trouble. Then, that evening, I loosened things up and adjusted configurations to better utilize the available resources. Though I always expect there to be "gotchas", I've thus far observed much improved response times and throughput. If you do experience any problems, please do not hesitate to let us know. We hope you enjoy the improvements to SciPy.org. Thanks, -- Aaron River Sr IT Administrator Enthought, Inc. --

4 3

Multiplying list of matrices with list of vectors
by David Reichert 30 Sep '10

30 Sep '10

Hi, I have a list of matrices W_k I'd like to multiply with a list of vectors v_k, or another way of looking at it, writing all W_k into a 3d array and all v_k into a 2d matrix/array, I'd like to compute matrix R as R_ik = sum_j W_ijk h_jk. Is there a fast way of doing that in numpy? Regards, David The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

2 2

FYI: Nose
by Charles R Harris 30 Sep '10

30 Sep '10

Looks like official nose support for Python 3.x might be in the pipeline<http://groups.google.com/group/nose-dev/browse_thread/thread/edab248023917b…> . Chuck

1 0

indexed arrays ignoring duplicates
by Damien Morton 29 Sep '10

29 Sep '10

lets say i have arrays: a = array((1,2,3,4,5)) indices = array((1,1,1,1)) and i perform operation: a[indices] += 1 the result is array([1, 3, 3, 4, 5]) in other words, the duplicates in indices are ignored if I wanted the duplicates not to be ignored, resulting in: array([1, 6, 3, 4, 5]) how would I go about this? the example above is somewhat trivial, what follows is exactly what I am trying to do: def inflate(self,pressure): faceforces = pressure * cross(self.verts[self.faces[:,1]]-self.verts[self.faces[:,0]], self.verts[self.faces[:,2]]-self.verts[self.faces[:,0]]) self.verts[self.faces[:,0]] += faceforces self.verts[self.faces[:,1]] += faceforces self.verts[self.faces[:,2]] += faceforces def constrain_lengths(self): vectors = self.verts[self.constraints[:,1]] - self.verts[self.constraints[:,0]] lengths = sqrt(sum(square(vectors), axis=1)) correction = 0.5 * (vectors.T * (1 - (self.restlengths / lengths))).T self.verts[self.constraints[:,0]] += correction self.verts[self.constraints[:,1]] -= correction def compute_normals(self): facenormals = cross(self.verts[self.faces[:,1]]-self.verts[self.faces[:,0]], self.verts[self.faces[:,2]]-self.verts[self.faces[:,0]]) self.normals.fill(0) self.normals[self.faces[:,0]] += facenormals self.normals[self.faces[:,1]] += facenormals self.normals[self.faces[:,2]] += facenormals lengths = sqrt(sum(square(self.normals), axis=1)) self.normals = (self.normals.T / lengths).T Ive been getting some very buggy results as a result of duplicates being ignored in my indexed inplace add/sub operations.

4 10

looking for code advice
by Gordon Wrigley 29 Sep '10

29 Sep '10

Hi First the disclaimer: This is my first numpy experience, so I have next to no idea what I'm doing. I've muddled through and managed to put together some code for my current problem and now that I have it going I'd like to hear any comments people may have on both my solution and other ways of approaching the problem. I have two goals here, I'd like to make the process run faster and I'd like to broaden my understanding of numpy as I can see from my brief use of it that it is a remarkably powerful tool. Now to the problem at hand. I find this difficult to explain but will try as best I can. The best word I have for the process is decimation. The input and output are both 3 dimensional arrays of uint8's. The output is half the size of the input along each dimension. Each cell [x,y,z] in the output corresponds to the 2x2x2 block [2*x:2*x+2, 2*y:2*y+2, 2*z:2*z+2] in the input. The tricky bit is in how the correspondence works. If all the cells in the input block have the same value then the cell in the output block will also have that value. Otherwise the output cell will have the value MIXED. Here is my current solution, from my limited testing it seems to produce the result I'm after. def decimate(data_in): in_x, in_y, in_z = data_in.shape out_x = in_x / 2 out_y = in_y / 2 out_z = in_z / 2 out_shape = out_x, out_y, out_z out_size = product(out_shape) # figure out which chunks are homogeneous reshaped_array = data_in.reshape(out_x, 2, out_y, 2, out_z, 2).transpose(0,2,4,1,3,5).reshape(out_x, out_y, out_z, 8) min_array = numpy.amin(reshaped_array, axis=3) max_array = numpy.amax(reshaped_array, axis=3) equal_array = numpy.equal(min_array, max_array) # select the actual value for the homogeneous chunks and MIXED for the heterogeneous decimated_array = data_in[::2,::2,::2] mixed_array = numpy.tile(MIXED, out_size).reshape(out_shape) data_out = numpy.where(equal_array, decimated_array, mixed_array) return data_out For the curious this is will be used to build a voxel octtree for a 3d graphics application. The final setup will be more complicated, this is the minimum that will let me get up and running. Regards Gordon P.S. congrats on numpy, it is a very impressive tool, I've only scraped the surface and it's already impressed me several times over.

2 3

Appending/combining masked arrays
by Gökhan Sever 29 Sep '10

29 Sep '10

Hello, Consider these two simple masked arrays: I[188]: a = np.ma.masked_equal([1,2,3], value=2) I[189]: b = np.ma.masked_equal([4,3,2], value=2) An operation like this voids the mask: I[190]: np.append(a,b) O[190]: masked_array(data = [1 2 3 4 3 2], mask = False, fill_value = 999999) In my real use case I have two lists (shown simplified versions): I[193]: all_measured[5::14][1:] O[193]: [masked_array(data = [425.82268 441.8043 432.69865 433.75158 420.42552 469.73359 483.80741 427.66887 466.7487 452.64255 438.14488 428.38871 416.38598 432.92884 440.74705 415.00694 430.1807 446.02079 428.1408 428.21708 461.37897 453.43518 433.90081 426.88591 451.15683 426.07399 410.7971 455.19179 389.01905 485.69204 505.35355 523.30598 502.00168 491.85421 485.75839 473.37061 459.24917 438.47531 424.09222 411.82773 409.27676 366.24813 362.0136 385.61986 350.38855 357.10589 390.84878 390.53565 332.60864 368.45913], mask = [False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False], fill_value = 1e+20) , masked_array(data = [762.00897 756.79155 773.59503 757.97867 746.20204 752.0657], mask = [False False False False False False], fill_value = 1e+20) ] I[194]: all_predicted[5::14][1:] O[194]: [masked_array(data = [601.587925396 615.382975948 637.565961135 662.845855035 630.180285797 910.714363555 886.048093691 912.616380221 1087.38406572 789.0075947 777.900831884 733.319025182 750.579326854 752.627618389 696.521605131 633.362074267 722.437789869 730.89750503 692.179530427 703.786215707 808.592649936 1006.89797524 818.839286207 767.260255009 787.622382926 831.332970348 949.016807581 783.981396594 643.768619685 654.417348215 681.516301642 753.577103851 654.092465489 628.484105951 691.461588689 800.901347497 630.894132084 610.977386345 512.926749811 653.74866061 587.915074604 531.106658494 562.265237436 606.32672755 563.281067561 546.715211886 604.210379352 475.66452212 454.426293217 656.039874394], mask = [False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False], fill_value = 1e+20) , masked_array(data = [891.251806903 833.185882945 840.250000752 831.649215796 883.534378034 841.970022166], mask = [False False False False False False], fill_value = 1e+20) ] These lists have 42 varying size masked arrays in each. I want to be able to combine each list in one array --preferably in a masked array for not losing the mask information so that I can perform some overall statistics. What is the way to solve this issue? Thanks -- Gökhan

3 7

numpydoc: "Other Parameters" section
by Michael Droettboom 29 Sep '10

29 Sep '10

We're having trouble using numpydoc formatted docstrings for one of our own project. It seems that the "Other parameters" section is not getting included in the output. A grep across Numpy's own source code (git master) reveals that this kind of section is used in only one place -- the docstring for "recarray". Curiously, the "Other Parameters" section is displayed in the documentation editor here: http://docs.scipy.org/numpy/docs/numpy.core.records.recarray/ But not in the generated Numpy docs online here: http://docs.scipy.org/doc/numpy/reference/generated/numpy.recarray.html Is this a bug? Mike -- Michael Droettboom Science Software Branch Space Telescope Science Institute Baltimore, Maryland, USA

3 3

convert FORTRAN exponential format text to float
by Thomas Kirk Gamble 29 Sep '10

29 Sep '10

I need to convert numbers read from a text file to floating point. The numbers are in the format 1.538D-06 (written by a FORTRAN application) and have varying amounts of whitespace between them from line to line. The function fromstring() deals with the whitespace just fine but 'dtype=float' doesn't correctly convert the data. It sees every thing up to the 'D' and ignores the rest(I assume expecting an 'e' to indicate the exponential). I was able to get around this using re.sub() to change the 'D' to 'e' in the string before using fromstring(), but I was wondering if python has any way to directly read this data as float? Google didn't find an answer.

2 4

Trying out datarray
by John Salvatier 28 Sep '10

28 Sep '10

I am not sure if this is the correct place for such questions, but here goes: I am curious about datarray, but I haven't been able to get it to work. The module datarray does not appear to have a class DataArray (or DatArray). So I am confused how I am supposed to use it. Can anyone advise? My other question is whether datarray will be able to handle multiple data types in the same object; i.e. does it have the functionality of recarrays and R data.frames? John

3 3

can savetxt write arrays with ndim > 2 ?
by Mark Bakker 28 Sep '10

28 Sep '10

Hello list, I tried to savetxt an array with ndim = 3, but I get an error: In [252]: savetxt('test.dat',a) /Library/Frameworks/Python.framework/Versions/6.2/lib/python2.6/site-packages/numpy/lib/io.py in savetxt(fname, X, fmt, delimiter) 784 785 for row in X: --> 786 fh.write(format % tuple(row) + '\n') 787 788 import re TypeError: float argument required, not numpy.ndarray I don't see this anywhere in the documentation of savetxt. And if that is a restriction it should probably be caught with an assertion. Numpy version 1.4.0. I see why ndim>2 may cause problems, as there is now way on 'loadtxt' to figure out the shape of the array, but I think it should throw an assertion and the documentation should be updated. Thanks, Mark

1 0