Mailman 3 January 2011 - NumPy-Discussion

How to tell if I succeeded to build numpy with amd, umfpack and lapack
by Samuel John Feb. 19, 2011

Feb. 19, 2011

Hi there! I have successfully built numpy 1.5 on ubuntu lucid (32 for now). I think I got ATLAS/lapack/BLAS support, and if I > ldd linalg/lapack_lite.so I see that my libptf77blas.so etc. are successfully linked. :-) However, how to I find out, if (and where) libamd.a and libumfpack.a have been found and (statically) linked. As far as I understand, I they are not present, a fallback in pure python is used, right? Is there a recommended way, I can query against which libs numpy has been built? So I can be sure numpy uses my own compiled versions of libamd, lapack and so forth. And the fftw3 is no longer supported, I guess (even if it is still mentioned in the site.cfg.example) Bests, Samuel -- Dipl.-Inform. Samuel John - - - - - - - - - - - - - - - - - - - - - - - - - PhD student, CoR-Lab(.de) and Neuroinformatics Group, Faculty of Technology, D33594 Bielefeld in cooperation with the HONDA Research Institute Europe GmbH jabber: samueljohn(a)jabber.org - - - - - - - - - - - - - - - - - - - - - - - - -

4 6

using loadtxt() for given number of rows?
by Robert Cimrman Feb. 14, 2011

Feb. 14, 2011

Hi, I work with text files which contain several arrays separated by a few lines of other information, for example: POINTS 4 float -5.000000e-01 -5.000000e-01 0.000000e+00 5.000000e-01 -5.000000e-01 0.000000e+00 5.000000e-01 5.000000e-01 0.000000e+00 -5.000000e-01 5.000000e-01 0.000000e+00 CELLS 2 8 3 0 1 2 3 2 3 0 (yes, that's the legacy VTK format, but take it just as an example) I have used custom Python code with loops to read similar files, so the speed was not too good. Now I wonder if it would be possible to use the numpy.loadtxt() function for the "array-like" parts. It supports passing an open file object in, which is good, but it wants to read the entire file, which does not work in this case. It seems to me, that an additional parameter to loadtxt(), say "nrows" or "numrows", would do the job, so that the function does not try reading the entire file. Another possibility would be to raise an exception as it is now, but also to return the data succesfully read so far. What do you think? Is this worth a ticket? r.

3 5

Installing numpy
by Jonathan Tu Feb. 13, 2011

Feb. 13, 2011

Hi, I need to reinstall numpy because the cluster I am using was recently overhauled. I am wondering if numpy works with Python 2.7 now. Also, I would like numpy to run as fast as possible. The last time I did this, I was advised to install ATLAS by hand, as the one that comes with RHEL is not suitable. The first time I tried this, I kept running into problems that I think were due to mismatched fortran compilers. Is there a good resource for how to do this? I am fairly new to Linux. Thanks, Jonathan Tu

5 8

Non-deterministic floating point behavior in numpy 1.5.1 ?
by Grégory Guyomarc'h Feb. 2, 2011

Feb. 2, 2011

Hello, I have noticed strange non-deterministic behaviours with numpy 1.5.1 when using floating point arrays. The following script were run on 6 different machines, all Intel Core i7: - 3 of them running numpy 1.5.1 with either Python 2.7.1 (x86) or 2.5.2 (x86) and, - 3 of them running numpy 1.3 and Python 2.5.2.(x86). import numpy x = numpy.array([[0.00010876945607980702], [0.22568137594619658], [5.6435218858623557]]) for i in range(10): m = numpy.array([[36.0 * 36.0, 36.0, 1.0] for j in range(6)]) y = (numpy.dot(m, x) - 13.90901663) * 1000.0 print y[0] The output on each machine running 1.5.1 are similar to this one: [ 5.00486230e-06] [ 5.00486408e-06] [ 5.00486230e-06] [ 5.00486408e-06] [ 5.00486230e-06] [ 5.00486408e-06] [ 5.00486230e-06] [ 5.00486408e-06] [ 5.00486230e-06] [ 5.00486408e-06] I cannot make sense of the changes of the least significant digits across different iterations of the for loop since its body is actually constant. Note that this behavior is hard to reproduce: on some machines I had to insert dummy print statements here and there to reproduce the bug or increase the number of iterations or the length of the arrays inside the loop. Also, I could not reproduce it with older versions of numpy such as 1.3. Is this behavior expected ? Is there a way to make sure the results of a numpy floating point computations remains the same for multiple runs ? Thanks, Gregory.

3 2

Vectorize or rewrite function to work with array inputs?
by DParker＠chromalloy.com Feb. 1, 2011

Feb. 1, 2011

I have several functions like the example below that I would like to make compatible with array inputs. The problem is the conditional statements give a ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all(). I can use numpy.vectorize, but if possible I'd prefer to rewrite the function. Does anyone have any advice the best way to modify the code to accept array inputs? Thanks in advance for any assistance. NAN = float('nan') def air_gamma(t, far=0.0): """ Specific heat ratio (gamma) of Air/JP8 t - static temperature, Rankine [far] - fuel air ratio [- defaults to 0.0 (dry air)] air_gamma - specific heat ratio """ if far < 0.: return NAN elif far < 0.005: if t < 379. or t > 4731.: return NAN else: air_gamma = -3.472487e-22 * t ** 6. + 6.218811e-18 * t ** 5. - 4.428098e-14 * t ** 4. + 1.569889e-10 * t ** 3. - 0.0000002753524 * t ** 2. + 0.0001684666 * t + 1.368652 elif far < 0.069: if t < 699. or t > 4731.: return NAN else: a6 = 4.114808e-20 * far ** 3. - 1.644588e-20 * far ** 2. + 3.103507e-21 * far - 3.391308e-22 a5 = -6.819015e-16 * far ** 3. + 2.773945e-16 * far ** 2. - 5.469399e-17 * far + 6.058125e-18 a4 = 4.684637e-12 * far ** 3. - 1.887227e-12 * far ** 2. + 3.865306e-13 * far - 4.302534e-14 a3 = -0.00000001700602 * far ** 3. + 0.000000006593809 * far ** 2. - 0.000000001392629 * far + 1.520583e-10 a2 = 0.00003431136 * far ** 3. - 0.00001248285 * far ** 2. + 0.000002688007 * far - 0.0000002651616 a1 = -0.03792449 * far ** 3. + 0.01261025 * far ** 2. - 0.002676877 * far + 0.0001580424 a0 = 13.65379 * far ** 3. - 3.311225 * far ** 2. + 0.3573201 * far + 1.372714 air_gamma = a6 * t ** 6. + a5 * t ** 5. + a4 * t ** 4. + a3 * t ** 3. + a2 * t ** 2. + a1 * t + a0 elif far >= 0.069: return NAN else: return NAN return air_gamma David Parker Chromalloy - TDAG

7 14

Strange behaviour of numpy.asarray() in corner case
by Friedrich Romstedt Feb. 1, 2011

Feb. 1, 2011

Python 2.6.6 (r266:84374, Aug 31 2010, 11:00:51) [GCC 4.0.1 (Apple Inc. build 5493)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.5.1' >>> class X: ... pass ... >>> numpy.asarray([X(), numpy.asarray([1, 1])]).shape (2,) >>> numpy.asarray([numpy.asarray([1, 1]), X()]).shape () >>> I would expect (2,) in the second case too. The following works ok: >>> numpy.asarray([1, X()]).shape (2,) >>> numpy.asarray([X(), 1]).shape (2,) Friedrich

2 3

create a numpy array of images
by Asmi Shah Jan. 31, 2011

Jan. 31, 2011

Hi guys, I am using python for a while now and I have a requirement of creating a numpy array of microscopic tiff images ( this data is 3d, meaning there are 100 z slices of 512 X 512 pixels.) How can I create an array of images? i then would like to use visvis for visualizing this in 3D. any help is highly appreciated to get me started.. Thanks,.. -- Regards, Asmi Shah

5 4

Inversion of near singular matrices.
by Algis Kabaila Jan. 31, 2011

Jan. 31, 2011

Hi, I am interested in determining if a matrix is singular or "nearly singular" - very ill conditioned. The problem occurs in structural engineering applications. My OS is kubuntu 10.10 (32 bit) Python 2.6.6 numpy and numpy.linalg binaries from ubuntu repositories. The attached tar ball has a little CLI script that generates singular or near singular matrices (because of the inevitable roundoffs) for matrices with elements from sequence 1, 2, 3, 4 etc. The dimension of matrix nn can be passed as command line parameter via sys.argv[1] . If argv[1] does not exist, the 5x5 default matrix is used. for nn = 3 and 4 numpy does not raise an exception for nn = 5 it does raise an exception for nn = 6, 7 np not raises exception for nn = 8 np does raise exception for nn = 9 np does not raise exception for higher nn values np mostly raises the exception, but for nn = 23 and nn=120 it does NOT raise the exception. It is worht noting that in practical problems of engineering analyisis the ill conditioned matrix is not "exact" - there always are approximations and roundoff errors. So my question is: how can one reliably detect singularity (or near singularity) and raise an exception? Many thanks for your attention, Al. -- Algis http://akabaila.pcug.org.au/StructuralAnalysis.pdf

6 12

numpy.linalg.svd documentation
by Jason Grout Jan. 31, 2011

Jan. 31, 2011

The SVD documentation seems a bit misleading. It says: Factors the matrix a as u * np.diag(s) * v, where u and v are unitary and s is a 1-d array of a‘s singular values. However, that only is true (i.e., you just have to do np.diag(s) to get S) in general if full_matrices is False, which is not the default. Otherwise, you have to something like in the first example in the docstring. I'm not sure what the right fix is here. Changing the default for full_matrices seems too drastic. But then having u*np.diag(s)*v in the first line doesn't work if you have a rectangular matrix. Perhaps the first line could be changed to: Factors the matrix a as u * S * v, where u and v are unitary and S is a matrix with shape (a.shape[0], a.shape[1]) with np.diag(S)=s, where s is a 1-d array of a‘s singular values. It sounds more confusing that way, but at least it's correct. Maybe even better would be to add a shape option to np.diag, and then just make the first line of the svd docstring say u*np.diag(s,shape=(a.shape[0],a.shape[1]))*v Jason

5 8

Generator arrays
by Travis Oliphant Jan. 31, 2011

Jan. 31, 2011

Just to start the conversation, and to find out who is interested, I would like to informally propose generator arrays for NumPy 2.0. This concept has as one use-case, the deferred arrays that Mark Wiebe has proposed. But, it also allows for "compressed arrays", on-the-fly computed arrays, and streamed or generated arrays. Basically, the modification I would like to make is to have an array flag (MEMORY) that when set means that the data attribute of a numpy array is a pointer to the address in memory where the data begins with the strides attribute pointing to a C-array of integers (in other words, all current arrays are MEMORY arrays) But, when the MEMORY flag is not set, the data attribute instead points to a length-2 C-array of pointers to functions [read(N, output_address, self->index_iter, self->extra), write(N, input_address, self->index_iter, self->extra)] Either of these could then be NULL (i.e. if write is NULL, then the array must be read-only). When the MEMORY flag is not set, the strides member of the ndarray structure is a pointer to the index_iter object (which could be anything that the particular read and write methods need it to be). The array structure should also get a member to hold the "extra" argument (which would hold any state that the array needed to hold on to in order to correctly perform the read or write operations --- i.e. it could hold an execution graph for deferred evaluation). The index_iter structure is anything that the read and write methods need to correctly identify *where* to write. Now, clearly, we could combine index_iter and extra into just one "structure" that holds all needed state for read and write to work correctly. The reason I propose two slots is because at least mentally in the use case of having these structures be calculation graphs, one of these structures is involved in "computing the location to read/write" and the other is involved in "computing what to read/write" The idea is fairly simple, but with some very interesting potential features: * lazy evaluation (of indexing, ufuncs, etc.) * fancy indexing as views instead of copies (really just another example of lazy evaluation) * compressed arrays * generated arrays (from computation or streamed data) * infinite arrays * computed arrays * missing-data arrays * ragged arrays (shape would be the bounding box --- which makes me think of ragged arrays as examples of masked arrays). * arrays that view PIL data. One could build an array with a (logically) infinite number of elements (we could use -2 in the shape tuple to indicate that). We don't need examples of all of these features for NumPy 2.0 to be released, because to really make this useful, we would need to modify all "calculation" code to produce a NON MEMORY array. What to do here still needs a lot of thought and experimentation. But, I can think about a situation where all NumPy calculations that produce arrays provide the option that when they are done inside of a particular context, a user-supplied behavior over-rides the default return. I want to study what Mark is proposing and understand his new iterator at a deeper level before providing more thoughts here. That's the gist of what I am thinking about. I would love feedback and comments. The other things I would like to see in NumPy 2.0 that have not been discussed lately (that could affect the ABI) are: * a geometry member to the data structure (that allows labels to dimensions and axes to be provided -- ala data_array) * small array performance improvements that Mark Wiebe has suggested (including the addition of an optional low-level loop that is used when you have contiguous data) * completed datetime implementation * pointer data-types (i.e. the memory location holds a pointer to another part of an ndarray) --- very useful for "join" - type arrays If anybody is interested in helping with any of these (and has time to do it, let me know). Some of this I could fund (especially if you are willing to come to Austin and be an intern for Enthought). Best regards, -Travis P.S. I hope to have more time this year to hang-out here on the numpy-discussion list (but we will see....)

6 8