Mailman 3 November 2012 - NumPy-Discussion

random.choice
by Alan G Isaac 13 Nov '12

13 Nov '12

I just noticed that 1.7 is scheduled to add a random.choice function. I wonder if the best structure has been chosen. Specifically, it does not provide for array flattening, and it does not provide for subarray choice. Back in 2006 (?) Robert Kern suggested something like the below (forgive any mistranslation). I think the included functionality does belong in a choice function. I do like the provision for repeated sampling in the current proposal, however. Cheers, Alan Isaac def choice(x, axis=None): """Select an element or subarray uniformly randomly. If axis is None, then a single element is chosen from the entire array. Otherwise, a subarray is chosen from the given axis. """ x = np.asarray(x) if axis is None: length = np.multiply.reduce(x.shape) n = random.randint(length) return x.flat[n] else: n = random.randint(x.shape[axis]) idx = map(slice, x.shape) idx[axis] = n return x[tuple(idx)]

3 16

Scipy dot
by Nicolas SCHEFFER 13 Nov '12

13 Nov '12

Hi, I've written a snippet of code that we could call scipy.dot, a drop-in replacement for numpy.dot. It's dead easy, and just answer the need of calling the right blas function depending on the type of arrays, C or F order (aka slowness of np.dot(A, A.T)) While this is not the scipy mailing list, I was wondering if this snippet would relevant and/or useful to others, numpy folks, scipy folks or could be integrated directly in numpy (so that we keep the nice A.dot(B) syntax) This bottleneck of temporary copies has been a problem for lots of users and it seems everybody has their own snippets. This code is probably not written as it should, I hope the community can help improving it! ;) First FIXME is to make it work for arrays of dimensions other than 2. Suggestions highly appreciated! Thanks! === Code (also on http://pastebin.com/QrRk0kEf) def dot(A, B, out=None): """ A drop in replaement for numpy.dot Computes A.B optimized using fblas call note: unlike in numpy the returned array is in F order""" import scipy.linalg as sp gemm = sp.get_blas_funcs('gemm', arrays=(A,B)) if out is None: lda, x, y, ldb = A.shape + B.shape if x != y: raise ValueError("matrices are not aligned") dtype = np.max([x.dtype for x in (A, B)]) out = np.empty((lda, ldb), dtype, order='F') if A.flags.c_contiguous and B.flags.c_contiguous: gemm(alpha=1., a=A.T, b=B.T, trans_a=True, trans_b=True, c=out, overwrite_c=True) if A.flags.c_contiguous and B.flags.f_contiguous: gemm(alpha=1., a=A.T, b=B, trans_a=True, c=out, overwrite_c=True) if A.flags.f_contiguous and B.flags.c_contiguous: gemm(alpha=1., a=A, b=B.T, trans_b=True, c=out, overwrite_c=True) if A.flags.f_contiguous and B.flags.f_contiguous: gemm(alpha=1., a=A, b=B, c=out, overwrite_c=True) return out == Timing (EPD, MKL): In [15]: A = np.array(np.random.randn(1000, 1000), 'f') In [16]: %timeit np.dot(A, A) 100 loops, best of 3: 7.19 ms per loop In [17]: %timeit np.dot(A.T, A.T) 10 loops, best of 3: 27.7 ms per loop In [18]: %timeit np.dot(A, A.T) 100 loops, best of 3: 18.3 ms per loop In [19]: %timeit np.dot(A.T, A) 100 loops, best of 3: 18.7 ms per loop In [20]: %timeit dot(A, A) 100 loops, best of 3: 7.16 ms per loop In [21]: %timeit dot(A.T, A.T) 100 loops, best of 3: 6.67 ms per loop In [22]: %timeit dot(A, A.T) 100 loops, best of 3: 6.79 ms per loop In [23]: %timeit dot(A.T, A) 100 loops, best of 3: 7.02 ms per loop

10 35

Compiling NumPy on Windows for Python 3.3
by Peter Cock 13 Nov '12

13 Nov '12

Dear all, Since the NumPy 1.7.0b2 release didn't include a Windows (32 bit) installer for Python 3.3, I am considering compiling it myself for local testing. What compiler is recommended? Thanks, Peter

2 11

(no subject)
by Jeffrey Spencer 12 Nov '12

12 Nov '12

http://www.a1bailbondsmd.com/wp-includes/ugoogle.html

1 0

Double-ended queues
by William Furnass 09 Nov '12

09 Nov '12

Hi all, I want to be able to within a loop a) apply a mathematical operation to all elements in a vector (can be done atomically) then b) pop zero or more elements from one end of the vector and c) push zero or more elements on to the other end. So far I've used a collections.deque to store my vector as it should be more efficient than a numpy array for the appending and deletion of elements. However, I was wondering whether performance could be improved through the use of a homogeneously-typed double-ended queue i.e. a linked list equivalent of numpy.ndarray. Has anyone previously considered whether it would be worth including such a thing within the numpy package? Cheers, Will

6 6

Help compiling numpy with new gcc
by Filipe Pires Alvarenga Fernandes 09 Nov '12

09 Nov '12

Hi I am trying to compile numpy with gcc 4.7.1 and I am having the following issue. "RuntimeError: Broken toolchain: cannot link a simple C program" I noticed that I need to pass the flag '-fno-use-linker-plugin' to be able to compile it. However, even though I did pass it by exporting the CFLAGS, it does not work. I guess that numpy do not use the CFLAGS for its internal extensions. How can I pass that option to it? Error below: [ 11s] compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/usr/include/python2.7 -c' [ 11s] gcc: _configtest.c [ 11s] gcc -pthread _configtest.o -o _configtest [ 11s] gcc: fatal error: -fuse-linker-plugin, but liblto_plugin.so not found [ 11s] compilation terminated. [ 11s] gcc: fatal error: -fuse-linker-plugin, but liblto_plugin.so not found [ 11s] compilation terminated. [ 11s] failure. [ 11s] removing: _configtest.c _configtest.o [ 11s] Traceback (most recent call last): [ 11s] File "setup.py", line 214, in <module> [ 11s] setup_package() [ 11s] File "setup.py", line 207, in setup_package [ 11s] configuration=configuration ) [ 11s] File "/home/abuild/rpmbuild/BUILD/numpy/numpy/distutils/core.py", line 186, in setup [ 11s] return old_setup(**new_attr) [ 11s] File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup [ 11s] dist.run_commands() [ 11s] File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands [ 11s] self.run_command(cmd) [ 11s] File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command [ 11s] cmd_obj.run() [ 11s] File "/home/abuild/rpmbuild/BUILD/numpy/numpy/distutils/command/build.py", line 37, in run [ 11s] old_build.run(self) [ 11s] File "/usr/lib64/python2.7/distutils/command/build.py", line 127, in run [ 11s] self.run_command(cmd_name) [ 11s] File "/usr/lib64/python2.7/distutils/cmd.py", line 326, in run_command [ 11s] self.distribution.run_command(command) [ 11s] File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command [ 11s] cmd_obj.run() [ 11s] File "/home/abuild/rpmbuild/BUILD/numpy/numpy/distutils/command/build_src.py", line 152, in run [ 11s] self.build_sources() [ 11s] File "/home/abuild/rpmbuild/BUILD/numpy/numpy/distutils/command/build_src.py", line 163, in build_sources [ 11s] self.build_library_sources(*libname_info) [ 11s] File "/home/abuild/rpmbuild/BUILD/numpy/numpy/distutils/command/build_src.py", line 298, in build_library_sources [ 11s] sources = self.generate_sources(sources, (lib_name, build_info)) [ 11s] File "/home/abuild/rpmbuild/BUILD/numpy/numpy/distutils/command/build_src.py", line 385, in generate_sources [ 11s] source = func(extension, build_dir) [ 11s] File "numpy/core/setup.py", line 648, in get_mathlib_info [ 11s] raise RuntimeError("Broken toolchain: cannot link a simple C program") [ 11s] RuntimeError: Broken toolchain: cannot link a simple C program [ 11s] error: Bad exit status from /var/tmp/rpm-tmp.yO2SIE (%build) [ 11s] [ 11s] [ 11s] RPM build errors: [ 11s] Bad exit status from /var/tmp/rpm-tmp.yO2SIE (%build)

3 2

testing with amd libm/acml
by Neal Becker 08 Nov '12

08 Nov '12

I'm trying to do a bit of benchmarking to see if amd libm/acml will help me. I got an idea that instead of building all of numpy/scipy and all of my custom modules against these libraries, I could simply use: LD_PRELOAD=/opt/amdlibm-3.0.2/lib/dynamic/libamdlibm.so:/opt/acml5.2.0/gfortran64/lib/libacml.so <my program here> I'm hoping that both numpy and my own dll's then will take advantage of these libraries. Do you think this will work?

5 18

numexpr question
by Neal Becker 08 Nov '12

08 Nov '12

I'm interested in trying numexpr, but have a question (not sure where's the best forum to ask). The examples I see use ne.evaluate ("some string...") When used within a loop, I would expect the compilation from the string form to add significant overhead. I would have thought a pre-compiled form would be available, similar to a precompiled regexp. yes/no?

2 1

strange behavior of numpy.unique
by Phillip Feldman 07 Nov '12

07 Nov '12

numpy.unique behaves as I would expect for small inputs like the following: In [12]: x= [0, 0, 1, 0, 1, 2, 0, 1, 2, 3] In [13]: unique(x, return_index=True) Out[13]: (array([0, 1, 2, 3]), array([0, 2, 5, 9], dtype=int64)) But, when I give it something larger, the return index values do not always correspond to the first occurrences in the input. The documentation is silent on the question of how the return index values are chosen when a given element of x appears more than once. Either the documentation should be clarified, or better yet, the behavior should be changed.

4 4

np 1.7b2 PyArray_BYTES(obj)=ptr fail
by Frédéric Bastien 06 Nov '12

06 Nov '12

Hi, I don't know if that was raised or not, but in np1.7b2 doing this fail with this error message: PyArray_BYTES(obj)=ptr file:lne_number:offset: error: lvalue required as left operatnd of assignment. I tried with PyArray_DATA(obj)=ptr and this also fail. Do you want to remove this feature now? I would think this change will be done at the same time as the one related to the macro NPY_NO_DEPRECATED_API. If I missed the discussion about this, tell me. thanks Fred

2 10

NumPy-Discussion November 2012