Mailman 3 April 2012 - NumPy-Discussion

Bitwise operations and unsigned types
by Chris Laumann 10 Apr '12

10 Apr '12

Hi all- I've been trying to use numpy arrays of ints as arrays of bit fields and mostly this works fine. However, it seems that the bitwise_* ufuncs do not support unsigned integer dtypes: In [142]: np.uint64(5)&3 --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /Users/claumann/<ipython-input-142-65e3301d5d07> in <module>() ----> 1 np.uint64(5)&3 TypeError: ufunc 'bitwise_and' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe'' This seems odd as unsigned ints are the most natural bitfields I can think of -- the sign bit is just confusing when doing bit manipulation. Python itself of course doesn't make much a distinction between ints, longs, unsigned etc. Is this a bug? Thanks, Chris -- Chris Laumann Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

5 15

Re: [Numpy-discussion] NpyAccessLib method documentation?
by William Johnston 10 Apr '12

10 Apr '12

Hello, Anyone there? williamj

2 1

Why is numpy.abs so much slower on complex64 than complex128 under windows 32-bit?
by Henry Gomersall 10 Apr '12

10 Apr '12

Here is the body of a post I made on stackoverflow, but it seems to be a non-obvious issue. I was hoping someone here might be able to shed light on it... On my 32-bit Windows Vista machine I notice a significant (5x) slowdown when taking the absolute values of a fairly large |numpy.complex64| array when compared to a |numpy.complex128| array. |>>> import numpy >>> a= numpy.random.randn(256,2048) + 1j*numpy.random.randn(256,2048) >>> b= numpy.complex64(a) >>> timeit c= numpy.float32(numpy.abs(a)) 10 loops, best of3: 27.5 ms per loop >>> timeit c= numpy.abs(b) 1 loops, best of3: 143 ms per loop | Obviously, the outputs in both cases are the same (to operating precision). I do not notice the same effect on my Ubuntu 64-bit machine (indeed, as one might expect, the double precision array operation is a bit slower). Is there a rational explanation for this? Is this something that is common to all windows? In a related note of confusion, the times above are notably (and consistently) different (shorter) to that I get doing a naive `st = time.time(); numpy.abs(a); print time.time()-st`. Is this to be expected? Cheers, Henry

3 6

Slice specified axis
by Tony Yu 10 Apr '12

10 Apr '12

Is there a way to slice an nd-array along a specified axis? It's easy to slice along a fixed axis, e.g.: axis = 0: >>> array[start:end] axis = 1: >>> array[:, start:end] ... But I need to do this inside of a function that accepts arrays of any dimension, and the user can operate on any axis of the array. My current solution looks like the following: >>> aslice = lambda axis, s, e: (slice(None),) * axis + (slice(s, e),) >>> array[aslice(axis, start, end)] which works, but I'm guessing that numpy has a more readable way of doing this that I've overlooked. Thanks, -Tony

5 9

NpyAccessLib method documentation?
by William Johnston 09 Apr '12

09 Apr '12

Hello, Is there NpyAccessLib documentation available? I need to use DLLImport for a C# IronPython DLR app and am not sure which methods to include. Thank you. Regards, William Johnston

1 0

NumPy EIG much slower than MATLAB EIG
by Kamesh Krishnamurthy 09 Apr '12

09 Apr '12

Hello all, I profiled NumPy EIG and MATLAB EIG on the same Macbook pro, and both were linking to the Accelerate framework BLAS. NumPy turns out to be ~4x slower. I've posted details on Stackoverflow: http://stackoverflow.com/q/9955021/974568 Can someone please let me know the reason for the performance gap? Thanks, Kamesh

8 10

Testsuite fails with Python 2.7.3rc1 and 3.2.3rc1 (Debian)
by Sandro Tosi 08 Apr '12

08 Apr '12

Hello, I've reported http://projects.scipy.org/numpy/ticket/2085 and Ralf asked for bringing that up here: is anyone able to replicate the problem described in that ticket? The debian bug tracking the problem is: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=664672 Cheers, -- Sandro Tosi (aka morph, morpheus, matrixhasu) My website: http://matrixhasu.altervista.org/ Me at Debian: http://wiki.debian.org/SandroTosi

3 11

problem with vectorized difference equation
by francesco82 07 Apr '12

07 Apr '12

Hello everyone, After reading the very good post http://technicaldiscovery.blogspot.com/2011/06/speeding-up-python-numpy-cyt… and the book by H. P. Langtangen 'Python scripting for computational science' I was trying to speed up the execution of a loop on numpy arrays being used to describe a simple difference equation. The actual code I am working on involves some more complicated equations, but I am having the same exact behavior as described below. To test the improvement in speed I wrote the following in vect_try.py: #!/usr/bin/python import numpy as np import matplotlib.pyplot as plt dt = 0.02 #time step time = np.arange(0,2,dt) #time array u = np.sin(2*np.pi*time) #input signal array def vect_int(u,y): #vectorized function n = u.size y[1:n] = y[0:n-1] + u[1:n] return y def sc_int(u,y): #scalar function y = y + u return y def calc_vect(u, func=vect_int): out = np.zeros(u.size) for i in xrange(u.size): out = func(u,out) return out def calc_sc(u, func=sc_int): out = np.zeros(u.size) for i in xrange(u.size-1): out[i+1] = sc_int(u[i+1],out[i]) return out To verify the execution time I've used the timeit function in Ipython: import vect_try as vt timeit vt.calc_vect(vt.u) --> 1000 loops, best of 3: 494 us per loop timeit vt.calc_sc(vt.u) -->10000 loops, best of 3: 92.8 us per loop As you can see the scalar implementation looping one item at the time (calc_sc) is 494/92.8~5.3 times faster than the vectorized one (calc_vect). My problem consists in the fact that I need to iterate the execution of calc_vect in order for it to operate on all the elements of the input array. If I execute calc_vect only once, it will only operate on the first slice of the vectors leaving the remaining untouched. My understanding was that the vector expression y[1:n] = y[0:n-1] + u[1:n] was supposed to iterate over all the array, but that's not happening for me. Can anyone tell me what I am doing wrong? Thanks! Francesco -- View this message in context: http://old.nabble.com/problem-with-vectorized-difference-equation-tp3364168… Sent from the Numpy-discussion mailing list archive at Nabble.com.

4 5

Improving NumPy's indexing / subsetting / fancy indexing implementation
by Wes McKinney 07 Apr '12

07 Apr '12

dear all, I've routinely found that: 1) ndarray.take is up to 1 order of magnitude faster than fancy indexing 2) Hand-coded Cython boolean indexing is many times faster than ndarray indexing 3) putmask is significantly faster than ndarray indexing For example, I stumbled on this tonight: straightforward cython code: def row_bool_subset(ndarray[float64_t, ndim=2] values, ndarray[uint8_t, cast=True] mask): cdef: Py_ssize_t i, j, n, k, pos = 0 ndarray[float64_t, ndim=2] out n, k = (<object> values).shape assert(n == len(mask)) out = np.empty((mask.sum(), k), dtype=np.float64) for i in range(n): if mask[i]: for j in range(k): out[pos, j] = values[i, j] pos += 1 return out In [1]: values = randn(1000000, 4) In [2]: mask = np.ones(1000000, dtype=bool) In [3]: import pandas._sandbox as sbx In [4]: result = sbx.row_bool_subset(values, mask) In [5]: timeit result = sbx.row_bool_subset(values, mask) 100 loops, best of 3: 9.58 ms per loop pure NumPy: In [6]: timeit values[mask] 10 loops, best of 3: 81.6 ms per loop Here's the kind of take performance problems that I routinely experience: In [12]: values = randn(1000000, 4) v In [13]: values.shape Out[13]: (1000000, 4) In [14]: indexer = np.random.permutation(1000000)[:500000] In [15]: timeit values[indexer] 10 loops, best of 3: 70.7 ms per loop In [16]: timeit values.take(indexer, axis=0) 100 loops, best of 3: 13.3 ms per loop When I can spare the time in the future I will personally work on these indexing routines in the C codebase, but I suspect that I'm not the only one adversely affected by these kinds of performance problems, and it might be worth thinking about a community effort to split up the work of retooling these methods to be more performant. We could use a tool like my vbench project (github.com/wesm/vbench) to make a list of the performance benchmarks of interest (like the ones above). Unfortunately I am too time constrained at least for the next 6 months to devote to a complete rewrite of the code in question. Possibly sometime in 2013 if no one has gotten to it yet, but this seems like someplace that we should be concerned about as the language performance wars continue to rage. - Wes

3 3

empty chararrays (ticket 1948)
by Ralf Gommers 07 Apr '12

07 Apr '12

Hi, In ticket 1948 a backwards compatibility issue with chararray is reported. Indexing a chararray with [] or a bool array of False used to return [] in numpy 1.2.1 (consistent with ndarray behavior), but now returns an empty string. Unfortunately this changed behavior has been present for the 1.5.x and 1.6.x releases. So the question is if this should be changed back or not? The change was likely unintentional; there's no test for it. Ralf

2 3