Mailman 3 May 2008 - NumPy-Discussion

Binary ufuncs: bitwise operators
by Charles R Harris May 27, 2008

May 27, 2008

Here are the bitwise operators. Note that combinations of signed and unsigned types lead to promotion to larger types. I don't think that is right, especially as the high bits will be zeroed anyway when the unsigned number is sign extended. Note also the object type used to promote the 'Q' (unsigned long long) type. On my machine i and l are the same precision, but different c-types, so it would also make me feel better if the rows/cols for l,L used the same types, making the functions symmetric in the arguments. bitwise_and not symmetric in1 ? , b , B , h , H , i , I , l , L , q , Q , f , d , g , F , D , G , -------------------------------------------------------------------------------------- in2 ? | ? , b , B , h , H , i , I , i , I , q , Q , err, err, err, err, err, err, in2 b | b , b , h , h , i , i , q , i , q , q , O , err, err, err, err, err, err, in2 B | B , h , B , h , H , i , I , i , I , q , Q , err, err, err, err, err, err, in2 h | h , h , h , h , i , i , q , i , q , q , O , err, err, err, err, err, err, in2 H | H , i , H , i , H , i , I , i , I , q , Q , err, err, err, err, err, err, in2 i | i , i , i , i , i , i , q , i , q , q , O , err, err, err, err, err, err, in2 I | I , q , I , q , I , q , I , q , I , q , Q , err, err, err, err, err, err, in2 l | l , l , l , l , l , l , q , l , q , q , O , err, err, err, err, err, err, in2 L | L , q , L , q , L , q , L , q , L , q , Q , err, err, err, err, err, err, in2 q | q , q , q , q , q , q , q , q , q , q , O , err, err, err, err, err, err, in2 Q | Q , O , Q , O , Q , O , Q , O , Q , O , Q , err, err, err, err, err, err, in2 f | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 d | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 g | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 F | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 D | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 G | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, bitwise_or not symmetric in1 ? , b , B , h , H , i , I , l , L , q , Q , f , d , g , F , D , G , -------------------------------------------------------------------------------------- in2 ? | ? , b , B , h , H , i , I , i , I , q , Q , err, err, err, err, err, err, in2 b | b , b , h , h , i , i , q , i , q , q , O , err, err, err, err, err, err, in2 B | B , h , B , h , H , i , I , i , I , q , Q , err, err, err, err, err, err, in2 h | h , h , h , h , i , i , q , i , q , q , O , err, err, err, err, err, err, in2 H | H , i , H , i , H , i , I , i , I , q , Q , err, err, err, err, err, err, in2 i | i , i , i , i , i , i , q , i , q , q , O , err, err, err, err, err, err, in2 I | I , q , I , q , I , q , I , q , I , q , Q , err, err, err, err, err, err, in2 l | l , l , l , l , l , l , q , l , q , q , O , err, err, err, err, err, err, in2 L | L , q , L , q , L , q , L , q , L , q , Q , err, err, err, err, err, err, in2 q | q , q , q , q , q , q , q , q , q , q , O , err, err, err, err, err, err, in2 Q | Q , O , Q , O , Q , O , Q , O , Q , O , Q , err, err, err, err, err, err, in2 f | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 d | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 g | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 F | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 D | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 G | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, bitwise_xor not symmetric in1 ? , b , B , h , H , i , I , l , L , q , Q , f , d , g , F , D , G , -------------------------------------------------------------------------------------- in2 ? | ? , b , B , h , H , i , I , i , I , q , Q , err, err, err, err, err, err, in2 b | b , b , h , h , i , i , q , i , q , q , O , err, err, err, err, err, err, in2 B | B , h , B , h , H , i , I , i , I , q , Q , err, err, err, err, err, err, in2 h | h , h , h , h , i , i , q , i , q , q , O , err, err, err, err, err, err, in2 H | H , i , H , i , H , i , I , i , I , q , Q , err, err, err, err, err, err, in2 i | i , i , i , i , i , i , q , i , q , q , O , err, err, err, err, err, err, in2 I | I , q , I , q , I , q , I , q , I , q , Q , err, err, err, err, err, err, in2 l | l , l , l , l , l , l , q , l , q , q , O , err, err, err, err, err, err, in2 L | L , q , L , q , L , q , L , q , L , q , Q , err, err, err, err, err, err, in2 q | q , q , q , q , q , q , q , q , q , q , O , err, err, err, err, err, err, in2 Q | Q , O , Q , O , Q , O , Q , O , Q , O , Q , err, err, err, err, err, err, in2 f | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 d | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 g | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 F | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 D | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, in2 G | err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, err, Chuck

1 0

Building universal (fat) OS X binaries with numpy distutils
by Zachary Pincus May 27, 2008

May 27, 2008

Hello all, I'm wondering if anyone could let me know what the current "best practices" are for building universal python extensions on OS X with numpy's distutils and fortran code. Currently, I've been doing what this message suggests: http://mail.python.org/pipermail/pythonmac-sig/2007-June/018986.html That is, get gfortran from http://r.research.att.com/tools/ , copy libgfortran.a to somewhere different like ~/staticlibs/, and then do this to build: export LDFLAGS="-undefined dynamic_lookup -bundle -arch i386 -arch ppc -Wl,-search_paths_first" python setup.py config_fc --fcompiler=gnu95 --arch="-arch i386 -arch ppc" build_ext -L ~/staticlibs/ build Is this still the best bet? Also, how best should one get python itself to compile as universal? (For py2app purposes...) Thanks, Zach

2 4

Re: [Numpy-discussion] Probable errors in type promotions in binary ufuncs.
by Charles R Harris May 26, 2008

May 26, 2008

On Mon, May 26, 2008 at 12:38 PM, Charles R Harris < charlesr.harris(a)gmail.com> wrote: > Continuing the ufunc saga, here are some probable error in the type > promotions. > The data is best viewed in a fixed spaced font, so you might want to use an > html aware mail client. > > The output type of the following functions should be symmetric in the input > types, but aren't. > Note the columns/rows for the l,L types. > <snip> I note that on my machine (32 bit) i and l are the same length. So the first part might only be an error in the sense that the associated dtype.char should be the same in these two cases. If instead this reflects a difference in the underlying c type, then this needs to be checked on 64 bit machines and with different compilers, so I've attached the script I've been using. Chuck

1 0

triangular matrix fill
by Tom Waite May 26, 2008

May 26, 2008

I have a question on filling a lower triangular matrix using numpy. This is essentially having two loops and the inner loop upper limit is the outer loop current index. In the inner loop I have a vector being multiplied by a constant set in the outer loop. For a matrix N*N in size, the C the code is: for(i = 0; i < N; ++i){ for(j = 0; j < i; ++j){ Matrix[i*N + j] = V1[i] * V2[j]; } } Thanks Tom

3 4

David, please check.
by Charles R Harris May 26, 2008

May 26, 2008

David, I merged the OBJECT_API and MULTIARRAY_API lists and fixed the SConstruct file in numpy/core, but you should check if I got it right. Chuck

2 3

build outside C code with scons within numpy distutils
by Zachary Pincus May 26, 2008

May 26, 2008

Hello all, I've been following David's work making numpy build with scons with some interest. I have a quick question about how one might use scons/ numpy distutils from an outside project. Specifically, I have a package that uses numpy and numpy.distutils to built itself. Unfortunately, there are some pure-C libraries that I call using ctypes, and as these libraries are are not python extensions, it is hard to get distutils to build them happily on all platforms. From what I understand, one of the benefits of having scons available was that building and packaging these sort of libraries would become easier. Is that the case? And if so, what would be the best way to add the scons stuff to a simple setup.py file written for numpy.distutils? Thanks, Zach

2 2

Re: [Numpy-discussion] Multiple Boolean Operations
by Peter Creasey May 25, 2008

May 25, 2008

Hi Andrea, 2008/5/23 "Andrea Gavana" <andrea.gavana(a)gmail.com>: > And so on. The probelm with this approach is that I lose the original > indices for which I want all the inequality tests to succeed: To have the original indices you just need to re-index your indices, as it were idx = flatnonzero(xCent >= xMin) idx = idx[flatnonzero(xCent[idx] <= xMax)] idx = idx[flatnonzero(yCent[idx] >= yMin)] idx = idx[flatnonzero(yCent[idx] <= yMax)] ... (I haven't tested this code, apologies for bugs) However, there is a performance penalty for doing all this re-indexing (I once fell afoul of this), and if these conditions "mostly" evaluate to True you can often be better off with one of the solutions already suggested. Regards, Peter

3 6

numpy.test error: test_hdquantiles
by Joshua Lippai May 25, 2008

May 25, 2008

I seem to be getting a few errors and failures with the current numpy SVN (1.2.0.dev5236). I get this output with numpy.test(1,10): ERROR: Ticket #396 ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/core/tests/test_regression.py", line 602, in check_poly1d_nan_roots self.failUnlessRaises(np.linalg.LinAlgError,getattr,p,"r") File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/unittest.py", line 320, in failUnlessRaises callableObj(*args, **kwargs) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/lib/polynomial.py", line 661, in __getattr__ return roots(self.coeffs) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/lib/polynomial.py", line 124, in roots roots = _eigvals(A) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/lib/polynomial.py", line 40, in _eigvals return eigvals(arg) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/linalg/decomp.py", line 478, in eigvals return eig(a,b=b,left=0,right=0,overwrite_a=overwrite_a) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/linalg/decomp.py", line 150, in eig a1 = asarray_chkfinite(a) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/lib/function_base.py", line 527, in asarray_chkfinite raise ValueError, "array must not contain infs or NaNs" ValueError: array must not contain infs or NaNs ====================================================================== ERROR: Ticket #396 ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/core/tests/test_regression.py", line 602, in check_poly1d_nan_roots self.failUnlessRaises(np.linalg.LinAlgError,getattr,p,"r") File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/unittest.py", line 320, in failUnlessRaises callableObj(*args, **kwargs) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/lib/polynomial.py", line 661, in __getattr__ return roots(self.coeffs) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/lib/polynomial.py", line 124, in roots roots = _eigvals(A) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/lib/polynomial.py", line 37, in _eigvals return eigvals(arg) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/linalg/decomp.py", line 478, in eigvals return eig(a,b=b,left=0,right=0,overwrite_a=overwrite_a) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/linalg/decomp.py", line 150, in eig a1 = asarray_chkfinite(a) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/lib/function_base.py", line 527, in asarray_chkfinite raise ValueError, "array must not contain infs or NaNs" ValueError: array must not contain infs or NaNs ====================================================================== ERROR: test_hdquantiles (numpy.ma.tests.test_morestats.TestQuantiles) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ma/tests/test_morestats.py", line 97, in test_hdquantiles hdq = hdquantiles_sd(data,[0.25, 0.5, 0.75]) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ma/morestats.py", line 168, in hdquantiles_sd result = _hdsd_1D(data.compressed(), p) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ma/morestats.py", line 144, in _hdsd_1D xsorted = numpy.sort(data.compressed()) AttributeError: 'numpy.ndarray' object has no attribute 'compressed' ====================================================================== FAIL: Tests the Marits-Jarrett estimator ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ma/tests/test_morestats.py", line 36, in test_mjci assert_almost_equal(mjci(data),[55.76819,45.84028,198.8788],5) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ma/testutils.py", line 134, in assert_almost_equal return assert_array_almost_equal(actual, desired, decimal, err_msg) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ma/testutils.py", line 227, in assert_array_almost_equal header='Arrays are not almost equal') File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ma/testutils.py", line 193, in assert_array_compare assert cond, msg AssertionError: Arrays are not almost equal (mismatch 33.3333333333%) x: array([ 55.76818915, 45.84027529, 198.8787528 ]) y: array([ 55.76819, 45.84028, 198.8788 ]) ====================================================================== FAIL: Test quantiles 1D - w/ mask. ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ma/tests/test_mstats.py", line 61, in test_1d_mask [24.833333, 50.0, 75.166666]) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ma/testutils.py", line 134, in assert_almost_equal return assert_array_almost_equal(actual, desired, decimal, err_msg) File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ma/testutils.py", line 227, in assert_array_almost_equal header='Arrays are not almost equal') File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy/ma/testutils.py", line 193, in assert_array_compare assert cond, msg AssertionError: Arrays are not almost equal (mismatch 66.6666666667%) x: array([ 24.83333333, 50. , 75.16666667]) y: array([ 24.833333, 50. , 75.166666]) ---------------------------------------------------------------------- Ran 1301 tests in 1.501s FAILED (failures=2, errors=3) <unittest._TextTestResult run=1301 errors=3 failures=2> >>> numpy.__version__ '1.2.0.dev5236' >>> scipy.__version__ '0.7.0.dev4386' I get that the failures are just NumPy being anal about the accuracy and rounding habits, but the errors concern me. The first two are the result of a ticket that's only supposed to pop up when SciPy isn't installed (which it is, and I even imported it before running the test). I assume those will go away by some strange voodoo magic on their own eventually because this has happened to me before with that particular ticket. The third error concerns me, though. Is anyone else getting an error with test_hdquantiles?

2 1

isnan surprise
by Keith Goodman May 25, 2008

May 25, 2008

>> x = np.array([1.0]) >> np.isnan(x) array([False], dtype=bool) # <----- Expected >> np.isnan(x,x) array([ 0.]) # <----- Surprise (to me) The same happens with isfinite, isinf, etc. My use case (self.x is an array): def isnan(self): y = self.copy() np.isnan(y.x, y.x) return y Then when I try to do myobj[myobj.isnan()] = 0 I get (since myobj.isnan() are floats) IndexError: arrays used as indices must be of integer (or boolean) type

2 2

ufunc oddities
by Charles R Harris May 25, 2008

May 25, 2008

Hi All, I'm writing tests for ufuncs and turned up some oddities: In [4]: degrees(True) Out[4]: 57.29578 In [5]: radians(True) Out[5]: 0.017453292 In [6]: sin(True) Out[6]: 0.84147096 Do we want numeric functions to apply to booleans? Chuck

7 38