Mailman 3 January 2012 - SciPy-Dev

installation of scipy on Mac OS X 10.7
by Ben Willmore 26 Jan '12

26 Jan '12

I have spent a day attempting to get a version of scipy that passes unit tests on Mac OS X 10.7. I hope the results of my investigations (below) are useful to others trying to do the same thing. Ben == Introduction I have attempted to obtain a version of scipy on Mac OS X 10.7 that passes unit tests, both by downloading it, or by compiling against the system's python, using the three C compilers available, and various compile flags. == Conclusions 1. Both Enthought Python 7.1 and the scipy superpack crash during scipy unit tests. 2. Compilation with the default C compiler (llvm-gcc-4.2) also leads to a crash during unit tests. 3. Compilation using either of the other C compilers (gcc-4.2 or clang) allows scipy to complete unit tests, only if the fortran flag --ff2c is used. In both cases, scipy passes almost all tests -- producing errors on one, and failing two more. With gcc-4.2, numpy passes all tests. With clang, numpy fails one test. 4. The following recipe was most successful for me: * Remove everything in /usr/local and /Library/Python/2.7/site-packages (BEWARE! These directories may well contain stuff you care about) * Install gfortran from here: http://r.research.att.com/gfortran-lion-5666-3.pkg mkdir ~/tmp cd ~/tmp git clone git://github.com/numpy/numpy.git cd numpy export CC=gcc-4.2 export CXX=g++-4.2 export FFLAGS=-ff2c python setupegg.py build --fcompiler=gfortran sudo python setupegg.py install cd ~/tmp git clone git://github.com/scipy/scipy.git cd scipy export CC=gcc-4.2 export CXX=g++-4.2 export FFLAGS=-ff2c python setupegg.py build --fcompiler=gfortran sudo python setupegg.py install (repeat for matplotlib, ipython ...) == Details of test results = Enthought python 7.1 numpy.test(): OK scipy.test(): Crash (Segmentation fault) = Scipy superpack numpy.test(): 3 failures: FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, 1, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -1, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, 0.0, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -0.0, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, inf, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -inf, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, nan, inf, inf, inf) FAIL: test_umath_complex.TestCsqrt.test_special_values(<ufunc 'sqrt'>, -inf, 1, 0.0, inf) scipy.test(): Crash (Abort trap) = Compilation from scratch (see above for recipe) = llvm-gcc-4.2 = (no compiler flags) numpy.test(): OK scipy.test(): Crash (Segmentation fault: 11) = gcc-4.2 = CC=gcc-4.2 CXX=g++-4.2 numpy.test(): OK scipy.test(): Hang (during ARPACK tests) = clang = CC=clang CXX=clang++ numpy.test(): 1 failure: FAIL: Test basic arithmetic function errors AssertionError: Type <type 'numpy.complex64'> did not raise fpe error ''. scipy.test(): Hang (during ARPACK tests) = llvm-gcc-4.2 and -ff2c = FFLAGS=-ff2c numpy.test(): OK scipy.test(): Crash (Segmentation fault: 11) = gcc-4.2 and -ff2c = CC=gcc-4.2 CXX=g++-4.2 FFLAGS=-ff2c numpy.test(): OK scipy.test(): 1 error, 2 failures: ERROR: test_arpack.test_hermitian_modes(True, <gen-hermitian>, 'F', 2, 'SA', None, None, <function aslinearoperator at 0x1098cd500>) FAIL: test_iterative.test_convergence(<function bicg at 0x1098cdd70>, <nonsymposdef>) FAIL: test_expon (test_morestats.TestAnderson) = clang and -ff2c = CC=clang CXX=clang++ FFLAGS=-ff2c numpy.test(): 1 failure: FAIL: Test basic arithmetic function errors AssertionError: Type <type 'numpy.complex64'> did not raise fpe error ''. scipy.test(): 1 error, 2 failures: ERROR: test_arpack.test_hermitian_modes(True, <gen-hermitian>, 'F', 2, 'SA', None, None, <function aslinearoperator at 0x1098cd500>) FAIL: test_iterative.test_convergence(<function bicg at 0x1098cdd70>, <nonsymposdef>) FAIL: test_expon (test_morestats.TestAnderson)

6 6

views and mask NA
by Charles R Harris 26 Jan '12

26 Jan '12

Hi All, I'd like some feedback on how mask NA should interact with views. The immediate problem is how to deal with the real and imaginary parts of complex numbers. If the original has a masked value, it should show up as masked in the real and imaginary parts. But what should happen on assignment to one of the masked views? This should probably clear the NA in the real/imag part, but not in the complex original. However, that does allow touching things under the mask, so to speak. Things get more complicated if the complex original is viewed as reals. In this case the mask needs to be "doubled" up, and there is again the possibility of touching things beneath the mask in the original. Viewing the original as bytes leads to even greater duplication. My thought is that touching the underlying data needs to be allowed in these cases, but the original mask can only be cleared by assignment to the original. Thoughts? Chuck

3 8

mwavepy scikit
by alex arsenovic 23 Jan '12

23 Jan '12

hello, my name is alex arsenovic. i am the author of the python module mwavepy, which is a package for RF/microwave engineering. homepage: http://code.google.com/p/mwavepy/ docs: http://packages.python.org/mwavepy/# it is my understanding that scipy doesnt currently have the functionality provided by mwavepy, and it seems as though it would be a valuable module to have, similar to the rf-toolbox in matlab. i was entertaining the idea of making a sci-kit for mwavepy, and was curious about the scipy-dev community's opinion on this. does a module like this belong as a sci-kit? if so, i have numerous questions as to what are the next steps. thanks, alex

6 9

Bug in use of np.sign() function with sparse csc_matrix?
by Warren Weckesser 22 Jan '12

22 Jan '12

In all the examples that I've tried with a sparse csc_matrix `a`, `sign(a)` always returns 1. I expect it to return the matrix of element-wise signs of a. For example: In [1]: from scipy.sparse import csc_matrix In [2]: a = csc_matrix([[0.0, 1.0, 2.0], [0.0, 0.0, -3.0], [0.0, 0.0, 0.0]]) In [3]: a.todense() Out[3]: matrix([[ 0., 1., 2.], [ 0., 0., -3.], [ 0., 0., 0.]]) In [4]: np.sign(a.todense()) Out[4]: matrix([[ 0., 1., 1.], [ 0., 0., -1.], [ 0., 0., 0.]]) In [5]: np.sign(a) # Incorrect result? Out[5]: 1 In [6]: import scipy In [7]: scipy.__version__ Out[7]: '0.11.0.dev-81dc505' In [8]: np.__version__ Out[8]: '1.6.1' I think that's a bug, but if someone knows better, let me know! Warren

3 5

Feedback wanted on a couple bug-fix pull requests
by Warren Weckesser 22 Jan '12

22 Jan '12

Reviews of the following bug fixes would be appreciated: The linkage function in scipy.cluster has several calls to malloc but the return value was not checked. This could result in a segfault when memory was low. This pull request checks the result of each malloc call in the function, and raises a MemoryError if it fails: https://github.com/scipy/scipy/pull/110 signal.lfilter could segfault if given object arrays. In this pull request, that is fixed by checking that all the objects are in fact numbers: https://github.com/scipy/scipy/pull/112 Thanks, Warren

1 1

discrete sine transforms
by Matt Terry 20 Jan '12

20 Jan '12

Hi, I noticed that scipy.fftpack has discrete cosine transforms, but not sine transforms. It also looks like the dst's are in the fftpack source, just not the scipy wrappers. Is there a special reason for not wrapping the dst's, or do they just lack a champion? -matt

3 6

FAIL: TNC: test 1 (approx. gradient)
by Nils Wagner 17 Jan '12

17 Jan '12

====================================================================== FAIL: TNC: test 1 (approx. gradient) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/optimize/tests/test_optimize.py", line 590, in test_tnc1b optimize.tnc.RCSTRINGS[rc]) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/utils.py", line 1213, in assert_allclose verbose=verbose, header=header) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/utils.py", line 677, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=1e-07, atol=1e-06 TNC failed with status: Converged (|f_n-f_(n-1)| ~= 0) (mismatch 100.0%) x: array(6.8110123995028524e-06) y: array(0.0) ---------------------------------------------------------------------- Ran 5186 tests in 173.913s FAILED (KNOWNFAIL=12, SKIP=28, failures=1) <nose.result.TextTestResult run=5186 errors=0 failures=1> >>> scipy.__version__ '0.11.0.dev-5e4b68f'

2 1

Strange truncation of autosummary docstring
by Warren Weckesser 17 Jan '12

17 Jan '12

I just added the rosenbrock functions to the `optimize` module docstring in optimize/__init__.py. When I rebuild the docs, the optimize page (which is html output of the docstring in __init__.py) shows this for rosen_der: rosen_der(x) The derivative (i.e. It should be rosen_der(x) The derivative (i.e. gradient) of the Rosenbrock function. The text after 'rosen_der(x)' should be the first line of the docstring. Is the truncation because of some cleverness in the numpydoc sphinx extension? Warren

3 2

Proposal for Scikit-Signal - a SciPy toolbox for signal processing
by Jaidev Deshpande 17 Jan '12

17 Jan '12

Hi I gave a talk at SciPy India 2011 about a Python implementation of the Hilbert-Huang Transform that I was working on. The HHT is a method used as an alternative to Fourier and Wavelet analyses of nonlinear and nonstationary data. Following the talk Gael Varoquaux said that there's room for a separate scikit for signal processing. He also gave a lightning talk about bootstrapping a SciPy community project soon after. So with this list let us start working out what the project should be like. For noobs like me, Gael's talk was quite a useful guide. Here's the link to a gist he made about it - https://gist.github.com/1433151 Here's the link to my SciPy talk: http://urtalk.kpoint.in/kapsule/gcc-57b6c86b-2f12-4244-950c-a34360a2cc1f/vi… I personally am researching nonlinear and nonstationary signal processing, I'd love to know what others can bring to this project. Also, let's talk about the limitations of the current signal processing tools available in SciPy and other scikits. I think there's a lot of documentation to be worked out, and there is also a lack of physically meaningful examples in the documentation. Thanks PS: I'm ccing a few people who might already be on the scipy-dev list. Sorry for the inconvenience.

20 73

check_finite pull request
by Christopher Jordan-Squire 17 Jan '12

17 Jan '12

Hi, This is an old pull request, but it's been active recently. https://github.com/scipy/scipy/pull/48 adds a check_finite flag to scipy.linalg functions that allows for disabling a check on whether there are infs/nans in the array. This can save (a lot) of speed in certain cases, such as when you're calling a linalg function over and over again on a matrix that you've already verified, outside the linalg function, consists of non-nan, finite numbers. It also has the potential for very bad things to happen. If you disable the checks and then pass an array to some of the functions, you'll silently get garbage out, or, even worse, the program will freeze. By default the checks remain on, of course, so this can only happen if the checks are intentionally disabled and garbage is passed in. What bad behavior you get depends on the function, and that in turn depends on the underlying linear algebra function called. (e.g., the LAPACK function called.) Click on the link for some more details, or review the old thread (from late August/early September). The previous consensus then appeared to be, "We should be able to find a better solution." Various ideas included writing a C/cython/assembly function to make the chkfinite function faster, or even adding some flag to the numpy arrays themselves, but as yet nothing has been added. (And several of the previous ideas proved less workable/fruitful than we'd hoped.) Which is unfortunate, since this feature addresses a common problem in data analysis fields, like statistics and machine learning. -Chris Jordan-Squire

1 0