Mailman 3 September 2007 - SciPy-Dev

Strange scipy.test crashes on OS X/Intel
by Tom Loredo Sept. 28, 2007

Sept. 28, 2007

Hi folks- With the problems I've been having with scipy-0.6.0 on RHEL 5, and my colleague with it on FC4, I experimented with it on my OS X machines. MacBook (Intel), OS 10.4, Python 2.5.1, numpy 1.0.3.1: Running scipy.test() the 1st time completed 1719 tests and reported 5 failures: FAIL: check_cosine_weighted_infinite (scipy.integrate.tests.test_quadpack.test_quad) FAIL: check_sine_weighted_finite (scipy.integrate.tests.test_quadpack.test_quad) FAIL: check_sine_weighted_infinite (scipy.integrate.tests.test_quadpack.test_quad) FAIL: check_dot (scipy.lib.tests.test_blas.test_fblas1_simple) FAIL: check_x_stride (scipy.lib.blas.tests.test_fblas.test_cgemv) I ran it again (in the same Python invocation) and got 4 of the 5 failures (all but check_x_stride). I ran it a third time, and it crashed with a seg fault: Found 42 tests for scipy.lib.lapack Found 41 tests for scipy.linalg.basic <module 'scipy.linalg.fblas' from '/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/linalg/fblas.so'> Segmentation fault The seg fault was 100% repeatable: Every time I ran scipy.test() a 3rd time, I got the same seg fault. I deleted all scipy stuff from site-packages, and tried this again with scipy 0.5.2.1. Different tests failed. I got bus errors or seg faults, but inconsistently, after 3-5 runs of scipy.test(), and at different places in the tests (but typically soon after the tests started, i.e., after just a few "..." lines). I removed scipy stuff, and reinstalled 0.6.0. This time I could run scipy.test() 9 times before getting a bus error on the 10th run. Subsequent Python invocations led to bus errors or seg faults sooner and sooner, until the original behavior was duplicated: seg fault on the 3rd run of scipy.test(), always here: Found 41 tests for scipy.linalg.basic <module 'scipy.linalg.fblas' from '/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/scipy/linalg/fblas.so'> Segmentation fault Subsequently, I always get the same crash on the 3rd run of scipy.test(). I tried similar tests on my G4 (PPC) desktop, OS 10.4, same numpy, but with an older Python (2.4.4) and scipy 0.5.2.1. I ran scipy.test() a dozen times with no problems apart from the 3 failures and 9 errors I've always gotten. I realize this is inconclusive, as the problem could be with Python 2.5.1 or gfortran or scipy, but I wanted to report it. It seems a notable coincidence that linalg seems involved in the crashes, since it is also giving me problems on RHEL 5, and my colleague on FC4. If there is anything further I can do to help diagnose these issues, please advise. Thanks, Tom ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/

3 2

Re: [SciPy-dev] Cannot import flapack, but it's there
by Tom Loredo Sept. 26, 2007

Sept. 26, 2007

Hi Jarrod- Thanks for the suggestion: > Could you try deleting the scipy directory and then re-installing it? > rm -rf /usr/local/lib/python2.5/site-packages/scipy I also deleted the scipy-0.6.0-py2.5.egg-info that gets automatically created (presumably by setuptools). No change---the Illegal instruction crash from the ndimage test, and if I disable that, the flapack ImportError. -Tom ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/

1 0

Cannot import flapack, but it's there
by Tom Loredo Sept. 26, 2007

Sept. 26, 2007

Hi again- This is with RHEL 5, Python 2.5.1, gcc 4.2.1, numpy 1.0.3.1, atlas 3.7.37. I've installed scipy 0.6.0. As noted in another post, an ndimage test causes a crash, so I've diabled it and continued. Now scipy.test(1,10) raises an import error: test1 (scipy.special.tests.test_spfun_stats.test_multigammaln) ... ok test_ararg (scipy.special.tests.test_spfun_stats.test_multigammaln) ... ok test_bararg (scipy.special.tests.test_spfun_stats.test_multigammaln) ... ok ====================================================================== ERROR: check_integer (scipy.io.tests.test_array_import.test_read_array) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.5/site-packages/scipy/io/tests/test_array_import.py", line 55, in check_integer from scipy import stats File "/usr/local/lib/python2.5/site-packages/scipy/stats/__init__.py", line 7, in <module> from stats import * File "/usr/local/lib/python2.5/site-packages/scipy/stats/stats.py", line 192, in <module> import scipy.linalg as linalg File "/usr/local/lib/python2.5/site-packages/scipy/linalg/__init__.py", line 8, in <module> from basic import * File "/usr/local/lib/python2.5/site-packages/scipy/linalg/basic.py", line 17, in <module> from lapack import get_lapack_funcs File "/usr/local/lib/python2.5/site-packages/scipy/linalg/lapack.py", line 17, in <module> from scipy.linalg import flapack ImportError: cannot import name flapack ---------------------------------------------------------------------- Ran 787 tests in 5.310s FAILED (errors=1) But if I go into site-packages/scipy/linalg, start python and try "import flapack" there is no error (there is indeed an flapack.so file there). A colleague trying an independent install of scipy 0.6.0 on a FC4 machine also has the tests fail at this point, though in her case she also gets "undefined symbol: isamax_". This makes me wonder if there may be a bug underlying this. Any help on diagnosing/fixing would be appreciated! -Tom ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/

2 1

SciPy SELinux and import bugs?
by Tom Loredo Sept. 26, 2007

Sept. 26, 2007

Hi folks- I've just "upgraded" to RHEL 5 and am going through the chore of installing a ton of software that I previously used under Fedora, including Python/NumPy/SciPy. I've gotten numpy-1.0.3.1 installed (with atlas & fftw) and it passes all tests. But with scipy-0.6.0, scipy.test(1,1) gives me an "Illegal instruction" error, and crashes out of Python. The first such error was an SELinux violation due to a text relocation requirement in _fftpack.so (whatever that means!). I executed the "chcon" context change command SELinux suggested as a workaround. But scipy.test(1,1) still crashes with "Illegal instruction" at apparently the same place, though now without raising an SELinux violation (at least, not one that appears in the SELinux troubleshooter). At this point I thought I'd give the current SVN SciPy (r3369) a try. It installs fine, but I get an error on import: >>> import scipy Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.5/site-packages/scipy/__init__.py", line 63, in <module> pkgload(verbose=SCIPY_IMPORT_VERBOSE,postpone=True) File "/usr/local/lib/python2.5/site-packages/numpy/_import_tools.py", line 177, in __call__ self._init_info_modules(packages or None) File "/usr/local/lib/python2.5/site-packages/numpy/_import_tools.py", line 100, in _init_info_modules self._init_info_modules(getattr(info_module,'depends',[])) File "/usr/local/lib/python2.5/site-packages/numpy/_import_tools.py", line 72, in _init_info_modules exec 'import %s.info as info' % (package_name) File "<string>", line 1, in <module> File "/home/inference/loredo/python/scripts/special.py", line 97, in <module> def factrl(n, ntop=0, prev=N.ones((33),N.Float)): AttributeError: 'module' object has no attribute 'Float' It is finding an old script of mine that happens to be named "special" that I haven't used and thus haven't upgraded from Numeric to NumPy. But that's beside the point---I think it must be a bug that SciPy's import mechanism is finding it. Any suggestions for dealing with either issue (Illegal instruction with 0.6.0, or the import issue for r3369) would be appreciated! -Tom Loredo PS: Regarding the text relocation issue in _fftpack.so, SELinux points to this web page for hints on how to fix it: http://people.redhat.com/drepper/selinux-mem.html ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/

2 1

New scipy 0.6.0 uploads
by Jarrod Millman Sept. 24, 2007

Sept. 24, 2007

There is a new scipy-0.6.0.tar.gz on the sourceforge page, which contains the missing scipy/linalg/src/fblaswrap_veclib_c.c. There is also now a scipy-0.6.0-py2.4-win32.egg and scipy-0.6.0-py2.5-win32.egg. Enjoy, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

4 6

Re: [SciPy-dev] SciPy 0.6.0 release
by dmitrey Sept. 24, 2007

Sept. 24, 2007

Matthieu Brucher wrote: > > The trac can create a zip file (although its name is not the best that > could be found) from a subdirectory, it can help for a starter ? > > Matthieu So how it can be done? I can't get anything related - neither via web search "zip trac" nor "zip" in http://trac.edgewall.org Regards, D.

1 0

scipy.misc.limits deprecated
by Jarrod Millman Sept. 23, 2007

Sept. 23, 2007

Hello, As of r3362, scipy.misc.limits is officially deprecated: http://projects.scipy.org/scipy/scipy/changeset/3362 If you need to work with the machine limits, please use numpy.finfo instead: http://scipy.org/scipy/numpy/browser/trunk/numpy/lib/getlimits.py For example, from numpy import finfo, single single_epsilon = finfo(single).eps -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

1 0

Re: [SciPy-dev] sandbox
by Jeff Whitaker Sept. 22, 2007

Sept. 22, 2007

Nils Wagner wrote: > On Sat, 22 Sep 2007 09:34:01 -0600 > Jeff Whitaker <jswhit(a)fastmail.fm> wrote: >> Nils Wagner wrote: >>> On Sat, 22 Sep 2007 08:37:31 -0600 >>> Jeff Whitaker <jswhit(a)fastmail.fm> wrote: >>> >>>> How do I import the sandbox packages? With 0.6.0 I get >>>> >>>> >>>>>>> import scipy.sandbox >>>>>>> dir(scipy.sandbox) >>>>>>> >>>> ['__builtins__', '__doc__', '__file__', '__name__', '__path__'] >>>> >>>> >>>> even though >>>> >>>> [jsw@Macintosh:/Users/jsw] ls -l >>>> /sw/lib/python2.5/site-packages/scipy/sandbox/ >>>> total 16 >>>> -rw-r--r-- 1 root admin 0 Sep 22 08:28 __init__.py >>>> -rw-r--r-- 1 root admin 144 Sep 22 08:04 __init__.pyc >>>> -rw-r--r-- 1 root admin 121 Sep 22 08:27 __init__.py~ >>>> drwxr-xr-x 9 root admin 306 Sep 22 08:22 buildgrid >>>> drwxr-xr-x 16 root admin 544 Sep 22 08:22 cdavid >>>> drwxr-xr-x 10 root admin 340 Sep 22 08:22 constants >>>> drwxr-xr-x 13 root admin 442 Sep 22 08:22 delaunay >>>> drwxr-xr-x 13 root admin 442 Sep 22 08:22 exmplpackage >>>> drwxr-xr-x 12 root admin 408 Sep 22 08:22 fdfpack >>>> drwxr-xr-x 39 root admin 1326 Sep 22 08:22 models >>>> drwxr-xr-x 22 root admin 748 Sep 22 08:22 multigrid >>>> drwxr-xr-x 6 root admin 204 Sep 22 08:22 netcdf >>>> drwxr-xr-x 16 root admin 544 Sep 22 08:22 numexpr >>>> drwxr-xr-x 26 root admin 884 Sep 22 08:22 pyem >>>> drwxr-xr-x 11 root admin 374 Sep 22 08:22 rbf >>>> drwxr-xr-x 10 root admin 340 Sep 22 08:22 rkern >>>> -rw-r--r-- 1 root admin 2719 Sep 22 07:47 setup.py >>>> -rw-r--r-- 1 root admin 1500 Sep 22 08:04 setup.pyc >>>> drwxr-xr-x 14 root admin 476 Sep 22 08:22 spline >>>> drwxr-xr-x 24 root admin 816 Sep 22 08:22 svm >>>> drwxr-xr-x 114 root admin 3876 Sep 22 08:22 xplt >>>> >>>> -Jeff >>>> >>>> >>>> -- >>>> Jeffrey S. Whitaker Phone : (303)497-6313 >>>> NOAA/OAR/CDC R/PSD1 FAX : (303)497-6449 >>>> 325 Broadway Boulder, CO, USA 80305-3328 >>>> >>>> _______________________________________________ >>>> Scipy-dev mailing list >>>> Scipy-dev(a)scipy.org >>>> http://projects.scipy.org/mailman/listinfo/scipy-dev >>>> >>> >>> >>> You have to enable them first. >>> Create a file enabled_packages.txt in your >>> directory scipy/scipy/sandbox with appropriate >>> entries >>> e.g. >>> >>> arpack >>> lobpcg >>> timeseries >>> multigrid >>> >>> >>> If you have rebuild scipy you can import them with >>> >>> from scipy.sandbox import arpack >>> >>> >>> Cheers, >>> Nils >>> >> >> Nils: I did this, and the packages were built and installed. I just >> can't import them. My scipy.sandbox __init__.py file is empty, is >> yours? >> >> -Jeff >> >> -- >> Jeffrey S. Whitaker Phone : (303)497-6313 >> NOAA/OAR/CDC R/PSD1 FAX : (303)497-6449 >> 325 Broadway Boulder, CO, USA 80305-3328 >> > ls -l > total 28 > drwxr-xr-x 3 root root 4096 2007-09-21 16:42 arpack > -rw-r--r-- 1 root root 0 2007-09-01 13:01 __init__.py > -rw-r--r-- 1 root root 153 2007-09-11 20:00 __init__.pyc > drwxr-xr-x 3 root root 4096 2007-09-11 20:52 lobpcg > drwxr-xr-x 3 root root 4096 2007-09-21 16:42 multigrid > -rw-r--r-- 1 root root 2735 2007-09-01 13:01 setup.py > -rw-r--r-- 1 root root 1094 2007-09-11 20:00 setup.pyc > drwxr-xr-x 3 root root 4096 2007-09-21 16:42 timeseries > nwagner@noname:/usr/local/lib64/python2.5/site-packages/scipy/sandbox> > > > Did you try from scipy.sandbox import multigrid ? > > Nils Nils: Ah, yes that works. I was trying from scipy.sandbox import netcdf which doesn't work (although all the rest apparently do). So it must be a problem just with the netcdf package. Thanks for the help -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 NOAA/OAR/CDC R/PSD1 FAX : (303)497-6449 325 Broadway Boulder, CO, USA 80305-3328

1 0

sandbox
by Jeff Whitaker Sept. 22, 2007

Sept. 22, 2007

How do I import the sandbox packages? With 0.6.0 I get >>> import scipy.sandbox >>> dir(scipy.sandbox) ['__builtins__', '__doc__', '__file__', '__name__', '__path__'] even though [jsw@Macintosh:/Users/jsw] ls -l /sw/lib/python2.5/site-packages/scipy/sandbox/ total 16 -rw-r--r-- 1 root admin 0 Sep 22 08:28 __init__.py -rw-r--r-- 1 root admin 144 Sep 22 08:04 __init__.pyc -rw-r--r-- 1 root admin 121 Sep 22 08:27 __init__.py~ drwxr-xr-x 9 root admin 306 Sep 22 08:22 buildgrid drwxr-xr-x 16 root admin 544 Sep 22 08:22 cdavid drwxr-xr-x 10 root admin 340 Sep 22 08:22 constants drwxr-xr-x 13 root admin 442 Sep 22 08:22 delaunay drwxr-xr-x 13 root admin 442 Sep 22 08:22 exmplpackage drwxr-xr-x 12 root admin 408 Sep 22 08:22 fdfpack drwxr-xr-x 39 root admin 1326 Sep 22 08:22 models drwxr-xr-x 22 root admin 748 Sep 22 08:22 multigrid drwxr-xr-x 6 root admin 204 Sep 22 08:22 netcdf drwxr-xr-x 16 root admin 544 Sep 22 08:22 numexpr drwxr-xr-x 26 root admin 884 Sep 22 08:22 pyem drwxr-xr-x 11 root admin 374 Sep 22 08:22 rbf drwxr-xr-x 10 root admin 340 Sep 22 08:22 rkern -rw-r--r-- 1 root admin 2719 Sep 22 07:47 setup.py -rw-r--r-- 1 root admin 1500 Sep 22 08:04 setup.pyc drwxr-xr-x 14 root admin 476 Sep 22 08:22 spline drwxr-xr-x 24 root admin 816 Sep 22 08:22 svm drwxr-xr-x 114 root admin 3876 Sep 22 08:22 xplt -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 NOAA/OAR/CDC R/PSD1 FAX : (303)497-6449 325 Broadway Boulder, CO, USA 80305-3328

3 3

SciPy 0.6.0 release
by Jarrod Millman Sept. 21, 2007

Sept. 21, 2007

I'm pleased to announce the release of SciPy 0.6.0. SciPy is a package of tools for science and engineering for Python. It includes modules for statistics, optimization, integration, linear algebra, Fourier transforms, signal and image processing, ODE solvers, and more. This release brings many bugfixes and speed improvements. Major changes since 0.5.2.1: * cluster o cleaned up kmeans code and added a kmeans2 function that adds several initialization methods * fftpack o fft speedups for complex data o major overhaul of fft source code for easier maintenance * interpolate o add Lagrange interpolating polynomial o fix interp1d so that it works for higher order splines * io o read and write basic .wav files * linalg o add Cholesky decomposition and solution of banded linear systems with Hermitian or symmetric matrices o add RQ decomposition * ndimage o port to NumPy API o fix byteswapping problem in rotate o better support for 64-bit platforms * optimize o nonlinear solvers module added o a lot of bugfixes and modernization * signal o add complex Morlet wavelet * sparse o significant performance improvements Thank you to everybody who contributed to the recent release. Enjoy, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

6 9