CVS scipy.test() segfaulting on me on a Fedora2 box
Hi all, I just updated to current CVS of f2py/scipy and Numeric 23.6, to try to root out some problems I was encountering with scipy.test(). Now I'm having a different problem, scipy is segfaulting on me :) I've narrowed it down to this: planck[pylab]> python -c 'import scipy;scipy.lib.lapack.test(verbosity=2)' !! No test file 'test_flapack.py' found for <module 'scipy.lib.lapack.flapack' from '...es/scipy/lib/lapack/flapack.so'> !! No test file 'test_clapack.py' found for <module 'scipy.lib.lapack.clapack' from '...es/scipy/lib/lapack/clapack.so'> Found 78 tests for scipy.lib.lapack !! No test file 'test_calc_lwork.py' found for <module 'scipy.lib.lapack.calc_lwork' from '...scipy/lib/lapack/calc_lwork.so'> Found 0 tests for __main__ check_gebal (scipy.lib.lapack.test_lapack.test_flapack_complex) ... ok check_heev (scipy.lib.lapack.test_lapack.test_flapack_complex)Segmentation fault If I go manually into test_lapack.py and comment out around line 93: #class test_flapack_complex(_test_lapack): # lapack = PrefixWrapper(flapack,'c') # decimal = 5 The whole suite runs fine: planck[pylab]> python -c 'import scipy;scipy.test(level=10,verbosity=2)' [...] check_basic (scipy.signal.signaltools.test_signaltools.test_wiener) ... ok ---------------------------------------------------------------------- Ran 1166 tests in 83.200s OK By digging a bit deeper into the testing code, I was able to track the segfault to this (a defined here is copied from the test code, the matrix so defined is correctly hermitian as required by cheev, so this shouldn't be a problem): In [1]: a = [[1,2,3],[2,2,3],[3,3,6]] In [2]: import scipy In [3]: scipy.lib.lapack.flapack.cheev(a) Segmentation fault The coredump is not terribly informative (unfortunately I don't have a debug build of python around to produce more details with): #0 0x55cd67c2 in ATL_cdotc_xp0yp0aXbX () from /usr/lib/python2.3/site-packages/scipy/lib/lapack/flapack.so #1 0xfee241b8 in ?? () #2 0x09bfb880 in ?? () #3 0xfee241f8 in ?? () #4 0x55cc4228 in cdotc_ () from /usr/lib/python2.3/site-packages/scipy/lib/lapack/flapack.so #5 0x4083e1dc in ?? () #6 0x560f1358 in ?? () from /usr/lib/python2.3/site-packages/scipy/lib/lapack/flapack.so #7 0x3ed413ce in ?? () #8 0x560f1358 in ?? () from /usr/lib/python2.3/site-packages/scipy/lib/lapack/flapack.so #9 0x3ed413ce in ?? () #10 0xfee24100 in ?? () #11 0x00000000 in ?? () Other calls to complex routines also appear to segfault, though I didn't test too many more. I wonder if anyone knows what may be going on here. This machine has in the past run successfully all scipy tests, and I did not change my ATLAS installation. For reference, here are all my version numbers (I just pulled scipy and f2py from CVS minutes ago, and grabbed Numpy from Sourceforge): In [9]: print sys.version 2.3.3 (#1, May 7 2004, 10:31:40) [GCC 3.3.3 20040412 (Red Hat Linux 3.3.3-7)] In [10]: Numeric.__version__ Out[10]: '23.6' In [11]: f2py2e.__version__.version Out[11]: '2.44.240_1906' In [12]: scipy.__version__ Out[12]: '0.3.2_300.4521' I installed scipy with a straight 'setup.py install', without touching any of the config files (as I've done successfully in the past). Thanks for any help. I'll gladly provide more info if needed, f
On Tue, 21 Dec 2004, Fernando Perez wrote:
Hi all,
I just updated to current CVS of f2py/scipy and Numeric 23.6, to try to root out some problems I was encountering with scipy.test(). Now I'm having a different problem, scipy is segfaulting on me :)
I've narrowed it down to this:
planck[pylab]> python -c 'import scipy;scipy.lib.lapack.test(verbosity=2)' !! No test file 'test_flapack.py' found for <module 'scipy.lib.lapack.flapack' from '...es/scipy/lib/lapack/flapack.so'> !! No test file 'test_clapack.py' found for <module 'scipy.lib.lapack.clapack' from '...es/scipy/lib/lapack/clapack.so'> Found 78 tests for scipy.lib.lapack !! No test file 'test_calc_lwork.py' found for <module 'scipy.lib.lapack.calc_lwork' from '...scipy/lib/lapack/calc_lwork.so'> Found 0 tests for __main__ check_gebal (scipy.lib.lapack.test_lapack.test_flapack_complex) ... ok check_heev (scipy.lib.lapack.test_lapack.test_flapack_complex)Segmentation fault
Todd had similar problem and my response: """ Second, some gcc Fortran compilers produce incorrect code when using -O3 optimization flag and there have been reports that they cause segfaults in heev tests. See the get_flags_opt method in scipy_distutils/gnufcompiler.py and replace the line if self.get_version()=='3.3.3': with if self.get_version()<='3.3.3': Do `rm -rf build` and rebuild scipy. Note that if this issue is related to g77 optimization bug then you should also rebuild Fortran lapack library (that used in completing atlas lapack library) with -O2 flag and before rebuilding scipy. """ seemed to help him. Todd was using gcc 3.2.2.
Other calls to complex routines also appear to segfault, though I didn't test too many more. I wonder if anyone knows what may be going on here. This machine has in the past run successfully all scipy tests, and I did not change my ATLAS installation.
These are new tests, so the issue were not discovered before. HTH, Pearu
Hi Pearu, Pearu Peterson schrieb:
Todd had similar problem and my response:
""" Second, some gcc Fortran compilers produce incorrect code when using -O3 optimization flag and there have been reports that they cause segfaults in heev tests. See the get_flags_opt method in scipy_distutils/gnufcompiler.py and replace the line
if self.get_version()=='3.3.3':
with
if self.get_version()<='3.3.3':
Do `rm -rf build` and rebuild scipy.
Note that if this issue is related to g77 optimization bug then you should also rebuild Fortran lapack library (that used in completing atlas lapack library) with -O2 flag and before rebuilding scipy. """
seemed to help him. Todd was using gcc 3.2.2.
Well, I'm using 3.3.3, so the code is already building with -02 only. I hardcoded this change (for 3.3.3) to plain -O instead, and this thing segfaults all the same. However, I did NOT rebuild ATLAS/LAPACK myself, I'm using those from the scipy ATLAS binaries page. Oh well, I'm switching to FC3 very soon, so I'll report again if the problem persists there. For now it's not an issue for me, since I don't need those routines. I figured I'd let you guys know about it, at least. Regards, and many thanks for the help, f
Hi I've been having the same problems with gcc version 3.4.1 (Mandrakelinux 10.1 3.4.1-4mdk). As suggested I recompiled BLAS/LAPACK with -O2 updated my ATLAS libs and changed gnufcompiler.py to: if self.get_version()<='3.4.2': to rebuild SciPy with -O2. However, no luck, I still get the segfault in scipy.test() with heev. Has anyone else tried compiling SciPy with 3.4.x compilers (I've had additional adventures with MinGW GCC 3.4.2 but will post these a bit later) ? Brett On Wednesday 22 December 2004 04:56, Fernando Perez wrote:
Well, I'm using 3.3.3, so the code is already building with -02 only. I hardcoded this change (for 3.3.3) to plain -O instead, and this thing segfaults all the same. However, I did NOT rebuild ATLAS/LAPACK myself, I'm using those from the scipy ATLAS binaries page.
Oh well, I'm switching to FC3 very soon, so I'll report again if the problem persists there. For now it's not an issue for me, since I don't need those routines. I figured I'd let you guys know about it, at least.
Regards, and many thanks for the help,
f
_______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.net http://www.scipy.net/mailman/listinfo/scipy-dev
-- Brett G. Olivier National Bioinformatics Network - Stellenbosch Node Triple-J Group for Molecular Cell Physiology, Stellenbosch University bgoli at sun dot ac dot za http://glue.jjj.sun.ac.za/~bgoli Tel +27-21-8082697 Fax +27-21-8085863 Mobile +27-82-7329306 There are many intelligent species in the universe, and they all own cats.
On Wed, 22 Dec 2004, Brett Olivier wrote:
I've been having the same problems with gcc version 3.4.1 (Mandrakelinux 10.1 3.4.1-4mdk). As suggested I recompiled BLAS/LAPACK with -O2 updated my ATLAS libs and changed gnufcompiler.py to: if self.get_version()<='3.4.2': to rebuild SciPy with -O2.
However, no luck, I still get the segfault in scipy.test() with heev. Has anyone else tried compiling SciPy with 3.4.x compilers (I've had additional adventures with MinGW GCC 3.4.2 but will post these a bit later) ?
I have gcc version 3.4.4, 3.3.5 in my debian sid box and have no trouble at all with scipy tests when building against these compilers. I am using also debian ATLAS 3.6.0 that is built against g77 3.3.3 using -O optimization. I'd suggest building scipy against Fortran BLAS/LAPACK libraries and see if the problem percist. For that, get blas and lapack sources from netlib, unpack them, and set up the following environment for building scipy: ATLAS=None LAPACK=None BLAS=None BLAS_SRC=<path to directory where blas.tgz was unpacked> LAPACK_SRC=<path to directory where lapack.tgz was unpacked/LAPACK> And in cvs/scipy/Lib/lib/lapack directory execute: rm -rf build python setup_lapack.py build The last command could be modified also to python setup_lapack.py config_fc --noopt build python setup_lapack.py config_fc --noopt --noarch build python setup_lapack.py config_fc --opt="-O" build so that you don't have to modify gnufcompiler.py and reinstall scipy_core all the time. To run lapack tests without installing scipy, execute python tests/test_lapack.py -v 10 HTH, Pearu
Pearu Peterson wrote:
I have gcc version 3.4.4, 3.3.5 in my debian sid box and have no trouble at all with scipy tests when building against these compilers. I am using also debian ATLAS 3.6.0 that is built against g77 3.3.3 using -O optimization.
Mmh, weird. Those seem to be the flags in the ATLAS binaries you shipped at scipy.org.
I'd suggest building scipy against Fortran BLAS/LAPACK libraries and see if the problem percist. For that, get blas and lapack sources from netlib, unpack them, and set up the following environment for building scipy:
As I said, I'll be soon updating these boxes to Fedora3, so I'd rather not spend too much time on this right now (I'm pretty behind on other things as it is :). Many thanks for the feedback though. At least I know I'm not the only one having these problems. Cheers, f
participants (3)
-
Brett Olivier -
Fernando Perez -
Pearu Peterson