scipy.linalg.eigvals error on 64bit RH3 & RH4
![](https://secure.gravatar.com/avatar/fd3ac07ad0dde7f04231ad1a886db35f.jpg?s=120&d=mm&r=g)
Has anyone else gotten these scipy.test() errors when running on RH3 and RH4 (both 64bit)? The following tests pass on RH5 64-bit and RH3,4,5 32-bit, so this seems to be a 64-bit RH issue prior to RH5. It looks like eigvals isn't returning the correct values: -David On Windows XP 32-bit: Python 2.5.4 (r254:67916, May 17 2009, 21:12:18) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
from numpy import sqrt from scipy.linalg import eigvals a = [[1,2,3],[1,2,3],[2,5,6]] w = eigvals(a) exact_w = [(9+sqrt(93))/2,0,(9-sqrt(93))/2] w array([ 9.32182538e+00+0.j, -9.59852072e-16+0.j, -3.21825380e-01+0.j]) exact_w [9.3218253804964775, 0, -0.32182538049647746]
On RH3.8 64-bit: Python 2.5.4 (r254:67916, May 20 2009, 22:57:32) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-56)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from numpy import sqrt from scipy.linalg import eigvals a = [[1,2,3],[1,2,3],[2,5,6]] w = eigvals(a) exact_w = [(9+sqrt(93))/2,0,(9-sqrt(93))/2] w array([ 9.43719064+0.j, -0.11536526+0.j, -0.32182538+0.j]) exact_w [9.3218253804964775, 0, -0.32182538049647746]
====================================================================== FAIL: test_simple (test_decomp.TestEig) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/tester/master/lib/python2.5/site-packages/scipy/linalg/tests/test_decomp.py", line 145, in test_simple assert_array_almost_equal(w,exact_w) File "/home/tester/master/lib/python2.5/site-packages/numpy/testing/utils.py", line 537, in assert_array_almost_equal header='Arrays are not almost equal') File "/home/tester/master/lib/python2.5/site-packages/numpy/testing/utils.py", line 395, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal (mismatch 66.6666666667%) x: array([ 9.43719064+0.j, -0.11536526+0.j, -0.32182538+0.j]) y: array([ 9.32182538, 0. , -0.32182538]) ====================================================================== FAIL: test_simple (test_decomp.TestEigVals) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/tester/master/lib/python2.5/site-packages/scipy/linalg/tests/test_decomp.py", line 114, in test_simple assert_array_almost_equal(w,exact_w) File "/home/tester/master/lib/python2.5/site-packages/numpy/testing/utils.py", line 537, in assert_array_almost_equal header='Arrays are not almost equal') File "/home/tester/master/lib/python2.5/site-packages/numpy/testing/utils.py", line 395, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal (mismatch 66.6666666667%) x: array([ 9.43719064+0.j, -0.11536526+0.j, -0.32182538+0.j]) y: array([ 9.32182538, 0. , -0.32182538]) ====================================================================== FAIL: test_simple_tr (test_decomp.TestEigVals) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/tester/master/lib/python2.5/site-packages/scipy/linalg/tests/test_decomp.py", line 122, in test_simple_tr assert_array_almost_equal(w,exact_w) File "/home/tester/master/lib/python2.5/site-packages/numpy/testing/utils.py", line 537, in assert_array_almost_equal header='Arrays are not almost equal') File "/home/tester/master/lib/python2.5/site-packages/numpy/testing/utils.py", line 395, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal (mismatch 66.6666666667%) x: array([ 9.43719064+0.j, -0.11536526+0.j, -0.32182538+0.j]) y: array([ 9.32182538, 0. , -0.32182538]) ====================================================================== FAIL: test (test_speigs.TestEigs) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/tester/master/lib/python2.5/site-packages/scipy/sparse/linalg/eigen/arpack/tests/test_speigs.py", line 34, in test assert_array_almost_equal(calc_vals, vals[0:nev], decimal=7) File "/home/tester/master/lib/python2.5/site-packages/numpy/testing/utils.py", line 537, in assert_array_almost_equal header='Arrays are not almost equal') File "/home/tester/master/lib/python2.5/site-packages/numpy/testing/utils.py", line 395, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal (mismatch 100.0%) x: array([ 0.03559256, 0.73900411, 0.73525312, 1.07213911]) y: array([ 0.25819889, 0.51639778, 0.77459667, 1.03279556])
![](https://secure.gravatar.com/avatar/764323a14e554c97ab74177e0bce51d4.jpg?s=120&d=mm&r=g)
On Mon, Jul 13, 2009 at 16:43, David Martin<frostedcheerios@gmail.com> wrote:
Has anyone else gotten these scipy.test() errors when running on RH3 and RH4 (both 64bit)? The following tests pass on RH5 64-bit and RH3,4,5 32-bit, so this seems to be a 64-bit RH issue prior to RH5. It looks like eigvals isn't returning the correct values:
Try to run the ATLAS tests for the version you built scipy against, if you can. Incorrect values from eigvals() usually mean that the underlying library is having problems, not scipy. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
![](https://secure.gravatar.com/avatar/fd3ac07ad0dde7f04231ad1a886db35f.jpg?s=120&d=mm&r=g)
Robert Kern wrote:
On Mon, Jul 13, 2009 at 16:43, David Martin<frostedcheerios@gmail.com> wrote:
Has anyone else gotten these scipy.test() errors when running on RH3 and RH4 (both 64bit)? The following tests pass on RH5 64-bit and RH3,4,5 32-bit, so this seems to be a 64-bit RH issue prior to RH5. It looks like eigvals isn't returning the correct values:
Try to run the ATLAS tests for the version you built scipy against, if you can. Incorrect values from eigvals() usually mean that the underlying library is having problems, not scipy.
Unfortunately I only had the .so libraries to link against so I couldn't run the ATLAS sanity tests against them. I built a new set of ATLAS libraries on RH3 64-bit and ran "make check" after the build, which showed all tests passing. DONE BUILDING TESTERS, RUNNING: SCOPING FOR FAILURES IN BIN TESTS: fgrep -e fault -e FAULT -e error -e ERROR -e fail -e FAIL \ bin/sanity.out 8 cases: 8 passed, 0 skipped, 0 failed 4 cases: 4 passed, 0 skipped, 0 failed 8 cases: 8 passed, 0 skipped, 0 failed 4 cases: 4 passed, 0 skipped, 0 failed 8 cases: 8 passed, 0 skipped, 0 failed 4 cases: 4 passed, 0 skipped, 0 failed 8 cases: 8 passed, 0 skipped, 0 failed 4 cases: 4 passed, 0 skipped, 0 failed DONE SCOPING FOR FAILURES IN CBLAS TESTS: fgrep -e fault -e FAULT -e error -e ERROR -e fail -e FAIL \ interfaces/blas/C/testing/sanity.out | \ fgrep -v PASSED DONE SCOPING FOR FAILURES IN F77BLAS TESTS: fgrep -e fault -e FAULT -e error -e ERROR -e fail -e FAIL \ interfaces/blas/F77/testing/sanity.out | \ fgrep -v PASSED DONE I rebuilt Numpy and Scipy against this new ATLAS build and re-ran both the Numpy and Scipy tests. The Numpy tests all pass, but most of the Scipy failures persist (apparently the new ATLAS libraries fixed the test_speigs.TestEigs issue): ====================================================================== FAIL: test_simple (test_decomp.TestEig) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/builder/master-1.0.4/lib/python2.5/site-packages/scipy/linalg/tests/test_decomp.py", line 145, in test_simple assert_array_almost_equal(w,exact_w) File "/home/builder/recipes/install/numpy-1.3.0-1.egg/numpy/testing/utils.py", line 537, in assert_array_almost_equal header='Arrays are not almost equal') File "/home/builder/recipes/install/numpy-1.3.0-1.egg/numpy/testing/utils.py", line 395, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal (mismatch 66.6666666667%) x: array([ 9.43719064+0.j, -0.11536526+0.j, -0.32182538+0.j]) y: array([ 9.32182538, 0. , -0.32182538]) ====================================================================== FAIL: test_simple (test_decomp.TestEigVals) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/builder/master-1.0.4/lib/python2.5/site-packages/scipy/linalg/tests/test_decomp.py", line 114, in test_simple assert_array_almost_equal(w,exact_w) File "/home/builder/recipes/install/numpy-1.3.0-1.egg/numpy/testing/utils.py", line 537, in assert_array_almost_equal header='Arrays are not almost equal') File "/home/builder/recipes/install/numpy-1.3.0-1.egg/numpy/testing/utils.py", line 395, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal (mismatch 66.6666666667%) x: array([ 9.43719064+0.j, -0.11536526+0.j, -0.32182538+0.j]) y: array([ 9.32182538, 0. , -0.32182538]) ====================================================================== FAIL: test_simple_tr (test_decomp.TestEigVals) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/builder/master-1.0.4/lib/python2.5/site-packages/scipy/linalg/tests/test_decomp.py", line 122, in test_simple_tr assert_array_almost_equal(w,exact_w) File "/home/builder/recipes/install/numpy-1.3.0-1.egg/numpy/testing/utils.py", line 537, in assert_array_almost_equal header='Arrays are not almost equal') File "/home/builder/recipes/install/numpy-1.3.0-1.egg/numpy/testing/utils.py", line 395, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal (mismatch 66.6666666667%) x: array([ 9.43719064+0.j, -0.11536526+0.j, -0.32182538+0.j]) y: array([ 9.32182538, 0. , -0.32182538]) ---------------------------------------------------------------------- -David
![](https://secure.gravatar.com/avatar/9820b5956634e5bbad7f4ed91a232822.jpg?s=120&d=mm&r=g)
David Martin wrote:
Unfortunately I only had the .so libraries to link against so I couldn't run the ATLAS sanity tests against them. I built a new set of ATLAS libraries on RH3 64-bit and ran "make check" after the build, which showed all tests passing.
What does the following return ?
from scipy.linalg import atlas_version
cheers, David
![](https://secure.gravatar.com/avatar/fd3ac07ad0dde7f04231ad1a886db35f.jpg?s=120&d=mm&r=g)
David Cournapeau wrote:
David Martin wrote:
Unfortunately I only had the .so libraries to link against so I couldn't run the ATLAS sanity tests against them. I built a new set of ATLAS libraries on RH3 64-bit and ran "make check" after the build, which showed all tests passing.
What does the following return ?
from scipy.linalg import atlas_version
Python 2.5.4 (r254:67916, May 20 2009, 22:57:32) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-56)] on linux2 Type "help", "copyright", "credits" or "license" for more information. from scipy.linalg import atlas_version atlas_version <module 'scipy.linalg.atlas_version' from '/home/tester/master/lib/python2.5/site-packages/scipy/linalg/atlas_version.so'> atlas_version.version() ATLAS version 3.8.3 built by builder on Mon Jul 13 20:09:54 CDT 2009: UNAME : Linux centos-3-x86_64-build 2.4.21-47.EL #1 SMP Tue Aug 1 07:58:43 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux INSTFLG : -1 0 -a 1 ARCHDEFS : -DATL_OS_Linux -DATL_ARCH_Core2 -DATL_CPUMHZ=2492 -DATL_SSE3 -DATL_SSE2 -DATL_SSE1 -DATL_USE64BITS -DATL_GAS_x8664 F2CDEFS : -DAdd__ -DF77_INTEGER=int -DStringSunStyle CACHEEDGE: 262144 F77 : g77, version GNU Fortran (GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-56)) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) F77FLAGS : -O -fPIC -g -m64 SMC : gcc, version gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) SMCFLAGS : -fomit-frame-pointer -mfpmath=sse -msse3 -O2 -fPIC -g -m64 SKC : gcc, version gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) SKCFLAGS : -fomit-frame-pointer -mfpmath=sse -msse3 -O2 -fPIC -g -m64
![](https://secure.gravatar.com/avatar/9820b5956634e5bbad7f4ed91a232822.jpg?s=120&d=mm&r=g)
David Martin wrote:
Python 2.5.4 (r254:67916, May 20 2009, 22:57:32) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-56)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from scipy.linalg import atlas_version atlas_version <module 'scipy.linalg.atlas_version' from '/home/tester/master/lib/python2.5/site-packages/scipy/linalg/atlas_version.so'> atlas_version.version() ATLAS version 3.8.3 built by builder on Mon Jul 13 20:09:54 CDT 2009: UNAME : Linux centos-3-x86_64-build 2.4.21-47.EL #1 SMP Tue Aug 1 07:58:43 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux INSTFLG : -1 0 -a 1 ARCHDEFS : -DATL_OS_Linux -DATL_ARCH_Core2 -DATL_CPUMHZ=2492 -DATL_SSE3 -DATL_SSE2 -DATL_SSE1 -DATL_USE64BITS -DATL_GAS_x8664 F2CDEFS : -DAdd__ -DF77_INTEGER=int -DStringSunStyle CACHEEDGE: 262144 F77 : g77, version GNU Fortran (GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-56)) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) F77FLAGS : -O -fPIC -g -m64 SMC : gcc, version gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) SMCFLAGS : -fomit-frame-pointer -mfpmath=sse -msse3 -O2 -fPIC -g -m64 SKC : gcc, version gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) SKCFLAGS : -fomit-frame-pointer -mfpmath=sse -msse3 -O2 -fPIC -g -m64
There are some problems with gcc 3.2/3.3 and using sse as the math unit (-mfpmath=sse), and I would not be surprised if g77 inherits them as well. Maybe Red Hat backported fixes, though, you would have to see this with your OS provider. One way to confirm this would be to compile your own gcc/gfortran, and recompile everything (lapack, atlas, numpy and scipy) with it. cheers, David
![](https://secure.gravatar.com/avatar/fd3ac07ad0dde7f04231ad1a886db35f.jpg?s=120&d=mm&r=g)
David Cournapeau wrote:
David Martin wrote:
Python 2.5.4 (r254:67916, May 20 2009, 22:57:32) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-56)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from scipy.linalg import atlas_version atlas_version <module 'scipy.linalg.atlas_version' from '/home/tester/master/lib/python2.5/site-packages/scipy/linalg/atlas_version.so'> atlas_version.version() ATLAS version 3.8.3 built by builder on Mon Jul 13 20:09:54 CDT 2009: UNAME : Linux centos-3-x86_64-build 2.4.21-47.EL #1 SMP Tue Aug 1 07:58:43 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux INSTFLG : -1 0 -a 1 ARCHDEFS : -DATL_OS_Linux -DATL_ARCH_Core2 -DATL_CPUMHZ=2492 -DATL_SSE3 -DATL_SSE2 -DATL_SSE1 -DATL_USE64BITS -DATL_GAS_x8664 F2CDEFS : -DAdd__ -DF77_INTEGER=int -DStringSunStyle CACHEEDGE: 262144 F77 : g77, version GNU Fortran (GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-56)) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) F77FLAGS : -O -fPIC -g -m64 SMC : gcc, version gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) SMCFLAGS : -fomit-frame-pointer -mfpmath=sse -msse3 -O2 -fPIC -g -m64 SKC : gcc, version gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) SKCFLAGS : -fomit-frame-pointer -mfpmath=sse -msse3 -O2 -fPIC -g -m64
There are some problems with gcc 3.2/3.3 and using sse as the math unit (-mfpmath=sse), and I would not be surprised if g77 inherits them as well. Maybe Red Hat backported fixes, though, you would have to see this with your OS provider. One way to confirm this would be to compile your own gcc/gfortran, and recompile everything (lapack, atlas, numpy and scipy) with it
Unfortunately if I upgrade gcc to 3.4.6, the Scipy egg which is built using that compiler will require libstdc++.so.6 instead of libstdc++.so.5. libstdc++.so.5 exists on a clean install of RH3 in /usr/lib, but libstdc++.so.6 does not, so any Scipy built with gcc 3.4.6 cannot be distributed and used on other RH3 platforms without updating the c libraries on those platforms before hand (and the goal of this build is a Scipy egg which I can package in an installer). -David
![](https://secure.gravatar.com/avatar/fd3ac07ad0dde7f04231ad1a886db35f.jpg?s=120&d=mm&r=g)
David Martin wrote:
David Cournapeau wrote:
David Martin wrote:
Python 2.5.4 (r254:67916, May 20 2009, 22:57:32) [GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-56)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from scipy.linalg import atlas_version atlas_version <module 'scipy.linalg.atlas_version' from '/home/tester/master/lib/python2.5/site-packages/scipy/linalg/atlas_version.so'>
atlas_version.version() ATLAS version 3.8.3 built by builder on Mon Jul 13 20:09:54 CDT 2009: UNAME : Linux centos-3-x86_64-build 2.4.21-47.EL #1 SMP Tue Aug 1 07:58:43 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux INSTFLG : -1 0 -a 1 ARCHDEFS : -DATL_OS_Linux -DATL_ARCH_Core2 -DATL_CPUMHZ=2492 -DATL_SSE3 -DATL_SSE2 -DATL_SSE1 -DATL_USE64BITS -DATL_GAS_x8664 F2CDEFS : -DAdd__ -DF77_INTEGER=int -DStringSunStyle CACHEEDGE: 262144 F77 : g77, version GNU Fortran (GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-56)) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) F77FLAGS : -O -fPIC -g -m64 SMC : gcc, version gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) SMCFLAGS : -fomit-frame-pointer -mfpmath=sse -msse3 -O2 -fPIC -g -m64 SKC : gcc, version gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-56) SKCFLAGS : -fomit-frame-pointer -mfpmath=sse -msse3 -O2 -fPIC -g -m64
There are some problems with gcc 3.2/3.3 and using sse as the math unit (-mfpmath=sse), and I would not be surprised if g77 inherits them as well. Maybe Red Hat backported fixes, though, you would have to see this with your OS provider. One way to confirm this would be to compile your own gcc/gfortran, and recompile everything (lapack, atlas, numpy and scipy) with it
Unfortunately if I upgrade gcc to 3.4.6, the Scipy egg which is built using that compiler will require libstdc++.so.6 instead of libstdc++.so.5. libstdc++.so.5 exists on a clean install of RH3 in /usr/lib, but libstdc++.so.6 does not, so any Scipy built with gcc 3.4.6 cannot be distributed and used on other RH3 platforms without updating the c libraries on those platforms before hand (and the goal of this build is a Scipy egg which I can package in an installer).
As a follow up, it seems the resulting dependency on the newer C libraries won't be a problem. I've rebuilt ATLAS, Numpy, and Scipy using gcc/g77 3.4.6 and all Scipy tests now pass. -David
participants (3)
-
David Cournapeau
-
David Martin
-
Robert Kern