Fwd: Package: scipy-0.11.0-0.1.rc2.fc18 Tag: f18-updates-candidate Status: failed Built by: orion
This is a plea for some help. We've been having trouble getting scipy to pass all of the tests in the Fedora 18 build with python 3.3 (although it seems to build okay in Fedora 19). Below are the logs of the build. There appears to be some kind of memory corruption that manifests itself a little differently on 32-bit vs. 64-bit. I really have no idea myself how to pursue debugging this, though I'm happy to provide any more needed information. Thanks, Orion -------- Original Message -------- Subject: Package: scipy-0.11.0-0.1.rc2.fc18 Tag: f18-updates-candidate Status: failed Built by: orion Date: Thu, 20 Sep 2012 21:01:54 +0000 (UTC) From: Fedora Koji Build System <buildsys@fedoraproject.org> To: ausil@fedoraproject.org, jspaleta@fedoraproject.org, voronov@fedoraproject.org, torwangjl@fedoraproject.org, alagunambi@fedoraproject.org, urkle@fedoraproject.org, orion@fedoraproject.org Package: scipy-0.11.0-0.1.rc2.fc18 Tag: f18-updates-candidate Status: failed Built by: orion ID: 350761 Started: Thu, 20 Sep 2012 20:39:45 UTC Finished: Thu, 20 Sep 2012 21:01:32 UTC scipy-0.11.0-0.1.rc2.fc18 (350761) failed on buildvm-33.phx2.fedoraproject.org (x86_64), buildvm-35.phx2.fedoraproject.org (i386), buildvm-34.phx2.fedoraproject.org (noarch): BuildError: error building package (arch x86_64), mock exited with status 1; see build.log for more information SRPMS: scipy-0.11.0-0.1.rc2.fc18.src.rpm Failed tasks: ------------- Task 4509076 on buildvm-33.phx2.fedoraproject.org Task Type: buildArch (scipy-0.11.0-0.1.rc2.fc18.src.rpm, x86_64) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=build.log http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=mock_output.log http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=state.log Task 4509077 on buildvm-35.phx2.fedoraproject.org Task Type: buildArch (scipy-0.11.0-0.1.rc2.fc18.src.rpm, i686) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=build.log http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=mock_output.log http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=state.log Task 4509063 on buildvm-34.phx2.fedoraproject.org Task Type: build (f18-candidate, /scipy:cb69bd06f0d930fbe8840d89b918b617e28af63f) Closed tasks: ------------- Task 4509064 on buildvm-30.phx2.fedoraproject.org Task Type: buildSRPMFromSCM (/scipy:cb69bd06f0d930fbe8840d89b918b617e28af63f) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=build.log http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=checkout.log http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=mock_output.log http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=state.log Task Info: http://koji.fedoraproject.org/koji/taskinfo?taskID=4509063 Build Info: http://koji.fedoraproject.org/koji/buildinfo?buildID=350761
As far as how to proceed, I had a problem once with unuran numerical issues on an unreleased fedora. I used mock to build on the unreleased version, and mock has a chroot feature to get a shell and debug. Orion Poplawski wrote:
This is a plea for some help. We've been having trouble getting scipy to pass all of the tests in the Fedora 18 build with python 3.3 (although it seems to build okay in Fedora 19). Below are the logs of the build. There appears to be some kind of memory corruption that manifests itself a little differently on 32-bit vs. 64-bit. I really have no idea myself how to pursue debugging this, though I'm happy to provide any more needed information.
Thanks,
Orion
-------- Original Message -------- Subject: Package: scipy-0.11.0-0.1.rc2.fc18 Tag: f18-updates-candidate Status: failed Built by: orion Date: Thu, 20 Sep 2012 21:01:54 +0000 (UTC) From: Fedora Koji Build System <buildsys@fedoraproject.org> To: ausil@fedoraproject.org, jspaleta@fedoraproject.org, voronov@fedoraproject.org, torwangjl@fedoraproject.org, alagunambi@fedoraproject.org, urkle@fedoraproject.org, orion@fedoraproject.org
Package: scipy-0.11.0-0.1.rc2.fc18 Tag: f18-updates-candidate Status: failed Built by: orion ID: 350761 Started: Thu, 20 Sep 2012 20:39:45 UTC Finished: Thu, 20 Sep 2012 21:01:32 UTC
scipy-0.11.0-0.1.rc2.fc18 (350761) failed on buildvm-33.phx2.fedoraproject.org (x86_64), buildvm-35.phx2.fedoraproject.org (i386), buildvm-34.phx2.fedoraproject.org (noarch): BuildError: error building package (arch x86_64), mock exited with status 1; see build.log for more information SRPMS: scipy-0.11.0-0.1.rc2.fc18.src.rpm
Failed tasks: -------------
Task 4509076 on buildvm-33.phx2.fedoraproject.org Task Type: buildArch (scipy-0.11.0-0.1.rc2.fc18.src.rpm, x86_64) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=build.log
http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=mock_output.log
http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509076&name=state.log
Task 4509077 on buildvm-35.phx2.fedoraproject.org Task Type: buildArch (scipy-0.11.0-0.1.rc2.fc18.src.rpm, i686) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=build.log
http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=mock_output.log
http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509077&name=state.log
Task 4509063 on buildvm-34.phx2.fedoraproject.org Task Type: build (f18-candidate, /scipy:cb69bd06f0d930fbe8840d89b918b617e28af63f)
Closed tasks: -------------
Task 4509064 on buildvm-30.phx2.fedoraproject.org Task Type: buildSRPMFromSCM (/scipy:cb69bd06f0d930fbe8840d89b918b617e28af63f) logs: http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=build.log
http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=checkout.log
http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=mock_output.log
http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=root.log http://koji.fedoraproject.org/koji/getfile?taskID=4509064&name=state.log
Task Info: http://koji.fedoraproject.org/koji/taskinfo?taskID=4509063 Build Info: http://koji.fedoraproject.org/koji/buildinfo?buildID=350761
On 09/20/2012 05:33 PM, Orion Poplawski wrote:
This is a plea for some help. We've been having trouble getting scipy to pass all of the tests in the Fedora 18 build with python 3.3 (although it seems to build okay in Fedora 19). Below are the logs of the build. There appears to be some kind of memory corruption that manifests itself a little differently on 32-bit vs. 64-bit. I really have no idea myself how to pursue debugging this, though I'm happy to provide any more needed information.
Well, I've at least been able to determine that is happens in scipy/linalg/test_decomp.py test_dsbevx(), though it doesn't fail the same way each time: Compare dsbevx eigenvalues and eigenvectors ... Segmentation fault $ python3 /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python3.3/site-packages/scipy/linalg/tests/test_decomp.py ..............E.....E.........................................................................................................................................................................................................................................................................E ====================================================================== ERROR: Compare dsbevx eigenvalues and eigenvectors ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python3.3/site-packages/scipy/linalg/tests/test_decomp.py", line 373, in test_dsbevx evec_ = evec[:,argsort(w)] IndexError: index (1) out of range (0<=index<0) in dimension 1 ====================================================================== ERROR: Compare zhbevx eigenvalues and eigenvectors ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python3.3/site-packages/scipy/linalg/tests/test_decomp.py", line 395, in test_zhbevx evec_ = evec[:,argsort(w)] IndexError: index (1) out of range (0<=index<0) in dimension 1 ====================================================================== ERROR: test_decomp.test_lapack_misaligned ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python3.3/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/usr/lib/python3.3/site-packages/numpy/testing/decorators.py", line 214, in knownfailer raise KnownFailureTest(msg) numpy.testing.noseclasses.KnownFailureTest: Ticket #1152, triggers a segfault in rare cases. ---------------------------------------------------------------------- Ran 287 tests in 1.915s FAILED (errors=3) $ python3 /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python3.3/site-packages/scipy/linalg/tests/test_decomp.py ..............Segmentation fault $ python3 /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python3.3/site-packages/scipy/linalg/tests/test_decomp.py ..............E.....E....................................................................................................................Segmentation fault Valgrind reports lots of messages like: ==7641== Invalid write of size 8 ==7641== at 0x73AF360: ATL_dswap_xp0yp0aXbX (in /usr/lib/atlas/libatlas.so.3.0) ==7641== by 0x73AF1A2: ATL_dswap (in /usr/lib/atlas/libatlas.so.3.0) ==7641== by 0x722FEC7: atl_f77wrap_dswap_ (in /usr/lib/atlas/libptf77blas.so.3.0) ==7641== by 0x7238C27: dswap_ (in /usr/lib/atlas/libptf77blas.so.3.0) ==7641== by 0x79C405A: dsteqr_ (in /usr/lib/atlas/liblapack.so.3.0) ==7641== by 0x79A8DB4: dsbevx_ (in /usr/lib/atlas/liblapack.so.3.0) ==7641== by 0x7F56A85: f2py_rout_flapack_dsbevx (flapackmodule.c:25876) ==7641== by 0x7F5ECB9: fortran_call (fortranobject.c:346) ==7641== by 0x406CBB0: PyObject_Call (in /usr/lib/libpython3.3m.so.1.0) ==7641== by 0x412096C: PyEval_EvalFrameEx (in /usr/lib/libpython3.3m.so.1.0) ==7641== by 0x41236E4: PyEval_EvalFrameEx (in /usr/lib/libpython3.3m.so.1.0) ==7641== by 0x4125713: PyEval_EvalCodeEx (in /usr/lib/libpython3.3m.so.1.0) ==7641== Address 0x4c5e228 is 0 bytes after a block of size 16 alloc'd ==7641== at 0x4029E3F: malloc (vg_replace_malloc.c:270) ==7641== by 0x40B43D8: PyMem_Malloc (in /usr/lib/libpython3.3m.so.1.0) ==7641== by 0x6D124C1: ??? (in /usr/lib/python3.3/site-packages/numpy/core/multiarray.cpython-33m.so) ==7641== by 0x7F5F238: array_from_pyobj (fortranobject.c:630) ==7641== by 0x7F5693F: f2py_rout_flapack_dsbevx (flapackmodule.c:25864) ==7641== by 0x7F5ECB9: fortran_call (fortranobject.c:346) ==7641== by 0x406CBB0: PyObject_Call (in /usr/lib/libpython3.3m.so.1.0) ==7641== by 0x412096C: PyEval_EvalFrameEx (in /usr/lib/libpython3.3m.so.1.0) ==7641== by 0x41236E4: PyEval_EvalFrameEx (in /usr/lib/libpython3.3m.so.1.0) ==7641== by 0x4125713: PyEval_EvalCodeEx (in /usr/lib/libpython3.3m.so.1.0) ==7641== by 0x41233A1: PyEval_EvalFrameEx (in /usr/lib/libpython3.3m.so.1.0) ==7641== by 0x4125713: PyEval_EvalCodeEx (in /usr/lib/libpython3.3m.so.1.0) Also, looking at the spec, it was running the python3 checks first. I switched that to see what I would see with python2 and it doesn't fail, although I get plenty of warnings/notices: + python -c 'import scipy; scipy.test('\''full'\'')' ..............................................................................................................................................................................................................................K........................................................................................................K..................................................................K..K...............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................! .......... ...........................................................SSSSSS......SSSSSS......SSSS..............................................................................................................................................................................................................................................................................................................................................................................K................................................................................................................................................................................................................SSSSS.K..........S.....................................................................................................................................................................................................................................................................................................................! .......... .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................! .......... ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................K..................................................................K................................................................................................................................................................KK.....................................................................................................................................................................! .......... ..................................................................................................................................................................................................F.................../builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/special/tests/test_basic.py:1645: RuntimeWarning: divide by zero encountered in iv c1 = special.iv(v, x) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/special/tests/test_basic.py:1645: RuntimeWarning: overflow encountered in iv c1 = special.iv(v, x) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/special/tests/test_basic.py:1646: RuntimeWarning: invalid value encountered in iv c2 = special.iv(v, x+0j) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/special/tests/test_basic.py:1652: RuntimeWarning: divide by zero encountered in divide dc = abs(c1/c2 - 1) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/special/tests/test_basic.py:1652: RuntimeWarning: invalid value encountered in divide dc = abs(c1/c2 - 1) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/special/tests/test_basic.py:1659: RuntimeWarning: overflow encountered in iv assert_(dc[k] < 2e-7, (v[k], x[k], special.iv(v[k], x[k]), special.iv(v[k], x[k]+0j))) F...................................................K.K........................................................./builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/special/tests/test_basic.py:153: RuntimeWarning: invalid value encountered in fdtri cephes.fdtri(1,1,0.5) ....................................................................................................................................................................................................................................................................................................................................K........K..............SSSSSSSS........................................................................................................................................................................................................................................................................................................................................./builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:4163: RuntimeWarning: overflow encountered in exp return exp(c*x-exp(x)-gamln(c)) ............/builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:4645: RuntimeWarning: invalid value encountered in sqrt vals = 2*(bt+1.0)*sqrt(b-2.0)/((b-3.0)*sqrt(b)) ............................................................................................................................................................................................................................../builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:6787: RuntimeWarning: divide by zero encountered in log lvals = where(vals==0.0,0.0,log(vals)) ............................S................/builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/optimize/optimize.py:1605: RuntimeWarning: invalid value encountered in double_scalars tmp2 = (x - v)*(fx - fw) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/optimize/optimize.py:1606: RuntimeWarning: invalid value encountered in double_scalars p = (x - v)*tmp2 - (x - w)*tmp1; /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/optimize/optimize.py:447: RuntimeWarning: invalid value encountered in subtract and max(abs(fsim[0] - fsim[1:])) <= ftol): /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/optimize/optimize.py:2105: RuntimeWarning: invalid value encountered in subtract if (fx2 - fval) > delta: /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/optimize/optimize.py:2113: RuntimeWarning: invalid value encountered in subtract if (2.0*(fx - fval) <= ftol*(abs(fx) + abs(fval)) + 1e-20): break /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:2421: RuntimeWarning: overflow encountered in double_scalars k = gd*g2c*g2cd - g1c**2 * g1cd**2 /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:2421: RuntimeWarning: invalid value encountered in double_scalars k = gd*g2c*g2cd - g1c**2 * g1cd**2 /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:1924: RuntimeWarning: invalid value encountered in double_scalars Lhat = muhat - Shat*mu /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:1222: RuntimeWarning: divide by zero encountered in log return log(self._pdf(x, *args)) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:2332: RuntimeWarning: divide by zero encountered in divide return where(b > 1, a/(b-1.0), inf) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:2334: RuntimeWarning: divide by zero encountered in divide return where(b > 2, a*(a+1.0)/((b-2.0)*(b-1.0)), inf) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:1676: RuntimeWarning: invalid value encountered in subtract mu2 = mu2p - mu*mu /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/optimize/optimize.py:1604: RuntimeWarning: invalid value encountered in double_scalars tmp1 = (x - w)*(fx - fv) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/optimize/optimize.py:1873: RuntimeWarning: invalid value encountered in double_scalars w = xb - ((xb - xc)*tmp2 - (xb - xa)*tmp1) / denom /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:2980: RuntimeWarning: divide by zero encountered in double_scalars g1 = 2*(v2+2*v1-2)/(v2-6)*sqrt((2*v2-4)/(v1*(v2+v1-2))) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:2980: RuntimeWarning: invalid value encountered in sqrt g1 = 2*(v2+2*v1-2)/(v2-6)*sqrt((2*v2-4)/(v1*(v2+v1-2))) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:3192: RuntimeWarning: divide by zero encountered in divide val = (-1.0/c)**n * sum(comb(n,k)*(-1)**k / (1.0-c*k),axis=0) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:4583: RuntimeWarning: overflow encountered in double_scalars var -= nc*nc*df* val1**2 / 2.0 / val2**2 /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/optimize/optimize.py:2105: RuntimeWarning: invalid value encountered in double_scalars if (fx2 - fval) > delta: /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/optimize/optimize.py:2113: RuntimeWarning: invalid value encountered in double_scalars if (2.0*(fx - fval) <= ftol*(abs(fx) + abs(fval)) + 1e-20): break ./builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:1222: RuntimeWarning: invalid value encountered in log return log(self._pdf(x, *args)) /builddir/build/BUILDROOT/scipy-0.11.0-0.1.rc2.fc18.i386/usr/lib/python2.7/site-packages/scipy/stats/distributions.py:5025: RuntimeWarning: divide by zero encountered in divide return where(x < c, 2*x/c, 2*(1-x)/(1-c)) ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... I don't know if any of these are normal. -- Orion Poplawski Technical Manager 303-415-9701 x222 NWRA, Boulder Office FAX: 303-415-9702 3380 Mitchell Lane orion@nwra.com Boulder, CO 80301 http://www.nwra.com
participants (2)
-
Neal Becker -
Orion Poplawski