RE: [SciPy-dev] CVS scipy.test() segfaulting on me on a Fedora2 box
Ok, I've got the reorganized version from cvs and still don't get a segfault using the distro BLAS and gcc 3.4.2. -----Original Message----- From: scipy-dev-bounces@scipy.net on behalf of Fernando Perez Sent: Wed 12/22/2004 12:07 AM To: SciPy Developers List Subject: Re: [SciPy-dev] CVS scipy.test() segfaulting on me on a Fedora2 box Charles Harris wrote:
Hi Fernando,
Am I missing something? I don't get scipy.lib when I install from cvs, so I am wondering if something has changed and I need to checkout scipy again?
I did a fresh checkout, b/c I recalled hearing about a discussion on a directory reorg. Since CVS sucks so badly with layout operations, I played it safe and did a clean checkout.
Anyway, scipy.linalg.lapack.flapack.cheev(a) runs fine here on Fedora Core 3, but I am using the blas libraries that came with the distro. Is Atlas to be preferred to blas?
ATLAS provides a tuned BLAS, optimized for your specific architecture. The functionality is the same as that in Fedora's generic BLAS, but the performance is higher. Cheers, f _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.net http://www.scipy.net/mailman/listinfo/scipy-dev
Charles Harris wrote:
Ok,
I've got the reorganized version from cvs and still don't get a segfault using the distro BLAS and gcc 3.4.2.
I suspect the problem lies with ATLAS. I am using atlas3.6.0_Linux_P4SSE2_2HT.tgz from the scipy website, which from reading the included makefile, was built as follows: F77 = /home/pearu/bin/g77 F77FLAGS = -fomit-frame-pointer -O -fno-second-underscore I think the problem is ATLAS, b/c the top of the backtrace I included yesterday had this: #0 0x55cd67c2 in ATL_cdotc_xp0yp0aXbX () from /usr/lib/python2.3/site-packages/scipy/lib/lapack/flapack.so That first symbol is, I imagine, an ATLAS call. But Pearu seems to have built these binaries with pretty conservative flags (-O), and when you use the distro BLAS (no ATLAS) you don't get any segfault. So perhaps it's a bug in ATLAS and/or g77 which appears even with basic optimizations. Cheers, f
On Wed, 22 Dec 2004, Fernando Perez wrote:
Charles Harris wrote:
Ok,
I've got the reorganized version from cvs and still don't get a segfault using the distro BLAS and gcc 3.4.2.
I suspect the problem lies with ATLAS. I am using atlas3.6.0_Linux_P4SSE2_2HT.tgz from the scipy website, which from reading the included makefile, was built as follows:
F77 = /home/pearu/bin/g77 F77FLAGS = -fomit-frame-pointer -O -fno-second-underscore
I think the problem is ATLAS, b/c the top of the backtrace I included yesterday had this:
#0 0x55cd67c2 in ATL_cdotc_xp0yp0aXbX () from /usr/lib/python2.3/site-packages/scipy/lib/lapack/flapack.so
That first symbol is, I imagine, an ATLAS call. But Pearu seems to have built these binaries with pretty conservative flags (-O), and when you use the distro BLAS (no ATLAS) you don't get any segfault. So perhaps it's a bug in ATLAS and/or g77 which appears even with basic optimizations.
Actually the Fortran lapack library that was used to complete ATLAS lapack, may have been compiled with -O3, I don't remember exactly anymore. If somebody could confirm that compiling lapack library with -O2 will fix the issue, then I'll look for rebuilding atlas binaries at scipy site. Pearu
On Wed, 2004-12-22 at 13:59, Pearu Peterson wrote:
On Wed, 22 Dec 2004, Fernando Perez wrote:
Charles Harris wrote:
Ok,
I've got the reorganized version from cvs and still don't get a segfault using the distro BLAS and gcc 3.4.2.
I suspect the problem lies with ATLAS. I am using atlas3.6.0_Linux_P4SSE2_2HT.tgz from the scipy website, which from reading the included makefile, was built as follows:
F77 = /home/pearu/bin/g77 F77FLAGS = -fomit-frame-pointer -O -fno-second-underscore
I think the problem is ATLAS, b/c the top of the backtrace I included yesterday had this:
#0 0x55cd67c2 in ATL_cdotc_xp0yp0aXbX () from /usr/lib/python2.3/site-packages/scipy/lib/lapack/flapack.so
That first symbol is, I imagine, an ATLAS call. But Pearu seems to have built these binaries with pretty conservative flags (-O), and when you use the distro BLAS (no ATLAS) you don't get any segfault. So perhaps it's a bug in ATLAS and/or g77 which appears even with basic optimizations.
Actually the Fortran lapack library that was used to complete ATLAS lapack, may have been compiled with -O3, I don't remember exactly anymore. If somebody could confirm that compiling lapack library with -O2 will fix the issue, then I'll look for rebuilding atlas binaries at scipy site.
Pearu
Can anyone confirm that using -O2 for lapack fixes this? I thought I'd worked around this (crash in ATL_cdotc_xp0yp0aXbX ()) using Pearu's advice but I'm seeing it again. Was the scipy ATLAS ever re-built? Regards, Todd
On Fri, 7 Jan 2005, Todd Miller wrote:
Can anyone confirm that using -O2 for lapack fixes this? I thought I'd worked around this (crash in ATL_cdotc_xp0yp0aXbX ()) using Pearu's advice but I'm seeing it again. Was the scipy ATLAS ever re-built?
No. Rebuilding all scipy ATLAS libraries is a rather time consuming task and I was hoping that someone could confirm that the proposed fix works before I'd spend time on it. Otherwise its just a waste my time if it turns out that using -O2 does not fix the issue. Pearu
On Fri, 2005-01-07 at 16:58 -0600, Pearu Peterson wrote:
On Fri, 7 Jan 2005, Todd Miller wrote:
Can anyone confirm that using -O2 for lapack fixes this? I thought I'd worked around this (crash in ATL_cdotc_xp0yp0aXbX ()) using Pearu's advice but I'm seeing it again. Was the scipy ATLAS ever re-built?
No. Rebuilding all scipy ATLAS libraries is a rather time consuming task and I was hoping that someone could confirm that the proposed fix works before I'd spend time on it. Otherwise its just a waste my time if it turns out that using -O2 does not fix the issue.
Sorry Pearu. I think I panicked here and then saw that this was still an open issue while I was looking for hints. SciPy's head built fine for me today and all the tests passed. If I recall correctly, there is some important update to the head that's missing from the numarray branch so when I built the numarray branch, it failed. I'll resolve it Monday morning and report back but I think this was just confusion on my part. Regards, Todd
On Fri, 2005-01-07 at 21:28, Todd Miller wrote:
On Fri, 2005-01-07 at 16:58 -0600, Pearu Peterson wrote:
On Fri, 7 Jan 2005, Todd Miller wrote:
Can anyone confirm that using -O2 for lapack fixes this? I thought I'd worked around this (crash in ATL_cdotc_xp0yp0aXbX ()) using Pearu's advice but I'm seeing it again. Was the scipy ATLAS ever re-built?
No. Rebuilding all scipy ATLAS libraries is a rather time consuming task and I was hoping that someone could confirm that the proposed fix works before I'd spend time on it. Otherwise its just a waste my time if it turns out that using -O2 does not fix the issue.
Sorry Pearu. I think I panicked here and then saw that this was still an open issue while I was looking for hints. SciPy's head built fine for me today and all the tests passed. If I recall correctly, there is some important update to the head that's missing from the numarray branch so when I built the numarray branch, it failed. I'll resolve it Monday morning and report back but I think this was just confusion on my part.
Regards, Todd
I was able to resolve this (again). The root problem was incorrect detection of ATLAS so other people seeing the ATL_cdotc_xp0yp0aXbX() crash should check their build output for correct ATLAS info. In my case the ATLAS detection failure was brought on by using a debug Python build which produces "extra" output about reference counts. Pearu already debugged this for me but it never made it onto the numarray branch and I forgot about it over the holidays. My apologies... Todd
So if we want to start a new "topical" library project hosted on scipy.org, are there any guidelines about where in CVS it is to be put? Under "topical"? (can we think up a better name than topical?). Under the name we want to give it? (By the way, when is a switch to subversion planned?). Thanks, Perry
participants (5)
-
Charles Harris -
Fernando Perez -
Pearu Peterson -
Perry Greenfield -
Todd Miller