Hi,
I've compile sfepy un a fedoracore 8 64bit machine (8 cores) but some tests run very slow. The debug output show the test stopping a long time in lines like:
sfepy: nls: iter: 0, residual: 3.515775e-04 (rel: 1.000000e+00)
I've installed the latest numpy, scipy and umfpack scikits from source. I've installed (among others) the following devel packages:
atlas.x86_64 lapack-devel.x86_64 suitesparse-devel.x86_64
and inserted the section
[umfpack] library_dirs=/usr/lib64 include_dirs = /usr/include/suitesparse
both in numpy and in scipy site.cfg.
What may be the problem?
Thanks,
~ Antonio
Hi Antonio,
Antonio wrote:
Hi,
I've compile sfepy un a fedoracore 8 64bit machine (8 cores) but some tests run very slow. The debug output show the test stopping a long time in lines like:
first sfepy is not parallelized (yet), but it should not run slower than on other machines. On my machine (a quad-core 64bit, gentoo), all tests pass in:
32 test file(s) executed in 41.48 s, 0 failure(s) of 42 test(s)
you should get similar results.
sfepy: nls: iter: 0, residual: 3.515775e-04 (rel: 1.000000e+00)
I've installed the latest numpy, scipy and umfpack scikits from source. I've installed (among others) the following devel packages:
atlas.x86_64 lapack-devel.x86_64 suitesparse-devel.x86_64
and inserted the section
[umfpack] library_dirs=/usr/lib64 include_dirs = /usr/include/suitesparse
both in numpy and in scipy site.cfg.
To verify umfpack installaion: try to go to sfepy/solvers/ls.py and try in a python shell the imports starting at line 56 - one of them should import umfpack.
Also, how did you compile sfepy extension modules - 'make' or 'python setup.py build_ext --inplace'? I have noticed that the latter options produces a slightly slower code, but nothing significant.
r.
Hi Robert,
2009/7/23 Robert Cimrman <cimr...@ntc.zcu.cz>:
Hi Antonio,
Antonio wrote:
Hi,
I've compile sfepy un a fedoracore 8 64bit machine (8 cores) but some tests run very slow. The debug output show the test stopping a long time in lines like:
first sfepy is not parallelized (yet), but it should not run slower than on other machines. On my machine (a quad-core 64bit, gentoo), all tests pass in:
32 test file(s) executed in 41.48 s, 0 failure(s) of 42 test(s)
you should get similar results.
In fact on a dual core ubuntu machine I run all the test in less then a minute.
On fedoracore8 I don't know how slow is but surely more than 20min (I interrupted the tests) that is definitely too much.
sfepy: nls: iter: 0, residual: 3.515775e-04 (rel: 1.000000e+00)
I've installed the latest numpy, scipy and umfpack scikits from source. I've installed (among others) the following devel packages:
atlas.x86_64 lapack-devel.x86_64 suitesparse-devel.x86_64
and inserted the section
[umfpack] library_dirs=/usr/lib64 include_dirs = /usr/include/suitesparse
both in numpy and in scipy site.cfg.
To verify umfpack installaion: try to go to sfepy/solvers/ls.py and try in a python shell the imports starting at line 56 - one of them should import umfpack.
The imports you cite are these four:
import scipy.linsolve.umfpack as um import scipy.splinalg.dsolve.umfpack as um import scipy.sparse.linalg.dsolve.umfpack as um import scikits.umfpack as um
the first two fail while the last two succeed. Does this mean that the problem lies in the scipy installation, not in the umfpack scikits?
Also, how did you compile sfepy extension modules - 'make' or 'python setup.py build_ext --inplace'? I have noticed that the latter options produces a slightly slower code, but nothing significant.
I entered the sfepy source dir and typed 'make'. Didn't know the other way
r.
Antonio
Antonino Ingargiola wrote:
Hi Robert,
2009/7/23 Robert Cimrman <cimr...@ntc.zcu.cz>:
Hi Antonio,
Hi,
I've compile sfepy un a fedoracore 8 64bit machine (8 cores) but some tests run very slow. The debug output show the test stopping a long time in lines like: first sfepy is not parallelized (yet), but it should not run slower than on other machines. On my machine (a quad-core 64bit, gentoo), all tests
Antonio wrote: pass in:
32 test file(s) executed in 41.48 s, 0 failure(s) of 42 test(s)
you should get similar results.
In fact on a dual core ubuntu machine I run all the test in less then a minute.
On fedoracore8 I don't know how slow is but surely more than 20min (I interrupted the tests) that is definitely too much.
It definitely looks like umfpack is not used.
sfepy: nls: iter: 0, residual: 3.515775e-04 (rel: 1.000000e+00)
I've installed the latest numpy, scipy and umfpack scikits from source. I've installed (among others) the following devel packages:
atlas.x86_64 lapack-devel.x86_64 suitesparse-devel.x86_64
and inserted the section
[umfpack] library_dirs=/usr/lib64 include_dirs = /usr/include/suitesparse
both in numpy and in scipy site.cfg. To verify umfpack installaion: try to go to sfepy/solvers/ls.py and try in a python shell the imports starting at line 56 - one of them should import umfpack.
The imports you cite are these four:
import scipy.linsolve.umfpack as um import scipy.splinalg.dsolve.umfpack as um import scipy.sparse.linalg.dsolve.umfpack as um import scikits.umfpack as um
the first two fail while the last two succeed. Does this mean that the problem lies in the scipy installation, not in the umfpack scikits?
At least one should succeed, so it's ok - this is just because umfpack is a moving target depending on scipy version.
Try to hardcode that umfpack should be used - modify line 77: if True: self.sls.use_solver(useUmfpack=True, assumeSortedIndices=True)
Also, how did you compile sfepy extension modules - 'make' or 'python setup.py build_ext --inplace'? I have noticed that the latter options produces a slightly slower code, but nothing significant.
I entered the sfepy source dir and typed 'make'. Didn't know the other way
They should be equivalent anyway.
r.
Hi Robert,
2009/7/23 Robert Cimrman <cimr...@ntc.zcu.cz>: <snip> On my machine (a quad-core 64bit, gentoo), all tests
pass in:
32 test file(s) executed in 41.48 s, 0 failure(s) of 42 test(s)
you should get similar results.
In fact on a dual core ubuntu machine I run all the test in less then a minute.
On fedoracore8 I don't know how slow is but surely more than 20min (I interrupted the tests) that is definitely too much.
It definitely looks like umfpack is not used.
Is possible that is another library that is not used causing the long runtime? For example atlas/lapack?
<snip>
To verify umfpack installaion: try to go to sfepy/solvers/ls.py and try in a python shell the imports starting at line 56 - one of them should import umfpack.
The imports you cite are these four:
import scipy.linsolve.umfpack as um import scipy.splinalg.dsolve.umfpack as um import scipy.sparse.linalg.dsolve.umfpack as um import scikits.umfpack as um
the first two fail while the last two succeed. Does this mean that the problem lies in the scipy installation, not in the umfpack scikits?
At least one should succeed, so it's ok - this is just because umfpack is a moving target depending on scipy version.
Try to hardcode that umfpack should be used - modify line 77: if True: self.sls.use_solver(useUmfpack=True, assumeSortedIndices=True)
Done, and the tests are still slow...
Now I'm timing the tests and found a segfault after 12 minutes during the test_lcbc_3d.py but only if I run all the tests. On the contrary, if run only the test_lcbc_3d.py test it passes in 6.6 seconds.
A test like test_elasticity_small_strain.py passes but after 9 minutes of running.
Thanks,
~ Antonio
Antonino Ingargiola wrote:
Hi Robert,
2009/7/23 Robert Cimrman <cimr...@ntc.zcu.cz>: <snip> On my machine (a quad-core 64bit, gentoo), all tests
pass in:
32 test file(s) executed in 41.48 s, 0 failure(s) of 42 test(s)
you should get similar results. In fact on a dual core ubuntu machine I run all the test in less then a minute.
On fedoracore8 I don't know how slow is but surely more than 20min (I interrupted the tests) that is definitely too much. It definitely looks like umfpack is not used.
Is possible that is another library that is not used causing the long runtime? For example atlas/lapack?
Maybe. I am really not sure what happens.
<snip>
Try to hardcode that umfpack should be used - modify line 77: if True: self.sls.use_solver(useUmfpack=True, assumeSortedIndices=True)
Done, and the tests are still slow...
Now I'm timing the tests and found a segfault after 12 minutes during the test_lcbc_3d.py but only if I run all the tests. On the contrary, if run only the test_lcbc_3d.py test it passes in 6.6 seconds.
1.46 s for me...
A test like test_elasticity_small_strain.py passes but after 9 minutes of running.
6.11 s ...
So something is badly wrong.
This test ends for me with an output like (use --debug): sfepy: nls: iter: 0, residual: 1.220492e+00 (rel: 1.000000e+00) sfepy: rezidual: 0.00 [s] sfepy: solve: 0.37 [s] sfepy: matrix: 0.06 [s] sfepy: nls: iter: 1, residual: 5.173555e-16 (rel: 4.238910e-16)
So the solution of the linear system takes 0.37 s.
I suspect there is something fishy with ATLAS + umfpack installation - is the fast lapack/blas actually used to compile umfpack?
I am now about to leave to Leipzig for the EuroSciPy conference, but there should be an internet access, so I should able to assist you during the weekend, too.
r.
2009/7/24 Robert Cimrman <cimr...@ntc.zcu.cz>:
Antonino Ingargiola wrote:
Hi Robert,
2009/7/23 Robert Cimrman <cimr...@ntc.zcu.cz>: <snip> On my machine (a quad-core 64bit, gentoo), all tests
pass in:
32 test file(s) executed in 41.48 s, 0 failure(s) of 42 test(s)
you should get similar results. In fact on a dual core ubuntu machine I run all the test in less then a minute.
On fedoracore8 I don't know how slow is but surely more than 20min (I interrupted the tests) that is definitely too much. It definitely looks like umfpack is not used.
Is possible that is another library that is not used causing the long runtime? For example atlas/lapack?
Maybe. I am really not sure what happens.
How can I check if scipy is using atlas/lapack/umfpack? Maybe I should ask on the scipy mailing list....
<snip>
Try to hardcode that umfpack should be used - modify line 77: if True: self.sls.use_solver(useUmfpack=True, assumeSortedIndices=True)
Done, and the tests are still slow...
Now I'm timing the tests and found a segfault after 12 minutes during the test_lcbc_3d.py but only if I run all the tests. On the contrary, if run only the test_lcbc_3d.py test it passes in 6.6 seconds.
1.46 s for me...
A test like test_elasticity_small_strain.py passes but after 9 minutes of running.
6.11 s ...
So something is badly wrong.
This test ends for me with an output like (use --debug): sfepy: nls: iter: 0, residual: 1.220492e+00 (rel: 1.000000e+00) sfepy: rezidual: 0.00 [s] sfepy: solve: 0.37 [s] sfepy: matrix: 0.06 [s] sfepy: nls: iter: 1, residual: 5.173555e-16 (rel: 4.238910e-16)
So the solution of the linear system takes 0.37 s.
On fedorecore8 the final output is:
sfepy: nls: iter: 0, residual: 1.220492e+00 (rel: 1.000000e+00) sfepy: rezidual: 0.01 [s] sfepy: solve: 19.86 [s] sfepy: matrix: 0.08 [s] sfepy: nls: iter: 1, residual: 3.040216e-15 (rel: 2.490976e-15) ... converged: True +++ test_get_solution: ok ... ||isotropic - general||: 0.000000e+00 ... ||isotropic - general||: 0.000000e+00 ... ||isotropic - general||: 0.000000e+00 +++ test_linear_terms: ok
all passed in 561.92 s
I suspect there is something fishy with ATLAS + umfpack installation - is the fast lapack/blas actually used to compile umfpack?
I don't now. I'm quite new to fedoracore. I've installed the distro-provided packages:
lapack.x86_64 lapack-devel.x86_64 atlas.x86_64 atlas-devel.x86_64 suitesparse.x86_64 suitesparse-devel.x86_64
I am now about to leave to Leipzig for the EuroSciPy conference, but there should be an internet access, so I should able to assist you during the weekend, too.
r.
Thanks a lot :)
~ Antonio
participants (3)
-
Antonino Ingargiola
-
Antonio
-
Robert Cimrman