petsc - UMFPACK and scipy
Ondrej, Please can you show me an example where petsc solvers are "better" than UMFPACK. Nils 4) About the petsc - I know it's another dependence. However, I noticed you are using umfpack in SciPy. So why not petsc? I think it contains much more (sometimes better) solvers (depends on the problem). It's seems logical to me, to either use nothing, or the best library available, which I believe is petsc.
Nils Wagner wrote:
Ondrej,
Please can you show me an example where petsc solvers are "better" than UMFPACK.
Nils
4) About the petsc - I know it's another dependence. However, I noticed you are using umfpack in SciPy. So why not petsc? I think it contains much more (sometimes better) solvers (depends on the problem). It's seems logical to me, to either use nothing, or the best library available, which I believe is petsc. I can give you one situation where adding dependency makes things more complicated: when you are a packager. I am trying to "evangelize" numpy/scipy, and one problem people face is installation. When you are a user, adding dependency is great, it gives you more code, more API to leverage. When you are a packager, each dependency is a mess.
I am working on rpm and debian package of numpy and scipy, and 99.9 % of the problems are the dependencies. LAPACK and BLAS are already quite difficult to package correctly (debian was the only distribution to do it correctly for a long time), UMFPACK is kind of a pain to compile too (depends on two other packages), and let's not even start talking about ATLAS, which pose significant challenges by its very nature (again, only debian has done it correctly, Fedora copying their method). And I have only experience on linux, where at least every distribution uses the same compiler suite. Most of those libraries are not really commonly used, and as such, are not provided by distributors most of the time. I wouldn't be surprised if this is one of the reason why Robert is reluctant to add more dependencies. David
Hi David David Cournapeau wrote:
UMFPACK is kind of a pain to compile too (depends on two other packages), and let's not even start talking about ATLAS, [...] Just in case you didn't know, UMFPACK (as well as Tim Davis' other code) is part of Debian/Ubuntu. It's named 'libufsparse'. AFAIK it was not in Fedora Core 6 (but this may have changed).
Cheers JP
John Pye wrote:
Hi David
David Cournapeau wrote:
UMFPACK is kind of a pain to compile too (depends on two other packages), and let's not even start talking about ATLAS, [...]
Just in case you didn't know, UMFPACK (as well as Tim Davis' other code) is part of Debian/Ubuntu. It's named 'libufsparse'. AFAIK it was not in Fedora Core 6 (but this may have changed).
I know that Debian/Ubuntu makes it much easier to build numpy and scipy than other distributions, basically 2-3 lines to install them and all necessary dependencies :) But I was speaking for other distributions. Sadly, my experience with other distributions (FC 5-6 and openSUSE, at least) show that they are not on par with debian, at least for those packages. And then, you have other OS. Anyway, I was just highlighting why adding dependencies makes the packaging job more difficult. David
John Pye wrote:
Hi David
David Cournapeau wrote:
UMFPACK is kind of a pain to compile too (depends on two other packages), and let's not even start talking about ATLAS, [...]
Just in case you didn't know, UMFPACK (as well as Tim Davis' other code) is part of Debian/Ubuntu. It's named 'libufsparse'. AFAIK it was not in Fedora Core 6 (but this may have changed).
I know that Debian/Ubuntu makes it much easier to build numpy and scipy than other distributions, basically 2-3 lines to install them and all necessary dependencies :) But I was speaking for other distributions. Sadly, my experience with other distributions (FC 5-6 and openSUSE, at least) show that they are not on par with debian, at least for those packages. And then, you have other OS. Anyway, I was just highlighting why adding dependencies makes the packaging job more difficult. Perhaps the focus should be on getting UMFPACK added to those distros, rather than working out how to build them as part of scipy? The only
David Cournapeau wrote: place where this isn't so great is Windows. Windows is always a special case :-)
John Pye wrote:
Perhaps the focus should be on getting UMFPACK added to those distros, rather than working out how to build them as part of scipy?
Since no one is suggesting building the UMFPACK library as part of the scipy build process, it appears that everyone already agrees with you. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
I think Python projects which are not just python bindings to some C++/Fortran library (and SciPy definitely is not) should work in a way that you just download them and import in python, without any installation and it should just work, at least basic things. This way SciPy can have any number of (optional) modules, as it will not add to the complexity of the installation. Of course you need NumPy, but I assume NumPy is already installed. But I just use apt-get install python-scipy anyway, so I don't care. Ondrej On 4/18/07, Robert Kern <robert.kern@gmail.com> wrote:
John Pye wrote:
Perhaps the focus should be on getting UMFPACK added to those distros, rather than working out how to build them as part of scipy?
Since no one is suggesting building the UMFPACK library as part of the scipy build process, it appears that everyone already agrees with you.
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
Ondrej Certik wrote:
I think Python projects which are not just python bindings to some C++/Fortran library (and SciPy definitely is not) should work in a way that you just download them and import in python, without any installation and it should just work, at least basic things.
On linux at least, I don't see any other solution than packaging numpy/scipy with rpm /deb/ whatever the distribution uses. This is the only reliable way to distribute binaries on Linux; trying otherwise is cumbersome, difficult, and a waste of time I think. David
Ondrej Certik wrote:
I think Python projects which are not just python bindings to some C++/Fortran library (and SciPy definitely is not) should work in a way that you just download them and import in python, without any installation and it should just work, at least basic things.
But most of the things in scipy that aren't *just* wrappers around C or FORTRAN *use* just-wrappers of C and FORTRAN libraries. I'm sorry that this doesn't meet your standards, but there's really no getting around it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
John Pye wrote:
Perhaps the focus should be on getting UMFPACK added to those distros, rather than working out how to build them as part of scipy? The only place where this isn't so great is Windows. Windows is always a special case :-)
Talk is cheap :) This is exactly what I've started a few days ago, you are more than welcomed to help me: http://software.opensuse.org/download/home:/ashigabou/ I've already binary packages for numpy + scipy using NETLIB BLAS and LAPACK (also packaged by myself, as I had various problems with the ones distributed by fedora and open suse), am working on atlas 3.7.* (which is by far the most difficult), and intend to do at least fftw3 and umfpack along. cheers, David
Nils Wagner wrote:
Please can you show me an example where petsc solvers are "better" than UMFPACK.
Petsc is really a superpackage providing many parallel linear solvers (iterative, direct, preconditioners, ...) together with nonlinear solvers, time steppers, etc. The solvers can be both petsc-native or external packages, nevertheless all are accessed via a uniform interface. IMHO UMFPACK is one of the optional external solvers petsc can use, so to answer your question, petsc can do anything that UMFPACK does and much more. r.
On 4/13/07, Robert Cimrman <cimrman3@ntc.zcu.cz> wrote:
Nils Wagner wrote:
Please can you show me an example where petsc solvers are "better" than UMFPACK.
Petsc is really a superpackage providing many parallel linear solvers (iterative, direct, preconditioners, ...) together with nonlinear solvers, time steppers, etc. The solvers can be both petsc-native or external packages, nevertheless all are accessed via a uniform interface. IMHO UMFPACK is one of the optional external solvers petsc can use, so to answer your question, petsc can do anything that UMFPACK does and much more.
Yes, it's exactly like this. Thus, there is a question whether SciPy should support sparse solvers (my answer is yes) and if so, then it should support petsc, otherwise, for example me, I am not going to use it, as I want to try several solvers according to the problem. What I am trying to say is that I don't want to write two versions of my code - one for petsc and second one for SciPy. And from the zen of python: There should be one-- and preferably only one --obvious way to do it. Ondra
Ondrej Certik wrote:
On 4/13/07, Robert Cimrman <cimrman3@ntc.zcu.cz> wrote:
Nils Wagner wrote:
Please can you show me an example where petsc solvers are "better" than UMFPACK. Petsc is really a superpackage providing many parallel linear solvers (iterative, direct, preconditioners, ...) together with nonlinear solvers, time steppers, etc. The solvers can be both petsc-native or external packages, nevertheless all are accessed via a uniform interface. IMHO UMFPACK is one of the optional external solvers petsc can use, so to answer your question, petsc can do anything that UMFPACK does and much more.
Yes, it's exactly like this. Thus, there is a question whether SciPy should support sparse solvers (my answer is yes) and if so, then it should support petsc, otherwise, for example me, I am not going to use it, as I want to try several solvers according to the problem.
My problems tend to be such that only a direct solvers work :)
What I am trying to say is that I don't want to write two versions of my code - one for petsc and second one for SciPy. And from the zen of python:
There should be one-- and preferably only one --obvious way to do it.
Well, you can use very well both petsc and scipy/numpy together. afaik petsc4py depends on numpy, so this you need in any case, and scipy is a set of very useful modules built on top of numpy (particularly its multidimensional array data type), addressing different fields of (scientific) computation, not just solving linear systems. It is true that the sparse matrix support in scipy is not as mature as some users need, but this can change :). So for now, you can use petsc (or <put your favourite sparse matrix package here>) for sparse stuff if you like, and scipy for other things that are not in petsc. There is no contradiction, imho. Just my 2kc, r.
Robert Cimrman wrote:
Ondrej Certik wrote:
On 4/13/07, Robert Cimrman <cimrman3@ntc.zcu.cz> wrote:
Nils Wagner wrote:
Please can you show me an example where petsc solvers are "better" than UMFPACK.
Petsc is really a superpackage providing many parallel linear solvers (iterative, direct, preconditioners, ...) together with nonlinear solvers, time steppers, etc. The solvers can be both petsc-native or external packages, nevertheless all are accessed via a uniform interface. IMHO UMFPACK is one of the optional external solvers petsc can use, so to answer your question, petsc can do anything that UMFPACK does and much more.
Yes, it's exactly like this. Thus, there is a question whether SciPy should support sparse solvers (my answer is yes) and if so, then it should support petsc, otherwise, for example me, I am not going to use it, as I want to try several solvers according to the problem.
My problems tend to be such that only a direct solvers work :)
What I am trying to say is that I don't want to write two versions of my code - one for petsc and second one for SciPy. And from the zen of python:
There should be one-- and preferably only one --obvious way to do it.
Well, you can use very well both petsc and scipy/numpy together. afaik petsc4py depends on numpy, so this you need in any case, and scipy is a set of very useful modules built on top of numpy (particularly its multidimensional array data type), addressing different fields of (scientific) computation, not just solving linear systems. It is true that the sparse matrix support in scipy is not as mature as some users need, but this can change :). So for now, you can use petsc (or <put your favourite sparse matrix package here>) for sparse stuff if you like, and scipy for other things that are not in petsc. There is no contradiction, imho.
Just my 2kc,
r. _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
Unfortunately petsc4py has no tutorial. http://www.cimec.org.ar/python/petsc4py.html#tutorial I guess that many users prefer well documented packages. Nils
Unfortunately petsc4py has no tutorial. http://www.cimec.org.ar/python/petsc4py.html#tutorial
I guess that many users prefer well documented packages.
Well, is there a tutorial for umfpack in scipy? I only found this: http://www.scipy.org/doc/api_docs/scipy.sparse.html But that's about the same amount of documentation as the petsc4py has in form of the docstrings. Ondrej
Ondrej Certik wrote:
Unfortunately petsc4py has no tutorial. http://www.cimec.org.ar/python/petsc4py.html#tutorial
I guess that many users prefer well documented packages.
Well, is there a tutorial for umfpack in scipy? I only found this:
http://www.scipy.org/doc/api_docs/scipy.sparse.html
But that's about the same amount of documentation as the petsc4py has in form of the docstrings.
Ondrej _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
Try
from scipy import * help (linsolve)
Nils
Robert Cimrman wrote:
Nils Wagner wrote:
Please can you show me an example where petsc solvers are "better" than UMFPACK.
Petsc is really a superpackage providing many parallel linear solvers (iterative, direct, preconditioners, ...) together with nonlinear solvers, time steppers, etc. The solvers can be both petsc-native or external packages, nevertheless all are accessed via a uniform interface. IMHO UMFPACK is one of the optional external solvers petsc can use, so to answer your question, petsc can do anything that UMFPACK does and much more.
r. _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
The current UMFPACK version is 5.0.3. Which versions are supported by scipy ? As I mentioned in a previous email I still have trouble to install other versions than 4.4. Any pointer, how to install more recent versions of UMFPACK, would be appreciated. When you are talking about superpackages how about Sundials ? http://www.llnl.gov/CASC/sundials/ AFAIK the ode solvers in scipy cannot handle events and a DAE solver is also missing in scipy. http://www.wolfram.com/products/mathematica/newin51/eventhandling.html Are there plans to add this functionality to scipy (in the form of scikits) ? Nils
Nils Wagner wrote:
Robert Cimrman wrote:
Nils Wagner wrote:
Please can you show me an example where petsc solvers are "better" than UMFPACK.
Petsc is really a superpackage providing many parallel linear solvers (iterative, direct, preconditioners, ...) together with nonlinear solvers, time steppers, etc. The solvers can be both petsc-native or external packages, nevertheless all are accessed via a uniform interface. IMHO UMFPACK is one of the optional external solvers petsc can use, so to answer your question, petsc can do anything that UMFPACK does and much more.
The current UMFPACK version is 5.0.3. Which versions are supported by scipy ? As I mentioned in a previous email I still have trouble to install other versions than 4.4. Any pointer, how to install more recent versions of UMFPACK, would be appreciated.
I can use 5.0 without problems, so 5.0.3 should work too. I have downloaded the whole UFsparse suite, edited UFconfig/UFconfig.mk, cd into UMFPACK and typed 'make' (normal UMFPACK installation procedure). Then I edited my numpy/site.cfg to reflect the installation paths. I do remember you had some problems with this step, but I do not know why. Any other recent UMFPACK users out there? r. ps: I am leaving for one week, so I will not be able to answer UMFPACK related questions :)
On 4/13/07, Robert Cimrman <cimrman3@ntc.zcu.cz> wrote:
I can use 5.0 without problems, so 5.0.3 should work too.
I have downloaded the whole UFsparse suite, edited UFconfig/UFconfig.mk, cd into UMFPACK and typed 'make' (normal UMFPACK installation procedure). Then I edited my numpy/site.cfg to reflect the installation paths. I do remember you had some problems with this step, but I do not know why.
Any other recent UMFPACK users out there?
I'm using SuiteSparse version 2.1.1 dated 09/11/2006 which includes UMFPACK 5.0.1. IIRC I'm also using GotoBLAS for UMFPACK. I installed SuiteSparse to /opt/SuiteSparse and copied all the UMFPACK, CHOLMOD, AMD etc. header files to /usr/include. I remember having problems when trying to point SciPy at the header files in /opt/SuiteSparse. My site.cfg has the following entries: [amd] library_dirs = /opt/SuiteSparse/AMD/Lib include_dirs = /opt/SuiteSparse/AMD/Include amd_libs = amd [umfpack] library_dirs = /opt/SuiteSparse/UMFPACK/Lib include_dirs = /opt/SuiteSparse/UMFPACK/Include umfpack_libs = umfpack When running SciPy's setup.py I get the following output: umfpack_info: amd_info: FOUND: libraries = ['amd'] library_dirs = ['/opt/SuiteSparse/AMD/Lib'] swig_opts = ['-I/opt/SuiteSparse/AMD/Include'] define_macros = [('SCIPY_AMD_H', None)] include_dirs = ['/opt/SuiteSparse/AMD/Include'] FOUND: libraries = ['umfpack', 'amd'] library_dirs = ['/opt/SuiteSparse/UMFPACK/Lib', '/opt/SuiteSparse/AMD/Lib'] swig_opts = ['-I/opt/SuiteSparse/UMFPACK/Include', '-I/opt/SuiteSparse/AMD/Include'] define_macros = [('SCIPY_UMFPACK_H', None), ('SCIPY_AMD_H', None)] include_dirs = ['/opt/SuiteSparse/UMFPACK/Include', '/opt/SuiteSparse/AMD/Include'] -- Nathan Bell wnbell@gmail.com
Nils Wagner wrote:
When you are talking about superpackages how about Sundials ?
You keep asking this question over and over again, and you get the same answer every time: it will be wrapped when the people who want it wrapped put the effort into wrapping it. If you want to see wrappers for SUNDIALS, stop asking the question and start writing code. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
participants (7)
-
David Cournapeau -
John Pye -
Nathan Bell -
Nils Wagner -
Ondrej Certik -
Robert Cimrman -
Robert Kern