Enhancement Request: SuperLU_dist/SuperLU_mt replace current scipy.sparse.superLU
Hi dev-list, I've logged an enhancement request on github to bring the parallel versions of SuperLU into scipy. As requested I'm circulating the enhancement here to solicit comment/feedback/engagement. Content of request is as follows (with light editing and links): "Is your feature request related to a problem? Please describe. The direct solution of sparse matrices is a common problem that arises across many domains. Current scipy.sparse.linalg provides spsolve and splu to solve such classes of problems. Neither solution appears to utilise more than one or two threads leaving a lot of the available compute on modern machines unused. Other solutions (MUMPS, Pardiso) exploit multicore processors but are either minimally supported and/or locked behind proprietary libraries. Describe the solution you'd like Fortunately SuperLU has two multithreaded implementations superLU_mt and superLU_dt, both actively maintained (SuperLU: Home Page (nersc.gov) <https://portal.nersc.gov/project/sparse/superlu/>). Ideally SciPy would replace the current sequential implementation with one of the parallel implementations to yield large performance improvements. Describe alternatives you've considered pyMKL (Pardiso dwfmarchant/pyMKL: Python wrappers to Intel MKL routines (github.com) <https://github.com/dwfmarchant/pyMKL>) - MKL with issues associated with that for non Intel processors, unmaintained relatively to scipy, PyTrilinos (implements all three superLU types but only available on osx and linux PyTrilinos | Trilinos <https://trilinos.github.io/pytrilinos.html>), petsc (petsc4py PETSc / petsc · GitLab <https://gitlab.com/petsc/petsc>) (implements all three superLU types but only available on osx and linux). All are relatively inaccessible compared to ubiquity and accessibility of scipy. Additional context (e.g. screenshots) Anecdotally large number of projects use various non scipy sparse solvers to achieve high level of performance. This comes with considerable time and maintenance investment. It would be fantastic for scipy to provide a 'performance competitive' sparse direct solver out of the hood as it does for dense matrices." Cheers, Neil
Hi dev-list, Further last, realised that I failed to link to the issue itself despite linking to virtually everything else: Replace superLU sequential with superLU dist · Issue #14096 · scipy/scipy (github.com) <https://github.com/scipy/scipy/issues/14096> Cheers, Neil On Fri, 21 May 2021 at 10:04, Neil Godber <njgodber@gmail.com> wrote:
Hi dev-list,
I've logged an enhancement request on github to bring the parallel versions of SuperLU into scipy. As requested I'm circulating the enhancement here to solicit comment/feedback/engagement.
Content of request is as follows (with light editing and links):
"Is your feature request related to a problem? Please describe.
The direct solution of sparse matrices is a common problem that arises across many domains. Current scipy.sparse.linalg provides spsolve and splu to solve such classes of problems. Neither solution appears to utilise more than one or two threads leaving a lot of the available compute on modern machines unused. Other solutions (MUMPS, Pardiso) exploit multicore processors but are either minimally supported and/or locked behind proprietary libraries.
Describe the solution you'd like Fortunately SuperLU has two multithreaded implementations superLU_mt and superLU_dt, both actively maintained (SuperLU: Home Page (nersc.gov) <https://portal.nersc.gov/project/sparse/superlu/>). Ideally SciPy would replace the current sequential implementation with one of the parallel implementations to yield large performance improvements.
Describe alternatives you've considered pyMKL (Pardiso dwfmarchant/pyMKL: Python wrappers to Intel MKL routines (github.com) <https://github.com/dwfmarchant/pyMKL>) - MKL with issues associated with that for non Intel processors, unmaintained relatively to scipy, PyTrilinos (implements all three superLU types but only available on osx and linux PyTrilinos | Trilinos <https://trilinos.github.io/pytrilinos.html>), petsc (petsc4py PETSc / petsc · GitLab <https://gitlab.com/petsc/petsc>) (implements all three superLU types but only available on osx and linux). All are relatively inaccessible compared to ubiquity and accessibility of scipy.
Additional context (e.g. screenshots) Anecdotally large number of projects use various non scipy sparse solvers to achieve high level of performance. This comes with considerable time and maintenance investment. It would be fantastic for scipy to provide a 'performance competitive' sparse direct solver out of the hood as it does for dense matrices."
Cheers,
Neil
On Fri, May 21, 2021 at 2:11 AM Neil Godber <njgodber@gmail.com> wrote:
Hi dev-list,
Further last, realised that I failed to link to the issue itself despite linking to virtually everything else:
Replace superLU sequential with superLU dist · Issue #14096 · scipy/scipy (github.com) <https://github.com/scipy/scipy/issues/14096>
Cheers, Neil
On Fri, 21 May 2021 at 10:04, Neil Godber <njgodber@gmail.com> wrote:
Hi dev-list,
Thanks for the proposal Neil.
I've logged an enhancement request on github to bring the parallel versions of SuperLU into scipy. As requested I'm circulating the enhancement here to solicit comment/feedback/engagement.
Content of request is as follows (with light editing and links):
"Is your feature request related to a problem? Please describe.
The direct solution of sparse matrices is a common problem that arises across many domains. Current scipy.sparse.linalg provides spsolve and splu to solve such classes of problems. Neither solution appears to utilise more than one or two threads leaving a lot of the available compute on modern machines unused. Other solutions (MUMPS, Pardiso) exploit multicore processors but are either minimally supported and/or locked behind proprietary libraries.
Describe the solution you'd like Fortunately SuperLU has two multithreaded implementations superLU_mt and superLU_dt, both actively maintained (SuperLU: Home Page (nersc.gov) <https://portal.nersc.gov/project/sparse/superlu/>). Ideally SciPy would replace the current sequential implementation with one of the parallel implementations to yield large performance improvements.
The relevant one seems to be superLU_mt. The `_dt` flavor is for distributed computing, which is out of scope for SciPy. superLU_mt says it has both pthreads and OpenMP interfaces - we'd prefer pthreads (and actually forbid OpenMP within SciPy).
The last release of superLU_mt is v3.1, from March 2016. Our current copy of SuperLU is 5.2.1, from May 2016 - but then there's a bunch of patches applied to it, and then https://github.com/scipy/scipy/pull/12243 did a sync with SuperLU master in May 2020. So it would be necessary to figure out if we can get superLU_mt from upstream master as well, and that we don't lose bug fixes we applied since 2016. Could you look into that Neil? And do you know if the parallel and sequential versions are developed in sync or not? Cheers, Ralf Describe alternatives you've considered
pyMKL (Pardiso dwfmarchant/pyMKL: Python wrappers to Intel MKL routines (github.com) <https://github.com/dwfmarchant/pyMKL>) - MKL with issues associated with that for non Intel processors, unmaintained relatively to scipy, PyTrilinos (implements all three superLU types but only available on osx and linux PyTrilinos | Trilinos <https://trilinos.github.io/pytrilinos.html>), petsc (petsc4py PETSc / petsc · GitLab <https://gitlab.com/petsc/petsc>) (implements all three superLU types but only available on osx and linux). All are relatively inaccessible compared to ubiquity and accessibility of scipy.
Additional context (e.g. screenshots) Anecdotally large number of projects use various non scipy sparse solvers to achieve high level of performance. This comes with considerable time and maintenance investment. It would be fantastic for scipy to provide a 'performance competitive' sparse direct solver out of the hood as it does for dense matrices."
Cheers,
Neil
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
Hi Ralf, Thanks for your email and happy to assist in whatever way I can. I've reached out to Dr Xiaoyei Li of Lawrence Berkley who appears to be the principal maintainer of the code base with a request to host the _mt code on github. Given the other two versions are already hosted I can't see that being an issue. I've also queried whether developments are done in sync and if so, if there is a pending patching for mt in the similar fashion to sequential. I will revert with further information as I receive it. Cheers, Neil On Mon, 24 May 2021 at 04:33, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Fri, May 21, 2021 at 2:11 AM Neil Godber <njgodber@gmail.com> wrote:
Hi dev-list,
Further last, realised that I failed to link to the issue itself despite linking to virtually everything else:
Replace superLU sequential with superLU dist · Issue #14096 · scipy/scipy (github.com) <https://github.com/scipy/scipy/issues/14096>
Cheers, Neil
On Fri, 21 May 2021 at 10:04, Neil Godber <njgodber@gmail.com> wrote:
Hi dev-list,
Thanks for the proposal Neil.
I've logged an enhancement request on github to bring the parallel versions of SuperLU into scipy. As requested I'm circulating the enhancement here to solicit comment/feedback/engagement.
Content of request is as follows (with light editing and links):
"Is your feature request related to a problem? Please describe.
The direct solution of sparse matrices is a common problem that arises across many domains. Current scipy.sparse.linalg provides spsolve and splu to solve such classes of problems. Neither solution appears to utilise more than one or two threads leaving a lot of the available compute on modern machines unused. Other solutions (MUMPS, Pardiso) exploit multicore processors but are either minimally supported and/or locked behind proprietary libraries.
Describe the solution you'd like Fortunately SuperLU has two multithreaded implementations superLU_mt and superLU_dt, both actively maintained (SuperLU: Home Page (nersc.gov) <https://portal.nersc.gov/project/sparse/superlu/>). Ideally SciPy would replace the current sequential implementation with one of the parallel implementations to yield large performance improvements.
The relevant one seems to be superLU_mt. The `_dt` flavor is for distributed computing, which is out of scope for SciPy. superLU_mt says it has both pthreads and OpenMP interfaces - we'd prefer pthreads (and actually forbid OpenMP within SciPy).
The last release of superLU_mt is v3.1, from March 2016. Our current copy of SuperLU is 5.2.1, from May 2016 - but then there's a bunch of patches applied to it, and then https://github.com/scipy/scipy/pull/12243 did a sync with SuperLU master in May 2020. So it would be necessary to figure out if we can get superLU_mt from upstream master as well, and that we don't lose bug fixes we applied since 2016. Could you look into that Neil? And do you know if the parallel and sequential versions are developed in sync or not?
Cheers, Ralf
Describe alternatives you've considered
pyMKL (Pardiso dwfmarchant/pyMKL: Python wrappers to Intel MKL routines (github.com) <https://github.com/dwfmarchant/pyMKL>) - MKL with issues associated with that for non Intel processors, unmaintained relatively to scipy, PyTrilinos (implements all three superLU types but only available on osx and linux PyTrilinos | Trilinos <https://trilinos.github.io/pytrilinos.html>), petsc (petsc4py PETSc / petsc · GitLab <https://gitlab.com/petsc/petsc>) (implements all three superLU types but only available on osx and linux). All are relatively inaccessible compared to ubiquity and accessibility of scipy.
Additional context (e.g. screenshots) Anecdotally large number of projects use various non scipy sparse solvers to achieve high level of performance. This comes with considerable time and maintenance investment. It would be fantastic for scipy to provide a 'performance competitive' sparse direct solver out of the hood as it does for dense matrices."
Cheers,
Neil
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
Hi Ralf + ScipyDev, Dr Li has helpfully uploaded the SuperLU_mt code base to github https://github.com/xiaoyeli/superlu_mt . Dr Li indicated that most development effort is focused on _dt, but that sequential and mt remain supported and bug reports will be attended to. Cheers, Neil On Mon, 24 May 2021 at 11:09, Neil Godber <njgodber@gmail.com> wrote:
Hi Ralf,
Thanks for your email and happy to assist in whatever way I can. I've reached out to Dr Xiaoyei Li of Lawrence Berkley who appears to be the principal maintainer of the code base with a request to host the _mt code on github. Given the other two versions are already hosted I can't see that being an issue. I've also queried whether developments are done in sync and if so, if there is a pending patching for mt in the similar fashion to sequential.
I will revert with further information as I receive it.
Cheers, Neil
On Mon, 24 May 2021 at 04:33, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Fri, May 21, 2021 at 2:11 AM Neil Godber <njgodber@gmail.com> wrote:
Hi dev-list,
Further last, realised that I failed to link to the issue itself despite linking to virtually everything else:
Replace superLU sequential with superLU dist · Issue #14096 · scipy/scipy (github.com) <https://github.com/scipy/scipy/issues/14096>
Cheers, Neil
On Fri, 21 May 2021 at 10:04, Neil Godber <njgodber@gmail.com> wrote:
Hi dev-list,
Thanks for the proposal Neil.
I've logged an enhancement request on github to bring the parallel versions of SuperLU into scipy. As requested I'm circulating the enhancement here to solicit comment/feedback/engagement.
Content of request is as follows (with light editing and links):
"Is your feature request related to a problem? Please describe.
The direct solution of sparse matrices is a common problem that arises across many domains. Current scipy.sparse.linalg provides spsolve and splu to solve such classes of problems. Neither solution appears to utilise more than one or two threads leaving a lot of the available compute on modern machines unused. Other solutions (MUMPS, Pardiso) exploit multicore processors but are either minimally supported and/or locked behind proprietary libraries.
Describe the solution you'd like Fortunately SuperLU has two multithreaded implementations superLU_mt and superLU_dt, both actively maintained (SuperLU: Home Page (nersc.gov) <https://portal.nersc.gov/project/sparse/superlu/>). Ideally SciPy would replace the current sequential implementation with one of the parallel implementations to yield large performance improvements.
The relevant one seems to be superLU_mt. The `_dt` flavor is for distributed computing, which is out of scope for SciPy. superLU_mt says it has both pthreads and OpenMP interfaces - we'd prefer pthreads (and actually forbid OpenMP within SciPy).
The last release of superLU_mt is v3.1, from March 2016. Our current copy of SuperLU is 5.2.1, from May 2016 - but then there's a bunch of patches applied to it, and then https://github.com/scipy/scipy/pull/12243 did a sync with SuperLU master in May 2020. So it would be necessary to figure out if we can get superLU_mt from upstream master as well, and that we don't lose bug fixes we applied since 2016. Could you look into that Neil? And do you know if the parallel and sequential versions are developed in sync or not?
Cheers, Ralf
Describe alternatives you've considered
pyMKL (Pardiso dwfmarchant/pyMKL: Python wrappers to Intel MKL routines (github.com) <https://github.com/dwfmarchant/pyMKL>) - MKL with issues associated with that for non Intel processors, unmaintained relatively to scipy, PyTrilinos (implements all three superLU types but only available on osx and linux PyTrilinos | Trilinos <https://trilinos.github.io/pytrilinos.html>), petsc (petsc4py PETSc / petsc · GitLab <https://gitlab.com/petsc/petsc>) (implements all three superLU types but only available on osx and linux). All are relatively inaccessible compared to ubiquity and accessibility of scipy.
Additional context (e.g. screenshots) Anecdotally large number of projects use various non scipy sparse solvers to achieve high level of performance. This comes with considerable time and maintenance investment. It would be fantastic for scipy to provide a 'performance competitive' sparse direct solver out of the hood as it does for dense matrices."
Cheers,
Neil
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
Hi Ralf, Saw your comment on the issue. Is there anything else I can do to help? Unfortunately my C/C++ skills are lacking so I'm not sure how much help I on the actual integration itself. Cheers, Neil On Sat, 29 May 2021 at 17:00, Neil Godber <njgodber@gmail.com> wrote:
Hi Ralf + ScipyDev,
Dr Li has helpfully uploaded the SuperLU_mt code base to github https://github.com/xiaoyeli/superlu_mt . Dr Li indicated that most development effort is focused on _dt, but that sequential and mt remain supported and bug reports will be attended to.
Cheers, Neil
On Mon, 24 May 2021 at 11:09, Neil Godber <njgodber@gmail.com> wrote:
Hi Ralf,
Thanks for your email and happy to assist in whatever way I can. I've reached out to Dr Xiaoyei Li of Lawrence Berkley who appears to be the principal maintainer of the code base with a request to host the _mt code on github. Given the other two versions are already hosted I can't see that being an issue. I've also queried whether developments are done in sync and if so, if there is a pending patching for mt in the similar fashion to sequential.
I will revert with further information as I receive it.
Cheers, Neil
On Mon, 24 May 2021 at 04:33, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Fri, May 21, 2021 at 2:11 AM Neil Godber <njgodber@gmail.com> wrote:
Hi dev-list,
Further last, realised that I failed to link to the issue itself despite linking to virtually everything else:
Replace superLU sequential with superLU dist · Issue #14096 · scipy/scipy (github.com) <https://github.com/scipy/scipy/issues/14096>
Cheers, Neil
On Fri, 21 May 2021 at 10:04, Neil Godber <njgodber@gmail.com> wrote:
Hi dev-list,
Thanks for the proposal Neil.
I've logged an enhancement request on github to bring the parallel versions of SuperLU into scipy. As requested I'm circulating the enhancement here to solicit comment/feedback/engagement.
Content of request is as follows (with light editing and links):
"Is your feature request related to a problem? Please describe.
The direct solution of sparse matrices is a common problem that arises across many domains. Current scipy.sparse.linalg provides spsolve and splu to solve such classes of problems. Neither solution appears to utilise more than one or two threads leaving a lot of the available compute on modern machines unused. Other solutions (MUMPS, Pardiso) exploit multicore processors but are either minimally supported and/or locked behind proprietary libraries.
Describe the solution you'd like Fortunately SuperLU has two multithreaded implementations superLU_mt and superLU_dt, both actively maintained (SuperLU: Home Page (nersc.gov) <https://portal.nersc.gov/project/sparse/superlu/>). Ideally SciPy would replace the current sequential implementation with one of the parallel implementations to yield large performance improvements.
The relevant one seems to be superLU_mt. The `_dt` flavor is for distributed computing, which is out of scope for SciPy. superLU_mt says it has both pthreads and OpenMP interfaces - we'd prefer pthreads (and actually forbid OpenMP within SciPy).
The last release of superLU_mt is v3.1, from March 2016. Our current copy of SuperLU is 5.2.1, from May 2016 - but then there's a bunch of patches applied to it, and then https://github.com/scipy/scipy/pull/12243 did a sync with SuperLU master in May 2020. So it would be necessary to figure out if we can get superLU_mt from upstream master as well, and that we don't lose bug fixes we applied since 2016. Could you look into that Neil? And do you know if the parallel and sequential versions are developed in sync or not?
Cheers, Ralf
Describe alternatives you've considered
pyMKL (Pardiso dwfmarchant/pyMKL: Python wrappers to Intel MKL routines (github.com) <https://github.com/dwfmarchant/pyMKL>) - MKL with issues associated with that for non Intel processors, unmaintained relatively to scipy, PyTrilinos (implements all three superLU types but only available on osx and linux PyTrilinos | Trilinos <https://trilinos.github.io/pytrilinos.html>), petsc (petsc4py PETSc / petsc · GitLab <https://gitlab.com/petsc/petsc>) (implements all three superLU types but only available on osx and linux). All are relatively inaccessible compared to ubiquity and accessibility of scipy.
Additional context (e.g. screenshots) Anecdotally large number of projects use various non scipy sparse solvers to achieve high level of performance. This comes with considerable time and maintenance investment. It would be fantastic for scipy to provide a 'performance competitive' sparse direct solver out of the hood as it does for dense matrices."
Cheers,
Neil
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
Hi Dev-List, Regarding Replace superLU sequential with superLU dist · Issue #14096 · scipy/scipy (github.com) <https://github.com/scipy/scipy/issues/14096>, would it be preferable to retain the sequential version a,d simply adding the mt version as an option or would should simply swap out sequential for mt? Cheers, Neil On Sun, 30 May 2021 at 09:40, Neil Godber <njgodber@gmail.com> wrote:
Hi Ralf,
Saw your comment on the issue. Is there anything else I can do to help? Unfortunately my C/C++ skills are lacking so I'm not sure how much help I on the actual integration itself.
Cheers, Neil
On Sat, 29 May 2021 at 17:00, Neil Godber <njgodber@gmail.com> wrote:
Hi Ralf + ScipyDev,
Dr Li has helpfully uploaded the SuperLU_mt code base to github https://github.com/xiaoyeli/superlu_mt . Dr Li indicated that most development effort is focused on _dt, but that sequential and mt remain supported and bug reports will be attended to.
Cheers, Neil
On Mon, 24 May 2021 at 11:09, Neil Godber <njgodber@gmail.com> wrote:
Hi Ralf,
Thanks for your email and happy to assist in whatever way I can. I've reached out to Dr Xiaoyei Li of Lawrence Berkley who appears to be the principal maintainer of the code base with a request to host the _mt code on github. Given the other two versions are already hosted I can't see that being an issue. I've also queried whether developments are done in sync and if so, if there is a pending patching for mt in the similar fashion to sequential.
I will revert with further information as I receive it.
Cheers, Neil
On Mon, 24 May 2021 at 04:33, Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Fri, May 21, 2021 at 2:11 AM Neil Godber <njgodber@gmail.com> wrote:
Hi dev-list,
Further last, realised that I failed to link to the issue itself despite linking to virtually everything else:
Replace superLU sequential with superLU dist · Issue #14096 · scipy/scipy (github.com) <https://github.com/scipy/scipy/issues/14096>
Cheers, Neil
On Fri, 21 May 2021 at 10:04, Neil Godber <njgodber@gmail.com> wrote:
Hi dev-list,
Thanks for the proposal Neil.
I've logged an enhancement request on github to bring the parallel versions of SuperLU into scipy. As requested I'm circulating the enhancement here to solicit comment/feedback/engagement.
Content of request is as follows (with light editing and links):
"Is your feature request related to a problem? Please describe.
The direct solution of sparse matrices is a common problem that arises across many domains. Current scipy.sparse.linalg provides spsolve and splu to solve such classes of problems. Neither solution appears to utilise more than one or two threads leaving a lot of the available compute on modern machines unused. Other solutions (MUMPS, Pardiso) exploit multicore processors but are either minimally supported and/or locked behind proprietary libraries.
Describe the solution you'd like Fortunately SuperLU has two multithreaded implementations superLU_mt and superLU_dt, both actively maintained (SuperLU: Home Page (nersc.gov) <https://portal.nersc.gov/project/sparse/superlu/>). Ideally SciPy would replace the current sequential implementation with one of the parallel implementations to yield large performance improvements.
The relevant one seems to be superLU_mt. The `_dt` flavor is for distributed computing, which is out of scope for SciPy. superLU_mt says it has both pthreads and OpenMP interfaces - we'd prefer pthreads (and actually forbid OpenMP within SciPy).
The last release of superLU_mt is v3.1, from March 2016. Our current copy of SuperLU is 5.2.1, from May 2016 - but then there's a bunch of patches applied to it, and then https://github.com/scipy/scipy/pull/12243 did a sync with SuperLU master in May 2020. So it would be necessary to figure out if we can get superLU_mt from upstream master as well, and that we don't lose bug fixes we applied since 2016. Could you look into that Neil? And do you know if the parallel and sequential versions are developed in sync or not?
Cheers, Ralf
Describe alternatives you've considered
pyMKL (Pardiso dwfmarchant/pyMKL: Python wrappers to Intel MKL routines (github.com) <https://github.com/dwfmarchant/pyMKL>) - MKL with issues associated with that for non Intel processors, unmaintained relatively to scipy, PyTrilinos (implements all three superLU types but only available on osx and linux PyTrilinos | Trilinos <https://trilinos.github.io/pytrilinos.html>), petsc (petsc4py PETSc / petsc · GitLab <https://gitlab.com/petsc/petsc>) (implements all three superLU types but only available on osx and linux). All are relatively inaccessible compared to ubiquity and accessibility of scipy.
Additional context (e.g. screenshots) Anecdotally large number of projects use various non scipy sparse solvers to achieve high level of performance. This comes with considerable time and maintenance investment. It would be fantastic for scipy to provide a 'performance competitive' sparse direct solver out of the hood as it does for dense matrices."
Cheers,
Neil
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@python.org https://mail.python.org/mailman/listinfo/scipy-dev
On Tue, Jun 1, 2021 at 6:40 AM Neil Godber <njgodber@gmail.com> wrote:
Hi Dev-List,
Regarding Replace superLU sequential with superLU dist · Issue #14096 · scipy/scipy (github.com) <https://github.com/scipy/scipy/issues/14096>, would it be preferable to retain the sequential version a,d simply adding the mt version as an option or would should simply swap out sequential for mt?
Typically we'd want to use the `workers=None` API pattern (see, e.g. `scipy.fft.fft`) to avoid auto-parallelization by default. Reason: that messes with multiprocessing. I'd hope that we would be able to replace the sequential superLU code with the MT version, rather than carry duplicate code around. This assumes that asking for 1 thread from the MT version gives identical results and performance as the sequential version. That would need checking.
On Sun, 30 May 2021 at 09:40, Neil Godber <njgodber@gmail.com> wrote:
Hi Ralf,
Saw your comment on the issue. Is there anything else I can do to help? Unfortunately my C/C++ skills are lacking so I'm not sure how much help I on the actual integration itself.
Not sure - someone needs to volunteer to dig into the C code, without that not much is going to happen. After initial conversion is done, extra testing and writing more tests will be useful contributions. Cheers, Ralf
participants (2)
-
Neil Godber -
Ralf Gommers