Hi everyone, I've been working with fairly large sparse matrices on a multiprocessor system lately and noticed that scipy.sparse is single-threaded. Since I needed faster computations, I've quickly added some OpenMP #pragma directives in scipy/sparse/sparsetools to the functions that I've been using in order to enable multithreading, what worked out nicely. I wondered if you would be interested in a more complete OpenMP-enabled version of scipy.sparse.setuptools. I've attached the patch of the quick-and-dirty changes that I made so far to this mail, to give you an idea. Best regards Max
Hello Max, I don't know how the community at large feels about it, but I would certainly love to know more about what you've done. I am days away from immersing myself in OpenMP coding for similar things. Cheers, Eric Carlson
On 2012-03-05, at 5:15 AM, Maximilian Nickel <nickel@dbs.ifi.lmu.de> wrote:
Hi everyone, I've been working with fairly large sparse matrices on a multiprocessor system lately and noticed that scipy.sparse is single-threaded. Since I needed faster computations, I've quickly added some OpenMP #pragma directives in scipy/sparse/sparsetools to the functions that I've been using in order to enable multithreading, what worked out nicely. I wondered if you would be interested in a more complete OpenMP-enabled version of scipy.sparse.setuptools. I've attached the patch of the quick-and-dirty changes that I made so far to this mail, to give you an idea.
Best regards Max <sparse-openmp.patch>
I am actually interested in this topic as well, especially considering Cython's recent support for them via cython.parallel. One thing I've heard Mark Florisson say is that code running with OpenMP pragmas in any thread other than the main one can cause bad things to happen, so I guess any SciPy policy on the inclusion of OpenMP pragmas will have to take the possibility of existing client code running a CPU bound thread in the background (which may be pretty remote, if current code doesn't release the GIL). Of course the caveats must also be weighed and, if accepted, quite clearly documented. David
With respect to OpenMP, it is worth noting that with MinGW (gcc on Windows) it requires an LGPL pthreads library. So it would mean we have to build SciPy with MSVC on Windows to avoid the LGPL licence taint. Sturla On 05.03.2012 11:15, Maximilian Nickel wrote:
Hi everyone, I've been working with fairly large sparse matrices on a multiprocessor system lately and noticed that scipy.sparse is single-threaded. Since I needed faster computations, I've quickly added some OpenMP #pragma directives in scipy/sparse/sparsetools to the functions that I've been using in order to enable multithreading, what worked out nicely. I wondered if you would be interested in a more complete OpenMP-enabled version of scipy.sparse.setuptools. I've attached the patch of the quick-and-dirty changes that I made so far to this mail, to give you an idea.
Best regards Max
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Thu, Apr 26, 2012 at 1:08 PM, Sturla Molden <sturla@molden.no> wrote:
With respect to OpenMP, it is worth noting that with MinGW (gcc on Windows) it requires an LGPL pthreads library. So it would mean we have to build SciPy with MSVC on Windows to avoid the LGPL licence taint.
Is there an LGPL license taint? So long as pthreads is dynamically linked this shouldn't place any requirements on scipy users or distributors. (And if there is a taint, wouldn't we have the same problem on Linux? pthreads there is also LGPL, along with the rest of libc.) - N
participants (5)
-
David Warde-Farley -
Eric Carlson -
Maximilian Nickel -
Nathaniel Smith -
Sturla Molden