On 06/06/07, Eugen Wintersberger <eugen.wintersberger@jku.at> wrote:

Hi there I have just started with using threads in python and want to use them together with the integration routines in scipy. However, there seems to be a serious problem with threadsafty. I want to run two threads which perform integration simultaneously. However, when I do so the program exits with a Segmentation Fault. If I use only one thread the program runs without problems. Is there any simple possibility to use scipy in threads?

Much of scipy is based on publicly-available FORTRAN routines. This has the advantage that they tend to be numerically robust and efficient, but it has the disadvantage that they often have interfaces that are, ah, old-fashioned. Many of them have python wrappers that make them somewhat more comfortable to use from python, but thread-safety (let alone parallelism) has not always been a goal. So sometimes, yes, some scipy routines just aren't thread-safe. It's not documented, either, as far as I can tell. You should also be aware of python's Global Interpreter Lock, which doesn't help with this and which means that you often don't get the parallelism you were hoping for. How can you avoid these problems? Well, for the specific problem of integrating known functions, scipy includes (for example) scipy.integrate.quadrature which is written in scipy with a perfectly reasonable interface that should be perfectly thread-safe. (And as a bonus, for difficult integration problems it should do some significantly large vector operations, which release the GIL and allow other threads to run concurrently.) It's not a very smart integrator. A more general solution to the problem would be to wrap your favourite non-thread-safe scipy routines in a simple routine that made sure only one thread was in each at a time. You could even write a little decorator. The ideal solution, of course, is to file bugs on the scipy trac when you find one that isn't thread-safe, so somebody can at least put a mutex on it inside scipy, and maybe if possible make it properly thread-safe. Or, hmm. Someone could write a little file full of unit tests that check various scipy routines for thread safety. That would accelerate the above process. Anne