[Cython] OpenMP support

[Stefan Behnel]

Dag Sverre Seljebotn, 11.03.2011 08:56:
> Basically, I'm +1 to anything that can make me
> pretend the GIL doesn't exist, even if it comes with a 2x performance hit:
> Because that will make me write parallell code (which I can't be bothered
> to do in Cython currently), and I have 4 cores on the laptop I use for
> debugging, so I'd still get a 2x speedup.
>
> Perhaps the long-term solution is something like an "autogil" mode could
> work where Cython automatically releases the GIL on blocks where it can
> (such as a typed for-loop), and acquires it back when needed (an
> exception-raising if-block within said for-loop).

I assume you mean this to become a decorator or other option written into 
the code.


> And when doing
> multi-threading, GIL-requiring calls are dispatched to a master GIL-holding
> thread (which would not be a worker thread, i.e. on 4 cores you'd have 4
> workers + 1 GIL-holding support thread). So the advice for speeding up code
> is simply "make sure your code is all typed", just like before, but people
> can follow that advice without even having to learn about the GIL.

The GIL does not only protect the interpreter core. It also protects C 
level data structures in user code and keeps threaded code from running 
amok. Releasing and acquiring it doesn't come for free either, so besides 
likely breaking code that was not specifically written to be reentrant, 
releasing it automatically may also introduce a performance penalty for 
many users.

I'm very happy the GIL exists, and I'm against anything that tries to 
disable it automatically. Threading is an extremely dangerous programming 
model. The GIL has its gotchas, too, but it still simplifies it quite a 
bit. Actually, threading is so complex and easy to get wrong, that any 
threaded code should always be written specifically to support threading. 
Explicitly acquiring and releasing the GIL is really just a minor issue on 
that path.

Stefan