Adam Jorgensen, 04.11.2011 08:53:
On 4 November 2011 09:41, Nick Coghlan wrote:
On Fri, Nov 4, 2011 at 5:03 PM, Adam Jorgensen wrote:
The GIL makes them pseudo-pointless in CPython anyway and the headaches arising from threading are very frustrating.
This is just plain false.
The first part, yes. The second - depends. Threading, especially when applied to the wrong task, is a very good way to give you headaches.
Threads are still an excellent way to take a synchronous operation and make it asynchronous. Take a look at concurrent.futures in 3.2, which makes it trivial to take independent blocking tasks and run them in parallel. The *only* time the GIL causes problems is when you have CPU bound threads written in pure Python. That's only a fraction of all of the Python apps out there, many of which are either calling out to calculations in C or FORTRAN (scientific community, financial community) or else performing IO bound tasks (most everyone else with a network connection).
I would love to seem some actual stats on this? How many multi-threaded apps are hitting the GIL barrier, etc...
In the numerics corner, multi-threaded CPU bound code is surprisingly common. In multi-server setups, people commonly employ MPI&friends, but especially on multi-core machines (usually less than 64 cores), threading is quite widely used. Proof? Cython just gained support for OpenMP based parallel loops, due to popular request. However, computational code usually doesn't hit the "GIL barrier", as you call it, because the really heavy computations don't run in the interpreter but straight on the iron. So, no, neither I/O bound tasks nor CPU bound numerics tasks get in conflict with the GIL, unless you do something wrong. That's the main theme, BTW. If you're a frequent reader of c.l.py, you'll quickly notice that those who complain most loudly about the GIL usually just do so because they do something wrong. Threading isn't the silver bullet that you shoot at the task at hand. It's *one* way to solve *some* kinds of concurrency problems, and certainly not a trivial one. Stefan