[Python-Dev] Pythonic concurrency

[Phillip J. Eby]

At 10:47 AM 10/7/2005 -0600, Bruce Eckel wrote:
>Also, look at the work that Scott Meyers, Andrei Alexandrescu, et al
>did on the "Double Checked Locking" idiom, showing that it was broken
>under threading. That was by no means "trivial and obvious" during all
>the years that people thought that it worked.

One of the nice things about the GIL is that it means double-checked 
locking *does* work in Python.  :)


>My own experience in discussions with folks who think that threading
>is transparent usually uncovers, after a few appropriate questions,
>that said person doesn't actually understand the depth of the issues
>involved. A common story is someone who has written a few programs and
>convinced themselves that these programs work (the "it works for me"
>proof of correctness). Thus, concurrency must be easy.
>
>I know about this because I have learned the hard way throughout many
>years, over and over again. Every time I've thought that I understood
>concurrency, something new has popped up and shown me a whole new
>aspect of things that I have heretofore missed. Then I start thinking
>"OK, now I finally understand concurrency."

The day when I knew, beyond all shadow of a doubt, that the people who say 
threading is easy are full of it, is when I wrote an event-driven 
co-operative multitasking system in Python and managed to create a race 
condition in *single-threaded code*.

Of course, due to its nature, a race condition in an event-driven system is 
at least reproducible given the same sequence of events, and it's fixable 
using "turns" (as described in a paper posted here yesterday).  With 
threads, it's not anything like reproducible, because pre-emptive threading 
is non-deterministic.

What the GIL-ranters don't get is that the GIL actually gives you just 
enough determinism to be able to write threaded programs that don't crash, 
and that maybe will even work if you treat every point of interaction 
between threads as a minefield and program with appropriate care.  So, if 
threads are "easy" in Python compared to other langauges, it's *because of* 
the GIL, not in spite of it.