[Python-Dev] Pythonic concurrency - cooperative MT

[Antoine Pitrou]

> (C)  That scheduler is non-preemptive.  A single
> greedy generator can starve all the others.

Instead of looking at this as a problem, you could look at it as a
feature. Since generators can't be switched at arbitrary places, the
programmer has to define his/her synchronization points explicitly.
Let me contrast this:
- in preemptive MT, threads can switch at every moment by default, and
you have to explicitly wrap some pieces of code in critical sections (or
other primitives) to protect the semantics of your program; every time
you forget a critical section, you introduce a bug
- in cooperative MT, threads can only switch where the programmer
explicitly allows it; every time you forget a synchronization point, you
give your program worse performance (latency), but you *don't* introduce
a bug

So you have a scheme where you seek optimal performance (latency) but
you take the risk of a huge number of difficult bugs, and you have a
scheme where good performance needs more careful coding *but* the
paradigm avoids difficult bugs /by construction/.

By the way, the cooperative MT is exactly the Twisted approach, except
implemented in a different manner (event loop instead of explicit
cooperative tasks).

Regards

Antoine.