[Tutor] threading mind set
steve at pearwood.info
Mon May 14 02:31:36 CEST 2012
Russel Winder wrote:
> On Sun, 2012-05-13 at 10:22 +1000, Steven D'Aprano wrote:
>> carlo locci wrote:
>>> Hello All,
>>> I've started to study python a couple of month ago(and I truly love it :)),
>>> however I'm having some problems understanding how to modify a sequential
>>> script and make it multithreaded (I think it's because I'm not used to
>>> think in that way),
>> No, that's because multithreading and parallel processing is hard.
> Shared memory multithreading may be hard due to locks, semaphores,
> monitors, etc., but concurrency and parallelism need not be hard.
No hard compared to what?
> Using processes and message passing, using dataflow, actors or CSP,
> parallelism and concurrency is far more straightforward. Not easy,
> agreed, but then programming isn't easy.
My argument is that once you move beyond the one-operation-after-another
programming model, almost any parallel processing problem is harder than the
equivalent sequential version, inherently due to the parallelism. Except
perhaps for "embarrassingly parallel" problems, parallelism adds complexity
even if your framework abstracts away most of the tedious detail like semaphores.
Once you move beyond sequential execution, you have to think about issues that
don't apply to sequential programs: how to divide the task up between
processes/threads/actors/whatever, how to manage their synchronization,
resource starvation (e.g. deadlocks, livelocks), etc.
We have linear minds and it doesn't take that many real-time parallel tasks to
overwhelm the human brain. I'm not saying that people can't reason in
parallel, because we clearly can and do, but it's inherently harder than
> The GIL in Python is a bad thing for parallelism. Using the
> multiprocessing package or concurrent.futures gets over the problem.
> Well sort of, these processes are a bit heavyweight compared to what can
> be achieved on the JVM or with Erlang.
Python doesn't have a GIL. Some Python implementations do, most obviously
CPython, the reference implementation. But Jython and IronPython don't. If the
GIL is a problem for your program, consider running it on Jython or IronPython.
More information about the Tutor