[Python-ideas] A concurrency survey of sorts

[Adam Jorgensen]

On 4 November 2011 09:41, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Fri, Nov 4, 2011 at 5:03 PM, Adam Jorgensen
> <adam.jorgensen.za at gmail.com> wrote:
> > The GIL makes them pseudo-pointless in CPython anyway and the headaches
> > arising from
> > threading are very frustrating.
>
> This is just plain false. Threads are still an excellent way to take a
> synchronous operation and make it asynchronous. Take a look at
> concurrent.futures in 3.2, which makes it trivial to take independent
> blocking tasks and run them in parallel. The *only* time the GIL
> causes problems is when you have CPU bound threads written in pure
> Python. That's only a fraction of all of the Python apps out there,
> many of which are either calling out to calculations in C or FORTRAN
> (scientific community, financial community) or else performing IO
> bound tasks (most everyone else with a network connection).
>

I would love to seem some actual stats on this? How many multi-threaded
apps are hitting the GIL barrier, etc...

Anyway, I consider myself refuted...


>
> People need to remember that *concurrency is a hard problem*. That's
> why we layer abstractions on top of it. The threading and
> multiprocessing modules are both fairly low level, so they offer lots
> of ways to shoot yourself in the foot, but also a lot of power and
> flexibility.
>
> The concurrent.futures model is a higher level abstraction that's much
> easier to get right.
>
> > Personally I would rather see an Actors library...
>
> And what is an actors library going to use as its concurrency
> mechanism if the threading and multiprocessing modules aren't there
> under the hood?
>

I said nothing about removing the multi-processing module, although it
would be nice if it spawned child processes didn't randomly zombify
for no good reason.

Regardless, I still think the GIL should be fixed or the threading module
removed. It's disingenuous to have a threading module when it doesn't work
as advertised due to an interpreter "feature" of dubious merit anyway.


>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20111104/ba1c4a20/attachment.html>