[Python-Dev] Pythonic concurrency - cooperative MT

[Guido van Rossum]

On 9/30/05, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le vendredi 30 septembre 2005 à 08:32 -0500, skip at pobox.com a écrit :
> >     >> (C)  That scheduler is non-preemptive.  A single greedy generator can
> >     >>      starve all the others.
> >
> >     Antoine> Instead of looking at this as a problem, you could look at it
> >     Antoine> as a feature.
> >
> > Apple looked at it as a feature for years.  Not anymore.  <wink>
>
> You are missing the context:
> - at the OS level, it is not good to have cooperative MT because any
> badly-written app (there are lots of ;-)) can cause lack of
> responsiveness for the whole machine
> - at the application level, cooperative MT can be good to enforce robust
> semantics without disrupting other apps
>
> The latter is what we are discussing AFAIK: a different concurrency
> scheme inside a single application - not accross the whole OS or
> desktop.

I like this analysis.

> Actually, I think different concurrency schemes must be chosen and mixed
> depending on the semantics. For example, when you have a networked GUI
> app, it can be good to have the network in one preemptive thread (using
> e.g. Twisted) and the GUI in another preemptive thread (using GTK,
> wx...). This is because GUI and network have different latency
> characteristics and requirements.
> (of course, you can replace "preemptive thread" with "process" in the
> above description)

I'm skeptical of this. I think a traditional GUI+network app can
combine the network and UI tasks in one process -- usually they
interact somewhat and that's where the bugs show up. Also note that
this is not an application area where MP is very interesting -- most
of the time you're blocked waiting for the user or for a socket (or
likely both :) so a single CPU is all you need. I've never heard
someone complain that the GIL is in the way for these types of apps.

> So whatever innovatice concurrency scheme Python may come out, it should
> still be mixable with more traditional concurrency schemes, because
> required properties vary wildly even inside a single app.

I don't think you've proved that yet.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)