[Python-Dev] Pythonic concurrency - cooperative MT

[Jp Calderone]

On Fri, 30 Sep 2005 17:26:27 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>Hi,
>
>> I've never heard
>> someone complain that the GIL is in the way for these types of apps.
>
>I've never said so either.
>I was just saying that it can be useful to mix cooperative threading and
>preemptive threading in the same app, i.e. have different domains of
>cooperative threading which are preemptively scheduled by the OS. That
>has nothing to do with the GIL, I think (but I don't know much in Python
>internals).
>
>For instance the mix between Twisted and GUI event loops is a very
>common topic on the twisted mailing-list, because people have trouble
>getting it right inside a single system thread. The Twisted people have
>started advocating something called the ThreadedSelectReactor, which I
>haven't looked at but whose name suggests that some operations are
>performed in a helper system thread.
>

"Advocating" might be putting it too strongly :)  "Experimenting with" describes the current state of things most accurately.

The problem it aims to solve is integration with cooperative threading systems which don't work very well.  An example of such a loop is the wx event loop.  

Whenever a modal dialog is displayed or a menu is activated, wx's loop stops cooerating.  The only way we've found thus far to avoid this is to run wx in a separate thread, so even when it stops cooperating, network code can still get scheduled.

Jp