[Python-Dev] coroutines and microthreads
Thu, 16 Nov 2000 21:29:52 -0500
> I'll first give an example. Tim Peters gave a standard example to
> return the fringe of a list (checked in as Demo/threads/fcmp.py).
> Using my proposed API this example could be rewritten a bit
> cleaner, as follows:
> from coro import coroutine, suspend, EarlyExit # Provisional module name
> def fringe(L):
> for x in L:
> if type(x) is type(L):
> def printinorder(L):
> c = coroutine(f, L)
> while 1:
> print c(),
> except EarlyExit:
> x = [0, 1, [2, ], [4,5], [[]]]
> printinorder(x) # prints "0 1 2 3 4 5 6"
> This is about as simple as it gets. It supports both generators (like
> the example here) and coroutines (like Tim's Demo/threads/squasher.py
This looks very nice and clean. It looks sufficient for the type
of thing I'm doing (and planning to do), but is it really a full
coroutine API? That is, doesn't the fact that you always
suspend to the guy who just activated you make this a
generator API? (OTOH, if I want A and B to talk to each other
as "coroutines", is it sufficient to make them both "generators"
and then glue them together with another routine that just
> Besides the APIs shown here (coroutine(), suspend(), and EarlyExit) I
> propose a function current() which returns the current coroutine
> object. There should also be a way to kill a coroutine (or at least
> to send an exception). When a coroutine falls through at its end,
> *some* other coroutine needs to be resumed.
- I've found it handy that Christian's stuff lets you grab a
coroutine as well as return one (that is, either side can
instigate it). Not sure if that's necessary.
- What do you mean by "kill a coroutine"? You can't interrupt
one, so isn't it sufficient that when it goes out of scope it gets
- It appears from your example that falling off the end
automatically raises an EarlyExit. I think I see more
arguments for that than against it :-).
> I'm much less clear about microthreads (uthreads). Last time, I
> thought this was a fancy name for coroutines. It isn't! Microthreads
> are "almost real" threads, with round-robin scheduling. What makes
> them attractive to some is the fact that they don't use operating
> system resources: there's no OS-level stack or kernel process table
> entry associated with them. This also accounts for their biggest
> weakness: when a microthread is suspended for I/O, *all* microthreads
> are suspended. In limited contexts, such as a network server, this
> can be solved by an approach similar to that in Gordon's
> SelectDispatcher. (It would be a titanic effort to do this for every
> potentially blocking I/O operation, and it probably wouldn't work well
> with C extensions.)
Using the raw Win32 API, I think you could come pretty close.
I've been wondering if it's possible to do something that would
get Cameron to quit raving about Tcl's event loop ;-).
> I'm not sure what applications require the round-robin scheduling
> property of uthreads -- certainly Gordon's application would seem to
> be doing just fine without it (I'm not sure if it is being scheduled
> round-robin or not).
Each coroutine (or whatever they are) runs until it calls one of
the SelectDispatcher methods that suspends it. The socket
methods suspend it until select says the socket is ready;
yield suspends it till the next time round the select loop
(which has a timeout). So piggish routines are encouraged to
yield once in a while.
(While I'm doing things the other way 'round, I believe I could
use your API without changing SelectDispatcher's API at all.)
> Proper round-robin scheduling for uthreads requires explicit switching
> code in the Python interpreter. Stackless provides this, at the same
> place where in regular Python the global interpreter lock is released
> and re-acquired to give other threads a chance to run. Is this
While I'm not really a uthread user, I think they would give you
an unqualified "yes". The advantage to explicitly yeilding is
that (with proper thought) you don't need nasty things like
locks; the disadvantage (as demonstrated by a particular OS
with a rabidly fanatical following) is that one jerk can ruin it for