On Thu, Oct 27, 2011 at 9:39 PM, Mark Shannon <mark@hotpy.org> wrote:
Greg Ewing wrote:
Mark Shannon wrote:
Why not have proper co-routines, instead of hacked-up generators?
What do you mean by a "proper coroutine"?
A parallel, non-concurrent, thread of execution. It should be able to transfer control from arbitrary places in execution, not within generators.
Stackless provides coroutines. Greenlets are also coroutines (I think).
Lua has them, and is implemented in ANSI C, so it can be done portably. See: http://www.jucs.org/jucs_10_7/coroutines_in_lua/de_moura_a_l.pdf
(One of the examples in the paper uses coroutines to implement generators, which is obviously not required in Python :) )
(It's kinda late here and I'm writing this without looking up details in the source, so don't be shocked if some of the technical points don't quite align with reality. The gist should still hold) It's definitely an interesting point to think about. The thing that makes generators fall short of being true coroutines is that they don't really have the separate stacks that coroutines need. Instead, they borrow the stack of whatever thread invoked next(), send() or throw(). This means that, whenever a generator yields, that stack needs to be unwound, suspending each affected generator in turn, strung together by references between the generator objects rather than remaining a true frame stack. This means that *every* frame in the stack must be a generator frame for the suspension to reach the outermost generator frame - ordinary function frames can't be suspended like that. So, let's suppose that instead of trying to change the way calls work (to create generator frames all the way down), the coroutine PEP instead proposed a new form of *yield* expression: coyield The purpose of 'codef' would then be to declare that a function maintains it's *own* stack frame, independent of that of the calling thread. Unlike 'yield' (which only suspends the current frame), 'coyield' would instead suspend the entire coroutine, returning control to the frame that invoked next(), send() or throw() on the coroutine. Notably, *coyield* would *not* have any special effect on the specific function that used it - instead, it would just be a runtime error if coyield was encountered and there was no coroutine frame on the stack. That actually sounds reasonably feasible to me (and I like it better than the "generator frames all the way down" approach). There are likely to be some nasty implementation problems teasing out the thread local state from the interpreter core though (and it poses interesting problems for other things like the decimal module that also use thread local state). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia