[Python-ideas] Implementing Coroutines (was Cofunctions - Back to Basics)

[Mark Shannon]

Hi,

Full coroutines have a number of advantages over the proposed 
cofunctions. For one, coroutines require *no* language changes.

Here is a simple coroutine class interface (feel free to substitute/add 
your own names/signatures).

class Coroutine:

    def __init__(self, callable):
       ...

    def resume(value):
       '''Passes value to coroutine, either as initial parameter to 

          callable or as return value of yield'''

    def co_yield(value):
        'Yields (returns) value back to caller of resume() method.'

    def stop():
        'Stops the coroutine'

In order to implement this, it must be possible to transfer control 
*sideways* from one frame to another, without unwinding either stack.
In order to do this, Python-to-Python calls within the interpreter must 
not make calls at the C (OS/hardware) level.

This is quite simple to implement in practice.
Currently CPython does something like this (for Python to Python calls).

In interpreter:
     call PyFunction.__call__(parameters)

In PyFunction.__call__():
     Create new frame with parameters.
     Call interpreter with new frame.

By changing the call sequence to something like:

In interpreter:
     frame = PyFunction.make_frame(parameters)
     Push frame to frame-stack
     jump to start of new function.	

we have a 'stackless' implementation which can support coroutines.

The implementation of Coroutines in a portable fashion is explained on 
pages 7 and 8 of this paper:
http://www.lua.org/doc/jucs05.pdf

(Sorry to keep quoting Lua papers, but it is a cleaner VM than CPython. 
Python is a better language, though :) )

Stackless uses a slightly different approach, not for any fundamental 
reason, but because their goal is to minimise the diff with CPython.

Adding coroutines may create problems for the other implementations.
PyPy already has a stackless implementation, but Jython and IronPython 
might find to it harder to implement.
Any Jython or IronPython folks care to comment?

In summary, the cofunction proposal is a work-around for a limitation in 
the VM. By fixing the VM we can have proper coroutines.
Surely, it is better to make the VM support the features we want/need 
rather than bend those features to fit the VM?

Cheers,
Mark.