[Python-Dev] Simple generators, round 2

[Tim Peters]

[Neil Schemenauer]
> Apparently they [Icon-style generators] are good for lots of other
> things too.  Tonight I implemented passing values using resume().
>  Next, I decided to see if I had enough magic juice to tackle the
> coroutine example from Gordon's stackless tutorial.  Its turns out
> that I didn't need the extra functionality.  Generators are enough.
>
> The code is not too long so I've attached it.  I figure that some
> people might need a break from 2.1 release issues.

I'm afraid we were buried alive under them at the time, and I don't want this
one to vanish in the bit bucket!

> I think the generator version is even simpler than the coroutine
> version.
>
> [Example code for the Dahl/Hoare "squasher" program elided -- see
>  the archive]

This raises a potentially interesting point:  is there *any* application of
coroutines for which simple (yield-only-to-immediate-caller) generators
wouldn't suffice, provided that they're explicitly resumable?

I suspect there isn't.  If you give me a coroutine program, and let me add a
"control loop", I can:

1. Create an Icon-style generator for each coroutine "before the loop".

2. Invoke one of the coroutines "before the loop".

3. Replace each instance of

       coroutine_transfer(some_other_coroutine, some_value)

   within the coroutines by

       yield some_other_coroutine, some_value

4. The "yield" then returns to the control loop, which picks apart
   the tuple to find the next coroutine to resume and the value to
   pass to it.

This starts to look a lot like uthreads, but built on simple generator
yield/resume.

It loses some things:

A. Coroutine A can't *call* routine B and have B do a co-transfer
   directly.  But A *can* invoke B as a generator and have B yield
   back to A, which in turn yields back to its invoker ("the control
   loop").

B. As with recursive Icon-style generators, a partial result generated
   N levels deep in the recursion has to suspend its way thru N
   levels of frames, and resume its way back down N levels of frames
   to get moving again.  Real coroutines can transmit results directly
   to the ultimate consumer.

OTOH, it may gain more than it loses:

A. Simple to implement in CPython without threads, and at least
   possible likewise even for Jython.

B. C routines "in the middle" aren't necessarily show-stoppers.  While
   they can't exploit Python's implementation of generators directly,
   they *could* participate in the yield/resume *protocol*, acting "as
   if" they were Python routines.  Just like Python routines have to
   do today, C routines would have to remember their own state and
   arrange to save/restore it appropriately across calls (but to the
   C routines, they *are* just calls and returns, and nothing trickier
   than that -- their frames truly vanish when "suspending up", so
   don't get in the way).

the-meek-shall-inherit-the-earth<wink>-ly y'rs  - tim