[Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342

[Phillip J. Eby]

At 08:03 PM 6/16/2005 -0700, Guido van Rossum wrote:
>Someone should really come up with some realistic coroutine examples
>written using PEP 342 (with or without "continue EXPR").

How's this?

    def echo(sock):
        while True:
            try:
                data = yield nonblocking_read(sock)
                yield nonblocking_write(sock, data)
            except ConnectionLost:
                pass

    def run_server(sock, handler):
        while True:
            connected_socket = yield nonblocking_accept(sock)
            schedule_coroutine(handler(connected_socket))

    schedule_coroutine(
        run_server(
            setup_listening_socket("localhost","echo"),
            echo
    )

Of course, I'm handwaving a lot here, but this is a much clearer example 
than anything I tried to pull out of the coroutines I've written for actual 
production use.  That is, I originally started this email with a real 
routine from a complex multiprocess application doing lots of IPC, and 
quickly got bogged down in explaining all the details of things like 
yielding to semaphores and whatnot.  But I can give you that example too, 
if you like.

Anyway, the handwaving above is only in explanation of details, not in 
their implementability.    It would be pretty straightforward to use 
Twisted's callback facilities to trigger next() or throw() calls to resume 
the coroutine in progress.  In fact, schedule_coroutine is probably 
implementable as something like this in Twisted:

     def schedule_coroutine(geniter, *arg):
         def resume():
             value = geniter.next(*arg)
             if value is not None:
                 schedule_coroutine(value)
         reactor.callLater(0, resume)

This assumes, of course, that you only yield between coroutines.  A better 
implementation would need to be more like the events.Task class in 
peak.events, which can handle yielding to Twisted's "Deferreds" and various 
other kinds of things that can provide callbacks.  But this snippet is 
enough to show that yield expressions let you write event-driven code 
without going crazy writing callback functions.

And of course, you can do this without yield expressions today, with a 
suitably magic function, but it doesn't read as well:

            yield nonblocking_accept(sock); connected_socket = events.resume()

This is how I actually do this stuff today.  'events.resume()' is a magic 
function that uses sys._getframe() to peek at the argument passed to the 
equivalent of 'next()' on the Task that wraps the 
generator.  events.resume() can also raise an error if the equivalent of 
'throw()' was called instead.

With yield expressions, the code in those Task methods would just do 
next(arg) or throw(*sys.exc_info()) on the generator-iterator, and 
'events.resume()' and its stack hackery could go away.