[Python-ideas] Cofunctions - Back to Basics

[Nick Coghlan]

On Fri, Oct 28, 2011 at 11:01 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> If all you are saying is that you can't suspend an arbitrary function at at
> arbitrary point, well, true, but that's a good thing, surely? The idea of a
> function is that it has one entry point, it does its thing, and then it
> returns. If you want different behaviour, don't use a function.
>
> Or do you mean something else? Actual working Python code (or not working,
> as the case may be) would probably help.

The number 1 use case for coroutines is doing asynchronous I/O (and
other event loop based programming) elegantly. That's why
libraries/frameworks like Twisted and gevent exist. The *only* reason
to add support to the core is if we can do it in a way that makes
programs that use coroutines as straightforward to write as threaded
programs.

Coroutines are essentially useful for the same reason any form of
cooperative multitasking is useful - it's a way to structure a program
as concurrently executed sequences of instructions without incurring
the overhead of an OS level thread (or process!) for each operation
stream.

>> Ideally, we would like to make it possible to write code like this:
>>
>>    def coroutine_friendly_io(*args, **kwds):
>>        if in_coroutine():
>>            return coyield AsychronousRequest(async_op, *args, **kwds)
>>        else:
>>            return sync_op(*args, **kwds)
>
> Why would you want one function to do double duty as both blocking and
> non-blocking? Particularly when the two branches don't appear to share any
> code (at least not in the example as given). To me, this seems like "Do What
> I Mean" magic which would be better written as a pair of functions:
>
>    def coroutine_friendly_io(*args, **kwds):
>        yield from AsychronousRequest(async_op, *args, **kwds)
>
>    def blocking_io(*args, **kwargs):
>        return sync_op(*args, **kwds)

If you can't merge the synchronous and asynchronous version of your
I/O routines, it means you end up having to write everything in the
entire stack twice - once in an "event loop friendly" way (that
promises never to block - generator based coroutines are just a way to
help with writing code like that) and once in the normal procedural
way. That's why Twisted has its own version of virtually all the I/O
libraries in the standard library - the impedance mismatch between our
the standard library's blocking I/O model and the needs of event loop
based programming is usually just too big to overcome at the
library/framework level without significant duplication of
functionality. If the "block or suspend?" decision can instead be made
at the lowest layers (as is currently possible with greenlets), then
the intervening layers *don't need to care* what actually happens
under the hood and the two worlds can start to move closer together.

Compare the current situation with cooperative multitasking to the
comparative ease of programming with threaded code, where, with the
GIL in place, there's a mixture of pre-emptive multi-tasking (the
interpreter can switch threads between any two bytecodes) and
cooperative multi-tasking (voluntarily relinquishing the GIL), all of
which is handled at the *lowest* layer in the stack, and the upper
layers generally couldn't care in the least which thread they're
running in. You still have the inherent complexity of coping with
shared data access in a concurrent environment to deal with, of
course, but at least the code you're running in the individual threads
is just ordinary Python code.

The cofunction PEP as it stands does nothing to help with that
bifurcation problem, so I now see it as basically pointless.

By contrast, the greenlets module (and, before that, Stackless itself)
helps bring cooperative multitasking up to the same level as threaded
programming - within a greenlet, you're just writing ordinary Python
code. It just so happens that some of the functions you call may
suspend your frame stack and start running a different one for a
while.

The fact that greenlets exists (and works) demonstrates that it is
possible to correctly manage and execute multiple Python stacks within
a single OS thread. It's probably more worthwhile to focus on
providing either symmetric (greenlets style, "switch to specific
coroutine") or asymmetric (Lua style, "suspend and pass control back
to the frame that invoked the coroutine") coroutines rather than
assuming that the current "generators all the way down" restriction is
an inviolable constraint.

As far as a concrete example goes, let's consider a simple screen echo
utility that accepts an iterable and prints whatever comes out:

    def echo_all(iterable):
        for line in iterable:
            print(line)

So far so good. I can run that in my main OS thread, or in a subthread
and it will all work happily. I can even pass in an operation that
reads lines from a file or a socket.

OK, the socket case sounds potentially interesting, and it's obvious
how it works with blocking IO, but what about asynchronous IO and an
event loop? In current Python, the answer is that it *doesn't* work,
*unless* you use greenlets (or Stackless). If you want to use
generator based coroutines instead, you have to write a *new* version
of echo_all and it's a mess, because you have requests to the event
loop (e.g. "let me know when this socket has data ready") and the
actual data to be printed intermingled in the same communications
channel. And that all has to be manually passed back up to the event
loop, because there's no standardised way for the socket read
operation to say "look, just suspend this entire call stack until the
socket has some data for me".

Having something like a codef/coyield pair that allows the entire
coroutine stack to be suspended means a coroutine can use the given
"echo_all" function as is. All that needs to happen is that:

1. echo_all() gets invoked (directly or indirectly) from a coroutine
defined with codef
2. the passed in iterable uses coyield whenever it wants to pass
control back to the event loop that called the coroutine in the first
place

The intervening layer in the body of echo_all() can then remain
blissfully unaware that anything unusual is going on, just as it
currently has no idea when a socket read blocks, allowing other OS
threads to execute while waiting for data to arrive.

Yes, the nonblocking I/O operations would need to have a protocol that
they use to communicate with the event loop, but again, one of the
virtues of true coroutines is that the details of that event loop
implementation specific protocol can be hidden behind a standardised
API.

Under such a scheme, 'return x' and 'yield x' and function calls would
all retain their usual meanings.

Instead of directly invoking 'coyield x', nonblocking I/O operations
might write:

  event_loop.resume_when_ready(handle) # Ask event loop to let us know
when data is available

The need for trampoline scheduling largely goes away, since you only
need to go back to the event loop when you're going to do something
that may block.

It's even possible that new keywords wouldn't be needed (as greenlets
demonstrates).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia