[Python-ideas] Cofunctions - Back to Basics

[Steven D'Aprano]

Nick Coghlan wrote:
> On Fri, Oct 28, 2011 at 5:33 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>> On 27 October 2011 20:15, Arnaud Delobelle <arnodel at gmail.com> wrote:
>>
>>>> That article states that Python has coroutines as of 2.5 -- that's
>>>> incorrect, isn't it?
>>> Generator functions + trampoline = coroutines
>> If I understand, the cofunctions of this thread aren't coroutines as
>> such, they are something intended to make writing coroutines easier in
>> some way. My problem is that it's not at all obvious how they help.
>> That's partly because the generator+trampoline idiom, although
>> possible, is not at all common so that there's little in the way of
>> examples, and even less in the way of common understanding, of how the
>> idiom works and what problems there are putting it into practice.
> 
> I highly recommend reading the article Mark Shannon linked earlier in

If you're talking about this:

http://www.jucs.org/jucs_10_7/coroutines_in_lua/de_moura_a_l.pdf

I have read it, and while all very interesting, I don't see how it 
answers the big questions about motivating use-cases for cofunctions as 
described in this PEP.

One specific thing I took out of this is that only the main body of a 
Python generator can yield. That is, if I write this:

def gen():
     def inner():
         yield 1
         yield 2
     yield 0
     inner()  # yield 1 and 2
     yield 3

it does not behave as I would like. Instead, I have to write this:

def gen():
     def inner():
         yield 1
         yield 2
     yield 0
     # In Python 3.3, the next 2 lines become "yield from inner()"
     for x in inner():
         yield x
     yield 3


I can see how that would be a difficulty, particularly when you move 
away from simple generators yielding values to coroutines that accept 
values, but isn't that solved by the "yield from" syntax?


> the thread. I confess I didn't finish the whole thing, but even the
> first half of it made me realise *why* coroutine programming in Python
> (sans Stackless or greenlet) is such a pain: *every* frame on the
> coroutine stack has to be a generator frame in order to support
> suspending the generator.

I understand that, or at least I *think* I understand that, but I don't 
understand what that implies in practice when writing Python code.

If all you are saying is that you can't suspend an arbitrary function at 
at arbitrary point, well, true, but that's a good thing, surely? The 
idea of a function is that it has one entry point, it does its thing, 
and then it returns. If you want different behaviour, don't use a function.

Or do you mean something else? Actual working Python code (or not 
working, as the case may be) would probably help.


> Ideally, we would like to make it possible to write code like this:
> 
>     def coroutine_friendly_io(*args, **kwds):
>         if in_coroutine():
>             return coyield AsychronousRequest(async_op, *args, **kwds)
>         else:
>             return sync_op(*args, **kwds)

Why would you want one function to do double duty as both blocking and 
non-blocking? Particularly when the two branches don't appear to share 
any code (at least not in the example as given). To me, this seems like 
"Do What I Mean" magic which would be better written as a pair of functions:

     def coroutine_friendly_io(*args, **kwds):
         yield from AsychronousRequest(async_op, *args, **kwds)

     def blocking_io(*args, **kwargs):
         return sync_op(*args, **kwds)



-- 
Steven