[Python-Dev] Proposal for 2.5: Returning values from PEP 342 enhanced generators

[Piet Delport]

PEP 255 ("Simple Generators") closes with:

> Q. Then why not allow an expression on "return" too?
>
> A. Perhaps we will someday.  In Icon, "return expr" means both "I'm
>    done", and "but I have one final useful value to return too, and
>    this is it".  At the start, and in the absence of compelling uses
>    for "return expr", it's simply cleaner to use "yield" exclusively
>    for delivering values.

Now that Python 2.5 gained enhanced generators (multitudes rejoice!), i think
there is a compelling use for valued return statements in cooperative
multitasking code, of the kind:

def foo():
    Data = yield Client.read()
    [...]
    MoreData = yield Client.read()
    [...]
    return FinalResult

def bar():
    Result = yield foo()

For generators written in this style, "yield" means "suspend execution of the
current call until the requested result/resource can be provided", and
"return" regains its full conventional meaning of "terminate the current call
with a given result".

The simplest / most straightforward implementation would be for "return Foo"
to translate to "raise StopIteration, Foo". This is consistent with "return"
translating to "raise StopIteration", and does not break any existing
generator code.

(Another way to think about this change is that if a plain StopIteration means
"the iterator terminated", then a valued StopIteration, by extension, means
"the iterator terminated with the given value".)

Motivation by real-world example:

One system that could benefit from this change is Christopher Armstrong's
defgen.py[1] for Twisted, which he recently reincarnated (as newdefgen.py) to
use enhanced generators. The resulting code is much cleaner than before, and
closer to the conventional synchronous style of writing.

[1] the saga of which is summarized here:
    http://radix.twistedmatrix.com/archives/000114.html

However, because enhanced generators have no way to differentiate their
intermediate results from their "real" result, the current solution is a
somewhat confusing compromise: the last value yielded by the generator
implicitly becomes the result returned by the call. Thus, to return
something, in general, requires the idiom "yield Foo; return". If valued
returns are allowed, this would become "return Foo" (and the code implementing
defgen itself would probably end up simpler, as well).