[Python-Dev] Yield-From Implementation Updated for Python 3

[Guido van Rossum]

On Sun, Aug 1, 2010 at 11:16 AM, P.J. Eby <pje at telecommunity.com> wrote:
> Just so you know, you don't need to wait for this to be added to Python in
> order to have such a construct; it just won't have the extra syntax sugar.
>  See the sample code I posted here using a "@From.container" decorator, and
> a "yield From()" call:
>
>  http://mail.python.org/pipermail/python-dev/2010-July/102320.html
>
> This code effectively reduces your generator nesting depth to a constant, no
> matter how deeply you nest sub-generator invocations.  It's not as efficient
> as the equivalent C implementation, but if you're actually being affected by
> nesting overhead now, it will nonetheless provide you with some immediate
> relief, if you backport it to 2.x code.  (It's not very 3.x-ish as it sits,
> really.)

Hi Phillip,

Your code is fiendishly clever (and I mean that as a compliment). Even
though only a few weeks ago I wrote a trampoline along these lines
(more similar to the one in PEP 342, example 3, but extended to
support async I/O), I don't understand all the details and corner
cases (e.g. the concatenation of stacks, which seems to have to do
with the special-casing of From objects in __new__) but I trust that
it works.

I am curious whether, if you need a trampoline for async I/O anyway,
there would be a swaying argument for integrating this functionality
into the general trampoline (as in the PEP 342 example), or whether it
would be better to separate the async I/O trampoline from the "From"
trampoline. The latter would in a sense be more future-proof because
once PEP 380 becomes a reality the From class can be dropped in favor
of yield-from syntax (and raise StopIteration(x) in favor of return
x). But it seems a bit of a waste to have two different trampolines,
especially since the trampoline itself is so hard to understand
(speaking for myself here :-). ISTM that the single combined
trampoline is easier to understand than the From class.

I am CC'ing Raymond so he can loop the author of Monocle (his
colleague) into this discussion -- I'd like to hear their opinion
(Monocle seems to require a decorator but it looks like it uses the
PEP 342 approach to simulate yield-from while avoiding stack buildup).

PS For Raymond and his colleague: I also still hope that Monocle will
change its mind about the proper convention for returning a value from
a coroutine -- I really think it should be raise StopIteration(value)
rather than yield value, since the latter (a) doesn't signal
sufficiently strongly (to the human reader) that the generator will
not be resumed ever again, and (b) could easily be confused with the
other uses of yield. (IOW I think yield is being overloaded too much
in Monocle.)

-- 
--Guido van Rossum (python.org/~guido)