[Python-Dev] PEP 380 (yield from a subgenerator) comments

[Guido van Rossum]

On Thu, Mar 26, 2009 at 10:07 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 09:24 PM 3/25/2009 -0700, Guido van Rossum wrote:
>> ISTR that the motivation for adding new syntax is that the best you
>> can do using a trampoline library is still pretty cumbersome to use
>> when you have to write a lot of tasks and subtasks, and when using
>> tasks is just a tool for getting things done rather than an end goal
>> in itself. I agree that the motivation and the comparison should be
>> added to the PEP (perhaps moving the trampoline sample
>> *implementation* to a reference or an appendix, since it is only the
>> appearance of the trampoline-*using* code that matters).
>
> In particular, it should explain why these choices are so costly as to
> justify new syntax and a complex implementation:
>
> * decorator clearly identifying intent to create a task vs. no indication of
> task-ness
> * "yield Return(value)" vs. "return value"
> * "result = yield subgenerator()" vs. "result = yield from subgenerator()"
>
> Especially since the first two differences arguably make the non-"yield
> from" code clearer, and the third makes it more compact.
>
>
>> with judicious use of decorators and helper classes you can get a
>> reasonable approximation, and I think Twisted uses something like
>> this, so it's not just theory. I think the best you can do without new
>> syntax though is still pretty cumbersome and brittle, which is why I
>> have encouraged your PEP.
>
> On the "cumbersome" side, there are only three differences, as I've shown
> above... and one of them uses less syntax than the PEP.

I wasn't talking about keystrokes so much as to having to remember to
use thes incantations and understand what they do and what their
limitations are.

> I'm not sure what you mean by brittle.  Twisted and PEAK have both had
> generator-based tasks for ages, and have been used in production for years,
> even before generators had send()/throw() -- the addition of yield
> expressions made it possible to get rid of the one previous brittleness,
> where you needed to do things like:
>
>      yield subgenerator(); result = resume()
>
> in order to pass values or exceptions in.  Since send()/throw() was added,
> the need to call a function after each yield was eliminated, at least from
> PEAK and my newer Trellis library; haven't looked at Twisted's tasks in a
> while.  I believe there are other generator-based task libraries available
> on PyPI, but can't comment on their robustness.

(Well here is Greg's requested use case for .send(). :-)

By brittle I meant again having to be aware of those details of the
mechanism that exist because of syntactic limitations, e.g.
accidentally writing "return X" instead of "yield Return(X)".

> (The existence of throw(), by the way, makes it possible to produce
> tracebacks that look *exactly* as if you had called a series of functions,
> rather than iterating over a bunch of generators.  The sample code I gave
> should do this correctly, as it was created by yanking out working, tested
> code from my Trellis library, and cutting out all the Trellis-specific
> bits.)
>
> If someone can find any semantic differences between the code I posted and
> the yield-from proposal (apart from the absence of the "for x in y: yield x"
> part of the functionality), I'd like to know about it...  and it should go
> in the PEP.

I'll leave this up to Greg.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)