[Python-Dev] Revised**8 PEP on Yield-From

[Nick Coghlan]

Greg Ewing wrote:
> Here's a new draft of the PEP. I've added a Motivation
> section and removed any mention of inlining.

I like this version a lot better - reading the two examples on your site
helped as well.

> There is a new expansion that incorporates recent ideas,
> including the suggested handling of StopIteration raised
> by a throw() call (i.e. if it wasn't the one thrown in,
> treat it as a return value).

The spec for GeneratorExit handling means that the if statement around
the raise statement needs an extra condition: a thrown in GeneratorExit
should *always* be reraised, even if the subgenerator converts it to
StopIteration (which it is allowed to do by PEP 342 and the relevant
documentation).

> ------------------------------------------------------------
> In general, the semantics can be described in terms of the iterator
> protocol as follows:

"iterator protocol and generator API" or "generator protocol" (which is
a phrase you already use later in the PEP) would be more accurate, as
send() and throw() aren't part of the basic iterator protocol.

> Fine Details
> ------------
> 
> The implicit GeneratorExit resulting from closing the delegating
> generator is treated as though it were passed in using ``throw()``.
> An iterator having a ``throw()`` method is expected to recognize
> this as a request to finalize itself.
> 
> If a call to the iterator's ``throw()`` method raises a StopIteration
> exception, and it is *not* the same exception object that was thrown
> in, its value is returned as the value of the ``yield from`` expression
> and the delegating generator is resumed.

As mentioned above, I believe this should be overruled in the case of
GeneratorExit. Since correctly written generators are permitted to
convert GeneratorExit to StopIteration, the 'yield from' expression
should detect when that has happened and reraise the original exception.

> Finalization
> ------------
> 
> There was some debate as to whether explicitly finalizing the delegating
> generator by calling its ``close()`` method while it is suspended at a
> ``yield from`` should also finalize the subiterator. An argument against
> doing so is that it would result in premature finalization of the
> subiterator if references to it exist elsewhere.
> 
> Consideration of non-refcounting Python implementations led to the
> decision that this explicit finalization should be performed, so that
> explicitly closing a factored generator has the same effect as doing
> so to an unfactored one in all Python implementations.
> 
> The assumption made is that, in the majority of use cases, the subiterator
> will not be shared. The rare case of a shared subiterator can be
> accommodated by means of a wrapper that blocks ``throw()`` and ``send()``
> calls, or by using a means other than ``yield from`` to call the
> subiterator.

With the currently semantics (calling close() if throw() isn't
available), it is also necessary to block close() in order to share an
iterator. Given the conclusion that shared iterators are actually better
handled by looping or explicit next() calls, I'm actually OK with that -
really focusing 'yield from' specifically on the ability to factor
monolithic generator functions into smaller components is probably a
good idea, since mere iteration is already easy.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------