[Python-ideas] Revised**11 PEP on Yield-From

[Jacob Holm]

Greg Ewing wrote:
> Draft 12 of the PEP.
>
> Fixed a bug in the expansion (didn't handle
> StopIteration raised by throw).
>

Just so you know, I now agree that a long expansion with multiple 
"try...except StopIteration" blocks is the right thing to do.  There are 
only two cases I can see where it makes a difference compared to what I 
suggested:

   1. Throwing StopIteration to an iterator without a throw() method.  I
      would prefer treating this case *exactly* as if the iterator had a
      trivial throw method:  "def throw(self, et, ev=None, tb=None):
      raise et, ev, tb".  Meaning that the StopIteration *should* be
      caught by yield-from.  Treating it like this makes it easier to
      write wrappers that don't accidentally change the semantics of an
      obscure corner case.  Principle of least surprise and all that... 
      It is easy enough to change the expansion to do this by expanding
      the try block around the throw() call or by actually using such a
      trivial throw method as a fallback.  Alternatively, the expansion
      can be rewritten using functools.partial as in the bottom of this
      mail.  It has identical semantics to draft 12 of the PEP, except
      for the handling of the missing throw method. I actually like that
      version because it is careful about what exceptions to catch, but
      still only has one "try...except StopIteration". YMMV.
   2. Calling an iterator.close() that raises a StopIteration. 
      Arguably, such a close() is an error, so getting an exception in
      the caller is better than swallowing it and turning it into a
      normal return.  Especially since we only called close() as part of
      handling GeneratorExit in the first place.

An unrelated question...  What should happen with an iterator that has a 
throw or close attribute that just happens to have the value None?  
Should that be treated as an error because None is not callable, or 
should it be treated as if the attribute wasn't there?  The expansion 
handles it as if the attribute wasn't there, but IIRC your patch will 
raise a TypeError trying to call None.

> Removed paragraph about StopIteration left over
> from an earlier version.
>
> Added some discussion about rejected ideas.
>

Looks good, except...

> Suggestion: If ``close()`` is not to return a value, then raise an
> exception if StopIteration with a non-None value occurs.
>
> Resolution: No clear reason to do so. Ignoring a return value is not
> considered an error anywhere else in Python.
>   

I may have been unclear about why I thought this should raise a 
RuntimeError.  As I see it there are only two code patterns in a 
generator that would have close() catch a StopIteration with a non-None 
value.

    * An explicit catch of GeneratorExit followed by "return Value". 
      This is harmless and potentially useful, although probably an
      abuse of GeneratorExit (that was one of the early arguments for
      not returning a value from close).  Not raising a RuntimeError in
      close makes it simpler to share a code path between the common and
      the forced exit.
    * An implicit catch of GeneratorExit, followed by "return Value". 
      By an "implicit catch", I mean either a catch of "BaseException"
      or a "finally" clause.  In both cases, "return Value" will hide
      the original exception and that is almost certainly a bug. 
      Raising a RuntimeError would let you discover this bug early.

The question now is whether it is better to catch n00b errors or to 
allow careful programmers a bit more freedom in how they structure their 
code.  When I started writing this mail I was leaning towards catching 
errors, but I have now changed my mind.  I think giving more power to 
experienced users is more important.

Best regards
- Jacob

------------------------------------------------------------------------

    _i = iter(EXPR)
    _p = partial(next, _i)
    while 1:
        try:
            _y = _p()
        except StopIteration as _e:
            _r = _e.value
            break
        try:
            _s = yield _y
        except GeneratorExit:
            _m = getattr(_i, 'close', None)
            if _m is not None:
                _m()
            raise
        except:
            _m = getattr(_i, 'throw', None)
            if _m is None:
                def _m(et, ev, tb):
                    raise et, ev, tb
            _p = partial(_m, *sys.exc_info())
        else:
            if _s is None:
                _p = partial(next, _i)
            else:
                _p = partial(_i.send, _s)
    RESULT = _r