[Python-ideas] Revised**11 PEP on Yield-From

[Jacob Holm]

Trying again, as the last version was mangled. (Thanks to Aahz for 
pointing that out).  I hope this is better...


Greg Ewing wrote:
 > Draft 12 of the PEP.
 >
 > Fixed a bug in the expansion (didn't handle StopIteration raised by
 > throw).
 >

Just so you know, I now agree that a long expansion with multiple
"try...except StopIteration" blocks is the right thing to do.  There
are only two cases I can see where it makes a difference compared to
what I suggested:

   1. Throwing StopIteration to an iterator without a throw() method.

      I would prefer treating this case *exactly* as if the iterator
      had a trivial throw method:

         def throw(self, et, ev=None, tb=None):
             raise et, ev, tb

      In other words, I think the StopIteration *should* be caught by
      yield-from.

      Treating it like this makes it easier to write wrappers that
      don't accidentally change the semantics of an obscure corner
      case.  Principle of least surprise and all that...

      It is easy enough to change the expansion to do this by expanding
      the try block around the throw() call or by actually using such a
      trivial throw method as a fallback.  Alternatively, the expansion
      can be rewritten using functools.partial as in the bottom of this
      mail.  It has identical semantics to draft 12 of the PEP, except
      for the handling of the missing throw method. I actually like
      that version because it is careful about what exceptions to
      catch, but still only has one "try...except StopIteration". YMMV.

   2. Calling an iterator.close() that raises a StopIteration.

      Arguably, such a close() is an error, so getting an exception in
      the caller is better than swallowing it and turning it into a
      normal return.  Especially since we only called close() as part
      of handling GeneratorExit in the first place.

An unrelated question...  What should happen with an iterator that has
a throw or close attribute that just happens to have the value None?
Should that be treated as an error because None is not callable, or
should it be treated as if the attribute wasn't there?  The expansion
handles it as if the attribute wasn't there, but IIRC your patch will
raise a TypeError trying to call None.

 > Removed paragraph about StopIteration left over from an earlier
 > version.
 >
 > Added some discussion about rejected ideas.
 >

Looks good, except...

 > Suggestion: If ``close()`` is not to return a value, then raise an
 > exception if StopIteration with a non-None value occurs.
 >
 > Resolution: No clear reason to do so. Ignoring a return value is not
 > considered an error anywhere else in Python.
 >

I may have been unclear about why I thought this should raise a
RuntimeError.  As I see it there are only two code patterns in a
generator that would have close() catch a StopIteration with a non-None
value.

    * An explicit catch of GeneratorExit followed by "return Value".

      This is harmless and potentially useful, although probably an
      abuse of GeneratorExit (that was one of the early arguments for
      not returning a value from close).  Not raising a RuntimeError in
      close makes it simpler to share a code path between the common
      and the forced exit.

    * An implicit catch of GeneratorExit, followed by "return Value".

      By an "implicit catch", I mean either a catch of "BaseException"
      or a "finally" clause.  In both cases, "return Value" will hide
      the original exception and that is almost certainly a bug.
      Raising a RuntimeError would let you discover this bug early.

The question now is whether it is better to catch n00b errors or to
allow careful programmers a bit more freedom in how they structure
their code.  When I started writing this mail I was leaning towards
catching errors, but I have now changed my mind.  I think giving more
power to experienced users is more important.

Best regards
- Jacob

------------------------------------------------------------------------

    _i = iter(EXPR)
    _p = partial(next, _i)
    while 1:
        try:
            _y = _p()
        except StopIteration as _e:
            _r = _e.value
            break
        try:
            _s = yield _y
        except GeneratorExit:
            _m = getattr(_i, 'close', None)
            if _m is not None:
                _m()
            raise
        except:
            _m = getattr(_i, 'throw', None)
            if _m is None:
                def _m(et, ev, tb):
                    raise et, ev, tb
            _p = partial(_m, *sys.exc_info())
        else:
            if _s is None:
                _p = partial(next, _i)
            else:
                _p = partial(_i.send, _s)
    RESULT = _r