[Python-ideas] Possible PEP 380 tweak

[Nick Coghlan]

On Sat, Oct 30, 2010 at 1:47 PM, Guido van Rossum <guido at python.org> wrote:
> I agree that you've poked a hole in my proposal. If we can change the
> expansion of yield-from to restore the equivalency between gtally()
> and the simplest gtally2(), thereby restoring the original refactoring
> principle, we might be able to save it. Otherwise I declare defeat.
> Right now I am too tired to think of such an expansion, but I recall
> trying my hand at one a few nights ago and realizing that I'd
> introduced another problem. So this does not look too hopeful,
> especially since I really don't like extending GeneratorExit for the
> purpose.

I tried to make the original version work as well, but always ran into
one of two problems:
- breaking GeneratorExit for resource cleanup
- "leaking" inner return values so they looked like they came from the
outer function.

Here's a crazy idea though. What if gtally2() could be written as follows:

def gtally2():
  return from gtally()

If we make the generator tail call explicit, then the interpreter can
do the right thing (i.e. raise StopIteration-with-a-value instead of
reraising GeneratorExit) and we don't need to try to shoehorn two
different sets of semantics into the single yield-from construct.

To give some formal semantics to the new statement:

    # RETURN FROM semantics
    _i = iter(EXPR)
    _m, _a = next, (_i,)
    # _m is a function or a bound method;
    #  _a is a tuple of arguments to call _m with;
    # both are set to other values further down
    while 1:
        # Move the generator along
        # Unlike YIELD FROM, we allow StopIteration to
        # escape, since this is a tail call
        _y = _m(*_a)

        # Yield _y and process what came back in
        try:
            _s = yield _y
        except GeneratorExit as _e:
            # Request to exit
            try:
                # Don't reuse _m, since we're bailing out of the loop
                _close = _i.close
            except AttributeError:
                pass
            else:
                # Unlike YIELD FROM, we use StopIteration
                # to return the value of the inner close() call
                raise StopIteration(_close())
            # If there is no inner close() attribute, return None
            raise StopIteration
        except BaseException as _e:
            # An exception was thrown in; pass it along
            _a = sys.exc_info()
            try:
                _m = _i.throw
            except AttributeError:
                # Can't throw it in; throw it back out
                raise _e
        else:
            # A value was sent in; pass it along
            if _s is None:
                _m, _a = next, (_i,)
            else:
                _m, _a = _i.send, (_s,)
    # Unlike YIELD FROM, this is a statement, so there is no RESULT

Summary of the differences between return from and yield from:
- statement, not an expression
- an inner StopIteration is allowed to propogate
- a thrown in GeneratorExit is converted to StopIteration

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia