[Python-ideas] Yield-From: Finalization guarantees

[Jacob Holm]

Nick Coghlan wrote:
> Jacob Holm wrote:
>   
>> Greg Ewing wrote:
>>     
>>> Jacob Holm wrote:
>>>
>>>       
>>>> will also remove some behavior that could have been useful, such as
>>>> the ability to suppress the GeneratorExit if you know what you are
>>>> doing.
>>>>         
>>> I'm not convinced there are any use cases for suppressing
>>> GeneratorExit in the first place. Can you provide an
>>> example that couldn't be easily done some other way?
>>>       
>> I don't have any real use cases, just a few examples of things you can
>> do in #2 that become a bit uglier in #3 or #4.
>>     
>
> You appear to be thinking of GeneratorExit as a way to ask a generator
> to finish normally such that it still makes sense to try to return a
> value after a GeneratorExit has been thrown in to the current frame, 
>   

Yes.  I am thinking that when using this for refactoring, there are 
likely to be cases where the closing generator needs to provide some 
final piece of information to its caller so that the caller can do *its* 
finalization.  Using return for that purpose has a number of control 
flow advantages.   If you insist we shouldn't use return for this, we 
should make close raise a RuntimeError like this:

def close(self):
    try:
        self.throw(GeneratorExit)
    except StopIteration, e:
        if e.value is not None:
            raise RuntimeError('generator responded to GeneratorExit by returning with a value')
    except GeneratorExit:
        pass
    else:
        raise RuntimeError('generator ignored GeneratorExit')


Of course I would prefer to use "return e.value" instead of the first 
RuntimeError, because that seems like the obvious thing to expect when 
you close a generator containing "try..except GeneratorExit: return 
value".   And once we have close returning a value, it would be nice to 
have access to that value in the context of the yield-from expression.  
Attaching it to the GeneratorExit (re)raised by yield-from seems like 
the only logical choice.  As my third code fragment showed, you could 
then explicitly recatch the GeneratorExit and get the value there.

> but that really isn't its role.
>
> Instead, it's more of an "Abandon Ship! Abandon Ship! All hands to the
> lifeboats!" indication that gives the generator a chance to release any
> resources it might be holding and bail out. 

That might be the prevailing wisdom concerning GeneratorExit, at least 
partly based on the fact that the only way to communicate anything 
useful out of a closing generator is to raise another exception.   
Thinking a bit about coroutines, it would be nice to use "send" for the 
normal communication and "close" to shut it down and getting a final 
result.  Example:

def averager():
    count = 0
    sum = 0
    while 1:
        try: 
            val = (yield)
        except GeneratorExit:
            return sum/count
        else:
            sum += val
            count += 1

avg = averager()
avg.next() # start coroutine
avg.send(1.0)
avg.send(2.0)
print avg.close()  # prints 1.5


To do something similar today requires either a custom exception, or the 
use of special values to tell the generator to yield the result.  I find 
this version a lot cleaner.

> The reason that close()
> accepts a StopIteration as well as a GeneratorExit is that the former
> still indicates that the generator has finalised itself, so the
> objective of calling close() has been achieved and there is no need to
> report an error.
>   

I have argued before that accepting StopIteration in close is likely to 
hide bugs in the closed generator, because the StopIteration may come 
from a return in a finally clause.  However, since we *are* accepting 
StopIteration we might as well make it useful.

> Any code that catches GeneratorExit without reraising it is highly
> suspect, just like code that suppresses SystemExit and KeyboardInterrupt.
>   

Explicitly catching GeneratorExit and then returning is a valid use 
today that I wouldn't consider suspect.  Catching GeneratorExit and then 
exiting the except block by other means than a raise or return is 
suspect, but has valid uses.

Best regards,

- Jacob