[Python-ideas] Possible PEP 380 tweak

Guido van Rossum guido at python.org
Tue Oct 26 18:56:41 CEST 2010

On Tue, Oct 26, 2010 at 3:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Tue, Oct 26, 2010 at 1:14 PM, Guido van Rossum <guido at python.org> wrote:
>> On Mon, Oct 25, 2010 at 6:35 PM, Jacob Holm <jh at improva.dk> wrote:
>>> Throwing and catching GeneratorExit is not common, and according to some
>>> shouldn't be used for this purpose at all.
>> Well, *throwing* it is close()'s job. And *catching* it ought to be
>> pretty rare. Maybe this idiom would be better:
>> def sum():
>>  total = 0
>>  try:
>>    while True:
>>      value = yield
>>      total += value
>>  finally:
>>    return total

> Rereading my previous post that Jacob linked, I'm still a little
> uncomfortable with the idea of people deliberately catching
> GeneratorExit to turn it into a normal value return to be reported by
> close(). That said, I'm even less comfortable with the idea of
> encouraging the moral equivalent of a bare except clause :)

My bad. I should have stopped at "except GeneratorExit: return total".

> I see two realistic options here:
> 1. Use GeneratorExit for this, have g.close() return a value and I
> (and others that agree with me) just get the heck over it.

This is still my preferred option.

> 2. Add a new GeneratorReturn exception and a new g.finish() method
> that follows the same basic algorithm Guido suggested, only with a
> different exception type:
> class GeneratorReturn(Exception): # Note: ordinary exception, unlike
> GeneratorExit
>  pass
> def finish(gen):
>  try:
>   gen.throw(GeneratorReturn)
>   raise RuntimeError("Generator ignored GeneratorReturn")
>  except StopIteration as err:
>   if err.args:
>     return err.args[0]
>  except GeneratorReturn:
>   pass
>  return None

IMO there are already too many special exceptions and methods.

> (Why "finish" as the suggested name for the method? I'd prefer
> "return", but that's a keyword and "return_" is somewhat ugly. Pairing
> GeneratorReturn with finish() is my second choice, for the "OK, time
> to wrap things up and complete your assigned task" connotations, as
> compared to the "drop everything and clean up the mess" connotations
> of GeneratorExit and close())
> I'd personally be +1 on option 2 (since it addresses the immediate use
> case while maintaining appropriate separation of concerns between
> guaranteed resource cleanup and graceful completion of coroutines) and
> -0 on option 1 (unsurprising, given my previously stated objections to
> failing to maintain appropriate separation of concerns).

Hm, I guess I'm more in favor of minimal mechanism. The clincher for
me is pretty much that the extended g.close() semantics are a very
simple mod to the existing gen_close() function in genobject.c -- it
currently always returns None but could very easily be changed to
extract the return value from err.args when it catches StopIteration
(but not GeneratorExit).

it also looks like my proposal doesn't get in the way of anything --
if the generator doesn't catch GeneratorExit g.close() will return
None, and if the caller of g.close() doesn't expect a value, they can
just ignore it.

Finally note that this still looks like a relatively esoteric use
case: when using "var = yield from generator()" the the return value
from the generator (written as "return X" and implemented as "raise
StopIteration(X)") will automatically be delivered to var, and there's
no need to call g.close(). In this case there is also no reason for
the generator to catch GeneratorExit -- that is purely needed for the
idiom of writing "inside-out iterators" using this pattern in the
generator (as I mentioned on the parent thread):

    while True:
      value = yield
      <use value>
  except GeneratorExit:
    raise StopIteration(<result>)  # Or "return <result>" in PEP 380 syntax

Now, if I may temporarily go into wild-and-crazy mode (this *is*
python-ideas after all :-), we could invent some ad-hoc syntax for
this pattern, e.g.:

  for value in yield:
    <use value>
  return <result>

IOW the special form:

  for <var> in yield:

would translate into:

    while True:
      <var> = yield
  except GeneratorExit:

If (and this is a big if) the
while-True-yield-inside-try-except-GeneratorExit pattern somehow
becomes popular we could reconsider this syntactic extension or some
variant. (I have to add that the syntactic ice is a bit thin here,
since "for <var> in (yield)" already has a meaning, and a totally
different one of course. A variant could be "for <var> from yield" or
some other abuse of keywords.

But let me stop here before people think I've just volunteered my
retirement... :-)

> (I should note that this differs from the previous suggestion of a
> GeneratorReturn exception in the context of PEP 380. Those suggestions
> were to use it as a replacement for StopIteration when a generator
> contained a return statement. The suggestion here is to instead use it
> as a replacement for GeneratorExit in order to request
> prompt-but-graceful completion of a generator rather than just bailing
> out immediately).


--Guido van Rossum (python.org/~guido)

More information about the Python-ideas mailing list