[Python-ideas] Possible PEP 380 tweak

Nick Coghlan ncoghlan at gmail.com
Wed Oct 27 00:14:14 CEST 2010

On Wed, Oct 27, 2010 at 3:33 AM, Guido van Rossum <guido at python.org> wrote:
> On Tue, Oct 26, 2010 at 7:44 AM, Jacob Holm <jh at improva.dk> wrote:
>> A different way to handle this would be to change the PEP 380 expansion
>> as follows:
>> [...]
>> - except GeneratorExit as _e:
>> + except (GeneratorReturn, GeneratorExit) as _e:
>> [...]
> That just strikes me as one more reason why a separate GeneratorReturn
> is a bad idea.
> In my ideal world, you almost never need to catch or raise
> StopIteration; you don't raise GeneratorExit (that is close()'s job)
> but you catch it to notice that your data source is finished, and then
> you return a value. (And see my crazy idea in my previous post to get
> rid of that too. :-)

Jacob's "implications for PEP 380" exploration started to give me some
doubts, but I think there are actually some flaws in his argument.
Accordingly, I would like to make one more attempt at explaining why I
think throwing in a separate exception for this use case is valuable
(and *doesn't* require any changes to PEP 380).

As I see it, there's a bit of a disconnect between many PEP 380 use
cases and any mechanism or idiom which translates a thrown in
exception into an ordinary StopIteration. If you expect your thrown in
exception to always terminate the generator in some fashion, adopting
the latter idiom in your generator will make it potentially unsafe to
use in a "yield from" expression that isn't the very last yield
operation in any outer generator.

Consider the following:

def example(arg):
    yield arg
  except GeneratorExit
    return "Closed"
  return "Finished"

def outer_ok1(arg):  # close() after next() returns "Closed"
  return yield from example(arg)

def outer_ok2(arg): # close() after next() returns None
  yield from example(arg)

def outer_broken(arg): # close() after next() gives RuntimeError
  val = yield from example(arg)
  yield val

# All 3 cases: close() before next() returns None
# All 3 cases: close() after 2x next() returns None

Using close() to say "give me your return value" creates the risk of
hitting those runtime errors in a generator's __del__ method, and
exceptions in __del__ are always a bit ugly.

Keeping the "give me your return value" and "clean up your resources"
concerns separate by adding a new method and thrown exception means
that close() is less likely to unpredictably raise RuntimeError (and
when it does, will reliably indicate a genuine bug in a generator
somewhere that is suppressing GeneratorExit).

As far as PEP 380's semantics go, I think it should ignore the
existence of anything like GeneratorReturn completely. Either one of
the generators in the chain will catch the exception and turn it into
StopIteration, or they won't. If they convert it to StopIteration, and
they aren't the last generator in the chain, then maybe what actually
needs to happen at the outermost level is something like this:

class GeneratorReturn(Exception): pass

def finish(gen):
    gen.throw(GeneratorReturn) # Ask generator to wrap things up
  except StopIteration as err:
    if err.args:
      return err.args[0]
  except GeneratorReturn:
    # Asking nicely didn't work, so force resource cleanup
    # and treat the result as if the generator had already
    # been exhausted or hadn't started yet
  return None


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-ideas mailing list