[Python-ideas] Possible PEP 380 tweak

[Nick Coghlan]

On Thu, Oct 28, 2010 at 6:22 AM, Jacob Holm <jh at improva.dk> wrote:
> Actually, AFAICT outer_broken will *not* give a RuntimeError on close()
> after next().  This is due to the special-casing of GeneratorExit in PEP
> 380.  That special-casing is also the basis for both my suggested
> modifications.

Ah, you're quite right - I'd completely forgotten about the
GeneratorExit special-casing in the PEP 380 semantics, so I was
arguing from a faulty premise. With that error corrected, I can
happily withdraw my objection to idioms that convert GeneratorExit to
StopIteration (since any yield from expressions will reraise the
GeneratorExit in that case).

The "did-it-really-finish?" question can likely be answered by
slightly improving generator state introspection from the Python level
(as I believe Guido suggested earlier in the thread). That way close()
can keep the gist of its current semantics (return something if the
generator ends up in an inactive state, raise RuntimeError if it
yields another value), while frameworks can object to other unexpected
states if they want to.

As it turns out, the information on generator state is already there,
just not in a particularly user friendly format ("not started" =
"g.gi_frame is not None and g.gi_frame.f_lasti == -1", "terminated" =
"g.gi_frame is None").

So, without any modifications at all to the current incarnation of PEP
380, it is already possible to write:

def finish(gen):
   frame = gen.gi_frame
   if frame is None:
       raise RuntimeError('finish() on exhausted/closed generator')
   if frame.f_lasti == -1:
       raise RuntimeError('finish() on not yet started generator')
   try:
       gen.throw(GeneratorExit)
   except StopIteration as err:
       if err.args:
           return err.args[0]
       return None
   except GeneratorExit:
       pass
   else:
       raise RuntimeError('Generator ignored GeneratorExit')
   raise RuntimeError('Generator failed to return a value')

I think I'm finally starting to understand *your* question/concern
though. Given the current PEP 380 expansion, the above definition of
finish() and the following two generators:

def g_inner():
  yield
  return "Hello world!"

def g_outer():
  yield (yield from g_inner())

You would get the following result (as g_inner converts GeneratorExit
to StopIteration, then yield from propogates that up the stack):
>>> g = g_outer()
>>> next(g)
>>> finish(g)
"Hello world!"

Oops?

I'm wondering if this part of the PEP 380 expansion:
                    if _e is _x[1] or isinstance(_x[1], GeneratorExit):
                        raise

Should actually look like:
                    if _e is _x[1]:
                        raise
                    if isinstance(_x[1], GeneratorExit):
                        raise GeneratorExit(*_e.args)

Once that distinction is made, you can more easily write helper
functions and context managers that allow code to do the "right thing"
according to the needs of a particular framework or application.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia