[Python-ideas] Possible PEP 380 tweak

[Jacob Holm]

On 2010-10-28 00:52, Nick Coghlan wrote:
> On Thu, Oct 28, 2010 at 6:22 AM, Jacob Holm <jh at improva.dk> wrote:
>> Actually, AFAICT outer_broken will *not* give a RuntimeError on close()
>> after next().  This is due to the special-casing of GeneratorExit in PEP
>> 380.  That special-casing is also the basis for both my suggested
>> modifications.
> 
> Ah, you're quite right - I'd completely forgotten about the
> GeneratorExit special-casing in the PEP 380 semantics, so I was
> arguing from a faulty premise. With that error corrected, I can
> happily withdraw my objection to idioms that convert GeneratorExit to
> StopIteration (since any yield from expressions will reraise the
> GeneratorExit in that case).
> 

Looks like we are still not on exactly the same page though...  You seem
to be arguing from the version at
http://www.python.org/dev/peps/pep-0380, whereas I am looking at
http://mail.python.org/pipermail/python-ideas/attachments/20090419/c7d72ba8/attachment-0001.txt,
which is newer.



> The "did-it-really-finish?" question can likely be answered by
> slightly improving generator state introspection from the Python level
> (as I believe Guido suggested earlier in the thread). That way close()
> can keep the gist of its current semantics (return something if the
> generator ends up in an inactive state, raise RuntimeError if it
> yields another value), while frameworks can object to other unexpected
> states if they want to.
> 
> As it turns out, the information on generator state is already there,
> just not in a particularly user friendly format ("not started" =
> "g.gi_frame is not None and g.gi_frame.f_lasti == -1", "terminated" =
> "g.gi_frame is None").
> 
> So, without any modifications at all to the current incarnation of PEP
> 380, it is already possible to write:
> 
> def finish(gen):
>    frame = gen.gi_frame
>    if frame is None:
>        raise RuntimeError('finish() on exhausted/closed generator')
>    if frame.f_lasti == -1:
>        raise RuntimeError('finish() on not yet started generator')
>    try:
>        gen.throw(GeneratorExit)
>    except StopIteration as err:
>        if err.args:
>            return err.args[0]
>        return None
>    except GeneratorExit:
>        pass
>    else:
>        raise RuntimeError('Generator ignored GeneratorExit')
>    raise RuntimeError('Generator failed to return a value')
> 

Yes.  I forgot about the "not yet started" case in my earlier versions.



> I think I'm finally starting to understand *your* question/concern
> though. Given the current PEP 380 expansion, the above definition of
> finish() and the following two generators:
> 
> def g_inner():
>   yield
>   return "Hello world!"
> 
> def g_outer():
>   yield (yield from g_inner())
> 
> You would get the following result (as g_inner converts GeneratorExit
> to StopIteration, then yield from propogates that up the stack):
>>>> g = g_outer()
>>>> next(g)
>>>> finish(g)
> "Hello world!"
> 
> Oops?
> 

Well.  Not with the newest expansion.  Not that the None you will get
from that one is any better.


> I'm wondering if this part of the PEP 380 expansion:
>                     if _e is _x[1] or isinstance(_x[1], GeneratorExit):
>                         raise
> 
> Should actually look like:
>                     if _e is _x[1]:
>                         raise
>                     if isinstance(_x[1], GeneratorExit):
>                         raise GeneratorExit(*_e.args)
> 

In the newer expansion, I would change:

            except GeneratorExit as _e:
                try:
                    _m = getattr(_i, 'close')
                except AttributeError:
                    pass
                else:
                    _m()
                raise _e

Into:

            except GeneratorExit as _e:
                try:
                    _m = getattr(_i, 'close')
                except AttributeError:
                    pass
                else:
                    raise GeneratorExit(_m())
                raise _e

(Which can cleaned up a bit btw., by removing _e and using direct
attribute access instead of getattr)

> Once that distinction is made, you can more easily write helper
> functions and context managers that allow code to do the "right thing"
> according to the needs of a particular framework or application.
> 

Yes.  OTOH, I have argued for this change before with no luck.

- Jacob