<div dir="ltr"><div><div class="gmail_extra"><div class="gmail_quote">On 15 November 2014 19:29, Chris Angelico <span dir="ltr"><<a href="mailto:rosuav@gmail.com" target="_blank">rosuav@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">PEP: 479<br>
Title: Change StopIteration handling inside generators<br>
Version: $Revision$<br>
Last-Modified: $Date$<br>
Author: Chris Angelico <<a href="mailto:rosuav@gmail.com">rosuav@gmail.com</a>><br>
Status: Draft<br>
Type: Standards Track<br>
Content-Type: text/x-rst<br>
Created: 15-Nov-2014<br>
Python-Version: 3.5<br>
Post-History: 15-Nov-2014<br>
<br>
<br>
Abstract<br>
========<br>
<br>
This PEP proposes a semantic change to ``StopIteration`` when raised<br>
inside a generator, unifying the behaviour of list comprehensions and<br>
generator expressions somewhat.<br>
<br>
<br>
Rationale<br>
=========<br>
<br>
The interaction of generators and ``StopIteration`` is currently<br>
somewhat surprising, and can conceal obscure bugs. An unexpected<br>
exception should not result in subtly altered behaviour, but should<br>
cause a noisy and easily-debugged traceback. Currently,<br>
``StopIteration`` can be absorbed by the generator construct.<br></blockquote><div><br></div><div>Thanks for the write-up!<br></div><div><br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Proposal<br>
========<br>
<br>
If a ``StopIteration`` is about to bubble out of a generator frame, it<br>
is replaced with some other exception (maybe ``RuntimeError``, maybe a<br>
new custom ``Exception`` subclass, but *not* deriving from<br>
``StopIteration``) which causes the ``next()`` call (which invoked the<br>
generator) to fail, passing that exception out. From then on it's<br>
just like any old exception. [3]_<br></blockquote><div><br></div><div>[snip]<br> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Alternate proposals<br>
===================<br>
<br>
Supplying a specific exception to raise on return<br>
-------------------------------------------------<br>
<br>
Nick Coghlan suggested a means of providing a specific<br>
``StopIteration`` instance to the generator; if any other instance of<br>
``StopIteration`` is raised, it is an error, but if that particular<br>
one is raised, the generator has properly completed.<br></blockquote><div><br></div><div>I think you can skip mentioning this particular idea in the PEP - I didn't like it even when I posted it, and both of Guido's ideas are much better :)<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Making return-triggered StopIterations obvious<br>
----------------------------------------------<br>
<br>
For certain situations, a simpler and fully backward-compatible<br>
solution may be sufficient: when a generator returns, instead of<br>
raising ``StopIteration``, it raises a specific subclass of<br>
``StopIteration`` which can then be detected. If it is not that<br>
subclass, it is an escaping exception rather than a return statement.<br></blockquote><br></div><div class="gmail_quote">There's an additional subtlety with this idea: if we add a new GeneratorReturn exception as a subclass of StopIteration, then generator iterators would likely also have to change to replace GeneratorReturn with a regular StopIteration (chaining appropriately via __cause__, and copying the return value across).<br></div><br><div class="gmail_quote">From the point of view of calling "next()" directly (rather than implicitly) this particular change makes it straightforward to distinguish between "the generator I called just finished" and "something inside the generator threw StopIteration". Due to the subclassing, implict next() invocations (e.g. in for loops, comprehensions, and container constructors) won't notice any difference.<br></div><br></div><div class="gmail_extra">With such a change, we would actually likely modify the following code in contextlib._GeneratorContextManager.__exit__:<br><br> try:<br> self.gen.throw(exc_type, value, traceback)<br> raise RuntimeError("generator didn't stop after throw()")<br> except StopIteration as exc:<br></div><div class="gmail_extra"> # Generator suppressed the exception<br></div><div class="gmail_extra"> # unless it's a StopIteration instance we threw in<br></div><div class="gmail_extra"> return exc is not value<br> except:<br> if sys.exc_info()[1] is not value:<br> raise<br><br></div><div class="gmail_extra">To be the slightly more self-explanatory:<br><br></div><div class="gmail_extra"> try:<br> self.gen.throw(type, value, traceback)<br> raise RuntimeError("generator didn't stop after throw()")<br> except GeneratorReturn:<br><div class="gmail_extra"> # Generator suppressed the exception<br></div> return True<br> except:<br> if sys.exc_info()[1] is not value:<br> raise<br><br></div><div class="gmail_extra">The current proposal in the PEP actually doesn't let us simplify this contextlib code, but rather means we would have to make it more complicated to impedance match generator semantics with the context management protocol. To handle that change, we'd have to make the code something like the following (for clarity, I've assumed a new RuntimeError subclass, rather than RuntimeError itself):<br><br> try:<br> self.gen.throw(exc_type, value, traceback)<br> raise RuntimeError("generator didn't stop after throw()")<br> except StopIteration as exc:<br><div class="gmail_extra"> # Could becomes "return True" once the __future__ becomes the default<br></div><div class="gmail_extra"> return exc is not value<br></div> except UnexpectedStopIteration as exc:<br> if exc.__cause__ is not value:<br> raise<br> except:<br> if sys.exc_info()[1] is not value:<br> raise<br></div><br></div>I definitely see value in adding a GeneratorReturn subclass to be able to tell the "returned" vs "raised StopIteration" cases apart from outside the generator (the current dance in contextlib only works because we have existing knowledge of the exact exception that was thrown in). I'm substantially less convinced of the benefit of changing generators to no longer suppress StopIteration. Yes, it's currently a rather odd corner case, but changing it *will* break code (at the very least, anyone using an old version of contextlib2, or who are otherwise relying on their own copy of contextlib rather than standard library one).<br><div><br><div class="gmail_extra">Cheers,<br>Nick.<br></div><div class="gmail_extra"></div><div class="gmail_extra"><br>-- <br><div class="gmail_signature">Nick Coghlan | <a href="mailto:ncoghlan@gmail.com">ncoghlan@gmail.com</a> | Brisbane, Australia</div>
</div></div></div>