
I have a strong suspicion that I'm missing something; I have been persuaded both directions too often to believe I have a grip on the real issue. So I'm putting out some assumptions; please tell me if I'm wrong, and maybe make them more explicit in the PEP. (1) The change will only affect situations where StopIteration is currently raised as an Exception -- i.e., it leaks past the bounds of a loop. (2) This can happen because of an explicit raise StopIteration. This is currently a supported idiom, and that is changing with PEP 479. (2a) Generators in the unwind path will now need to catch and reraise. (3) It can also happen because of an explicit next statement (as opposed the the implicit next of a loop). This is currently supported; after PEP 479, the next statement should be wrapped in a try statement, so that the intent will be explicit. (4) It can happen because of "yield from" yielding from an iterator, rather than a generator? (5) There is no other case where this can happen? (So the generator comprehension case won't matter unless it also includes one of the earlier cases.) -jJ

On Sun, Nov 30, 2014 at 1:04 PM, Jim J. Jewett <jimjjewett@gmail.com> wrote:
Where a StopIteration would come up out of the generator. Inside the generator function, it's exactly the same as it is in any other function; you can raise it, you can catch it, everything's normal.
(2) This can happen because of an explicit raise StopIteration. This is currently a supported idiom, and that is changing with PEP 479.
Correct. There is nothing that explicitly-raised StopIteration can do in 3.0-3.4 that a return statement can't do in 3.0-3.7. There is the downside that "raise StopIteration(value)" works on 2.7 where "return value" is a syntax error; the PEP currently has no solution for this.
(2a) Generators in the unwind path will now need to catch and reraise.
More likely, catch and return; if your code was allowing "next(iter)" to have the effect of potentially terminating the function, then you now have to spell that "try: next(iter); except StopIteration: return", which makes it clear that there's control flow here.
Correct, as per previous point. As you say, the intent will be explicit: take a value, and if there aren't any more, stop processing.
(4) It can happen because of "yield from" yielding from an iterator, rather than a generator?
No; as I understand it (though maybe I'm wrong too), "yield from" will yield every value the other iterator yields, and will then quietly emit a value if the iterator raises StopIteration, or will allow any other exception to propagate. The StopIteration coming from the iterator is absorbed by the "yield from" construct. To completely propagate it out, "return (yield from iter)" should cover all three results (yielded value, returned value, raised exception).
Correct. In a generator expression (I assume that's what you mean?), the most likely way to leak a StopIteration is the "or stop()" hack, which has always been at least slightly dubious, and is now going to be actively rejected. Control flow in a generator expression is now the same as in a comprehension, with no early-abort option; if you want that, the best way is to break the expression into an out-of-line generator function. This is now very similar to the restrictions on lambda; you can't (eg) raise exceptions in a lambda function, and if anyone comes to python-list asking how to do so, the best response is "use def instead of lambda". ChrisA

On 30 November 2014 at 12:31, Chris Angelico <rosuav@gmail.com> wrote:
Chris is correct here. The key yield from related change due to PEP 479 is to eliminate a subtle difference in the interaction between generators and StopIteration that currently exists when delegating to a subgenerator. If you throw StopIteration directly into a suspended generator, it propagates back out:
However, if the generator is instead suspended inside a *subgenerator*, then the thrown in exception will terminate the subgenerator, and execution of the delegating generator will resume:
Under PEP 479, the StopIteration thrown in StopIteration will be turned into RuntimeError regardless of whether or not a subgenerator is involved - the difference in the subgenerator case is that the transformation will happen when the innermost generator is terminated, and it will then appear as RuntimeError in the delegating generator. This change in semantics will make it possible to fix a latent defect in contextlib.contextmanager, where using a subgenerator as part of the context manager implementation may currently lead to inadvertently suppressing StopIteration in the body of a covered with statement. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Sun, Nov 30, 2014 at 1:04 PM, Jim J. Jewett <jimjjewett@gmail.com> wrote:
Where a StopIteration would come up out of the generator. Inside the generator function, it's exactly the same as it is in any other function; you can raise it, you can catch it, everything's normal.
(2) This can happen because of an explicit raise StopIteration. This is currently a supported idiom, and that is changing with PEP 479.
Correct. There is nothing that explicitly-raised StopIteration can do in 3.0-3.4 that a return statement can't do in 3.0-3.7. There is the downside that "raise StopIteration(value)" works on 2.7 where "return value" is a syntax error; the PEP currently has no solution for this.
(2a) Generators in the unwind path will now need to catch and reraise.
More likely, catch and return; if your code was allowing "next(iter)" to have the effect of potentially terminating the function, then you now have to spell that "try: next(iter); except StopIteration: return", which makes it clear that there's control flow here.
Correct, as per previous point. As you say, the intent will be explicit: take a value, and if there aren't any more, stop processing.
(4) It can happen because of "yield from" yielding from an iterator, rather than a generator?
No; as I understand it (though maybe I'm wrong too), "yield from" will yield every value the other iterator yields, and will then quietly emit a value if the iterator raises StopIteration, or will allow any other exception to propagate. The StopIteration coming from the iterator is absorbed by the "yield from" construct. To completely propagate it out, "return (yield from iter)" should cover all three results (yielded value, returned value, raised exception).
Correct. In a generator expression (I assume that's what you mean?), the most likely way to leak a StopIteration is the "or stop()" hack, which has always been at least slightly dubious, and is now going to be actively rejected. Control flow in a generator expression is now the same as in a comprehension, with no early-abort option; if you want that, the best way is to break the expression into an out-of-line generator function. This is now very similar to the restrictions on lambda; you can't (eg) raise exceptions in a lambda function, and if anyone comes to python-list asking how to do so, the best response is "use def instead of lambda". ChrisA

On 30 November 2014 at 12:31, Chris Angelico <rosuav@gmail.com> wrote:
Chris is correct here. The key yield from related change due to PEP 479 is to eliminate a subtle difference in the interaction between generators and StopIteration that currently exists when delegating to a subgenerator. If you throw StopIteration directly into a suspended generator, it propagates back out:
However, if the generator is instead suspended inside a *subgenerator*, then the thrown in exception will terminate the subgenerator, and execution of the delegating generator will resume:
Under PEP 479, the StopIteration thrown in StopIteration will be turned into RuntimeError regardless of whether or not a subgenerator is involved - the difference in the subgenerator case is that the transformation will happen when the innermost generator is terminated, and it will then appear as RuntimeError in the delegating generator. This change in semantics will make it possible to fix a latent defect in contextlib.contextmanager, where using a subgenerator as part of the context manager implementation may currently lead to inadvertently suppressing StopIteration in the body of a covered with statement. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (3)
-
Chris Angelico
-
Jim J. Jewett
-
Nick Coghlan