[Python-Dev] PEP 479: Change StopIteration handling inside generators

[Isaac Schwabacher]

On 14-11-26, Guido van Rossum  wrote:
> On Wed, Nov 26, 2014 at 8:54 AM, Paul Moore wrote:
> 
> > On 26 November 2014 at 16:24, Isaac Schwabacher wrote:
> > > This actually leads to a good example of why the PEP is necessary:
> > [...]
> > 
> > Oh! If that's the current behaviour, then it probably needs to go into
> > the PEP as a motivating example. It's far more convincing than most of
> > the other arguments I've seen. Just one proviso - is it fixable in
> > contextlib *without* a language change? If so, then it loses a lot of
> > its value.
> 
> It's hard to use as an example because the behavior of contextlib is an integral part of it -- currently for me the example boils down to "there is a bug in contextlib". Maybe it would have been caught earlier with the change in the PEP, but when using it as a motivating example you have to show the code containing the bug, not just a demonstration.

How is this a bug in contextlib? The example behaves the way it does because gen.throw(StopIteration) behaves differently depending on whether gen is paused at a yield or a yield from. What *should* contextlib.contextmanager do in this instance? It has faithfully forwarded the StopIteration raised in the protected block to the generator, and the generator has forwarded this to the subgenerator, which has elected to fail and report success. The bug is in the subgenerator, because it fails to treat StopIteration as an error. But the subgenerator can't in general be converted to treat StopIteration as an error, because clearly it's used in other places than as a nested context manager (otherwise, it would itself be decorated with @contextlib.contextmanager and accessed as such, instead of yielded from). And in those places, perhaps it needs to simply allow StopIteration to bubble up. And can we factor out the error checking so that we don't have to duplicate subgenerator? Well... yes, but it's tricky because we'll introduce an extra yield from in the process, so we have to put the handling in the subgenerator itself and wrap the *non*-context-manager uses.

ijs

> If you want to try though, I'm happy to entertain a pull request for the PEP.