On 16 November 2014 00:37, Chris Angelico <rosuav@gmail.com> wrote:

On Sun, Nov 16, 2014 at 1:13 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
>> For certain situations, a simpler and fully backward-compatible
>> solution may be sufficient: when a generator returns, instead of
>> raising ``StopIteration``, it raises a specific subclass of
>> ``StopIteration`` which can then be detected. If it is not that
>> subclass, it is an escaping exception rather than a return statement.
>
> There's an additional subtlety with this idea: if we add a new
> GeneratorReturn exception as a subclass of StopIteration, then generator
> iterators would likely also have to change to replace GeneratorReturn with a
> regular StopIteration (chaining appropriately via __cause__, and copying the
> return value across).

Would have to do so automatically, meaning this is no simpler than the
current proposal? Or would have to be always explicitly written to
handle it?

When GeneratorReturn escaped a generator frame, the interpreter would automatically convert it into an ordinary StopIteration instance.

It's still simpler because it won't need the __future__ dance (as it doesn't involve any backwards incompatible changes).

> I definitely see value in adding a GeneratorReturn subclass to be able to
> tell the "returned" vs "raised StopIteration" cases apart from outside the
> generator (the current dance in contextlib only works because we have
> existing knowledge of the exact exception that was thrown in). I'm
> substantially less convinced of the benefit of changing generators to no
> longer suppress StopIteration. Yes, it's currently a rather odd corner case,
> but changing it *will* break code (at the very least, anyone using an old
> version of contextlib2, or who are otherwise relying on their own copy of
> contextlib rather than standard library one).

This is why it's proposed to use __future__ to protect it.

Using __future__ still imposes a large cost on the community - docs need updating, code that relies on the existing behaviour has to be changed, developers need to adjust their mental models of how the language works.

There needs to be a practical payoff for those costs - and at the moment, it's looking like we can actually get a reasonably large fraction of the gain without most of the pain by instead pursuing Guido's idea of a separate StopIteration subclass to distinguish returning from the outermost generator frame from raising StopIteration elsewhere in the generator.

If anyone's
still using an old version of contextlib2 once 3.7 comes along, it'll
break; but is there any reason to use Python 3.7 with a contextlib
from elsewhere than its standard library?

Same reason folks use it now: consistent behaviour and features across a range of Python versions.

However, that's not the key point - the key point is that working through the exact changes that would need to be made in contextlib persuaded me that I was wrong when I concluded that contextlib wouldn't be negatively affected.

It's not much more complicated, but if we can find a fully supported example like that in the standard library, what other things might folks be doing with generators that *don't* fall into the category of "overly clever code that we don't mind breaking"?

(I'm not familiar with
contextlib2 or what it offers.)

contexlib2 ~= 3.3 era contextlib that runs as far back as 2.6 (I initially created it as a proving ground for the idea that eventually become contextlib.ExitStack).

Cheers,
Nick.

Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia