On 18 November 2014 14:50, Guido van Rossum <guido@python.org> wrote:

I think we've gone through enough clarifications of the PEP now to be clear on the proposal. I saw in one of your earliest replies (right after Chris posted his first draft) that you're hesitant to support the PEP because of what would have to change to contextlib. What I couldn't quite read is whether you think that the proposal by itself is not an improvement, or whether you're just worried about compatibility.

I think it's an improvement - I really like the fact that it brings generators into line with your reasoning in the with statement PEP that flow control constructs should be locally visible. At the moment, "raise StopIteration" in a generator context is effectively non-local flow control, as it means any function call (explicit or implicit) or yield point may gracefully stop generator execution, rather than only return statements.

From the point of view of a generator *consumer*, I'm now happy that the backwards incompatibility is limited to cases where you throw in StopIteration and want to check if you got the same StopIteration instance back - you'll now also need to check for RuntimeError with __cause__ set to the StopIteration instance you threw in. You can construct scenarios where such a check will give a false positive, but they're getting seriously contrived at that point.

That's obscure enough that I think it's on par with other behavioural tweaks we've included in the "Porting to Python X.Y" guides in the past.
Apparently you know of a large group of users who use an older 3rd party version of contextlib, and for whom that older, 3rd party contextlib should keep working with future versions of Python 3 without updating their version of contextlib -- did I get that right? What exactly is the constraint there that makes their version of contextlib immutable even though the version of Python they are using may move forward?

I don't even remember how that came up now, but it was entirely hypothetical, and I think it can be ignored as a concern. As you say, being able to update to Python 3.5 without also being able to update to a new version of contextlib2 would just be weird.

Even if such a strange scenario did somehow come up, it would still be possible to switch to a conditional import where they used the stdlib version if available, and only fell back to contextlib2 on earlier versions of Python.
Separate from this special case, I am also worried about backward compatibility, and I have yet to get a good idea for how widespread code is that depends on StopIteration bubbling out from generators. I also don't have a good idea how often this issue bites users, but I have a feeling it does bite. E.g. this quote from c.l.py (https://mail.python.org/pipermail/python-list/2014-November/680775.html):

I did find it annoying occasionally that raising StopIteration inside a
generator expression conveys a different behavior than elsewhere. It did
take me quite a while to understand why that is so, but after that it
did not cause me much of a headache anymore.

One advantage of needing a __future__ import on the generator author side is that you always have the option of changing your mind, and *never* making the new behaviour the default.

That wouldn't be a *good* outcome, but I don't think it would be intolerable.

OTOH, I'm also not sure the status quo is sufficiently problematic to be worth changing. Yes, it's a little weird, but is it *that* much weirder than the unavoidable issues with exceptions thrown in __next__, __getitem__, __getattr__ and other special methods where a particular type of exception is handled directly by the interpreter?


Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia