[Python-Dev] PEP 479: Change StopIteration handling inside generators

[Nick Coghlan]

On 27 November 2014 at 09:50, Guido van Rossum <guido at python.org> wrote:
> On Wed, Nov 26, 2014 at 3:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> This is actually the second iteration of this bug: the original
>> implementation *always* suppressed StopIteration. PJE caught that one before
>> Python 2.5 was released, but we didn't notice that 3.3 had brought it back
>> in a new, more subtle form :(
>>
>> It's worth noting that my "allow_implicit_stop" idea in the other thread
>> wouldn't affect subgenerators - those would still convert StopIteration to
>> RuntimeError unless explicitly silenced.
>
> You've lost me in this subthread. Am I right to conclude that the PEP change
> doesn't cause problems for contextlib(*), but that the PEP change also
> probably wouldn't have helped diagnose any contextlib bugs?

I think the PEP 479 semantics would have made both bugs (the one PJE
found in 2.5, and the newer one Isaac pointed out here) less cryptic,
in that they would have caused RuntimeError to be raised, rather than
silently consuming the StopIteration and continuing execution after
the with statement body.

With the new semantics, contextlib just needs to be updated to cope
with the StopIteration -> RuntimeError conversion, and Isaac's
"spurious success" bug will be fixed*.

Without PEP 479, I believe my only recourse to systematically
eliminate the risk of generator based context managers silently
consuming StopIteration would be to implement the "StopIteration ->
gen.close()" workaround, and that would be a backwards incompatible
change in its own right.

Cheers,
Nick.

P.S. *(This does mean I was wrong about allow_implicit_stop being
useful to contextlib, but I still think the decorator is useful for
cases where StopIteration is being used to terminate the generator on
purpose)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia