On 12/11/2014 10:14 AM, Oscar Benjamin wrote:
On 10 December 2014 at 20:46, Guido van Rossum
wrote:
The PEP is still very confused. Primarily the fact is that the PEP is attempting to address an issue which affects all iterators but proposing a solution which is only about generators. The PEP seems to suggest that there is something special about generators in this respect when there really isn't. For example:
""" The interaction of generators and StopIteration is currently somewhat surprising, and can conceal obscure bugs. An unexpected exception should not result in subtly altered behaviour, but should cause a noisy and easily-debugged traceback. Currently, StopIteration can be absorbed by the generator construct. """
There is no interaction between generators and StopIteration. The issue isn't about generators it is about the iterator protocol. StopIteration cannot be absorbed by the generator construct.
What is very confusing is the ambiguous use of 'generator' to mean either 'generator function' or 'generator class instance'. Since I do not know what either of you mean by 'generator' in each instance above, I do not know exactly what either of you meant. If the PEP and discussants consistency used 'generator' to mean "instance of the internal generator class, initialized with a suspended instance of a *user-written* generator function body", then I think some of the confusion would dissipate. The PEP is about the conjunction of the following facts (and a few more, but I think these are the most salient): a) generator function bodies follow a different protocol than iterator __next__ methods, one that does not require them to *ever* raise StopIteration; b) if a g.f. body does raise StopIteration, it might be a substitute for 'return', but it might be a bug -- and apparently bugs do happen in real code; and c) generator.__next__ currently trusts that *user-written* g.f.s are never buggy. The PEP proposal is, either literally or in effect, that generator.__next should stop trusting StopIteration from a g.f., thereby disallowing the sometimes convenient but optional substitution. The justification is that bugs should not pass silently, and now they do.
I think the PEP would be clearer if it properly acknowledged that the problem is a problem for all iterators. The question then is why the fix is only targeted at generators and what should be done about the same problem that occurs in many other forms. The PEP rationale avoids these issues by falsely claiming that generators are special.
Guido today add the following, apparently in response to the above. +When implementing a regular ``__next__()`` method, the only way to +indicate the end of the iteration is to raise ``StopIteration``. So +catching ``StopIteration`` here and converting it to ``RuntimeError`` +would defeat the purpose. This is a reminder of the special status of +generator functions: in a generator function, raising +``StopIteration`` is redundant since the iteration can be terminated +by a simple ``return``. I would put this answer slightly differently. A __next__ method is supposed to raise StopIteration if *and only if* there are no more items to return. The __next__ method author is responsible for fulfilling the contract. Core developers are responsible for generator.__next__; we are not responsible for iterators that others write. Anyone who writes an iterator class whose instances are initialized with 3rd party non-iterator code that is executed in .__next__, should think about what to do if that code raises StopIteration. It would be possible for the PEP to recommend that other .__next__ methods executing external code might do something similar to what generator.__next__ will do. try: execute(self.external_code) except StopIteration as e: raise RuntimeError(<whatever>) -- Terry Jan Reedy