[Python-ideas] Generators are iterators
tjreedy at udel.edu
Fri Dec 12 05:44:45 CET 2014
On 12/11/2014 10:14 AM, Oscar Benjamin wrote:
> On 10 December 2014 at 20:46, Guido van Rossum <guido at python.org> wrote:
> The PEP is still very confused. Primarily the fact is that the PEP is
> attempting to address an issue which affects all iterators but
> proposing a solution which is only about generators. The PEP seems to
> suggest that there is something special about generators in this
> respect when there really isn't. For example:
> The interaction of generators and StopIteration is currently somewhat
> surprising, and can conceal obscure bugs. An unexpected exception
> should not result in subtly altered behaviour, but should cause a
> noisy and easily-debugged traceback. Currently, StopIteration can be
> absorbed by the generator construct.
> There is no interaction between generators and StopIteration. The
> issue isn't about generators it is about the iterator protocol.
> StopIteration cannot be absorbed by the generator construct.
What is very confusing is the ambiguous use of 'generator' to mean
either 'generator function' or 'generator class instance'. Since I do
not know what either of you mean by 'generator' in each instance above,
I do not know exactly what either of you meant.
If the PEP and discussants consistency used 'generator' to mean
"instance of the internal generator class, initialized with a suspended
instance of a *user-written* generator function body", then I think some
of the confusion would dissipate.
The PEP is about the conjunction of the following facts (and a few more,
but I think these are the most salient): a) generator function bodies
follow a different protocol than iterator __next__ methods, one that
does not require them to *ever* raise StopIteration; b) if a g.f. body
does raise StopIteration, it might be a substitute for 'return', but it
might be a bug -- and apparently bugs do happen in real code; and c)
generator.__next__ currently trusts that *user-written* g.f.s are never
The PEP proposal is, either literally or in effect, that
generator.__next should stop trusting StopIteration from a g.f., thereby
disallowing the sometimes convenient but optional substitution. The
justification is that bugs should not pass silently, and now they do.
> I think the PEP would be clearer if it properly acknowledged that the
> problem is a problem for all iterators. The question then is why the
> fix is only targeted at generators and what should be done about the
> same problem that occurs in many other forms. The PEP rationale avoids
> these issues by falsely claiming that generators are special.
Guido today add the following, apparently in response to the above.
+When implementing a regular ``__next__()`` method, the only way to
+indicate the end of the iteration is to raise ``StopIteration``. So
+catching ``StopIteration`` here and converting it to ``RuntimeError``
+would defeat the purpose. This is a reminder of the special status of
+generator functions: in a generator function, raising
+``StopIteration`` is redundant since the iteration can be terminated
+by a simple ``return``.
I would put this answer slightly differently.
A __next__ method is supposed to raise StopIteration if *and only if*
there are no more items to return. The __next__ method author is
responsible for fulfilling the contract. Core developers are
responsible for generator.__next__; we are not responsible for iterators
that others write. Anyone who writes an iterator class whose instances
are initialized with 3rd party non-iterator code that is executed in
.__next__, should think about what to do if that code raises StopIteration.
It would be possible for the PEP to recommend that other .__next__
methods executing external code might do something similar to what
generator.__next__ will do.
except StopIteration as e:
Terry Jan Reedy
More information about the Python-ideas