[Python-ideas] Generators are iterators
Oscar Benjamin
oscar.j.benjamin at gmail.com
Fri Dec 12 22:10:00 CET 2014
On 12 December 2014 at 20:06, Terry Reedy <tjreedy at udel.edu> wrote:
>
>
> On 12/12/2014 7:42 AM, Oscar Benjamin wrote:> On 12 December 2014 at 10:34,
> Nick Coghlan <ncoghlan-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> wrote:
>
>>> specific problem deemed worthy of being fixed is that the presence of
>>> "yield" in a function body can implicitly suppress StopIteration
>>> exceptions raised elsewhere in that function body (or in functions it
>>> calls).
>
> This makes no sense to me either.
I'm wondering if Nick is referring to two different things at once: On
the one hand "yield from" which is an iterator-consumer suppresses
StopIteration from a child iterator and on the other hand every
generator interacts with a parent iterator-consumer that will catch
any StopIteration it raises. When generator function f "yield from"s
to generator function g both things occur in tandem in two different
frames.
<snip>
>
>> The problem (insofar as it is) is a problem for all
>> iterators since all iterators interact with iterator-consumers and it
>> is the iterator-consumer that catches the StopIteration. Here is an
>> example using map:
>>
>>>>> def func(x):
>> ... if x < 0:
>> ... raise StopIteration
>> ... return x ** 2
>
> This function violates the implicit guideline, which I think should be more
> explicit in the doc, that the only functions that should expose
> StopIteration to the outside world are iterator.__next__ functions and the
> corresponding builtin next(). It is not surprising that this violation
> causes map to misbehave. However, changing the .__next__ methods of map and
> filter iterators should be a new thread, which I will write after posting
> this.
The function was just supposed to generate the effect. More plausible
examples can be made. I tripped over this myself once when doing
something with iterators that required me to extract the first item
before processing the rest. A simple demonstration:
def sum(iterable):
iterator = iter(iterable)
total = next(iterator)
for item in iterator:
total += item
return total
results = list(map(sum, data_sources))
> The g.f. PEP says that generator functions should also follow this guideline
> and not raise StopIteration (but should always return when not yielding).
> ...
>>> The change in the PEP is to change that side effect such that
>>> those exceptions are converted to RuntimeError rather than silently
>>> suppressed - making generator function bodies behave less like
>>> __next__ method implementations.
>>
>> Fine but generator function bodies are __next__ method
>> implementations.
>
> This is about as wrong, or at least as confusing, as saying that the
> function passed to map is a __next__ method implementation. There is only
> one generator.__next__, just as there is only one map.__next__. Both
> .__next__ methods execute the user function (or function frame) passed in to
> the corresponding .__init__ to calculate the next value to return. So the
> external functions assist the __next__ methods but are not __next__ methods
> themselves. In particular, they should not raise StopIteration.
The idea is that it happens by accident.
Oscar
More information about the Python-ideas
mailing list