[Python-ideas] Propagating StopIteration value

[Oscar Benjamin]

On 7 October 2012 23:43, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
>
> Before pep 380 filter(lambda x: True, obj) returned an object that was
> the same kind of iterator as obj (it would yield the same values). Now
> the "kind of iterator" that obj is depends not only on the values that
> it yields but also on the value that it returns. Since filter does not
> pass on the same return value, filter(lambda x: True, obj) is no
> longer the same kind of iterator as obj. The same considerations apply
> to many other functions such as map, itertools.groupby,
> itertools.dropwhile.
>

I really should have checked this before posting but I didn't have
Python 3.3 available:

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import itertools
>>>
>>> def f():
...     return 'Returned from generator!'
...     yield
...
>>> next(filter(lambda x:True, f()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration: Returned from generator!

So filter does propagate the same StopIteration instance. However map does not:

>>> next(map(None, f()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

The itertools module is inconsistent in this respect as well. As
already mentioned itertools.chain() hides the value:

>>> next(itertools.chain(f(), f()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> next(itertools.chain(f()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

Other functions may or may not:

>>> next(itertools.dropwhile(lambda x:True, f()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration: Returned from generator!
>>> next(itertools.groupby(f()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

These next two seem wrong since there are two iterables (but I don't
think they can be done differently):

>>> def g():
...     return 'From the other generator...'
...     yield
...
>>> next(itertools.compress(f(), g()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration: Returned from generator!
>>> next(zip(f(), g()))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration: Returned from generator!

I guess this should be treated as undefined behaviour? Perhaps it
should be documented as such so that anyone who chooses to rely on it
was warned.

Also some of the itertools documentation is ambiguous in relation to
returning vs yielding values from an iterator. Those on the builtin
functions page are defined carefully:

    http://docs.python.org/py3k/library/functions.html#filter
    filter(function, iterable)
    Construct an iterator from those elements of iterable for which
function returns true.

    http://docs.python.org/py3k/library/functions.html#map
    map(function, iterable, ...)
    Return an iterator that applies function to every item of
iterable, yielding the results.

But some places in the itertools module use 'return' in place of 'yield':

    http://docs.python.org/py3k/library/itertools.html#itertools.filterfalse
    itertools.filterfalse(predicate, iterable)
    Make an iterator that filters elements from iterable returning
only those for which the predicate is False. If predicate is None,
return the items that are false.

    http://docs.python.org/py3k/library/itertools.html#itertools.groupby
    itertools.groupby(iterable, key=None)
    Make an iterator that returns consecutive keys and groups from the
iterable. The key is a function computing a key value for each
element. If not specified or is None, key defaults to an identity
function and returns the element unchanged.


Oscar