On 7 October 2012 23:43, Oscar Benjamin <oscar.j.benjamin@gmail.com> wrote:
Before pep 380 filter(lambda x: True, obj) returned an object that was the same kind of iterator as obj (it would yield the same values). Now the "kind of iterator" that obj is depends not only on the values that it yields but also on the value that it returns. Since filter does not pass on the same return value, filter(lambda x: True, obj) is no longer the same kind of iterator as obj. The same considerations apply to many other functions such as map, itertools.groupby, itertools.dropwhile.
I really should have checked this before posting but I didn't have Python 3.3 available: Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
import itertools
def f(): ... return 'Returned from generator!' ... yield ... next(filter(lambda x:True, f())) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration: Returned from generator!
So filter does propagate the same StopIteration instance. However map does not:
next(map(None, f())) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration
The itertools module is inconsistent in this respect as well. As already mentioned itertools.chain() hides the value:
next(itertools.chain(f(), f())) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration next(itertools.chain(f())) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration
Other functions may or may not:
next(itertools.dropwhile(lambda x:True, f())) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration: Returned from generator! next(itertools.groupby(f())) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration
These next two seem wrong since there are two iterables (but I don't think they can be done differently):
def g(): ... return 'From the other generator...' ... yield ... next(itertools.compress(f(), g())) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration: Returned from generator! next(zip(f(), g())) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration: Returned from generator!
I guess this should be treated as undefined behaviour? Perhaps it should be documented as such so that anyone who chooses to rely on it was warned. Also some of the itertools documentation is ambiguous in relation to returning vs yielding values from an iterator. Those on the builtin functions page are defined carefully: http://docs.python.org/py3k/library/functions.html#filter filter(function, iterable) Construct an iterator from those elements of iterable for which function returns true. http://docs.python.org/py3k/library/functions.html#map map(function, iterable, ...) Return an iterator that applies function to every item of iterable, yielding the results. But some places in the itertools module use 'return' in place of 'yield': http://docs.python.org/py3k/library/itertools.html#itertools.filterfalse itertools.filterfalse(predicate, iterable) Make an iterator that filters elements from iterable returning only those for which the predicate is False. If predicate is None, return the items that are false. http://docs.python.org/py3k/library/itertools.html#itertools.groupby itertools.groupby(iterable, key=None) Make an iterator that returns consecutive keys and groups from the iterable. The key is a function computing a key value for each element. If not specified or is None, key defaults to an identity function and returns the element unchanged. Oscar