[Python-ideas] Map and filter should also convert StopIteration to RuntimeError
steve at pearwood.info
Sat Dec 13 15:45:08 CET 2014
On Fri, Dec 12, 2014 at 04:14:07PM -0500, Terry Reedy wrote:
> This proposal comes from Oscar Benjamin's comments on the PEP in the
> 'Generator are iterators' thread. In one post, he gave this example.
> >>> def func(x):
> ... if x < 0:
> ... raise StopIteration
> ... return x ** 2
> >>> it = map(func, [1, 2, 3, -1, 2])
> >>> list(it)
> [1, 4, 9]
> >>> list(it) # map continues to yield values...
> Function func violates what I think should be the guideline.
The term used by the docs is that it is "broken", which in this context
has a technical meaning. An iterator is broken if it fails the rule that
once it raises StopIteration, it will continue to always raise
Whether or not map() and filter() return a "broken iterator" is
completely at the mercy of the user's function. func() can silently
break the iterator in many different ways:
- it may call os._exit() or os.abort()
- it may call threaded code which deadlocks
- it may call time.sleep(2**10000)
- it may enter an infinite loop
to mention just a few. The documentation's promise that map() will
return an iterator that "applies function to every item of iterable" is
not an unconditional promise. It cannot possibly be, and neither map()
nor Python can detect every possible failure condition in advance.
It is not Python's responsibility to police that all iterators are
non-broken. That is the responsibility of the coder. If you write func()
such that it "breaks" map(), then you either have a good reason for
doing so, or you are responsible for your own actions. We are all
consenting adults here. Let's not complicate things in a futile attempt
to prevent people from shooting themselves in the foot.
There's a thread on python-dev at the moment decrying the difficulty of
writing polylingual Python 2 + 3 code and how some people find it sucks
all the joy out of writing Python code. Every backwards incompatible
change we add just makes it more difficult to deal with the 2/3
transition. New features are an incentive to upgrade to 3.x. This is not
an incentive to upgrade, but it will make it harder to write and
understand polylingual iterator code. Gratuitously fixing perceived
weaknesses in the iterator protocol which have been there since it was
first introduced will, in my opinion, cause more pain than benefit.
In another thread, Nick wrote to Oscar:
The problem you think PEP 479 is trying to solve *is* the
one where the discussion started, but it is *not* the one
where it ended when Guido accepted PEP 479. The problems
PEP 479 solves are generator specific - they have nothing
to do with the iterator protocol in general.
Let's leave the iterator protocol alone.
More information about the Python-ideas