[Python-ideas] Map and filter should also convert StopIteration to RuntimeError
Guido van Rossum
guido at python.org
Sat Dec 13 21:48:38 CET 2014
On Fri, Dec 12, 2014 at 10:10 PM, Raymond Hettinger <
raymond.hettinger at gmail.com> wrote:
> > On Dec 12, 2014, at 4:36 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>
> > Terry Reedy wrote:
> >> PEP 479 reverses this acceptance by having generator.__next turn
> StopIteration raised in a user-written generator function body into a
> RuntimeError. I propose that other builtin iterator.__next__ methods that
> execute a passed in function do the same.
> > Maybe also any user-written function whose
> > name isn't "__next__"?
> That does seem to be where this is headed :-(
> I'm -1 on this proposal.
Me too. PEP 479 was a clear win. __next__ methods are a much murkier area
and we should not mess with them.
> Generators are part of the language internals and language spec, so Guido
> can reasonably decide to take this in whatever direction he wants. In
> contrast, now you're moving on to parts of the language library that merely
> call functions and return results.
> It is not the responsibility of accumulate(), filter(), map(), or any
> other higher-order functions to impose rules about what those functions are
> allowed to do -- they can take as much time as they want, they can hold
> the GIL, they can manipulate signals, they can use tons of memory, they can
> return scalar values or complex objects, and they can raise any exception,
> but now StopIteration would become an event that gets special treatment.
> Further, this new made-up rule (for which there is zero demonstrated need
> in any language I know) would have to be applied throughout the standard
> library and possibly be extended to third-party code. It would be yet
> another language idiosyncrasy that would have to be learned, remembered,
> and StackOverflowed about for an eternity.
> In the case of generators, the PEP 479 rule is easily applied (not hard to
> search for or to mitigate). In contrast, the "fix" in this case would need
> to be applied to the *called* functions or their callees, possibly far
> removed from the map/filter/accumulate call. If that function is in
> third-party code or in a C-library, then the mitigation would require
> redesigning the call logic completely (not fun) or to wrap the function in
> something transforms a StopIteration into a custom exception and then
> re-catches the custom exception upstream from the higher-order function
> (also not fun).
> For anyone who has tested code that is currently working correctly but
> becomes broken by this proposal, the simplest mitigation will be for them
> to write their own variants of
> map/filter/accumulate/dropwhile/takewhile/groupby/starmap that just ignore
> this proposal and restore the behavior that has been happily in place for
When it comes to map() and filter() and friends the situation is murkier
yet. In Python 2, raising StopIteration in the function passed to
map/filter was just bubbled out. In Python 3, where these have become lazy,
a StopIteration raised by the function terminates the iteration. Was that
intentional? I doubt it. Is it now a feature? Maybe (though it would make
more sense to use something like takewhile). Could it mask a subtle bug?
Probably (but it doesn't sound like a common situation -- map/filter
functions are usually small and simple). Should we "fix" it? I don't think
so. It's not sufficiently broken, the fix would lead us onto a slippery
slope. Enough is enough.
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-ideas