[Python-ideas] Map and filter should also convert StopIteration to RuntimeError

[Raymond Hettinger]

> On Dec 12, 2014, at 4:36 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> 
> Terry Reedy wrote:
>> PEP 479 reverses this acceptance by having generator.__next turn StopIteration raised in a user-written generator function body into a RuntimeError.  I propose that other builtin iterator.__next__ methods that execute a passed in function do the same.
> 
> Maybe also any user-written function whose
> name isn't "__next__"?

That does seem to be where this is headed :-(

I'm -1 on this proposal.   

Generators are part of the language internals and language spec, so Guido can reasonably decide to take this in whatever direction he wants.   In contrast, now you're moving on to parts of the language library that merely call functions and return results.  

It is not the responsibility of accumulate(), filter(), map(), or any other higher-order functions to impose rules about what those functions are allowed to do  -- they can take as much time as they want, they can hold the GIL, they can manipulate signals, they can use tons of memory, they can return scalar values or complex objects, and they can raise any exception, but now StopIteration would become an event that gets special treatment.  

Further, this new made-up rule (for which there is zero demonstrated need in any language I know) would have to be applied throughout the standard library and possibly be extended to third-party code. It would be yet another language idiosyncrasy that would have to be learned, remembered, and StackOverflowed about for an eternity.

In the case of generators, the PEP 479 rule is easily applied (not hard to search for or to mitigate).  In contrast, the "fix" in this case would need to be applied to the *called* functions or their callees, possibly far removed from the map/filter/accumulate call.  If that function is in third-party code or in a C-library, then the mitigation would require redesigning the call logic completely (not fun) or to wrap the function in something transforms a StopIteration into a custom exception and then re-catches the custom exception upstream from the higher-order function (also not fun).

For anyone who has tested code that is currently working correctly but becomes broken by this proposal, the simplest mitigation will be for them to write their own variants of map/filter/accumulate/dropwhile/takewhile/groupby/starmap that just ignore this proposal and restore the behavior that has been happily in place for ages.


Raymond