[Python-ideas] PEP 479: Change StopIteration handling inside generators

[Guido van Rossum]

On Tue, Nov 25, 2014 at 9:48 AM, Chris Angelico <rosuav at gmail.com> wrote:

> On Wed, Nov 26, 2014 at 4:30 AM, Wolfgang Maier
> <wolfgang.maier at biologie.uni-freiburg.de> wrote:
> > Well, I'm not familiar with every implementation detail of the
> interpreter
> > so I can't judge how difficult to implement certain things would be, but
> one
> > solution that I could think of is:
>
> I don't know much about the internal details of CPython either, but
> let's just ignore that for the moment and consider specs for the
> Python language. AFAIK, not one of the concerns raised (by PEP 479 or
> your proposal here) is CPython-specific.
>
> > allow StopIteration to be raised anywhere, but let it bubble up only
> *one*
> > frame.
> > So if the next outer frame does not deal with it, the exception would be
> > converted to UnhandledStopIteration (or something else) when it's about
> to
> > bubble out of that outer frame.
> > The builtin next() would simply reset the frame count by catching and
> > reraising StopIteration raised inside its argument (whether that's an
> > iterator's __next__ or a generator; note that in this scenario using
> raise
> > StopIteration instead of return inside a generator would remain
> possible).
>
> Interesting. Makes a measure of sense, and doesn't have much magic to it.
>
> > So different from the current PEP where a StopIteration must be dealt
> with
> > explicitly using try/except only inside generators, but bubbles up
> > everywhere else, here StopIteration will be special everywhere, i.e., it
> > must be passed upwards explicitly through all frames or will get
> converted.
> >
> > Back to Steven's generator expression vs comprehension example:
> >
> > iterable = [iter([])]
> > list(next(x) for x in iterable)
> >
> > would raise UnhandledStopIteration since there is no way, inside the
> > generator expression to catch the StopIteration raised by next(x).
>
> Downside of this is that it's harder to consciously chain iterators,
> but maybe that's a cost that has to be paid.
>
> Suggestion for this: Have a new way of "raise-and-return". It's mostly
> like raise, except that (a) it can't be caught by a try/except block
> in the current function (because there's no point), and (b) it
> bypasses the "this exception must not pass unnoticed". It could then
> also be used for anything else that needs the "return any object, or
> signal lack of return value" option, covering AttributeError and so
> on.
>
> So it'd be something like this:
>
> class X:
>     def __iter__(self): return self
>     def __next__(self):
>         if condition: return value
>         signal StopIteration
>
> The 'signal' statement would promptly terminate the function (not sure
> exactly how it'd interact with context managers and try/finally, but
> something would be worked out), and then raise StopIteration in the
> calling function. Any other StopIteration which passes out of a
> function would become a RuntimeError.
>
> Magic required: Some way of knowing which exceptions should be covered
> by this ban on bubbling; also, preferably, some way to raise
> StopIteration in the calling function, without losing the end of the
> backtrace.
>
> This could be a viable proposal. It'd be rather more complicated than
> PEP 479, though, and would require a minimum of five hundred
> bikeshedding posts before it comes to any kind of conclusion, but if
> you feel this issue is worth it, I'd certainly be an interested
> participant in the discussion.
>

It's not viable. It will break more code than PEP 479, and it will incur a
larger interpreter overhead (every time any exception bubbles out of any
frame we'd have to check whether it is (derived from) StopIteration and
replace it, rather than only when exiting a generator frame. (The check for
StopIteration is relatively expensive -- it's easy to determine that an
exception *is* StopIteration, but in order that it doesn't derive from
StopIteration you have to walk the inheritance tree.)

Please stop panicking.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20141125/5fffcaa0/attachment-0001.html>