[Python-Dev] GeneratorExit inheriting from Exception

[Guido van Rossum]

On 3/25/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> The kind of code I'm talking about would be an *existing* Python 2.4 generator
> that happens to do something like:
>
>    def gen(tasks):
>        """yield the results of a bunch of task functions"""
>        for task in tasks:
>            try:
>                yield (task, task())
>            except Exception, ex:
>                yield ExceptionOccurred(task, ex)

This is purely hypothetical. It doesn't look like good style at all.

> If you run such a generator on Python 2.5, but don't run it to completion
> before it is garbage collected, you will get an error message printed on
> stderr saying that an exception was ignored when this generator was cleaned
> up. If you use the new PEP 342 features to try to explicitly close it before
> it is garbage collected, you'll get the exception directly.

I think this is fine. The code breaks with the new yield semantics.
But that's because the except clause was overly broad. It's easy to
rewrite it like this, which is better style anyway because the scope
of the try/except is limited.

  try:
    value = (task, task())
  except Exception, ex:
    value = ExceptionOccurred(task, ex)
  yield value

> The culprit is the RuntimeError raised when the generator's close() method
> gets upset because the generator swallowed GeneratorExit.
>
> If GeneratorExit inherits directly from BaseException, such unexpected
> behaviour won't happen - the only way for an existing generator to break is if
> it contained a bare except clause, and that code was *already* dubious (e.g.
> it probably swallowed KeyboardInterrupt).
>
> I don't have any actual live examples of a generator with a broad exception
> clause like the one above, but toy generators like the one above are legal in
> 2.4 and result in spurious errors with current SVN.

I don't want to cater to hypotheticals. Unless you find real code out
there doing this kind of thing I don't believe the problem is real.

I like to resolve corner cases so that *likely* situations are handled
reasonably.

Just in case you feel inclined to argue this further, let me argue
that there's also a *downside* to making GeneratorExit inherit from
BaseException: if it ever "leaks" out of some code that was supposed
to catch it but somehow didn't, and there's an outer "except
Exception:" trying to protect against buggy code, that except clause
is bypassed.

So perhaps we can turn this into a requirement for exceptions that
inherit from BaseException instead of Exception: the chance that they
get raised by buggy code should be nihil. I think that SystemExit and
KeyboardExit both qualify -- the former is raised by *non-buggy* code
with the intention of falling all the way through; the latter is not
raised by code at all but by the end user.

I don't think GeneratorExit qualifies.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)