[Python-Dev] GeneratorExit is unintuitive and uneccessary

[Nick Coghlan]

Igor Bukanov wrote:
> This example also suggests how to fix generators. One just need to
> change the close method so it would cause return executed right after
> the yield instead of throw.
> 
> I.e. replace the current text from
> http://www.python.org/dev/peps/pep-0342/
<snip close() = throw(GeneratorExit)>

> by simpler one:
<snip close() = externally forced return>

Simpler is in the eye of the beholder - the current close() merely uses
throw() to raise a particular kind of exception 'asynchronously' (which is
already a familiar concept due to KeyboardInterrupt). What you're suggesting
is a completely new flow control concept that doesn't exist anywhere else in
the language.

I suggested during the PEP 352 discussions that GeneratorExit should inherit
directly from BaseException, so that the fix to your example would be to
replace the bare "except:" with "except Exception:" (because it can swallow
KeyboardInterrupt the version you posted is buggy regardless of how g.close()
works).

Guido didn't like that idea [1], so the correct way to write your exception
handler is to either explicitly catch & reraise GeneratorExit before doing a
general "except Exception:" catch-all clause, or to do the yield outside the
try-except clause (but inside the try-finally) (obviously, this makes more
sense when what you're yielding is an expression rather than a single variable).

I think Guido is correct that GeneratorExit should be a normal exception from
a technical point of view (as GE should never reach the outer layers of a
program), but I also believe it is going to be a PITA from a practical point
of view because I think the natural way of writing a generator with exception
handling is to include the yield inside the try/except block instead of
storing the result in a variable and yielding afterwards.

However, since the status quo is the result of BDFL preference, it's going to
be up to Guido as to whether or not he considers this enough of a potential
pain to change his mind. As there isn't actually any new evidence here (it's
all still hypothetical), I don't really expect that to happen :)

That said, with the PEP 342 implementation now being a bit more mature, I am
curious as to exactly *how* GeneratorExit is supposed to leak out of code that
was supposed to catch it (which was Guido's main concern at the time he
decided that GE should remain a normal Exception). GeneratorExit really
shouldn't be getting raised by anything other than g.close(), which takes
great pains to ensure that the exception gets handled correctly (and if it
*doesn't* get handled correctly, the resulting exception that reports that
fact is a RuntimeError, not GeneratorExit).

> This not only fixes the above discrepancy between normal flow control
> and generators, removes GeneratorExit and simplifies the generator
> protocol, but also bring a new feature allowing to have easy to grasp
> feature table of the iterator methods:
> 
> next: continue after yield

This isn't right, though. next()/send() just resume execution after a yield. 
They are only equivalent to continue if the yield is at the end of a loop body.

> throw: raise after yield
> close: return after yield

In the current implementation, close is just a special case of throw
(g.throw(GeneratorExit)), the same as next() is just a special case of send
(g.send(None)).

2 concepts to understand (send(), throw()) is simpler to my mind than 3
(send(), throw(), close()).

Cheers,
Nick.

[1] http://mail.python.org/pipermail/python-dev/2006-March/062825.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org