Exception chaining and generator finalisation
While updating my yield-from impementation for Python 3.1.2, I came across a quirk in the way that the new exception chaining feature interacts with generators. If you close() a generator, and it raises an exception inside a finally clause, you get a double-barrelled traceback that first reports a GeneratorExit, then "During handling of the above exception, another exception occurred", followed by the traceback for the exception raised by the generator. To my mind, the fact that GeneratorExit is involved is an implementation detail that shouldn't be leaking through like this. Does anyone think this ought to be fixed, and if so, how? Should GeneratorExit be exempt from being implicitly set as the context of another exception? Should any other exceptions also be exempt? Demonstration follows: Python 3.1.2 (r312:79147, Jul 31 2010, 21:23:14) [GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin Type "help", "copyright", "credits" or "license" for more information.
def g(): ... try: ... yield 1 ... finally: ... raise ValueError("Spanish inquisition") ... gi = g() next(gi) 1 gi.close() Traceback (most recent call last): File "<stdin>", line 3, in g GeneratorExit
During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 5, in g ValueError: Spanish inquisition -- Greg
On Sun, Aug 1, 2010 at 11:01 AM, Greg Ewing
While updating my yield-from impementation for Python 3.1.2, I came across a quirk in the way that the new exception chaining feature interacts with generators.
If you close() a generator, and it raises an exception inside a finally clause, you get a double-barrelled traceback that first reports a GeneratorExit, then "During handling of the above exception, another exception occurred", followed by the traceback for the exception raised by the generator.
To my mind, the fact that GeneratorExit is involved is an implementation detail that shouldn't be leaking through like this.
Does anyone think this ought to be fixed, and if so, how? Should GeneratorExit be exempt from being implicitly set as the context of another exception? Should any other exceptions also be exempt?
I don't see it as an implementation detail - it's part of the spec of generator finalisation in PEP 342 that GeneratorExit is thrown in to the incomplete generator at the point of the most recent yield. Trying to hide that doesn't benefit anybody. SystemExit and KeyboardInterrupt behave the same way: Python 3.2a0 (py3k:82729, Jul 9 2010, 20:26:08) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import sys try: ... sys.exit(1) ... finally: ... raise RuntimeError("Ooops") ... Traceback (most recent call last): File "<stdin>", line 2, in <module> SystemExit: 1
During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 4, in <module> RuntimeError: Ooops
try: ... input("Hit Ctrl-C now") ... finally: ... raise RuntimeError("Ooops") ... Hit Ctrl-C nowTraceback (most recent call last): File "<stdin>", line 2, in <module> KeyboardInterrupt
During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 4, in <module> RuntimeError: Ooops Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan wrote:
I don't see it as an implementation detail - it's part of the spec of generator finalisation in PEP 342
It doesn't seem like something you need to know in this situation, though. All it tells you is that the finalisation is happening because the generator is being closed rather than completing on its own. I suppose it doesn't do any harm, but it seems untidy to clutter up the traceback with irrelevant and possibly confusing information.
Hit Ctrl-C nowTraceback (most recent call last): File "<stdin>", line 2, in <module> KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "<stdin>", line 4, in <module> RuntimeError: Ooops
That's a bit different, because the fact that the program was terminated by Ctrl-C could be useful information. -- Greg
On Sun, 01 Aug 2010 13:01:32 +1200
Greg Ewing
While updating my yield-from impementation for Python 3.1.2, I came across a quirk in the way that the new exception chaining feature interacts with generators.
If you close() a generator, and it raises an exception inside a finally clause, you get a double-barrelled traceback that first reports a GeneratorExit, then "During handling of the above exception, another exception occurred", followed by the traceback for the exception raised by the generator.
It only happens if you call close() explicitly:
def g(): ... try: yield 1 ... finally: 1/0 ... gi = g() next(gi) 1 del gi Exception ZeroDivisionError: ZeroDivisionError('division by zero',) in
ignored gi = g() next(gi) 1 next(gi) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in g ZeroDivisionError: division by zero
Regards Antoine.
On Sun, Aug 1, 2010 at 1:25 PM, Greg Ewing
Nick Coghlan wrote:
I don't see it as an implementation detail - it's part of the spec of generator finalisation in PEP 342
It doesn't seem like something you need to know in this situation, though. All it tells you is that the finalisation is happening because the generator is being closed rather than completing on its own.
That may be important though (e.g. if the generator hasn't been written to correctly take into account the possibility of exceptions being thrown in, then knowing the exception happened when GeneratorExit in particular was thrown in rather than when next() was called or a different exception was thrown in may matter for the debugging process). Basically, I disagree with your assumption that knowing GeneratorExit was involved won't be significant in figuring why the generator threw an exception at all, so I see this as providing useful exception context information rather than being untidy noise. A toy example, that isn't obviously broken at first glance, but in fact fails when close() is called: def toy_gen(): try: yield 1 except Exception as ex: exc = ex else: exc = None finally: if exc is not None: print(type(exc))
g = toy_gen() next(g) 1 g.throw(NameError)
Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration g = toy_gen() next(g) 1 g.close() Traceback (most recent call last): File "<stdin>", line 3, in toy_gen GeneratorExit
During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 9, in toy_gen UnboundLocalError: local variable 'exc' referenced before assignment Without knowing GeneratorExit was thrown, the UnboundLocalError would be rather confusing. Given GeneratorExit to work with though, it shouldn't be hard for a developer to realise that "exc" won't be set when a thrown exception inherits directly from BaseException rather than from Exception. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Antoine Pitrou wrote:
It only happens if you call close() explicitly:
Well, that's only because the exception is being ignored and you're not getting a traceback at all. If you arrange to get a traceback, the same thing happens. import traceback as tb def g(): try: try: yield 1 finally: raise ValueError("Hovercraft contains eels") except Exception: tb.print_exc() gi = g() next(gi) del gi -- Greg
def g():
... try: yield 1 ... finally: 1/0 ...
gi = g() next(gi)
1
del gi
Exception ZeroDivisionError: ZeroDivisionError('division by zero',) in
ignored gi = g() next(gi)
1
next(gi)
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 3, in g ZeroDivisionError: division by zero
Regards
Antoine.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg.ewing%40canterbury.ac...
participants (3)
-
Antoine Pitrou -
Greg Ewing -
Nick Coghlan