[Twisted-Python] inlineCallbacks loses custom exception type?
Hi, I am hitting an issue with inlineCallbacks-decorated functions. It seems that from within an inlineCallbacks-decorated func, if you yield on another deferred-returning func, and that deferred-returning func returns a deferred which will errback with a Failure wrapping a user defined exception type, then the user defined exception gets repackaged as a common Exception by the time you are in the inlineCallbacks-decorated function. It seems to be related to the fact that inside an inlineCallbacks-decorated func, if you yield on a func, and that func returns a deferred which errs back, twisted will convert the errback into a synchronous exception that it raises for you. That’s all a bit wordy, so maybe some code will help clarify: $ cat inlinecallbacks-loses-exception-types.py #!/usr/bin/env python import sys, traceback from twisted.internet import defer, reactor class MyException(Exception): pass def go2(): "An ordinary deferred-returning function that will return a deferred which is ready to errback()" try: raise MyException('xyz') except Exception as e: return defer.fail() # errs back with current exception context return defer.succeed(1) # unreachable code @defer.inlineCallbacks def go_inlinecb(): try: x = yield go2() print 'no exception raised' except MyException as e: print 'got my custom exception' traceback.print_exc() except Exception as e: print 'got a garden variety exception' traceback.print_exc() def go_noinlinecb(): d = go2() def cb(data): print 'got data in cb: %s' % (data) def eb(failure): try: failure.raiseException() except MyException as e: print 'got my custom exception' traceback.print_exc() except Exception as e: print 'got a garden variety exception' traceback.print_exc() d.addCallbacks(cb, eb) if sys.argv[1] == 'inline': reactor.callWhenRunning(go_inlinecb) elif sys.argv[1] == 'noinline': reactor.callWhenRunning(go_noinlinecb) reactor.callLater(2, reactor.stop) reactor.run() $ ./inlinecallbacks-loses-exception-types.py inline got a garden variety exception Traceback (most recent call last): File "./inlinecallbacks-loses-exception-types.py", line 21, in go_inlinecb x = yield go2() Exception: xyz $ ./inlinecallbacks-loses-exception-types.py noinline got my custom exception Traceback (most recent call last): File "./inlinecallbacks-loses-exception-types.py", line 36, in eb failure.raiseException() File "/sw/external/twisted-py27-11.0.0/lib/python/twisted/python/failure.py", line 338, in raiseException raise self.type, self.value, self.tb MyException: xyz So as you can see, it seems that if you use inlineCallbacks, you cannot ‘except MyException’ and catch your own custom exceptions. Somehow a generic Exception is constructed. Is there any workaround or is this a known issue? Thanks, -- Benjamin Rutt
Using the slightly simplified code (nothing uses the reactor, so do stuff synchronosuly): import sys, traceback from twisted.internet import defer class MyException(Exception): pass def go2(): "An ordinary deferred-returning function that will return a deferred which is ready to errback()" try: raise MyException('xyz') except Exception: return defer.fail() # errs back with current exception context return defer.succeed(1) # unreachable code @defer.inlineCallbacks def go_inlinecb(): try: yield go2() print 'no exception raised' except MyException: print 'got my custom exception' traceback.print_exc() except Exception: print 'got a garden variety exception' traceback.print_exc() def go_noinlinecb(): d = go2() def cb(data): print 'got data in cb: %s' % (data) def eb(failure): try: failure.raiseException() except MyException: print 'got my custom exception' traceback.print_exc() except Exception: print 'got a garden variety exception' traceback.print_exc() d.addCallbacks(cb, eb) if sys.argv[1] == 'inline': go_inlinecb() elif sys.argv[1] == 'noinline': go_noinlinecb() I get the follwing results: % python test.py noinline got my custom exception Traceback (most recent call last): File "test.py", line 36, in eb failure.raiseException() File "/usr/lib64/python2.7/site-packages/twisted/python/failure.py", line 370, in raiseException raise self.type, self.value, self.tb MyException: xyz % python test.py inline got my custom exception Traceback (most recent call last): File "test.py", line 21, in go_inlinecb yield go2() MyException: xyz This is with % python --version Python 2.7.3 % python -c 'import twisted; print twisted.__version__' 12.2.0 Tom
That's a useful data point, thanks. I usually don't use unreleased revisions of twisted, but I'll try the latest release for starters and see what the result is. Forgot to mention before, I am on python 2.7.1 and twisted 11.0.0, so there's a chance this is fixed in a later release from what I am running. On Fri, Dec 7, 2012 at 10:00 AM, <exarkun@twistedmatrix.com> wrote:
-- Benjamin Rutt
Using the slightly simplified code (nothing uses the reactor, so do stuff synchronosuly): import sys, traceback from twisted.internet import defer class MyException(Exception): pass def go2(): "An ordinary deferred-returning function that will return a deferred which is ready to errback()" try: raise MyException('xyz') except Exception: return defer.fail() # errs back with current exception context return defer.succeed(1) # unreachable code @defer.inlineCallbacks def go_inlinecb(): try: yield go2() print 'no exception raised' except MyException: print 'got my custom exception' traceback.print_exc() except Exception: print 'got a garden variety exception' traceback.print_exc() def go_noinlinecb(): d = go2() def cb(data): print 'got data in cb: %s' % (data) def eb(failure): try: failure.raiseException() except MyException: print 'got my custom exception' traceback.print_exc() except Exception: print 'got a garden variety exception' traceback.print_exc() d.addCallbacks(cb, eb) if sys.argv[1] == 'inline': go_inlinecb() elif sys.argv[1] == 'noinline': go_noinlinecb() I get the follwing results: % python test.py noinline got my custom exception Traceback (most recent call last): File "test.py", line 36, in eb failure.raiseException() File "/usr/lib64/python2.7/site-packages/twisted/python/failure.py", line 370, in raiseException raise self.type, self.value, self.tb MyException: xyz % python test.py inline got my custom exception Traceback (most recent call last): File "test.py", line 21, in go_inlinecb yield go2() MyException: xyz This is with % python --version Python 2.7.3 % python -c 'import twisted; print twisted.__version__' 12.2.0 Tom
That's a useful data point, thanks. I usually don't use unreleased revisions of twisted, but I'll try the latest release for starters and see what the result is. Forgot to mention before, I am on python 2.7.1 and twisted 11.0.0, so there's a chance this is fixed in a later release from what I am running. On Fri, Dec 7, 2012 at 10:00 AM, <exarkun@twistedmatrix.com> wrote:
-- Benjamin Rutt
participants (3)
-
Benjamin Rutt -
exarkun＠twistedmatrix.com -
Tom Prince