Re: [Twisted-Python] A kinder and more consistent defer.inlineCallbacks
Hi Glyph
"glyph" == glyph <glyph@divmod.com> writes:
inlineCallbacks appears to have a bug: 'raise' before 'yield' in a generator results in a synchronous exception rather than an errback, although its documentation does not explain this.
glyph> Ugh, scratch that. No, it doesn't have this bug. Yes, you're right. glyph> I did some quick testing and saw some tracebacks, but apparently glyph> wasn't paying very close attention to them Nor was I, sorry! glyph> If func "raises before it gets to its first yield", we get the right glyph> behavior. If it just raises and doesn't yield *anywhere*, then it's glyph> not a generator and that's the same as your other case of glyph> accidentally-not- returning-a-generator. Still worth debugging, but glyph> not as serious. Agreed. I also agree with your earlier remarks about dropping the isinstance(result, GeneratorType). That leaves me with an alternative: def altInlineCallbacks(f): def unwindGenerator(*args, **kwargs): try: result = f(*args, **kwargs) except Exception, e: # f was not a generator. return failure.Failure() return _inlineCallbacks(None, result, Deferred()) return mergeFunctionMetadata(f, unwindGenerator) and still the problem that _inlineCallbacks raises if result doesn't have a send() method, etc. I'm also not sure of the best way to check for that, but don't think it should be in the _inlineCallbacks loop. --- BTW, there is a coding lesson here (at least for me): The reason I started to think there was a problem with Exceptions, tracebacks and _inlineCallbacks was from running code like this: @inlineCallbacks def f(): try: # something except Exception: # clean up raise When I ran it, the traceback of the Exception that propagated back to the attached errback would (sometimes) show the exception as having being triggered in _inlineCallbacks, like this: 2008/11/22 17:29 +0200 [-] File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 726, in _inlineCallbacks 2008/11/22 17:29 +0200 [-] result = g.send(result) The problem, as I guess will be obvious, is that my raise was delivering whatever sys.exc_info returned after the clean-up was done. In my case this was confusing as although the clean-up code had succeeded, it had called things that also made use of inlineCallbacks and the exception I was finally raising, and whose traceback I was printing, was a StopIteration raised inside *another* call of _inlineCallbacks! So it looked like _inlineCallbacks was somehow raising instead of sending a failure back to the errback... Re-raising the original exception made everything make sense. Thanks a lot for taking a look. Terry
On 22 Nov, 05:50 pm, terry@jon.es wrote:
Agreed. I also agree with your earlier remarks about dropping the isinstance(result, GeneratorType). That leaves me with an alternative:
def altInlineCallbacks(f): def unwindGenerator(*args, **kwargs): try: result = f(*args, **kwargs) except Exception, e: # f was not a generator. return failure.Failure() ^ I hope you mean "defer.fail()". and still the problem that _inlineCallbacks raises if result doesn't have a send() method, etc. I'm also not sure of the best way to check for that, but don't think it should be in the _inlineCallbacks loop.
As with the other case we mistakenly diagnosed here, it doesn't actually raise. It returns a failed Deferred. Consider: from twisted.internet.defer import inlineCallbacks @inlineCallbacks def notDeferred(): return object() def ok(result): ae = result.trap(AttributeError) print 'OK', ae notDeferred().addErrback(ok) The only thing that (potentially) needs to be done here is to produce a more useful error message. The other case, where inlineCallbacks decorates a function that itself raises an exception rather than returns an object, is the only way you won't get a Deferred.
participants (2)
-
glyph＠divmod.com -
Terry Jones