[Twisted-Python] unhandled exception in deferred has a delayed logging ?
while migrating some functionality to a deferred callback chain and debugging my app, i noticed this oddity... if a function doesn't have the proper args/kwargs definition, no error is raised and everything just hangs for example, if i have these 2 class functions in an internet.TimerService managed class ... def start_chain( self ): database.get_dbPool().runInteraction( self.get_update_batch )\ .addCallback( self.process_batch ) def get_update_batch(self): return True and then improperly don't have an arg to capture the return value in the callback... def process_batch( self ): pass then the reactor just seems to hang. if i kill the process, then the Unhandled Error and Traceback message prints. If i wait until the next interval that the timerservice runs, then the Unhandled Error and Traceback message will print then ( but that could be minutes/hours later ). has anyone else noticed this before ? are there any good ways around this , other than setting the timer service to be within seconds during bugtesting ?
On Wed, Sep 26, 2012 at 7:46 PM, Jonathan Vanasco <twisted-python@2xlp.com>wrote:
The reactor isn't hanging, it's running just fine. The problem is you expect the Deferred's error to be logged immediately, which is not guaranteed; the error is only logged if the object is garbage collected, which may take a while, or never happen if you've kept extra references around. The latter can happen by accident.
are there any good ways around this , other than setting the timer service to be within seconds during bugtesting ?
1. Add a logging errback to all Deferreds at the end of the callback chain: d.addErrback(log.err). 2. Less effective, but also helpful, is making sure you delete all unnecessary references to a Deferred once you've fired it, to make sure it gets garbage collected: self.d.callback(None) del self.d # remove extra reference that prevents garbage collection http://bit.ly/NS6ptj explains the issue. -- Itamar Turner-Trauring, Future Foundries LLC http://futurefoundries.com/ — Twisted consulting, training and support.
thanks so much for the quick response, and i wish i thought of looking in the FAQ ( just searched online, and nothing relevant came up ) Will defer.setDebugging(True) make these errors print/log sooner ? i'm in the process of getting all the addErrback() commands and making this app "proper", which is how i found this out in the first place. On Sep 26, 2012, at 8:20 PM, Itamar Turner-Trauring wrote:
// Jonathan Vanasco c. 646.729.6436 | 415.501.9815 e. jonathan@2xlp.com w. http://findmeon.com/user/jvanasco linkedin. http://linkedin.com/in/jonathanvanasco blog. http://destructuring.net
On 27/09/12 03:47, Jonathan Vanasco wrote:
I don't think so. I think you'll just get more info in the error. You could using a "loopingcall" to call gc.collect() periodically but it's hacky, and only helps with things you've cleared all refs to. If you've got a dangling ref, it'll just never get gc'ed.
i'm in the process of getting all the addErrback() commands and
That's your better option.
On Wed, Sep 26, 2012 at 7:46 PM, Jonathan Vanasco <twisted-python@2xlp.com>wrote:
The reactor isn't hanging, it's running just fine. The problem is you expect the Deferred's error to be logged immediately, which is not guaranteed; the error is only logged if the object is garbage collected, which may take a while, or never happen if you've kept extra references around. The latter can happen by accident.
are there any good ways around this , other than setting the timer service to be within seconds during bugtesting ?
1. Add a logging errback to all Deferreds at the end of the callback chain: d.addErrback(log.err). 2. Less effective, but also helpful, is making sure you delete all unnecessary references to a Deferred once you've fired it, to make sure it gets garbage collected: self.d.callback(None) del self.d # remove extra reference that prevents garbage collection http://bit.ly/NS6ptj explains the issue. -- Itamar Turner-Trauring, Future Foundries LLC http://futurefoundries.com/ — Twisted consulting, training and support.
thanks so much for the quick response, and i wish i thought of looking in the FAQ ( just searched online, and nothing relevant came up ) Will defer.setDebugging(True) make these errors print/log sooner ? i'm in the process of getting all the addErrback() commands and making this app "proper", which is how i found this out in the first place. On Sep 26, 2012, at 8:20 PM, Itamar Turner-Trauring wrote:
// Jonathan Vanasco c. 646.729.6436 | 415.501.9815 e. jonathan@2xlp.com w. http://findmeon.com/user/jvanasco linkedin. http://linkedin.com/in/jonathanvanasco blog. http://destructuring.net
On 27/09/12 03:47, Jonathan Vanasco wrote:
I don't think so. I think you'll just get more info in the error. You could using a "loopingcall" to call gc.collect() periodically but it's hacky, and only helps with things you've cleared all refs to. If you've got a dangling ref, it'll just never get gc'ed.
i'm in the process of getting all the addErrback() commands and
That's your better option.
participants (5)
-
exarkun＠twistedmatrix.com -
Glyph -
Itamar Turner-Trauring -
Jonathan Vanasco -
Phil Mayers