[Twisted-Python] DeferredList and errbacks
I just noticed [what appears to me to be] a contradiction in the twisted documentation. The "Using Deferreds" howto [1] says, "A standard DeferredList will never call errback," while the "Twisted FAQ" [2] states, "Always add errbacks!" in response to the question, "My Deferred or DeferredList never fires, so my program just mysteriously hangs! What's wrong?" So, which is it? I suppose what this is getting at is that each deferred in a DeferredList should have an errback attached separately, but this isn't super-clear to me. Is this the right way to interpret the above statements? thanks, Lenny G. [1] http://twistedmatrix.com/documents/current/howto/defer [2] http://twistedmatrix.com/documents/current/howto/faq#auto19 __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On Tue, Apr 05, 2005 at 04:13:36PM -0700, Lenny G Arbage wrote:
I just noticed [what appears to me to be] a contradiction in the twisted documentation. The "Using Deferreds" howto [1] says, "A standard DeferredList will never call errback," while the "Twisted FAQ" [2] states, "Always add errbacks!" in response to the question, "My Deferred or DeferredList never fires, so my program just mysteriously hangs! What's wrong?"
So, which is it?
I suppose what this is getting at is that each deferred in a DeferredList should have an errback attached separately, but this isn't super-clear to me. Is this the right way to interpret the above statements?
By default, DeferredList waits for all its Deferreds to fire (with a result or an error), and returns a big list of results of the form: [(success_bool, value), (success_bool, value), ...] Where 'success_bool' is True if the corresponding Deferred succeeded (i.e. fired a callback), and False if it failed (i.e. fired an errback). 'value' will be a result or a Failure instance accordingly. So a standard DeferredList will never call errback, because it includes errors in the result passed to its callback. Error handling in this case could be handled by the user of the DeferredList, because it would have access to any underlying failures via the callback. But you're correct, the original deferreds would still need errbacks attached -- or pass consumeErrors=True to DeferredList, and it will do this for you. And of course, the situation is a bit different again if you pass fireOnOneErrback=True... You're right that the docs are a bit confusing here. Please file a bug about it at http://twistedmatrix.com/bugs. The "Other Behaviours" section of http://twistedmatrix.com/documents/current/howto/defer tries to address this, but perhaps we could do better. -Andrew.
Andrew Bennetts <andrew-twisted@puzzling.org> writes:
By default, DeferredList waits for all its Deferreds to fire (with a result or an error), and returns a big list of results of the form: [(success_bool, value), (success_bool, value), ...]
Where 'success_bool' is True if the corresponding Deferred succeeded (i.e. fired a callback), and False if it failed (i.e. fired an errback). 'value' will be a result or a Failure instance accordingly. So a standard DeferredList will never call errback, because it includes errors in the result passed to its callback.
Note however, that while the DeferredList will never errback, it doesn't stop the errback chains of the individual deferreds. It inserts itself into those chains to gather the information, but continues to let the failure propagate. So if you expect the DeferredList to be the final aggregation of information for all of the individual deferreds, but don't want unterminated errbacks (which often, but not always, result in default logging), you still need to install additional errbacks on each of the deferreds you hand to the DeferredList. And you need to add them after the deferreds are installed in the DeferredList or its the result of your individual errbacks that will flow to the DeferredList result. -- David
On 21 Apr 2005 14:39:38 -0400, David Bolen <db3l@fitlinxx.com> wrote:
Andrew Bennetts <andrew-twisted@puzzling.org> writes:
By default, DeferredList waits for all its Deferreds to fire (with a result or an error), and returns a big list of results of the form: [(success_bool, value), (success_bool, value), ...]
Where 'success_bool' is True if the corresponding Deferred succeeded (i.e. fired a callback), and False if it failed (i.e. fired an errback). 'value' will be a result or a Failure instance accordingly. So a standard DeferredList will never call errback, because it includes errors in the result passed to its callback.
Note however, that while the DeferredList will never errback, it doesn't stop the errback chains of the individual deferreds. It inserts itself into those chains to gather the information, but continues to let the failure propagate.
So if you expect the DeferredList to be the final aggregation of information for all of the individual deferreds, but don't want unterminated errbacks (which often, but not always, result in default logging), you still need to install additional errbacks on each of the deferreds you hand to the DeferredList. And you need to add them after the deferreds are installed in the DeferredList or its the result of your individual errbacks that will flow to the DeferredList result.
Alternatively, pass consumeErrors=True to DeferredList() and it will do something equivalent for all Deferreds you put into it. Jp
Jp Calderone <exarkun@divmod.com> writes:
Alternatively, pass consumeErrors=True to DeferredList() and it will do something equivalent for all Deferreds you put into it.
Hrmm, and of course that was also mentioned in the post I was replying to (which you were kind enough not to highlight)... duh! Looks like that got added after my initial work with DeferredList and I didn't catch it becoming available. It's particularly ironic, because I think I participated in the discussion at the time with respect to deprecating the addDeferred method that led to the patch. Ah well, learn something new every day ... definitely going to start using it. -- David
participants (4)
-
Andrew Bennetts -
David Bolen -
Jp Calderone -
Lenny G Arbage