[Twisted-Python] Multiple deferreds that depend on each other
I know people ask about deferreds all the time (and i've read through a bunch of those questions), but there is still one thing I don't quite grasp. If I want to do a number of actions in a row that each return a deferred but depend on eachother, do I need to nest those callbacks? Ie if i want to: 1. Call database function using adbabi 2. Call xmlrpc method using result from 1 3. Call database using result from 2 Would I just put 2 inside the callback for 1 and then put 3 inside the callback for 2? How would I do that in one function? All the examples I have seen have a function calling one method returning a deferred and then placing the logic (doing something with the result) in the callback, but do not deal with multiple deferreds that depend on each other.
Hi Landreville,
If I want to do a number of actions in a row that each return a deferred but depend on eachother, do I need to nest those callbacks? Ie if i want to: 1. Call database function using adbabi 2. Call xmlrpc method using result from 1 3. Call database using result from 2 Would I just put 2 inside the callback for 1 and then put 3 inside the callback for 2?
By multiple deferreds that depend on each other, you probably mean a linear firing of actions based on callbacks of previous actions. The only way to do that is to trigger actions in callbacks of the deferreds in question, as you allude to. Yeah, so roughly something like this would work: def db_call(arg): dfr = make_db_call_in_thread(arg) return dfr def xmlrpc_call(arg): dfr = make_call(arg) return dfr dfr = db_call(some_initial_arg) dfr.addCallback(xmlrpc) dfr.addCallback(db_call)
How would I do that in one function? All the examples I have seen have a function calling one method returning a deferred and then placing the logic (doing something with the result) in the callback, but do not deal with multiple deferreds that depend on each other.
To do it in one function, you could use something like: def complex_call(): def db_call(arg): dfr = make_db_call_in_thread(arg) return dfr def xmlrpc_call(arg): dfr = make_call(arg) return dfr dfr = db_call(some_initial_arg) dfr.addCallback(xmlrpc_call) dfr.addCallback(db_call) dfr.addErrback(my_errback_handler) return dfr Or you can make each handler a method on a class. Another way to do it is to use inlineCallbacks: from twisted.internet import defer @defer.inlineCallbacks def example(): res = yield db_call(some_initial_arg) rpc_res = yield xmlrpc_call(res) db_res = yield db_call(rpc_res) In the above example, it's all still asynchronous (if not parallel), but it looks like it's synchronous, using the magic of decorators and Python 2.5+'s ability to retrieve results from yield statements. Hope that makes sense. Cheers, Reza -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: reza@tweetdeck.com twitter: @rlotun
Oh I like the look of the inlineCallbacks. Is there a way that I can still
use inspect.getargspec to retrieve the arg names of my original method, as
opposed to the generator returned by inlineCallbacks.
I noticed it copies the __doc__ over, but is there an attribute where I can
still inspect my original method?
On Tue, Nov 17, 2009 at 11:08 AM, Reza Lotun
Hi Landreville,
If I want to do a number of actions in a row that each return a deferred but depend on eachother, do I need to nest those callbacks? Ie if i want to: 1. Call database function using adbabi 2. Call xmlrpc method using result from 1 3. Call database using result from 2 Would I just put 2 inside the callback for 1 and then put 3 inside the callback for 2?
By multiple deferreds that depend on each other, you probably mean a linear firing of actions based on callbacks of previous actions. The only way to do that is to trigger actions in callbacks of the deferreds in question, as you allude to.
Yeah, so roughly something like this would work:
def db_call(arg): dfr = make_db_call_in_thread(arg) return dfr
def xmlrpc_call(arg): dfr = make_call(arg) return dfr
dfr = db_call(some_initial_arg) dfr.addCallback(xmlrpc) dfr.addCallback(db_call)
How would I do that in one function? All the examples I have seen have a function calling one method returning a deferred and then placing the logic (doing something with the result) in the callback, but do not deal with multiple deferreds that depend on each other.
To do it in one function, you could use something like:
def complex_call(): def db_call(arg): dfr = make_db_call_in_thread(arg) return dfr
def xmlrpc_call(arg): dfr = make_call(arg) return dfr
dfr = db_call(some_initial_arg) dfr.addCallback(xmlrpc_call) dfr.addCallback(db_call) dfr.addErrback(my_errback_handler) return dfr
Or you can make each handler a method on a class. Another way to do it is to use inlineCallbacks:
from twisted.internet import defer @defer.inlineCallbacks def example(): res = yield db_call(some_initial_arg) rpc_res = yield xmlrpc_call(res) db_res = yield db_call(rpc_res)
In the above example, it's all still asynchronous (if not parallel), but it looks like it's synchronous, using the magic of decorators and Python 2.5+'s ability to retrieve results from yield statements.
Hope that makes sense.
Cheers, Reza
-- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: reza@tweetdeck.com twitter: @rlotun
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (2)
-
Landreville -
Reza Lotun