Re: [Twisted-Python] Simpler Twisted deferred code via decorated callbacks

Here's a description of what the 'callback' decorator in my code does. Here's some normal Twisted code, assuming you do from twisted.web.client import getPage (yes, getPage is kind-of obsolete, but it's a concrete and conceptually simple deferred-returning function I like to use in examples). def pageLength(url): def _length(page): return len(page) return getPage(url).addCallback(_length) Using the 'callback' decorator you could instead write: def pageLength(url): @callback def _length(page): return len(page) return _length(getPage(url) The syntactic difference in this case is minimal. The saving is more significant in other cases, but let's consider this simplest case. The decorator returns a wrapping function that essentially does this: 1. Look at all its arguments. For any that are Deferreds, put them into a deferred list, which I'll call DL for this explanation. This is done for positional args (line 25) and keyword args (line 31). 2. If no arguments are Deferreds, just call the original _length function with the original arguments, and return the result in a Deferred using maybeDeferred (line 44). So, any function that is called with a set of arguments that are not Deferred instances just returns a deferred that fires with the function result or fails if the function raises (maybeDeferred takes care of that for us). 3. The interesting part is when some of the arguments are deferreds (line 36). In that case, we have a DeferredList called DL that will eventually fire when all the arguments are available. The arguments are collected, as they arrive, into a list (fags) and a dictionary (fkw). Args that were not deferreds are already put into fargs and fkw (lines 30 and 35). 4. If DL fires normally, we can go ahead and call the wrapped original function with all the arguments it was supposed to receive. In our _length case, there was only one argument, a deferred returning the content of a web page, so the original _length gets called with the page content. A lambda to call the original function is added to DL as a callback (line 41), and DL is returned. Note that DL has already fired, so adding the callback function to call the original function results in the function being called immediately. By adding it as a callback to DL, we can return a deferred (DL) from the wrapper. 5. When DL is made, I pass fireOnOneErrback=True to it. So if any of the arguments to the wrapped function result in an error (i.e., in our example, getPage fails), then DL will fail immediately. In that case, an errback (getSubFailure) attached to DL will pull the original failure out of DL and return it. Because one of the arguments to the original function has failed to materialize, we obviously can't call the original function. By just returning DL (which has failed), the failure is propagated (in a deferred) to any caller we may have. If you look at the code for the 'callback' decorator in decorate.py, the above should help to make things clearer. I'm happy to answer any questions, of course. I hope the description makes it clear how the decorator works on a function called with multiple deferred arguments. It's quite like a DeferredList that you can put non-deferreds objects into and add a callback to (where the callback is the decorated function). I implemented that once years ago, but didn't bother to post it. The 'errback' decorator is similar in structure, but operates differently of course. I'll send a separate mail describing it. Terry
participants (1)
-
Terry Jones