[Twisted-Python] Returning a deferred, maybe
We have several functions in the code which return either a synchronous value or a deferred. In these cases, the caller has to either inspect the return type and do different things depending whether the value is available right then, or call the function using meybeDeferred. It seems like you can save the caller some grief by ensuring that you always return a deferred value (eg. by wrapping a synchronous result with defer.succeed(), as suggested in http://twistedmatrix.com/ projects/core/documentation/howto/gendefer.html). Should this be an encouraged practice? -wsv
On Thu, 15 Jun 2006 16:40:59 -0700, Wilfredo Sánchez Vega <wsanchez@wsanchez.net> wrote:
It seems like you can save the caller some grief by ensuring that you always return a deferred value (eg. by wrapping a synchronous result with defer.succeed(), as suggested in http://twistedmatrix.com/ projects/core/documentation/howto/gendefer.html). Should this be an encouraged practice?
There are 2 reasons for this practice. One is a sort of silly micro-optimization (which vanishes anyway if the caller uses maybeDeferred) and the other is an issue of convenience for the callee. We should forget about this and just make Deferred faster. Newer interfaces have tended toward just saying "@return a Deferred which..." rather than "@return a value or a Deferred" unless they are highly public APIs like IResource and forgetting to wrap a 2-byte string return value is a real pain. In other words: document your interfaces precisely, and decide whose responsibility is to ensure that the value is a Deferred at some point, and you should probably prefer requiring it is a Deferred in most cases -- but I don't think that this is the sort of thing that should be in the coding standard. It varies from situation to situation.
I guess I'm not seeing any justification for returning a deferred some of the time other than: - the micro-optimization (which your saying is silly) and - convenience for the callee (but wrapping a result in defer.succeed () is hardly a burden, and you are basically just passed the inconvenience down to every caller, which is usually a bad allocation of convenience-ness-es). The reason I thought it a good idea to require always returning a deferred is that I had a lot of code which depended on locateChild() to return a non-deferred value, which is clearly broken. But none of the unit tests caught this, because I've never traversed a resource which returned a deferred in locateChild(). dreid pointed this out to me, so I changed some routines to fix this and had them always return a deferred, and sure enough, I ended up having to fix a *lot* of code. The point being that callers of functions that _usually_ return non-deferred values will usually work, enough so that it's hard to catch them with unit tests unless you go out of your way to ensure that deferreds are tested, which is unlikely if you also wrote the offending code. -wsv On Jun 15, 2006, at 5:14 PM, glyph@divmod.com wrote:
On Thu, 15 Jun 2006 16:40:59 -0700, Wilfredo Sánchez Vega <wsanchez@wsanchez.net> wrote:
It seems like you can save the caller some grief by ensuring that you always return a deferred value (eg. by wrapping a synchronous result with defer.succeed(), as suggested in http:// twistedmatrix.com/ projects/core/documentation/howto/ gendefer.html). Should this be an encouraged practice?
There are 2 reasons for this practice. One is a sort of silly micro-optimization (which vanishes anyway if the caller uses maybeDeferred) and the other is an issue of convenience for the callee. We should forget about this and just make Deferred faster.
Newer interfaces have tended toward just saying "@return a Deferred which..." rather than "@return a value or a Deferred" unless they are highly public APIs like IResource and forgetting to wrap a 2- byte string return value is a real pain.
In other words: document your interfaces precisely, and decide whose responsibility is to ensure that the value is a Deferred at some point, and you should probably prefer requiring it is a Deferred in most cases -- but I don't think that this is the sort of thing that should be in the coding standard. It varies from situation to situation.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Thu, 15 Jun 2006 19:24:20 -0700, Wilfredo Sánchez Vega <wsanchez@wsanchez.net> wrote:
I guess I'm not seeing any justification for returning a deferred some of the time other than:
- the micro-optimization (which your saying is silly) and
- convenience for the callee (but wrapping a result in defer.succeed () is hardly a burden, and you are basically just passed the inconvenience down to every caller, which is usually a bad allocation of convenience- ness-es).
In the case of, for example, web1's IResource, callers are in framework code and callees are in application code. Which means maybe two-dozen callers and like a billion callees. In the case of the *new* IResource, this might not be the right factoring since you are really supposed to subclass stuff and not implement directly; and in any event, in the long term hopefully most people will be using nevow+web2 and avoiding touching the resource model directly.
The point being that callers of functions that _usually_ return non- deferred values will usually work, enough so that it's hard to catch them with unit tests unless you go out of your way to ensure that deferreds are tested, which is unlikely if you also wrote the offending code.
Which reminds me. Somebody really *should* be trying to get that C deferred stuff James did into Twisted proper; one Deferred for every locateChild sounds excruciatingly expensive for long URLs, as the code currently stands.
On Thu, 15 Jun 2006 22:53:22 -0400, glyph@divmod.com wrote:
[snip]
Which reminds me. Somebody really *should* be trying to get that C deferred stuff James did into Twisted proper; one Deferred for every locateChild sounds excruciatingly expensive for long URLs, as the code currently stands.
FWIW, last I checked, it was an insignificant performance improvement over the current Deferred implementation, at least for web2. Jean-Paul
On Jun 15, 2006, at 11:01 PM, Jean-Paul Calderone wrote:
On Thu, 15 Jun 2006 22:53:22 -0400, glyph@divmod.com wrote:
[snip]
Which reminds me. Somebody really *should* be trying to get that C deferred stuff James did into Twisted proper; one Deferred for every locateChild sounds excruciatingly expensive for long URLs, as the code currently stands.
FWIW, last I checked, it was an insignificant performance improvement over the current Deferred implementation, at least for web2.
Which is not all that surprising, as web2 doesn't make very many deferreds. The optimization of not returning a deferred when you have an immediate result is not really so minor if you'd otherwise be making a ton of them. However, be careful, it's possible that in some cases the ambiguous result is actually slower, as "isinstance" turns out to be quite an expensive operation. So if you end up returning the deferred/result out multiple levels, testing at each level, at some point all the isinstance calls add up to be slower than just always returning a deferred. James
participants (4)
-
glyph＠divmod.com -
James Y Knight -
Jean-Paul Calderone -
Wilfredo Sánchez Vega