[Twisted-Python] Please help with Deferreds
Can anyone explain the behaviour of the following? #!/usr/bin/python import sys from twisted.internet import defer, reactor d=defer.Deferred() d.callback(0) d2=defer.Deferred() d.addCallback(lambda x,d2=d2: d2) # switch the comment sign on the next two and it'll # print not 1 but a Deferred. Why? #reactor.callLater(0, lambda d2=d2: d2.callback(1)) d2.callback(1) d.addCallback(lambda x:sys.stdout.write(repr(x)+'\n')) d.addBoth(lambda x:reactor.stop()) reactor.run() -- tv@{{hq.yok.utu,havoc,gaeshido}.fi,{debian,wanderer}.org,stonesoft.com} double a,b=4,c;main(){for(;++a<2e6;c-=(b=-b)/a++);printf("%f\n",c);}
On Sat, Aug 24, 2002 at 10:05:30PM +0300, Tommi Virtanen wrote:
Can anyone explain the behaviour of the following?
#!/usr/bin/python import sys from twisted.internet import defer, reactor d=defer.Deferred() d.callback(0) d2=defer.Deferred() d.addCallback(lambda x,d2=d2: d2)
So you've effectively done: d = defer.Deferred() d2 = defer.Deferred() d.callback(d2)
# switch the comment sign on the next two and it'll # print not 1 but a Deferred. Why? #reactor.callLater(0, lambda d2=d2: d2.callback(1))
Which is effectively: # reactor.callLater(0, d2.callback, 1)
d2.callback(1)
d.addCallback(lambda x:sys.stdout.write(repr(x)+'\n')) d.addBoth(lambda x:reactor.stop())
Note that these callbacks will fire immediately upon adding them.
reactor.run()
And of course, you don't need to do reactor.run() unless you're using the reactor.callLater version of this code. So, this is easy... Consider what happens to your first Deferred, d, with that line uncommented: d = defer.Deferred() # 1. Create it d.callback(d2) # 2. call it with some value (d2) reactor.callLater(....) # 3. this doesn't happen yet d.addCallback(....) # 4. this fires *immediately*, and writes the value from 2., which is d2 So that's why it prints the Deferred in that case. Now back the way the code is originally: d = defer.Deferred() # 1. Create it d.callback(d2) # 2. call it with some value (d2) d2.callback(1) # 3. call d2 with another value (1) d.addCallback(....) # 4. this fires *immediately*, and writes the value from 2. -- which is 1??? That does confuse me. Somehow, if a callback has a result which is another Deferred, it does something special to automatically call that Deferred, which I didn't realise and seems nasty to me. I thought that was only meant to happen if you explicitly called chainDeferred? I've just re-read doc/howto/defer and I see that this is in fact explained under the heading "Chain Deferreds", but I'm still curious as to why this behaviour is necessary... "explicit is better than implicit", etc :) Anyway, I hope you now understand why that code behaves the way it does... -Andrew.
On Mon, Aug 26, 2002 at 11:43:46AM +1000, Andrew Bennetts wrote:
#!/usr/bin/python import sys from twisted.internet import defer, reactor d=defer.Deferred() d.callback(0) d2=defer.Deferred() d.addCallback(lambda x,d2=d2: d2)
So you've effectively done:
d = defer.Deferred() d2 = defer.Deferred() d.callback(d2)
Not really. You'll notice that Deferred addCallbacks handlers returning Deferreds is handled differently from d.callback(Deferred()); the former gets special treatment, the latter doesn't. Your version never outputs "1", it outputs repr(d2) for the callLater version and repr(d) for the immediate one. My version outputs "1" for the immediate case and repr(d2) for the callLater version.
Consider what happens to your first Deferred, d, with that line uncommented: d = defer.Deferred() # 1. Create it d.callback(d2) # 2. call it with some value (d2) reactor.callLater(....) # 3. this doesn't happen yet d.addCallback(....) # 4. this fires *immediately*, and writes the value from 2., which is d2
Yes, but only because you modified it to be such.
So that's why it prints the Deferred in that case. Now back the way the code is originally: d = defer.Deferred() # 1. Create it d.callback(d2) # 2. call it with some value (d2) d2.callback(1) # 3. call d2 with another value (1) d.addCallback(....) # 4. this fires *immediately*, and writes the value from 2. -- which is 1???
That does confuse me. Somehow, if a callback has a result which is another Deferred, it does something special to automatically call that Deferred, which I didn't realise and seems nasty to me. I thought that was only meant to happen if you explicitly called chainDeferred?
Yes, exactly. And I'm asking why this mechanism doesn't work with callLater.
I've just re-read doc/howto/defer and I see that this is in fact explained under the heading "Chain Deferreds", but I'm still curious as to why this behaviour is necessary... "explicit is better than implicit", etc :)
And it can't be emulated with chainDeferred, for the simple reason of needing to hold both Deferreds at once. The thing registered with addCallback doesn't have a reference to the Deferred it is added to, and it could be added to many Deferreds. Handy: def foo(dummy): d2=Deferred() do_something(d2) return d2 d=Deferred() d.addCallback(foo) Cumbersome: def foo(dummy, d): d2=Deferred() do_something(d2) d2.chainDeferred(d) return dummy d=Deferred() d.addCallback(foo, d) Plus, it allows you to use more things as callbacks; many information retrieval funcs return either the wanted result or a Deferred. They are very nice to use as callbacks, chaining them to retrieve values for each other.
Anyway, I hope you now understand why that code behaves the way it does...
No, I don't. You only managed to miss the point :) -- :(){ :|:&};:
On Mon, 26 Aug 2002 20:50:34 +0300, Tommi Virtanen
On Mon, Aug 26, 2002 at 11:43:46AM +1000, Andrew Bennetts wrote:
That does confuse me. Somehow, if a callback has a result which is another Deferred, it does something special to automatically call that Deferred, which I didn't realise and seems nasty to me. I thought that was only meant to happen if you explicitly called chainDeferred?
Yes, exactly. And I'm asking why this mechanism doesn't work with callLater.
I'm in the process of checking in the fix for this bug. Thanks for finding it!
I've just re-read doc/howto/defer and I see that this is in fact explained under the heading "Chain Deferreds", but I'm still curious as to why this behaviour is necessary... "explicit is better than implicit", etc :)
And it can't be emulated with chainDeferred, for the simple reason of needing to hold both Deferreds at once. The thing registered with addCallback doesn't have a reference to the Deferred it is added to, and it could be added to many Deferreds.
Plus, it allows you to use more things as callbacks; many information retrieval funcs return either the wanted result or a Deferred. They are very nice to use as callbacks, chaining them to retrieve values for each other.
Yes. There are actually many cases where it's not possible to use deferreds in the way described in the "cumbersome" example you provided without changing framework code, and that's bad. This requirement evolved from repeat experience with those cases breaking very basic database functionality using twisted.enterprise. -- | <`'> | Glyph Lefkowitz: Travelling Sorcerer | | < _/ > | Lead Developer, the Twisted project | | < ___/ > | http://www.twistedmatrix.com |
On Mon, Aug 26, 2002 at 08:50:34PM +0300, Tommi Virtanen wrote:
On Mon, Aug 26, 2002 at 11:43:46AM +1000, Andrew Bennetts wrote:
#!/usr/bin/python import sys from twisted.internet import defer, reactor d=defer.Deferred() d.callback(0) d2=defer.Deferred() d.addCallback(lambda x,d2=d2: d2)
So you've effectively done:
d = defer.Deferred() d2 = defer.Deferred() d.callback(d2)
Not really. You'll notice that Deferred addCallbacks handlers returning Deferreds is handled differently from d.callback(Deferred()); the former gets special treatment, the latter doesn't.
Your version never outputs "1", it outputs repr(d2) for the callLater version and repr(d) for the immediate one.
My version outputs "1" for the immediate case and repr(d2) for the callLater version.
Hmm. Good point. Deferreds are more complicated that I thought :( I think this implicit chaining behaviour needs to become more prominently discussed in the documentation, because that's what confused me... Probably what needs updating in doc/howto/defer is the diagram explaining the chain of callback processing (or the text next to it), which never at any point hints at this behaviour (that's described later). I'll probably checkin a doc fix along the lines of "Note that this behaves differently if a callback returns a Deferred; see 'Chaining Deferreds' for more details". -Andrew.
participants (3)
-
Andrew Bennetts
-
Glyph Lefkowitz
-
Tommi Virtanen