[Twisted-Python] RPC design questions
I'm currently making some light-weight RPC/PubSub over WebSockets using Twisted. I have two design-like questions where I'd be happy for advice/options ... The first is definitely Twisted related, the second not strictly, .. Tobias 1) I'd like to chain RPC calls, i.e. self.call(23, "square").addCallback(self.call, "sqrt").addCallback(self.show) https://github.com/oberstet/Autobahn/blob/dev-rpc/demo/rpc/simple/simple_cli... self.call(<arg>, <procedure>) will return a deferred, <arg> is the marshalled argument for the RPC, and <procedure> is the remote procedure identifier. Now, the nice thing is, I can chain the result from one call to the next like in the example above. What I find less nice is that I have to have that order : <arg>, <procedure> since the result of the first deferred will be passed as the first argument to the second, and only then will the additional arguments be passed (in above example the "sqrt") How can I retain the - which I find - natural order for arguments? self.call(<procedure>, <arg>) self.call("square", 6).addCallback(self.call, "sqrt").addCallback(self.show) => not working, the 2nd self.call receives 36, "sqrt" .. 2) The server side methods a hooked up using decorators, like class SimpleServerProtocol(AutobahnServerProtocol): @AutobahnRpc def square(self, arg): return arg * arg https://github.com/oberstet/Autobahn/blob/dev-rpc/demo/rpc/simple/simple_ser... Here, AutobahnServerProtocol derives (indirectly) from Twisted Protocol. The decorator will register the method for RPC under "square". https://github.com/oberstet/Autobahn/blob/dev-rpc/lib/python/autobahn/autoba... That is all nice and simple, however I am wondering if it's a good idea to do that auto-registering on a Protocol derived class. I mean, the alternative could be having the user call something like registerRpcObject(<any class instance with RPC decorators>, <base URI>) in Protocol.connectionMade(), which then would auto-register all decorated methods in <any class instance ..> What do you think would be better?
On Mon, Aug 22, 2011 at 11:09 PM, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote:
self.call(<procedure>, <arg>) self.call("square", 6).addCallback(self.call, "sqrt").addCallback(self.show)
The easiest way to do this is probably: from functools import partial self.call("square", 6).addCallback(partial(self.call, "sqrt")).addCallback(self.show) -- mithrandi, i Ainil en-Balandor, a faer Ambar
Thanks alot! Did not knew about partial .. I checked the solution you proposed: works. It rectifies the arg order, but introduces more boilerplate to write (and making usage convenient is my goal). === In the meantime, I've come up with 3 others options: No chaining (no problem here): self.call("add", 23*23, 5).addCallback(self.show) Chaining Option 1: self.call("square", 23).addCallback(self.rcall, "add", 5).addCallback(self.show) Chaining Option 2: self.call("square", 23).addCallback(lambda res: self.call("add", res, 5)).addCallback(self.show) Chaining Option 3: self.call("square", 23).call("add", 5).addCallback(self.show) Option 1 ======= Makes use of rcall() vs call() where rcall() does the reordering of arguments def rcall(self, *args): a = [] a.append(args[1]) # procedure ID if args[0]: a.append(args[0]) # result from previous deferred a.extend(args[2:]) # new args return self.call(*a) Pro: terse, can handle callback&errback Con: user needs to remember to use rcall() not call() Option 2 ====== Similar to yours .. just using lambda. Pro: standard Python (as yours), can handle callbacks&errbacks Con: verbose Option 3 ======= Makes use of class AutobahnDeferred(Deferred): def call(self, *args): return self.addCallback(self.protocol.rcall, *args) Pro: most terse Con: only supports single callback no errback ==== Currently my thinking is: why not provide all variants? Anything why I shouldn't do?
mithrandi, i Ainil en-Balandor, a faer Ambar
ok. nice;) but what does it mean?
On Aug 23, 2011, at 10:37 AM, Tobias Oberstein wrote:
Con: subclassing in general is a bad idea. Subclassing Deferred is an even worse idea. What if Deferred one day gets a new method called 'call' that does something different? Your code would all break.
Anything why I shouldn't do?
Providing three different ways to do the same thing just so that you can use different syntax depending on your mood is a recipe for making your code hard to read. Pick one style, stick to it as much as possible - consistency is more important than the benefits of any one particular style :). (Except the subclassing one. Don't do that.) -glyph
Thanks for your advice .. that's what I was looking for;) Makes sense, I'll stick to your advice, kick the subclassing approach and the stuff with the reordering version rcall(). That leaves "partial" and "lambda" # Option 1 : prints 31 d2 = self.call("square", 6).addCallback(lambda res: self.call("sub", res, 5)).addCallback(self.show) # Option 3 : prints -31 d4 = self.call("square", 6).addCallback(partial(self.call, "sub", 5)).addCallback(self.show) Lambda is the most general, since it can insert the previous deferred's result anywhere in the next call. Partial only simulates append result to end of args of next call. But lambda is also most verbose. I'll sleep one night and then see if I document and endorse both or only lambda. Obviously, I can't hinder anyone using both. Built in stuff. Thanks again, Tobias Von: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] Im Auftrag von Glyph Lefkowitz Gesendet: Dienstag, 23. August 2011 20:30 An: Twisted general discussion Betreff: Re: [Twisted-Python] RPC design questions On Aug 23, 2011, at 10:37 AM, Tobias Oberstein wrote: class AutobahnDeferred(Deferred): def call(self, *args): return self.addCallback(self.protocol.rcall, *args) Pro: most terse Con: only supports single callback no errback Con: subclassing in general is a bad idea. Subclassing Deferred is an even worse idea. What if Deferred one day gets a new method called 'call' that does something different? Your code would all break. Anything why I shouldn't do? Providing three different ways to do the same thing just so that you can use different syntax depending on your mood is a recipe for making your code hard to read. Pick one style, stick to it as much as possible - consistency is more important than the benefits of any one particular style :). (Except the subclassing one. Don't do that.) -glyph
I follow Glyphs advice and only keep the lambda version with standard Twisted Deferred. However, I've got a (I believe) more serious "design" problem. The RPC stuff works in Twisted, but also from JavaScript. So I have 2 audiences (Twisted and JS developers). On JS, I use the Deferreds that come standard with jQuery since 1.5. Now compare the following Twisted code (self.call does the RPC and returns a Twisted Deferred): # prints 23 d2 = self.call("square", 23).addCallback(lambda res: \ self.call("sqrt", res)).addCallback(self.show) # prints 23 d3 = self.call("square", 23).addCallback(lambda res: \ self.call("sqrt", res).addCallback(self.show)) (note the subtle difference in bracketing) with the following JS // prints 529 sess.call("square", 23).then(function(res) { return sess.call("sqrt", res); }).then(console.log); // prints 23 sess.call("square", 23).then(function(res) { sess.call("sqrt", res).then(console.log); }); === The JS is structurally/syntactically similar to the Py version. However they behave differently. There seems to be a fundamental difference between Twisted and jQuery Deferreds. When calling then() on a jQuery Deferred D, it seems to return D, not any Deferred that might be returned within the handler within then(). http://api.jquery.com/deferred.then Whereas the Twisted Deferred addCallback() returns the inner Deferred. Am I getting something wrong? I'd be glad on any hints/clarification whats going on here .. Tobias == The complete examples are on: JS: https://github.com/oberstet/Autobahn/blob/master/demo/rpc/simple/simple_clie... Py Client: https://github.com/oberstet/Autobahn/blob/master/demo/rpc/simple/simple_clie... Py Server: https://github.com/oberstet/Autobahn/blob/master/demo/rpc/simple/simple_serv... Library: https://github.com/oberstet/Autobahn/tree/master/lib/python => python setup.py install
No, this is something jQuery and CommonJS got wrong; callbacks don't chain in their implementation. Firing a jQuery promise invokes each callback in order with the same argument. Firing a Deferred in Twisted invokes the first callback then passes its return value as the arg to the next callback, and so forth.
Ok. This is most unpleasant. In particular since my options then are: 1) write my own, sane (Twisted like, which I think is the sane way) JS deferred, not use jQuery/CommonJS (which everyone else does) 2) open the door for users running into issues when they do both Twisted & JS/jQuery and don't understand the subtle difference bad. of course not a Twisted mailing list topic anymore .. I'll go to jQuery list. Thanks! Tobias On 25.08.11 00:36, "Allen Short" <washort@twistedmatrix.com> wrote: No, this is something jQuery and CommonJS got wrong; callbacks don't chain in their implementation. Firing a jQuery promise invokes each callback in order with the same argument. Firing a Deferred in Twisted invokes the first callback then passes its return value as the arg to the next callback, and so forth.
Found something: it seems in jQuery 1.6 they have introduced something which corresponds more closely to Twisted Deferreds callback/errback chains: // prints 23 sess.call("square", 23).pipe(function(res) { return sess.call("sqrt", res); }).pipe(console.log); http://api.jquery.com/deferred.pipe/ I am still wondering why they called the other stuff "then()" when it's not then, but "also()" and "pipe()" is the real <then> ;) And now they have then() _and_ pipe(). This is all asking for confusion. - Whatever. Tobias On 25.08.11 00:52, "Tobias Oberstein" <tobias.oberstein@tavendo.de> wrote: Ok. This is most unpleasant. In particular since my options then are: 1) write my own, sane (Twisted like, which I think is the sane way) JS deferred, not use jQuery/CommonJS (which everyone else does) 2) open the door for users running into issues when they do both Twisted & JS/jQuery and don't understand the subtle difference bad. of course not a Twisted mailing list topic anymore .. I'll go to jQuery list. Thanks! Tobias On 25.08.11 00:36, "Allen Short" <washort@twistedmatrix.com> wrote: No, this is something jQuery and CommonJS got wrong; callbacks don't chain in their implementation. Firing a jQuery promise invokes each callback in order with the same argument. Firing a Deferred in Twisted invokes the first callback then passes its return value as the arg to the next callback, and so forth.
My understanding (partially aided by this post: http://www.sitepen.com/blog/2010/05/03/robust-promises-with-dojo-deferred-1-...) is that Dojo provides both twisted-style "addCallback" and CommonJS-style "then" methods.
On 25 August 2011 00:30, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote: <snip>
I agree. I had the same frustration a few months ago and various Twisted people pointed out missing features in Jquery deferred: * http://twistedmatrix.com/pipermail/twisted-python/2011-June/024146.html Mochikit is the best (and original) JS port of deferred..Dojo and Google Closure both credit Mochikit (and Twisted) in their source code. I'm planning to revert back to it. I guess it would be useful / constructive to compile a list of the weaknesses in the jquery / commonjs deferred API and the reasons and use cases for the Twisted deferred API. Perhaps they can be persuaded to evolve towards the Twisted API in time. -RichardW.
Jasper, Allen, Richard, thanks all for the pointers. I am confused now;) No, seriously, I need to _test_ those libs, to see what they actually do. I.e. I suspect the "new Dojo Deferreds" with "then" return something different than the broken "then" in jQuery 1.5. This is all messy, personally, I want the Twisted semantics in JS, but want to choose something "mainstream" to make it easy for users to adopt. Well. I'll give me an hour tomorrow to look into Dojo and Mochikit. On 25.08.11 02:01, "Richard Wall" <m-lists@the-moon.net> wrote: On 25 August 2011 00:30, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote: <snip>
I agree. I had the same frustration a few months ago and various Twisted people pointed out missing features in Jquery deferred: * http://twistedmatrix.com/pipermail/twisted-python/2011-June/024146.html Mochikit is the best (and original) JS port of deferred..Dojo and Google Closure both credit Mochikit (and Twisted) in their source code. I'm planning to revert back to it. I guess it would be useful / constructive to compile a list of the weaknesses in the jquery / commonjs deferred API and the reasons and use cases for the Twisted deferred API. Perhaps they can be persuaded to evolve towards the Twisted API in time. -RichardW.
Note that as far as I know, MochiKit isn't actively developed anymore. Google's Closure has a fork of MochiKit's Deferred's, which seems to incorporate some of Dojo's code as well. http://code.google.com/p/closure-library/source/browse/trunk/third_party/clo... On Wed, Aug 24, 2011 at 8:20 PM, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote:
-- Jasper
Hi Jasper, On Wed, 24 Aug 2011 19:48:14 -0500, Jasper St. Pierre <jstpierre@mecheye.net> wrote:
Although it is certainly not "mainstream", like jQuery, or Dojo, or Closure, the most accurate implementation of Deferred in JavaScript is in Nevow.Athena. I thought it might be worth mentioning, even though it isn't part of a widely-adopted JS lib. Hope this helps, L. Daniel Burr
I've been trying to find the code, but I'm too stupid, can't find http://bazaar.launchpad.net/~divmod-dev/divmod.org/trunk/files Where is it? Do you have a pointer to the JS Deferred?
On Thu, Aug 25, 2011 at 10:43 AM, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote:
http://bazaar.launchpad.net/~divmod-dev/divmod.org/trunk/view/head:/Nevow/ne... jml
I have collected the Deferreds from Nevow MochiKit Closure jQuery (and Twisted) here https://github.com/oberstet/jdeferred/tree/master/others for convenient comparison. I'd be interested in baking a JS Deferred which - replicates the Twisted Deferred in semantics and syntax (API) as closely as possible - has no external dependencies (totally self contained) - contains only this: a JS Deferred Anyone else? I know this is kinda OT for this list, but it could serve Twisted indirectly: by propagating it's mature, sane Deferred concept/terminology instead of half-baked, flawned others. Cheers, Tobias
On Thu, Aug 25, 2011 at 4:29 AM, Tobias Oberstein < tobias.oberstein@tavendo.de> wrote:
In some ways the popular JS design is better, since it provides immutable-once-resolved promises. You can have that without breaking chaining though; see how E's promises work. I don't think Twisted's interface for Deferreds is the final answer, but chaining is important to keep :)
I thought I can strictly stick to the advice (only keep one variant -- I thought I keep "lambda"), but today Python surprised me (which happens not so often). Check out these snippets: (1) def get(self, keys): for key in keys: self.call("keyvalue:get", key).addCallback(lambda value: self.show(key, value)) (2) def get(self, keys): for key in keys: self.call("keyvalue:get", key).addCallback(partial(self.show, key)) (3) def get(self, keys): for key in keys: self.call("keyvalue:get", key).addCallback(lambda value, key = key: self.show(key, value)) == (2) and (3) will output the same. Not (1). The reason is, that lambda establishes a closure over the reference "key", which changes with loop iteration, and when the RPCs come back, that "key" will have the last value of the list iterated over. The result of (1) was not what I expected, and it took me some googling to find the answers. (3) is somewhat a hack on lambda to establish closures over the value when the closure is constructed, not the reference. (2) does that by default. I have some background in Erlang, where everything is immutable, and thus the issue is non. This is where the half-functional character of Python has bitten me badly. It may be nothing new to many, but it certainly was for me. http://blog.markfeeney.com/2009/11/python-lambda-partial-and-scopes.html http://stackoverflow.com/questions/938429/scope-of-python-lambda-functions-a... http://math.andrej.com/2009/04/09/pythons-lambda-is-broken/ == Anyway, perhaps it's still easier for users to grasp (3) than having another construct (2). What do you think? Von: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] Im Auftrag von Glyph Lefkowitz Gesendet: Dienstag, 23. August 2011 20:30 An: Twisted general discussion Betreff: Re: [Twisted-Python] RPC design questions On Aug 23, 2011, at 10:37 AM, Tobias Oberstein wrote: class AutobahnDeferred(Deferred): def call(self, *args): return self.addCallback(self.protocol.rcall, *args) Pro: most terse Con: only supports single callback no errback Con: subclassing in general is a bad idea. Subclassing Deferred is an even worse idea. What if Deferred one day gets a new method called 'call' that does something different? Your code would all break. Anything why I shouldn't do? Providing three different ways to do the same thing just so that you can use different syntax depending on your mood is a recipe for making your code hard to read. Pick one style, stick to it as much as possible - consistency is more important than the benefits of any one particular style :). (Except the subclassing one. Don't do that.) -glyph
On Mon, Aug 22, 2011 at 11:09 PM, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote:
self.call(<procedure>, <arg>) self.call("square", 6).addCallback(self.call, "sqrt").addCallback(self.show)
The easiest way to do this is probably: from functools import partial self.call("square", 6).addCallback(partial(self.call, "sqrt")).addCallback(self.show) -- mithrandi, i Ainil en-Balandor, a faer Ambar
Thanks alot! Did not knew about partial .. I checked the solution you proposed: works. It rectifies the arg order, but introduces more boilerplate to write (and making usage convenient is my goal). === In the meantime, I've come up with 3 others options: No chaining (no problem here): self.call("add", 23*23, 5).addCallback(self.show) Chaining Option 1: self.call("square", 23).addCallback(self.rcall, "add", 5).addCallback(self.show) Chaining Option 2: self.call("square", 23).addCallback(lambda res: self.call("add", res, 5)).addCallback(self.show) Chaining Option 3: self.call("square", 23).call("add", 5).addCallback(self.show) Option 1 ======= Makes use of rcall() vs call() where rcall() does the reordering of arguments def rcall(self, *args): a = [] a.append(args[1]) # procedure ID if args[0]: a.append(args[0]) # result from previous deferred a.extend(args[2:]) # new args return self.call(*a) Pro: terse, can handle callback&errback Con: user needs to remember to use rcall() not call() Option 2 ====== Similar to yours .. just using lambda. Pro: standard Python (as yours), can handle callbacks&errbacks Con: verbose Option 3 ======= Makes use of class AutobahnDeferred(Deferred): def call(self, *args): return self.addCallback(self.protocol.rcall, *args) Pro: most terse Con: only supports single callback no errback ==== Currently my thinking is: why not provide all variants? Anything why I shouldn't do?
mithrandi, i Ainil en-Balandor, a faer Ambar
ok. nice;) but what does it mean?
On Aug 23, 2011, at 10:37 AM, Tobias Oberstein wrote:
Con: subclassing in general is a bad idea. Subclassing Deferred is an even worse idea. What if Deferred one day gets a new method called 'call' that does something different? Your code would all break.
Anything why I shouldn't do?
Providing three different ways to do the same thing just so that you can use different syntax depending on your mood is a recipe for making your code hard to read. Pick one style, stick to it as much as possible - consistency is more important than the benefits of any one particular style :). (Except the subclassing one. Don't do that.) -glyph
Thanks for your advice .. that's what I was looking for;) Makes sense, I'll stick to your advice, kick the subclassing approach and the stuff with the reordering version rcall(). That leaves "partial" and "lambda" # Option 1 : prints 31 d2 = self.call("square", 6).addCallback(lambda res: self.call("sub", res, 5)).addCallback(self.show) # Option 3 : prints -31 d4 = self.call("square", 6).addCallback(partial(self.call, "sub", 5)).addCallback(self.show) Lambda is the most general, since it can insert the previous deferred's result anywhere in the next call. Partial only simulates append result to end of args of next call. But lambda is also most verbose. I'll sleep one night and then see if I document and endorse both or only lambda. Obviously, I can't hinder anyone using both. Built in stuff. Thanks again, Tobias Von: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] Im Auftrag von Glyph Lefkowitz Gesendet: Dienstag, 23. August 2011 20:30 An: Twisted general discussion Betreff: Re: [Twisted-Python] RPC design questions On Aug 23, 2011, at 10:37 AM, Tobias Oberstein wrote: class AutobahnDeferred(Deferred): def call(self, *args): return self.addCallback(self.protocol.rcall, *args) Pro: most terse Con: only supports single callback no errback Con: subclassing in general is a bad idea. Subclassing Deferred is an even worse idea. What if Deferred one day gets a new method called 'call' that does something different? Your code would all break. Anything why I shouldn't do? Providing three different ways to do the same thing just so that you can use different syntax depending on your mood is a recipe for making your code hard to read. Pick one style, stick to it as much as possible - consistency is more important than the benefits of any one particular style :). (Except the subclassing one. Don't do that.) -glyph
I follow Glyphs advice and only keep the lambda version with standard Twisted Deferred. However, I've got a (I believe) more serious "design" problem. The RPC stuff works in Twisted, but also from JavaScript. So I have 2 audiences (Twisted and JS developers). On JS, I use the Deferreds that come standard with jQuery since 1.5. Now compare the following Twisted code (self.call does the RPC and returns a Twisted Deferred): # prints 23 d2 = self.call("square", 23).addCallback(lambda res: \ self.call("sqrt", res)).addCallback(self.show) # prints 23 d3 = self.call("square", 23).addCallback(lambda res: \ self.call("sqrt", res).addCallback(self.show)) (note the subtle difference in bracketing) with the following JS // prints 529 sess.call("square", 23).then(function(res) { return sess.call("sqrt", res); }).then(console.log); // prints 23 sess.call("square", 23).then(function(res) { sess.call("sqrt", res).then(console.log); }); === The JS is structurally/syntactically similar to the Py version. However they behave differently. There seems to be a fundamental difference between Twisted and jQuery Deferreds. When calling then() on a jQuery Deferred D, it seems to return D, not any Deferred that might be returned within the handler within then(). http://api.jquery.com/deferred.then Whereas the Twisted Deferred addCallback() returns the inner Deferred. Am I getting something wrong? I'd be glad on any hints/clarification whats going on here .. Tobias == The complete examples are on: JS: https://github.com/oberstet/Autobahn/blob/master/demo/rpc/simple/simple_clie... Py Client: https://github.com/oberstet/Autobahn/blob/master/demo/rpc/simple/simple_clie... Py Server: https://github.com/oberstet/Autobahn/blob/master/demo/rpc/simple/simple_serv... Library: https://github.com/oberstet/Autobahn/tree/master/lib/python => python setup.py install
No, this is something jQuery and CommonJS got wrong; callbacks don't chain in their implementation. Firing a jQuery promise invokes each callback in order with the same argument. Firing a Deferred in Twisted invokes the first callback then passes its return value as the arg to the next callback, and so forth.
Ok. This is most unpleasant. In particular since my options then are: 1) write my own, sane (Twisted like, which I think is the sane way) JS deferred, not use jQuery/CommonJS (which everyone else does) 2) open the door for users running into issues when they do both Twisted & JS/jQuery and don't understand the subtle difference bad. of course not a Twisted mailing list topic anymore .. I'll go to jQuery list. Thanks! Tobias On 25.08.11 00:36, "Allen Short" <washort@twistedmatrix.com> wrote: No, this is something jQuery and CommonJS got wrong; callbacks don't chain in their implementation. Firing a jQuery promise invokes each callback in order with the same argument. Firing a Deferred in Twisted invokes the first callback then passes its return value as the arg to the next callback, and so forth.
Found something: it seems in jQuery 1.6 they have introduced something which corresponds more closely to Twisted Deferreds callback/errback chains: // prints 23 sess.call("square", 23).pipe(function(res) { return sess.call("sqrt", res); }).pipe(console.log); http://api.jquery.com/deferred.pipe/ I am still wondering why they called the other stuff "then()" when it's not then, but "also()" and "pipe()" is the real <then> ;) And now they have then() _and_ pipe(). This is all asking for confusion. - Whatever. Tobias On 25.08.11 00:52, "Tobias Oberstein" <tobias.oberstein@tavendo.de> wrote: Ok. This is most unpleasant. In particular since my options then are: 1) write my own, sane (Twisted like, which I think is the sane way) JS deferred, not use jQuery/CommonJS (which everyone else does) 2) open the door for users running into issues when they do both Twisted & JS/jQuery and don't understand the subtle difference bad. of course not a Twisted mailing list topic anymore .. I'll go to jQuery list. Thanks! Tobias On 25.08.11 00:36, "Allen Short" <washort@twistedmatrix.com> wrote: No, this is something jQuery and CommonJS got wrong; callbacks don't chain in their implementation. Firing a jQuery promise invokes each callback in order with the same argument. Firing a Deferred in Twisted invokes the first callback then passes its return value as the arg to the next callback, and so forth.
My understanding (partially aided by this post: http://www.sitepen.com/blog/2010/05/03/robust-promises-with-dojo-deferred-1-...) is that Dojo provides both twisted-style "addCallback" and CommonJS-style "then" methods.
On 25 August 2011 00:30, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote: <snip>
I agree. I had the same frustration a few months ago and various Twisted people pointed out missing features in Jquery deferred: * http://twistedmatrix.com/pipermail/twisted-python/2011-June/024146.html Mochikit is the best (and original) JS port of deferred..Dojo and Google Closure both credit Mochikit (and Twisted) in their source code. I'm planning to revert back to it. I guess it would be useful / constructive to compile a list of the weaknesses in the jquery / commonjs deferred API and the reasons and use cases for the Twisted deferred API. Perhaps they can be persuaded to evolve towards the Twisted API in time. -RichardW.
Jasper, Allen, Richard, thanks all for the pointers. I am confused now;) No, seriously, I need to _test_ those libs, to see what they actually do. I.e. I suspect the "new Dojo Deferreds" with "then" return something different than the broken "then" in jQuery 1.5. This is all messy, personally, I want the Twisted semantics in JS, but want to choose something "mainstream" to make it easy for users to adopt. Well. I'll give me an hour tomorrow to look into Dojo and Mochikit. On 25.08.11 02:01, "Richard Wall" <m-lists@the-moon.net> wrote: On 25 August 2011 00:30, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote: <snip>
I agree. I had the same frustration a few months ago and various Twisted people pointed out missing features in Jquery deferred: * http://twistedmatrix.com/pipermail/twisted-python/2011-June/024146.html Mochikit is the best (and original) JS port of deferred..Dojo and Google Closure both credit Mochikit (and Twisted) in their source code. I'm planning to revert back to it. I guess it would be useful / constructive to compile a list of the weaknesses in the jquery / commonjs deferred API and the reasons and use cases for the Twisted deferred API. Perhaps they can be persuaded to evolve towards the Twisted API in time. -RichardW.
Note that as far as I know, MochiKit isn't actively developed anymore. Google's Closure has a fork of MochiKit's Deferred's, which seems to incorporate some of Dojo's code as well. http://code.google.com/p/closure-library/source/browse/trunk/third_party/clo... On Wed, Aug 24, 2011 at 8:20 PM, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote:
-- Jasper
Hi Jasper, On Wed, 24 Aug 2011 19:48:14 -0500, Jasper St. Pierre <jstpierre@mecheye.net> wrote:
Although it is certainly not "mainstream", like jQuery, or Dojo, or Closure, the most accurate implementation of Deferred in JavaScript is in Nevow.Athena. I thought it might be worth mentioning, even though it isn't part of a widely-adopted JS lib. Hope this helps, L. Daniel Burr
I've been trying to find the code, but I'm too stupid, can't find http://bazaar.launchpad.net/~divmod-dev/divmod.org/trunk/files Where is it? Do you have a pointer to the JS Deferred?
On Thu, Aug 25, 2011 at 10:43 AM, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote:
http://bazaar.launchpad.net/~divmod-dev/divmod.org/trunk/view/head:/Nevow/ne... jml
I have collected the Deferreds from Nevow MochiKit Closure jQuery (and Twisted) here https://github.com/oberstet/jdeferred/tree/master/others for convenient comparison. I'd be interested in baking a JS Deferred which - replicates the Twisted Deferred in semantics and syntax (API) as closely as possible - has no external dependencies (totally self contained) - contains only this: a JS Deferred Anyone else? I know this is kinda OT for this list, but it could serve Twisted indirectly: by propagating it's mature, sane Deferred concept/terminology instead of half-baked, flawned others. Cheers, Tobias
On Thu, Aug 25, 2011 at 4:29 AM, Tobias Oberstein < tobias.oberstein@tavendo.de> wrote:
In some ways the popular JS design is better, since it provides immutable-once-resolved promises. You can have that without breaking chaining though; see how E's promises work. I don't think Twisted's interface for Deferreds is the final answer, but chaining is important to keep :)
I thought I can strictly stick to the advice (only keep one variant -- I thought I keep "lambda"), but today Python surprised me (which happens not so often). Check out these snippets: (1) def get(self, keys): for key in keys: self.call("keyvalue:get", key).addCallback(lambda value: self.show(key, value)) (2) def get(self, keys): for key in keys: self.call("keyvalue:get", key).addCallback(partial(self.show, key)) (3) def get(self, keys): for key in keys: self.call("keyvalue:get", key).addCallback(lambda value, key = key: self.show(key, value)) == (2) and (3) will output the same. Not (1). The reason is, that lambda establishes a closure over the reference "key", which changes with loop iteration, and when the RPCs come back, that "key" will have the last value of the list iterated over. The result of (1) was not what I expected, and it took me some googling to find the answers. (3) is somewhat a hack on lambda to establish closures over the value when the closure is constructed, not the reference. (2) does that by default. I have some background in Erlang, where everything is immutable, and thus the issue is non. This is where the half-functional character of Python has bitten me badly. It may be nothing new to many, but it certainly was for me. http://blog.markfeeney.com/2009/11/python-lambda-partial-and-scopes.html http://stackoverflow.com/questions/938429/scope-of-python-lambda-functions-a... http://math.andrej.com/2009/04/09/pythons-lambda-is-broken/ == Anyway, perhaps it's still easier for users to grasp (3) than having another construct (2). What do you think? Von: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] Im Auftrag von Glyph Lefkowitz Gesendet: Dienstag, 23. August 2011 20:30 An: Twisted general discussion Betreff: Re: [Twisted-Python] RPC design questions On Aug 23, 2011, at 10:37 AM, Tobias Oberstein wrote: class AutobahnDeferred(Deferred): def call(self, *args): return self.addCallback(self.protocol.rcall, *args) Pro: most terse Con: only supports single callback no errback Con: subclassing in general is a bad idea. Subclassing Deferred is an even worse idea. What if Deferred one day gets a new method called 'call' that does something different? Your code would all break. Anything why I shouldn't do? Providing three different ways to do the same thing just so that you can use different syntax depending on your mood is a recipe for making your code hard to read. Pick one style, stick to it as much as possible - consistency is more important than the benefits of any one particular style :). (Except the subclassing one. Don't do that.) -glyph
participants (8)
-
Allen Short -
Glyph Lefkowitz -
Jasper St. Pierre -
Jonathan Lange -
L. Daniel Burr -
Richard Wall -
Tobias Oberstein -
Tristan Seligmann