[Twisted-Python] Noob alert - Looking for a framework for creating a long polling service
Hi guys, I am a front-end developer, starting with django with some php in the belt. I am looking for a framework for doing a long polling service where : clients would connect to a soecific 'channel' and wait for a response Where a client admin would push a response only to clients connected to the same channel It's a bit like a chat where everyone listening and only one is talking with multiple channels I am a bit lost in the twisted doc, I know that twisted do a LOT of thing, and was wondering, if twisted is appropriate for this task, and do you have a book to recommend for people like me (by that I mean that I will not be using low level api's) Thank you!
Hi,
I am looking for a framework for doing a long polling service where :
clients would connect to a soecific 'channel' and wait for a response
Where a client admin would push a response only to clients connected to the same channel
There is a project built on top of Twisted that does exactly this, called Orbited (http://orbited.org/). It works, and I use it for internal real-time stats web pages. The one caveat I have with it is that it recently appears to be somewhat inactive - though I use it so much I might fork it and continue development ;-) A good article showing you how to use Orbited is: http://cometdaily.com/2008/10/10/scalable-real-time-web-architecture-part-2-... Other articles are featured on the orbited site. If you're looking for a scalable, long-polling queue broadcaster *in general*, I highly suggest nginx_http_push - an addon module for the popular C-based asynchronous web server nginx (http://pushmodule.slact.net/). Start with Orbited though, and see where it takes you ;-) Cheers, Reza -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: reza@tweetdeck.com twitter: @rlotun
On Mar 31, 2010, at 4:26 PM, Reza Lotun wrote:
Hi,
I am looking for a framework for doing a long polling service where :
clients would connect to a soecific 'channel' and wait for a response
Where a client admin would push a response only to clients connected to the same channel
There is a project built on top of Twisted that does exactly this, called Orbited (http://orbited.org/). It works, and I use it for internal real-time stats web pages. The one caveat I have with it is that it recently appears to be somewhat inactive - though I use it so much I might fork it and continue development ;-)
It seems to have completely stalled just before the big 0.8 refactoring effort. Have you continued, or do you plan to continue development where it was left off? They all seem to have gone off to play with node.js instead.
If you're looking for a scalable, long-polling queue broadcaster *in general*, I highly suggest nginx_http_push - an addon module for the popular C-based asynchronous web server nginx (http://pushmodule.slact.net/).
Very cool, I hadn't seen that before. S
There's some simple sample code here: http://zenmachine.wordpress.com/2009/09/19/twisted-and-comet-comet-in-60-sec... The formatting's off, but otherwise it's a succinct demo of the kind of functionality you may be looking for. Cheers D On 1 April 2010 09:58, ssteinerX@gmail.com <ssteinerx@gmail.com> wrote:
On Mar 31, 2010, at 4:26 PM, Reza Lotun wrote:
Hi,
I am looking for a framework for doing a long polling service where :
clients would connect to a soecific 'channel' and wait for a response
Where a client admin would push a response only to clients connected to the same channel
There is a project built on top of Twisted that does exactly this, called Orbited (http://orbited.org/). It works, and I use it for internal real-time stats web pages. The one caveat I have with it is that it recently appears to be somewhat inactive - though I use it so much I might fork it and continue development ;-)
It seems to have completely stalled just before the big 0.8 refactoring effort.
Have you continued, or do you plan to continue development where it was left off?
They all seem to have gone off to play with node.js instead.
If you're looking for a scalable, long-polling queue broadcaster *in general*, I highly suggest nginx_http_push - an addon module for the popular C-based asynchronous web server nginx (http://pushmodule.slact.net/).
Very cool, I hadn't seen that before.
S
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Hi, I'm a long term user of twisted with nevow on top mainly using the athena part with LivePage and LiveElement for RPC between Python and JScript. nevow/athena opens up a whole new world of transparently calling from server to browser and vice versa on a permanent connection. With a slight change (three lines) a nevow/athena 'permanent connection' also survives network outages of up to a minute like the ones experienced in mobile 3G networks. Recently I've combined twisted/nevow with qooxdoo http://www.qooxdoo.org allowing injection/removal of widgets including code into a livepage thus providing not only a 'real app' but a real desktop user experience in a browser. If you want to have a look at some demos please feel free to hit http://nqx.thieprojects.ch minimal nevow/qooxdoo samples http://telinfo.thieprojects.ch Swiss telephone book lookup http://differenzler.thieprojects.ch Minmal Jass cardgame To see code (use Firebug, JScript is not obfuscated) done starting from 2006 till recently. Part of the above mentioned technology was used in a larger card game setup for the Swiss National TV providing a multi player game site with various games of Jass. Currently around 25k games are played by about 70k users a day. http://www.samschtigjass.ch The 'Einzelspieler-Version' can be played without logging in. I've also written experimental stuff bridging XMPP and browser with nevow/athena, using XMPP/pubsub for real time data exchange, though for this I have no public available samples. This is definitely a plug but after four years of working in the field of RIAs it seems to me that nevow/athena is the most advanced of all implementations revolving around 'long polling', permanent connections, websockets and related technologies. It solves a host of problems which you have to tackle when using other technologies. HTH, Werner On 31.03.2010 22:58, ssteinerX@gmail.com wrote:
On Mar 31, 2010, at 4:26 PM, Reza Lotun wrote:
Hi,
I am looking for a framework for doing a long polling service where :
clients would connect to a soecific 'channel' and wait for a response
Where a client admin would push a response only to clients connected to the same channel
There is a project built on top of Twisted that does exactly this, called Orbited (http://orbited.org/). It works, and I use it for internal real-time stats web pages. The one caveat I have with it is that it recently appears to be somewhat inactive - though I use it so much I might fork it and continue development ;-)
It seems to have completely stalled just before the big 0.8 refactoring effort.
Have you continued, or do you plan to continue development where it was left off?
They all seem to have gone off to play with node.js instead.
If you're looking for a scalable, long-polling queue broadcaster *in general*, I highly suggest nginx_http_push - an addon module for the popular C-based asynchronous web server nginx (http://pushmodule.slact.net/).
Very cool, I hadn't seen that before.
S
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Hi Steve,
Have you continued, or do you plan to continue development where it was left off?
Well, it's something I've only be toying with. I can handle all the specific Twisted fixes I'd like to make: - handling lost connections gracefully - proper use of twistd and daemonization - massive code cleanups The only problem is that I'm not that knowledgeable about the Javascript browser hacks they've employed - their Orbited.js is more than half "magic" to me (probably due to me not having an opportunity to work on a large js project). It's something that I'd like to get into, if someone is willing to take the lead on the javascript side. It could potentially be as simple as slotting in js.io (http://github.com/mcarter/js.io), which Orbited.js has apparently become, and which appears to have been the plan for Orbited 0.8.
They all seem to have gone off to play with node.js instead.
I don't *really* blame them, considering most of the project's merit lay in the Javascript realm, and node.js is basically a (less featurful) version of Twisted for Javascript (of course with the added bonus of using a heavily JITed and optimized Javascript VM).
Very cool, I hadn't seen that before.
Yes, very neat. Haven't done much with it, but I think it's a real winner when you're doing something simple like long-polling/ broadcasting from a queue. Cheers, Reza -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: reza@tweetdeck.com twitter: @rlotun
On Sat, Apr 3, 2010 at 6:03 AM, Reza Lotun <rlotun@gmail.com> wrote:
Hi Steve,
Have you continued, or do you plan to continue development where it was left off?
Well, it's something I've only be toying with. I can handle all the specific Twisted fixes I'd like to make: - handling lost connections gracefully - proper use of twistd and daemonization - massive code cleanups
I would very much support you continuing development on Orbited. In fact, I believe I can help, or already have patches for most of the above issues you mention. A lot of this is result of my "Hotdot" project, "Create realtime webapps using Django + Orbited + Twisted": http://github.com/clemesha/hotdot Specifically, see here for Orbited and twistd working together: http://github.com/clemesha/hotdot/blob/master/server.py and see the below for patches that improve handling lost connections gracefully: http://gist.github.com/256582 Finally, it would be awesome if you put up your improvements on Github (just a suggestion :-), then I can easily fork and starting adding improvements, and you can pull and we can move this forward. -Alex p.s. I haven't found many issues with the frontend JavaScript code, but if you have specific issues that you could point out, I'd love to try to help.
The only problem is that I'm not that knowledgeable about the Javascript browser hacks they've employed - their Orbited.js is more than half "magic" to me (probably due to me not having an opportunity to work on a large js project). It's something that I'd like to get into, if someone is willing to take the lead on the javascript side. It could potentially be as simple as slotting in js.io (http://github.com/mcarter/js.io), which Orbited.js has apparently become, and which appears to have been the plan for Orbited 0.8.
They all seem to have gone off to play with node.js instead.
I don't *really* blame them, considering most of the project's merit lay in the Javascript realm, and node.js is basically a (less featurful) version of Twisted for Javascript (of course with the added bonus of using a heavily JITed and optimized Javascript VM).
Very cool, I hadn't seen that before.
Yes, very neat. Haven't done much with it, but I think it's a real winner when you're doing something simple like long-polling/ broadcasting from a queue.
Cheers, Reza
-- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: reza@tweetdeck.com twitter: @rlotun
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- Alex Clemesha clemesha.org
On Apr 3, 2010, at 2:37 PM, Alex Clemesha wrote:
On Sat, Apr 3, 2010 at 6:03 AM, Reza Lotun <rlotun@gmail.com> wrote:
Hi Steve,
Have you continued, or do you plan to continue development where it was left off?
Well, it's something I've only be toying with. I can handle all the specific Twisted fixes I'd like to make: - handling lost connections gracefully - proper use of twistd and daemonization - massive code cleanups
I would very much support you continuing development on Orbited. In fact, I believe I can help, or already have patches for most of the above issues you mention. A lot of this is result of my "Hotdot" project, "Create realtime webapps using Django + Orbited + Twisted": http://github.com/clemesha/hotdot
Specifically, see here for Orbited and twistd working together: http://github.com/clemesha/hotdot/blob/master/server.py
and see the below for patches that improve handling lost connections gracefully: http://gist.github.com/256582
Finally, it would be awesome if you put up your improvements on Github (just a suggestion :-), then I can easily fork and starting adding improvements, and you can pull and we can move this forward.
I had worked with your hotdot stuff a while ago, and I think I had even sent you some django csrf patches for Django 1.2 compatibility (it was a while ago...) Back then, I had started on refactoring the one-big-javascript file in Orbited a few months ago but, at the time, was assured that 0.8 was "just around the corner", so I left it alone. Well...now I have need and, since Athena seems to be tightly tied to Nevow, cometd is itself, websocket is websocket, so I'm looking again. Since Orbited seems to have gone into "unmaintained mode", I agree, it may just be time for a fork.
p.s. I haven't found many issues with the frontend JavaScript code, but if you have specific issues that you could point out, I'd love to try to help.
Mostly the problems I had with it were the lack of doc, and the all-in-one 2700 line .js file, and the complete lack of unit tests of any type.
The only problem is that I'm not that knowledgeable about the Javascript browser hacks they've employed - their Orbited.js is more than half "magic" to me (probably due to me not having an opportunity to work on a large js project). It's something that I'd like to get into, if someone is willing to take the lead on the javascript side. It could potentially be as simple as slotting in js.io (http://github.com/mcarter/js.io), which Orbited.js has apparently become, and which appears to have been the plan for Orbited 0.8.
The js.io site documentation link says "Sorry, we don't have documentation yet", and the doc directory on github has a powerpoint file and an echo server so I'm not sure that's going to help so much.
They all seem to have gone off to play with node.js instead.
I don't *really* blame them, considering most of the project's merit lay in the Javascript realm, and node.js is basically a (less featurful) version of Twisted for Javascript (of course with the added bonus of using a heavily JITed and optimized Javascript VM).
Very cool, I hadn't seen that before.
Yes, very neat. Haven't done much with it, but I think it's a real winner when you're doing something simple like long-polling/ broadcasting from a queue.
Well, if there's interest on reviving Orbited either as itself, or as a new project with some code from Orbited as well as some of Alex's stuff, I'm in. S
On Apr 3, 2010, at 3:10 PM, ssteinerX@gmail.com wrote:
On Apr 3, 2010, at 2:37 PM, Alex Clemesha wrote:
On Sat, Apr 3, 2010 at 6:03 AM, Reza Lotun <rlotun@gmail.com> wrote:
Hi Steve,
Have you continued, or do you plan to continue development where it was left off?
Well, it's something I've only be toying with. I can handle all the specific Twisted fixes I'd like to make: - handling lost connections gracefully - proper use of twistd and daemonization - massive code cleanups
I would very much support you continuing development on Orbited. In fact, I believe I can help, or already have patches for most of the above issues you mention. A lot of this is result of my "Hotdot" project, "Create realtime webapps using Django + Orbited + Twisted": http://github.com/clemesha/hotdot
I've cloned orbited to BitBucket to take up a fork of the project. http://bitbucket.org/ssteinerx/orbited-new/overview/ I really need this for a project and since the old Orbited seems to be abandoned, I'm taking it on. Contributions gladly accepted. If you're interested in working on it, I'll be happy to give you write access. I've sent an e-mail to CarterMichael@gmail.com to see whether he'll give an official blessing to this fork once we've moved it forward a bit. Thanks, S
participants (6)
-
Alex Clemesha -
Cedric Dugas -
Donal McMullan -
Reza Lotun -
ssteinerX＠gmail.com -
Werner Thie