Mailman 3 [Twisted-Python] Noob alert - Looking for a framework for creating a long polling service - Twisted

newer
[Twisted-Python] Weekly Bug Summary

[Twisted-Python] Noob alert - Looking for a framework for creating a long polling service

older
[Twisted-Python] session management

Cedric Dugas

31 Mar 2010 31 Mar '10

4:52 p.m.

Hi guys, I am a front-end developer, starting with django with some php in the belt. I am looking for a framework for doing a long polling service where : clients would connect to a soecific 'channel' and wait for a response Where a client admin would push a response only to clients connected to the same channel It's a bit like a chat where everyone listening and only one is talking with multiple channels I am a bit lost in the twisted doc, I know that twisted do a LOT of thing, and was wondering, if twisted is appropriate for this task, and do you have a book to recommend for people like me (by that I mean that I will not be using low level api's) Thank you!

Attachments:

attachment.htm (text/html — 747 bytes)

Show replies by thread

Reza Lotun

31 Mar 31 Mar

8:26 p.m.

New subject: [Twisted-Python] Noob alert - Looking for a framework for creating a long polling service

Hi,

...

I am looking for a framework for doing a long polling service where :

clients would connect to a soecific 'channel' and wait for a response

Where a client admin would push a response only to clients connected to the same channel

There is a project built on top of Twisted that does exactly this, called Orbited (http://orbited.org/). It works, and I use it for internal real-time stats web pages. The one caveat I have with it is that it recently appears to be somewhat inactive - though I use it so much I might fork it and continue development ;-) A good article showing you how to use Orbited is: http://cometdaily.com/2008/10/10/scalable-real-time-web-architecture-part-2-... Other articles are featured on the orbited site. If you're looking for a scalable, long-polling queue broadcaster *in general*, I highly suggest nginx_http_push - an addon module for the popular C-based asynchronous web server nginx (http://pushmodule.slact.net/). Start with Orbited though, and see where it takes you ;-) Cheers, Reza -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: reza@tweetdeck.com twitter: @rlotun

ssteinerX＠gmail.com

8:58 p.m.

New subject: [Twisted-Python] Noob alert - Looking for a framework for creating a long polling service

On Mar 31, 2010, at 4:26 PM, Reza Lotun wrote:

...

Hi,

...
I am looking for a framework for doing a long polling service where :

clients would connect to a soecific 'channel' and wait for a response

Where a client admin would push a response only to clients connected to the same channel

There is a project built on top of Twisted that does exactly this, called Orbited (http://orbited.org/). It works, and I use it for internal real-time stats web pages. The one caveat I have with it is that it recently appears to be somewhat inactive - though I use it so much I might fork it and continue development ;-)

It seems to have completely stalled just before the big 0.8 refactoring effort. Have you continued, or do you plan to continue development where it was left off? They all seem to have gone off to play with node.js instead.

...

If you're looking for a scalable, long-polling queue broadcaster *in general*, I highly suggest nginx_http_push - an addon module for the popular C-based asynchronous web server nginx (http://pushmodule.slact.net/).

Very cool, I hadn't seen that before. S

Donal McMullan

9:48 p.m.

New subject: [Twisted-Python] Noob alert - Looking for a framework for creating a long polling service

There's some simple sample code here: http://zenmachine.wordpress.com/2009/09/19/twisted-and-comet-comet-in-60-sec... The formatting's off, but otherwise it's a succinct demo of the kind of functionality you may be looking for. Cheers D On 1 April 2010 09:58, ssteinerX@gmail.com wrote:

...

On Mar 31, 2010, at 4:26 PM, Reza Lotun wrote:

...
Hi,

...
I am looking for a framework for doing a long polling service where :

clients would connect to a soecific 'channel' and wait for a response

Where a client admin would push a response only to clients connected to the same channel

There is a project built on top of Twisted that does exactly this, called Orbited (http://orbited.org/). It works, and I use it for internal real-time stats web pages. The one caveat I have with it is that it recently appears to be somewhat inactive - though I use it so much I might fork it and continue development ;-)

It seems to have completely stalled just before the big 0.8 refactoring effort.

Have you continued, or do you plan to continue development where it was left off?

They all seem to have gone off to play with node.js instead.

...
If you're looking for a scalable, long-polling queue broadcaster *in general*, I highly suggest nginx_http_push - an addon module for the popular C-based asynchronous web server nginx (http://pushmodule.slact.net/).

Very cool, I hadn't seen that before.

S

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Werner Thie

1 Apr 1 Apr

7:12 a.m.

New subject: [Twisted-Python] Noob alert - Looking for a framework for creating a long polling service

Hi, I'm a long term user of twisted with nevow on top mainly using the athena part with LivePage and LiveElement for RPC between Python and JScript. nevow/athena opens up a whole new world of transparently calling from server to browser and vice versa on a permanent connection. With a slight change (three lines) a nevow/athena 'permanent connection' also survives network outages of up to a minute like the ones experienced in mobile 3G networks. Recently I've combined twisted/nevow with qooxdoo http://www.qooxdoo.org allowing injection/removal of widgets including code into a livepage thus providing not only a 'real app' but a real desktop user experience in a browser. If you want to have a look at some demos please feel free to hit http://nqx.thieprojects.ch minimal nevow/qooxdoo samples http://telinfo.thieprojects.ch Swiss telephone book lookup http://differenzler.thieprojects.ch Minmal Jass cardgame To see code (use Firebug, JScript is not obfuscated) done starting from 2006 till recently. Part of the above mentioned technology was used in a larger card game setup for the Swiss National TV providing a multi player game site with various games of Jass. Currently around 25k games are played by about 70k users a day. http://www.samschtigjass.ch The 'Einzelspieler-Version' can be played without logging in. I've also written experimental stuff bridging XMPP and browser with nevow/athena, using XMPP/pubsub for real time data exchange, though for this I have no public available samples. This is definitely a plug but after four years of working in the field of RIAs it seems to me that nevow/athena is the most advanced of all implementations revolving around 'long polling', permanent connections, websockets and related technologies. It solves a host of problems which you have to tackle when using other technologies. HTH, Werner On 31.03.2010 22:58, ssteinerX@gmail.com wrote:

...

On Mar 31, 2010, at 4:26 PM, Reza Lotun wrote:

...
Hi,

...
I am looking for a framework for doing a long polling service where :

clients would connect to a soecific 'channel' and wait for a response

Where a client admin would push a response only to clients connected to the same channel

There is a project built on top of Twisted that does exactly this, called Orbited (http://orbited.org/). It works, and I use it for internal real-time stats web pages. The one caveat I have with it is that it recently appears to be somewhat inactive - though I use it so much I might fork it and continue development ;-)

It seems to have completely stalled just before the big 0.8 refactoring effort.

Have you continued, or do you plan to continue development where it was left off?

They all seem to have gone off to play with node.js instead.

...
If you're looking for a scalable, long-polling queue broadcaster *in general*, I highly suggest nginx_http_push - an addon module for the popular C-based asynchronous web server nginx (http://pushmodule.slact.net/).

Very cool, I hadn't seen that before.

S

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Reza Lotun

3 Apr 3 Apr

1:03 p.m.

New subject: [Twisted-Python] Noob alert - Looking for a framework for creating a long polling service

Hi Steve,

...

Have you continued, or do you plan to continue development where it was left off?

Well, it's something I've only be toying with. I can handle all the specific Twisted fixes I'd like to make: - handling lost connections gracefully - proper use of twistd and daemonization - massive code cleanups The only problem is that I'm not that knowledgeable about the Javascript browser hacks they've employed - their Orbited.js is more than half "magic" to me (probably due to me not having an opportunity to work on a large js project). It's something that I'd like to get into, if someone is willing to take the lead on the javascript side. It could potentially be as simple as slotting in js.io (http://github.com/mcarter/js.io), which Orbited.js has apparently become, and which appears to have been the plan for Orbited 0.8.

...

They all seem to have gone off to play with node.js instead.

I don't *really* blame them, considering most of the project's merit lay in the Javascript realm, and node.js is basically a (less featurful) version of Twisted for Javascript (of course with the added bonus of using a heavily JITed and optimized Javascript VM).

...

Very cool, I hadn't seen that before.

Yes, very neat. Haven't done much with it, but I think it's a real winner when you're doing something simple like long-polling/ broadcasting from a queue. Cheers, Reza -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: reza@tweetdeck.com twitter: @rlotun

Alex Clemesha

6:37 p.m.

New subject: [Twisted-Python] Noob alert - Looking for a framework for creating a long polling service

On Sat, Apr 3, 2010 at 6:03 AM, Reza Lotun wrote:

...

Hi Steve,

...
Have you continued, or do you plan to continue development where it was left off?

Well, it's something I've only be toying with. I can handle all the specific Twisted fixes I'd like to make: - handling lost connections gracefully - proper use of twistd and daemonization - massive code cleanups

I would very much support you continuing development on Orbited. In fact, I believe I can help, or already have patches for most of the above issues you mention. A lot of this is result of my "Hotdot" project, "Create realtime webapps using Django + Orbited + Twisted": http://github.com/clemesha/hotdot Specifically, see here for Orbited and twistd working together: http://github.com/clemesha/hotdot/blob/master/server.py and see the below for patches that improve handling lost connections gracefully: http://gist.github.com/256582 Finally, it would be awesome if you put up your improvements on Github (just a suggestion :-), then I can easily fork and starting adding improvements, and you can pull and we can move this forward. -Alex p.s. I haven't found many issues with the frontend JavaScript code, but if you have specific issues that you could point out, I'd love to try to help.

...

The only problem is that I'm not that knowledgeable about the Javascript browser hacks they've employed - their Orbited.js is more than half "magic" to me (probably due to me not having an opportunity to work on a large js project). It's something that I'd like to get into, if someone is willing to take the lead on the javascript side. It could potentially be as simple as slotting in js.io (http://github.com/mcarter/js.io), which Orbited.js has apparently become, and which appears to have been the plan for Orbited 0.8.

...
They all seem to have gone off to play with node.js instead.

I don't *really* blame them, considering most of the project's merit lay in the Javascript realm, and node.js is basically a (less featurful) version of Twisted for Javascript (of course with the added bonus of using a heavily JITed and optimized Javascript VM).

...
Very cool, I hadn't seen that before.

Yes, very neat. Haven't done much with it, but I think it's a real winner when you're doing something simple like long-polling/ broadcasting from a queue.

Cheers, Reza

-- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: reza@tweetdeck.com twitter: @rlotun

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

-- Alex Clemesha clemesha.org

ssteinerX＠gmail.com

7:10 p.m.

New subject: [Twisted-Python] Noob alert - Looking for a framework for creating a long polling service

On Apr 3, 2010, at 2:37 PM, Alex Clemesha wrote:

...

On Sat, Apr 3, 2010 at 6:03 AM, Reza Lotun wrote:

...
Hi Steve,

...
Have you continued, or do you plan to continue development where it was left off?

Well, it's something I've only be toying with. I can handle all the specific Twisted fixes I'd like to make: - handling lost connections gracefully - proper use of twistd and daemonization - massive code cleanups

I would very much support you continuing development on Orbited. In fact, I believe I can help, or already have patches for most of the above issues you mention. A lot of this is result of my "Hotdot" project, "Create realtime webapps using Django + Orbited + Twisted": http://github.com/clemesha/hotdot

Specifically, see here for Orbited and twistd working together: http://github.com/clemesha/hotdot/blob/master/server.py

and see the below for patches that improve handling lost connections gracefully: http://gist.github.com/256582

Finally, it would be awesome if you put up your improvements on Github (just a suggestion :-), then I can easily fork and starting adding improvements, and you can pull and we can move this forward.

I had worked with your hotdot stuff a while ago, and I think I had even sent you some django csrf patches for Django 1.2 compatibility (it was a while ago...) Back then, I had started on refactoring the one-big-javascript file in Orbited a few months ago but, at the time, was assured that 0.8 was "just around the corner", so I left it alone. Well...now I have need and, since Athena seems to be tightly tied to Nevow, cometd is itself, websocket is websocket, so I'm looking again. Since Orbited seems to have gone into "unmaintained mode", I agree, it may just be time for a fork.

...

p.s. I haven't found many issues with the frontend JavaScript code, but if you have specific issues that you could point out, I'd love to try to help.

Mostly the problems I had with it were the lack of doc, and the all-in-one 2700 line .js file, and the complete lack of unit tests of any type.

...

...
The only problem is that I'm not that knowledgeable about the Javascript browser hacks they've employed - their Orbited.js is more than half "magic" to me (probably due to me not having an opportunity to work on a large js project). It's something that I'd like to get into, if someone is willing to take the lead on the javascript side. It could potentially be as simple as slotting in js.io (http://github.com/mcarter/js.io), which Orbited.js has apparently become, and which appears to have been the plan for Orbited 0.8.

The js.io site documentation link says "Sorry, we don't have documentation yet", and the doc directory on github has a powerpoint file and an echo server so I'm not sure that's going to help so much.

...

...
...
They all seem to have gone off to play with node.js instead.

I don't *really* blame them, considering most of the project's merit lay in the Javascript realm, and node.js is basically a (less featurful) version of Twisted for Javascript (of course with the added bonus of using a heavily JITed and optimized Javascript VM).

...
Very cool, I hadn't seen that before.

Yes, very neat. Haven't done much with it, but I think it's a real winner when you're doing something simple like long-polling/ broadcasting from a queue.

Well, if there's interest on reviving Orbited either as itself, or as a new project with some code from Orbited as well as some of Alex's stuff, I'm in. S

ssteinerX＠gmail.com

4 Apr 4 Apr

12:24 a.m.

New subject: [Twisted-Python] Noob alert - Looking for a framework for creating a long polling service, Forking Orbited

On Apr 3, 2010, at 3:10 PM, ssteinerX@gmail.com wrote:

...

On Apr 3, 2010, at 2:37 PM, Alex Clemesha wrote:

...
On Sat, Apr 3, 2010 at 6:03 AM, Reza Lotun wrote:

...
Hi Steve,

...
Have you continued, or do you plan to continue development where it was left off?

Well, it's something I've only be toying with. I can handle all the specific Twisted fixes I'd like to make: - handling lost connections gracefully - proper use of twistd and daemonization - massive code cleanups

I would very much support you continuing development on Orbited. In fact, I believe I can help, or already have patches for most of the above issues you mention. A lot of this is result of my "Hotdot" project, "Create realtime webapps using Django + Orbited + Twisted": http://github.com/clemesha/hotdot

I've cloned orbited to BitBucket to take up a fork of the project. http://bitbucket.org/ssteinerx/orbited-new/overview/ I really need this for a project and since the old Orbited seems to be abandoned, I'm taking it on. Contributions gladly accepted. If you're interested in working on it, I'll be happy to give you write access. I've sent an e-mail to CarterMichael@gmail.com to see whether he'll give an official blessing to this fork once we've moved it forward a bit. Thanks, S

5135

Age (days ago)

5139

Last active (days ago)

List overview

Download

8 comments

6 participants

participants (6)

Alex Clemesha
Cedric Dugas
Donal McMullan
Reza Lotun
ssteinerX＠gmail.com
Werner Thie

[Twisted-Python] Noob alert - Looking for a framework for creating a long polling service

Cedric Dugas

Reza Lotun

ssteinerX＠gmail.com

Donal McMullan

Werner Thie

Reza Lotun

Alex Clemesha

ssteinerX＠gmail.com

ssteinerX＠gmail.com

tags

participants (6)