[Twisted-Python] Protocol for socket.io
![](https://secure.gravatar.com/avatar/917e9407f1027d3e6e4bc1a16d49b632.jpg?s=120&d=mm&r=g)
Hi: I searched the archive but couldn't find an answer so here goes: I am also new to all this I need to write a server that receives socket.io requests. I see Twisted has support for WebSocket. I also see that the Tornado2 socket.io is used alongside Twisted (I didn't know that this was possible). I read that socket.io uses WebSocket whenever possible. I have just started to look at the js implementation of socket.io and the specification. I am trying to gauge how hard is it to write a socket.io protocol/handler. Am I barking up the wrong tree? Or do I really need to learn Tornado? Cheers, Andrew
![](https://secure.gravatar.com/avatar/1846c8040fcf70e9b55bb7bfcdb78bc4.jpg?s=120&d=mm&r=g)
Hi Andrew, No idea about socket.io, but if you're free to pick whatever you want, and you just want "websockets, damnit!" then sockjs is actually better, and has excellent twisted support. hth lvh
![](https://secure.gravatar.com/avatar/f8d14732ac045849989e5236418a547e.jpg?s=120&d=mm&r=g)
Laurens Van Houtven <_ <at> lvh.io> writes:
Hi Andrew, No idea about socket.io, but if you're free to pick whatever you want, and
you just want "websockets, damnit!" then sockjs is actually better, and has excellent twisted support.hthlvh Agreed on using sockjs. There are at least two sockjs server implementation using twisted that i know of: sockjs-twisted [1] and (shameless plug) sockjs-cyclone [2] As per rpc and pubsub support i have personally used wamp [3] and websocket- multiplex [4] with great success. Hope this helps, Flavio [1] https://github.com/DesertBus/sockjs-twisted [2] https://github.com/flaviogrossi/sockjs-cyclone [3] http://wamp.ws/ [4] https://github.com/sockjs/websocket-multiplex
![](https://secure.gravatar.com/avatar/917e9407f1027d3e6e4bc1a16d49b632.jpg?s=120&d=mm&r=g)
Hi Laurens and Folks: Thanks for all the replies. I'll try desert-sockjs first. Looking through the documentation, it is not clear to me how I would handle socket.io events (unfortunately I am new to socket.io). There doesn't seem to be an event decorator like Socket.io for Tornadio2. I am looking the documentation describing what an event looks like at the HTTP level. Cheers, Andrew On Saturday, January 25, 2014 1:52 PM, Laurens Van Houtven <_@lvh.io> wrote: Hi Andrew, No idea about socket.io, but if you're free to pick whatever you want, and you just want "websockets, damnit!" then sockjs is actually better, and has excellent twisted support. hth lvh
![](https://secure.gravatar.com/avatar/1846c8040fcf70e9b55bb7bfcdb78bc4.jpg?s=120&d=mm&r=g)
Hi Andrew, SockJS is a different project from socket.io; it just tries to solve a similar problem (WebSockets everywhere). If you literally need socket.io and not a similar thing in the same problem space, then SockJS will not help you. You can read about the spec for SockJS here: https://github.com/sockjs/sockjs-protocol hth lvh
![](https://secure.gravatar.com/avatar/96342dbb350e3be883a4cb2f7c3f2423.jpg?s=120&d=mm&r=g)
As I understand it, socket.io is a browser-side library that uses any of a number of technologies, depending on its environment, to communicate with the server. SSE, Websockets, and long polling are all on the list. Last I looked, though, I didn't see any support for the negotiation of which of these protocols to use with a Twisted server. If that's changed, I'd certainly be interested! Dustin On Sat, Jan 25, 2014 at 1:42 PM, Andrew Francis <andrewfr_ice@yahoo.com> wrote:
![](https://secure.gravatar.com/avatar/235f5f4ffd3736018800ef8064551a3f.jpg?s=120&d=mm&r=g)
Hello, I looked for something that fit your needs few days ago and i came across txsockjs and i create a server based on twisted and the https://pypi.python.org/pypi/txsockjs module and i made it in few minutes. really good stuff. Of course the client part have to be written in javascript with sockjs object/api. If you are sure that you just need websocket and websocket only, there are also some pure "websocket" implementation in twisted, or you can probably write your own following the RFC. For you information there is a kind of flame war on socket.io vs sockjs, i am not sure which one won the fight, but from my read, and only from what i have read, sockjs is more promissing and more supported and more actively maintained. But as i said and based on my experience and preference only, i did a lot of research about what i could use to write web app with websocket support + ajax/comet/long polling technologies and i choose txsockjs. regards. Sofiane Akermoun 2014-01-25 Dustin J. Mitchell <dustin@v.igoro.us>:
-- Sofiane AKERMOUN akersof@gmail.com
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Jan 25, 2014, at 12:21 PM, Dustin J. Mitchell <dustin@v.igoro.us> wrote:
It would be great if someone built a package to do this (more interop is always better), but I think interest has generally fizzled on Socket.IO implementations in the past because SockJS does pretty much exactly the same thing, and as Laurens said, our friends over at Desert Bus for Hope maintain great Twisted support for it <https://github.com/DesertBus/sockjs-twisted> <https://pypi.python.org/pypi/txsockjs>. It even has endpoints support, including a string parser! <https://github.com/DesertBus/sockjs-twisted#endpoints>. Socket.IO is just a negotiation protocol for deciding whether to use websockets though. If you started with Autobahn I can't imagine it would be more than a day or two of work to implement. -glyph
![](https://secure.gravatar.com/avatar/1846c8040fcf70e9b55bb7bfcdb78bc4.jpg?s=120&d=mm&r=g)
On Sun, Jan 26, 2014 at 10:46 AM, Glyph <glyph@twistedmatrix.com> wrote:
socket.io adds a few things like "RPC support" and "pubsub", both those are things that should be spoken over some transport, not part of some transport, and are trivial to layer over an existing browser transport. I like SockJS because it's literally just "websockets damnit!". Also, it doesn't use Flash, which has excellent benefits including no gnarly XML files to server and not having to wait three seconds to figure out if a connection works or not (yes, really). Also: protip, speak SockJS over TLS exclusively. It's not even a security thing (except yes, that too), but if you do end up speaking websockets you'd be amazed how much Enterprise Quality Web Middleware Server Hardware Firewall(TM) screws up websocket connections. TLS, OTOH, they just leave be. hth lvh
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Jan 26, 2014, at 1:53 AM, Laurens Van Houtven <_@lvh.io> wrote:
you'd be amazed how much Enterprise Quality Web Middleware Server Hardware Firewall(TM) screws up websocket connections
Having had the life-transforming experience of supporting customers using Government-Quality Web Middleware Server Hardware Firewall™, in a pre-websockets world, with a two-way web application... No, no I probably wouldn't. -glyph
![](https://secure.gravatar.com/avatar/83111e3ce141b383d5169be6edd768d1.jpg?s=120&d=mm&r=g)
socket.io adds a few things like "RPC support" and "pubsub", ...
Do you have pointers for that? I can see socket.io supports "topic based pubsub" like functionality via "rooms" : https://github.com/LearnBoost/socket.io/wiki/Rooms What about RPC? /Tobias
![](https://secure.gravatar.com/avatar/1846c8040fcf70e9b55bb7bfcdb78bc4.jpg?s=120&d=mm&r=g)
Hi Tobias, Hm. I can't find it in the current version. Either I am misremembering, or they expect you to use the pubsub thing as an RPC mechanism somehow (i.e. the room name is the procedure name). hth lvh
![](https://secure.gravatar.com/avatar/83111e3ce141b383d5169be6edd768d1.jpg?s=120&d=mm&r=g)
Hi Laurens,
Hm. I can't find it in the current version. Either I am misremembering, or they expect you to use the pubsub thing as an RPC mechanism somehow (i.e. the room name is the procedure name).
I see. Using PubSub to implement RPC seems hackish at best, if it works at all. E.g. how to make sure only 1 implementing endpoint subscribes to the Room (to receive procedure calls), how to direct call results to calling clients (since only the caller should receive his result), etc etc Unsurprisingly (probably), I'd like to point to WAMP: http://wamp.ws/ Which was designed from the ground up for exactly this: 1 protocol that covers _both_ RPC and PubSub, and runs over WebSocket. Autobahn|Python implements both WebSocket and WAMP (https://github.com/tavendo/AutobahnPython) End of marketing message;) /Tobias
![](https://secure.gravatar.com/avatar/1846c8040fcf70e9b55bb7bfcdb78bc4.jpg?s=120&d=mm&r=g)
Hi Andrew, No idea about socket.io, but if you're free to pick whatever you want, and you just want "websockets, damnit!" then sockjs is actually better, and has excellent twisted support. hth lvh
![](https://secure.gravatar.com/avatar/f8d14732ac045849989e5236418a547e.jpg?s=120&d=mm&r=g)
Laurens Van Houtven <_ <at> lvh.io> writes:
Hi Andrew, No idea about socket.io, but if you're free to pick whatever you want, and
you just want "websockets, damnit!" then sockjs is actually better, and has excellent twisted support.hthlvh Agreed on using sockjs. There are at least two sockjs server implementation using twisted that i know of: sockjs-twisted [1] and (shameless plug) sockjs-cyclone [2] As per rpc and pubsub support i have personally used wamp [3] and websocket- multiplex [4] with great success. Hope this helps, Flavio [1] https://github.com/DesertBus/sockjs-twisted [2] https://github.com/flaviogrossi/sockjs-cyclone [3] http://wamp.ws/ [4] https://github.com/sockjs/websocket-multiplex
![](https://secure.gravatar.com/avatar/917e9407f1027d3e6e4bc1a16d49b632.jpg?s=120&d=mm&r=g)
Hi Laurens and Folks: Thanks for all the replies. I'll try desert-sockjs first. Looking through the documentation, it is not clear to me how I would handle socket.io events (unfortunately I am new to socket.io). There doesn't seem to be an event decorator like Socket.io for Tornadio2. I am looking the documentation describing what an event looks like at the HTTP level. Cheers, Andrew On Saturday, January 25, 2014 1:52 PM, Laurens Van Houtven <_@lvh.io> wrote: Hi Andrew, No idea about socket.io, but if you're free to pick whatever you want, and you just want "websockets, damnit!" then sockjs is actually better, and has excellent twisted support. hth lvh
![](https://secure.gravatar.com/avatar/1846c8040fcf70e9b55bb7bfcdb78bc4.jpg?s=120&d=mm&r=g)
Hi Andrew, SockJS is a different project from socket.io; it just tries to solve a similar problem (WebSockets everywhere). If you literally need socket.io and not a similar thing in the same problem space, then SockJS will not help you. You can read about the spec for SockJS here: https://github.com/sockjs/sockjs-protocol hth lvh
![](https://secure.gravatar.com/avatar/96342dbb350e3be883a4cb2f7c3f2423.jpg?s=120&d=mm&r=g)
As I understand it, socket.io is a browser-side library that uses any of a number of technologies, depending on its environment, to communicate with the server. SSE, Websockets, and long polling are all on the list. Last I looked, though, I didn't see any support for the negotiation of which of these protocols to use with a Twisted server. If that's changed, I'd certainly be interested! Dustin On Sat, Jan 25, 2014 at 1:42 PM, Andrew Francis <andrewfr_ice@yahoo.com> wrote:
![](https://secure.gravatar.com/avatar/235f5f4ffd3736018800ef8064551a3f.jpg?s=120&d=mm&r=g)
Hello, I looked for something that fit your needs few days ago and i came across txsockjs and i create a server based on twisted and the https://pypi.python.org/pypi/txsockjs module and i made it in few minutes. really good stuff. Of course the client part have to be written in javascript with sockjs object/api. If you are sure that you just need websocket and websocket only, there are also some pure "websocket" implementation in twisted, or you can probably write your own following the RFC. For you information there is a kind of flame war on socket.io vs sockjs, i am not sure which one won the fight, but from my read, and only from what i have read, sockjs is more promissing and more supported and more actively maintained. But as i said and based on my experience and preference only, i did a lot of research about what i could use to write web app with websocket support + ajax/comet/long polling technologies and i choose txsockjs. regards. Sofiane Akermoun 2014-01-25 Dustin J. Mitchell <dustin@v.igoro.us>:
-- Sofiane AKERMOUN akersof@gmail.com
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Jan 25, 2014, at 12:21 PM, Dustin J. Mitchell <dustin@v.igoro.us> wrote:
It would be great if someone built a package to do this (more interop is always better), but I think interest has generally fizzled on Socket.IO implementations in the past because SockJS does pretty much exactly the same thing, and as Laurens said, our friends over at Desert Bus for Hope maintain great Twisted support for it <https://github.com/DesertBus/sockjs-twisted> <https://pypi.python.org/pypi/txsockjs>. It even has endpoints support, including a string parser! <https://github.com/DesertBus/sockjs-twisted#endpoints>. Socket.IO is just a negotiation protocol for deciding whether to use websockets though. If you started with Autobahn I can't imagine it would be more than a day or two of work to implement. -glyph
![](https://secure.gravatar.com/avatar/1846c8040fcf70e9b55bb7bfcdb78bc4.jpg?s=120&d=mm&r=g)
On Sun, Jan 26, 2014 at 10:46 AM, Glyph <glyph@twistedmatrix.com> wrote:
socket.io adds a few things like "RPC support" and "pubsub", both those are things that should be spoken over some transport, not part of some transport, and are trivial to layer over an existing browser transport. I like SockJS because it's literally just "websockets damnit!". Also, it doesn't use Flash, which has excellent benefits including no gnarly XML files to server and not having to wait three seconds to figure out if a connection works or not (yes, really). Also: protip, speak SockJS over TLS exclusively. It's not even a security thing (except yes, that too), but if you do end up speaking websockets you'd be amazed how much Enterprise Quality Web Middleware Server Hardware Firewall(TM) screws up websocket connections. TLS, OTOH, they just leave be. hth lvh
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
On Jan 26, 2014, at 1:53 AM, Laurens Van Houtven <_@lvh.io> wrote:
you'd be amazed how much Enterprise Quality Web Middleware Server Hardware Firewall(TM) screws up websocket connections
Having had the life-transforming experience of supporting customers using Government-Quality Web Middleware Server Hardware Firewall™, in a pre-websockets world, with a two-way web application... No, no I probably wouldn't. -glyph
![](https://secure.gravatar.com/avatar/83111e3ce141b383d5169be6edd768d1.jpg?s=120&d=mm&r=g)
socket.io adds a few things like "RPC support" and "pubsub", ...
Do you have pointers for that? I can see socket.io supports "topic based pubsub" like functionality via "rooms" : https://github.com/LearnBoost/socket.io/wiki/Rooms What about RPC? /Tobias
![](https://secure.gravatar.com/avatar/1846c8040fcf70e9b55bb7bfcdb78bc4.jpg?s=120&d=mm&r=g)
Hi Tobias, Hm. I can't find it in the current version. Either I am misremembering, or they expect you to use the pubsub thing as an RPC mechanism somehow (i.e. the room name is the procedure name). hth lvh
![](https://secure.gravatar.com/avatar/83111e3ce141b383d5169be6edd768d1.jpg?s=120&d=mm&r=g)
Hi Laurens,
Hm. I can't find it in the current version. Either I am misremembering, or they expect you to use the pubsub thing as an RPC mechanism somehow (i.e. the room name is the procedure name).
I see. Using PubSub to implement RPC seems hackish at best, if it works at all. E.g. how to make sure only 1 implementing endpoint subscribes to the Room (to receive procedure calls), how to direct call results to calling clients (since only the caller should receive his result), etc etc Unsurprisingly (probably), I'd like to point to WAMP: http://wamp.ws/ Which was designed from the ground up for exactly this: 1 protocol that covers _both_ RPC and PubSub, and runs over WebSocket. Autobahn|Python implements both WebSocket and WAMP (https://github.com/tavendo/AutobahnPython) End of marketing message;) /Tobias
participants (7)
-
Andrew Francis
-
Dustin J. Mitchell
-
Flavio Grossi
-
Glyph
-
Laurens Van Houtven
-
Sofiane Akermoun
-
Tobias Oberstein