Hi Chris,

It will probably not surprise you that I like Twisted :-)

On Wed, Mar 12, 2014 at 8:28 AM, Chris Withers <chris@simplistix.co.uk> wrote:

So, I see python now has a plethora of async frameworks and I need to try and pick one to use from:

- asyncio/tulip
- tornado
- twisted

Twisted has a thing called Cyclone, which I hear (but that's only hearsay) gives you Tornado's API on top of twisted, so you get all of the stuff below for free.

>From my side, I'm looking to experimentally build a network testing tool that will need to speak a fair few protocols, both classic tcp and multicast-based, and have a web api living on top of it that most likely will have a websocket for pumping data to the browser. It'll also need to write out JUnit-compatible xml results, but that's like the easiest bit ;-)

I don't know which protocols you are interested in specifically; later on in the e-mail you mention that it's *probably* not already extant. Twisted comes with a huge number of protocols already implemented. It has TCP support (duh) as well as an interface for UDP multicast.

Twisted has a third party project called txsockjs which works excellently, both by itself and in the context of other (HTTP) resources. sockjs is a protocol which is basically "websockets, damnit", even when the consumer is a bad browser like IE6.

I'd like to be able to serve the rest of the web api using a pyramid wsgi app if possible, and I'd like to be able to write the things that process requests in and validation out in a synchronous fashion, most likely spinning off a thread for each one.

Twisted comes with a threadpool-backed WSGI server. (When I say threadpool-backed I mean that the WSGI requests are handled in threads; the IO itself of course comes from the Twisted reactor).

The protocols are all financial (do we really not have a pure-python FIX library?!) but none are likely to have existing python implementations.

I find twisted to be a great tool for writing protocol implementations. I have written tools for querying all sorts of gnarly proprietary protocols over all sorts of gnarly transports (packet radio; it's totally a thing), and more recently for doing crazy things like multiplexing stream transports over existing RPC protocols. (Like, you see a local port come up, and that actually creates a virtual stream connection over an existing RPC thing to some virtual server on the other end of the wire: https://www.youtube.com/watch?v=W_jEIvugwes).

Twisted has had so many people write so many protocols in it that also the testing tools (MemoryReactor, StringTransport) are great. Especially if you are writing something very close to a wire protocol you will undoubtedly enjoy those amenities. There's also tons of composable things for receiving delimited lines, nestrings, etc. It's hard to tell what you will be looking for since I don't know details about your protocol, but having written more than a few protocol implementations I'm going to wager a guess and say Twisted has it or a third party thing for twisted has it.

Even though I mostly write Clojure now, I still write my networking stuff in Twisted. Macros are cool. Eleven years worth of battle-tested code is better.

That said, tulip is nice in that you can also write protocol implementations that will look similar :-)

How should I pick between the options? What would people recommend and why?

Twisted, emphatically and without reservation, for all the above reasons. It's stable. All the stuff you need has been tried and tested extensively in production. It runs on PyPy, and usually a damn sight faster than on CPython, too.

A common criticism of Twisted is that it "takes over" your codebase. I am speaking at PyCon in about a month to demonstrate that that isn't true. (It just looks that way, because once people use it, they don't want to go back... ;-))

hth
lvh