[Python-ideas] re-implementing Twisted for fun and profit

[Glyph]

On Oct 14, 2012, at 7:47 PM, Shane Green <shane at umbrellacode.com> wrote:

> Hm, just jumping in out of turn (async ;-)  here, but I prototyped pretty clean versions of asyncore.dispatcher and asynchat.async_chat type classes built on top of a promise-based asynchronous I/O socket-monitor.  Code ended up looking something like this following: 

> this.socket.accept().then(this.handle_connection)
> 
> # With a handle_connection() kind of like…
> def handle_connection(conn): 
> 	# Create new channel and add to socket map, then…
> 	if (this.running()): 
> 		this.accept().then(this.handle_connection)

As I explained in a previous message, I think this is the wrong way to go, because:

It's error-prone.  It's very easy to forget to call this.accept().then(...).  What if you get an exception? How do you associate it with 'this'?  (Why do you have to constantly have application code check 'this.running'?)
It's inefficient.  You have to allocate a promise for every single operation. (Not a big deal for 'accept()' but kind of a big deal for 'recv()'.
It's hard to share resources. What if multiple layers try to call .accept() or .read_until() from different promise contexts?
As a bonus fourth point, this uses some wacky new promise abstraction which isn't Deferreds, and therefore (most likely) forgets to implement some part of the callback-flow abstraction that you really need for layered, composable systems :).

We implemented something very like this in Twisted in a module called called "twisted.web2.stream" and it was a big problem and had very poor performance and I just this week fixed yet another instance of the 'oops I forgot to call .read() again in my exception handler' bug in a system where it's still in use.  Please don't repeat this mistake in the standard library.

-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121015/bf9c8126/attachment.html>