[Python-ideas] re-implementing Twisted for fun and profit
shane at umbrellacode.com
Mon Oct 15 10:03:33 CEST 2012
Your points regarding performance are good ones. My tests indicated it was slightly slower than asyncore. The API I based it on is actually quite thorough, and addresses many of the shortcomings Deferreds (in Twisted) have. Namely, all callbacks registered with a given Promise instance, receive the output of the original operation; chaining is fully supported but explicitly (this.then(that).then(that)…), rather than having a Deferred whose value automatically assumes that of each callback, making them necessarily dependent handlers fired before them, with a default guaranteed behaviour being that only the first one actually receives the output of the originating application. I haven't come across many instances where one wants to chain their callback by accident, but many examples where multiple parties were interested in the same operation's output. Finally, I'm not sure you're other points differ greatly from the gotchas of I/O programming in general. Uncoordinated access by multiple threads tends to be problematic. Again, though, you're point about efficiency and the less than ideal "an instance for every" arrangement are good ones. Just throwing it out there as a source of ideas, and hopefully to unseat Deferreds as the defacto callback standard four discussion because the promise pattern is more flexible and robust.
805-452-9666 | shane at umbrellacode.com
On Oct 15, 2012, at 12:45 AM, Glyph <glyph at twistedmatrix.com> wrote:
> On Oct 14, 2012, at 7:47 PM, Shane Green <shane at umbrellacode.com> wrote:
>> Hm, just jumping in out of turn (async ;-) here, but I prototyped pretty clean versions of asyncore.dispatcher and asynchat.async_chat type classes built on top of a promise-based asynchronous I/O socket-monitor. Code ended up looking something like this following:
>> # With a handle_connection() kind of like…
>> def handle_connection(conn):
>> # Create new channel and add to socket map, then…
>> if (this.running()):
> As I explained in a previous message, I think this is the wrong way to go, because:
> It's error-prone. It's very easy to forget to call this.accept().then(...). What if you get an exception? How do you associate it with 'this'? (Why do you have to constantly have application code check 'this.running'?)
> It's inefficient. You have to allocate a promise for every single operation. (Not a big deal for 'accept()' but kind of a big deal for 'recv()'.
> It's hard to share resources. What if multiple layers try to call .accept() or .read_until() from different promise contexts?
> As a bonus fourth point, this uses some wacky new promise abstraction which isn't Deferreds, and therefore (most likely) forgets to implement some part of the callback-flow abstraction that you really need for layered, composable systems :).
> We implemented something very like this in Twisted in a module called called "twisted.web2.stream" and it was a big problem and had very poor performance and I just this week fixed yet another instance of the 'oops I forgot to call .read() again in my exception handler' bug in a system where it's still in use. Please don't repeat this mistake in the standard library.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-ideas