[Twisted-Python] From asyncore to twisted

Hello, I'm looking for some advice on how to get started with an asyncore -> Twisted conversion. Being new to twisted I have some trouble getting started (re)designing my system. All ideas are welcome since I'm also doing this to learn.
I am about to convert a message-passing daemon written with asyncore to Twisted, and I have two goals in mind. Code that is easier to maintain and expand, and higher reliability. The last part would probably be achievable while sticking with the present code, but since I keep hearing I really should use Twisted instead, I figured I'd try it out :-).
The daemon listens for connections from different peers. Some peers just leave a message and disconnect, some peers connect as receivers and linger waiting for messages. A message will be given to the daemon by a messenger, and forwarded by the daemon to exactly one of the connected receivers. The daemon keeps a data structure (a dict with identifier: channel, actually) of receivers which it cycles through in a round-robin fashion.
There are more scenarios as well; (recievers may send status messages back to the daemon that the daemon logs, some messengers linger waiting for confirmation from a receiver and a few other) but they are mostly just variations of the above.
I'm a bit confused about what parts should be in the Protocol and what should be separated away from it. Should I make a specialized Factory or make a Service (or several depending on what type of client connection it is)?
I realize there is a lot in my description that's open for interpretation, but since I'm mostly after design ideas and good practices, I'm hoping that's OK..
Best regards, Daniel Brandt

On Thu, May 1, 2008 at 6:09 PM, Daniel Brandt daniel.brandt@gmail.com wrote:
Hello, I'm looking for some advice on how to get started with an asyncore -> Twisted conversion. Being new to twisted I have some trouble getting started (re)designing my system. All ideas are welcome since I'm also doing this to learn.
I am about to convert a message-passing daemon written with asyncore to Twisted, and I have two goals in mind. Code that is easier to maintain and expand, and higher reliability. The last part would probably be achievable while sticking with the present code, but since I keep hearing I really should use Twisted instead, I figured I'd try it out :-).
The daemon listens for connections from different peers. Some peers just leave a message and disconnect, some peers connect as receivers and linger waiting for messages. A message will be given to the daemon by a messenger, and forwarded by the daemon to exactly one of the connected receivers. The daemon keeps a data structure (a dict with identifier: channel, actually) of receivers which it cycles through in a round-robin fashion.
One common strategy in twisted is to keep a list of your connected clients (Protocol instances) as part of the factory. See:
http://twistedmatrix.com/projects/core/documentation/howto/servers.html
So in the dataReceived method on you Protocol, you would do something like:
def dataReceived(self): self.factory.clients.append(self)
There are more scenarios as well; (recievers may send status messages back to the daemon that the daemon logs, some messengers linger waiting for confirmation from a receiver and a few other) but they arees, mostly just variations of the above.
I'm a bit confused about what parts should be in the Protocol and what should be separated away from it. Should I make a specialized Factory or make a Service (or several depending on what type of client connection it is)?
I think generally it is a good idea to have a specialized Factory as this binds global state for you Protocol instances. The server howto linked above describes this pattern quite well.
I realize there is a lot in my description that's open for interpretation, but since I'm mostly after design ideas and good practices, I'm hoping that's OK..
The finger tutorial does a good job of explaining many aspects of twisted, and this might give you some ideas of how to leverage its parts for your application:
http://twistedmatrix.com/projects/core/documentation/howto/tutorial/index.ht...

On 1 May, 10:09 pm, daniel.brandt@gmail.com wrote:
I'm looking for some advice on how to get started with an asyncore -> Twisted conversion. Being new to twisted I have some trouble getting started (re)designing my system. All ideas are welcome since I'm also doing this to learn.
I suggest that you do these things one at a time.
While I am practically overflowing with advice about how to architect Twisted applications, I don't think it would be appropriate to give it all to you right now.
Get your application running on Twisted under more or less the same architecture, and then once you've ported over the basics - protocols and transports, timed events - you should notice that you can delete some of the things in your asyncore application that were doing things that Twisted is now doing for you.
Once you've got a fully working Twisted app (with, of course, full unit test coverage) that's the time to start looking at how to best improve it. Trying to simultaneously convert to Twisted and improve your design will lead to things breaking, leaving you thinking "this used to work *before*..." and scratching your head, wondering if it's the new design that's buggy or it's some detail of Twisted you don't understand.
I'm a bit confused about what parts should be in the Protocol and what should be separated away from it. Should I make a specialized Factory or make a Service (or several depending on what type of client connection it is)?
A Protocol represents an individual connection between a client and server; a factory roughly represents a listening (or connecting) port where incoming connections may be initiated; when connections are started via a Factory, Protocols are created.
Keep the things on the Protocol that were on your asyncore "channels", and use factories for objects that previously had a "handle_connect" method.
I wouldn't worry at all about Services just yet; the only place you need to care about them is your start-up and shut-down code.
participants (3)
-
Daniel Brandt
-
Drew Smathers
-
glyph@divmod.com