I am new to this Twisted Architecture so if my plan makes no sense, please guide me down a smoother path.

I have a program using Asyncore as the communications interface but I want to switch to Twisted.

Server1                                Server 2
<-------TCP------|    | <---TCP-------
                     | My App |
----TCP--------->|    |----TCP----->

Basically MyApp takes stuff from Server 1, modifies it in some way, then sends it to server 2 and vice versa for Server 2 to Server 1. It is a translator between two ICD's languages spoken by two different systems. All data in and out is raw.

I have modules written that can parse the raw data into objects, translate the objects into the other ICD, then repack the objects into raw data. Currently, My App uses asyncore to manage the socket between the servers as pictured above. Asyncore sticks the raw data on a Queue for the next object to retrieve. Each object is a thread that blocks on the Queue until data arrives. When a object receives data on the Queue, it performs its operation and sticks the modified data on another queue for the next thread to handle. Eventually, data ends up on a Queue being monitored by an Asyncore dispatcher that grabs the data and sends it to a server.

Server1 ----TCP----->Asyncore.recv() --put--> Queue --get-->Parser--put--> Queue --get--> Translator --put--> Queue --get--> Packer --put---> Queue --get-->Asyncore.send()

I would like to use Twisted for the following reasons:

1. Add a gui, stand alone or web based, allowing users to view statistics being generated by modules and to inject messages to one of the servers. Other services may be necessary in the future
2. We have a problem with asyncore behaving differently on Windows servers then Linux servers with regards to reconnecting after network drops. It is greatly beneficial if we can run on either platform.
3. Twistd is a great way to run this program as a daemon on Linux but still works on Windows
4. My own personal education
5. Twisted seems so cool

Questions that have arisen as i try to code this in Twisted.

1. What is the proper method for gluing my processing modules to Twisted?

I have followed the finger tutorial and it seems I should call them from a Service class. Finger tutorial part 3
This leads to question 2

I have toyed with using DeferredQueue's to pass data between the modules. I would pretty much follow the pattern above, but the DeferredQueue's should allow my modules to NOT run as threads. Instead they just fire events when data arrives on their Queue.

2. How do I have My App send data to the servers via events?

In the finger tutorial Part 3 there are getUser and getUsers methods that are called by the FingerProtocol class via the FingerClientFactory. This all makes sense for processing data from the network and responding to it, but suppose I want some other object, say my Packer object, to fire an event that will make FingerProtocol send some data from Packer to the server? How do I expose a method to do this if Packer only knows about the Service, but the Service has no access to methods in FingerProtocol?

I have come up with two solutions to this problem, neither of which is impressive to me.

First solution, use DeferredQueue's to pass all the data around. This does work as a prototype, but I have no Service class wrapping it.

class QueueProtocol(Protocol):
    "Handles the network receive and send work for a send and a receive queue"
    def dataReceived(self, data):
        print "Received Data:",data
        self.factory.putData(data)

    def connectionMade(self):
        print "Conn Made"
        self.sendData()

    def sendData(self):
        print "sendData"
        self.factory.getData().addCallback(self.sendHelp)

    def sendHelp(self,data):
        print "SendHelp",data
        self.transport.write(data+"\r\n")
        self.sendData()

class QueueClientFactory(ClientFactory):
    "Handle the Connection"
    protocol = QueueProtocol
    def __init__(self,recQueue, sendQueue):
        self.recQueue = recQueue #both deferredQueue's
        self.sendQueue = sendQueue

    def putData(self,data):
        self.recQueue.put(data)

    def getData(self):
        "Returns a defered because sendQueue is a DeferredQueue"
        return self.sendQueue.get()

Second solution, that I have not managed to code. I have the feeling I'm way of the mark here.

Each component (yes I have been reading the Zope docs) in the application knows about interfaces. All components will implement a receive(data) method and a process(data) method. Each of these objects can pass data to other components by adding another components receive(data) method as a callback to its own process(data) method.

Parse
def receive(data):

self.process(data).addCallback( Translate.receive(data) )

The Parse object receives data from somewhere. The process method takes this raw data and parses it into an object. When process has completed the callback method, defined in the Translate object, is called with the new object as a parameter.

Parse's Factory class will take an instance of Translate in __init__.

Some guidance would make my brain hurt less.

Thanks,

--

Sean Roark