Questions about asyncore

[Frank Millman]

Hi all

I have been using my own home-brewed client/server technique for a
while, using socket and select. It seems to work ok. The server can
handle multiple clients. It does this by creating a new thread for
each connection. Each thread runs its own select loop.

I am making some fairly big changes, so I thought I would look at
asyncore. I modified my program to use asyncore without much trouble,
and it feels nicer. It uses async I/O instead of threading, and it
relieves me of having to run my own select loop.

I have two questions. They both relate to whether I am using the
module as intended. The documentation is rather sparse, and I know
from experience that just getting something working is no guarantee
that I am getting the full benefit.

Firstly, having got asyncore working, I had a look at asynchat. As far
as I can see I get very little benefit from using it. I have already
set up a 'messaging' protocol between server and client, where all
messages consist of 5 digits for the message length, followed by the
message. The message consists of a pickled tuple, where the first
element is a message identifier, and the rest is the message body.

This is how it works in asyncore -

    def __init__(self,channel):
        [...]
        self.msgLength = 0
        self.recvData = ''  # temporary buffer

    def handle_read(self):
        self.recvData += self.recv(8192)
        if not self.msgLength:
            if len(self.recvData) < 5:  # 1st 5 bytes = msg length
                return
            self.msgLength = int(self.recvData[:5])
            self.recvData = self.recvData[5:]
        if len(self.recvData) < self.msgLength:
            return
        data = loads(self.recvData[:self.msgLength])
        self.recvData = self.recvData[self.msgLength:]
        self.msgLength = 0
        [handle data]

This is how I got it working in asynchat -

    def __init__(self,channel):
        [...]
        self.recvData = ''  # temporary buffer
        self.set_terminator(5)
        self.gotMsgLength = False

    def collect_incoming_data(self, data):
        self.recvData += data

    def found_terminator(self):
        if self.gotMsgLength:  # what follows is the message
            data = loads(self.recvData)
            self.set_terminator(5)
            self.gotMsgLength = False
            [handle data]
        else:  # what follows is the message length
            self.set_terminator(int(self.recvData))
            self.gotMsgLength = True
        self.recvData = ''

It may be slightly neater, but it does not seem worth adding an extra
layer just for that. Does asynchat give me any other benefits I may
have overlooked?

My second question relates to writing a dummy client program to test
the server. I just want to send it some messages and print the
responses. Some messages incorporate data extracted from previous
responses, so I have to wait for the reply before continuing.

I am using asyncore for this as well. However, the only way I can
think of to get it working is to run asyncore.dispatcher in a separate
thread.

To send messages, I append them to a list of messages to be sent. The
dispatcher method writable() returns True if there is anything in the
list, else False.

To receive messages, I run a 'while 1' loop in the main thread, with a
sleep of 0.1, until recvData has something in it.

It works, but it seems odd to use a separate thread, as one of the
points of asyncore is to avoid multi-threading. Is there a better way
to write the client program?

Thanks

Frank Millman