[Python-Dev] Synchronous and Asynchronous servers in the standard library

[Josiah Carlson]

A recent patch to offer an SMTP server (SocketServer derivative) sparked
the below...

Question:
Does we (and by 'we' I mean those in charge of developing Python) want
to offer both asynchronous (deriving from asyncore, asynchat, etc.) and
synchronous versions of server software in the standard library?


Observations:
There is currently 1 async server, and 5 sync servers in the standard
library.

If we want to offer both sync and async versions, it makes sense to
reduce programmer effort and share as much code as possible.


Idea:
Set up a framework that allows Python programmers to produce async and
sync variants of their servers easily.  I have a partial example
'implementation' below.

It seems to make sense (if such a framework is desired) to place all
such modules in a 'Server' package, and to describe precisely what is
necessary to make such a package usable, used, and added to in a PEP
called something like "A flexible server framework".


Future Python:
This would obviously be a 2.5 feature/module, and I would be willing to
write both the PEP (perhaps after the 1st of January, 2005 as I have a
journal paper write before then) (and afterwards) translate the current
standard library servers to the framework.

Depending on how popular the framework is through the lifetime of Python
2.5 (if it makes it into 2.5), it may make sense to deprecate the
original servers in 2.6 or 2.7 (if ever).  I would be willing to
volunteer to maintain the server package, or at least as many of the
protocols as I understand, for 2.5 and beyond (I don't know much about
XMLRPC, so translation is about as much as I could probably do).


 - Josiah



In the case of 'chatty' protocols like SMTP, POP, etc., where the
following complete, merely a prototype...


class SyncServerBase:
    def send(self, data):
        if data is None:
            self.sock.close()
        else:
            self.sock.write(data)
    def close_when_done(self):
        self.send(None)
    ...

class AsyncServerBase(asyncore.dispatcher):
    def send(self, data):
        #handle buffering with deque, etc.
    def close_when_done(self):
        #toss a None into the queue
    ...

class AsynchatMixin(asynchat.async_chat):
    #perhaps write a better buffered reader and writer...
    limit = None
    def readable(self):
        #handle optional request size limit
        return limit is None or len(self.ac_in_buffer) < self.limit
    def found_terminator(self):
        self.request = self.ac_in_buffer
        self.ac_in_buffer = ''
        self.handle_request()
    ...

class SMTP_Handler:
    def SMTP_HELO(self, cmd):
        ....
    def SMTP_MAIL(self, cmd):
        ...
        self.send(response)
        ...
    ...

class SMTP_Sync(SyncServerBase, SMTP_Handler):
    def handle_request(self):
        #does all of the reading necessary to read from a socket and
        #handle dispatch to SMTP_*


class SMTP_Async(AsyncServerBase, AsynchatMixin, SMTP_Handler):
    terminator = '\r\n'
    def handle_request(self):
        #command line (or email content) is provided in self.request
        #handles dispatch to SMTP_*, reading a line/request is provided
        #in this case by the AsynchatMixin

Etcetera.