[Python-Dev] Synchronous and Asynchronous servers in the standard library

[Phillip J. Eby]

At 10:39 AM 11/7/04 -0800, Josiah Carlson wrote:
>It seems as though there has already been discussion of getting portions
>of Twisted into the Python standard library.  Being as I have not
>developed with Twisted, nor am I really a part of the Twisted community,
>I'm not sure that I am really the right person to try to figure out what
>parts of Twisted should or should not be included with Python 2.5.
>
>Regardless of the asyncore vs Twisted issue, a proper (meta) framework
>should separate out the calls for "what happens to the internal state
>when the server receives a reqest like 'foo'", and how the data actually
>gets to those calls.  If done correctly, if/when portions of Twisted
>make it into the standard library, it shouldn't be terribly difficult to
>offer a mechanism to use Twisted as a base (for someone with Twisted and
>stdlib Twisted experience), if such a thing is desired.
>
>I suppose the question comes down to; should there be a meta framework
>for developing servers in Python that specifies a separation between the
>protocol logic and how data gets to that protocol logic?  The desire is
>for a mechanism to allow people to create a single version of a server
>in their favorite framework (SocketServer, asyncore, Twisted, or other),
>and for others to be able to use them in their favorite framework with
>little difficulty.

I believe that this may be an unrealistic goal, even though I have done 
something rather similar myself.  There is a myriad assortment of policy 
issues you'll have to swim upstream past, just to get to the mechanism 
issues.  For example, Twisted assumes its reactor is an interpreter-global, 
maybe even process-global event loop, whereas asyncore and peak.events 
allow multiple event loops.

I've dabbled in integrating Twisted and peak.events, such that code written 
for peak.events can run with or without Twisted, and such that an 
asynchronous server can be run synchronously.  So, it's certainly possible 
to do something like what you're describing, but I'm not sure it's worth 
the effort versus someone simply installing PEAK and/or Twisted.

Now, if the Python standard library included a lightweight task switching 
facility akin to Armin's greenlets, it might be more practical to include 
asynchronous communications facilities in the stdlib.  Essentially, one 
could implement a set of communication primitives that task-switched to a 
global scheduler, or one could supply the scheduler as part of the calls, 
or provide the primitives as methods of a scheduler, etc.  This would 
bypass most of the architectural policy issues.

However, without something like this, you are basically just going to be 
re-inventing either Twisted or peak.events, depending on whether you prefer 
a continuation-passing or generator-pseudothread architecture.