[Python-Dev] Synchronous and Asynchronous servers in the standard library

[Bob Ippolito]

On Nov 10, 2004, at 13:38, Josiah Carlson wrote:

> James Y Knight <foom at fuhm.net> wrote:
>>
>> But, the real point is that you can use the same class for SMTP and
>> SMTP-over-ssl, which is fortunate, because you need to switch to the
>> over-ssl implementation half way through the connection.
>
> You statement is a no-op.  One could implement the greater part of most
> any application in a single class, the question is about how much can 
> be
> shared.
>
> At the minimal level, self.send() and self.recv() need to have 
> different
> implementations for the different with/without SSL variants (assuming
> one would merely wrap the bare socket).  No surprise there.  At a 
> higher
> level, because SSL-ifying a socket is a blocking operation (requires at
> least one round-trip if I remember correctly), discussion about async
> SMTP+TLS is pretty moot at this point (unless one uses/abuses tlslite).

Depends on what you mean by blocking.  As far as Twisted 
implementations are concerned, nothing is supposed to block Python code 
from executing unless it has nothing to do and it's sitting on a 
select(...) or equivalent (depends on which reactor you are using).

Such round trip operations are typically done with a Deferred: a 
placeholder for a success value or failure exception (wrapped in a 
Failure instance) that will be available sometime in the future.  When 
you call some function that needs to do a "blocking" operation, it will 
return a Deferred object.  Once you have that Deferred object you can 
attach a chain of success / failure callbacks to it.  When the producer 
of the Deferred object is ready to do so, it will start the callback 
chain by passing the Deferred object a the success or failure value.  
That value (well, the return value of the previous callback in the 
chain) will be passed along the chain until the chain ends.

The added bonus to using Deferreds is that they're not just for 
sockets.  Deferreds can be used to get the return value of a *blocking* 
function by using the "callInThread" mechanism, which dispatches this 
function to a thread from the reactor's thread pool and returns a 
Deferred object that will be passed the return value of that function.  
You can also use them to get the result of some long-running 
computation (that has been broken into small chunks, executing once per 
runloop iteration by way of a timer).

>> This discussion really seems useless to me. Asyncore certainly has 
>> some
>> serious deficiencies, and arguing each and every one of them is okay
>> because alternatively "nobody needs that feature" or "If X worked in
>> some way (which it doesn't), then everything would be fine" just seems
>> silly. I've only started developing with Twisted fairly recently, so I
>> don't know the entire history of its development, but I'm pretty sure
>> it wasn't started just for fun...
>
> Asyncore was written to scratch Sam Rushing's itch.  I'm not up on
> Twisted's history.

I can't speak for its original raison d'etre, but I moved from asyncore 
to Twisted because I ran into limitations in asyncore that Twisted had 
already taken care of.  As an added bonus, there was a larger more 
responsive community behind it, and a lot of existing protocol 
implementations (most if not all of which are more complete than any 
standard library counterpart, if one exists) that came standard with 
Twisted I could piggy-back off of.

Originally I had stayed away from Twisted because it was big and 
complicated in comparison to asyncore, but at some point a light went 
off in my head and I realized that it would make my applications 
SIGNIFICANTLY shorter, had methods for doing the things I needed to do 
that asyncore didn't do well, and solved some problems I didn't even 
know I would have.

-bob