[Python-Dev] thread safe SMTP module

[Josiah Carlson]

Gordon Messmer <yinyang at eburg.com> wrote:
> After some discussion, Aahz suggested that I discuss the problem here, 
> on python-dev.  He seemed to think that the problem I saw may have been 
> an indication of a bug in python.  Could anyone take a look at that 
> thread and say whether it looks like a bug, or working with non-blocking 
> sockets was the right thing to do?

His most recent message in that thread actually said, "No, at this point
I think this is neither bug nor about thread blocking on I/O.  I think
it's about sockets dying and the inability of sockets in blocking mode
to recover.  I have seen this kind of behavior in single-threaded
systems."

I would concur with Aahz, more or less.  I have also seen this behavior
on single-threaded blocking systems.  The real issue may be related to
blocking sockets breaking coupled with socket.makefile(...).readline()
not being able to handle the breakage.

If you can, try to dig into the file object implementation returned by
socket.makefile().  If you don't want to (I would understand), try
replacing smtplib's use of 'self.file.readline()' with a Python variant
that handles failures more gracefully.  Heck, you could even add
timeouts (which are being added to httplib and either urllib or urllib2).


 - Josiah

As an aside, for even minimally loaded systems, I've noticed that
file.readline() isn't any faster than just using a while loop with
socket.recv() and checking for '\n'.  I believe that it would make sense
to replace certain libraries' (poplib, smtplib, etc.) use of
file.readline() with that of a _readline() method that has a default
implementation using either a file.readline() or sock.recv() in a loop. 
By pulling it out into a single method, the user could easily override
it if it isn't doing what they want/expect.