[Python-Dev] SocketServer issues

[Kristján Valur Jónsson]

Hi there.
I want to mention some issues I've had with the socketserver module, and discuss if there's a way to make it nicer.
So, for a long time we were able to create magic stackless mixin classes for it, like ThreadingMixIn, and assuming we had the appropriate socket replacement library, be able to use it nicely using tasklets.
Then, at some point, the run_forever loop was changed to support timeout through the use of select.select() before every socket.accept() call.  This was very awkward because the whole concept of select() really goes contrary to the approach of using microthreads, non-blocking IO and all that.
The only reason for this select call, was to support timeout for the accept.  And even for vanilla applications, it necessiates an extra kernel call for every accept, just for the timeout.

The way around this for me has to do local modifications to the SocketServer and just get rid of the select.

So, my first question is:  Why not simply rely on the already built-in timeout support in the socket module?  Setting the correct timeout value on the accepting socket, will achieve the same thing.  Of course, one then has to reset the timeout value on the accepted socket, but this is minor.

Second: Of late the SocketServer has grown additional features and attributes.  In particular, it now has two event objects, __shutdown_request and __is_shut_down.
Notice the double underscores.
They make it impossible to subclass the SocketServer class to provide a different implementation of run_forever().  Is there any good reason why these attributes have been made "private" like this?  Having just seen Raymond's talk on how to subclass right, this looks like the wrong way to use the double leading underscores.

So, two things really:
The use of select.select in SocketServer makes it necessary to subclass it to write a new version of run_forever() for those that wish to use a non-blocking IO library instead of socket. And the presence of these private attributes make it (theoretically) impossible to specialize run_forever in a mix-in class.

Any thoughs?  Is anyone interested in seeing how the timeouts can be done without using select.select()?  And what do you think about removing the double underscores from there and thus making serve_forever owerrideable?

Kristján
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120314/7c8ba3eb/attachment.html>