[Python-Dev] Fw: SocketServer and makefile() [from comp.lang.python]

[Guido van Rossum]

> I ran into this same problem on the client side.
> 
> The server does a makefile() so that it can do readline() to fetch the HTTP
> request line and then the MIME headers. The *problem* is that if you do
> something like:
> 
>     f = sock.makefile()
>     line = f.readline()
>     data = sock.recv(1000)
> 
> You're screwed if you have buffering enabled. "f" will read in a bunch of
> data -- past the end of the line. That data now sits inside f's buffer and
> is not available to the sock.recv() call.
> 
> If you forget about sock and just stick to f, then you'd be okay. But
> SocketServer and/or BaseHTTPServer doesn't -- it uses both objects to do the
> reading.
> 
> Solution? Don't use rfile for reading, but go for the socket itself. Or
> revamp the two classes to forget about the socket once the files (wfile and
> rfile) are created. The latter might not be possible, tho.

I was about to say that you have it backwards, and that you should
only use rfile & wfile, when I realized that CGIHTTPServer.py needs
this!  The subprocess needs to be able to read the rest of the socket,
for POST requests.  So you're right.

Solution?  The buffer size should be an instance or class variable.
Then SocketServer can set it to buffered by default, and CGIHTTPServer
can set it to unbuffered.

> Dunno why the unbuffered reading would be slow. I'd think it would still
> read large chunks at a time when you request it.

System call overhead?  I had the same complaint about Windows, where
apparently winsock makes you pay more of a performance penalty than
Unix does in the same case.

--Guido van Rossum (home page: http://www.pythonlabs.com/~guido/)