[New-bugs-announce] [issue4879] Allo buffering for HTTPResponse

Kristján Valur Jónsson report at bugs.python.org
Thu Jan 8 11:43:56 CET 2009

New submission from Kristján Valur Jónsson <kristjan at ccpgames.com>:

This is related to issue 4336.  While that issue finally revolved around 
fixing the Nagle problem for xmlrpc and other http clients, here I 
propose to fix another performance bottleneck with xmlrpc, reading the 
HTTP Headers.
HTTPResponse creates a socket.fileobject() with zero buffering which 
means that the readline() operations used to read the headers become 
very inefficient since individual socket.recv() calls are used for each 
The reason for this is to support users who subsequently do 
socket.recv() on the underlying socket, and so we don't want to read too 
much of the headers. Note that there is no example of this in the 
standard library.
In the provided patch, I have removed some vestigial code from 
xmrlpclib.py which attempted to use recv() even though it never did 
(because the connection has been closed when HTTPConnection returns the 
response).  Even so, Guido has expressed his concern that we need to 
keep the support for this recv() behaviour in place.
Therefore, I have added a new optional argument, nobuffer=True, which 
users that promise not to call recv() can set to false.  This will then 
cause the generated fileobject from HTTPResponse to have default 
buffering, greatly increasing the performance of the reading of the 

components: Library (Lib)
files: nobuffer.patch
keywords: patch, patch
messages: 79406
nosy: krisvale
severity: normal
status: open
title: Allo buffering for HTTPResponse
type: performance
versions: Python 2.7
Added file: http://bugs.python.org/file12649/nobuffer.patch

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list