[issue2576] httplib read() very slow due to lack of socket buffer

Chris Withers report at bugs.python.org
Wed Aug 12 09:54:24 CEST 2009


Chris Withers <chris at simplistix.co.uk> added the comment:

I tried to use the following to change the buffersize for a download:

from base64 import encodestring
from httplib import HTTPResponse,HTTPConnection,HTTPSConnection,_UNKNOWN
from datetime import datetime

class FHTTPResponse(HTTPResponse):

    def __init__(self, sock, debuglevel=0, strict=0, method=None):
        print "creating response"
        self.fp = sock.makefile('rb',4096)
        self.debuglevel = debuglevel
        self.strict = strict
        self._method = method

        self.msg = None

        # from the Status-Line of the response
        self.version = _UNKNOWN # HTTP-Version
        self.status = _UNKNOWN  # Status-Code
        self.reason = _UNKNOWN  # Reason-Phrase

        self.chunked = _UNKNOWN         # is "chunked" being used?
        self.chunk_left = _UNKNOWN      # bytes left to read in current 
chunk
        self.length = _UNKNOWN          # number of bytes left in 
response
        self.will_close = _UNKNOWN      # conn will close at end of 
respons    
class FHTTPConnection(HTTPConnection):
    response_class = FHTTPResponse

class FHTTPSConnection(HTTPSConnection):
    response_class = FHTTPResponse

conn = FHTTPSConnection('localhost')
headers = {}
auth = 'Basic '+encodestring('usernmae:password').strip()
headers['Authorization']=
t = datetime.now()
print t
conn.request('GET','/somefile.zip',None,headers)
print 'request:',datetime.now()-t
response = conn.getresponse()
print 'response:',datetime.now()-t
data = response.read()
print 'read:',datetime.now()-t

..however, I saw absolutely no change in download speed.

Aren, I notice in your pastebin code that you do response.read(10485700) 
in a loop rather than just one response.read(), why is that?

----------
nosy: +cjw296

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2576>
_______________________________________


More information about the Python-bugs-list mailing list