httplib incredibly slow :-(
Chris Withers
chris at simplistix.co.uk
Tue Aug 11 16:25:47 EDT 2009
Hi All,
I'm using the following script to download a 150Mb file:
from base64 import encodestring
from httplib import HTTPConnection
from datetime import datetime
conn = HTTPSConnection('localhost')
headers = {}
auth = 'Basic '+encodestring('username:password').strip()
headers['Authorization']=auth
t = datetime.now()
print t
conn.request('GET','/somefile.zip',None,headers)
print 'request:',datetime.now()-t
response = conn.getresponse()
print 'response:',datetime.now()-t
data = response.read()
print 'read:',datetime.now()-t
The output shows it takes over 20 minutes to do this.
However, this is on a local network, and downloading the same file in IE
takes under 3 seconds!
I saw this issue:
http://bugs.python.org/issue2576
I tried changing the buffer size to 4096 in a subclass as the issue
suggested, but I didn't see the reported speed improvement.
I'm using Python 2.6.2.
Does anyone know of an alternative library for creating http requests
and getting their responses that's faster but hopefully has a similar
interface?
Chris
--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
More information about the Python-list
mailing list