urllib2.open always send 'Accept-Encoding: gzip, deflate' under Windows

Yuan HOng hongyuan1306 at gmail.com
Tue Feb 3 09:56:15 CET 2009


I wish to send a request without 'Accept-Encoding: gzip, deflate' in the
request header. Under Linux, this is not a problem, I got 'Accept-Encoding:
identity' by default.

However, under Windows (XP SP3 with Python 2.5), no matter what I try, I
always got 'Accept-Encoding: gzip, deflate' in the request header. There is
no method in urllib2 to remove or disable the sending of a header. So what I
can do is try to override the miraculously added header, like:

>>> opener = urllib2.build_opener()
>>> opener.addheaders = [('Accept-encoding', 'identity;q=1.0, gzip;q=0')]
>>> opener.open('some test url')


>>> request = urllib2.Request('http://www.homemaster.cn',
headers={'Accept-encoding': 'identity'})
>>> urllib2.urlopen(request)

But I always still get the annoying 'gzip, deflate' in the header (captured
by Wireshark). And in the latter case, it seems my 'Accept-encoding':
'identity' is changed into '---------------: --------', presumably because
it clashes with the system generated header of 'gzip, deflate':

GET / HTTP/1.1
Accept-Encoding: gzip, deflate
Host: www.homemaster.cn
User-Agent: Python-urllib/2.5
Connection: close
---------------: --------

Strangely I neither find the string 'gzip, deflate' in urllib2.py nor in
httplib.py. Where does it come and how can I remove it or replace it with

Hong Yuan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20090203/9c21fd75/attachment.html>

More information about the Python-list mailing list