[Python-Dev] Issue with HTTP basic proxy authentication in urllib2
ivano at deakin.edu.au
Tue Jan 20 22:48:37 EST 2004
I have only been scripting in python for about 3 months
and have mainly been performing HTTP requests to cgi scripts.
I have had had a lot of trouble with POST method and believe I have
tracked this down to the fact that I'm having to perform Basic Proxy
When the HTTP header for Basic authentication is created in urllib2
the username and password are encoded using the encodestring() method
of the base64 module. This adds a newline '\n' to the end of the returned
encoded string (as documented).
def proxy_open(self, req, proxy, type):
orig_type = req.get_type()
type, r_type = splittype(proxy)
host, XXX = splithost(r_type)
if '@' in host:
user_pass, host = host.split('@', 1)
if ':' in user_pass:
user, password = user_pass.split(':', 1)
user_pass = base64.encodestring('%s:%s' %
(unquote(user),Harry_Connick_Jr - She.albm
req.add_header('Proxy-authorization', 'Basic ' + user_pass)
This newline then becomes the end of the HTTP Proxy authentication header:
Proxy-authorization: Basic asdfSADFwaer%asfdas=\n
When the HTTP request is assembled the required CRLF is added to the end
of each header along with the one extra to indicate the end of the
This would seem OK except some HTTP servers appear to interperet the newline
after the proxy authentication header as an extra blank line in the
missing any data sent through in the body of the request. This is not a
with the GET method (obviously) or a POST method which is not sent though
an HTTP Proxy server requiring authentication.
I made the very simple addition to urllib2.py of
to remove the trailing '\n' from the encoded username/password before it
added to the HTTP headers and everything works without a problem.
I'm not sure if this is an issue which has been raised before. I
glanced through the
summaries of the python-dev mailing list for the past year and found no
mention of it.
Technically I don't see it as a bug in urllib2.py but an incorrect
implementation of HTTP
header parsing by servers (The Proxy server may even be the culprit)
PS. If there is any need to discuss this could I please be CC'd as I am
not a member of the mailing, Thanks
Metabolic Research Unit,
Waurn Ponds 3217,
Ph: 61 3 52272195
"the money I save won't buy my youth again"
More information about the Python-Dev