[New-bugs-announce] [issue16095] urllib2 failing with squid proxy and digest authentication

[Pietro Battiston]

New submission from Pietro Battiston:

If you run the following code:

#! /usr/bin/python
import urllib2

MyHTTPPasswordMgr = urllib2.HTTPPasswordMgr
proxy = urllib2.ProxyHandler({'http': 'http://proxybiblio2.si.unimib.it:8080'})
auth = urllib2.ProxyDigestAuthHandler(MyHTTPPasswordMgr())
auth.add_password(None, "proxybiblio2.si.unimib.it", "a", "b" )
opener = urllib2.build_opener(proxy, auth, urllib2.HTTPHandler)
urllib2.install_opener(opener)
conn = urllib2.urlopen('http://webofknowledge.com')

an "HTTP Error 407: Proxy Authentication Required" is raised, and under the hood here's what's happening:
- the request is made without authentication
- the server replies it must be made with digest authentication, and gives the nonce
- the error is raised.

Instead, urllib2 should now try to connect with the username and password, and do so up to 5 times (as hardcoded in urllib2.http_error_auth_reqed), and then raise a "HTTP Error 401: digest auth failed". And it's indeed what it does if you replace the line "MyHTTPPasswordMgr = urllib2.HTTPPasswordMgr" with


class MyHTTPPasswordMgr(urllib2.HTTPPasswordMgr):
    def find_user_password(self, realm, authuri):
        return "a", "b"


So the problem is in HTTPPasswordMgr, which is apparently unable to match the authentication data with the realm. Some tests¹ suggest that this can vary according to the proxy engine and to the proxy address format (works with apache, but doesn't if then you add "http://" in front of the proxy address).

This reminds a bit bug 680577, and in particular I noticed that (possibly unrelated) the behaviour reported in the following message:
http://bugs.python.org/msg14444
has not changed:

In [1]: import urllib2

In [2]: urllib2.HTTPPasswordMgr().is_suburi("/foo/spam", "/foo/eggs")Out[2]: True


This affects also python 3.2, you can try the following:

#! /usr/bin/python
from urllib import request
MyHTTPPasswordMgr = request.HTTPPasswordMgr
proxy = request.ProxyHandler({'http': 'http://proxybiblio2.si.unimib.it:8080'})
auth = request.ProxyDigestAuthHandler(MyHTTPPasswordMgr())
auth.add_password(None, "proxybiblio2.si.unimib.it", "a", "b" )
opener = request.build_opener(proxy, auth, request.HTTPHandler)
request.install_opener(opener)
conn = request.urlopen('http://webofknowledge.com')


¹ http://lists.python.it/pipermail/python/2012-September/013309.html (in Italian)

----------
components: Library (Lib)
messages: 171650
nosy: toobaz
priority: normal
severity: normal
status: open
title: urllib2 failing with squid proxy and digest authentication
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16095>
_______________________________________