urllib2 http authorization question

Matthew Wilson matt at overlook.homelinux.net
Sun Dec 14 12:07:27 EST 2003


I am writing a script to check on my router's external IP address.  My
ISP refreshes my IP very often and I use dyndns for the hostname for my
computer. My Netgear mr814 router has a webserver that uses HTTP basic
authorization.

My script uses urllib2 to connect to the router and read the html page
with the current external IP address. This is the function I wrote to
lookup the router's external IP:

def mr814(router_user, router_password):
    "Looks up the IP from the router and returns it as a string."
    req = urllib2.Request('http://192.168.0.1/sysstatus.html')
    req.add_header("USER-AGENT",
                   "Mozilla/4.76 [en] (X11; U; Linux 2.4.1-0.1.9 i586)")
    req.add_header("AUTHORIZATION", string.strip("Basic " +
        base64.encodestring("%s:%s" % (router_user, router_password))))
    IP = re.compile('(\d{1,3}\.){3}\d{1,3}')
    for line in urllib2.urlopen(req):
        if IP.search(line):
            ip = line.strip()
            return ip

The first time I run this script, I always get a urllib2.HTTPError.  But
then If I ran the exact same script again afterward, I get access.

>>> lookup.mr814('username', 'password')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "lookup.py", line 13, in mr814
    for line in urllib2.urlopen(req):
  File "/usr/lib/python2.3/urllib2.py", line 136, in urlopen
    return _opener.open(url, data)
  File "/usr/lib/python2.3/urllib2.py", line 333, in open
    '_open', req)
  File "/usr/lib/python2.3/urllib2.py", line 313, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.3/urllib2.py", line 849, in http_open
    return self.do_open(httplib.HTTP, req)
  File "/usr/lib/python2.3/urllib2.py", line 843, in do_open
    return self.parent.error('http', req, fp, code, msg, hdrs)
  File "/usr/lib/python2.3/urllib2.py", line 359, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.3/urllib2.py", line 313, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.3/urllib2.py", line 419, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 401: Unauthorized
>>> lookup.mr814('admin', '54wins')
'66.72.200.255'

This is baffling me.  Does the webserver ignore the AUTHORIZATION header
on the first request?  Should I do something like this:

>>> try:
...     lookup.mr814('admin', '54wins')
... except urllib2.HTTPError: #I only want to catch 401 here, but how?
...     print "I got back the 401 error.  Trying again..."
...     lookup.mr814('admin', '54wins')
... except:
...     print "something else went wrong."

If that is recommended, how do I distinguish between 401 errors, vs
other errors?

Thanks for the help.




More information about the Python-list mailing list