[Tutor] Seeing response from authorization page with urllib2

[eryksun]

On Tue, Sep 11, 2012 at 4:29 AM, Ray Jones <crawlzone at gmail.com> wrote:
>
> But when I attempt to get any part of an authorization-required page
> using urllib2.urlopen(), I immediately receive the 401 error. Even the
> intended object variable is left in an undefined state, so using info()
> doesn't seem to work. How can I get that information from the server?

The HTTPError object has the information you want: url, code, reason,
headers -- along with the methods info, read, readline, and readlines.

http://docs.python.org/library/urllib2#urllib2.HTTPError

For example, if you catch the error as e, then you can look at
e.headers['www-authenticate'] to find the realm.

Basic HTTP Authentication:
http://docs.python.org/library/urllib2#examples

You can also use an HTTPPasswordMgrWithDefaultRealm with a catch-all
realm (realm=None):

    pass_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
    auth_handler = urllib2.HTTPBasicAuthHandler(pass_mgr)
    auth_handler.add_password(
        realm=None,
        uri='https://mahler:8092/site-updates.py',
        user='klem',
        passwd='kadidd!ehopper')
    opener = urllib2.build_opener(auth_handler)

Optionally, install the opener:

    urllib2.install_opener(opener)

Or just use opener.open().