[Tutor] Cookies and authorization - urllib2 vs urllib

D. Hartley denise.hartley at gmail.com
Thu Jun 23 22:59:07 CEST 2005

>From Christian: 

> Try subclassing urllib.FancyURLopener and overriding the
> prompt_user_passwd() method.  That should get you what you need :-)

Well, I used urllib.FancyURLopener, and can open and look at the url, like this:

import urllib
opener2 = urllib.FancyURLopener({})
f = opener2.open("http://www.pythonchallenge.com/pc/return/romance.html")

..but to get at the cookies, I need to use urllib2.build_opener
instead of urllib.FancyURLopener so that I can have access to
urllib2.HTTPCookieProcessor, which does not seem to be an option in
the regular urllib module?  Sorry if this seems like a dense question,
I have little-to-no experience with cookies (and very little with
urllib itself), so the examples sometimes leave me hanging!

Does anyone know a way to use an opener (like that above from urllib2)
that can process cookies AND can pass in a user/pass (like
FancyURLopener from urllib)? I'm not having much luck trying to do
both things at once!


P.S. Kent - thank you for the helpful tutorials on authentication,
they really cleared up the process a great deal.  The only problem is:

When I create an opener to process the cookies, it looks like this:

opener = 

..where myjar is cookielib.CookieJar()

But in the examples for authentication, when I create the opener:

opener = urllib2.build_opener(authhandler)

..where authhandler is urllib2.HTTPBasicAuthHandler(passwordmanager)

So both use build_opener, but the thing I pass in, from what I am
looking at so far, has to be a cookie processor OR an authenticator. 
How can I do both at once?

More information about the Tutor mailing list