[Tutor] urllib2 with a proxy
Lloyd Kvam
pythonTutor at venix.com
Fri Nov 12 20:49:51 CET 2004
The realm is defined by the host you connect to. A URL consists of
(protocol)://(host)/(resource)?(parameters)
Each resource can be assigned to a security realm by the system
administrator on the host. When the host wants to check to see if you
should be allowed access to the resource, it sends a response back to
your browser:
HTTP/1.0 401 Unauthorized
WWW-Authenticate: Basic realm="Some string which is the realm name"
Your browser will ask you for your username and password and remember
them along with the realm name. Then whenever the host returns
Unauthorized for that realm, the browser will supply your username and
password.
Looking below:
"http//:proxy:8080"
should be:
"http://proxy:8080"
On Fri, 2004-11-12 at 20:03, kbond wrote:
> Kent,
>
> I did test your suggestion but unfortunatly it is still not working.
> Somehow I do not understand why it is so complex to go trough my
> proxy. I do not have any trouble if I use firefox or mozilla.
> I am completely stuck by this tiny detail.
> If someone on this list has clear and simple explanation about what is
> realm and host in "add_password('realm', 'host', 'username',
> 'password')" that will also help me a lot.
>
> Thank you for your help any pointer or code sample are more than welcome.
>
> *code** *
> import urllib2
> # set up authentication info
> authinfo = urllib2.HTTPBasicAuthHandler()
> #add_password('realm', 'host', 'username', 'password')
> authinfo.add_password(None, "http://google.com", 'username', 'pwd')
> proxy_support = urllib2.ProxyHandler({"http" : "http//:proxy:8080"})
> # build a new opener that adds authentication and caching FTP handlers
> opener = urllib2.build_opener(proxy_support,
> authinfo,urllib2.CacheFTPHandler())
> # install it
> urllib2.install_opener(opener)
> f = urllib2.urlopen('http://google.com')
> buf = f.read()
> print buf
> f.close()
>
>
> *error*
>
> Traceback (most recent call last):
> File "testProxy.py", line 12, in ?
> f = urllib2.urlopen('http://google.com')
> File "E:\users\install\Python\lib\urllib2.py", line 129, in urlopen
> return _opener.open(url, data)
> File "E:\users\install\Python\lib\urllib2.py", line 326, in open
> '_open', req)
> File "E:\users\install\Python\lib\urllib2.py", line 306, in _call_chain
> result = func(*args)
> File "E:\users\install\Python\lib\urllib2.py", line 491, in <lambda>
> lambda r, proxy=url, type=type, meth=self.proxy_open: \
> File "E:\users\install\Python\lib\urllib2.py", line 498, in proxy_open
> if '@' in host:
> TypeError: iterable argument required
>
>
>
> kbond wrote:
>
> > Thank you for your help kent.
> > Already the second time that you are helping me.
> > In fact I face this proxy trouble right after your first answer I was
> > about to try your suggestion to solve the clientForm trouble when this
> > proxy came into the danse.
> >
> > When I started my project I was far to imagine that talking to the
> > world wide web was that difficult by script.
> > Unfortunatly I will not be able to test before monday because I do not
> > use a proxy at home.
> > Do you have an idea of what are realm and host in the add_password()
> > method?
> > I am making the assomption that host stand for the host I want to
> > contact but I do not have any idea of what realm is?
> > Do you know how I can get it? Why firefox don't need this information
> > to go out?
> >
> > You will find below the code I will try on monday.
> > Thank you for your help
> >
> > *Code*
> > import urllib2
> > # set up authentication info
> > authinfo = urllib2.HTTPBasicAuthHandler()
> > #add_password('realm', 'host', 'username', 'password')
> > authinfo.add_password(None, "http://google.com", 'userName', 'pwd')
> > proxy_support = urllib2.ProxyHandler({"http" : "http//:proxy:8080"})
> > # build a new opener that adds authentication and caching FTP handlers
> > opener = urllib2.build_opener(proxy_support,
> > authinfo,urllib2.CacheFTPHandler())
> > # install it
> > urllib2.install_opener(opener)
> > f = urllib2.urlopen('http://google.com')
> > buf = f.read()
> > print buf
> > f.close()
> >
> > Kent Johnson wrote:
> >
> >> I'm not familiar with this library but looking at your code and the
> >> docs I can make some guesses where the problem might be:
> >> - The second argument to ProxyHandler should be a complete URL, e.g.
> >> 'http://proxy:8080'. This is the cause of the exception you are seeing.
> >> - Try using a HTTPBasicAuthHandler instead of a HTTPDigestAuthHandler
> >> unless you are sure your proxy uses digest authentication
> >> - You should pass an instance of CacheFTPHandler to build_opener, not
> >> the class itself. In other words, pass CacheFTPHandler() with
> >> parentheses.
> >>
> >> HTH
> >> Kent
> >>
> >> At 10:56 AM 11/6/2004 -0500, kbond wrote:
> >>
> >>> hello,
> >>>
> >>> I am trying to get access an internet page using urllib2. I am able
> >>> to do so form home where I do not have proxy but for some reason I
> >>> am unable to do it form my company.
> >>>
> >>> I found many ressource on the internet explaining how to do it but I
> >>> was unable to apply it to my contexte. The interesting thing is that
> >>> I did manage to configure Firefox (my fovourite browser) so I guess
> >>> I should have all the information to do it by script.
> >>> I think I am doing something is the usage of "add_password". Could
> >>> you please explain me what realm and host are? and how can I get them?
> >>>
> >>> You will find bellow my firefox configuration.
> >>> It would be great If someone can help me to put all this together.
> >>> Thank you for your help
> >>>
> >>> *Code*
> >>> import urllib2
> >>> # set up authentication info
> >>> authinfo = urllib2.HTTPDigestAuthHandler()
> >>> #proxy_auth_handler.add_password('realm', 'host', 'username',
> >>> 'password')
> >>> authinfo.add_password(None, "http://google.com", 'userName', 'pwd')
> >>> * *
> >>> proxy_support = urllib2.ProxyHandler({"http" : "proxy:8080"})
> >>> # build a new opener that adds authentication and caching FTP handlers
> >>> opener = urllib2.build_opener(proxy_support,
> >>> authinfo,urllib2.CacheFTPHandler)
> >>> # install it
> >>> urllib2.install_opener(opener)
> >>> f = urllib2.urlopen('http://google.com')
> >>> buf = f.read()
> >>> print buf
> >>> f.close()
> >>>
> >>> *Error Message*
> >>>
> >>> Traceback (most recent call last):
> >>> File "test.py", line 18, in ?
> >>> f = urllib2.urlopen('http://google.com')
> >>> File "E:\users\install\Python\lib\urllib2.py", line 129, in urlopen
> >>> return _opener.open(url, data)
> >>> File "E:\users\install\Python\lib\urllib2.py", line 326, in open
> >>> '_open', req)
> >>> File "E:\users\install\Python\lib\urllib2.py", line 306, in
> >>> _call_chain
> >>> result = func(*args)
> >>> File "E:\users\install\Python\lib\urllib2.py", line 491, in <lambda>
> >>> lambda r, proxy=url, type=type, meth=self.proxy_open: \
> >>> File "E:\users\install\Python\lib\urllib2.py", line 498, in proxy_open
> >>> if '@' in host:
> >>> TypeError: iterable argument required
> >>>
> >>> *Firefox configuration*
> >>>
> >>>
> >>> _______________________________________________
> >>> Tutor maillist - Tutor at python.org
> >>> http://mail.python.org/mailman/listinfo/tutor
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> Tutor maillist - Tutor at python.org
> >> http://mail.python.org/mailman/listinfo/tutor
> >>
> >>
> >
> > _______________________________________________
> > Tutor maillist - Tutor at python.org
> > http://mail.python.org/mailman/listinfo/tutor
> >
> >
> _______________________________________________
> Tutor maillist - Tutor at python.org
> http://mail.python.org/mailman/listinfo/tutor
--
Lloyd Kvam
Venix Corp
More information about the Tutor
mailing list