[Tutor] urllib2 with a proxy

[kbond]

Kent,

I did test your suggestion but unfortunatly it is still not working.
Somehow I do not understand  why it  is so  complex  to  go trough  my 
proxy.  I do not have  any trouble if I use  firefox or mozilla.
I am completely stuck by this tiny detail.
If someone on this list has clear and simple explanation about what is 
realm and host in "add_password('realm', 'host', 'username', 
'password')" that will also help me a lot.

Thank you for your help any pointer or code sample are more than welcome.

*code** *
import urllib2
# set up authentication info
authinfo = urllib2.HTTPBasicAuthHandler()
#add_password('realm', 'host', 'username', 'password')
authinfo.add_password(None, "http://google.com", 'username', 'pwd')
proxy_support = urllib2.ProxyHandler({"http" : "http//:proxy:8080"})
# build a new opener that adds authentication and caching FTP handlers
opener = urllib2.build_opener(proxy_support,
authinfo,urllib2.CacheFTPHandler())
# install it
urllib2.install_opener(opener)
f = urllib2.urlopen('http://google.com')
buf = f.read()
print buf
f.close()


*error*

Traceback (most recent call last):
 File "testProxy.py", line 12, in ?
   f = urllib2.urlopen('http://google.com')
 File "E:\users\install\Python\lib\urllib2.py", line 129, in urlopen
   return _opener.open(url, data)
 File "E:\users\install\Python\lib\urllib2.py", line 326, in open
   '_open', req)
 File "E:\users\install\Python\lib\urllib2.py", line 306, in _call_chain
   result = func(*args)
 File "E:\users\install\Python\lib\urllib2.py", line 491, in <lambda>
   lambda r, proxy=url, type=type, meth=self.proxy_open: \
 File "E:\users\install\Python\lib\urllib2.py", line 498, in proxy_open
   if '@' in host:
TypeError: iterable argument required



kbond wrote:

> Thank you for your help kent.
> Already the second time that you are helping me.
> In fact I face this proxy trouble right after your first answer I was 
> about to try your suggestion to solve the clientForm trouble when this 
> proxy came into the danse. 
>
> When I started my project I was far to imagine that talking to the 
> world wide web was that difficult by script.
> Unfortunatly I will not be able to test before monday because I do not 
> use a proxy at home.
> Do you have an idea of what are realm and  host in the add_password() 
> method?
> I am making the assomption that host stand for the host I want to 
> contact but I do not have any idea of what realm is?
> Do you know how I can get it? Why firefox don't need this information 
> to go out?
>
> You will find below the code I will try on monday.
> Thank you for your help
>
> *Code*
> import urllib2
> # set up authentication info
> authinfo = urllib2.HTTPBasicAuthHandler()
> #add_password('realm', 'host', 'username', 'password')
> authinfo.add_password(None, "http://google.com", 'userName', 'pwd')
> proxy_support = urllib2.ProxyHandler({"http" : "http//:proxy:8080"})
> # build a new opener that adds authentication and caching FTP handlers
> opener = urllib2.build_opener(proxy_support, 
> authinfo,urllib2.CacheFTPHandler())
> # install it
> urllib2.install_opener(opener)
> f = urllib2.urlopen('http://google.com')
> buf = f.read()
> print buf
> f.close()
>
> Kent Johnson wrote:
>
>> I'm not familiar with this library but looking at your code and the 
>> docs I can make some guesses where the problem might be:
>> - The second argument to ProxyHandler should be a complete URL, e.g. 
>> 'http://proxy:8080'. This is the cause of the exception you are seeing.
>> - Try using a HTTPBasicAuthHandler instead of a HTTPDigestAuthHandler 
>> unless you are sure your proxy uses digest authentication
>> - You should pass an instance of CacheFTPHandler to build_opener, not 
>> the class itself. In other words, pass CacheFTPHandler() with 
>> parentheses.
>>
>> HTH
>> Kent
>>
>> At 10:56 AM 11/6/2004 -0500, kbond wrote:
>>
>>> hello,
>>>
>>> I am trying to get access an internet page  using urllib2. I am able 
>>> to do so form home where I do not have proxy but for some reason I 
>>> am unable to do it form my company.
>>>
>>> I found many ressource on the internet explaining how to do it but I 
>>> was unable to apply it to my contexte. The interesting thing is that 
>>> I did manage to configure Firefox (my fovourite browser) so I guess  
>>> I should have all the information to do it by script.
>>> I think I am doing something is the usage of "add_password". Could 
>>> you please explain me what realm and host are? and how can I get them?
>>>
>>> You will find bellow my firefox configuration.
>>> It would be great If someone can help me to put all this together.
>>> Thank you for your help
>>>
>>> *Code*
>>> import urllib2
>>> # set up authentication info
>>> authinfo = urllib2.HTTPDigestAuthHandler()
>>> #proxy_auth_handler.add_password('realm', 'host', 'username', 
>>> 'password')
>>> authinfo.add_password(None, "http://google.com", 'userName', 'pwd')  
>>> * *
>>> proxy_support = urllib2.ProxyHandler({"http" : "proxy:8080"})
>>> # build a new opener that adds authentication and caching FTP handlers
>>> opener = urllib2.build_opener(proxy_support, 
>>> authinfo,urllib2.CacheFTPHandler)
>>> # install it
>>> urllib2.install_opener(opener)
>>> f = urllib2.urlopen('http://google.com')
>>> buf = f.read()
>>> print buf
>>> f.close()
>>>
>>> *Error Message*
>>>
>>> Traceback (most recent call last):
>>>  File "test.py", line 18, in ?
>>>    f = urllib2.urlopen('http://google.com')
>>>  File "E:\users\install\Python\lib\urllib2.py", line 129, in urlopen
>>>    return _opener.open(url, data)
>>>  File "E:\users\install\Python\lib\urllib2.py", line 326, in open
>>>    '_open', req)
>>>  File "E:\users\install\Python\lib\urllib2.py", line 306, in 
>>> _call_chain
>>>    result = func(*args)
>>>  File "E:\users\install\Python\lib\urllib2.py", line 491, in <lambda>
>>>    lambda r, proxy=url, type=type, meth=self.proxy_open: \
>>>  File "E:\users\install\Python\lib\urllib2.py", line 498, in proxy_open
>>>    if '@' in host:
>>> TypeError: iterable argument required
>>>
>>> *Firefox configuration*
>>>
>>>
>>> _______________________________________________
>>> Tutor maillist  -  Tutor at python.org
>>> http://mail.python.org/mailman/listinfo/tutor
>>
>>
>>
>>
>> _______________________________________________
>> Tutor maillist  -  Tutor at python.org
>> http://mail.python.org/mailman/listinfo/tutor
>>
>>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>