[Python-bugs-list] [ python-Bugs-413135 ] urllib2 fails with proxy requiring auth

[noreply@sourceforge.net]

Bugs item #413135, was updated on 2001-04-02 07:14
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=413135&group_id=5470

Category: Python Library
Group: None
>Status: Closed
Priority: 5
Submitted By: Paul Moore (pmoore)
Assigned to: Moshe Zadka (moshez)
Summary: urllib2 fails with proxy requiring auth

Initial Comment:
The following program:

import urllib2

proxy_info = {
    'user' : 'my_name', 'pass' : 'my_pass',
    'host' : "my-proxy", 'port' : 80
}
 
# build a new opener that uses a proxy requiring
# authorization
proxy_support = urllib2.ProxyHandler(
    {"http" :
     "http://%(user)s:%(pass)s@%(host)s:%(port)d"
     % proxy_info})
opener = urllib2.build_opener(proxy_support, 
urllib2.HTTPHandler)
 
# install it
urllib2.install_opener(opener)
 
f = urllib2.urlopen('http://www.python.org/')
print f.headers
print f.read()


fails with the following error on Python 2.1b2 (on 
Windows)

C:\Data>python21 proxy_auth.py
Traceback (most recent call last):
  File "proxy_auth.py", line 18, in ?
    f = urllib2.urlopen('http://www.python.org/')
  File "c:\applications\python21\lib\urllib2.py", line 
135, in urlopen
    return _opener.open(url, data)
  File "c:\applications\python21\lib\urllib2.py", line 
318, in open
    '_open', req)
  File "c:\applications\python21\lib\urllib2.py", line 
297, in _call_chain
    result = func(*args)
  File "c:\applications\python21\lib\urllib2.py", line 
823, in http_open
    return self.do_open(httplib.HTTP, req)
  File "c:\applications\python21\lib\urllib2.py", line 
801, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error host not found>

A similar error occurred in beta 1, but this was 
reported as bug 406683. The fix is in beta 2. I 
applied the fix manually in beta 1, and it worked, so 
I can only assume that something else changed in the 
transition from beta 1 to beta 2, which broke this 
again.


----------------------------------------------------------------------

>Comment By: Moshe Zadka (moshez)
Date: 2001-04-11 00:45

Message:
Logged In: YES 
user_id=11645

You're right!
I've fixed this in urllib2.py v 1.12

----------------------------------------------------------------------

Comment By: Paul Moore (pmoore)
Date: 2001-04-10 02:46

Message:
Logged In: YES 
user_id=113328

I found the problem. In urllib2.py, class 
AbstractHTTPHandler, method do_open, the first line is now 

        host = urlparse.urlparse(req.get_full_url())[1]

It used to be

        host = req.get_host()

With the old version, the code works (with my proxy). With 
the new version it doesn't, as it passes the destination 
host, rather than the proxy name (and so loses the proxy 
info totally).

Paul.


----------------------------------------------------------------------

Comment By: Paul Moore (pmoore)
Date: 2001-04-10 02:45

Message:
Logged In: YES 
user_id=113328

I found the problem. In urllib2.py, class 
AbstractHTTPHandler, method do_open, the first line is now 

        host = urlparse.urlparse(req.get_full_url())[1]

It used to be

        host = req.get_host()

With the old version, the code works (with my proxy). With 
the new version it doesn't, as it passes the destination 
host, rather than the proxy name (and so loses the proxy 
info totally).

Paul.


----------------------------------------------------------------------

Comment By: Moshe Zadka (moshez)
Date: 2001-04-09 07:11

Message:
Logged In: YES 
user_id=11645

I've just tested with my installation of Python 2.1b2 and it
works. So I cannot reproduce the problem, and I need more
information from you: can you insert prints in the correct
places (e.g. do_open) to see what host urllib2 *thinks* it
is trying to access?
Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=413135&group_id=5470