[Python-bugs-list] [ python-Bugs-445815 ] urllib doesn't handle proxy exceptions

[noreply@sourceforge.net]

Bugs item #445815, was opened at 2001-07-29 19:01
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445815&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Dylan Jay (djsq)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib doesn't handle proxy exceptions

Initial Comment:
When using urllib in a firewall environment when a 
proxy is set, the proxy will be used for all 
connections regardless of if the host is located 
inside or outside the firewall. Localhost is a good 
example of this.
Under a windows environment the registry contains a 
proxy exception list which contains patterns which 
match hosts that should not be directed through the 
proxy. This should be used.
Under other environments perhaps an environment 
variable be used to determine the exceptions
eg
http_proxy_expceptions 
= "*.mydomain.com;*.myotherdomain.com"

Localhost should never be directed through a proxy.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-11 20:51

Message:
Logged In: YES 
user_id=6380

Look at the urllib.py in Python 2.2b2. Does that do what you
want?

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-12-11 15:11

Message:
Logged In: NO 

ok, there are really two problems here

1. If I have a proxy set and I go say 
print urllib.urlopen("http://localhost").read()
I go to localhost.com outside the firewall. This is because 
urllib is not recognizing 'localhost' as a special case. 

2. There is no way to override the default behaviour of 
always using a proxy when it is available. At least under 
windows, if urllib can find the registry entries for the 
proxy it will use them whether you want to or not. 

The solutions I can see are as follows

1. I think the special case handling for 'localhost' is 
broken. localhost should always resolve to the local 
computer regardless of presence of a proxy.

2. There are a couple of options here. 

 - extra arguments on url calls
 - setting modes on urllib
 - or being able to set up internal link patterns (as 
suggested in my orginal post).

 The latter is what windows internet explorer does (I think 
netscape too). You specify a simple pattern that if the 
domain matches, will result in the url call being made 
locally without the use of the proxy. 
This information is already in the windows registery so 
that would be the ideal place to get this information on 
the windows platform. On other platforms I would suggest an 
evornonment variable.

PS. The code you reference just gets the normal proxy 
information. Not the proxy exception information.

PPS. I tested this behaviour on win2k, python 2.1

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-11 14:46

Message:
Logged In: YES 
user_id=6380

I think there's code in urllib.py that tries to do exactly
wht you're saying. It's towards the end of urllib.py, after
"elif os.name == 'nt':". Can you perhaps instrument that
code with a few print statements to see why it doesn't do
what you expect? I can't debug this because we don't have a
proxy set up here, and I have no idea how to configure this
information on Windows.

It might help if you explained is which Windows version (95,
98, 2K, ME, XT?), of if you have experienced this on
multiple Windows versions.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=445815&group_id=5470