[issue9637] docs do not say that urllib uses HTTP_PROXY
New submission from Kirikaza <kirikaza@rambler.ru>: In practice urllib reads HTTP_PROXY firstly and then if HTTP_PROXY is empty urllib reads http_proxy. Documentation (http://docs.python.org/library/urllib.html) says nothing about HTTP_PROXY. Maybe it affects all the versions of Python. ---------- assignee: docs@python components: Documentation messages: 114319 nosy: docs@python, kirikaza priority: normal severity: normal status: open title: docs do not say that urllib uses HTTP_PROXY versions: Python 2.6 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: Are you talking about top-level code within the urllib module or code within defined functions. If the former, can you quote or point to the place in the file? If the latter, which functions? Just urlopen or others? Does urllib2.urlopen have the same issue? (That replaces urllib.urlopen in 3.x). Having answered the above, please suggest a specific change in a specific place in the docs. ---------- nosy: +terry.reedy stage: -> needs patch versions: +Python 2.7 -Python 2.6 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
Kirikaza <kirikaza@rambler.ru> added the comment:
Are you talking about ... I have read no line of code from urllib module. I just try to use urllib.urlopen() and I see it uses not only http_proxy but also HTTP_PROXY and urlopen() prefers the latter variable.
a specific change in a specific place
Let's consider a two-lines sample proxy.py: import urllib urllib.urlopen('http://python.org/ftp/').readlines() Some tests: $ http_proxy= HTTP_PROXY= proxy.py ['<!DOCTYPE HTML ... \n'] $ http_proxy=lower:80 HTTP_PROXY= proxy.py IOError: invalid proxy for http: 'lower:80' $ http_proxy= HTTP_PROXY=UPPER:80 proxy.py IOError: invalid proxy for http: 'UPPER:80' $ http_proxy=lower:80 HTTP_PROXY=UPPER:80 proxy.py IOError: invalid proxy for http: 'UPPER:80' place: 20.5.1, paragraph 7 (the only one where *_proxy are discussed) change: add some words about HTTP_PROXY and maybe about FTP_PROXY
does urllib2.urlopen ... (in 3.x) I don't have Python 3.x so I can't say anything.
---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
Kirikaza <kirikaza@rambler.ru> added the comment: I missed the print statement in that example... So I attached the file with correct code. ---------- Added file: http://bugs.python.org/file18594/proxy.py _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
Changes by Neil Muller <drnlmuller+bugs@gmail.com>: ---------- versions: +Python 3.2 -Python 2.7 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
Neil Muller <drnlmuller+bugs@gmail.com> added the comment: The problem is that the getproxies_environment function converts all environment variables to lower case before deciding whether to use the name. This means that whichever ends up last in os.environment will be used. The attached patch does two things. It changes the behaviour to prefer the all lower case version if it exists, and adds a note to the ProxyHandler description noting that the environment variable used is not case sensitive, although the all lower case version will be preferred. The behaviour change is debatable, but preferring the lower case name is likely to be less surprising in general. ---------- keywords: +patch nosy: +Neil Muller Added file: http://bugs.python.org/file19678/urllib_PROXY.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
Neil Muller <drnlmuller+bugs@gmail.com> added the comment: This patch is now more than a year old with no comment. What needs to happen to get this bug fixed? ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: Senthil, any comment on this? ---------- nosy: +orsenthil _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
Senthil Kumaran <senthil@uthcode.com> added the comment: Sorry, this escaped for so long. First thing is, having different values for HTTP_PROXY and HTTP_proxy and http_proxy is plain wrong at the user side. They all should be same and in that case the possible problem which Kirikaza and Neil Muller state is extremely unlikely (and if the user faces any issue, it is due to improper HTTP configuration). The reason that lowering of environ vars is performed is similar to any string match with input from user where we want to ignore the case sensitivity. Having said that, I think, I shall just add teh info on lowering and verification of proxy environment to the docs. I am -1 on changing any behavior for this issue. ---------- assignee: docs@python -> orsenthil _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
Roundup Robot <devnull@psf.upfronthosting.co.za> added the comment: New changeset 3370fa13ed73 by Senthil Kumaran in branch '3.2': Issue9637 - Explain in getproxies_environment that <scheme>_proxy environ variable case does not matter. http://hg.python.org/cpython/rev/3370fa13ed73 New changeset eb028b3c62c9 by Senthil Kumaran in branch 'default': Merge from 3.2 - Issue9637 - Explain in getproxies_environment that <scheme>_proxy environ variable case does not matter. http://hg.python.org/cpython/rev/eb028b3c62c9 ---------- nosy: +python-dev _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
Roundup Robot <devnull@psf.upfronthosting.co.za> added the comment: New changeset 4a30eae3b945 by Senthil Kumaran in branch '2.7': port to 2.7 - Issue9637 - Details that case of Proxy env var does not matter. http://hg.python.org/cpython/rev/4a30eae3b945 ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
Senthil Kumaran <senthil@uthcode.com> added the comment: This is fixed. Thanks for your contribution. ---------- resolution: -> fixed stage: needs patch -> committed/rejected status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue9637> _______________________________________
participants (5)
-
Kirikaza
-
Neil Muller
-
Roundup Robot
-
Senthil Kumaran
-
Terry J. Reedy