[issue13211] urllib2.HTTPError does not have 'reason' attribute.
New submission from Jason R. Coombs <jaraco@jaraco.com>: The urllib2 docs indicate that HTTPError is a subclass of URLError and that URLError has an attribute of 'reason', but HTTPError does not have this attribute. The docs should be updated to reflect this deviance. It appears the Python 3.2 docs no longer include documentation for URLError Python 2.7.2
try:urllib2.urlopen('http://api.wordnik.com/v4/word.json/foo/examples') ... except urllib2.HTTPError as exc: ... print(dir(exc)) ... ['_HTTPError__super_init', '__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__getitem__', '__getslice__', '__hash__', '__init__', '__iter__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__unicode__', '__weakref__', 'args', 'close', 'code', 'errno', 'filename', 'fileno', 'fp', 'getcode', 'geturl', 'hdrs', 'headers', 'info', 'message', 'msg', 'next', 'read', 'readline', 'readlines', 'strerror', 'url']
The same issue exists in Python 3.2.2. Here's what I propose: - For Python 3.2 and 3.3, update HTTPError to supply a @property, aliasing .msg (will .msg always be a suitable .reason?). - For Python 2.7, document the deviance, such as by adding the following wording to the HTTPError docs: "Unlike URLError, HTTPError does not supply a reason attribute. The reason can be retrieved through the msg attribute." ---------- assignee: docs@python components: Documentation messages: 145805 nosy: docs@python, jason.coombs priority: normal severity: normal status: open title: urllib2.HTTPError does not have 'reason' attribute. versions: Python 2.7, Python 3.2, Python 3.3 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Petri Lehtinen <petri@digip.org> added the comment:
It appears the Python 3.2 docs no longer include documentation for URLError
Both URLError and HTTPError are documented in 3.2 and 3.3: http://docs.python.org/py3k/library/urllib.error.html#urllib.error.URLError http://docs.python.org/dev/library/urllib.error.html#urllib.error.URLError It needs to be investigated whether each point raising a HTTPError sets a good msg. If this is the case, reason can be aliased to msg. This could be done for 2.7, too, because this is a bug (deviation from what the documentation says) rather than a new feature. ---------- keywords: +easy nosy: +petri.lehtinen stage: -> needs patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Jason R. Coombs <jaraco@jaraco.com> added the comment: I scanned through the libs for Python 2.7, 3.2, and 3.3 and there is no construction of HTTPError that does not pass a string for msg. I believe it would be reasonable to alias reason to msg. I'll put together the changesets. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Changes by Senthil Kumaran <senthil@uthcode.com>: ---------- nosy: +orsenthil _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Changes by Jason R. Coombs <jaraco@jaraco.com>: ---------- hgrepos: +88 keywords: +needs review, patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Changes by Jason R. Coombs <jaraco@jaraco.com>: Added file: http://bugs.python.org/file23627/fffeff7721c0.diff _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Jason R. Coombs <jaraco@jaraco.com> added the comment: I've created three changesets, addressing the issue in 2.7, 3.2, and 3.3, including tests. Please review and comment. If there are no objections, I'll push the changesets after 24 hours. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Petri Lehtinen <petri@digip.org> added the comment: Perhaps the reason should include the status code, too? It makes HTTP errors much more useful, as you'll immediately see what's going on from the status code. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Jason R. Coombs <jaraco@jaraco.com> added the comment: My initial instinct was to agree - the status code is useful. However, in looking at the FTP code, it sometimes just sets other objects (socket.error for example) as the 'reason'. The docs say 'reason' is a string or another exception. I'm tempted to leave it as is. This fix is mainly to provide a reasonable value for .reason. The __str__ and .code are already exposed, so to create a reason with a code would create yet another string representation of the error which is different from every other representation already present. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Petri Lehtinen <petri@digip.org> added the comment: Ok. Sounds that you've already evaluated this alternative, so I'm fine with it. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Senthil Kumaran <senthil@uthcode.com> added the comment: Hi Jason & Petri, urllib2.HTTPError never had a reason attribute. In the docs, it is mentioned that only URLError has the reason attribute. The HTTPError sublasses URLError and addinforurl class, but the further initialization happens only in the addinforurl class atrs. If you got confused that the HTTPError as args and not reason, the 'args' is not coming from URLError. HTTPError is raised for peculiar conditions such like authentication failures and it is 'used' as expected failure for certain authentication conditions. URLError is not so, it is seen more as an exception like socket errors. The example your illustrated is an Authentication failure and as per docs, it is guaranteed to have code attribute to verify the kind of HTTP error are getting. msg is corresponding HTTP error code msg. Take this example for URLError which will have reason attribute, it will work in both 2.7,3.2 and 3.3 import urllib.request, urllib.error, urllib.parse try: urllib.request.urlopen('http://aninvalidsite/something') except urllib.error.URLError as exc: print(exc.reason) Because this is a socket error, the reason as "[Errno -2] Name or service not known" and HTTPError may not be a proper exception for this. This is more of an IOError which urllib calls a URLError. I am not sure, how the need for 'reason' attribute for HTTPError exception was felt as the docs just say about 'code'. (HTTPError is informed as a subclass of URLError, but HTTPError does not call the URLError 's __init__ and acts standalone. I am not sure, how to go about with this bug. If a new .reason attribute has to be added to HTTPError, then it is a feature request though, I wonder why we need when code and msg serve an adequate purpose. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Petri Lehtinen <petri@digip.org> added the comment: My impression was that because HTTPError is a subclass of URLError, it should have the same attributes, and maybe some extra. A user reading the docs doesn't know whether it calls the base class __init__ or not. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Jason R. Coombs <jaraco@jaraco.com> added the comment: That was my point. If HTTPError is a subclass of URLError, then an HTTPError _is an_ URLError, and thus should implement the same public interface. The problem is better illustrated with this request: try: urllib.request.urlopen('http://api.wordnik.com/v4/word.json/foo/examples') except urllib2.URLError as exc: # We caught a URLError, what's the reason? print(exc.reason) This code will fail with an attribute error, but only when the except clause catches an HTTPError (as it does in this case). The documentation explicitly states that HTTPError is a subclass of URLError and it doesn't say anything about the addinfourl interface. The documentation strongly suggests (though implicitly) that HTTPError.reason will be present as it is with URLError. If HTTPError does not implement the reason attribute, then there is no value in making it the subclass of URLError, and HTTPError should probably have the same parent class as URLError. However, this change is even more drastic and would almost certainly violate backward compatibility constraints. The proposal I've made is generally backward compatible, and addresses the underlying bug (that URLError.__init__ is never called), and the symptom that a HTTPError does not implement the documented public interface. Perhaps there's a better approach, but I believe based on the example code above, the implementation is broken. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Changes by Jason R. Coombs <jaraco@jaraco.com>: ---------- components: +Library (Lib) _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Roundup Robot <devnull@psf.upfronthosting.co.za> added the comment: New changeset ee94b89f65ab by Jason R. Coombs in branch '2.7': Issue #13211: Add .reason attribute to HTTPError to implement parent class (URLError) interface. http://hg.python.org/cpython/rev/ee94b89f65ab New changeset abfe76a19f63 by Jason R. Coombs in branch '3.2': Issue #13211: Add .reason attribute to HTTPError to implement parent class (URLError) interface. http://hg.python.org/cpython/rev/abfe76a19f63 New changeset deb60efd32eb by Jason R. Coombs in branch 'default': Merged fix for #13211 from 3.2 http://hg.python.org/cpython/rev/deb60efd32eb ---------- nosy: +python-dev _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Changes by Jason R. Coombs <jaraco@jaraco.com>: ---------- resolution: -> fixed status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Antoine Pitrou <pitrou@free.fr> added the comment: The 3.x buildbots are all broken: http://www.python.org/dev/buildbot/all/waterfall?category=3.x.stable&category=3.x.unstable ---------- nosy: +pitrou _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Arfrever Frehtes Taifersar Arahesis <Arfrever.FTA@GMail.Com> added the comment: I suspect that this problem is caused by the fix for issue #12555. ---------- nosy: +Arfrever _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Jason R. Coombs <jaraco@jaraco.com> added the comment: Antoine, I think Arfrever is on to something here. It does appear the new test added to test_urllib2 is failing, but for reasons I don't exactly understand. I'm initializing the HTTPError instance according to its signature in urllib.error, but it fails to accept keyword arguments. I guess for the sake of this test, I can pass positional arguments, though this added limitation should probably be addressed in #12555. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Roundup Robot <devnull@psf.upfronthosting.co.za> added the comment: New changeset a3ddee916808 by Jason R. Coombs in branch 'default': Pass positional arguments - HTTPError is not accepting keyword arguments. Reference #13211 and #12555. http://hg.python.org/cpython/rev/a3ddee916808 ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Jason R. Coombs <jaraco@jaraco.com> added the comment: After yet another commit, the build bots are green again: http://hg.python.org/cpython/rev/8fa1dc66de5d ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Changes by Terry J. Reedy <tjreedy@udel.edu>: ---------- versions: +Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Roundup Robot added the comment: New changeset e1ba514ddcd2 by Senthil Kumaran in branch '3.2': Fix issue13211 - Document the reason attribute for urllib.error.HTTPError http://hg.python.org/cpython/rev/e1ba514ddcd2 ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
Changes by Berker Peksag <berker.peksag@gmail.com>: ---------- stage: needs patch -> resolved _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue13211> _______________________________________
participants (8)
-
Antoine Pitrou -
Arfrever Frehtes Taifersar Arahesis
-
Berker Peksag
-
Jason R. Coombs
-
Petri Lehtinen
-
Roundup Robot
-
Senthil Kumaran
-
Terry J. Reedy