Python3: Is this a bug in urllib?

Wed Oct 20 07:41:54 EDT 2010

Johannes Bauer wrote:

> Hi,
> 
> I've experienced the following behavior with Python3 of which I do not
> know if it's a bug or not. On two Python3.1 implementations, Python's
> urllib hangs when encountering a HTTP 301 (Redirect).
> 
> The code to reproduce is a one-liner (actually, two-liner), Python from
> Ubuntu tree:
> 
> Python 3.1.2 (r312:79147, Apr 15 2010, 15:35:48)
> [GCC 4.4.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from urllib import request;
>>>> request.URLopener().open("http://google.de")
> 
> Also occurs on another Python version (Gentoo):
> 
> Python 3.1.2 (release31-maint, Jun  9 2010, 23:58:21)
> [GCC 4.3.4] on linux2
> 
> The exchanged HTTP is:
> 
> GET http://google.de HTTP/1.1
> Accept-Encoding: identity
> Host: google.de
> User-Agent: Python-urllib/3.1
> 
> HTTP/1.1 301 Moved Permanently
> Via: 1.1 IMMPWISA01
> Connection: Keep-Alive
> Proxy-Connection: Keep-Alive
> Content-Length: 218
> Expires: Thu, 18 Nov 2010 15:18:40 GMT
> Date: Tue, 19 Oct 2010 15:18:40 GMT
> Location: http://www.google.de/
> Content-Type: text/html; charset=UTF-8
> Server: gws
> Cache-Control: public, max-age=2592000
> X-XSS-Protection: 1; mode=block
> 
> <HTML><HEAD><meta http-equiv="content-type"
> content="text/html;charset=utf-8">
> <TITLE>301 Moved</TITLE></HEAD><BODY>
> <H1>301 Moved</H1>
> The document has moved
> <A HREF="http://www.google.de/">here</A>.
> </BODY></HTML>
> 
> Although the content might indicate looping forever, it just hangs with
> no web traffic whatsoever (the TCP connection stays open, however).
> 
> When interrupting with Ctrl-C, this is the calltrace:
> 
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib/python3.1/urllib/request.py", line 1454, in open
>     return getattr(self, name)(url)
>   File "/usr/lib/python3.1/urllib/request.py", line 1628, in open_http
>     return self._open_generic_http(http.client.HTTPConnection, url, data)
>   File "/usr/lib/python3.1/urllib/request.py", line 1624, in
> _open_generic_http
>     response.status, response.reason, response.msg, data)
>   File "/usr/lib/python3.1/urllib/request.py", line 1644, in http_error
>     return self.http_error_default(url, fp, errcode, errmsg, headers)
>   File "/usr/lib/python3.1/urllib/request.py", line 1648, in
> http_error_default
>     void = fp.read()
>   File "/usr/lib/python3.1/socket.py", line 214, in readinto
>     return self._sock.recv_into(b)
> KeyboardInterrupt
> 
> Can anyone tell me if this is a bug or expected behavior?

While I'm not 100 percent sure it looks like a bug to me and I think you 
should report it at http://bugs.python.org

Peter