Python3: Is this a bug in urllib?

Tue Oct 19 12:47:10 EDT 2010

Hi,

I've experienced the following behavior with Python3 of which I do not
know if it's a bug or not. On two Python3.1 implementations, Python's
urllib hangs when encountering a HTTP 301 (Redirect).

The code to reproduce is a one-liner (actually, two-liner), Python from
Ubuntu tree:

Python 3.1.2 (r312:79147, Apr 15 2010, 15:35:48)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib import request; request.URLopener().open("http://google.de")

Also occurs on another Python version (Gentoo):

Python 3.1.2 (release31-maint, Jun  9 2010, 23:58:21)
[GCC 4.3.4] on linux2

The exchanged HTTP is:

GET http://google.de HTTP/1.1
Accept-Encoding: identity
Host: google.de
User-Agent: Python-urllib/3.1

HTTP/1.1 301 Moved Permanently
Via: 1.1 IMMPWISA01
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Content-Length: 218
Expires: Thu, 18 Nov 2010 15:18:40 GMT
Date: Tue, 19 Oct 2010 15:18:40 GMT
Location: http://www.google.de/
Content-Type: text/html; charset=UTF-8
Server: gws
Cache-Control: public, max-age=2592000
X-XSS-Protection: 1; mode=block

<HTML><HEAD><meta http-equiv="content-type"
content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.de/">here</A>.
</BODY></HTML>

Although the content might indicate looping forever, it just hangs with
no web traffic whatsoever (the TCP connection stays open, however).

When interrupting with Ctrl-C, this is the calltrace:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.1/urllib/request.py", line 1454, in open
    return getattr(self, name)(url)
  File "/usr/lib/python3.1/urllib/request.py", line 1628, in open_http
    return self._open_generic_http(http.client.HTTPConnection, url, data)
  File "/usr/lib/python3.1/urllib/request.py", line 1624, in
_open_generic_http
    response.status, response.reason, response.msg, data)
  File "/usr/lib/python3.1/urllib/request.py", line 1644, in http_error
    return self.http_error_default(url, fp, errcode, errmsg, headers)
  File "/usr/lib/python3.1/urllib/request.py", line 1648, in
http_error_default
    void = fp.read()
  File "/usr/lib/python3.1/socket.py", line 214, in readinto
    return self._sock.recv_into(b)
KeyboardInterrupt

Can anyone tell me if this is a bug or expected behavior?

Regards,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa <hidbv3$om2$1 at speranza.aioe.org>