urllib2 timeout issue
Peter Otten
__peter__ at web.de
Wed Oct 16 07:22:34 EDT 2013
Jérôme wrote:
> Hi all.
>
> I'm having troubles with urllib2 timeout.
>
> See the following script :
>
> ----------------------------
> import urllib2
> result = urllib2.urlopen("http://dumdgdfgdgmyurl.com/")
> print result.readline()
> ----------------------------
>
> If run on my Debian Wheezy computer, or on my Debian Squeeze server,
> the answer is instantaneous :
>
> [...]
> urllib2.URLError: <urlopen error [Errno -2] Name or service not known>
>
> When run on my Raspberry Pi with Raspian Wheezy, the answer is
> identical but it takes 10 seconds.
>
> I tried
>
> result = urllib2.urlopen("http://dumdgdfgdgmyurl.com/", timeout=5)
>
> but I get the same results : instantaneous on Debian, 10 secondes on
> RPi.
>
> I also added this, as suggested on some StackOverflow pages :
>
> import socket
> socket.setdefaulttimeout(5)
>
> and it didn't make any difference.
>
> In both cases, Python version is "Python 2.7.3".
>
> Am I missing something ?
The problem might be ipv6-related. I'm currently working around a similar
annoying delay with the monkey-patch demonstrated below:
$ cat patch_getaddrinfo.py
import sys
import time
import urllib2
def monkeypatch_getaddrinfo():
import socket
gai = socket.getaddrinfo
def getaddrinfo(*args):
return gai(args[0], args[1], 2, *args[3:])
socket.getaddrinfo = getaddrinfo
if "-4" in sys.argv:
monkeypatch_getaddrinfo()
start = time.time()
try:
urllib2.urlopen("http://example.com/")
finally:
print time.time() - start
$ python -S patch_getaddrinfo.py
20.320786953
$ python -S patch_getaddrinfo.py -4
0.305027008057
As you may infer from the above I don't know what is happening exactly and
how to fix it properly...
More information about the Python-list
mailing list