Problem with svn on community buildbot

I have a community buildbot: http://www.python.org/dev/buildbot/community/all/g5%20OSX%202.5/builds/14/st... which is failing the svn checkout of the 2.5 branch: svn: PROPFIND request failed on '/projects/python/branches/release25-maint' svn: PROPFIND of '/projects/python/branches/release25-maint': Could not resolve hostname `svn.python.org': Temporary failure in name resolution (http://svn.python.org) The svn command is: /opt/local/bin/svn checkout --revision 67742 --non-interactive http://svn.python.org/projects/python/branches/release25-maint build Any idea what the problem might be? Thanks, Skip

svn: PROPFIND of '/projects/python/branches/release25-maint': Could not resolve hostname `svn.python.org': Temporary failure in name resolution (http://svn.python.org)
Any idea what the problem might be?
Well - can you resolve `svn.python.org' on that machine (e.g. when using ping(1))? Regards, Martin

Martin> Well - can you resolve `svn.python.org' on that machine Martin> (e.g. when using ping(1))? Yup: $ host svn.python.org svn.python.org has address 82.94.164.164 svn.python.org has IPv6 address 2001:888:2000:d::a4 $ ping svn.python.org PING svn.python.org (82.94.164.164): 56 data bytes 64 bytes from 82.94.164.164: icmp_seq=0 ttl=50 time=134.041 ms 64 bytes from 82.94.164.164: icmp_seq=1 ttl=50 time=135.441 ms 64 bytes from 82.94.164.164: icmp_seq=2 ttl=50 time=135.352 ms ^C --- svn.python.org ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max/stddev = 134.041/134.945/135.441/0.640 ms $ telnet svn.python.org 80 Trying 82.94.164.164... Connected to svn.python.org. Escape character is '^]'. ^] telnet> quit Connection closed. Skip

I don't know is this is related, but from my end, access to svn.python.org has been extremely slow recently: $ time curl -o /dev/null http://svn.python.org % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 353 100 353 0 0 4 0 0:01:28 0:01:15 0:00:13 0 real 1m15.045s user 0m0.004s sys 0m0.004s I've seen similar slowdowns accessing bugs.python.org, but not now. It looks like it has something to do with IPv6: $ host svn.python.org svn.python.org has address 82.94.164.164 svn.python.org has IPv6 address 2001:888:2000:d::a4 $ time curl -v -o /dev/null http://svn.python.org * About to connect() to svn.python.org port 80 (#0) * Trying 2001:888:2000:d::a4... Operation timed out * Trying 82.94.164.164... connected ... No slowdown when IPv6 lookup is disabled with -4 option to curl: $ time curl -4 -o /dev/null http://svn.python.org % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 353 100 353 0 0 774 0 --:--:-- --:--:-- --:--:-- 0 real 0m0.463s user 0m0.004s sys 0m0.004s On Sun, Dec 14, 2008 at 8:58 AM, <skip@pobox.com> wrote:
Martin> Well - can you resolve `svn.python.org' on that machine Martin> (e.g. when using ping(1))?
Yup:
$ host svn.python.org svn.python.org has address 82.94.164.164 svn.python.org has IPv6 address 2001:888:2000:d::a4 $ ping svn.python.org PING svn.python.org (82.94.164.164): 56 data bytes 64 bytes from 82.94.164.164: icmp_seq=0 ttl=50 time=134.041 ms 64 bytes from 82.94.164.164: icmp_seq=1 ttl=50 time=135.441 ms 64 bytes from 82.94.164.164: icmp_seq=2 ttl=50 time=135.352 ms ^C --- svn.python.org ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max/stddev = 134.041/134.945/135.441/0.640 ms $ telnet svn.python.org 80 Trying 82.94.164.164... Connected to svn.python.org. Escape character is '^]'. ^] telnet> quit Connection closed.
Skip _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/alexander.belopolsky%40gma...

Alexander> It looks like it has something to do with IPv6: Alexander> $ host svn.python.org svn.python.org has address Alexander> 82.94.164.164 svn.python.org has IPv6 address Alexander> 2001:888:2000:d::a4 ... Alexander> No slowdown when IPv6 lookup is disabled with -4 option to Alexander> curl: ... But I have no problem on my laptop which is sitting right next to the G5 which is having problems. Both show an IPv6 address for svn.python.org. Skip

I don't know is this is related
It shouldn't. AFAIK, buildbot makes its internet connections through twisted, and twisted doesn't use IPv6. Also, the diagnostics (cannot resolve name) doesn't match connectivity problems.
$ time curl -v -o /dev/null http://svn.python.org * About to connect() to svn.python.org port 80 (#0) * Trying 2001:888:2000:d::a4... Operation timed out
Hmm. Can you debug this further? Do you have IPv6 connectivity at all? Do you have a global v6 address? What happens if you do a v6 traceroute? Regards, Martin

Please see below for more svn debugging, but now I also traced down the delays I observe when I go to bugs.python.com to the same issue. The offending download is the style sheet and that explains why curl does not show it when pointed to the main page: $ curl -v -o /dev/null http://python.org/styles/screen-switcher-default.css * About to connect() to python.org port 80 (#0) * Trying 2001:888:2000:d::a2... Operation timed out The offending main page element is: $ curl http://bugs.python.org 2>/dev/null | grep screen-switcher-default <link media="screen" href="http://python.org/styles/screen-switcher-default.css" type="text/css" id="screen-switcher-stylesheet" rel="stylesheet" /> On Sun, Dec 14, 2008 at 2:55 PM, "Martin v. Löwis" <martin@v.loewis.de> wrote: ..
$ time curl -v -o /dev/null http://svn.python.org * About to connect() to svn.python.org port 80 (#0) * Trying 2001:888:2000:d::a4... Operation timed out
Hmm. Can you debug this further?
Do you have IPv6 connectivity at all? I don't think so.
Do you have a global v6 address? No, only private inet6 address:
$ ifconfig en0 en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::21f:5bff:fef3:c0a4%en0 prefixlen 64 scopeid 0x4 inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255 ...
What happens if you do a v6 traceroute?
$ traceroute6 -v svn.python.org traceroute6 to svn.python.org (2001:888:2000:d::a4) from fdbd:a375:403a:51c6:21f:5bff:fef3:c0a4, 30 hops max, 12 byte packets 1 * 24 bytes from fe80::216:cbff:fec1:c94c%en0 to fe80::21f:5bff:fef3:c0a4: icmp type 136 (Neighbor Advertisement) code 0 0000: fe800000 00000000 0216cbff fec1c94c 0010: 00000000 00000000 32 bytes from fe80::216:cbff:fec1:c94c%en0 to fe80::21f:5bff:fef3:c0a4: icmp type 135 (Neighbor Solicitation) code 0 0000: fe800000 00000000 021f5bff fef3c0a4 0010: 01010016 cbc1c94c 00000000 00000000 * *

I've found a work-around in Firefox: go to about:config page an change network.dns.disableIPv6 to true. Does anyone know a similar setting in Safari? On Sun, Dec 14, 2008 at 4:52 PM, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
Please see below for more svn debugging, but now I also traced down the delays I observe when I go to bugs.python.com to the same issue. The offending download is the style sheet and that explains why curl does not show it when pointed to the main page:
$ curl -v -o /dev/null http://python.org/styles/screen-switcher-default.css * About to connect() to python.org port 80 (#0) * Trying 2001:888:2000:d::a2... Operation timed out
The offending main page element is: $ curl http://bugs.python.org 2>/dev/null | grep screen-switcher-default <link media="screen" href="http://python.org/styles/screen-switcher-default.css" type="text/css" id="screen-switcher-stylesheet" rel="stylesheet" />
On Sun, Dec 14, 2008 at 2:55 PM, "Martin v. Löwis" <martin@v.loewis.de> wrote: ..
$ time curl -v -o /dev/null http://svn.python.org * About to connect() to svn.python.org port 80 (#0) * Trying 2001:888:2000:d::a4... Operation timed out
Hmm. Can you debug this further?
Do you have IPv6 connectivity at all? I don't think so.
Do you have a global v6 address? No, only private inet6 address:
$ ifconfig en0 en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::21f:5bff:fef3:c0a4%en0 prefixlen 64 scopeid 0x4 inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255 ...
What happens if you do a v6 traceroute?
$ traceroute6 -v svn.python.org traceroute6 to svn.python.org (2001:888:2000:d::a4) from fdbd:a375:403a:51c6:21f:5bff:fef3:c0a4, 30 hops max, 12 byte packets 1 * 24 bytes from fe80::216:cbff:fec1:c94c%en0 to fe80::21f:5bff:fef3:c0a4: icmp type 136 (Neighbor Advertisement) code 0 0000: fe800000 00000000 0216cbff fec1c94c 0010: 00000000 00000000
32 bytes from fe80::216:cbff:fec1:c94c%en0 to fe80::21f:5bff:fef3:c0a4: icmp type 135 (Neighbor Solicitation) code 0 0000: fe800000 00000000 021f5bff fef3c0a4 0010: 01010016 cbc1c94c 00000000 00000000 * *

I've found a work-around in Firefox: go to about:config page an change network.dns.disableIPv6 to true.
I'd advise against using such a work-around. The infrastructure is designed to cope with that case transparently; if it is not transparent, your system must be somehow misconfigured (it could also be the case that applications are buggy - but I don't think this is the case you are facing). The proper solution is to fix your system (although I'm still uncertain what precisely the problem might be). Regards, Martin

On Sun, Dec 14, 2008 at 5:18 PM, "Martin v. Löwis" <martin@v.loewis.de> wrote:
I've found a work-around in Firefox: go to about:config page an change network.dns.disableIPv6 to true.
I'd advise against using such a work-around. The infrastructure is designed to cope with that case transparently; if it is not transparent, your system must be somehow misconfigured ...
I've never had similar issues with any site other than those in python.org domain and I had these problems with bug.python.org on several systems in different locations. Another work-around, which happens to work for all browsers and svn is to disable IPv6 in network preferences (my system is Mac OS 10.5.5). since I don't have IPv6 connectivity, I think this is a solution I can live with, but I wonder why is it necessary for python.org to be registered as both an IPv4 and v6 domain? Google does not do that: $ host google.com google.com has address 72.14.205.100 google.com has address 74.125.45.100 google.com has address 209.85.171.100 google.com mail is handled by 10 smtp4.google.com. google.com mail is handled by 10 smtp1.google.com. google.com mail is handled by 10 smtp2.google.com. google.com mail is handled by 10 smtp3.google.com. $ host ipv6.google.com ipv6.google.com is an alias for ipv6.l.google.com. ipv6.l.google.com has IPv6 address 2001:4860:0:2001::68

live with, but I wonder why is it necessary for python.org to be registered as both an IPv4 and v6 domain? Google does not do that:
Google works in changing that: http://www3.ietf.org/proceedings/08jul/slides/plenaryw-4.pdf Other systems have been doing it for many years now: martin@mira:~$ host www.freebsd.org www.freebsd.org has address 69.147.83.33 www.freebsd.org has IPv6 address 2001:4f8:fff6::21 Regards, Martin
participants (3)
-
"Martin v. Löwis"
-
Alexander Belopolsky
-
skip@pobox.com