Does Buildbot retain a constant TCP socket to its server? I'm seeing this: http://buildbot.python.org/all/builders/AMD64%20Debian%20root%203.3/builds/0 """ Results: Retry exception slave lost """ I have two internet connections; one is faster, but tends to drop socket connections after a few hours, and the other is a bit slower but more reliable (as long as the connection doesn't go idle for too long - neither of my upstreams is particularly MUD-friendly). Would it help to set a routing rule to send traffic to python.org:9020 via the connection that won't drop out? ChrisA
On Tue, 31 Dec 2013 11:24:26 +1100 Chris Angelico <rosuav@gmail.com> wrote:
Does Buildbot retain a constant TCP socket to its server? I'm seeing this:
http://buildbot.python.org/all/builders/AMD64%20Debian%20root%203.3/builds/0
I'm quite sure it does. It's necessary to get a real-time view of the test log.
Would it help to set a routing rule to send traffic to python.org:9020 via the connection that won't drop out?
The only way to know is to try it out :) Regards Antoine.
On Tue, Dec 31, 2013 at 11:42 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Tue, 31 Dec 2013 11:24:26 +1100 Chris Angelico <rosuav@gmail.com> wrote:
Does Buildbot retain a constant TCP socket to its server? I'm seeing this:
http://buildbot.python.org/all/builders/AMD64%20Debian%20root%203.3/builds/0
I'm quite sure it does. It's necessary to get a real-time view of the test log.
Would it help to set a routing rule to send traffic to python.org:9020 via the connection that won't drop out?
The only way to know is to try it out :)
If that's what it does, then it almost certainly will help. I put all my MUD connections through the other connection. The change is made, I'll trigger another build and see if it works. Thanks for the help! ChrisA
Am 31.12.13 01:24, schrieb Chris Angelico:
Does Buildbot retain a constant TCP socket to its server?
In short: yes. A little bit longer: It uses the Twisted PerspectiveBroker protocol. That has nearly-transparent reconnects (but as your case shows, not fully transparent), and does regular ping messages to keep the connection alive. So it should be able to handle a failover from one link to the other, but it's certainly better to bind it to the more reliable transport. I believe you can somehow configure the frequency of ping messages so that you network doesn't believe the connection goes idle, plus it will attempt a reconnect if the network did indeed cancel the connection. If you wanted to study this further, you could look into the slave's twisted log file. Regards, Martin
On Thu, Jan 2, 2014 at 9:17 AM, "Martin v. Löwis" <martin@v.loewis.de> wrote:
So it should be able to handle a failover from one link to the other, but it's certainly better to bind it to the more reliable transport. I believe you can somehow configure the frequency of ping messages so that you network doesn't believe the connection goes idle, plus it will attempt a reconnect if the network did indeed cancel the connection.
If you wanted to study this further, you could look into the slave's twisted log file.
I should have mentioned that I'd already peeked into the log and that's why I posted. There was something in there about it sending application-level keepalives (every 600 seconds, IIRC), which looked good; but it wasn't sufficient, as seen. After I switched it to the other connection, I didn't see any more problems (and it's completed a few builds, though 2.7 seems to be having trouble in the tests - not peculiar to this bot though). ChrisA
participants (3)
-
"Martin v. Löwis" -
Antoine Pitrou -
Chris Angelico