[Python-Dev] funky buildbot problems again...

[exarkun at twistedmatrix.com]

On 08:31 pm, janssen at parc.com wrote:
>My Intel Snow Leopard 2 build slave has gone into outer-space again.
>
>When I look at it, I see buildslave taking up most of a CPU (80%), and
>nothing much else going on.  The twistd log says:
>
>[... much omitted ...]
>2011-04-04 08:35:47-0700 [-] sending app-level keepalive
>2011-04-04 08:45:47-0700 [-] sending app-level keepalive
>2011-04-04 08:55:47-0700 [-] sending app-level keepalive
>2011-04-04 09:03:15-0700 [Broker,client] lost remote
>2011-04-04 09:03:15-0700 [Broker,client] lost remote
>2011-04-04 09:03:15-0700 [Broker,client] lost remote
>2011-04-04 09:03:15-0700 [Broker,client] lost remote
>2011-04-04 09:03:15-0700 [Broker,client] lost remote
>2011-04-04 09:03:15-0700 [Broker,client] Lost connection to 
>dinsdale.python.org:9020
>2011-04-04 09:03:15-0700 [Broker,client] 
><twisted.internet.tcp.Connector instance at 0x101629ab8> will retry in 
>3 seconds
>2011-04-04 09:03:15-0700 [Broker,client] Stopping factory 
><buildslave.bot.BotFactory instance at 0x1016299e0>
>2011-04-04 09:03:18-0700 [-] Starting factory 
><buildslave.bot.BotFactory instance at 0x1016299e0>
>2011-04-04 09:03:18-0700 [-] Connecting to dinsdale.python.org:9020
>2011-04-04 09:03:18-0700 [Uninitialized] Connection to 
>dinsdale.python.org:9020 failed: Connection Refused
>2011-04-04 09:03:18-0700 [Uninitialized] 
><twisted.internet.tcp.Connector instance at 0x101629ab8> will retry in 
>8 seconds
>2011-04-04 09:03:18-0700 [Uninitialized] Stopping factory 
><buildslave.bot.BotFactory instance at 0x1016299e0>
>2011-04-04 09:03:27-0700 [-] Starting factory 
><buildslave.bot.BotFactory instance at 0x1016299e0>
>2011-04-04 09:03:27-0700 [-] Connecting to dinsdale.python.org:9020
>
>So it's been spinning its wheels for 3 days.

Does this mean that the "2011-04-04 09:03:27-0700 [-] Connecting to 
dinsdale.python.org:9020" message in the logs is the last one you see 
until you restart the slave?

Or does it mean that the logs go on and on for three days with these 
"Connecting to dinsdale...." / "Connection Refused" / "... will retry in 
N seconds" cycles, thousands and thousands of times?

What does the buildmaster's info page for this slave say when the slave 
is in this state?  In particular, what does it say about 
"connects/hour"?

Jean-Paul