I've checked disk; we still have a couple hundred MB to play with, so I don't think it's related.
On Tue, Jul 7, 2020 at 11:31 AM Victor Stinner vstinner@python.org wrote:
There is an on-going problem of low disk space which might be related.
The buildbot server only has a disk of 8 GB which is full most of the time. Sadly, we failed to get a new machine with a larger disk.
I would suggest to stop storing large XML files which fill the disk.
Victor
Le mar. 7 juil. 2020 à 18:02, Matt Billenstein matt@vazor.com a écrit :
Ok, looked at the logs and couldn't see anything obvious going on - tried rebooting the machine and now I can't connect to it - I'm on a trip and won't have physical access until mid-August, so it may be unavailable until then.
m
On Tue, Jul 07, 2020 at 12:45:49PM +0200, Victor Stinner wrote:
Hi Matt,
Yes, there are humans on the other side of the list :-)
I'm not aware of any action done on the buildbot server recently.
All builds of the "x86-64 macOS 3.x" worker failed with "retry lost connection" for 3 days: https://buildbot.python.org/all/#/builders/275
I suggest you to investigate on the worker side, since other workers look fine.
On the server, I found these logs:
2020-07-07 01:43:19+0000 [Broker,110,67.169.78.30] Got exception in RepeatedActionHandler Traceback (most recent call last): File "/data/buildbot/venv/lib/python3.8/site-packages/twisted/internet/defer.py", line 1475, in gotResult _inlineCallbacks(r, g, status) File "/data/buildbot/venv/lib/python3.8/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks result = result.throwExceptionIntoGenerator(g) File "/data/buildbot/venv/lib/python3.8/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/data/buildbot/venv/lib/python3.8/site-packages/buildbot/util/deferwaiter.py", line 96, in _do_action log.err(e, 'Got exception in RepeatedActionHandler') --- <exception caught here> --- File "/data/buildbot/venv/lib/python3.8/site-packages/buildbot/util/deferwaiter.py", line 94, in _do_action yield self._action() twisted.spread.pb.PBConnectionLost: [Failure instance: Traceback (failure with no frames):
: Connection to the other side was lost in a non-clean fashion. ] 2020-07-07 01:43:19+0000 [Broker,110,67.169.78.30]
.lostRemote 2020-07-07 01:43:19+0000 [Broker,110,67.169.78.30] Worker.detached(billenstein-macos) 2020-07-07 01:43:19+0000 [Broker,110,67.169.78.30] releaseLocks( ): [] 2020-07-07 01:43:19+0000 [Broker,110,67.169.78.30] .lostRemote 2020-07-07 01:43:19+0000 [Broker,110,67.169.78.30] .lostRemote 2020-07-07 01:43:19+0000 [Broker,110,67.169.78.30] <Build x86-64 macOS custom number:31 results:success>.lostRemote 2020-07-07 01:43:19+0000 [Broker,110,67.169.78.30] stopping currentStep Test(command=['make', 'buildbottest', 'TESTOPTS=-j2 --junit-xml test-results.xml ${BUILDBOT_TESTOPTS}', 'TESTPYTHONOPTS=', 'TESTTIMEOUT=900'], timeout=1200, usePTY=False, env={}) 2020-07-07 01:43:19+0000 [Broker,110,67.169.78.30] RemoteCommand.interrupt [Failure instance: Traceback (failure with no frames):
Victor
Le mar. 7 juil. 2020 � 06:05, Matt Billenstein via Buildbot-status buildbot-status@python.org a �crit :
Anyone monitoring this email address?
The worker is running - not sure where these disconnects are coming from - is something going on with the master?
m
On Tue, Jul 07, 2020 at 02:43:19AM +0000, buildbot@python.org wrote:
The Buildbot working for 'Python' has noticed that the worker named billenstein-macos went away.
It last disconnected at Tue Jul 7 01:43:19 2020.
The admin on record (as reported by WORKER:info/admin) was Matt Billenstein matt@vazor.com .
Sincerely, -The Buildbot
Buildbot-status mailing list -- buildbot-status@python.org To unsubscribe send an email to buildbot-status-leave@python.org https://mail.python.org/mailman3/lists/buildbot-status.python.org/ Member address: vstinner@python.org
-- Night gathers, and now my watch begins. It shall not end until my death.
-- Night gathers, and now my watch begins. It shall not end until my death.