Mailman 3 - Python-Buildbots

buildbots
by Kesha DeVita 02 Jun '21

02 Jun '21

hi, i wasnt sure if you were still looking for people that have os x installed to run a buildbot. But I do and would love to help in any way I can. Thanks Kesha

3 2

Adding a new x86 ubuntu / freebsd / clang / gcc buildbot
by Senthil Kumaran 14 May '21

14 May '21

Hello, I have been monitoring https://buildbot.python.org/all/#/builders for few weeks, and I have a desire to be involved with few aspects. 1. Help with triaging. 2. Running Buildbots. 3. Improving the state after identifying the areas that need improvement. For running buildbot, as a test, I setup a ubuntu docker on my machine and followed the instructions https://devguide.python.org/buildworker/#buildworker I am at the state, wherein I will need an identifier and password. > you will need to obtain an identifier and password for your worker so it can join the fleet. For the moment, this will be experimental, but I will settle upon either Ubuntu or FreeBSD, Clang / GCC and run the worker. - FreeBSD CLang might be a good addition. - Linux + GCC is something that my default system and I am comfortable triaging. Either ways, is it possible for me to obtain an identifier and password? How do we go about it? Thank you, Senthil

1 0

AIX buildbot reconfiguration request
by David Edelsohn 14 May '21

14 May '21

Michael Felt and I have created a new VM at OSUOSL as a dedicated Python buildbot for AIX. I plan to switch the buildbot from the GCC Compile Farm AIX system to the dedicated system. I would like to add an XLC builder to the worker. Can someone help with the configuration file changes for the buildbot master or should I try to make the changes myself and create pull requests on github? Thanks, David

3 3

buildmaster worker naming
by Jeremy Kloth 29 Mar '21

29 Mar '21

I'm looking to add another Windows 10 worker to the "buildbot fleet". Since Win10 updates semi-regularly, is it an issue if the worker name (BuilderConfig.name) gets changed over the lifetime of the worker? Currently it would be "Windows10 20H2", but once the next update is released, the name should be updated to "Windows10 21H1". Mostly I'm looking to differentiate the would be two Win10 workers, but there is a bit of practicality to this as well. Each update (or version, as it were) does bring changes and features to the base OS, which can be noticed by test_locale failing (https://bugs.python.org/issue37945) on recent versions, but not on the current Windows 10 worker (at version 1803). Related to that, if renaming a worker is OK over its lifetime, then would it be permissible to suffix "1803" to the current Win10 worker name? I'll submit the PR for the new worker once there is clarification. Thanks for your time. -- Jeremy Kloth

5 6

SLES s390x worker update to SLES 15
by David Edelsohn 04 Mar '21

04 Mar '21

FYI, I am in the process of updating the SLES s390x worker to SLES 15. Thanks, David

1 0

Build-bots and IPv6 only servers, redo AIX bots for more isolation (cleanliness) of build environments
by Michael Felt 19 Nov '20

19 Nov '20

Hi all, I do not recall ever reading about instructions for having a buildbot on an IPv6 only server. As I have not done ' a lot' with IPv6 only hosts - I was wondering if there is also an IPv6 address (or is an IPv6 lookup of the same host sufficient). I would like to organize the AIX bots - better - one designed and hosted in an AIX+GNU environment as well as one "designed and hosted" in a GNU-free environment. Currently the bots run in a mixed environment - so it is hard to know, for sure, which (shared) libraries are being used. The intent is to have unique LPAR/WPAR environments to keep the build environments sterile - and use IPv6 for further isolation. Question: has CPython ever been tested in an IPv6-only environment (i.e., will IPv4 tests break automatically when the only IPv4 (if even then) is 127.0.0.1. Regards, Michael

3 2

Buildbot for SmartOS
by Gaige B Paulsen 06 Nov '20

06 Nov '20

Hi, ClueTrust would like to offer up buildbot worker(s) for python running on SmartOS (Illumos-derived). I believe we’ve got the configuration working, but we’ll need a worker name/password in order to confirm operation and set things up further (at least according to the wiki, if that isn’t the case, please point me to more up-to-date instructions). Thanks, -Gaige

2 1

ACTION NEEDED: upgrade your buildbot client configuration
by Victor Stinner 02 Sep '20

02 Sep '20

Hi Buildbot owners, You need to manually update your buildbot owner clients configuration: * Change the buildbot server host to: buildbot-api.python.org (it was buildbot.python.org previously) * Change the keepalive to: 60 (seconds, it was usually 600 previously) You should also clean up manually the temporary directory (/tmp) to remove old "ccXXX" and "tmpXXX" files: rm -f /tmp/{cc,tmp}* Stop the client, remove these files, and then start again the client. For example, on some Red Hat workers, /tmp contained up to 20 GB of these "temporary" files! These files are leaked by buildbot builds interrupted by closed TCP connections. Note: we might pass the "-pipe" option to GCC to avoid leaking temporary "ccXXX" files when GCC is killed by an interrupted job. But I'm not sure how to do that. -- So, what happened? https://bugs.python.org/issue41642 One week ago, Ernest and Pablo migrated the buildbot server to a new machine with a larger disk. The old machine only had 8 GB disk and its disk was frequently full because of the large (JUnit) XML files. The new machine is now behind a load balancer which is great to have clean x.509 certificates on HTTPS (https://buildbot.python.org/). The TCP port 9020 used by buildbot clients also goes through the load balancer for consistency with the whole PSF infrastructure. The problem is that the load balancer seems to like closing idle TCP connections open for longer than 60 seconds. Ernest extended the timeout to 24 hours, but it seems like the load balancers still closes TCP connections of buildbot clients (workers). Ernest exposed the TCP port 9020 directly to the Internet under a new host name: buildbot-api.python.org. This host is not behind the load balancer, and so is not affected by the closed TCP connections issue. Please update your buildbot client configuration to use the new "buildbot-api.python.org" host. I also suggest to use a keepalive of 60 seconds, it should prevent further TCP connection issue in the future (if one day, the host goes behind a load balancer again). Victor -- Night gathers, and now my watch begins. It shall not end until my death.

1 0

Cleaning up leftover directories that fill up the disk space of the buildbots
by Charalampos Stratakis 26 Aug '20

26 Aug '20

Hello all, As many of you have noticed beforehand, buildbot doesn't clean-up by default the old directories on a worker, which are not required anymore by the buildmaster. Since version 2.7.0 there will be a notice at twistd.log within the buildarea, about the directories that can be removed but there is also an option to always remove them which can be added to buildbot.tac of the worker config [0]. Unfortunately it's not turned on by default and it has to be configured on each worker individually but it can save some hassle in the long run. So within the buildbot.tac residing at the buildarea just change the value of "delete_leftover_dirs" to 1 and restart the service. That should do the trick and you can verify the change by checking to see that the directories mentioned beforehand at twistd.log do not exist anymore. Note that if you created the worker config before version 2.7.0, the value will not be there, so just add the "delete_leftover_dirs=1" and add it to the constructor of the worker. example snippet of that buildbot.tac section: maxretries = None use_tls = 0 delete_leftover_dirs = 1 s = Worker(buildmaster_host, port, workername, passwd, basedir, keepalive, umask=umask, maxdelay=maxdelay, numcpus=numcpus, allow_shutdown=allow_shutdown, maxRetries=maxretries, useTls=use_tls, delete_leftover_dirs=delete_leftover_dirs) [0] https://github.com/buildbot/buildbot/commit/7a3943551f64cd04315d17ec89411a8… -- Regards, Charalampos Stratakis Software Engineer Python Maintenance Team, Red Hat

1 0

Python and "xlc" based builds running again.
by Michael Felt 01 Aug '20

01 Aug '20

Hi All, I was very happy, back in March, when I discovered that the test failure that was occurring when building with xlc (and I could not find a cause for) was 'magically' resolved - and likewise, frustrated when just as magically builds suddenly went into a 'segmentation fault' during builds. This was the reason I had switched off my bot - no need for consistent 'red' flags, if a cause could not be found. Happily, again 'magically', whatever was causing this disappeared - and a second error (re: unicopde and test_io) got fixed in master - and master runs 'green' on master. The 3.9 bot (both xlc and gcc) still run red - waiting for the backport of the test_io issue 40401 (see request from Serley to Victor): https://bugs.python.org/issue41401#msg374409 Would be very nice to have all running 'green' again. The 'open' issue with xlc (v13 and v16 in any case) and _decimal - there is someone looking at that (i.e., I know who to ask for more help) can be worked on now - now the seg fault is gone. Michael p.s. - I hope you all agree this is not a suitable message for python-dev. imho - it is more about what is holding up a green status with a bot than an issue in Python (core).

1 0