** Note: This message contains images and is better visualized in discourse: https://discuss.python.org/t/now-you-can-test-a-pr-with-the-buildbots-before... **
## Testing PRs with buildbots :robot:
After some work (and updating the buildbot server to Python3.8), I am happy to announce that after some work, is now possible to test a Pull Request with the buildbot fleet on demand by just adding a simple label (:hammer: test-with-buildbots):
When you add this label to an existing Pull Request:
A build will be immediately triggered in a selected subset of the stable buildbots. Every buildbot will report to the PR the build status (like Travis, Azure Pipelines...etc).
As long as the label is in the Pull Request *every new commit to the pr will be tested with the buildbots* in the same way that Travis or Azure Pipelines test every commit that is pushed to the PR.
Clicking the "Details" link in the GitHub status will bring you to the builder page if you want to read logs or see the build live.
The checks are *not required* to merge the PR. This means that if you decide so, you can proceed and merge the PR even if a buildbot is failing (for example, if the failure is unrelated to the PR).
In the buildbot page you can
distinguish all the builders that are building PRs by a new Pull Request
label.
## When to use the new label :man_student:
Using the buildbots before merging will be especially useful in the following situations:
You are reviewing a big C PR and you want to make sure that there are no refleaks by using the refleak-detection builders. These builders take more time but will tell you if the PR has some sneaky reference leaks. :medal_sports:
You want to make sure that the PR builds on all platforms. We have a big
collection of platforms (various versions of Windows, macOS, Linux distros) and configurations (debug, non-debug, PGO/LTO, LTO, clang sanitizers...) and if you want to make sure that a PR that you suspect may behave differently by platform (like adding new POSIX functions) will not break later, is a good practice to test with the buildbots first.
- Because you don't want us to suffer afterwards if your PR fails on some obscure platform ;)
Notes
Each individual builder has a limited amount of PRs that can be tested simultaneously (depends on the builder) so it may take a while to start building your PR. So please, be patient! You can check the status of all builders in the buildbot page :mantelpiece_clock:
Remember that the buildbots are donated machines, so remember to be a good citizen and double-check the code of the PR before adding the label.
This new addition should allow you to gain confidence when merging a PR with minimum impact in your existing workflow and effort. If you have any feedback or feature request, go ahead and tell us :)
Kind regards from rainy London, Pablo Galindo Salgado