On 15 June 2017 at 23:38, Victor Stinner <victor.stinner@gmail.com> wrote:
2017-06-15 5:31 GMT+02:00 Nick Coghlan <ncoghlan@gmail.com>:
For example, something that would be genuinely helpful would be a bot monitoring PR comments that could automate the "custom-build" dance for core developers (i.e. we'd be able to write something like "BuildBot: test custom build", and it would go away, kick off a custom BuildBot run by pushing the PR to the "custom-build" branch, and then report back the links for failed builds like http://buildbot.python.org/all/builders/x86%20Tiger%20custom/builds/5 )
I don't think that buildbots have enough resources to run tests on each PR. We get too many PR and each PR is updated regulary. On some buildbots, a single build can take 1 hour (or longer)...
I wasn't proposing running every PR through BuildBot, I was suggesting having a more straightforward way of triggering custom builds directly from the PR we want to test after post-merge CI has indicated it's necessary.
However, I think there's a way of handling that which doesn't require any new automation code to be written. Instead, we can have the policy be to post a link to https://docs.python.org/devguide/buildbots.html#custom-builders on the PR that is being reverted.
That way, even if folks didn't notice the problem with the buildbots themselves:
- They get a ping on a PR that they're necessarily following
- They get a reminder of how to test the revised PR across the buildbot fleet prior to merging it again
Hopefully reversions will continue to be rare (since relatively few changes are likely to be as platform dependent as PEP 538, and Windows/*nix differences are already covered in pre-merge CI), but when they do come up, the reminder of how to manually trigger pre-merge cross-platform CI is likely to be useful, and doesn't require much additional effort on the part of the folks doing the reversion.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia