On 15 June 2017 at 23:38, Victor Stinner email@example.com wrote:
2017-06-15 5:31 GMT+02:00 Nick Coghlan firstname.lastname@example.org:
For example, something that would be genuinely helpful would be a bot monitoring PR comments that could automate the "custom-build" dance for core developers (i.e. we'd be able to write something like "BuildBot: test custom build", and it would go away, kick off a custom BuildBot run by pushing the PR to the "custom-build" branch, and then report back the links for failed builds like http://buildbot.python.org/all/builders/x86%20Tiger%20custom/builds/5 )
I don't think that buildbots have enough resources to run tests on each PR. We get too many PR and each PR is updated regulary. On some buildbots, a single build can take 1 hour (or longer)...
I wasn't proposing running every PR through BuildBot, I was suggesting having a more straightforward way of triggering custom builds directly from the PR we want to test after post-merge CI has indicated it's necessary.
However, I think there's a way of handling that which doesn't require any new automation code to be written. Instead, we can have the policy be to post a link to https://docs.python.org/devguide/buildbots.html#custom-builders on the PR that is being reverted.
That way, even if folks didn't notice the problem with the buildbots themselves:
buildbot fleet prior to merging it again
Hopefully reversions will continue to be rare (since relatively few changes are likely to be as platform dependent as PEP 538, and Windows/*nix differences are already covered in pre-merge CI), but when they do come up, the reminder of how to manually trigger pre-merge cross-platform CI is likely to be useful, and doesn't require much additional effort on the part of the folks doing the reversion.