2017-06-15 5:31 GMT+02:00 Nick Coghlan <ncoghlan@gmail.com>:
I'm not necessarily opposed to such a policy change, but if folks really want guaranteed green post-merge buildbots for all platforms (rather than just guaranteed green for Linux & Windows, sometimes red for everything else), then I think a better place to focus effort would be in making it easier for us to test things across the full BuildBot fleet in pre-merge CI.
For example, something that would be genuinely helpful would be a bot monitoring PR comments that could automate the "custom-build" dance for core developers (i.e. we'd be able to write something like "BuildBot: test custom build", and it would go away, kick off a custom BuildBot run by pushing the PR to the "custom-build" branch, and then report back the links for failed builds like http://buildbot.python.org/all/builders/x86%20Tiger%20custom/builds/5 )
I don't think that buildbots have enough resources to run tests on each PR. We get too many PR and each PR is updated regulary. On some buildbots, a single build can take 1 hour (or longer)... Moreover, that's an idea, but we need "someone" to do the job. I don't know anyone available to do it.
I'm proposing a pragmatic solution to a concrete issue. I'm just to do by best with our limited resources (human and CPU time). Again, failures on random platforms not catched by our pre-commit CI occur, but are -hopefully- rare!
I dislike having a long list of "nice to have" list for our infra. I prefer to focus on the existing bricks and don't put too much pressure on people who give their time to care of our tooling and infra ;-)
(Yes, I would also prefer to have a fleet of buildbots at the pre-commit stage if I can get it for free :-))
Victor