[Python-Dev] How to watch buildbots?

[Nick Coghlan]

On 30 May 2018 at 22:30, Ivan Pozdeev via Python-Dev <python-dev at python.org>
wrote:

> What's the big idea of separate buildbots anyway? I thought the purpose of
> CI is to test everything _before_
> it breaks the main codebase. Then it's the job of the contributor rather
> than maintainer to fix any breakages.
>

> So, maybe making them be driven by Github checks would be a better time
> investment.
> Especially since we've got VSTS checks just recently, so whoever was doing
> that still knows how to interface with this Github machinery.
>
> If the bots cancel a previous build if a new one for the same PR arrives,
> this will not lead to a significant load difference 'cuz the number of
> actively developed PRs is stable and roughly equal to the number of merges
> according to the open/closed tickets dynamics.
>

There are a few key details here:

1. We currently need to run post-merge CI anyway, as we're not doing
linearised commits (where core devs just approve a change without merging
it, and then a gating system like Zuul ensures that the tests are run
against the latest combination of the target branch and the PR before
merging the change)
2. Since the buildbots are running on donated dedicated machines (rather
than throwaway instances from a dynamic CI provider), we need to review the
code before we let it run on the contributed systems
3. The buildbot instances run *1* build at a time, which would lead to
major PR merging bottlenecks during sprints if we made them a gating
requirement
4. For the vast majority of PRs, the post-merge cross-platform testing is a
formality, since the code being modified is using lower level
cross-platform APIs elsewhere in the standard library, so if it works on
Windows, Linux, and Mac OS X, it will work everywhere Python runs
5. We generally don't *want* to burden new contributors with the task of
dealing with the less common (or harder to target) platforms outside the
big 3 - when they do break, it often takes a non-trivial amount of platform
knowledge to understand what's different about the platform in question

Cheers,
Nick.

P.S. That said, if VSTS or Travis were to offer FreeBSD as an option for
pre-merge CI, I'd suggest we enable it, at least in an advisory capacity -
it's a better check against Linux-specific assumptions creeping into the
code base than Mac OS X, since the latter is regularly different enough
from other *nix systems that we need to give it dedicated code paths.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180530/e5680deb/attachment.html>