[Python-Dev] Keeping an eye on Travis CI, AppVeyor and buildbots: revert on regression

[Victor Stinner]

2018-06-07 4:45 GMT+02:00 Mariatta Wijaya <mariatta.wijaya at gmail.com>:
> Are there APIs we can use to check the status of builbots?

Buildbots offer different ways to send notifications: emails and IRC
bot for example.

If you want to *poll* for recent builds, I don't know. I would suggest
to use notifications (push) rather than polling.

> Maybe we can have the our bots check for the buildbot status in backport
> PRs.

Right now, I'm not confortable with this idea because there is a risk
to scare newcomers with false alarms and real bugs which are not
coming from their changes, but may be known or are new but still
unrelated to their changes.

Moreover, even when a buildbot fails because of a real regression, a
build may include multiple changes (I saw builds with more than 25
changes on some buildbots). Buildbot is different than a CI: slow
buildbots are able to pack test multiple new changes at once. So
again, there is a high risk of false alarms.

Maybe I'm too conservative and we can try something with good
documentation and proper warnings to explain properly such hypotetical
notifications on pull requests.

See also my other email which explains this differently:
https://mail.python.org/pipermail/python-dev/2018-May/153759.html

Victor