[Python-Dev] Looking after the buildbots (in general)

[Stephen J. Turnbull]

Steve Holden writes:

 > But I see rules being established ("there's a language moratorium: no
 > changes!", "no release should be made unless the buildbots are *all*
 > green") and then ignored apparently on a whim. This doesn't give people
 > any confidence that the rules actually mean much, and I think ignoring
 > the latter rule can negatively affect quality.

I don't see this.  In the first case, you've misstated the rule: it's
"no changes to the Python language", and what is and is not part of
the language is subject to a certain amount of interpretation.  There
are several PEPs waiting on the moratorium despite everybody loving
them, and the decisions on borderline changes (which have gone both
ways, mostly denials) are establishing precedents that narrow the
scope for "interpretation".  I think it's very reasonable to assess
the moratorium as *very successful* with respect to it being a rule
that is obeyed in spirit and according to the letter.

In the second case, I don't recall it being stated as a project rule.
The buildbots were considered untrustworthy by many from the get-go,
and I do recall discussion of the "community buildbots" which
effectively resulted in community 'bots being fully deprecated.

Some RMs have nevertheless chosen to take them very seriously and want
them fixed if they're broken, others consider them a useful indicator
but are willing to proceed if there are strong indications that the
'bot is broken rather than CPython.  That's something that can be left
up to the release manager or not, as the project chooses, but my
impression to date has been that this is a matter of RM policy, not
project policy.

Note that following the latter rule can also negatively affect
quality, if scarce developer effort is devoted to fixing somebody
else's software rather than working on Python.

FWIW my assessment is that for the moment all of the RMs take the
buildbots pretty seriously, which is good (that seems to be consensus
opinion), with some variations in intensity, which is also (IMO YMMV)
good.  No need for change here yet (IMO YMMV), although community
members (anybody who cares) should prod RMs who seem to be neglecting
buildbot results.  In another cycle or so, the bots will probably be
ready for a project-wide rule.

I agree with J-P's suggestion that the place to start is asking
developers to bookmark the bot pages relevant to them, and visit it
(with appropriate lag) after committing.  For one thing, if people see
the 'bots deprecating their perfectly good changes, they'll have some
incentive to work on the bots and beat them into shape.  That can help
take some load off the people who have concentrated on the bots.  It
will also mean that a decision to condition releases on green bots
will be taken based on much broader experience rather than hearsay
about their reliability.