[python-committers] Wrongly stopping merges discourages merging.

[Brett Cannon]

On Sun, 3 Jun 2018 at 13:23 Terry Reedy <tjreedy at udel.edu> wrote:

> When we used hg, core dev committers could actually commit to the
> repository when they judged it appropriate.  When we moved to github,
> Brett, with whoever's approval,


Since this seems very much directed at me, I should mention any authority I
have has either come from Guido and/or the PEP process. If people are
unhappy with my work then please feel free to bring it up and we can
discuss having my privileges taken away.


> decided that we should no longer be
> trusted to make commits without approval of a couple of mindless robots.
>   However, the premise that the robots should be trusted is false.  So I
> again request that there be a manual override for when the robots are
> obviously giving false failures.
>

Please realize that every time we have switched off CI, we have ended up
with a broken branch, so it's a trade-off between these occasional hiccups
or occasionally broken branches (and as Victor has pointed out, we are not
always good as a group about making sure we notice when stuff breaks). Also
note that because we now have branches that are almost always stable we
have users who actually run from a checkout directly instead of waiting for
a release (which also benefits us by helping to surface bugs earlier than
e.g. an RC).


>
> Exhibit 1. For at least a couple of weeksin may, faults in the asyncio
> test (and another) caused the asyncio test to randomly fail about half
> the time.  With one retest, each CI bot failed about 1/4 the time.  At
> least one bot of the two bots failed about 1/2 the time.  The AppVeyor
> queue ballooned.
>
> One could decrease the frustration and time to success (but only partly)
>   by only re-starting the bot that failed.  Doing so for Travis is
> fairly easy.  Doing so with AppVeyor is obscure and error prone.
>
> I twice requested that the randomly failing tests be disabled.  Victor
> said he wanted to keep monitoring what they did.  I think he overly
> discounted the pain and frustration of having good merges blocked.  I
> think either 1) bad tests should be disabled, or 2. the CI code should
> be able to ignore failures by bad tests, or 3. responsible core devs
> should be able to.
>
> Exhibit 2. AppVeyor is badly broken.
>
> This morning Cheryl Sabella submitted a nice patch fixing an annoying
> behavior of IDLE's editor/shell/output windows.  The CI tests passed,
> the patch worked great, it only needed expansion of the placeholder
> blurb.  I was really excited.
>
> With some trepidation, I made the edit.  Unfortunately, both CI bots
> rerun the code tests even when the code is unchanged.  Blurb edits
> should be treated as doc-only changes, with only the blurb rechecked.
>
> My trepidation turned out to be well-founded.  My excitement is gone.
> After an error, AppVeyor just quit without reporting any failure cause.
> https://ci.appveyor.com/project/python/cpython/build/3.8build16869
>
> Since then, there have been 2 successes and 7 similar unexplained failures.
> https://ci.appveyor.com/project/python/cpython/history
>
> https://ci.appveyor.com/project/python/cpython/build/3.6build16871/job/c2q80njh9clnfgjt
> https://ci.appveyor.com/project/python/cpython/build/3.8build16872
> https://ci.appveyor.com/project/python/cpython/build/3.7build16873
> https://ci.appveyor.com/project/python/cpython/build/3.8build16874
>
> https://ci.appveyor.com/project/python/cpython/build/3.6build16876/job/t9nyt59wkwcn68nk
> https://ci.appveyor.com/project/python/cpython/build/3.8build16877
> https://ci.appveyor.com/project/python/cpython/build/3.8build16878
>
> The last is my restart.  The time wasted by this broken blockbot is time
> not spent doing something useful.  I would really like to have this
> patch in the .rc in a week -- along with a few others that should follow.
>
> Guido once asked what is off-putting about being a core developer.  This
> is one thing.
>

So both examples seem very focused on AppVeyor and the first one somewhat
at CI overall. As stated in another email, I have turned off AppVeyor being
required so 1.5 of these issues have been dealt with (and based on a PR I
looked it the requirement retroactively went away for all open PRs).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-committers/attachments/20180604/a0061ccf/attachment-0001.html>