[core-workflow] Starting the improved workflow discussion again

[Brett Cannon]

On Mon, Jul 20, 2015 at 7:15 PM Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 21 July 2015 at 05:49, Brett Cannon <bcannon at gmail.com> wrote:
> > In my ideal workflow scenario, these are the steps a patch would take:
> >
> > Issue is created
> > Issue is triaged to have right affected versions, etc.
> > Patch is uploaded
> > CI kicks the patch off for all branches and OSs that are affected
> > CI flags what branches and OSs did (not) pass or apply cleanly to
> > If necessary, another patch that works in a specific branch that is
> affected
> > is uploaded (obviously requires some way to flag that a patch applies to
> a
> > specific branch, deciding how to deal with Misc/NEWS, etc.)
> > Code review -- with a tool other than Rietveld -- from a core developer
> with
> > feedback
> > New version of patch uploaded, usual CI kicked off
> > If everything looks good and CI is green, get patch approval from a core
> dev
> > Approval submits the patch(es) to the appropriate branches
> > CI triggered yet again, and if tests fail then patch(es) are
> automatically
> > rolled back
> >
> > Now I realize this is not about to launch immediately. There are changes
> to
> > Roundup in there, a reliable test suite that actually fails only on
> failures
> > and not because it's flaky, etc. But the key point here is that
> everything
> > that can be automated is, and code reviews can occur entirely through a
> > browser.
>
> I think you're conflating some different issues here, at least two of
> which are worth separating out from each other:
>
> 1. Completely online workflow for documentation editing
> 2. Pre-commit CI for CPython
>

I wasn't conflating them so much as not worrying about #1 as I know that's
not a hard problem to solve like the CPython-specific workflow is.


>
> Both of the current forge.python.org proposals are aimed primarily at
> the first problem, since they start out with purely documentation
> repos like the developer guide and the PEPs. Hopefully we can also
> eventually separate out "version independent" repos for the how to
> guides and the tutorial. In addition to a completely online process
> for documentation editing, review, and approval, the other key benefit
> to these repos would be that *access management* would also be online,
> rather than requiring shell access to hg.python.org.
>
> Documentation projects are a good starting point for this side of
> things, as they're inherently lower risk. The worst thing
> documentation can do is give readers bad advice, it can't force them
> to follow it.
>
> This means that for forge.python.org, I think "What about CPython?"
> should be something we take into account as a "What's next?" for the
> service, but our near term focus should be on making things like the
> developer guide and PEPs trivial to suggest edits to, trivial to
> approve edits to, and trivial to grant approval rights over. Those
> levels of access (who can suggest edits, who can approve edits, who
> can approve edit rights for others) should also all be completely
> transparent and changes in them should be tracked automatically rather
> than requiring manual updates to a text file.
>

OK, then let's choose the devguide or the PEPs to test Kalithea out on and
see how it goes since no one has experience with the service while I bet
everyone has experience with at least GitHub. If you can get whomever has
done the most amount of work on the devguide lately to sign off on it --
probably Carol Willing -- then I say get a test instance of forge.python.org
up for the devguide and let's see what working with Kalithea is like (and
it also gets us more new contributor feedback than the PEPs would while
also not frustrating Guido when he deals with peps@ =).


>
> Pre-commit CI for CPython is a different story - as David points out,
> it is *not* OK to run code on the Buildbot fleet that hasn't been
> approved by a core developer. Folks are trusting *us* to run code on
> their systems, not random developers posting patches to
> bugs.python.org. Noah (quite sensibly) isn't interested in getting the
> PSF Infrastructure team involved in running random code from the
> internet either.
>
> That's where the system Kushal set up in collaboration with the CentOS
> folks potentially comes in:
> https://mail.python.org/pipermail/python-dev/2015-May/140050.html
>
> That's just a simple "smoke test" to say "Does the proposed change
> pass on x86_64 systems running CentOS 7?". If we could combine it with
> a similar system for running Windows smoke tests in Appveyor, I think
> we'd flush out a lot of basic cross-platform compatibility issues
> pre-commit, regardless of whether folks are working locally on a *nix
> system or a Windows one. (We wouldn't catch *everything*, because
> Linux is not FreeBSD is not Mac OS X, etc, but we'd catch a lot of
> them).
>

Right. Basic coverage is better than no coverage for initial patch testing
(after core dev approval).


>
> At the moment, running those requires that we be logged into IRC, be
> approved to trigger test runs, and indicate which issue we'd like to
> test.
>
> If we instead had a "test" link next to patch files in Roundup, then a
> core developer, completely online, could:
>
> 1. Read over the patch to see if we think its reasonable to smoke test
> 2. Trigger the smoke test directly from Roundup
> 3. Receive the results back as Roundup comments, with links to the run logs
>

SGTM


>
> As we gained further familiarity and confidence with the system, we
> could extend the trust for running pre-commit test runs to anyone that
> has been granted Developer privileges on the issue tracker, rather
> than restricting it specifically to core developers. (BTW, we should
> probably come up with an icon for folks with elevated tracker
> privileges - at the moment they're just marked as having signed the
> CLA if they aren't also CPython core developers)
>

Definitely wouldn't hurt, and it does raise their profiles on the issue
tracker.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20150721/b8fbb20b/attachment.html>