[core-workflow] Starting the improved workflow discussion again

[Maciej Szulik]

On Tue, Jul 21, 2015 at 4:15 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 21 July 2015 at 05:49, Brett Cannon <bcannon at gmail.com> wrote:
> > In my ideal workflow scenario, these are the steps a patch would take:
> >
> > Issue is created
> > Issue is triaged to have right affected versions, etc.
> > Patch is uploaded
> > CI kicks the patch off for all branches and OSs that are affected
> > CI flags what branches and OSs did (not) pass or apply cleanly to
> > If necessary, another patch that works in a specific branch that is
> affected
> > is uploaded (obviously requires some way to flag that a patch applies to
> a
> > specific branch, deciding how to deal with Misc/NEWS, etc.)
> > Code review -- with a tool other than Rietveld -- from a core developer
> with
> > feedback
> > New version of patch uploaded, usual CI kicked off
> > If everything looks good and CI is green, get patch approval from a core
> dev
> > Approval submits the patch(es) to the appropriate branches
> > CI triggered yet again, and if tests fail then patch(es) are
> automatically
> > rolled back
> >
> > Now I realize this is not about to launch immediately. There are changes
> to
> > Roundup in there, a reliable test suite that actually fails only on
> failures
> > and not because it's flaky, etc. But the key point here is that
> everything
> > that can be automated is, and code reviews can occur entirely through a
> > browser.
>
> I think you're conflating some different issues here, at least two of
> which are worth separating out from each other:
>
> 1. Completely online workflow for documentation editing
> 2. Pre-commit CI for CPython
>
> Both of the current forge.python.org proposals are aimed primarily at
> the first problem, since they start out with purely documentation
> repos like the developer guide and the PEPs. Hopefully we can also
> eventually separate out "version independent" repos for the how to
> guides and the tutorial. In addition to a completely online process
> for documentation editing, review, and approval, the other key benefit
> to these repos would be that *access management* would also be online,
> rather than requiring shell access to hg.python.org.
>
> Documentation projects are a good starting point for this side of
> things, as they're inherently lower risk. The worst thing
> documentation can do is give readers bad advice, it can't force them
> to follow it.
>
> This means that for forge.python.org, I think "What about CPython?"
> should be something we take into account as a "What's next?" for the
> service, but our near term focus should be on making things like the
> developer guide and PEPs trivial to suggest edits to, trivial to
> approve edits to, and trivial to grant approval rights over. Those
> levels of access (who can suggest edits, who can approve edits, who
> can approve edit rights for others) should also all be completely
> transparent and changes in them should be tracked automatically rather
> than requiring manual updates to a text file.
>
> Pre-commit CI for CPython is a different story - as David points out,
> it is *not* OK to run code on the Buildbot fleet that hasn't been
> approved by a core developer. Folks are trusting *us* to run code on
> their systems, not random developers posting patches to
> bugs.python.org. Noah (quite sensibly) isn't interested in getting the
> PSF Infrastructure team involved in running random code from the
> internet either.
>
> That's where the system Kushal set up in collaboration with the CentOS
> folks potentially comes in:
> https://mail.python.org/pipermail/python-dev/2015-May/140050.html
>
> That's just a simple "smoke test" to say "Does the proposed change
> pass on x86_64 systems running CentOS 7?". If we could combine it with
> a similar system for running Windows smoke tests in Appveyor, I think
> we'd flush out a lot of basic cross-platform compatibility issues
> pre-commit, regardless of whether folks are working locally on a *nix
> system or a Windows one. (We wouldn't catch *everything*, because
> Linux is not FreeBSD is not Mac OS X, etc, but we'd catch a lot of
> them).
>
> At the moment, running those requires that we be logged into IRC, be
> approved to trigger test runs, and indicate which issue we'd like to
> test.
>
> If we instead had a "test" link next to patch files in Roundup, then a
> core developer, completely online, could:
>
> 1. Read over the patch to see if we think its reasonable to smoke test
> 2. Trigger the smoke test directly from Roundup
> 3. Receive the results back as Roundup comments, with links to the run logs
>
>
We (openshift) have similar technique developed around vagrant + jenkins
where
we can kick of by commenting on a github PR to either run the tests or
perform
actual merge. Both of these operation run exactly the same suite of tests,
but only the merge performs actual, well merging. Obviously only certain
group
of core devs has the rights to tag the PRs. Additionally when such
tag/comment already exists all the new changes (eg. additional changes
after review)
are tested/merged and this is something that should be considered in this
case
as well (there are some slight nuances, but I don't want to go too much
into details).
Does the tag applies always or only once, and if once how you know
it was applied already? So having a tag on/off future would be desirable
imho.
I'd be happy to help triaging this problem as much as my time allows me ;)

Maciej


> As we gained further familiarity and confidence with the system, we
> could extend the trust for running pre-commit test runs to anyone that
> has been granted Developer privileges on the issue tracker, rather
> than restricting it specifically to core developers. (BTW, we should
> probably come up with an icon for folks with elevated tracker
> privileges - at the moment they're just marked as having signed the
> CLA if they aren't also CPython core developers)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> core-workflow mailing list
> core-workflow at python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20150721/026bddbc/attachment.html>