[core-workflow] Starting the improved workflow discussion again

[Nick Coghlan]

On 25 July 2015 at 14:47, Brett Cannon <bcannon at gmail.com> wrote:
> Basically Kalithea becomes a common frontend to bitbucket and github where
> we generate patches from PRs on either site and work with them on our end
> for reviews, applying them to our repo of record, etc.

Yep, that's my hoped for outcome - that way, folks can develop and
submit changes using their preferred tools (whether that's git or hg,
GitHub or BitBucket), while we only have to deal with a single set of
tooling on the backend. With the data models of git and hg being
isomorphic to each other, it's technically feasible to provide that
choice to contributors, and I think it's worth aiming to do so rather
than forcing them to adapt to one or the other before we'll accept
their contributions.

I currently expect triagers and core developers would need to go back
to the original services if we wanted to actually discuss the change
with the submitter (at least in the near term), but I think there's
actually potential value in offering that kind of split conversation -
while the forge discussion would still be public if contributors
wanted to go look at it (and even participate), the internal
discussion between triagers and core developers would be separated
from the third party contributor facing discussion. That's a standard
feature of service-oriented ticketing systems such that the requestor
doesn't get spammed about updates relating to internal implementation
details that aren't relevant to them.

Services like http://gerrithub.io/ show that this kind of external
review service integration is already possible with GitHub, and the
Gerrit plugin for that is open source:
https://gerrit.googlesource.com/plugins/github/+/master/README.md

We'd also hopefully be able to sync this up with bugs.python.org in a
way that made it possible to check incoming change requests against
the CLA records held there. That would likely require the ability to
voluntarily link bugs.python.org accounts with accounts on the
services where we decided to accept pull requests.

> How would we handle
> changes that require custom fixes in two branches? They just fix it in both
> and we automatically handle the merge/revert steps?

I don't think it's a coincidence that there's a correlation between
"project uses a pull request based workflow" and "project doesn't
provide maintenance releases for past feature releases" :)

Beyond CPython, the main DVCS based workflows I'm familiar with that
need to provide maintenance releases are Beaker and OpenStack, and
those both rely on Gerrit. Quibbles with the busy nature of the Gerrit
web UI aside, I think the underlying workflow is well designed for the
task (at least the way we had it configured for Beaker), and it should
be adaptable to Mercurial as well (especially with appropriate use of
changeset evolution to manage the "still in review" commit stacks).
While it's sheer vaporware, I speculated on how that model could
potentially be adapted to Kallithea earlier this year:
http://lists.sfconservancy.org/pipermail/kallithea-general/2015q1/000231.html

Once "accepted change proposals" are their own distinct entity that
exists within the target repo (rather than being produced as a live
diff against a separate clone), it becomes more feasible to offer the
option to cherry-pick/graft them to other branches, rather than
merging the heads. That way, the multiple branch workflow *always*
involves independent commits, but in the trivial cases, the forward
ports are a push-button exercise (with each branch getting an
independent CI run prior to being merged). The current model of
"commit locally, merge forward locally, push to remote" goes away in
favour duplicating change proposals on the server so they can be
readily applied to additional branches.

You do pick up a new risk where the forward porting step is missed,
but it's possible to deal with that at the issue tracker level by
checking if the code commits match the affected branches. My
experience with Beaker is that this risk is worth it, as you actually
get a big gain from the fact that "needs tweaks to work on the newer
branch" and "doesn't need to be forward ported at all" are better
accounted for in the default workflow - the "common" case isn't common
enough that the exceptional cases can be so readily ignored.

That said, it would be good to have server side automation for the
default case (forward porting an unmodified change), and that could
potentially be done if we finally picked one of the conflict-free NEWS
file automation ideas and pushed it through to completion.

This would technically be the topic of a "Migrate CPython to
forge.python.org" follow-up PEP rather than the current PEP 474
proposal, and I'm not sure when I'm going to have a chance to write
that - PyCon Australia is next weekend, and then the weekend after I'm
off to the US for a couple of weeks to attend Fedora's Flock
conference and PyGotham.

I can at least get the PEP 474 update done before I leave for the US,
though, and I can include some of these ideas in there as "possible
future work".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia