On Sat, Jan 2, 2016 at 8:45 PM, Brett Cannon email@example.com wrote:
I was purposefully trying to avoid having this discussion start until Monday, but since Ezio sort has a deadline for issue-related stuff we can start on that side now (I still have an email planned on Monday to outline the initial steps to moving things over and the very first hurdle to work through).
No hurry, there's still a week before the sprint so take your time. I also still have to merge all the GSoC improvements before looking at these other issues.
On Fri, 1 Jan 2016 at 17:57 Ezio Melotti firstname.lastname@example.org wrote:
On Fri, Jan 1, 2016 at 9:21 PM, Brett Cannon email@example.com wrote:
... We can start the discussion of how we want to handle the transition next week, but I'm going to try and step away from this whole workflow topic until Monday so I can spend the last couple of days of my vacation not thinking about this stuff. :)
This will likely require a new PEP, that should cover:
- the new workflow (including how to handle reviews -- see below);
- the steps required for the migration and a timeline;
- a list of things that will break and/or that will need to be
added/replaced before/during/after the migration; 4) the fate of hg.python.org;
Yes to all of that.
Some of these things might already be covered by existing PEPs, but I don't see them in PEP 507 and 481 (and I'm getting a bit lost among all the competing PEPs and multiple threads across at least a couple different MLs :).
It will be either be a new PEP or the GitHub PEP will simply be rewritten.
Above you said that GitHub will be used for reviews but we will keep using our bug tracker.
Yes, we are not moving to GitHub's issue tracker.
This leads to the following questions:
- Do GitHub reviews only work for pull requests?
What do you mean by this? Do you mean by patch upload?
With reviews here I mean inline comments similar to the ones you can add on Rietveld. I haven't used GitHub yet, but I assume this can be done for pull requests (same as with BitBucket) and perhaps for patches attached to the GitHub bug tracker. Since we are not going to use the GitHub bug tracker, only pull requests can be reviewed on GitHub. If someone uploads a patch on the bug tracker it won't be possible to review it unless we convert it to a pull request (or unless we keep Rietveld around, but we don't want that).
- Are we still going to support uploading diffs/patches to the
tracker (short term and long term)?
Well, "support" as in "allow". We won't be keeping Rietveld around (part of this move is so we can get off of Rietveld).
Only supporting pull requests will force all contributors to use GitHub (unless GitHub supports "external" pull requests). I think we should keep supporting plain patches as an alternative, and write tools to convert those to pull requests. If GitHub doesn't support "external" pull requests, we might also be able to write additional tools to do so from the tracker directly.
- If so, how/where are we going to review those diffs/patches?
I don't know. Is it possible to have the bot create a PR for an uploaded patch?
It should be possible -- we will need an account for b.p.o and then b.p.o can commit the patches on its clone (perhaps using a different branch for issue/patch) and send a pull request to the main clone (for the last bit we might need to use the GitHub API, unless there are other ways).
- Will patches be automatically converted to pull requests, or pull
requests converted to patches and attached to b.p.o issues?
I don't think we need to upload a patch file if we already have a PR,
The goal is to have everything in the same place, even if contributions are made both via pull requests and via patches uploaded to b.p.o. I think the best option is to convert all patches uploaded to b.p.o to pull requests, so that all the reviews happen on GitHub.
but we should either have a PR -> issue or issue -> PR mapping somehow. This can either be via bot command or specifying a PR # in the issue tracker.
b.p.o should automatically link to PRs when "PR num" appears in a message (this can be done easily). GitHub should also link to b.p.o issues when an issue number appears in the PR (I don't know if/how this can be done). For patches uploaded to b.p.o, a link to the generated PR can be added where the "review" link is now.
I also have no issue if people want to make PRs generate patches for the issue as well for backup purposes, but that might get a bit noisy for those following both an issue and a PR.
I'm not sure we need to generate a diff and attach it to the issue, but PRs related to the issue should probably be listed on b.p.o where the list of patches currently is. One possible reason to add patches is the new patch analysis tool written during GSoC that can extract metadata from patches (affected files, whether the patch has tests/docs, etc.) which in turn can be used during searches. The tool can also be adapted to generate the diff dynamically and extract metadata from there though.
- Are there plans to migrate existing Rietveld reviews to GitHub and
shut Rietveld down for good?
Yes, there are plans to shut Rietveld down. I suspect we will leave it up until GitHub is running in an equivalent fashion to our current workflow,
Good to know.
then we will have a deadline to work through any outstanding reviews and then close it up (and if we want to be thorough, set Rietveld so that it won't work on new issues and only pre-existing ones that already have a patch).
Or we could just copy-paste the reviews as messages on the tracker so that languishing reviews don't hold up the Rietveld demise.
- If not, will Rietveld stay around and be read-only? Or will it
still be used for patches uploaded to b.p.o?
We are trying to cut down on the infrastructure we maintain, so I don't want Rietveld sticking around indefinitely.
- What else is needed to integrate GitHub and b.p.o?
Basically a way to map an issue to a PR (or vice-versa). Probably the simplest solution is to allow pasting in a GitHub PR URL or the PR # to make the association. The other option is for the bot to accept a command to make the association. No reason we can't have both, though. But either way we should have a way to connect PRs to issues.
I'll have to think a bit about the best way to implement this and the UI (I already have some ideas).
If a bot can create PRs from a patch, then that would be nice for folks who don't want to use GitHub, but I don't consider that a priority. I'm happy to try to be accommodating to people who don't want to use GitHub for whatever reason, there is a limit to how much energy we should put into supporting that scenario past uploading a patch.
If we want to support this (and I think we should) we will either need a bot (actually a Roundup detector) or a human to do it, and the former sounds better.
About points 3 of the initial list, these are some examples of things that will need to be added/updated/replaced:
- the hgroundup hook that post messages to b.p.o when a commit
includes an issue number needs to be replaced;
- the hgirker hook used for deadparrot on #python-dev needs to be
replaced (probably there is already an irker-git hook that can be used/adapted);
- the hgbuildbot hook that triggers the buildbots on commit needs
to be replaced;
- other hg hooks might need to be rewritten/replaced;
- the script that generates bug tracker links to hg.p.o needs to
be updated (for both cs ids and paths);
- the hg code that converts issue links in commit messages to b.p.o
links needs to be replaced;
- other places where issue numbers appears on GitHub should generate
links to b.p.o;
- the hg-cpydev hg extension  should be ported to git (optional);
- the buildbots need to be updated if they are going to pull the
source from github and use git;
- the bug tracker will need to be updated to interact with github;
- the devguide needs to be updated (both to cover the new workflow
and update links/commands);
Yes to all of these (I think; depends if we change how something operates in terms of associations).
FWIW next weekend (9-10 January) we are organizing a sprint in Helsinki, and I'm planning to work on the bug tracker, so I might be able to start addressing some of these points.
I think deciding how to associate issues with PRs would be a great thing to work through. Otherwise we need to decide if we want to go with an issue tracker solution to the NEWS file or if we want a individual file solution (which happens to be bot-friendly). Those two things are probably the most critical starting points.
There's also another point that I haven't seen being discussed: how we will handle the author and the committer fields. Currently we only know the committer so, while migrating to git, this can either be duplicated in the author field or just preserved in the committer field leaving the author field empty. Afterwards we can start using the two fields for what they are meant for. A possible problem is how to update them when commits/merges are performed by bots (e.g. if b.p.o automatically converts a patch into a pull request, or if a core dev merges a patch by clicking on a button). Perhaps someone more familiar with Git/GitHub can clarify this (I think it's possible to specify author/committer manually, but I don't know how GitHub handles it on its side).
Best Regards, Ezio Melotti
Best Regards, Ezio Melotti
P.S. enjoy your last few days of vacation while you can ;)