On 2015-12-02 00:32, Donald Stufft wrote:

...

On Dec 1, 2015, at 6:24 PM, Brett Cannon <brett@python.org> wrote:

It's Dec 1, which means it's time for any questions people have about the proposed workflows so we can get answers by Dec 15.

I have one question that applies to both proposals and one specific to GitLab. The general one is whether both Guido and me can both be happy. :) Guido doesn't want intermediate commits nor what he calls "merge turds" to show up in the history. I want to be able to do merges from the browser. Do either GitHub or GitLab provide a way through the web UI to give Guido what he wants, or will it always require having a checkout and SSH keys set up in order to do a PR merge? If only Guido can be made happy then that means either proposal becomes an easy way for people to get code hosting for their forks and a review tool but not a PR management platform since merges would occur outside the website and merges would simply be a `git push` which is basically what we do now to do the final merge for a patch.

As far as I am aware, when you merge with the browser in GitHub it essentially does ``git merge —no-ff`` which means there will *always* be a merge commit. There’s no support for a rebase workflow (where you rebase the branch ontop of master) or for squash merges or for FF merges.

Merge commits are the single most idiotic feature in GitHub because GitHub enforces non fast-forward merges. Merge commits bloat and clutter the revision history with useless junk, e.g. http://ariya.ofilabs.com/2013/09/fast-forward-git-merge.html . We either have to live with the fact that CPython's revision history is going to contain lots of superfluous checkins or we cannot use the green merge button at all. By the way it is not possible to disable or hide the merge button. This means that we have to teach all committers to resist the temptation and do a manual merge. GitHub claims that non-ff merges are superior because they add context information to merges. The same can be accomplished with mandatory links to tickets and Reviewed-by, Tested-by and Signed-off-by lines. I'm -1 on GitHub as long as GitHub doesn't support fast-forward merges. Christian

Berker also said a bot could do the ff workflow which could also handle the NEWS problem as well as long term branches. Assuming that's true and we choose to solve those problems that way then it will be true on either platform. On Sun, 13 Dec 2015, 05:39 Donald Stufft <donald@stufft.io> wrote:

...

On Dec 13, 2015, at 7:55 AM, Christian Heimes <christian@python.org> wrote:

On 2015-12-02 00:32, Donald Stufft wrote:

...

...

On Dec 1, 2015, at 6:24 PM, Brett Cannon <brett@python.org> wrote:

It's Dec 1, which means it's time for any questions people have about

...

...

I have one question that applies to both proposals and one specific to

GitLab. The general one is whether both Guido and me can both be happy. :) Guido doesn't want intermediate commits nor what he calls "merge turds" to show up in the history. I want to be able to do merges from the browser. Do either GitHub or GitLab provide a way through the web UI to give Guido what he wants, or will it always require having a checkout and SSH keys set up in order to do a PR merge? If only Guido can be made happy then that means either proposal becomes an easy way for people to get code hosting for

the proposed workflows so we can get answers by Dec 15. their forks and a review tool but not a PR management platform since merges would occur outside the website and merges would simply be a `git push` which is basically what we do now to do the final merge for a patch.

...

As far as I am aware, when you merge with the browser in GitHub it

essentially does ``git merge —no-ff`` which means there will *always* be a merge commit. There’s no support for a rebase workflow (where you rebase the branch ontop of master) or for squash merges or for FF merges.

Merge commits are the single most idiotic feature in GitHub because GitHub enforces non fast-forward merges. Merge commits bloat and clutter the revision history with useless junk, e.g. http://ariya.ofilabs.com/2013/09/fast-forward-git-merge.html . We either have to live with the fact that CPython's revision history is going to contain lots of superfluous checkins or we cannot use the green merge button at all. By the way it is not possible to disable or hide the merge button. This means that we have to teach all committers to resist the temptation and do a manual merge.

GitHub claims that non-ff merges are superior because they add context information to merges. The same can be accomplished with mandatory links to tickets and Reviewed-by, Tested-by and Signed-off-by lines.

I'm -1 on GitHub as long as GitHub doesn't support fast-forward merges.

Christian

I actually prefer non-ff merges, but that’s something that’s going to be very opinion driven (to ff or not to ff). Obviously going with GitHub is (at least currently) choosing to go with non-ff most of the time.

----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

For Github, Yes and No. You can have status checks which will either prevent the merge, or will color it yellow to indicate that it shouldn’t be merged (but will still allow it). Unfortunately right now the implementation of preventing the merge unless all status checks pass ALSO makes it so that you cannot have any commits on the branch you’re merging into that are not contained within the PR. The practical implications of this is that anytime you merge a PR or push commits to master (or another branch), all PRs to that branch need to be rebased or have master merged back into them. They do provide a “Update Branch” button to do this automatically if there are no conflicts, however you only get that button if you have write permission to the incoming repository (e.g. if I fork python/cpython to dstufft/python and make a PR, you’d need write permission to dstufft/cpython to get that button). This makes mandatory checks pretty useless for most OSS projects.

On Dec 13, 2015, at 5:53 PM, Brett Cannon <brett@python.org> wrote:

And couldn't we have a pre-merge web hook that always rejected merges through the browser if we chose to?

On Sun, 13 Dec 2015, 14:31 Brett Cannon <brett@python.org <mailto:brett@python.org>> wrote: Berker also said a bot could do the ff workflow which could also handle the NEWS problem as well as long term branches. Assuming that's true and we choose to solve those problems that way then it will be true on either platform.

On Sun, 13 Dec 2015, 05:39 Donald Stufft <donald@stufft.io <mailto:donald@stufft.io>> wrote:

...

On Dec 13, 2015, at 7:55 AM, Christian Heimes <christian@python.org <mailto:christian@python.org>> wrote:

On 2015-12-02 00:32, Donald Stufft wrote:

...

...

On Dec 1, 2015, at 6:24 PM, Brett Cannon <brett@python.org <mailto:brett@python.org>> wrote:

It's Dec 1, which means it's time for any questions people have about the proposed workflows so we can get answers by Dec 15.

I have one question that applies to both proposals and one specific to GitLab. The general one is whether both Guido and me can both be happy. :) Guido doesn't want intermediate commits nor what he calls "merge turds" to show up in the history. I want to be able to do merges from the browser. Do either GitHub or GitLab provide a way through the web UI to give Guido what he wants, or will it always require having a checkout and SSH keys set up in order to do a PR merge? If only Guido can be made happy then that means either proposal becomes an easy way for people to get code hosting for their forks and a review tool but not a PR management platform since merges would occur outside the website and merges would simply be a `git push` which is basically what we do now to do the final merge for a patch.

As far as I am aware, when you merge with the browser in GitHub it essentially does ``git merge —no-ff`` which means there will *always* be a merge commit. There’s no support for a rebase workflow (where you rebase the branch ontop of master) or for squash merges or for FF merges.

Merge commits are the single most idiotic feature in GitHub because GitHub enforces non fast-forward merges. Merge commits bloat and clutter the revision history with useless junk, e.g. http://ariya.ofilabs.com/2013/09/fast-forward-git-merge.html <http://ariya.ofilabs.com/2013/09/fast-forward-git-merge.html> . We either have to live with the fact that CPython's revision history is going to contain lots of superfluous checkins or we cannot use the green merge button at all. By the way it is not possible to disable or hide the merge button. This means that we have to teach all committers to resist the temptation and do a manual merge.

GitHub claims that non-ff merges are superior because they add context information to merges. The same can be accomplished with mandatory links to tickets and Reviewed-by, Tested-by and Signed-off-by lines.

I'm -1 on GitHub as long as GitHub doesn't support fast-forward merges.

Christian

I actually prefer non-ff merges, but that’s something that’s going to be very opinion driven (to ff or not to ff). Obviously going with GitHub is (at least currently) choosing to go with non-ff most of the time.

----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

On Dec 13, 2015, at 01:55 PM, Christian Heimes wrote:

Merge commits are the single most idiotic feature in GitHub because GitHub enforces non fast-forward merges. Merge commits bloat and clutter the revision history with useless junk, e.g. http://ariya.ofilabs.com/2013/09/fast-forward-git-merge.html . We either have to live with the fact that CPython's revision history is going to contain lots of superfluous checkins or we cannot use the green merge button at all. By the way it is not possible to disable or hide the merge button. This means that we have to teach all committers to resist the temptation and do a manual merge.

GitHub claims that non-ff merges are superior because they add context information to merges. The same can be accomplished with mandatory links to tickets and Reviewed-by, Tested-by and Signed-off-by lines.

Coming from Bazaar, I was used to non-ff merges. One thing commit merges allows, which works well with bzr's strong main-line-of-development view of the world[1] is that log and bisect feel more natural. Meaning that when you need to bisect to find a problem, or want to see mainline changes, you generally won't see the branch author's intermediate commits. But sometimes you *do* want to dive into those sideline commits and of course there are options that allow you to do so. But the more I use git, and once I understood what they do to the graph, the more I've come to like fast-forward merges, and on projects I host on GitLab, I like having the ability to allow ff-merges. Doing this through the web ui usually requires rebasing though, either automatically (and GL has an option to allow rebase-before-ff-merge) or manually by the original branch author. The linear history is pretty nice. There are a couple of trade-offs though. If the rebase or merge doesn't squash commits, then it's possible for a bisect to land inside an intermediate commit, and if those intermediate commits don't always pass the test suite then bisect will see test failures, which could make bisect more difficult to use. Also, in practice, I usually have to do *some* clean up of submitted branches anyway (e.g. whitespace or add NEWS), so the choices are; 1) locally branch, cleanup, test, rebase, ff-merge, push; 2) cleanup locally after automerging; 3) insist that the cleanups be done by the original author before the merge is approved. I've found #1 to be the least work for drive-by contributions, but for core-contributor branches in Python, #3 will probably be more common and then we might miss ff-merges more. So bottom line, I like the *option* of ff-merging and it can often be the cleanest way to move history forward. At least with GL, you still have the option of doing a merge commit when that's what you want. Cheers, -Barry [1] which I'd argue correlates strongly to most project's reality, and is one thing I miss in git. Of course, with merge commits, you have `git log --first-parent` although I'm not aware of a similar ability with bisect.

On Dec 14, 2015, at 09:16 PM, Nick Coghlan wrote:

GitLab actually have a useful page discussing some of the trade-offs of different git workflows: http://doc.gitlab.com/ee/workflow/gitlab_flow.html

Yep, this is a good page for the start of a discussion we're going to need to have. IMO, it doesn't focus enough on projects like Python where we don't deploy anything but instead release regular products like tarballs. Picking a code hosting strategy will be the "easy" part of Brett's work <wink>. Trying to figure out the best workflow for git-based CPython development will be the "fun" part, and the current Mercurial flow is probably not the best approach if and when git is adopted.

One of the points they make regarding "no commits to master, only merges" is that it means that regardless of whether you're working on a simple change that can be handled on a single commit, or a multi-commit feature branch, there will always be a single merge commit at the end, and that's what you revert if you decide to back out the change - you never need to ask the question "Are there multiple commits I need to revert in order to back that change out?".

Right, that's definitely an advantage of no-ff merge commits. It's less of a concern when you have to branch locally to clean up contributions because at the end of that process, you can rebase and squash on master, and then you've only got one commit to ff. This rebase is safe because it's never pushed. E.g. Alice submits a merge request that fixes bug #3. She doesn't include a NEWS item, and her branch has some minor style issues that need cleaning up, but it's generally pretty good. Rather than nitpick Alice's contribution, Bob fetches Alice's branch, does a few more commits locally to clean things up, and then rebases his local copy of Alice's branch onto master, squashing commits. Everything looks good to Bob so he ff-merges his local branch onto master, and pushes master. In this case, linear ff-merges are cleaner and just as easy to revert as a merge-commit. It also makes it easy to cherry pick this change into stable release branches. In fact, we may find that bug fixes typically work this way anyway because if they need to be backported, it can be easier to work out all the cherry picks and do local testing on all the stable branches, then only push master and the stable branches once everything's sorted out. CI and the buildbots would then run on the results and ensure everything still works. For this reason, I'm a fan of a master-first development for bug fixes, with cherry picking to backport fixes to stable branches as appropriate. I'm love to hear about alternatives, but after a lot of experimentation, this seems to be an approach that usually works smoothly and is the easiest to understand what happened after the fact, but it's different than our current Mercurial workflow. Feature branches will work differently because they won't be backported. In those cases, an auto-merge with or without merge commits via the web ui could be a cleaner workflow.

If we were to adopt the "commit to master and backport" model for maintenance branches, then filtering out merge commits to master should also eliminate most of the clutter from "git log", while still presenting the meaningful changes to each branch.

There's probably no easy way to eliminate manual, local work on the part of the final committer for backporting changes from master into stable branches, but with a careful strategy, it's not that hard either. Sometimes there's no way around merge conflicts though. Cheers, -Barry

On Dec 14, 2015, at 11:31 AM, Guido van Rossum <guido@python.org> wrote:

TBH the argument from being able to roll back things doesn't feel very strong at Dropbox. We use a merge process where developers' commits are always squashed into a single commit in master once code review is finished (we have command-line tools for this based on Phabricator's Arcanist). And of course we also have to roll back stuff. But I've never heard of a problem where we had to roll back multiple commits to get back into a reasonable state.

You don’t really need the merge commit if you’re doing squash merges since you have a single atomic unit to roll back which is equal to the final, squashed, commit. It’s difficult to roll back the effect of merging an entire branch with multiple commits if you’re not squash merging but just regular merging if it was a FF merge without merge commits because there is no longer any record that a merge took place at all. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

On Dec 14, 2015, at 11:46 AM, Barry Warsaw <barry@python.org> wrote:

On Dec 14, 2015, at 11:35 AM, Donald Stufft wrote:

...

You don’t really need the merge commit if you’re doing squash merges

Just to be clear though, with a GitHub choice, you'd either have to accept merge commits, or do the squash merges locally and then push master because they aren't supported by GH's u/i, right?

Correct. Where “locally” could also include a robot to do merges based on a comment left on the PR, like “Ok to Merge”. There is no built in option to do anything but ``git merge —no-ff`` in Github’s UI. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

On Dec 14, 2015, at 3:48 PM, Brett Cannon <brett@python.org> wrote:

This also means neither GitHub or GitLab offer any specific benefit over each other from what I can tell (if we dropped unique NEWS entries and did cherry picks manually then I think GitLab EE gets us web-based squashing if I understand things properly). But this bot-based approach gets us the common-case managed entirely in the browser with a clean history.

Correct. You’re down to more nebulous things like “I like X’s UI better”, “X is more Free”, “X has a large community buy in” and such. Of course the downside is that we’d be diverging from the “normal” process for either tool so we’d be a bit special snowflake which increases the overhead for potential core developers… but I don’t think a bot command is a particularly onerous requirement for core developers to learn, and since it’s limited to just core developers it doesn’t affect the (hopefully) larger pool of external contributors. Since Gitlab CE is OSS you could potentially modify it to have these things baked directly into it, but that feels like a less optimal solution than a bot since we’d be essentially forking it and then maintaining it ourselves. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

On Mon, 14 Dec 2015 16:03:57 -0500, Barry Warsaw <barry@python.org> wrote:

On Dec 14, 2015, at 08:48 PM, Brett Cannon wrote:

...

What that comment would do is trigger a bot

As long as we a clear, responsive owner of the bot, and a plan for when it inevitably dies or does the wrong thing. If the bot is going to be in the critical loop, we can't allow its failure to block our work.

Well, the bot being down won't *block* anything, it will just remove the level of added convenience we are looking for (ie: you can always fall back to manual commit/push). Likewise, mistakes will probably have to be cleaned up manually, and we might have to shut it off until the bug gets fixed in that situation. That said, we really don't want the bot to have only a single maintainer, though it may initially have a single author, and ideally operations would be responsible for keeping it running. There are a number of advantages to having a bot be the way this works rather than depending on features of the provider's software, easier integration with other tools (ie: the tracker) being a big one. It would also presumably make switching providers somewhat easier. --David

On 15 December 2015 at 07:46, R. David Murray <rdmurray@bitdance.com> wrote:

That said, we really don't want the bot to have only a single maintainer, though it may initially have a single author, and ideally operations would be responsible for keeping it running.

There are a number of existing Python projects that provide the scaffolding for handling GitHub webhook requests (e.g. https://github.com/nickfrostatx/flask-hookserver is a project for setting up a GitHub bot as a Flask web service), so getting something up and running on Heroku should be relatively straightforward (Donald has been looking at PaaS hosting options of late, and a GitHub bot wouldn't benefit from a full VM in the Rackspace infrastructure).

There are a number of advantages to having a bot be the way this works rather than depending on features of the provider's software, easier integration with other tools (ie: the tracker) being a big one. It would also presumably make switching providers somewhat easier.

Right, GitLab offers its own webhook support (although I assume the event details are different), so going with a bot-based workflow even means that we could even offer a read-write GitLab mirror in the future, since the bot would handle the actual merge-and-commit process based on review comments, regardless of which server was used to submit the review. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 15 December 2015 at 17:32, Donald Stufft <donald@stufft.io> wrote:

Only if the review system is used. You still have split brain problems for people merging manually or pushing direct to master.

Right, but that's just "git-with-a-different-host" regardless of the specific web interface in use - if it was set up, any such mirror repo would be read/write from a "you can submit and work with change requests here", rather than "you can use git to push updates here". Cheers, Nick.

Sent from my iPhone

...

On Dec 14, 2015, at 10:58 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:

going with a bot-based workflow even means that we could even offer a read-write GitLab mirror in the future, since the bot would handle the actual merge-and-commit process based on review comments, regardless of which server was used to submit the review.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Dec 1, 2015, at 6:53 PM, Guido van Rossum <guido@python.org> wrote:

(A bigger question might be if merging a patch across three branches will become any easier. At this point I'm reluctant to accept small asyncio PRs because I have to merge then into 3.4, 3.5 and 3.6. But I expect that this would be a matter of git command line tricks regardless of the hosting platform.)

If the original PR was to the 3.4 branch, once you merge it you can, in the web ui, create a new PR that merges 3.4 into 3.5, then 3.5 into 3.6. That’s about the best that’s available right now. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

On Dec 01, 2015, at 03:53 PM, Guido van Rossum wrote:

(A bigger question might be if merging a patch across three branches will become any easier. At this point I'm reluctant to accept small asyncio PRs because I have to merge then into 3.4, 3.5 and 3.6. But I expect that this would be a matter of git command line tricks regardless of the hosting platform.)

I think that's true, but I'd love to be wrong about that. In Mailman 3 we have the maintenance branch and the devel branch. Usually bugs get fixed on devel first, and then I have to manually cherry-pick those comments from the master branch to the release-3.0 branch. This usually goes pretty smoothly except for (dum dum DUMM....) the NEWS file. I have to resolve those conflicts manually (Emacs' smerge-mode to the rescue), commit that, and then push to the origin. I've found it's usually easier to rebase the original commit onto master so there's just one commit that needs to be cherry-picked, and I find it's usually safer to do all this on a topic branch, get that all nice and clean, and then fast-forward merge onto the maintenance branch. Git branches are cheap so this works well - you just don't want to push those temporary topic branches.

in the past we've had such arrangements, and IIRC typically over the years the hosting company has lost interest. We're better off being hosted on the default terms by a thriving company than by a company that's just scraping by offering us an incentive.

There are more than a few assumptions in that paragraph that make it difficult to respond to <wink>. Cheers, -Barry

On Dec 01, 2015, at 08:47 PM, Barry Warsaw wrote:

However I think removing the intermediate commits require an interactive rebase to squash commits on the source branch. GitLab doesn't provide a through-the-web interface to interactive rebases. You'd have to use the command line for this, and I often do that when I want to make local cleanups to the source branch.

Oops, I'm wrong about that. `git merge` allows for --squash and a quick search found some reference to this for GitLab, but I don't know whether this happens automatically or not when the merge is accepted. I don't see an option to squash or not squash though, so I'd have to ask. Cheers, -Barry

On Dec 11, 2015, at 03:27 AM, Brett Cannon wrote:

Any news from the gitlab CEO about what they will offer and what the plan is? I'm still aiming to get everything squared away by Dec 15 so I have 2 weeks to think things through.

Brett, see my question from 01-Dec - I think we should still try to outline what we want before we approach GitLab with our detailed request. I want to avoid the inevitable "What will you offer us?" "Well, what do you want?" back and forth. Also, as PEP 481 is still in contention as the alternative, it still mentions Phabricator which I believe is off the table, so that PEP needs an update. And afaik, none of the workflow PEPs actually describe Nick's HOCAEIT suggestion for getting the best out of both GitLab and GitHub. I know you want to make a decision soon, but IMHO we need to get these issues resolved before we can move forward. Cheers, -Barry

On 12 December 2015 at 10:24, Brett Cannon <brett@python.org> wrote:

On Fri, 11 Dec 2015 at 07:33 Barry Warsaw <barry@python.org> wrote:

...

I know you want to make a decision soon, but IMHO we need to get these issues resolved before we can move forward.

Yes, but we have all known Nick long enough to know that he will just keep coming up with ideas so I can't leave it open indefinitely. :)

Hey, I resemble that remark! :) The HOCAEIT comment from Barry was a reference to "Have Our Cake, And Eat It Too", but I don't agree we need to write that up before you make a decision - that was mostly me thinking out loud about possible alternatives, rather than an offer to do the work to set it up. However, the emails about it may be a useful reference if Barry wanted to discuss it directly with GitLab (since they're clearly already moving in that direction given the Repository Mirroring additions in GitLab EE 8.2).

I'm not enthralled with the idea of trying to attempt any bi-directional coordination between GitHub and GitLab Even uni-directional coordination seems like it might be more hassle than it's worth (what's the point of telling people "GitHub is totally fine" and then having it all mirrored to GitLab, along with all of the technical maintenance and complications? Just to avoid upsetting some people from having to use GitHub since the existence of the tooling means we don't have to worry about vendor lock-in?)

An analogy I've been pondering to help explain the inclusiveness/exclusiveness aspects of decisions to use proprietary services as part of an open source project's infrastructure is to compare it to vegetarianism & veganism: when "Sign up to this proprietary service" is a precondition for participation, there are going to be folks who opt out, just as there will be folks who opt out of community events that don't offer vegetarian or vegan meal options.

From a practical community management perspective, the relevant part of the analogy in relation to development tools is that when choosing a fixed menu for a community event you either:

a. offer the vegetarian/vegan option as the only option, and aim to make sure that even the non-vegetarians enjoy it; or b. offer a non-vegetarian option by default, but aim to make sure the vegetarian/vegan option is still a decent one Switching from the analogy back to the question at hand, unless you're actually the Free Software Foundation, being inclusive of free software and privacy advocates when choosing workflow tools doesn't mean adopting a categorical ban on the use of proprietary software services, but it does mean considering what the impact will be on the folks that choose to opt out of the proprietary dependency, and ensuring that that alternative workflow is still a reasonable one. In the case of a GitHub-only CPython core workflow, nothing would change for strict free software advocates that aren't core developers: the process will be to upload patches to Roundup as they do today.

From there, we'll either continue reviewing them in Rietveld, or else change those scripts to submit a PR on GitHub instead.

However, folks that are both current core developers *and* strict free software/privacy advocates would effectively have their commit privileges revoked if they didn't sign up for a GitHub account, as you need one in order to upload your SSH keys. Since the goal of the workflow changes is to make more effective use of limited core developer time, reducing the size of that pool as a side effect really wouldn't be a good outcome. As such, it might be worth your while to try to quantify that more precisely by asking current core developers their views on getting a GitHub account: a. I already have one (happily) b. I already have one (begrudgingly) c. I'd happily get one d. I'd begrudgingly get one e. I wouldn't get one And make it clear that failing to fill in the survey will be interpreted as "not e". The folks that would potentially benefit from a GitLab instance being part of the setup are then those that fall into categories "b", "d", and "e", and you could ask a follow-up question regarding alternatives which they considered a preferable solution to signing up for a GitHub account: a. GitLab EE instance hosted and managed by GitLab, Inc b. GitLab EE instance hosted by the PSF, and managed by GitLab, Inc c. GitLab EE instance hosted and managed by the PSF d. GitLab CE instance hosted and managed by the PSF Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 12 December 2015 at 23:39, Donald Stufft <donald@stufft.io> wrote:

It seems pretty hypocritical to be against GitHub because it's proprietary and hosted by a company that can sell your data (both things I've seen people say against GitHub) and be pro GitLab EE (either hosted by Gitlab or ourselves) or pro using a Hosted Gitlab. I don't personally have a problem with it, I'm fine with the tooling not being free but it seems disingenuous to treat Gitlab EE (as opposed CE) as if it's free software. I'm particularly interested in a managed option of some kind to reduce to load on the infra team.

Yeah, that's why I suggested there'd need to be at least four options in a useful survey question (if Brett decides a survey is worth doing at all), since those would help tease out what folks are actually worried about, similar to the way that presenting a vegetarian option for a meal may still not be sufficient for vegans. Running through them: a. GitLab EE instance hosted and managed by GitLab, Inc * Being happier with this option than with the use of GitHub would indicate someone was bothered by GitHub specifically, rather than the use of proprietary software. Given the tech industry's propensity for flocking to proprietary vendors who then turn around to attempt to extract monopoly rents (cf. IBM, legacy UNIX vendors, Microsoft, Oracle, etc), this is certainly one of the factors in my own concerns - Silicon Valley venture capitalists are deliberately running a "ubiquity first, revenue later" strategy for GitHub, which means it's important to plan for an exit strategy if the flow of free money (and the associated free services) dries up when investors start getting impatient for their returns. While I've personally come to the conclusion that "Migrate-to-GitLab-or-some-other-service-if-the-need-arises" adequately addresses such concerns (along with the general market forces of GitHub doing battle with incumbents like Microsoft's Team Foundation Server and Atlassian's software lifecycle management portfolio), it may be that not everyone feels the same way. b. GitLab EE instance hosted by the PSF, and managed by GitLab, Inc * This gets into cases where folks are mainly concerned about having permanent copies of everything important from GitHub always ready to go on PSF controlled infrastructure, even if we're not using it actively. It's essentially the "GitLab-as-GitHub-exit-strategy" approach, with an additional hedge where PSF admins have direct control over the data, even if GitLab have been tasked with the day-to-day operation of the service. c. GitLab EE instance hosted and managed by the PSF * Similar to the previous option, this option would likely indicate that someone is more concerned by privacy and data control issues than they are by the use of proprietary software. Both GitLab and GitHub are private for profit entities, answerable to their investors moreso than to their users, so monetisation of user data, including becoming a data broker for information on user's online activities, is well within the prospective scope of corporate activities (even if neither company is pursuing such activities today). By contrast, the PSF is a non-profit entity, answerable to its members. If the PSF controls the identity information needed to participate in the core development process, then we can categorically assure participants that we're not selling their information to data brokers, whereas if we use third party services, we'll have to both tell people to check the Terms & Conditions for those services, as well as consider defining trigger conditions for what we consider to be unacceptable barriers to entry for participation in the core development process. d. GitLab CE instance hosted and managed by the PSF * This is the only option that gets all the way to "Free software needs free tools" - it denies even GitLab's open core development model as a legitimate revenue raising tactic, since it's still using the proprietary model to exercise coercive control over customers. The main advantage GitLab EE has over GitHub in that context is that the exit strategy from EE to CE is even simpler than that from GitHub to GitLab EE. I honestly don't know how the different areas of concern break down across the core contributor base, and I've seen enough demands for free software and privacy advocates to "stop being unreasonable" in various contexts to suspect that an anonymous survey (that is still somehow restricted to core developers) would be the only way to get an even halfway accurate assessment. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Wed, 2 Dec 2015 at 03:42 Berker Peksağ <berker.peksag@gmail.com> wrote:

On Wed, Dec 2, 2015 at 1:24 AM, Brett Cannon <brett@python.org> wrote:

...

It's Dec 1, which means it's time for any questions people have about the proposed workflows so we can get answers by Dec 15.

I have one question that applies to both proposals and one specific to GitLab. The general one is whether both Guido and me can both be happy. :) Guido doesn't want intermediate commits nor what he calls "merge turds" to show up in the history. I want to be able to do merges from the browser. Do

I agree with Guido. --no-ff merge is probably the only GitHub feature that I really dislike. It would be good to have a GitHub bot to handle ff merges.

For example, with a bot, a core developer can add one of the following comments to merge the PR;

@biggles merge # merges into master

@biggles merge 3.5+ # merges into 3.5 and master (or merges into master then cherry-pick it to 3.5)

This will also solve the pull request and email flood problem since a contributor won't have to open N pull requests if their patch needs to be backported to maintenance branches.

I was actually wondering if this was possible just last night. :) Do both GitHub and GitLab have the appropriate APIs to support something like this? It would be fantastic if a bot can get Guido the style of commits he wants while letting us trigger the merges through the browser (especially since a bot is nice, but we don't have to feel like we are stranded if it goes down since old-school solution of manual merges would still be possible). And could the bot, when told to cherry-pick/merge into another branch, create a PR if there is a merge conflict so that it can be fixed in a checkout and not lost track of? Otherwise at least leave a message in the PR that stated what versions to commit to that failed and hence not close the PR until all branches are properly handled? And this bot doesn't have to do it, but we should definitely make sure we have at least automated testing of all PRs on *some* OS and also a way to verify the PR contributor signed the CLA (the testing should be easy enough on either and the CLA should be doable somehow).

I will have some spare time to work on such a tool in the next months.

Great! Let's choose the platform first -- I'm aiming for Jan 1 -- and then if you still have time, Berker, it would be great if you can help with the tool. But we can definitely discuss potential workflows and what the bot would need to do now since that's probably platform-agnostic to make sure this is a tenable solution.

On Wed, 2 Dec 2015 at 10:50 R. David Murray <rdmurray@bitdance.com> wrote:

On Wed, 02 Dec 2015 18:34:49 +0000, Brett Cannon <brett@python.org> wrote:

...

And this bot doesn't have to do it, but we should definitely make sure we have at least automated testing of all PRs on *some* OS and also a way to

Kushal did some work in this direction (for unix builds), but we haven't done anything with it for a while.

Donald hooked up Travis to the GitHub test instance and runs the PRs using both gcc and clang, so I know at least GitHub + Travis has us covered. Barry has said that GitLab supports runners but you have to run them yourselves. I would also be curious to know if we could get a test coverage report somehow (e.g., can coveralls.io support be added and have it use https://bitbucket.org/ned/coveragepy/src/787158b1da024fd57b9a4f05c35042b7cdb... which requires a checkout of coverage.py).

...

verify the PR contributor signed the CLA (the testing should be easy enough on either and the CLA should be doable somehow).

This should be easy enough to do via the xmlrpc interface to the tracker, or (better) the rest interface if/when it lands.

I guess it would simply require people to add their GitHub IDs to their issue tracker profile. We probably should also make sure the process gets as automated as possible as I think Ewa still flips some bits manually. -Brett

--David _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct

On Dec 02, 2015, at 06:58 PM, Brett Cannon wrote:

Barry has said that GitLab supports runners but you have to run them yourselves.

There are now some shared runners that we can probably use. I haven't had much time to investigate them though. Cheers, -Barry

On Dec 2, 2015, at 2:21 PM, R. David Murray <rdmurray@bitdance.com> wrote:

That's interesting. I thought someone had already tried travis and our test suite took too long to run. Sounds like Kushal's stuff could be hooked up to gitlab.

Unless I did it wrong, the tests took 11 minutes to run, Travis allows up to 50. https://travis-ci.org/fake-python/cpython/jobs/89432399 ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

On Dec 2, 2015, at 2:45 PM, R. David Murray <rdmurray@bitdance.com> wrote:

On Wed, 02 Dec 2015 14:23:41 -0500, Donald Stufft <donald@stufft.io> wrote:

...

...

On Dec 2, 2015, at 2:21 PM, R. David Murray <rdmurray@bitdance.com> wrote:

That's interesting. I thought someone had already tried travis and our test suite took too long to run. Sounds like Kushal's stuff could be hooked up to gitlab.

Unless I did it wrong, the tests took 11 minutes to run, Travis allows up to 50.

https://travis-ci.org/fake-python/cpython/jobs/89432399

That looks OK. Maybe they changed their limits? 50 sounds way longer than I remember whoever it was (Kushal?) quoting.

It’s been 50 minutes for as long as I can remember, but there is an additional limit of 10 minutes without any output, so if it was being run in a way that didn’t cause output to occur for 10 minutes it’d get killed. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

On Dec 08, 2015, at 12:46 PM, Nick Coghlan wrote:

Something I've been pondering recently is whether we might be able to "have our cake and eat it too", by setting up the workflow to use GitHub to facilitate easy contributions by folks with a GitHub account (and allowing folks that don't want a GitHub account to continue uploading patches to the tracker), while using GitLab to continue to host the master repository used to make releases on a *.python.org subdomain.

The main prompt for that thought is the fact that, with the 8.2 release, GitLab introduced repository mirroring as a first class feature in the Enterprise Edition: https://about.gitlab.com/2015/11/22/gitlab-8-2-released/

With that setup, development work would happen on GitHub and Roundup, while release management work would mostly happen on GitLab, docs.python.org and www.python.org.

The question for GitLab then would be whether or not bidirectional syncing of pull requests/merge requests between GitHub & GitLab is on their radar as a possible future feature (one-way syncing is already possible, through projects like https://pypi.python.org/pypi/github2gitlab )

I don't know the answer to the latter, but I do think the repository mirroring feature could help us here. If we do this though, why wouldn't we also accept merge requests on the gitlab instance? I guess the problem is that if only gitlab can mirror, then commits to the gitlab repo have to be manually pushed to github. Maybe a webhook can automate that for us. The trick will be ensuring that we don't get into a situation where there are conflicts when mirroring. That can happen if changes are merged to both the github and gitlab repos. So I think one of them has to the be the canonical origin and the other a mirror. As far as processing pull requests or merge requests, I don't think it much matters. Someone could submit a pull request on github and it can be easily (though, not automatically) merged into a gitlab origin repo. What we can't do (I think) is let Alice commit and push to github while Bob commits and pushes to gitlab. Now we've got DAGs that probably needs manual reconciliation. I think we could handle that with protected branches so *only* mirroring will be allowed in the unofficial direction. Cheers, -Barry