PEP 481 - Migrate Some Supporting Repositories to Git and Github
As promised in the "Move selected documentation repos to PSF BitBucket account?" thread I've written up a PEP for moving selected repositories from hg.python.org to Github. You can see this PEP online at: https://www.python.org/dev/peps/pep-0481/ I've also reproduced the PEP below for inline discussion. ----------------------- Abstract ======== This PEP proposes migrating to Git and Github for certain supporting repositories (such as the repository for Python Enhancement Proposals) in a way that is more accessible to new contributors, and easier to manage for core developers. This is offered as an alternative to PEP 474 which aims to achieve the same overall benefits but while continuing to use the Mercurial DVCS and without relying on a commerical entity. In particular this PEP proposes changes to the following repositories: * https://hg.python.org/devguide/ * https://hg.python.org/devinabox/ * https://hg.python.org/peps/ This PEP does not propose any changes to the core development workflow for CPython itself. Rationale ========= As PEP 474 mentions, there are currently a number of repositories hosted on hg.python.org which are not directly used for the development of CPython but instead are supporting or ancillary repositories. These supporting repositories do not typically have complex workflows or often branches at all other than the primary integration branch. This simplicity makes them very good targets for the "Pull Request" workflow that is commonly found on sites like Github. However where PEP 474 wants to continue to use Mercurial and wishes to use an OSS and self-hosted and therefore restricts itself to only those solutions this PEP expands the scope of that to include migrating to Git and using Github. The existing method of contributing to these repositories generally includes generating a patch and either uploading them to bugs.python.org or emailing them to peps@python.org. This process is unfriendly towards non-comitter contributors as well as making the process harder than it needs to be for comitters to accept the patches sent by users. In addition to the benefits in the pull request workflow itself, this style of workflow also enables non techincal contributors, especially those who do not know their way around the DVCS of choice, to contribute using the web based editor. On the committer side the Pull Requests enable them to tell, before merging, whether or not a particular Pull Request will break anything. It also enables them to do a simple "push button" merge which does not require them to check out the changes locally. Another such feature that is useful in particular for docs, is the ability to view a "prose" diff. This Github specific feature enables a committer to view a diff of the rendered output which will hide things like reformatting a paragraph and show you what the actual "meat" of the change actually is. Why Git? -------- Looking at the variety of DVCS which are available today it becomes fairly clear that git has gotten the vast mindshare of people who are currently using it. The Open Hub (Previously Ohloh) statistics [#openhub-stats]_ show that currently 37% of the repositories Open Hub is indexing is using git which is second only to SVN (which has 48%) while Mercurial has just 2% of the indexed repositories (beating only bazaar which has 1%). In additon to the Open Hub statistics a look at the top 100 projects on PyPI (ordered by total download counts) shows us that within the Python space itself there is a majority of projects using git: === ========= ========== ====== === ==== Git Mercurial Subversion Bazaar CVS None === ========= ========== ====== === ==== 62 22 7 4 1 1 === ========= ========== ====== === ==== Chosing a DVCS which has the larger mindshare will make it more likely that any particular person who has experience with DVCS at all will be able to meaningfully use the DVCS that we have chosen without having to learn a new tool. In addition to simply making it more likely that any individual will already know how to use git, the number of projects and people using it means that the resources for learning the tool are likely to be more fully fleshed out and when you run into problems the liklihood that someone else had that problem and posted a question and recieved an answer is also far likelier. Thirdly by using a more popular tool you also increase your options for tooling *around* the DVCS itself. Looking at the various options for hosting repositories it's extremely rare to find a hosting solution (whether OSS or commerical) that supports Mercurial but does not support Git, on the flip side there are a number of tools which support Git but do not support Mercurial. Therefore the popularity of git increases the flexibility of our options going into the future for what toolchain these projects use. Also by moving to the more popular DVCS we increase the likelhood that the knowledge that the person has learned in contributing to these support repositories will transfer to projects outside of the immediate CPython project such as to the larger Python community which is primarily using Git hosted on Github. In previous years there was concern about how well supported git was on Windows in comparison to Mercurial. However git has grown to support Windows as a first class citizen. In addition to that, for Windows users who are not well aquanted with the Windows command line there are GUI options as well. On a techincal level git and Mercurial are fairly similar, however the git branching model is signifcantly better than Mercurial "Named Branches" for non-comitter contributors. Mercurial does have a "Bookmarks" extension however this isn't quite as good as git's branching model. All bookmarks live in the same namespace so it requires individual users to ensure that they namespace the branchnames themselves lest the risk collision. It also is an extension which requires new users to first discover they need an extension at all and then figure out what they need to do in order to enable that extension. Since it is an extension it also means that in general support for them outside of Mercurial core is going to be less than 100% in comparison to git where the feature is built in and core to using git at all. Finally users who are not used to Mercurial are unlikely to discover bookmarks on their own, instead they will likely attempt to use Mercurial's "Named Branches" which, given the fact they live "forever", are not often what a project wants their contributors to use. Why Github? ----------- There are a number of software projects or web services which offer functionality similar to that of Github. These range from commerical web services such as a Bitbucket to self-hosted OSS solutions such as Kallithea or Gitlab. This PEP proposes that we move these repositories to Github. There are two primary reasons for selecting Github: Popularity and Quality/Polish. Github is currently the most popular hosted repository hosting according to Alexa where it currently has a global rank of 121. Much like for Git itself by choosing the most popular tool we gain benefits in increasing the likelhood that a new contributor will have already experienced the toolchain, the quality and availablity of the help, more and better tooling being built around it, and the knowledge transfer to other projects. A look again at the top 100 projects by download counts on PyPI shows the following hosting locations: ====== ========= =========== ========= =========== ========== GitHub BitBucket Google Code Launchpad SourceForge Other/Self ====== ========= =========== ========= =========== ========== 62 18 6 4 3 7 ====== ========= =========== ========= =========== ========== In addition to all of those reasons, Github also has the benefit that while many of the options have similar features when you look at them in a feature matrix the Github version of each of those features tend to work better and be far more polished. This is hard to quantify objectively however it is a fairly common sentiment if you go around and ask people who are using these services often. Finally a reason to choose a web service at all over something that is self-hosted is to be able to more efficiently use volunteer time and donated resources. Every additional service hosted on the PSF infrastruture by the PSF infrastructure team further spreads out the amount of time that the volunteers on that team have to spend and uses some chunk of resources that could potentionally be used for something where there is no free or affordable hosted solution available. One concern that people do have with using a hosted service is that there is a lack of control and that at some point in the future the service may no longer be suitable. It is the opinion of this PEP that Github does not currently and has not in the past engaged in any attempts to lock people into their platform and that if at some point in the future Github is no longer suitable for one reason or another than at that point we can look at migrating away from Github onto a different solution. In other words, we'll cross that bridge if and when we come to it. Example: Scientific Python -------------------------- One of the key ideas behind the move to both git and Github is that a feature of a DVCS, the repository hosting, and the workflow used is the social network and size of the community using said tools. We can see this is true by looking at an example from a sub-community of the Python community: The Scientific Python community. They have already migrated most of the key pieces of the SciPy stack onto Github using the Pull Request based workflow starting with IPython and as more projects moved over it became a natural default for new projects. They claim to have seen a great benefit from this move, where it enables casual contributors to easily move between different projects within their sub-community without having to learn a special, bespoke workflow and a different toolchain for each project. They've found that when people can use their limited time on actually contributing instead of learning the different tools and workflows that not only do they contribute more to one project, that they also expand out and contribute to other projects. This move is also attributed to making it commonplace for members of that community to go so far as publishing their research and educational materials on Github as well. This showcases the real power behind moving to a highly popular toolchain and workflow, as each variance introduces yet another hurdle for new and casual contributors to get past and it makes the time spent learning that workflow less reusable with other projects. Migration ========= Through the use of hg-git [#hg-git]_ we can easily convert a Mercurial repository to a Git repository by simply pushing the Mercurial repository to the Git repository. People who wish to continue to use Mercurual locally can then use hg-git going into the future using the new Github URL, however they will need to re-clone their repositories as using Git as the server seems to trigger a one time change of the changeset ids. As none of the selected repositories have any tags, branches, or bookmarks other than the ``default`` branch the migration will simply map the ``default`` branch in Mercurial to the ``master`` branch in git. In addition since none of the selected projects have any great need of a complex bug tracker, they will also migrate their issue handling to using the GitHub issues. In addition to the migration of the repository hosting itself there are a number of locations for each particular repository which will require updating. The bulk of these will simply be changing commands from the hg equivilant to the git equivilant. In particular this will include: * Updating www.python.org to generate PEPs using a git clone and link to Github. * Updating docs.python.org to pull from Github instead of hg.python.org for the devguide. * Enabling the ability to send an email to python-checkins@python.org for each push. * Enabling the ability to send an IRC message to #python-dev on Freenode for each push. * Migrate any issues for these projects to their respective bug tracker on Github. This will restore these repositories to similar functionality as they currently have. In addition to this the migration will also include enabling testing for each pull request using Travis CI [#travisci]_ where possible to ensure that a new PR does not break the ability to render the documentation or PEPs. User Access =========== Moving to Github would involve adding an additional user account that will need to be managed, however it also offers finer grained control, allowing the ability to grant someone access to only one particular repository instead of the coarser grained ACLs available on hg.python.org. References ========== .. [#openhub-stats] `Open Hub Statistics <https://www.openhub.net/repositories/compare>` .. [#hg-git] `hg-git <https://hg-git.github.io/>` .. [#travisci] `Travis CI <https://travis-ci.org/>` --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Donald Stufft <donald <at> stufft.io> writes:
[words words words]
I strongly support this PEP. I'd like to share two pieces of information. Both of these are personal anecdotes: For the past several years, I've been a contributor on two major projects using mercurial, CPython and PyPy. PyPy has a strong culture of in-repo branching, basically all contributors regularly make branches in the main repo for their work, and we're very free in giving people commit rights, so almost everyone working on PyPy in any way has this level of access. This workflow works ok. I don't love it as much as git, but it's fine, it's not an impediment to my work. By contrast, CPython does not have this type of workflow, there are almost no in-tree branches besides the 2.7, 3.4, etc. ones. Despite being a regular hg user for years, I have no idea how to create a local-only branch, or a branch which is pushed to a remote (to use the git term). I also don't generally commit my own work to CPython, even though I have push privledges, because I prefer to *always* get code review on my work. As a result, I use a git mirror of CPython to do all my work, and generate patches from that. The conclusion I draw from this is that hg's workflow is probably fine if you're a committer on the project, or don't ever need to maintain multiple patches concurrently (and thus can just leave everything uncommitted in the repo). However, the hg workflow seems extremely defficient at non-committer contributors. The seconds experience I have is that of Django's migration to git and github. For a long time we were on SVN, and we were very resistant to moving to DVCS in general, and github in particular. Multiple times I said that I didn't see how exporting a patch and uploading it to trac was more difficult than sending a pull request. That was very wrong on my part. My primary observation is not about new contributors though, it's actually about the behavior of core developers. Before we were on github, it was fairly rare for core developers to ask for reviews for anything besides *gigantic* patches, we'd mostly just commit stuff to trunk. Since the switch to github, I've seen that core developers are *far* more likely to ask for reviews of their work before merging. Big +1 from me, thanks for writing this up Donald, Alex
On Nov 29, 2014, at 7:15 PM, Alex Gaynor <alex.gaynor@gmail.com> wrote:
Donald Stufft <donald <at> stufft.io> writes:
[words words words]
I strongly support this PEP. I'd like to share two pieces of information. Both of these are personal anecdotes:
For the past several years, I've been a contributor on two major projects using mercurial, CPython and PyPy. PyPy has a strong culture of in-repo branching, basically all contributors regularly make branches in the main repo for their work, and we're very free in giving people commit rights, so almost everyone working on PyPy in any way has this level of access. This workflow works ok. I don't love it as much as git, but it's fine, it's not an impediment to my work.
By contrast, CPython does not have this type of workflow, there are almost no in-tree branches besides the 2.7, 3.4, etc. ones. Despite being a regular hg user for years, I have no idea how to create a local-only branch, or a branch which is pushed to a remote (to use the git term). I also don't generally commit my own work to CPython, even though I have push privledges, because I prefer to *always* get code review on my work. As a result, I use a git mirror of CPython to do all my work, and generate patches from that.
The conclusion I draw from this is that hg's workflow is probably fine if you're a committer on the project, or don't ever need to maintain multiple patches concurrently (and thus can just leave everything uncommitted in the repo). However, the hg workflow seems extremely defficient at non-committer contributors.
I also don’t know how to do this. When I’m doing multiple things for CPython my “branching” strategy is essentially using hg diff to create a patch file with my “branch” name (``hg diff > my-branch.patch``), then revert all of my changes (``hg revert —all —no-backup``), then either work on a new “branch” or switch to an old “branch” by applying the corresponding patch (``patch -p1 < other-branch.patch``).
The seconds experience I have is that of Django's migration to git and github. For a long time we were on SVN, and we were very resistant to moving to DVCS in general, and github in particular. Multiple times I said that I didn't see how exporting a patch and uploading it to trac was more difficult than sending a pull request. That was very wrong on my part.
My primary observation is not about new contributors though, it's actually about the behavior of core developers. Before we were on github, it was fairly rare for core developers to ask for reviews for anything besides *gigantic* patches, we'd mostly just commit stuff to trunk. Since the switch to github, I've seen that core developers are *far* more likely to ask for reviews of their work before merging.
I’ve also seen this effect, not just in Django but that I also notice my own behavior. Projects where I have commit access but which aren’t on Github I find myself less likely to look for others to review them and find myself just committing directly to master/default/trunk while I do tend to look for reviews and create PRs to give others the chance to review on Github. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Sun, Nov 30, 2014 at 11:37 AM, Donald Stufft <donald@stufft.io> wrote:
I also don’t know how to do this. When I’m doing multiple things for CPython my “branching” strategy is essentially using hg diff to create a patch file with my “branch” name (``hg diff > my-branch.patch``), then revert all of my changes (``hg revert —all —no-backup``), then either work on a new “branch” or switch to an old “branch” by applying the corresponding patch (``patch -p1 < other-branch.patch``).
IMO, this is missing out on part of the benefit of a DVCS. When your patches are always done purely on the basis of files, and have to be managed separately, everything will be manual; and your edits won't (normally) contain commit messages, authorship headers, date/time stamps, and all the other things that a commit will normally have. Using GitHub automatically makes all that available; when someone forks the project and adds a commit, that commit will exist and have its full identity, metadata, etc, and if/when it gets merged into trunk, all that will be carried through automatically. I strongly support this PEP. ChrisA
On 11/29/2014 05:15 PM, Chris Angelico wrote:
On Sun, Nov 30, 2014 at 11:37 AM, Donald Stufft <donald@stufft.io> wrote:
I also don’t know how to do this. When I’m doing multiple things for CPython my “branching” strategy is essentially using hg diff to create a patch file with my “branch” name (``hg diff > my-branch.patch``), then revert all of my changes (``hg revert —all —no-backup``), then either work on a new “branch” or switch to an old “branch” by applying the corresponding patch (``patch -p1 < other-branch.patch``).
IMO, this is missing out on part of the benefit of a DVCS. When your patches are always done purely on the basis of files, and have to be managed separately, everything will be manual; and your edits won't (normally) contain commit messages, authorship headers, date/time stamps, and all the other things that a commit will normally have. Using GitHub automatically makes all that available; when someone forks the project and adds a commit, that commit will exist and have its full identity, metadata, etc, and if/when it gets merged into trunk, all that will be carried through automatically.
There is no reason to make this `hg diff` dance (but ignorance). - You can make plain commit with your changes. - You can export commit content using `hg export` - You can change you patch content will all kind of tools (amend, rebase, etc) - You can have multiple branches without any issue to handle concurrent workflow. We (Mercurial developer) will be again sprinting at Pycon 2015. We can probably arrange some workflow discussion//training there. -- Pierre-Yves David
On 11/29/2014 04:37 PM, Donald Stufft wrote:
On Nov 29, 2014, at 7:15 PM, Alex Gaynor <alex.gaynor@gmail.com> wrote:
Despite being a regular hg user for years, I have no idea how to create a local-only branch, or a branch which is pushed to a remote (to use the git term). I also don’t know how to do this.
Instead of collectively scratching your heads, could one of you guys do the research and figure out whether or not hg supports this workflow? One of the following two things must be true: 1. hg supports this workflow (or a reasonable fascimile), which may lessen the need for this PEP. 2. hg doesn't support this workflow, which may strengthen the need for this PEP. Saying "I've been using hg for years and I don't know whether it supports this IMPORTANT THING" is not a particularly compelling argument. //arry/
Larry Hastings <larry@hastings.org> writes:
On 11/29/2014 04:37 PM, Donald Stufft wrote:
On Nov 29, 2014, at 7:15 PM, Alex Gaynor <alex.gaynor@gmail.com> wrote:
Despite being a regular hg user for years, I have no idea how to create a local-only branch, or a branch which is pushed to a remote (to use the git term). I also don’t know how to do this.
Instead of collectively scratching your heads, could one of you guys do the research and figure out whether or not hg supports this workflow? One of the following two things must be true:
1. hg supports this workflow (or a reasonable fascimile), which may lessen the need for this PEP. 2. hg doesn't support this workflow, which may strengthen the need for this PEP.
Assuming git's "all work is done in a local branch" workflow, you could use bookmarks with hg http://lostechies.com/jimmybogard/2010/06/03/translating-my-git-workflow-wit... http://stevelosh.com/blog/2009/08/a-guide-to-branching-in-mercurial/#branchi... http://mercurial.selenic.com/wiki/BookmarksExtension#Usage http://stackoverflow.com/questions/1598759/git-and-mercurial-compare-and-con... -- Akira
On Nov 30, 2014, at 2:08 AM, Larry Hastings <larry@hastings.org> wrote:
On 11/29/2014 04:37 PM, Donald Stufft wrote:
On Nov 29, 2014, at 7:15 PM, Alex Gaynor <alex.gaynor@gmail.com> <mailto:alex.gaynor@gmail.com> wrote:
Despite being a regular hg user for years, I have no idea how to create a local-only branch, or a branch which is pushed to a remote (to use the git term). I also don’t know how to do this.
Instead of collectively scratching your heads, could one of you guys do the research and figure out whether or not hg supports this workflow? One of the following two things must be true: hg supports this workflow (or a reasonable fascimile), which may lessen the need for this PEP. hg doesn't support this workflow, which may strengthen the need for this PEP. Saying "I've been using hg for years and I don't know whether it supports this IMPORTANT THING" is not a particularly compelling argument.
Comments like this make me feel like I didn’t explain myself very well in the PEP. While I do think that supporting this workflow via an extension is worse than supporting it in core, this isn’t why this PEP exists. The current workflow for contributing is painful, for the repositories this is talking about if I’m a non-comitter I have to email patches to a particular closed mailing list and then sit around and wait. The Pull Request based workflow is *massively* better than uploading/emailing patches around. So the question then becomes, if we're going to move to a PR based workflow how do we do it? PEP 474 says that we should install some software that works with Mercurial and supports Pull Requests. Another thread suggested that we should just use to bitbucket whicih also supports Mercurial and use that. This PEP says that git and Github have the popular vote, which is extremely compelling as a feature because: * Popularity is the one thing you can't replicate featurewise. This feature or that you can find a way to work something like it out. * With popularity comes the fact that people will already know the tooling involved and how to use it. This means that a new contributor whom already knows git will spend less time learning our specific toolset and more time being able to meaningfully contribute. * With popularity also comes transferability, If I don't currently know a VCS tool and I learn git (and github) for the sake of these repositories then that knowledge will directly transfer to greater than 60% of the top 100 projects on PyPI. * With popularity also comes an increased amount of people writing about it, asking questions, etc. This means that if I have a problem while using this tool I am more likely to find someone who already had this problem and they are more likely to have found someone to help them solve it than I am with a less popular tool. I'm not sure why people seem to try and focus on the fact that with this extension or that you can make Mercurial replicate git better when that isn't really the major point of the PEP at all. If Mercurial and git were neck in neck in terms of mindshare and bitbucket and Github were as well then this PEP probably wouldn't exist because on a techincal level I think the two tools are probably close enough. However when most of the world is using one tool and you're using another, that does represent a barrier to entry. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Nov 30, 2014, at 11:30 AM, Donald Stufft <donald@stufft.io> wrote:
Comments like this make me feel like I didn’t explain myself very well in the PEP.
It’s been pointed out to me that Mercurial bookmarks have been core since 1.8 and since I felt like the technical arguments were really secondary to the social arguments I’ve gone ahead and removed that section from the PEP. I’ve also added the read only Mercurial repository on hg.python.org <http://hg.python.org/> See changes at https://hg.python.org/peps/rev/d31fe28e2766 --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 11/30/2014 08:30 AM, Donald Stufft wrote:
On Nov 30, 2014, at 2:08 AM, Larry Hastings <larry@hastings.org <mailto:larry@hastings.org>> wrote:
On 11/29/2014 04:37 PM, Donald Stufft wrote:
On Nov 29, 2014, at 7:15 PM, Alex Gaynor<alex.gaynor@gmail.com> wrote:
Despite being a regular hg user for years, I have no idea how to create a local-only branch, or a branch which is pushed to a remote (to use the git term). I also don’t know how to do this.
Instead of collectively scratching your heads, could one of you guys do the research and figure out whether or not hg supports this workflow? One of the following two things must be true:
1. hg supports this workflow (or a reasonable fascimile), which may lessen the need for this PEP. 2. hg doesn't support this workflow, which may strengthen the need for this PEP.
Saying "I've been using hg for years and I don't know whether it supports this IMPORTANT THING" is not a particularly compelling argument.
Comments like this make me feel like I didn’t explain myself very well in the PEP.
While I do think that supporting this workflow via an extension is worse than supporting it in core,
I was about to point that bookmark are no longer an extension for more than 3 years, but someone was faster than me. But I would like to reply a bit more on this extension FUD. There is three kind of Mercurial feature: 1) The one in Mercurial core 2) The one in official extension 3) the one in third party extension (1) and (2) Have the -same- level of support and stability. All of them are part of the same repo and same test suite, offer the same backward compatibility promise and are installed as part of the same package. Official extensions are usually not in core for various reasons: 1) It is exotic feature (eg: communication bugzilla extension) 2) Nobody did the work to move it into core (unfortunatly) (eg: progress bar extension) (similar situation: how long did it took to get pip shipped with python?) 3) We think it is a important feature but we are not happy with the current UX and would like somethign better before moving it into core (eg: histedit) Not using official extensions because they are extensions is similar to not use the python standard library because "They are modules, not part of the core language"
this isn’t why this PEP exists. The current workflow for contributing is painful, for the repositories this is talking about if I’m a non-comitter I have to email patches to a particular closed mailing list and then sit around and wait.
Your workflow issue does not seems to be particularly tied to the tool (Mercurial) itself but more to (1) very partial usage of the tool (2) current project workflow. It would not be hard to (1) use the tools in a less painful way (2) rethink project workflow to reduce the pain. This seems orthogonal to changing the tool.
The Pull Request based workflow is *massively* better than uploading/emailing patches around. So the question then becomes, if we're going to move to a PR based workflow how do we do it? PEP 474 says that we should install some software that works with Mercurial and supports Pull Requests. Another thread suggested that we should just use to bitbucket which also supports Mercurial and use that.
(note: Yes Manual upload of patch is terrible and tracking email by patch is terrible too. But github is fairly bad at doing pull request. It emphasis the final result instead of the actual pull request content. Encouraging massive patches and making the history muddier.)
This PEP says that git and Github have the popular vote, which is extremely compelling as a feature because:
Sure, Git//Github is more popular than Mercurial nowaday. But can I point to the irony of Python using the "more popular" argument? If most of people here would have seen this definitive argument, they would been currently writing Java code instead of discussing Python on this list. -- Pierre-Yves David
On Sun, 30 Nov 2014 00:15:55 +0000 (UTC) Alex Gaynor <alex.gaynor@gmail.com> wrote:
The seconds experience I have is that of Django's migration to git and github. For a long time we were on SVN, and we were very resistant to moving to DVCS in general, and github in particular. Multiple times I said that I didn't see how exporting a patch and uploading it to trac was more difficult than sending a pull request. That was very wrong on my part.
My primary observation is not about new contributors though, it's actually about the behavior of core developers. Before we were on github, it was fairly rare for core developers to ask for reviews for anything besides *gigantic* patches, we'd mostly just commit stuff to trunk. Since the switch to github, I've seen that core developers are *far* more likely to ask for reviews of their work before merging.
I don't know anything about Django's old SVN setup, but our code review tool (Rietveld) is actually quite good at that, and IMHO slightly better than github, since it will only send an e-mail at the final submission - by contrast, each individual comment you leave on github fires a separate notification, which can feel like spam. Our main problem for reviews these days is the lack of core developer time. For example, Serhiy often asks for reviews (he has many patches pending), which I personally don't have a lot of time to provide. Regards Antoine.
On Nov 29, 2014, at 7:43 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Sun, 30 Nov 2014 00:15:55 +0000 (UTC) Alex Gaynor <alex.gaynor@gmail.com> wrote:
The seconds experience I have is that of Django's migration to git and github. For a long time we were on SVN, and we were very resistant to moving to DVCS in general, and github in particular. Multiple times I said that I didn't see how exporting a patch and uploading it to trac was more difficult than sending a pull request. That was very wrong on my part.
My primary observation is not about new contributors though, it's actually about the behavior of core developers. Before we were on github, it was fairly rare for core developers to ask for reviews for anything besides *gigantic* patches, we'd mostly just commit stuff to trunk. Since the switch to github, I've seen that core developers are *far* more likely to ask for reviews of their work before merging.
I don't know anything about Django's old SVN setup, but our code review tool (Rietveld) is actually quite good at that, and IMHO slightly better than github, since it will only send an e-mail at the final submission - by contrast, each individual comment you leave on github fires a separate notification, which can feel like spam.
Our main problem for reviews these days is the lack of core developer time. For example, Serhiy often asks for reviews (he has many patches pending), which I personally don't have a lot of time to provide.
I think one of the issues with Reitveld isn’t related to Reitveld itself at all, it’s all the *other* stuff you have to do to get a patch into Reitveld to allow someone to review it at all. Generating a patch and uploading it to Roundup is a pain and it’s far easier to just commit things directly to default. As far as Reitveld vs Github itself goes for reviews, I don’t personally agree. Sending a single notification vs a notification per comment is going to be a very subjective point of view, I like more notifications for the simple fact that if I'm actively working on a PR while someone is reviewing it I can fix the issues as quickly as they are finding them. However if you move past that one thing, the Github PR review has a number of benefits over Reitveld itself. Since the repositories this PEP currently deals with are largely documentation-esque repositories the "Prose" diff is incredibly useful for viewing a content-based diff instead of a diff designed more for software where you don't normally reflow lines as much. In addition the Github diff viewer also makes it trivial to expand the context of the diff so you can see more of the surrounding file, even allowing you to expand it to the entire file. This is super useful when the automatic amount of context can't really give you enough information for reviewing the change or not. Another difference is in how the review comments are presented. With Reitveld the inline comments go away for each patchset you upload, regardless of if you've addressed the concerns in that comment or not. They do get bubbled up into the overall "Messages" view, however this has the opposite problem where it gives you all of the messages regardless of if they are still a problem with the latest code or not. In contrast the Github PR will hide old comments when they lines they addressed have changed, but still make it easy to see the old comments and the context in which they were made. There's also the UI itself, it's somewhat dated and I think it's not entirely unobvious that it suffers from something a lot of OSS projects suffer from in that it is a "Developer" made UI. I don't think it's a big secret that developers tend to make UIs that are not as nice as proper UX folks make, but OSS has historically had a problem attracting that kind of talent (and is often openly hostile towards it). Finally there's the transferability of knowledge at play too. I likely would not review someone else's patch unless I felt strongly about it on Reitveld largely because I don't know Reitveld very well and I'd rather spend the time that I'd need to learn to use it effectively working on other projects where I already know the entire toolchain well. This is one of the larger points in this PEP that the benefits to using popular tooling is that not only does people's existing knowledge transfer easily into your project, but the time they spend learning your tooling also transfers easily outside of this project. This makes it much more attractive to learn the tooling since the hypothetical person would be able to take that knowledge and apply it elsewhere. It is my experience, and this entirely ancedotal, that it's far easier to get reviews from non-committers and committers alike on projects which are hosted on Github than it is to get reviews on projects which are not hosted there. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 29 Nov 20:13, Donald Stufft wrote:
I think one of the issues with Reitveld isn’t related to Reitveld itself at all, it’s all the *other* stuff you have to do to get a patch into Reitveld to allow someone to review it at all. Generating a patch and uploading it to Roundup is a pain and it’s far easier to just commit things directly to default.
For the record, there is a mercurial extension to ease upload of patch to Rietveld: https://bitbucket.org/nicoe/hgreview/overview -- Cédric Krier - B2CK SPRL Email/Jabber: cedric.krier@b2ck.com Tel: +32 472 54 46 59 Website: http://www.b2ck.com/
On Sat Nov 29 2014 at 7:16:34 PM Alex Gaynor <alex.gaynor@gmail.com> wrote:
Donald Stufft <donald <at> stufft.io> writes:
[words words words]
I strongly support this PEP. I'd like to share two pieces of information. Both of these are personal anecdotes:
For the past several years, I've been a contributor on two major projects using mercurial, CPython and PyPy. PyPy has a strong culture of in-repo branching, basically all contributors regularly make branches in the main repo for their work, and we're very free in giving people commit rights, so almost everyone working on PyPy in any way has this level of access. This workflow works ok. I don't love it as much as git, but it's fine, it's not an impediment to my work.
By contrast, CPython does not have this type of workflow, there are almost no in-tree branches besides the 2.7, 3.4, etc. ones. Despite being a regular hg user for years, I have no idea how to create a local-only branch, or a branch which is pushed to a remote (to use the git term). I also don't generally commit my own work to CPython, even though I have push privledges, because I prefer to *always* get code review on my work. As a result, I use a git mirror of CPython to do all my work, and generate patches from that.
The conclusion I draw from this is that hg's workflow is probably fine if you're a committer on the project, or don't ever need to maintain multiple patches concurrently (and thus can just leave everything uncommitted in the repo). However, the hg workflow seems extremely defficient at non-committer contributors.
One way to come close to that using hg is to have your own clone that you never push to hg.python.org/cpython (e.g. cloning the Bitbucket clone of cpython or hosting on hg.python.org a personal clone). You can then specify the repo and branch on the issue tracker to generate your patch: https://docs.python.org/devguide/triaging.html#mercurial-repository . After that it's just like any other patch workflow for core devs. It's not quite as nice as maybe using named branches where you can just do a final hg merge/commit to get your changes committed, but if you're not going to commit your branches then you might as well get the automatic patch generation perk in the issue tracker rather than using git (unless there is some other git feature you use that you can't get in hg).
The seconds experience I have is that of Django's migration to git and github. For a long time we were on SVN, and we were very resistant to moving to DVCS in general, and github in particular. Multiple times I said that I didn't see how exporting a patch and uploading it to trac was more difficult than sending a pull request. That was very wrong on my part.
My primary observation is not about new contributors though, it's actually about the behavior of core developers. Before we were on github, it was fairly rare for core developers to ask for reviews for anything besides *gigantic* patches, we'd mostly just commit stuff to trunk. Since the switch to github, I've seen that core developers are *far* more likely to ask for reviews of their work before merging.
Why specifically? Did you have a web UI for reviewing patches previously? Do you have CI set up for patches now and didn't before? What features did you specifically gain from the switch to GitHub that you didn't have before? IOW was it the "magic" of GitHub or some technical solution that you got as part of the GitHub package and thus could theoretically be replicated on python.org? -Brett
On Sun Nov 30 2014 at 10:28:50 AM Brett Cannon <brett@python.org> wrote:
Why specifically? Did you have a web UI for reviewing patches previously? Do you have CI set up for patches now and didn't before? What features did you specifically gain from the switch to GitHub that you didn't have before? IOW was it the "magic" of GitHub or some technical solution that you got as part of the GitHub package and thus could theoretically be replicated on python.org?
-Brett
Previously someone looking for a review (read: any non-committer) would export a diff from their VCS, upload it as a patch to trac, and then reviewers could leave comments as trac comments. CPython's present process is a clear improvement, insofar as Rietveld allows inlining commenting, but it is otherwise basically the same. By contrast, the Github process does not require a patch author to leave their workflow, they simply "git push" to update a patch. We now also have CI for PRs, but that's a recent addition. It's not magic, it's a good UX :-) Alex
On Nov 30, 2014, at 11:28 AM, Brett Cannon <brett@python.org> wrote:
On Sat Nov 29 2014 at 7:16:34 PM Alex Gaynor <alex.gaynor@gmail.com <mailto:alex.gaynor@gmail.com>> wrote: Donald Stufft <donald <at> stufft.io <http://stufft.io/>> writes:
[words words words]
I strongly support this PEP. I'd like to share two pieces of information. Both of these are personal anecdotes:
For the past several years, I've been a contributor on two major projects using mercurial, CPython and PyPy. PyPy has a strong culture of in-repo branching, basically all contributors regularly make branches in the main repo for their work, and we're very free in giving people commit rights, so almost everyone working on PyPy in any way has this level of access. This workflow works ok. I don't love it as much as git, but it's fine, it's not an impediment to my work.
By contrast, CPython does not have this type of workflow, there are almost no in-tree branches besides the 2.7, 3.4, etc. ones. Despite being a regular hg user for years, I have no idea how to create a local-only branch, or a branch which is pushed to a remote (to use the git term). I also don't generally commit my own work to CPython, even though I have push privledges, because I prefer to *always* get code review on my work. As a result, I use a git mirror of CPython to do all my work, and generate patches from that.
The conclusion I draw from this is that hg's workflow is probably fine if you're a committer on the project, or don't ever need to maintain multiple patches concurrently (and thus can just leave everything uncommitted in the repo). However, the hg workflow seems extremely defficient at non-committer contributors.
One way to come close to that using hg is to have your own clone that you never push to hg.python.org/cpython <http://hg.python.org/cpython> (e.g. cloning the Bitbucket clone of cpython or hosting on hg.python.org <http://hg.python.org/> a personal clone). You can then specify the repo and branch on the issue tracker to generate your patch: https://docs.python.org/devguide/triaging.html#mercurial-repository <https://docs.python.org/devguide/triaging.html#mercurial-repository> . After that it's just like any other patch workflow for core devs. It's not quite as nice as maybe using named branches where you can just do a final hg merge/commit to get your changes committed, but if you're not going to commit your branches then you might as well get the automatic patch generation perk in the issue tracker rather than using git (unless there is some other git feature you use that you can't get in hg).
This doesn’t really work in a Pull Request workflow though I think is the point. Uploading patches is it’s own kind of terrible but yes if you’re just uploading patches you can probably do that. I don’t make branches in Mercurial because i’m afraid I’m going to push a permanent branch to hg.python.org <http://hg.python.org/> and screw something up.
The seconds experience I have is that of Django's migration to git and github. For a long time we were on SVN, and we were very resistant to moving to DVCS in general, and github in particular. Multiple times I said that I didn't see how exporting a patch and uploading it to trac was more difficult than sending a pull request. That was very wrong on my part.
My primary observation is not about new contributors though, it's actually about the behavior of core developers. Before we were on github, it was fairly rare for core developers to ask for reviews for anything besides *gigantic* patches, we'd mostly just commit stuff to trunk. Since the switch to github, I've seen that core developers are *far* more likely to ask for reviews of their work before merging.
Why specifically? Did you have a web UI for reviewing patches previously? Do you have CI set up for patches now and didn't before? What features did you specifically gain from the switch to GitHub that you didn't have before? IOW was it the "magic" of GitHub or some technical solution that you got as part of the GitHub package and thus could theoretically be replicated on python.org <http://python.org/>?
In most of the cases there was some form of a web UI for reviewing patches. I don’t believe there was a CI setup for patches previously and most projects do end up with some sort of CI on Github since Travis makes it particularly easy. Honestly though, I feel like the main reason for it is just that Github’s solution is easy to use, it works with the VCS which is great because that’s what I’m using already, and it’s highly polished. OSS has a long history of having rather poor UX, and poor UX tends to mean that people would rather not use some particular thing unless they *have* to. That is the way I look at Rietveld to be honest. I get frustrated trying to use it so I'd rather just not use it where I feel I can get away with it. It's not really enough to look at a feature matrix and then go "yup we something in all the checkboxes" which it seems that OSS software likes to try and do when comparing themselves to a well done, but non-OSS solution. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Sun, Nov 30, 2014, at 11:45, Donald Stufft wrote:
On Nov 30, 2014, at 11:28 AM, Brett Cannon <brett@python.org> wrote:
On Sat Nov 29 2014 at 7:16:34 PM Alex Gaynor <alex.gaynor@gmail.com <mailto:alex.gaynor@gmail.com>> wrote: Donald Stufft <donald <at> stufft.io <http://stufft.io/>> writes:
[words words words]
I strongly support this PEP. I'd like to share two pieces of information. Both of these are personal anecdotes:
For the past several years, I've been a contributor on two major projects using mercurial, CPython and PyPy. PyPy has a strong culture of in-repo branching, basically all contributors regularly make branches in the main repo for their work, and we're very free in giving people commit rights, so almost everyone working on PyPy in any way has this level of access. This workflow works ok. I don't love it as much as git, but it's fine, it's not an impediment to my work.
By contrast, CPython does not have this type of workflow, there are almost no in-tree branches besides the 2.7, 3.4, etc. ones. Despite being a regular hg user for years, I have no idea how to create a local-only branch, or a branch which is pushed to a remote (to use the git term). I also don't generally commit my own work to CPython, even though I have push privledges, because I prefer to *always* get code review on my work. As a result, I use a git mirror of CPython to do all my work, and generate patches from that.
The conclusion I draw from this is that hg's workflow is probably fine if you're a committer on the project, or don't ever need to maintain multiple patches concurrently (and thus can just leave everything uncommitted in the repo). However, the hg workflow seems extremely defficient at non-committer contributors.
One way to come close to that using hg is to have your own clone that you never push to hg.python.org/cpython <http://hg.python.org/cpython> (e.g. cloning the Bitbucket clone of cpython or hosting on hg.python.org <http://hg.python.org/> a personal clone). You can then specify the repo and branch on the issue tracker to generate your patch: https://docs.python.org/devguide/triaging.html#mercurial-repository <https://docs.python.org/devguide/triaging.html#mercurial-repository> . After that it's just like any other patch workflow for core devs. It's not quite as nice as maybe using named branches where you can just do a final hg merge/commit to get your changes committed, but if you're not going to commit your branches then you might as well get the automatic patch generation perk in the issue tracker rather than using git (unless there is some other git feature you use that you can't get in hg).
This doesn’t really work in a Pull Request workflow though I think is the point. Uploading patches is it’s own kind of terrible but yes if you’re just uploading patches you can probably do that. I don’t make branches in Mercurial because i’m afraid I’m going to push a permanent branch to hg.python.org <http://hg.python.org/> and screw something up.
Don't worry; we have a hook for that.
On 11/30/2014 08:45 AM, Donald Stufft wrote:
I don’t make branches in Mercurial because i’m afraid I’m going to push a permanent branch to hg.python.org <http://hg.python.org> and screw something up.
There is no need to be afraid there, Mercurial is not going to let you push new head/branch unless you explicitly use `hg push --force`. I you are really paranoid about this, you can configure your Mercurial to make all new commit as secret (no pushable) and explicitly make commit ready to push as such. This can be achieved by adding [phases] new-commit=secret See `hg help phases` for details. -- Pierre-Yves David
On Nov 30, 2014, at 8:11 PM, Pierre-Yves David <pierre-yves.david@ens-lyon.org> wrote:
On 11/30/2014 08:45 AM, Donald Stufft wrote:
I don’t make branches in Mercurial because i’m afraid I’m going to push a permanent branch to hg.python.org <http://hg.python.org> and screw something up.
There is no need to be afraid there, Mercurial is not going to let you push new head/branch unless you explicitly use `hg push --force`.
I you are really paranoid about this, you can configure your Mercurial to make all new commit as secret (no pushable) and explicitly make commit ready to push as such. This can be achieved by adding
[phases] new-commit=secret
See `hg help phases` for details.
Yea Benjamin mentioned that the hg.python.org repositories have commit hooks to prevent that from happening too. To be clear the fact I don’t really know Mercurial very well isn’t what I think is a compelling argument for not using Mercurial. It’s mostly a tangent to this PEP which is primarily focused on the “network effects” of using a more popular tool. The technical benefits mostly come from Github generally being a higher quality product than it’s competitors, both FOSS and not. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Sun, Nov 30, 2014 at 6:25 PM, Donald Stufft <donald@stufft.io> wrote:
The technical benefits mostly come from Github generally being a higher quality product than it’s competitors, both FOSS and not.
Here's a solution to allow contribution via PR while not requiring anything to switch VCS or hosting: 1. Set up mirrors of a desired repo on any hosting providers we choose. 2. Set up a webhook for PRs that automatically creates/re-uses a tracker ticket with the diff from the PR. The workflow does not change for the committer, but it gets easier to contribute. I did something like this for juju (https://github.com/juju/juju) when we switched to github, weren't satisfied with their code review tool, and switched to something else. We have a web hook that automatically creates a review request for new PRs and updates the review request when the PR gets updated. -eric
On Nov 30, 2014, at 8:41 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Sun, Nov 30, 2014 at 6:25 PM, Donald Stufft <donald@stufft.io> wrote:
The technical benefits mostly come from Github generally being a higher quality product than it’s competitors, both FOSS and not.
Here's a solution to allow contribution via PR while not requiring anything to switch VCS or hosting:
1. Set up mirrors of a desired repo on any hosting providers we choose. 2. Set up a webhook for PRs that automatically creates/re-uses a tracker ticket with the diff from the PR.
The workflow does not change for the committer, but it gets easier to contribute.
I did something like this for juju (https://github.com/juju/juju) when we switched to github, weren't satisfied with their code review tool, and switched to something else. We have a web hook that automatically creates a review request for new PRs and updates the review request when the PR gets updated.
-eric
Yea this is essentially what I meant. We already have “unofficial” mirrors for PEPs and CPython itself on Github that are updated a few times a day. It wouldn’t be very difficult I think to make them official mirrors and update them immediately after a push. Then just some integration with Roundup would enable people to send PRs on Github. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Sun, Nov 30, 2014 at 6:44 PM, Donald Stufft <donald@stufft.io> wrote:
Yea this is essentially what I meant. We already have “unofficial” mirrors for PEPs and CPython itself on Github that are updated a few times a day. It wouldn’t be very difficult I think to make them official mirrors and update them immediately after a push.
Then just some integration with Roundup would enable people to send PRs on Github.
Exactly. In my mind this eliminates almost all the controversy in this discussion. The question of switching hosting provider or DVCS becomes something to discuss separately (and probably just drop since at that point it doesn't improve much over the status quo). Of course, it does not address the more pressing concern of how to get contributions landed more quickly/steadily. :( However, that is largely what Nick addresses in PEP 462. :) -eric
On Nov 29, 2014, at 6:27 PM, Donald Stufft <donald@stufft.io> wrote:
[lots of words]
Just FYI, I’ve pushed an update to the PEP. Nothing major just some grammatical fixes and such. The revision is here: https://hg.python.org/peps/rev/6c6947dbd13f For whatever it's worth, the person who submitted that patch used Github's online editor to submit them and made a PR to my personal PEP repository where I work on my PEPs (https://github.com/dstufft/peps/pull/3). If someone wanted to see some of the features in action that is a nice PEP to look at. In particular if you hit "Files Changed" then beside the view button is an icon that looks like a piece of paper and when you over over it it'll say "Display the Rich Diff". Clicking on that you'll see the diff of the rendered output which lets you ignore things which were just reflowing the paragraphs and such. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 30 Nov 2014 09:28, "Donald Stufft" <donald@stufft.io> wrote:
As promised in the "Move selected documentation repos to PSF BitBucket account?" thread I've written up a PEP for moving selected repositories
from
hg.python.org to Github.
You can see this PEP online at: https://www.python.org/dev/peps/pep-0481/
I've also reproduced the PEP below for inline discussion.
Given that hg.python.org isn't going anywhere, you could also use hg-git to maintain read-only mirrors at the existing URLs and minimise any breakage (as well as ensuring a full historical copy remains available on PSF infrastructure). Then the only change needed would be to set up appropriate GitHub web hooks to replace anything previously based on a commit hook rather than periodic polling. The PEP should also cover providing clear instructions for setting up git-remote-hg with the remaining Mercurial repos (most notably CPython), as well as documenting a supported workflow for generating patches based on the existing CPython GitHub mirror. Beyond that, GitHub is indeed the most expedient option. My two main reasons for objecting to taking the expedient path are: 1. I strongly believe that the long term sustainability of the overall open source community requires the availability and use of open source infrastructure. While I admire the ingenuity of the "free-as-in-beer" model for proprietary software companies fending off open source competition, I still know a proprietary platform play when I see one (and so do venture capitalists looking to extract monopoly rents from the industry in the future). (So yes, I regret relenting on this principle in previously suggesting the interim use of another proprietary hosted service) 2. I also feel that this proposal is far too cavalier in not even discussing the possibility of helping out the Mercurial team to resolve their documentation and usability issues rather than just yelling at them "your tool isn't popular enough for us, and we find certain aspects of it too hard to use, so we're switching to something else rather than working with you to address our concerns". We consider the Mercurial team a significant enough part of the Python ecosystem that Matt was one of the folks specifically invited to the 2014 language summit to discuss their concerns around the Python 3 transition. Yet we'd prefer to switch to something else entirely rather than organising a sprint with them at PyCon to help ensure that our existing Mercurial based infrastructure is approachable for git & GitHub users? (And yes, I consider some of the core Mercurial devs to be friends, so this isn't an entirely abstract concern for me) Given my proposal to use BitBucket as a near term solution for enabling pull request based workflows, it's clear I consider the second argument the more significant of the two. However, if others consider some short term convenience that may or may not attract additional contributors more important than supporting the broader Python and open source communities (an argument I'm more used to hearing in the ruthlessly commercial environment of Red Hat, rather than in upstream contexts that tend to be less worried about "efficiency at any cost"), I'm not going to expend energy trying to prevent a change I disagree with on principle, but will instead work to eliminate (or significantly reduce) the current expedience argument in GitHub's favour. As a result, I'm -0 on the PEP, rather than -1 (and will try to stay out of further discussions). Given this proposal, I'm planning to refocus PEPs 474 & 462 specifically on resolving the CPython core workflow issues, since that will require infrastructure customisation regardless, and heavy customisation of GitHub based infrastructure requires opting in to the use of the GitHub specific APIs that create platform lockin. (Note that the argument in PEP 481 about saving overall infrastructure work is likely spurious - the vast majority of that work will be in addressing the complex CPython workflow requirements, and moving some support repos to GitHub does little to alleviate that) If folks decide they want to migrate the ancillary repos back from GitHub after that other infrastructure work is done, so be it, but if they don't, that's OK too. We're already running heterogeneous infrastructure across multiple services (especially if you also take PyPA into account), so having additional support repos externally hosted isn't that big a deal from a purely practical perspective. Regards, Nick.
-----------------------
Abstract ========
This PEP proposes migrating to Git and Github for certain supporting repositories (such as the repository for Python Enhancement Proposals) in
that is more accessible to new contributors, and easier to manage for core developers. This is offered as an alternative to PEP 474 which aims to achieve the same overall benefits but while continuing to use the Mercurial DVCS and without relying on a commerical entity.
In particular this PEP proposes changes to the following repositories:
* https://hg.python.org/devguide/ * https://hg.python.org/devinabox/ * https://hg.python.org/peps/
This PEP does not propose any changes to the core development workflow for CPython itself.
Rationale =========
As PEP 474 mentions, there are currently a number of repositories hosted on hg.python.org which are not directly used for the development of CPython but instead are supporting or ancillary repositories. These supporting repositories do not typically have complex workflows or often branches at all other
primary integration branch. This simplicity makes them very good targets for the "Pull Request" workflow that is commonly found on sites like Github.
However where PEP 474 wants to continue to use Mercurial and wishes to use an OSS and self-hosted and therefore restricts itself to only those solutions this PEP expands the scope of that to include migrating to Git and using Github.
The existing method of contributing to these repositories generally includes generating a patch and either uploading them to bugs.python.org or emailing them to peps@python.org. This process is unfriendly towards non-comitter contributors as well as making the process harder than it needs to be for comitters to accept the patches sent by users. In addition to the benefits in the pull request workflow itself, this style of workflow also enables non techincal contributors, especially those who do not know their way around the DVCS of choice, to contribute using the web based editor. On the committer side the Pull Requests enable them to tell, before merging, whether or not a particular Pull Request will break anything. It also enables them to do a simple "push button" merge which does not require them to check out the changes locally. Another such feature that is useful in particular for docs, is the ability to view a "prose" diff. This Github specific feature enables a committer to view a diff of the rendered output which will hide things
reformatting a paragraph and show you what the actual "meat" of the change actually is.
Why Git? --------
Looking at the variety of DVCS which are available today it becomes fairly clear that git has gotten the vast mindshare of people who are currently using it. The Open Hub (Previously Ohloh) statistics [#openhub-stats]_ show that currently 37% of the repositories Open Hub is indexing is using git which is second only to SVN (which has 48%) while Mercurial has just 2% of the indexed repositories (beating only bazaar which has 1%). In additon to the Open Hub statistics a look at the top 100 projects on PyPI (ordered by total download counts) shows us that within the Python space itself there is a majority of projects using git:
=== ========= ========== ====== === ==== Git Mercurial Subversion Bazaar CVS None === ========= ========== ====== === ==== 62 22 7 4 1 1 === ========= ========== ====== === ====
Chosing a DVCS which has the larger mindshare will make it more likely
particular person who has experience with DVCS at all will be able to meaningfully use the DVCS that we have chosen without having to learn a new tool.
In addition to simply making it more likely that any individual will already know how to use git, the number of projects and people using it means
resources for learning the tool are likely to be more fully fleshed out and when you run into problems the liklihood that someone else had that
and posted a question and recieved an answer is also far likelier.
Thirdly by using a more popular tool you also increase your options for tooling *around* the DVCS itself. Looking at the various options for hosting repositories it's extremely rare to find a hosting solution (whether OSS or commerical) that supports Mercurial but does not support Git, on the flip side there are a number of tools which support Git but do not support Mercurial. Therefore the popularity of git increases the flexibility of our options going into the future for what toolchain these projects use.
Also by moving to the more popular DVCS we increase the likelhood that the knowledge that the person has learned in contributing to these support repositories will transfer to projects outside of the immediate CPython
such as to the larger Python community which is primarily using Git hosted on Github.
In previous years there was concern about how well supported git was on Windows in comparison to Mercurial. However git has grown to support Windows as a first class citizen. In addition to that, for Windows users who are not well aquanted with the Windows command line there are GUI options as well.
On a techincal level git and Mercurial are fairly similar, however the git branching model is signifcantly better than Mercurial "Named Branches" for non-comitter contributors. Mercurial does have a "Bookmarks" extension however this isn't quite as good as git's branching model. All bookmarks live in
same namespace so it requires individual users to ensure that they namespace the branchnames themselves lest the risk collision. It also is an extension which requires new users to first discover they need an extension at all and then figure out what they need to do in order to enable that extension. Since it is an extension it also means that in general support for them outside of Mercurial core is going to be less than 100% in comparison to git where
feature is built in and core to using git at all. Finally users who are not used to Mercurial are unlikely to discover bookmarks on their own, instead they will likely attempt to use Mercurial's "Named Branches" which, given the fact they live "forever", are not often what a project wants their contributors to use.
Why Github? -----------
There are a number of software projects or web services which offer functionality similar to that of Github. These range from commerical web services such as a Bitbucket to self-hosted OSS solutions such as Kallithea or Gitlab. This PEP proposes that we move these repositories to Github.
There are two primary reasons for selecting Github: Popularity and Quality/Polish.
Github is currently the most popular hosted repository hosting according to Alexa where it currently has a global rank of 121. Much like for Git itself by choosing the most popular tool we gain benefits in increasing the
that a new contributor will have already experienced the toolchain, the quality and availablity of the help, more and better tooling being built around it, and the knowledge transfer to other projects. A look again at the top 100
by download counts on PyPI shows the following hosting locations:
====== ========= =========== ========= =========== ========== GitHub BitBucket Google Code Launchpad SourceForge Other/Self ====== ========= =========== ========= =========== ========== 62 18 6 4 3 7 ====== ========= =========== ========= =========== ==========
In addition to all of those reasons, Github also has the benefit that while many of the options have similar features when you look at them in a feature matrix the Github version of each of those features tend to work better and be far more polished. This is hard to quantify objectively however it is a fairly common sentiment if you go around and ask people who are using these services often.
Finally a reason to choose a web service at all over something that is self-hosted is to be able to more efficiently use volunteer time and donated resources. Every additional service hosted on the PSF infrastruture by the PSF infrastructure team further spreads out the amount of time that the volunteers on that team have to spend and uses some chunk of resources
could potentionally be used for something where there is no free or affordable hosted solution available.
One concern that people do have with using a hosted service is that there is a lack of control and that at some point in the future the service may no longer be suitable. It is the opinion of this PEP that Github does not currently and has not in the past engaged in any attempts to lock people into their
and that if at some point in the future Github is no longer suitable for one reason or another than at that point we can look at migrating away from Github onto a different solution. In other words, we'll cross that bridge if and when we come to it.
Example: Scientific Python --------------------------
One of the key ideas behind the move to both git and Github is that a feature of a DVCS, the repository hosting, and the workflow used is the social network and size of the community using said tools. We can see this is true by looking at an example from a sub-community of the Python community: The Scientific Python community. They have already migrated most of the key pieces of the SciPy stack onto Github using the Pull Request based workflow starting with IPython and as more projects moved over it became a natural default for new projects.
They claim to have seen a great benefit from this move, where it enables casual contributors to easily move between different projects within their sub-community without having to learn a special, bespoke workflow and a different toolchain for each project. They've found that when people can use their limited time on actually contributing instead of learning the different tools and workflows that not only do they contribute more to one project,
they also expand out and contribute to other projects. This move is also attributed to making it commonplace for members of that community to go so far as publishing their research and educational materials on Github as well.
This showcases the real power behind moving to a highly popular toolchain and workflow, as each variance introduces yet another hurdle for new and casual contributors to get past and it makes the time spent learning that workflow less reusable with other projects.
Migration =========
Through the use of hg-git [#hg-git]_ we can easily convert a Mercurial repository to a Git repository by simply pushing the Mercurial repository to the Git repository. People who wish to continue to use Mercurual locally can then use hg-git going into the future using the new Github URL, however
will need to re-clone their repositories as using Git as the server seems to trigger a one time change of the changeset ids.
As none of the selected repositories have any tags, branches, or bookmarks other than the ``default`` branch the migration will simply map the ``default`` branch in Mercurial to the ``master`` branch in git.
In addition since none of the selected projects have any great need of a complex bug tracker, they will also migrate their issue handling to using
GitHub issues.
In addition to the migration of the repository hosting itself there are a number of locations for each particular repository which will require updating. The bulk of these will simply be changing commands from the hg equivilant to the git equivilant.
In particular this will include:
* Updating www.python.org to generate PEPs using a git clone and link to Github. * Updating docs.python.org to pull from Github instead of hg.python.org for the devguide. * Enabling the ability to send an email to python-checkins@python.org for each push. * Enabling the ability to send an IRC message to #python-dev on Freenode for each push. * Migrate any issues for these projects to their respective bug tracker on Github.
This will restore these repositories to similar functionality as they currently have. In addition to this the migration will also include enabling testing for each pull request using Travis CI [#travisci]_ where possible to ensure
a way than the like that any that the problem project the the likelhood projects that platform that they the that
a new PR does not break the ability to render the documentation or PEPs.
User Access ===========
Moving to Github would involve adding an additional user account that will need to be managed, however it also offers finer grained control, allowing the ability to grant someone access to only one particular repository instead of the coarser grained ACLs available on hg.python.org.
References ==========
.. [#openhub-stats] `Open Hub Statistics < https://www.openhub.net/repositories/compare>` .. [#hg-git] `hg-git <https://hg-git.github.io/>` .. [#travisci] `Travis CI <https://travis-ci.org/>`
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
On Nov 29, 2014, at 8:12 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 30 Nov 2014 09:28, "Donald Stufft" <donald@stufft.io <mailto:donald@stufft.io>> wrote:
As promised in the "Move selected documentation repos to PSF BitBucket account?" thread I've written up a PEP for moving selected repositories from hg.python.org <http://hg.python.org/> to Github.
You can see this PEP online at: https://www.python.org/dev/peps/pep-0481/ <https://www.python.org/dev/peps/pep-0481/>
I've also reproduced the PEP below for inline discussion.
Given that hg.python.org <http://hg.python.org/> isn't going anywhere, you could also use hg-git to maintain read-only mirrors at the existing URLs and minimise any breakage (as well as ensuring a full historical copy remains available on PSF infrastructure). Then the only change needed would be to set up appropriate GitHub web hooks to replace anything previously based on a commit hook rather than periodic polling.
Ah yes, I meant to include that and just forgot to do it when I went to test hg-git to see how well it worked and whether I got different commit hashes on different machines. I also thought about adding a git.python.org <http://git.python.org/> which just acted as a read-only mirror of what was on Github, but I don’t know if that’s actually generally useful or not.
The PEP should also cover providing clear instructions for setting up git-remote-hg with the remaining Mercurial repos (most notably CPython), as well as documenting a supported workflow for generating patches based on the existing CPython GitHub mirror.
I can add this. I’ve never actually tried using git-remote-hg with CPython itself because I’ve made it segfault on other Mercurial repositories and I never figured out why so I just generally fight my way through using Mercurial on projects that themselves use Mercurial. I will absolutely test to see if git-remote-hg works with CPython and I can document using that to contribute to CPython. I’m not sure it needs to be part of the PEP or not? Feels like something that would be better inside the devguide itself but I’m not opposed to putting it both locations.
Beyond that, GitHub is indeed the most expedient option. My two main reasons for objecting to taking the expedient path are:
It's not entirely about expedience. I think a lot of the reason why we should look towards outsourcing some of these items is that volunteer time is not a fungible resource. Volunteers are generally only willing to work on things which they personally care about. This is entirely unlike a business where you have employees who will generally work on whatever you tell them to because that's what you're paying them for. To this end I personally don't really have an interest in trying to create a better code hosting platform than Github when Github is doing an amazing job in my opinion and they satisify my needs fine. Given the *current* state of tooling it appears that there are not a lot of people who both care about making that piece of software exist and are capable of competing with Github in terms of quality.
1. I strongly believe that the long term sustainability of the overall open source community requires the availability and use of open source infrastructure. While I admire the ingenuity of the "free-as-in-beer" model for proprietary software companies fending off open source competition, I still know a proprietary platform play when I see one (and so do venture capitalists looking to extract monopoly rents from the industry in the future). (So yes, I regret relenting on this principle in previously suggesting the interim use of another proprietary hosted service)
I somewhat agree. However I’m less concerned specifically about where projects are hosted exactly and more about the *ability* to move to a completely OSS infrastructure. In particular if at somepoint we need to move off of Github we can totally do that, it’s not particularly difficult. Currently you lose the higher quality polish of Github if you do that however if at some point in the future Github either turns evil or an OSS software offers a truly compelling alternative to Github then there is really nothing stopping a migration to another platform. As I said in the PEP I view this as a “let’s cross that bridge if/when we get to it”. The main thing we should look at is things that would be difficult to migrate away from. For code hosting in particular most of the truly valuable data is stored within the DVCS so migrating the bulk of the data is as simple as pushing the repository to a new location. The other data is within the issues, for these repositories I suggest moving the issues to Github entirely because I don’t suspect they’ll get many if any issues themselves so the amount of data stored in issues will be low. However I also think that long term sustainability of any particular project depends on attracting and retaining contributors. To this end going to where the people are already and paving inroads that reduce the barriers to contributing is an important thing to do as well. This proposal is aimed squarely at reducing barriers to contribution while also giving a nod to the first concern by selecting a platform that has done a lot to enable OSS and doing it in a way that the ability to leave that platform is maintained so that in some future we can migrate away if need be.
2. I also feel that this proposal is far too cavalier in not even discussing the possibility of helping out the Mercurial team to resolve their documentation and usability issues rather than just yelling at them "your tool isn't popular enough for us, and we find certain aspects of it too hard to use, so we're switching to something else rather than working with you to address our concerns". We consider the Mercurial team a significant enough part of the Python ecosystem that Matt was one of the folks specifically invited to the 2014 language summit to discuss their concerns around the Python 3 transition. Yet we'd prefer to switch to something else entirely rather than organising a sprint with them at PyCon to help ensure that our existing Mercurial based infrastructure is approachable for git & GitHub users? (And yes, I consider some of the core Mercurial devs to be friends, so this isn't an entirely abstract concern for me)
I was on the fence about including the bit about branches in the PEP itself and I ended up doing it only because multiple people brought it up when I asked them for a review. I also tried not to include the fact that I haven’t personally figured out how to actually use Mercurial effectively in the PEP itself because honestly I don’t think that is really the core idea behind moving to git and Github. I think if you look at any semi objective measurement between git and Mercurial’s effective popularity git is going to be a clear winner, and if you do the same thing for Github compared to any other code hosting service or software then Github wins that by any even larger margin. The reason the PEP primarily focuses on the popularity of the the tool is because as you mentioned, issues like poor documentation, bad support for a particular platform, a particular workflow not being very good can be solved by working with the tool authors to solve that particular problem. I wouldn’t consider those issues in a vacuum to be a good reason to migrate away from that tool. However there’s very little that CPython can do to get more people using Mercurial, and presumably the authors of Mercurial are already doing what they can to get people to use them. However by choosing, and continuing to choose, a tool that an order of magnitude less people use, we’re choosing to effectively make it harder to contribute. We’re increasing the likelhood that a contributor is going to have to learn our particular DVCS even if they already know one and we’re increasing the likelhood that we’re burdening users to learn a technology that isn’t going to transfer to most other projects that they might want to contribute to, even within the Python ecosystem itself. Even the suggestion of a way to make it approachable for git and Github users still acts as a barrier to contribution. Now it’s true that when you have to select against competing tools the tool you choose is going to be a barrier to *someone*. For example if we do choose to go with git, then that tool will be a barrier to people who already know Mercurial but don’t already know git. However by selecting the most popular tool you ensure that the particular barrier is a barrier to as few as people as possible. To be clear, I don’t consider the technical differences between Mercurial and git to be very large hurdles to overcome, it’s primarily about the mindshare and the fact that Mercurial doesn’t really enable us to do much that git doesn’t also allow us to do (other than support friends, which isn’t an unreasonable desire either) while keeping us from tapping into the collective power of the number of git users. To put it another way, Linux vs the BSDs, I actually prefer FreeBSD over Linux, but I almost never use it because Linux is vastly more popular and by selecting it I’m more likely to be able to transfer that knowledge to different scenarios and find help when I have a problem.
Given my proposal to use BitBucket as a near term solution for enabling pull request based workflows, it's clear I consider the second argument the more significant of the two.
However, if others consider some short term convenience that may or may not attract additional contributors more important than supporting the broader Python and open source communities (an argument I'm more used to hearing in the ruthlessly commercial environment of Red Hat, rather than in upstream contexts that tend to be less worried about "efficiency at any cost"), I'm not going to expend energy trying to prevent a change I disagree with on principle, but will instead work to eliminate (or significantly reduce) the current expedience argument in GitHub's favour.
As a result, I'm -0 on the PEP, rather than -1 (and will try to stay out of further discussions).
Given this proposal, I'm planning to refocus PEPs 474 & 462 specifically on resolving the CPython core workflow issues, since that will require infrastructure customisation regardless, and heavy customisation of GitHub based infrastructure requires opting in to the use of the GitHub specific APIs that create platform lockin. (Note that the argument in PEP 481 about saving overall infrastructure work is likely spurious - the vast majority of that work will be in addressing the complex CPython workflow requirements, and moving some support repos to GitHub does little to alleviate that
That was specifically about saving infrastructure work to supporting Kallithea (or whatever solution). I don’t suspect that CPython was planning on using Kallithea (maybe I’m wrong?) so forge.python.org was primarily aimed towards the non-CPython repositories.
If folks decide they want to migrate the ancillary repos back from GitHub after that other infrastructure work is done, so be it, but if they don't, that's OK too. We're already running heterogeneous infrastructure across multiple services (especially if you also take PyPA into account), so having additional support repos externally hosted isn't that big a deal from a purely practical perspective.
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Nov 29, 2014, at 9:01 PM, Donald Stufft <donald@stufft.io> wrote:
The PEP should also cover providing clear instructions for setting up git-remote-hg with the remaining Mercurial repos (most notably CPython), as well as documenting a supported workflow for generating patches based on the existing CPython GitHub mirror.
I can add this. I’ve never actually tried using git-remote-hg with CPython itself because I’ve made it segfault on other Mercurial repositories and I never figured out why so I just generally fight my way through using Mercurial on projects that themselves use Mercurial. I will absolutely test to see if git-remote-hg works with CPython and I can document using that to contribute to CPython. I’m not sure it needs to be part of the PEP or not? Feels like something that would be better inside the devguide itself but I’m not opposed to putting it both locations.
Nevermind. I’m not going to add this because git-remote-hg is busted with the latest version of Mercurial and I don’t think it’s really needed for this PEP (though would be good for the devguide at some point). --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 11/29/2014 06:01 PM, Donald Stufft wrote:
The reason the PEP primarily focuses on the popularity of the the tool is because as you mentioned, issues like poor documentation, bad support for a particular platform, a particular workflow not being very good can be solved by working with the tool authors to solve that particular problem. I wouldn’t consider those issues in a vacuum to be a good reason to migrate away from that tool.
As I understand it[1] my current employer (Facebook) picked Mercurial over git because these very reason end up being important. And this analysis have been validated by another big company[2]. Git implementation is very tied to the linux world and this slowed down its gain of a windows support. This is not something that will change by discussing with author: "btw can you rewrite your tool with different techno and concept?". Mercurial is extensible in Python, very extensible. In my previous job one of our client switched to Mercurial and was able to get an extension adding commands to match it exact previous code-review workflow in a couple of hundred line of python. (you could have the same for python). Mercurial developer are already connected to the Python community. They are invited to language submit, regular pycon speaker and attendees etc. All these things contradict "bah any project would not make a difference"
However there’s very little that CPython can do to get more people using Mercurial, and presumably the authors of Mercurial are already doing what they can to get people to use them.
Mercurial is an open source project. We have no communication department, no communication budget actually. Over the year, more and more contributors are actually paid to do so, but they usually focus on "making employer's users" happy. Something that rarely involves getting more outside-world users. We mostly rely on the network effect to gain more users, (yes, we are losing to git on this, but still growing anyway). Part of this network effect is having big project like CPython using Mercurial. It also imply that CPython dev are willing to look at how the tools works and that the Project tries to take advantage of the tools strength. This would turn the situation into mutual benefits. You are happy with Mercurial and we are happy with Python. However moving to git and github send a very different signal: If you want to be a successful command line tool, use C and bash. If you want to be a successful website use ruby on rails. -- Pierre-Yves David [1] This is a personal statement is not to be linked to the opinion of my employer PR department. [2] That I'm not naming in fear of there PR assasins.
On Nov 30, 2014, at 10:08 PM, Pierre-Yves David <pierre-yves.david@ens-lyon.org> wrote:
On 11/29/2014 06:01 PM, Donald Stufft wrote:
The reason the PEP primarily focuses on the popularity of the the tool is because as you mentioned, issues like poor documentation, bad support for a particular platform, a particular workflow not being very good can be solved by working with the tool authors to solve that particular problem. I wouldn’t consider those issues in a vacuum to be a good reason to migrate away from that tool.
As I understand it[1] my current employer (Facebook) picked Mercurial over git because these very reason end up being important. And this analysis have been validated by another big company[2].
Git implementation is very tied to the linux world and this slowed down its gain of a windows support. This is not something that will change by discussing with author: "btw can you rewrite your tool with different techno and concept?". Mercurial is extensible in Python, very extensible. In my previous job one of our client switched to Mercurial and was able to get an extension adding commands to match it exact previous code-review workflow in a couple of hundred line of python. (you could have the same for python). Mercurial developer are already connected to the Python community. They are invited to language submit, regular pycon speaker and attendees etc.
All these things contradict "bah any project would not make a difference"
However there’s very little that CPython can do to get more people using Mercurial, and presumably the authors of Mercurial are already doing what they can to get people to use them.
Mercurial is an open source project. We have no communication department, no communication budget actually. Over the year, more and more contributors are actually paid to do so, but they usually focus on "making employer's users" happy. Something that rarely involves getting more outside-world users. We mostly rely on the network effect to gain more users, (yes, we are losing to git on this, but still growing anyway). Part of this network effect is having big project like CPython using Mercurial. It also imply that CPython dev are willing to look at how the tools works and that the Project tries to take advantage of the tools strength. This would turn the situation into mutual benefits. You are happy with Mercurial and we are happy with Python.
However moving to git and github send a very different signal: If you want to be a successful command line tool, use C and bash. If you want to be a successful website use ruby on rails.
I want to adress this point specifically, because it’s not particularly related to the hg vs git discussion that Guido has asked people to stop doing. The idea that unless Python as a project always picks something written in Python over something written in something else we’re somehow signaling to the world that if you want to write X kind of tool you should do it in some other language is laughable. It completely ignores anything at all about the tools except what language they are written in.
On 11/30/2014 07:43 PM, Donald Stufft wrote:
The idea that unless Python as a project always picks something written in Python over something written in something else we’re somehow signaling to the world that if you want to write X kind of tool you should do it in some other language is laughable. It completely ignores anything at all about the tools except what language they are written in.
My point is not "Python should always" pick python over any other language. My point is: a tool written in python and with developer already involved with the community is something to takes in account. And is we simplify the technical debate (and shut it down as requested) by saying both are equivalent. This is part of the other consideration to take in account. -- Pierre-Yves David
Nick Coghlan <ncoghlan@gmail.com> writes:
1. I strongly believe that the long term sustainability of the overall open source community requires the availability and use of open source infrastructure.
I concur. This article <URL:http://mako.cc/writing/hill-free_tools.html> makes the arguments well, IMO.
2. I also feel that this proposal is far too cavalier in not even discussing the possibility of helping out the Mercurial team […] we'd prefer to switch to something else entirely rather than organising a sprint with them at PyCon to help ensure that our existing Mercurial based infrastructure is approachable for git & GitHub users?
Exactly. For such a core tool, instead of pushing proprietary platforms at the expense of software freedom, the sensible strategy for a project (Python) that hopes to be around in the long term is to use and improve the free software platforms.
As a result, I'm -0 on the PEP, rather than -1 (and will try to stay out of further discussions).
I don't get a vote. So I'm glad there are some within the Python core development team that can see the mistakes inherent in depending on non-free tools for developing free software. -- \ “The cost of a thing is the amount of what I call life which is | `\ required to be exchanged for it, immediately or in the long | _o__) run.” —Henry David Thoreau | Ben Finney
On Sun, Nov 30, 2014 at 4:06 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
I don't get a vote. So I'm glad there are some within the Python core development team that can see the mistakes inherent in depending on non-free tools for developing free software.
While this is a laudable view, this kind of extreme stance is contrary to any semblance of practicality. Compare: http://www.gnu.org/distros/free-distros.html http://www.gnu.org/distros/common-distros.html#Debian Debian is not considered sufficiently free because "people can readily learn about these nonfree packages by browsing Debian's online package database", even though you have to be very much explicit about these things (you have to go and enable the non-free repos). Yes, GitHub is proprietary. But all of your actual code is stored in git, which is free, and it's easy to push that to a new host somewhere else, or create your own host. This proposal is for repositories that don't need much in the way of issue trackers etc, so shifting away from GitHub shouldn't demand anything beyond moving the repos themselves. How bad is it, really? Is it worth fighting a philosophical battle for the sake of no real gain, sacrificing real benefits for the intangible "but it's not free" debate? Python is already using quite a bit of non-free software in its ecosystem. The Windows builds of CPython are made with Microsoft's compiler, and the recent discussion about shifting to Cygwin or MinGW basically boiled down to "but it ought to be free software", and that was considered not a sufficiently strong argument. In each case, the decision has impact on other people (using MSVC for the official python.org installers means extension writers need to use MSVC too; and using GitHub means that contributors are strongly encouraged, possibly required, to use GitHub); so why is it acceptable to use a non-free compiler, but not acceptable to use a non-free host? I admire and respect the people who, for their own personal use, absolutely and utterly refuse to use any non-free systems or software. It's great that they do it, because that helps encourage free software to be created. But for myself? I'll use whatever makes the most sense. Proprietary systems have inherent issues (the best-maintained non-free programs seem to have about the same bugginess as a poorly-maintained free program, or at least that's how it feels), but if the available free alternatives have even more issues, I'll not hobble myself for the purity of freedom. Practicality wins. ChrisA
On 30 November 2014 at 15:23, Chris Angelico <rosuav@gmail.com> wrote:
Python is already using quite a bit of non-free software in its ecosystem. The Windows builds of CPython are made with Microsoft's compiler, and the recent discussion about shifting to Cygwin or MinGW basically boiled down to "but it ought to be free software", and that was considered not a sufficiently strong argument. In each case, the decision has impact on other people (using MSVC for the official python.org installers means extension writers need to use MSVC too; and using GitHub means that contributors are strongly encouraged, possibly required, to use GitHub); so why is it acceptable to use a non-free compiler, but not acceptable to use a non-free host?
Relying on non-free software to support users of a non-free platform is rather different from *requiring* the use of non-free software to participate in core Python community design processes. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, Nov 30, 2014 at 8:54 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 30 November 2014 at 15:23, Chris Angelico <rosuav@gmail.com> wrote:
Python is already using quite a bit of non-free software in its ecosystem. The Windows builds of CPython are made with Microsoft's compiler, and the recent discussion about shifting to Cygwin or MinGW basically boiled down to "but it ought to be free software", and that was considered not a sufficiently strong argument. In each case, the decision has impact on other people (using MSVC for the official python.org installers means extension writers need to use MSVC too; and using GitHub means that contributors are strongly encouraged, possibly required, to use GitHub); so why is it acceptable to use a non-free compiler, but not acceptable to use a non-free host?
Relying on non-free software to support users of a non-free platform is rather different from *requiring* the use of non-free software to participate in core Python community design processes.
But what non-free software is required to use the community design processes? The GitHub client is entirely optional; I don't use it, I just use git itself. Using a free client to access a proprietary server isn't the same as using non-free software. ChrisA
Chris Angelico <rosuav@gmail.com> writes:
But what non-free software is required to use the community design processes? The GitHub client is entirely optional; I don't use it, I just use git itself. Using a free client to access a proprietary server isn't the same as using non-free software.
For that to remain true, there must be no advantage specifically to GitHub except as a commodity repository host. So all the verbiage about specific GitHub features in this PEP are undermined. On the other hand, if GitHub is being recommended specifically because of features that cannot be migrated elsewhere (such as its proprietary pull requests), then it is not possible to argue that Git alone is sufficient to realise these advantages. -- \ “DRM doesn't inconvenience [lawbreakers] — indeed, over time it | `\ trains law-abiding users to become [lawbreakers] out of sheer | _o__) frustration.” —Charles Stross, 2010-05-09 | Ben Finney
On Nov 30, 2014, at 11:17, Chris Angelico <rosuav@gmail.com> wrote:
On Sun, Nov 30, 2014 at 8:54 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 30 November 2014 at 15:23, Chris Angelico <rosuav@gmail.com> wrote:
Python is already using quite a bit of non-free software in its ecosystem. The Windows builds of CPython are made with Microsoft's compiler, and the recent discussion about shifting to Cygwin or MinGW basically boiled down to "but it ought to be free software", and that was considered not a sufficiently strong argument. In each case, the decision has impact on other people (using MSVC for the official python.org installers means extension writers need to use MSVC too; and using GitHub means that contributors are strongly encouraged, possibly required, to use GitHub); so why is it acceptable to use a non-free compiler, but not acceptable to use a non-free host?
Relying on non-free software to support users of a non-free platform is rather different from *requiring* the use of non-free software to participate in core Python community design processes.
But what non-free software is required to use the community design processes? The GitHub client is entirely optional; I don't use it, I just use git itself. Using a free client to access a proprietary server isn't the same as using non-free software.
Also keep in mind that unless you are using a very esoteric hardware setup and dedicated leased lines that you purchased yourself, you are likely to be using copyrighted, patented, proprietary software at a number of levels: the microcode implementation in your CPU the firmware in your GPU the firmware in your network card the boot code (e.g.: BIOS or EFI implementation) of your motherboard the firmware in your router or the firmware in your cable or DSL modem, if you thought to get a free router with OpenWRT or something in it the firmware in your ISP's router the firmware in the backbone's routers the firmware in the PSF's ISP's routers Does this sound like ridiculous nitpicking? Of course it does! If you refused to use all that stuff you just wouldn't be able to access the internet at all, regardless of your personal choices. Most layers of this stack are _less_ constrained to open standards and open data access than Github, and can require layers and layers of additional proprietary software or reverse engineering (ask anyone who has tried to play a video game on a Linux computer what the experience is like without gobs of proprietary blobs from nvidia or ATI). And as the story of BitKeeper shows, if a proprietary platform decides to do something bad, if the cost of migration is within your means, you can just leave. This is far more true of Github than Bitkeeper - Linux had to create a totally new VCS to migrate off of that, we just have to install Trac or Roundup or something again. (Which, as per the presently-proposed PEP, we don't even have to install them again, just change some configuration to put a few repositories back.) The monoculture about Github concerns me. I also have concerns about the long-term consequences of not having an all-free-software stack. But focusing on avoiding services like Github at this point in history is just a gigantic waste of time; it's resolving dependencies in the wrong order. The only reason to avoid Github is ideological purity, and even then it's not even "purity" because you still have to accept this other ideological contamination. Except, what about other ideological concepts that are important to the Python core developers, like equitable representation for women and minorities, and compassionate and polite interactions within the community? Clearly we can't require the use of Linux. If we treat these ideals as an equivalent priority as free software (even if we ignore the "invisible" dependencies like the list I just made above), then there is literally no software left that we can use to develop Python. Linux kernel and GNU low-level user-land development are a perpetual cesspit, and their leaders serve as a continuous embarrassment to our community. And speaking of equitable representation, one proven technique we have learned for dealing with that problem is making it for newcomers of all stripes to access the community. Like it or not, Github's popularity means that it's a place where most newcomers to programming are learning to use version control and bug tracking. This is a network effect that can have a real impact on people's lives. Requiring newcomers to learn our weird technology religion before they can contribute creates a real barrier to entry, which in turn makes our community more insular and homogenous. Some days you get the Git, and some days the Github gets you. The sooner we, as a community and a culture, can accept this and move on, the more time will be available to actually build replacements for these things. -glyph
On Sun, 30 Nov 2014 16:23:08 +1100 Chris Angelico <rosuav@gmail.com> wrote:
Yes, GitHub is proprietary. But all of your actual code is stored in git, which is free, and it's easy to push that to a new host somewhere else, or create your own host. This proposal is for repositories that don't need much in the way of issue trackers etc, so shifting away from GitHub shouldn't demand anything beyond moving the repos themselves.
I hope we're not proposing to move the issue trackers to github, otherwise I'm -1 on this PEP. Regards Antoine.
On Sun, Nov 30, 2014 at 7:01 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Sun, 30 Nov 2014 16:23:08 +1100 Chris Angelico <rosuav@gmail.com> wrote:
Yes, GitHub is proprietary. But all of your actual code is stored in git, which is free, and it's easy to push that to a new host somewhere else, or create your own host. This proposal is for repositories that don't need much in the way of issue trackers etc, so shifting away from GitHub shouldn't demand anything beyond moving the repos themselves.
I hope we're not proposing to move the issue trackers to github, otherwise I'm -1 on this PEP.
Regards
Antoine.
So I usually choose not to weigh in on discussions like these but there seems to be a lot of misdirection in some of these arguments. To start, I'm generally neutral about this proposal or Nick's proposal that spurred this one. I've found the most frustrating part of contributing to anything involving CPython to be the lack of reviewer time. I have had very small (2-line) patches take months (close to a year in reality) to get through in spite of periodically pinging the appropriate people. Moving to git/GitHub will not alleviate this at all. To be clear, the main reasoning behind Nick's was being able to submit changes without ever having to have a local copy of the repository in question on your machine. Having a complete web workflow for editing and contributing makes the barrier to entry far lower than switching VCS or anything else. BitBucket (apparently, although I've never used this) and GitHub both have this capability and *both* are free-as-in-beer systems. No one as I understand it is proposing that we use the per-distro proprietary interface to these websites. All data can be removed from GitHub using it's API and can generally be converted to another platform. The same goes for BitBucket although it's arguably easier to retrieve issue data from BitBucket than GitHub. That said, *the issue tracker is not covered by these proposals* so this is a moot point. Drop it already. If we're seriously considering moving to git as a DVCS, we should consider the real free-as-in-freedom alternatives that come very close to GitHub and can be improved by us (even though they're not written in Python). One of those is GitLab. We can self-host a GitLab instance easily or we can rely on gitlab.com. GitLab aims to provide a very similar user experience to GitHub and it's slowly approaching feature parity and experience parity. GitLab is also what a lot of people chose to switch to after the incident Steven mentioned, which I don't think is something we should dismiss or ignore. We should refocus the discussion with the following in mind: - Migrating "data" from GitHub is easy. There are free-as-in-freedom tools to do it and the only cost is the time it would take to monitor the process - GitHub has a toxic company culture that we should be aware of before moving to it. They have a couple blog posts about attempting to change it but employees became eerily silent after the incident and have remained so from what I've personally seen. - GitHub may be popular but there are popular FOSS solutions that exist that we can also self-host at something like forge.python.org - bugs.python.org is not covered by any of these proposals - The main benefit this proposal (and the proposal to move to BitBucket) are seeking to achieve is an online editing experience allowing for *anyone with a browser and an account* to contribute. This to me is the only reason I would be +1 for either of these proposals (if we can reach consensus).
On Sun Nov 30 2014 at 10:55:26 AM Ian Cordasco <graffatcolmingov@gmail.com> wrote:
On Sun, Nov 30, 2014 at 7:01 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Sun, 30 Nov 2014 16:23:08 +1100 Chris Angelico <rosuav@gmail.com> wrote:
Yes, GitHub is proprietary. But all of your actual code is stored in git, which is free, and it's easy to push that to a new host somewhere else, or create your own host. This proposal is for repositories that don't need much in the way of issue trackers etc, so shifting away from GitHub shouldn't demand anything beyond moving the repos themselves.
I hope we're not proposing to move the issue trackers to github, otherwise I'm -1 on this PEP.
Regards
Antoine.
So I usually choose not to weigh in on discussions like these but there seems to be a lot of misdirection in some of these arguments.
To start, I'm generally neutral about this proposal or Nick's proposal that spurred this one. I've found the most frustrating part of contributing to anything involving CPython to be the lack of reviewer time. I have had very small (2-line) patches take months (close to a year in reality) to get through in spite of periodically pinging the appropriate people. Moving to git/GitHub will not alleviate this at all.
To be clear, the main reasoning behind Nick's was being able to submit changes without ever having to have a local copy of the repository in question on your machine. Having a complete web workflow for editing and contributing makes the barrier to entry far lower than switching VCS or anything else. BitBucket (apparently, although I've never used this) and GitHub both have this capability and *both* are free-as-in-beer systems.
No one as I understand it is proposing that we use the per-distro proprietary interface to these websites.
All data can be removed from GitHub using it's API and can generally be converted to another platform. The same goes for BitBucket although it's arguably easier to retrieve issue data from BitBucket than GitHub. That said, *the issue tracker is not covered by these proposals* so this is a moot point. Drop it already.
If we're seriously considering moving to git as a DVCS, we should consider the real free-as-in-freedom alternatives that come very close to GitHub and can be improved by us (even though they're not written in Python). One of those is GitLab. We can self-host a GitLab instance easily or we can rely on gitlab.com. GitLab aims to provide a very similar user experience to GitHub and it's slowly approaching feature parity and experience parity. GitLab is also what a lot of people chose to switch to after the incident Steven mentioned, which I don't think is something we should dismiss or ignore.
We should refocus the discussion with the following in mind:
- Migrating "data" from GitHub is easy. There are free-as-in-freedom tools to do it and the only cost is the time it would take to monitor the process
- GitHub has a toxic company culture that we should be aware of before moving to it. They have a couple blog posts about attempting to change it but employees became eerily silent after the incident and have remained so from what I've personally seen.
- GitHub may be popular but there are popular FOSS solutions that exist that we can also self-host at something like forge.python.org
- bugs.python.org is not covered by any of these proposals
- The main benefit this proposal (and the proposal to move to BitBucket) are seeking to achieve is an online editing experience allowing for *anyone with a browser and an account* to contribute. This to me is the only reason I would be +1 for either of these proposals (if we can reach consensus).
But that's not just it. As you pointed out, Ian, getting patch submissions committed faster would be a huge improvement over what we have today. GitHub/Bitbucket/whatever could help with this by giving core devs basic CI to know that I patch is sound to some extent as well as push button commits of patches. For me personally, if I knew a simple patch integrated cleanly and passed on at least one buildbot -- when it wasn't a platform-specific fix -- then I could easily push a "Commit" button and be done with it (although this assumes single branch committing; doing this across branches makes all of this difficult unless we finally resolve our Misc/NEWS conflict issues so that in some instances it can be automated). Instead I have to wait until I have a clone I can push from, download a patch, apply it, run the unit tests myself, do the commit, and then repeat a subset of that to whatever branches make sense. It's a lot of work for which some things could be automated.
On Nov 30, 2014, at 11:44 AM, Brett Cannon <brett@python.org> wrote:
On Sun Nov 30 2014 at 10:55:26 AM Ian Cordasco <graffatcolmingov@gmail.com <mailto:graffatcolmingov@gmail.com>> wrote: On Sun, Nov 30, 2014 at 7:01 AM, Antoine Pitrou <solipsis@pitrou.net <mailto:solipsis@pitrou.net>> wrote:
On Sun, 30 Nov 2014 16:23:08 +1100 Chris Angelico <rosuav@gmail.com <mailto:rosuav@gmail.com>> wrote:
Yes, GitHub is proprietary. But all of your actual code is stored in git, which is free, and it's easy to push that to a new host somewhere else, or create your own host. This proposal is for repositories that don't need much in the way of issue trackers etc, so shifting away from GitHub shouldn't demand anything beyond moving the repos themselves.
I hope we're not proposing to move the issue trackers to github, otherwise I'm -1 on this PEP.
Regards
Antoine.
So I usually choose not to weigh in on discussions like these but there seems to be a lot of misdirection in some of these arguments.
To start, I'm generally neutral about this proposal or Nick's proposal that spurred this one. I've found the most frustrating part of contributing to anything involving CPython to be the lack of reviewer time. I have had very small (2-line) patches take months (close to a year in reality) to get through in spite of periodically pinging the appropriate people. Moving to git/GitHub will not alleviate this at all.
To be clear, the main reasoning behind Nick's was being able to submit changes without ever having to have a local copy of the repository in question on your machine. Having a complete web workflow for editing and contributing makes the barrier to entry far lower than switching VCS or anything else. BitBucket (apparently, although I've never used this) and GitHub both have this capability and *both* are free-as-in-beer systems.
No one as I understand it is proposing that we use the per-distro proprietary interface to these websites.
All data can be removed from GitHub using it's API and can generally be converted to another platform. The same goes for BitBucket although it's arguably easier to retrieve issue data from BitBucket than GitHub. That said, *the issue tracker is not covered by these proposals* so this is a moot point. Drop it already.
If we're seriously considering moving to git as a DVCS, we should consider the real free-as-in-freedom alternatives that come very close to GitHub and can be improved by us (even though they're not written in Python). One of those is GitLab. We can self-host a GitLab instance easily or we can rely on gitlab.com <http://gitlab.com/>. GitLab aims to provide a very similar user experience to GitHub and it's slowly approaching feature parity and experience parity. GitLab is also what a lot of people chose to switch to after the incident Steven mentioned, which I don't think is something we should dismiss or ignore.
We should refocus the discussion with the following in mind:
- Migrating "data" from GitHub is easy. There are free-as-in-freedom tools to do it and the only cost is the time it would take to monitor the process
- GitHub has a toxic company culture that we should be aware of before moving to it. They have a couple blog posts about attempting to change it but employees became eerily silent after the incident and have remained so from what I've personally seen.
- GitHub may be popular but there are popular FOSS solutions that exist that we can also self-host at something like forge.python.org <http://forge.python.org/>
- bugs.python.org <http://bugs.python.org/> is not covered by any of these proposals
- The main benefit this proposal (and the proposal to move to BitBucket) are seeking to achieve is an online editing experience allowing for *anyone with a browser and an account* to contribute. This to me is the only reason I would be +1 for either of these proposals (if we can reach consensus).
But that's not just it. As you pointed out, Ian, getting patch submissions committed faster would be a huge improvement over what we have today. GitHub/Bitbucket/whatever could help with this by giving core devs basic CI to know that I patch is sound to some extent as well as push button commits of patches.
For me personally, if I knew a simple patch integrated cleanly and passed on at least one buildbot -- when it wasn't a platform-specific fix -- then I could easily push a "Commit" button and be done with it (although this assumes single branch committing; doing this across branches makes all of this difficult unless we finally resolve our Misc/NEWS conflict issues so that in some instances it can be automated). Instead I have to wait until I have a clone I can push from, download a patch, apply it, run the unit tests myself, do the commit, and then repeat a subset of that to whatever branches make sense. It's a lot of work for which some things could be automated.
Well there’s two sides to the contribution process. There’s making things better/easier for people who *aren’t* committers and there is making things better/easier for people who *are* committers. Tacking extra things on to what we already have to improve the life of committers is easier in many ways. As committers they’ve likely already taken the time to learn the bespoke workflow that the Python project uses and have already gotten through that particular hurdle. Looking to standardize around popular tools is mostly about making it easier for *new* people and making it so that if they learn this set of tools they can go an immediately apply that to most of the other Python projects out there, or that if they are already contributing to those other Python projects they are probably aware of this particular toolchain and workflow and can apply that knowledge directly to the Python project. Moving to some of these tools happens to come with it features like really nice CI integration and a nice "Merge" button that also make it a lot nicer for the committer side of things. I think it's also hard to get a representation of the people for whom the bespoke workflow and less popular tooling are a problem for in a discussion on python-dev. My guess is most of those people would not have signed up for python-dev since, unless they were willing to take the time to learn that, so there is an amount of selection bias at play here as well. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Sun Nov 30 2014 at 12:00:20 PM Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 11:44 AM, Brett Cannon <brett@python.org> wrote:
On Sun Nov 30 2014 at 10:55:26 AM Ian Cordasco <graffatcolmingov@gmail.com> wrote:
On Sun, Nov 30, 2014 at 7:01 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Sun, 30 Nov 2014 16:23:08 +1100 Chris Angelico <rosuav@gmail.com> wrote:
Yes, GitHub is proprietary. But all of your actual code is stored in git, which is free, and it's easy to push that to a new host somewhere else, or create your own host. This proposal is for repositories that don't need much in the way of issue trackers etc, so shifting away from GitHub shouldn't demand anything beyond moving the repos themselves.
I hope we're not proposing to move the issue trackers to github, otherwise I'm -1 on this PEP.
Regards
Antoine.
So I usually choose not to weigh in on discussions like these but there seems to be a lot of misdirection in some of these arguments.
To start, I'm generally neutral about this proposal or Nick's proposal that spurred this one. I've found the most frustrating part of contributing to anything involving CPython to be the lack of reviewer time. I have had very small (2-line) patches take months (close to a year in reality) to get through in spite of periodically pinging the appropriate people. Moving to git/GitHub will not alleviate this at all.
To be clear, the main reasoning behind Nick's was being able to submit changes without ever having to have a local copy of the repository in question on your machine. Having a complete web workflow for editing and contributing makes the barrier to entry far lower than switching VCS or anything else. BitBucket (apparently, although I've never used this) and GitHub both have this capability and *both* are free-as-in-beer systems.
No one as I understand it is proposing that we use the per-distro proprietary interface to these websites.
All data can be removed from GitHub using it's API and can generally be converted to another platform. The same goes for BitBucket although it's arguably easier to retrieve issue data from BitBucket than GitHub. That said, *the issue tracker is not covered by these proposals* so this is a moot point. Drop it already.
If we're seriously considering moving to git as a DVCS, we should consider the real free-as-in-freedom alternatives that come very close to GitHub and can be improved by us (even though they're not written in Python). One of those is GitLab. We can self-host a GitLab instance easily or we can rely on gitlab.com. GitLab aims to provide a very similar user experience to GitHub and it's slowly approaching feature parity and experience parity. GitLab is also what a lot of people chose to switch to after the incident Steven mentioned, which I don't think is something we should dismiss or ignore.
We should refocus the discussion with the following in mind:
- Migrating "data" from GitHub is easy. There are free-as-in-freedom tools to do it and the only cost is the time it would take to monitor the process
- GitHub has a toxic company culture that we should be aware of before moving to it. They have a couple blog posts about attempting to change it but employees became eerily silent after the incident and have remained so from what I've personally seen.
- GitHub may be popular but there are popular FOSS solutions that exist that we can also self-host at something like forge.python.org
- bugs.python.org is not covered by any of these proposals
- The main benefit this proposal (and the proposal to move to BitBucket) are seeking to achieve is an online editing experience allowing for *anyone with a browser and an account* to contribute. This to me is the only reason I would be +1 for either of these proposals (if we can reach consensus).
But that's not just it. As you pointed out, Ian, getting patch submissions committed faster would be a huge improvement over what we have today. GitHub/Bitbucket/whatever could help with this by giving core devs basic CI to know that I patch is sound to some extent as well as push button commits of patches.
For me personally, if I knew a simple patch integrated cleanly and passed on at least one buildbot -- when it wasn't a platform-specific fix -- then I could easily push a "Commit" button and be done with it (although this assumes single branch committing; doing this across branches makes all of this difficult unless we finally resolve our Misc/NEWS conflict issues so that in some instances it can be automated). Instead I have to wait until I have a clone I can push from, download a patch, apply it, run the unit tests myself, do the commit, and then repeat a subset of that to whatever branches make sense. It's a lot of work for which some things could be automated.
Well there’s two sides to the contribution process.
There’s making things better/easier for people who *aren’t* committers and there is making things better/easier for people who *are* committers. Tacking extra things on to what we already have to improve the life of committers is easier in many ways. As committers they’ve likely already taken the time to learn the bespoke workflow that the Python project uses and have already gotten through that particular hurdle. Looking to standardize around popular tools is mostly about making it easier for *new* people and making it so that if they learn this set of tools they can go an immediately apply that to most of the other Python projects out there, or that if they are already contributing to those other Python projects they are probably aware of this particular toolchain and workflow and can apply that knowledge directly to the Python project.
Moving to some of these tools happens to come with it features like really nice CI integration and a nice "Merge" button that also make it a lot nicer for the committer side of things.
All very true, but if we can't improve both sides then we are simply going to end up with even more patches that we take a while to get around to. I want to end up with a solution that advances the situation for *both* committers and non-committers and I feel like that is being lost in the discussion somewhat. As the person who pushed for a migration to DVCS for non-committers I totally support improving the workflow for non-committers, but not at the cost of ignoring the latter half of the contribution workflow of committers which is a chronic problem. As the PEP points out, the devguide, devinabox, and the PEPs have such a shallow development process that hosting them on Bitbucket wouldn't be a big thing. But if we don't view this as a long-term step towards moving cpython development somehow we are bifurcating our code contributors between git and hg which will be annoying. Now it could be argued that it doesn't matter for the peps and devguide since they are purely text and can be done easily through a web UI and a simple CI in Travis can be set up to make sure that the docs compile cleanly. But moving devinabox where there is going to be a code checkout in order to execute code for testing, etc. will be an issue. So I guess my view is +0 for doc-only repos on GitHub as long as we make it clear we are doing it with the expectation that people will do everything through the web UI and never have to know git. But I can't advocate moving code over without moving ALL repos over to git -- hosting location doesn't matter to me -- to prevent having to know both DVCSs in order to do coding work related to Python; the cpython repo shouldn't become this vaunted repo that is special and so it's in hg long-term but everything else is on git. -Brett
I think it's also hard to get a representation of the people for whom the bespoke workflow and less popular tooling are a problem for in a discussion on python-dev. My guess is most of those people would not have signed up for python-dev since, unless they were willing to take the time to learn that, so there is an amount of selection bias at play here as well.
On Nov 30, 2014, at 2:19 PM, Brett Cannon <brett@python.org> wrote:
All very true, but if we can't improve both sides then we are simply going to end up with even more patches that we take a while to get around to. I want to end up with a solution that advances the situation for *both* committers and non-committers and I feel like that is being lost in the discussion somewhat. As the person who pushed for a migration to DVCS for non-committers I totally support improving the workflow for non-committers, but not at the cost of ignoring the latter half of the contribution workflow of committers which is a chronic problem.
As the PEP points out, the devguide, devinabox, and the PEPs have such a shallow development process that hosting them on Bitbucket wouldn't be a big thing. But if we don't view this as a long-term step towards moving cpython development somehow we are bifurcating our code contributors between git and hg which will be annoying. Now it could be argued that it doesn't matter for the peps and devguide since they are purely text and can be done easily through a web UI and a simple CI in Travis can be set up to make sure that the docs compile cleanly. But moving devinabox where there is going to be a code checkout in order to execute code for testing, etc. will be an issue.
So I guess my view is +0 for doc-only repos on GitHub as long as we make it clear we are doing it with the expectation that people will do everything through the web UI and never have to know git. But I can't advocate moving code over without moving ALL repos over to git -- hosting location doesn't matter to me -- to prevent having to know both DVCSs in order to do coding work related to Python; the cpython repo shouldn't become this vaunted repo that is special and so it's in hg long-term but everything else is on git.
So a goal of mine here is to sort of use these as a bit of a test bed. Moving CPython itself is a big and drastic change with a lot of implications, but moving the “support” repositories is not nearly as much, especially with a read only mirror on hg.python.org <http://hg.python.org/> which would allow things like the PEP rendering on www.python.org <http://www.python.org/> to stay the same if we wanted to. My hope was that we’d try this out, see how it works out, and if it seems to be a good thing, then at a later time we can either look at moving CPython itself or decide if it makes sense to do something different. Maybe this should be spelled out in the PEP? I’ve seen a few people say they were -1 because they didn’t want to split between hg on the CPython side and git on the supporting repos side. I’m not sure you can really get away from that because we’re *already* in that situation, things like the docs building script is a Git repo on Github, the python infrastructure itself is a git repo on Github, the new python.org <http://python.org/> website is a git repo on Github, the future PyPI is a git repo on GitHub. IOW I’m not sure what the best way forward is. I think moving to these tools for *all* repos is likely to be in the best interests of making things better for both sides of that coin however I didn’t want to go wholesale and try and make everything switch at all at once. If you think it makes sense to drop devinabox and make the dividing line between Code and not code (although I’d argue that line is already crossed with other code things already being on github) that’s fine with me. Or I can expand the scope if people think that makes more sense in the PEP too. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Sun Nov 30 2014 at 2:33:35 PM Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 2:19 PM, Brett Cannon <brett@python.org> wrote:
All very true, but if we can't improve both sides then we are simply going to end up with even more patches that we take a while to get around to. I want to end up with a solution that advances the situation for *both* committers and non-committers and I feel like that is being lost in the discussion somewhat. As the person who pushed for a migration to DVCS for non-committers I totally support improving the workflow for non-committers, but not at the cost of ignoring the latter half of the contribution workflow of committers which is a chronic problem.
As the PEP points out, the devguide, devinabox, and the PEPs have such a shallow development process that hosting them on Bitbucket wouldn't be a big thing. But if we don't view this as a long-term step towards moving cpython development somehow we are bifurcating our code contributors between git and hg which will be annoying. Now it could be argued that it doesn't matter for the peps and devguide since they are purely text and can be done easily through a web UI and a simple CI in Travis can be set up to make sure that the docs compile cleanly. But moving devinabox where there is going to be a code checkout in order to execute code for testing, etc. will be an issue.
So I guess my view is +0 for doc-only repos on GitHub as long as we make it clear we are doing it with the expectation that people will do everything through the web UI and never have to know git. But I can't advocate moving code over without moving ALL repos over to git -- hosting location doesn't matter to me -- to prevent having to know both DVCSs in order to do coding work related to Python; the cpython repo shouldn't become this vaunted repo that is special and so it's in hg long-term but everything else is on git.
So a goal of mine here is to sort of use these as a bit of a test bed. Moving CPython itself is a big and drastic change with a lot of implications, but moving the “support” repositories is not nearly as much, especially with a read only mirror on hg.python.org which would allow things like the PEP rendering on www.python.org to stay the same if we wanted to. My hope was that we’d try this out, see how it works out, and if it seems to be a good thing, then at a later time we can either look at moving CPython itself or decide if it makes sense to do something different. Maybe this should be spelled out in the PEP?
I’ve seen a few people say they were -1 because they didn’t want to split between hg on the CPython side and git on the supporting repos side. I’m not sure you can really get away from that because we’re *already* in that situation, things like the docs building script is a Git repo on Github, the python infrastructure itself is a git repo on Github, the new python.org website is a git repo on Github, the future PyPI is a git repo on GitHub.
That doesn't bother as that is support infrastructure around CPython but in no way directly tied to CPython releases. But devinabox, for instance, is specifically for helping people contribute to CPython, so asking people to use devinabox in git but then work in hg for some repos and git in others that devinabox checks out is just asking for trouble (e.g. devinabox checks out the peps, devguide, and cpython repos).
IOW I’m not sure what the best way forward is. I think moving to these tools for *all* repos is likely to be in the best interests of making things better for both sides of that coin however I didn’t want to go wholesale and try and make everything switch at all at once. If you think it makes sense to drop devinabox and make the dividing line between Code and not code (although I’d argue that line is already crossed with other code things already being on github) that’s fine with me. Or I can expand the scope if people think that makes more sense in the PEP too.
Depends what other people think, but for me it's "we are going to move to git long-term and we are starting an experiment with docs on GitHub to see if that impacts contributions and committer maintenance at least for docs, maybe for code eventually".
On Nov 30, 2014, at 3:26 PM, Brett Cannon <brett@python.org> wrote:
On Sun Nov 30 2014 at 2:33:35 PM Donald Stufft <donald@stufft.io <mailto:donald@stufft.io>> wrote:
On Nov 30, 2014, at 2:19 PM, Brett Cannon <brett@python.org <mailto:brett@python.org>> wrote:
All very true, but if we can't improve both sides then we are simply going to end up with even more patches that we take a while to get around to. I want to end up with a solution that advances the situation for *both* committers and non-committers and I feel like that is being lost in the discussion somewhat. As the person who pushed for a migration to DVCS for non-committers I totally support improving the workflow for non-committers, but not at the cost of ignoring the latter half of the contribution workflow of committers which is a chronic problem.
As the PEP points out, the devguide, devinabox, and the PEPs have such a shallow development process that hosting them on Bitbucket wouldn't be a big thing. But if we don't view this as a long-term step towards moving cpython development somehow we are bifurcating our code contributors between git and hg which will be annoying. Now it could be argued that it doesn't matter for the peps and devguide since they are purely text and can be done easily through a web UI and a simple CI in Travis can be set up to make sure that the docs compile cleanly. But moving devinabox where there is going to be a code checkout in order to execute code for testing, etc. will be an issue.
So I guess my view is +0 for doc-only repos on GitHub as long as we make it clear we are doing it with the expectation that people will do everything through the web UI and never have to know git. But I can't advocate moving code over without moving ALL repos over to git -- hosting location doesn't matter to me -- to prevent having to know both DVCSs in order to do coding work related to Python; the cpython repo shouldn't become this vaunted repo that is special and so it's in hg long-term but everything else is on git.
So a goal of mine here is to sort of use these as a bit of a test bed. Moving CPython itself is a big and drastic change with a lot of implications, but moving the “support” repositories is not nearly as much, especially with a read only mirror on hg.python.org <http://hg.python.org/> which would allow things like the PEP rendering on www.python.org <http://www.python.org/> to stay the same if we wanted to. My hope was that we’d try this out, see how it works out, and if it seems to be a good thing, then at a later time we can either look at moving CPython itself or decide if it makes sense to do something different. Maybe this should be spelled out in the PEP?
I’ve seen a few people say they were -1 because they didn’t want to split between hg on the CPython side and git on the supporting repos side. I’m not sure you can really get away from that because we’re *already* in that situation, things like the docs building script is a Git repo on Github, the python infrastructure itself is a git repo on Github, the new python.org <http://python.org/> website is a git repo on Github, the future PyPI is a git repo on GitHub.
That doesn't bother as that is support infrastructure around CPython but in no way directly tied to CPython releases. But devinabox, for instance, is specifically for helping people contribute to CPython, so asking people to use devinabox in git but then work in hg for some repos and git in others that devinabox checks out is just asking for trouble (e.g. devinabox checks out the peps, devguide, and cpython repos).
I’m not sure what you’re proposing here. If devinabox checks out peps, devguide, and cpython aren’t they going to have to use git and hg both unless we move all of those repositories together? Beyond just the tools the status quo of those is that if you do make a change you have different ways to submit a contribution depending on which repository it is.
IOW I’m not sure what the best way forward is. I think moving to these tools for *all* repos is likely to be in the best interests of making things better for both sides of that coin however I didn’t want to go wholesale and try and make everything switch at all at once. If you think it makes sense to drop devinabox and make the dividing line between Code and not code (although I’d argue that line is already crossed with other code things already being on github) that’s fine with me. Or I can expand the scope if people think that makes more sense in the PEP too.
Depends what other people think, but for me it's "we are going to move to git long-term and we are starting an experiment with docs on GitHub to see if that impacts contributions and committer maintenance at least for docs, maybe for code eventually”.
What do you mean by “docs”, is that the devguide and PEPs repository? Here’s another idea for an experiment that might be more generally useful. As we've said there are two sides to the coin here, non-comitters and comitters, a lot of the benefit of moving to Github is focused at non-comitters although there are benefits for comitters themselves. Django hosts it's git repository on Github and it's issue tracker is Trac. Although it doesn't require an issue to be created for small issues, you can just send a PR. What if we focused an experiment on the benefits to non-comitters? It's possible to maintain a git mirror of a Mercurial repository, in fact we already have that at github.com/python/cpython. What if we permit people to make PRs against that repository, and then take those PRs and paste them into roundup? Sort of like the "Remote hg repo" field. Then we can create some integration that would post a comment to the ticket whenever that PR is updated (sort of like the notification that happens when a new patch is uploaded). The cannonical repository would still be hg.python.org and in order to actually commit the PR commiters would need to turn the PR into a patch (trivially easy, just add .diff or .patch to the PR URL). If we ever move CPython to git (and Github) this sort of integration is going to be required anyways unless we plan on moving the issue tracker as well which I don't think we would since the Github issues are not fully featured enough for us. This would give us some real information on how people would prefer to contribute I think. If people start sending a lot of patches using Github PRs then we can look at moving the possibility of moving everything whole hog over to the git / Github. If people end up not really using it, or it doesn't seem to be actually useful we can look at disabling or (or keeping it around for people who like to use it). I still think moving at least the PEPs and devguide repository over to git / Github is a generally good idea that should be part of all of this as well. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 11/30/2014 4:45 PM, Donald Stufft wrote: I think you are stimulating more heated discussion than is necessary by trying to do too much, both in terms of physical changes and in terms of opinion persuasion. I am reminded of the integer division change. The initial discussion was initially over-heated partly because the actual change to Python was coupled, quite unnecessarily, with a metaphysical opinion as the the ontological status of natural numbers versus integers -- an opinion that many, including me, considered to be wrong. Once the metaphysical question was dropped, discussion went more smoothly.
Here’s another idea for an experiment that might be more generally useful.
I think a true experiment with one repository is easily justified. The PEP repository is an obvious choice because a) the main editor is in favor, b) many but not all core committers interact with it, and c) it is not tied to the tracker. The easier and more controversial option would be to move it completely to GitHub. I expect part of the result would be pull requests from committers who would otherwise commit directly to hg. The harder and, I think, more useful (generalizable) option would be to set up a mirror (or keep the hg version as a mirror ;-) and experiment with coordinating the two mirrors. Such an experiment should not preclude other experiments. If Brett wants to similarly experiment with devinabox on another site, let him. If the horrible-to-Nick prospect of possibly moving CPython to GitHub, if nothing else is done, provokes Nick to improve the workflow otherwise, great. If the mirror experiment is successful, the devguide might be the next experiment. It does not have any one maintainer, and *is* tied to the tracker. But herein lies the problem with the devguide. There are 22 issues, down just 1 from about a year ago. All but 2 are more than a year old. Many (most?) have patches, but enough consensus for anyone to push is hard. As with other doc issues, there is no 'test' for when a non-trivial patch is 'good enough' and hence, in my opinion, too much bikeshedding and pursuit of the perfect.
As we've said there are two sides to the coin here, non-comitters and comitters, a lot of the benefit of moving to Github is focused at non-comitters although there are benefits for comitters themselves.
For maintaining Idle, I do not see the benefit. Downloading patches from the tracker to my dev directory is trivial. I then apply to the current 3.x maintenance version, possibly with some hand editing, revise (always, that I can remember), and test. Once satisfied, I backport to 2.7.
What if we focused an experiment on the benefits to non-comitters?
Users benefit by more patches being applied. How do non-commiters benefit, really, by making it easier for them to submit patches that sit for years? Ignore that. Guido says that working with PEPs on GitHub would benefit him as a committer.
It's possible to maintain a git mirror of a Mercurial repository, in fact we already have that at github.com/python/cpython <http://github.com/python/cpython>. What if we permit people to make PRs against that repository, and then take those PRs and paste them into roundup? Sort of like the "Remote hg repo" field. Then we can create some integration that would post a comment to the ticket whenever that PR is updated (sort of like the notification that happens when a new patch is uploaded). The cannonical repository would still be hg.python.org <http://hg.python.org> and in order to actually commit the PR commiters would need to turn the PR into a patch (trivially easy, just add .diff or .patch to the PR URL).
This would be the focus of an experiment with the devguide, even if we have to generate some somewhat artificial pull requests for testing. I really hope you try to make the above work. The 3rd stage would be to expand on the above for doc patches. This is one area where we would get small ready-to-commit patches -- that do not need to be reported to the tracker. Would it be possible to automate the following? Turn a doc PR into a patch, apply the patch to all 3 branches (perhaps guided by the PR message), and generate a report, along with, currently, a 2.7 and 3.4 PR. (I am thinking about how to do some of this doc patches with hg on windows.) [snip premature discussion of moving cpython 'wholehog' to github.] Summary research plan: 3 experiments, each depending of the preceding. 1. Link 2 repositories, one with pull requests 2. Link the PRs with the tracker 3. Make PRs work better with our multibranch, 2-head monster. Report after each experiment (ending with 'success' or 'give-up'). -- Terry Jan Reedy
On Sun, 30 Nov 2014 22:06:03 -0500 Terry Reedy <tjreedy@udel.edu> wrote:
If the mirror experiment is successful, the devguide might be the next experiment. It does not have any one maintainer, and *is* tied to the tracker. But herein lies the problem with the devguide. There are 22 issues, down just 1 from about a year ago. All but 2 are more than a year old. Many (most?) have patches, but enough consensus for anyone to push is hard. As with other doc issues, there is no 'test' for when a non-trivial patch is 'good enough' and hence, in my opinion, too much bikeshedding and pursuit of the perfect.
Speaking as someone who contributed to the devguide, I think it has become good enough and have therefore largely stopped caring. Also, many requests seem to be of the "please add this thing" kind, which is a slippery slope. Regards Antoine.
On 11/30/2014 2:33 PM, Donald Stufft wrote:
So a goal of mine here is to sort of use these as a bit of a test bed. Moving CPython itself is a big and drastic change with a lot of implications, but moving the “support” repositories is not nearly as much, especially with a read only mirror on hg.python.org <http://hg.python.org> which would allow things like the PEP rendering on www.python.org <http://www.python.org> to stay the same if we wanted to. My hope was that we’d try this out, see how it works out, and if it seems to be a good thing, then at a later time we can either look at moving CPython itself or decide if it makes sense to do something different. Maybe this should be spelled out in the PEP?
I’ve seen a few people say they were -1 because they didn’t want to split between hg on the CPython side and git on the supporting repos side.
Being only recently somewhat comfortable with hg, I do not really want to have to learn git at this time.
I’m not sure you can really get away from that because we’re *already* in that situation, things like the docs building script is a Git repo on Github,
As long as I can do test doc builds with the scripts in /Docs, this does not matter to me. If the people who work with the site doc build script want it in git on Github, fine.
the python infrastructure itself is a git repo on Github,
Since I am not sure what you mean, its location does not matter to me.
the new python.org <http://python.org> website is a git repo on Github, the future PyPI is a git repo on GitHub.
Since I do not commit to either, ditto. As far as I am concerned, the people involved with specialized repositories may as well have their preference. This includes devinabox. I have made a few commits to the PEP repository, but only to the one I co-authored, 434. Using the web form might make it easier to suggest changes to other peoples' PEPs.
IOW I’m not sure what the best way forward is. I think moving to these tools for *all* repos is likely to be in the best interests of making things better for both sides of that coin however I didn’t want to go wholesale and try and make everything switch at all at once.
I agree with others that the current bottleneck is disposing of patches, especially those from non-core developers, not getting more patches to considier. I am quite sure that if we significantly reduce the current backlog of 4600 issues and more expeditiously handles the contributions we do get, will will get more. More than one person has said that they are disinclined to submit patches when they have multiple patches on the tracker already. So I think the focus should be on making better use of developer time and having more of it. -- Terry Jan Reedy
On Sun, 30 Nov 2014 19:19:50 +0000 Brett Cannon <brett@python.org> wrote:
As the PEP points out, the devguide, devinabox, and the PEPs have such a shallow development process that hosting them on Bitbucket wouldn't be a big thing. But if we don't view this as a long-term step towards moving cpython development somehow we are bifurcating our code contributors between git and hg which will be annoying. Now it could be argued that it doesn't matter for the peps and devguide since they are purely text and can be done easily through a web UI and a simple CI in Travis can be set up to make sure that the docs compile cleanly.
Web-based text editors are a poor UI so, while it *can* be done over the Web, it's not really a good thing to promote (unless we're talking mini-edits such as fixing typos). Regards Antoine.
On 11/30/2014 08:44 AM, Brett Cannon wrote:
For me personally, if I knew a simple patch integrated cleanly and passed on at least one buildbot -- when it wasn't a platform-specific fix -- then I could easily push a "Commit" button and be done with it (although this assumes single branch committing; doing this across branches makes all of this difficult unless we finally resolve our Misc/NEWS conflict issues so that in some instances it can be automated). Instead I have to wait until I have a clone I can push from, download a patch, apply it, run the unit tests myself, do the commit, and then repeat a subset of that to whatever branches make sense. It's a lot of work for which some things could be automated.
The Misc/NEWS issue could be easily solved. Mercurial allow to specify a custom merge tool for specific file. And I already succesfully wrote dedicated merge tools for file with similar issue. I've already discussed that with various people Larry, Nick, etc. And what is needed now is someone actually doing the work. Once you have such tool, you can have automatic pull request merge/rebasing through a web ui. -However-, You can only do that if you actually own the said interface. Because propriétary plateform are not going to let your run arbitrary code on their machine. -- Pierre-Yves David
On Sun, Nov 30, 2014 at 8:09 PM, Pierre-Yves David < pierre-yves.david@ens-lyon.org> wrote:
On 11/30/2014 08:44 AM, Brett Cannon wrote:
For me personally, if I knew a simple patch integrated cleanly and passed on at least one buildbot -- when it wasn't a platform-specific fix -- then I could easily push a "Commit" button and be done with it (although this assumes single branch committing; doing this across branches makes all of this difficult unless we finally resolve our Misc/NEWS conflict issues so that in some instances it can be automated). Instead I have to wait until I have a clone I can push from, download a patch, apply it, run the unit tests myself, do the commit, and then repeat a subset of that to whatever branches make sense. It's a lot of work for which some things could be automated.
The Misc/NEWS issue could be easily solved. Mercurial allow to specify a custom merge tool for specific file. And I already succesfully wrote dedicated merge tools for file with similar issue.
You might take a look at the hubflow/gitflow branching workflow diagrams? https://datasift.github.io/gitflow/IntroducingGitFlow.html (GitFlow -> Hubflow) feature/name, develop, hotfix/name, releases/v0.0.1, master I've already discussed that with various people Larry, Nick, etc. And what
is needed now is someone actually doing the work.
Once you have such tool, you can have automatic pull request merge/rebasing through a web ui. -However-, You can only do that if you actually own the said interface. Because propriétary plateform are not going to let your run arbitrary code on their machine.
On Nov 30, 2014, at 09:54 AM, Ian Cordasco wrote:
- Migrating "data" from GitHub is easy. There are free-as-in-freedom tools to do it and the only cost is the time it would take to monitor the process
*Extracting* data may be easy, but migrating it is a different story. As the Mailman project has seen in trying to migrate from Confluence to Moin, there is a ton of very difficult work involved after extracting the data. Parsing the data, ensuring that you have all the bits you need, fitting it into the new system's schema, working out the edge cases, adapting to semantic differences and gaps, ensuring that all the old links are redirected, and so on, were all exceedingly difficult[*]. Even converting between two FLOSS tools is an amazing amount of work. Look at what Eric Raymond did with reposurgeon to convert from Bazaar to git. It's a good thing that your data isn't locked behind a proprietary door, for now. That's only part of the story. But also, because github is a closed system, there's no guarantee that today's data-freeing APIs will still exist, continue to be functional for practical purposes, remain complete, or stay at parity with new features. Cheers, -Barry [*] And our huge gratitude goes to Paul Boddie for his amazing amount of work on the project.
On Nov 30, 2014, at 11:55 AM, Barry Warsaw <barry@python.org> wrote:
On Nov 30, 2014, at 09:54 AM, Ian Cordasco wrote:
- Migrating "data" from GitHub is easy. There are free-as-in-freedom tools to do it and the only cost is the time it would take to monitor the process
*Extracting* data may be easy, but migrating it is a different story. As the Mailman project has seen in trying to migrate from Confluence to Moin, there is a ton of very difficult work involved after extracting the data. Parsing the data, ensuring that you have all the bits you need, fitting it into the new system's schema, working out the edge cases, adapting to semantic differences and gaps, ensuring that all the old links are redirected, and so on, were all exceedingly difficult[*].
Even converting between two FLOSS tools is an amazing amount of work. Look at what Eric Raymond did with reposurgeon to convert from Bazaar to git.
I fail to see how this is a reasonable argument to make at all since, as you mentioned, converting between two FLOSS tools can be an amazing amount of work. Realistically the amount of work is going to be predicated on whether or not there is a tool that already handles the conversion for you. Assuming of course that the data is available to you at all. As a particularly relevant example, switching from Mercurial to Git is as easy as installing hg-git, creating a bookmark for master that tracks default, and then pushing to a git repository.
It's a good thing that your data isn't locked behind a proprietary door, for now. That's only part of the story. But also, because github is a closed system, there's no guarantee that today's data-freeing APIs will still exist, continue to be functional for practical purposes, remain complete, or stay at parity with new features.
This feels like Chicken Little-ing. If Github closed it’s APIs then you could still get at that data by scraping the web interface. However why would Github do that? That would piss off the vast majority of OSS projects who are currently hosted there and is likely to cause a pretty big migration off of Github for fear that Github is going to attempt to lock people onto Github. The popularity of Github *is* Github’s killer feature and doing something that is going to send the vast bulk of your users running for the hills sounds like something that they would have to be particularly stupid to do.
Cheers, -Barry
[*] And our huge gratitude goes to Paul Boddie for his amazing amount of work on the project. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Nov 30, 2014, at 12:09 PM, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 11:55 AM, Barry Warsaw <barry@python.org> wrote:
On Nov 30, 2014, at 09:54 AM, Ian Cordasco wrote:
- Migrating "data" from GitHub is easy. There are free-as-in-freedom tools to do it and the only cost is the time it would take to monitor the process
*Extracting* data may be easy, but migrating it is a different story. As the Mailman project has seen in trying to migrate from Confluence to Moin, there is a ton of very difficult work involved after extracting the data. Parsing the data, ensuring that you have all the bits you need, fitting it into the new system's schema, working out the edge cases, adapting to semantic differences and gaps, ensuring that all the old links are redirected, and so on, were all exceedingly difficult[*].
Even converting between two FLOSS tools is an amazing amount of work. Look at what Eric Raymond did with reposurgeon to convert from Bazaar to git.
I fail to see how this is a reasonable argument to make at all since, as you mentioned, converting between two FLOSS tools can be an amazing amount of work. Realistically the amount of work is going to be predicated on whether or not there is a tool that already handles the conversion for you. Assuming of course that the data is available to you at all.
As a particularly relevant example, switching from Mercurial to Git is as easy as installing hg-git, creating a bookmark for master that tracks default, and then pushing to a git repository.
When looking for a tool that did this (specifically Github -> Gitlab because the two are most similar) I found https://gitlab.com/sigmavirus24/issues-migration/blob/master/migrate.py which happens to be written by Ian. I would guess that he is likely speaking from experience about migrating off of Github to go to Gitlab. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Nov 30, 2014 11:09 AM, "Donald Stufft" <donald@stufft.io> wrote:
On Nov 30, 2014, at 11:55 AM, Barry Warsaw <barry@python.org> wrote:
On Nov 30, 2014, at 09:54 AM, Ian Cordasco wrote:
- Migrating "data" from GitHub is easy. There are free-as-in-freedom tools to do it and the only cost is the time it would take to monitor the process
*Extracting* data may be easy, but migrating it is a different story.
Mailman project has seen in trying to migrate from Confluence to Moin,
is a ton of very difficult work involved after extracting the data. Parsing the data, ensuring that you have all the bits you need, fitting it into
new system's schema, working out the edge cases, adapting to semantic differences and gaps, ensuring that all the old links are redirected, and so on, were all exceedingly difficult[*].
Even converting between two FLOSS tools is an amazing amount of work. Look at what Eric Raymond did with reposurgeon to convert from Bazaar to git.
I fail to see how this is a reasonable argument to make at all since, as you mentioned, converting between two FLOSS tools can be an amazing amount of work. Realistically the amount of work is going to be predicated on whether or not there is a tool that already handles the conversion for you. Assuming of course that the data is available to you at all.
As a particularly relevant example, switching from Mercurial to Git is as easy as installing hg-git, creating a bookmark for master that tracks default, and then pushing to a git repository.
It's a good thing that your data isn't locked behind a proprietary
door, for
now. That's only part of the story. But also, because github is a closed system, there's no guarantee that today's data-freeing APIs will still exist, continue to be functional for practical purposes, remain complete, or stay at parity with new features.
This feels like Chicken Little-ing. If Github closed it’s APIs then you could still get at that data by scraping the web interface. However why would Github do that? That would piss off the vast majority of OSS projects who are currently hosted there and is likely to cause a pretty big migration off of Github for fear that Github is going to attempt to lock people onto Github. The
As the there the popularity
of Github *is* Github’s killer feature and doing something that is going to send the vast bulk of your users running for the hills sounds like something that they would have to be particularly stupid to do.
Cheers, -Barry
[*] And our huge gratitude goes to Paul Boddie for his amazing amount
of work
on the project. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
I tend to agree with Donald that it's highly unlikely for GitHub to close off their APIs but Barry is right that it isn't impossible. That can be mitigated by regularly scheduling a back-up of that data using the tools we have now so that the sky doesn't appear to be falling if the worst case does occur. I also tend to disagree with Barry that it will be extraordinarily difficult because I have migrated issues and pull requests off of GitHub and was able to automate the entirety of it reliably with python. Admittedly, I'm very familiar with GitHub's API as the author of github3.py so for me this is a trivial task. I would also be willing to help set up migrations and back ups if we decide to use GitHub.
I don't feel it's my job to accept or reject this PEP, but I do have an opinion. The scope of the PSF organization is far beyond just the Python language -- it includes the Python developer community, the Python user community, 3rd party Python packages and their communities (even if some have created their own organizations). But I think that it is "scope creep" to try and be "pure" in our tooling or workflows. Python has a long history (all the way back to my choice of a MIT-style license for the first release) of mixing "free" and "non-free" uses and tools -- for example on Windows we consciously chose to align ourselves with the platform tooling rather than with the (minority) free tools available, Python has been ported to many commercial platforms, and I've always encouraged use of Python in closed-source situations. I bring this up to emphasize that (unlike GNU software and the FSF) Python has no additional hidden agenda of bringing freedom to all software. At least that's how I feel about it -- clearly some of the most vocal contributors to this thread feel differently. Now some entirely practical points. - I am basically the only remaining active PEP editor, so I see most PEP contributions by non-core-committers. Almost all of these uses github. Not bitbucket, not some other git host, but github. I spend a fair amount of time applying patches. It would most definitely be easier if I could get them to send me pull requests. - I am not worried about "lock in". The most important stuff is copied in the local git repos of hundreds of core devs and thousands of others. Pull requests are by nature short-lived -- and if you really want a history of the back-and-forth that led to the eventual set of diffs that was integrated, you could subscribe a mailing list to it to archive it. I'm sure there's a way to back up the issue tracker too. Finally. And this may actually be the most important point. Python people should be doing stuff that makes Python better (both taken in the most inclusive way possible). For stuff that's not unique to Python but can be used by many other open-source projects, such as compilers, DVCS tools, or mailing lists, we should not be wasting our precious time on building and maintaining our own tools or administering the servers on which they run. And historically we've not done a great job on maintenance and administration. Of course it's fun to make tools in Python, and to see them used beyond the Python world. But that's an entirely different argument from what I hear. -- --Guido van Rossum (python.org/~guido)
On Nov 30, 2014, at 1:05 PM, Guido van Rossum <guido@python.org> wrote:
I don't feel it's my job to accept or reject this PEP, but I do have an opinion.
So here’s a question. If it’s not your job to accept or reject this PEP, whose is it? This is probably an issue we’re never going to get actual consensus on so unless there is an arbitrator of who gets to decide I feel it’s probably a waste of my time to try and convince absolutely *everyone*.
The scope of the PSF organization is far beyond just the Python language -- it includes the Python developer community, the Python user community, 3rd party Python packages and their communities (even if some have created their own organizations). But I think that it is "scope creep" to try and be "pure" in our tooling or workflows.
Python has a long history (all the way back to my choice of a MIT-style license for the first release) of mixing "free" and "non-free" uses and tools -- for example on Windows we consciously chose to align ourselves with the platform tooling rather than with the (minority) free tools available, Python has been ported to many commercial platforms, and I've always encouraged use of Python in closed-source situations.
I bring this up to emphasize that (unlike GNU software and the FSF) Python has no additional hidden agenda of bringing freedom to all software. At least that's how I feel about it -- clearly some of the most vocal contributors to this thread feel differently.
Yes, this is how I feel about Python too, that it’s the pragmatic language/community not the purist language/community.
Now some entirely practical points.
- I am basically the only remaining active PEP editor, so I see most PEP contributions by non-core-committers. Almost all of these uses github. Not bitbucket, not some other git host, but github. I spend a fair amount of time applying patches. It would most definitely be easier if I could get them to send me pull requests.
- I am not worried about "lock in". The most important stuff is copied in the local git repos of hundreds of core devs and thousands of others. Pull requests are by nature short-lived -- and if you really want a history of the back-and-forth that led to the eventual set of diffs that was integrated, you could subscribe a mailing list to it to archive it. I'm sure there's a way to back up the issue tracker too.
Finally. And this may actually be the most important point. Python people should be doing stuff that makes Python better (both taken in the most inclusive way possible). For stuff that's not unique to Python but can be used by many other open-source projects, such as compilers, DVCS tools, or mailing lists, we should not be wasting our precious time on building and maintaining our own tools or administering the servers on which they run. And historically we've not done a great job on maintenance and administration.
Of course it's fun to make tools in Python, and to see them used beyond the Python world. But that's an entirely different argument from what I hear.
-- --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Sun, Nov 30, 2014 at 10:25 AM, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 1:05 PM, Guido van Rossum <guido@python.org> wrote:
I don't feel it's my job to accept or reject this PEP, but I do have an opinion.
So here’s a question. If it’s not your job to accept or reject this PEP, whose is it? This is probably an issue we’re never going to get actual consensus on so unless there is an arbitrator of who gets to decide I feel it’s probably a waste of my time to try and convince absolutely *everyone*.
I saved this question. I still don't know who should accept or reject the PEP. I tried to get out of it by asking Brett for the two repos he "owns", but he hasn't stated his preference (though he did acknowledge the responsibility). If it were really up to me I'd switch all "minor" repos to GitHub, but I feel I've run into sufficient opposition (most vocally from Nick) that I think "status quo wins" applies. I think Nick previously wanted to switch to BitBucket -- if he hasn't hardened his position I say we should do that. But if he no longer wants that, I have stopped caring after the 200th message. -- --Guido van Rossum (python.org/~guido)
On 11/30/2014 10:05 AM, Guido van Rossum wrote:
Python has a long history (all the way back to my choice of a MIT-style license for the first release) of mixing "free" and "non-free" uses and tools -- for example on Windows we consciously chose to align ourselves with the platform tooling rather than with the (minority) free tools available, Python has been ported to many commercial platforms, and I've always encouraged use of Python in closed-source situations.
For this I am grateful, and agree with.
Finally. And this may actually be the most important point. Python people should be doing stuff that makes Python better (both taken in the most inclusive way possible). For stuff that's not unique to Python but can be used by many other open-source projects, such as compilers, DVCS tools, or mailing lists, we should not be wasting our precious time on building and maintaining our own tools or administering the servers on which they run. And historically we've not done a great job on maintenance and administration.
My issues with GitHub range from selfish to philosophical: - (selfish) I don't want to learn git - (practical) from what I hear git can have issues with losing history -- in a project that has many volunteer and part-time developers, using a tool that can munge your data just doesn't seem very wise - (practical) supporting git and hg means learning two different workflows - (philosophical) in a commercial world we vote with our dollars (don't like how a company behaves? don't buy their product); in an OSS world we vote by whose services/software we use; I don't want to support, or appear to support, a company that is abusive and sexist towards its employees: it is not what the PSF supports, and it's definitely not what I support. Not everyone is suited to demonstrate in the streets, but it shouldn't be that hard to not use a company with acknowledged bad practices. -- ~Ethan~
On Nov 30, 2014, at 2:28 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 11/30/2014 10:05 AM, Guido van Rossum wrote:
Python has a long history (all the way back to my choice of a MIT-style license for the first release) of mixing "free" and "non-free" uses and tools -- for example on Windows we consciously chose to align ourselves with the platform tooling rather than with the (minority) free tools available, Python has been ported to many commercial platforms, and I've always encouraged use of Python in closed-source situations.
For this I am grateful, and agree with.
Finally. And this may actually be the most important point. Python people should be doing stuff that makes Python better (both taken in the most inclusive way possible). For stuff that's not unique to Python but can be used by many other open-source projects, such as compilers, DVCS tools, or mailing lists, we should not be wasting our precious time on building and maintaining our own tools or administering the servers on which they run. And historically we've not done a great job on maintenance and administration.
My issues with GitHub range from selfish to philosophical:
- (selfish) I don't want to learn git
Note: That you don’t actually have to learn git, you can clone a git repository with Mercurial using hg-git and continue to use Mercurial locally. The same of course can be said for the *other* way, but I’d argue that putting the burden of using things like hg-git or git-remote-hg on the less popular tool is a better overall decision.
- (practical) from what I hear git can have issues with losing history -- in a project that has many volunteer and part-time developers, using a tool that can munge your data just doesn't seem very wise
I have never heard of git losing history. Git goes out of it’s way not to lose things. Even unreferences commits don’t go away for months unless you purposely prune them. I’d personally call this FUD unless there’s some citation here.
- (practical) supporting git and hg means learning two different workflows
As I mentioned in my other email, we’re already supporting two different tools, and it’s a hope of mine to use this as a sort of testbed to moving the other repositories as well.
- (philosophical) in a commercial world we vote with our dollars (don't like how a company behaves? don't buy their product); in an OSS world we vote by whose services/software we use; I don't want to support, or appear to support, a company that is abusive and sexist towards its employees: it is not what the PSF supports, and it's definitely not what I support.
I’m assuming this is about Github. I’ll say that Github has, at least publicly, made steps towards doing better than they had previously there. I’m not a Github employee so I can’t speak towards that. It almost feels like there is some amount of co-opting this incident as a shield to hide behind. Most people who make this statement are more than happy to continue to use Linux for example, even though Linus is well documented being extremely abusive towards people and has more or less said that he’s never going to change that. I also think it’s hard to look at a company like bitbucket, for example, and say they are *better* than Github just because they didn’t have a public and inflammatory event. Attempting to reduce the cognitive burden for contributing and aligning ourselves with the most popular tools allows us to take advantage of the network effects of these tools popularity. This can be the difference between someone with limited amount of time being able to contribute or not, which can make real inroads towards making it easier for under privileged people to contribute much more than refusing to use a product of one group of people over another just because the other group hasn’t had a public and inflammatory event.
Not everyone is suited to demonstrate in the streets, but it shouldn't be that hard to not use a company with acknowledged bad practices.
-- ~Ethan~
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 11/30/2014 11:56 AM, Donald Stufft wrote:
On Nov 30, 2014, at 2:28 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
My issues with GitHub range from selfish to philosophical:
- (selfish) I don't want to learn git
Note: That you don’t actually have to learn git, you can clone a git repository with Mercurial using hg-git and continue to use Mercurial locally. The same of course can be said for the *other* way, but I’d argue that putting the burden of using things like hg-git or git-remote-hg on the less popular tool is a better overall decision.
Fair enough.
- (practical) from what I hear git can have issues with losing history -- in a project that has many volunteer and part-time developers, using a tool that can munge your data just doesn't seem very wise
I have never heard of git losing history. Git goes out of it’s way not to lose things. Even unreferences commits don’t go away for months unless you purposely prune them. I’d personally call this FUD unless there’s some citation here.
Okay.
- (practical) supporting git and hg means learning two different workflows
As I mentioned in my other email, we’re already supporting two different tools, and it’s a hope of mine to use this as a sort of testbed to moving the other repositories as well.
That should be in the PEP then.
- (philosophical) in a commercial world we vote with our dollars (don't like how a company behaves? don't buy their product); in an OSS world we vote by whose services/software we use; I don't want to support, or appear to support, a company that is abusive and sexist towards its employees: it is not what the PSF supports, and it's definitely not what I support.
I’m assuming this is about Github.
Yes.
I’ll say that Github has, at least publicly, made steps towards doing better than they had previously there. I’m not a Github employee so I can’t speak towards that.
It almost feels like there is some amount of co-opting this incident as a shield to hide behind. Most people who make this statement are more than happy to continue to use Linux for example, even though Linus is well documented being extremely abusive towards people and has more or less said that he’s never going to change that.
Linux is not my choice. ;) Linus is also one person, not an entire company.
I also think it’s hard to look at a company like bitbucket, for example, and say they are *better* than Github just because they didn’t have a public and inflammatory event.
In cases like this it's not "better" but "willing to work with". BitBucket might be as hostile as GitHub, only someone who has worked for both could really know.
[...] than refusing to use a product of one group of people over another just because the other group hasn’t had a public and inflammatory event.
We can only make decisions on information we have; pretending we don't have it, or that some other company /could/ be the same, is hiding our heads in the sand. If git is the wave of the future, there are other git hosts besides GitHub. -- ~Ethan~
Donald Stufft <donald@stufft.io> writes:
I have never heard of git losing history.
In my experience talking with Git users about this problem, that depends on a very narrow definition of “losing history”. Git encourages re-writing, and thereby losing prior versions of, the history of a branch. The commit information remains, but the history of how they link together is lost. That is a loss of information, which is not the case in the absence of such history re-writing. Git users differ in whether they consider that information loss important; but it is, objectively, losing history information. So Ethan's impression is correct on this point. -- \ “If you see an animal and you can't tell if it's a skunk or a | `\ cat, here's a good saying to help: ‘Black and white, stinks all | _o__) right. Tabby-colored, likes a fella.’” —Jack Handey | Ben Finney
On Nov 30, 2014, at 7:17 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
Donald Stufft <donald@stufft.io> writes:
I have never heard of git losing history.
In my experience talking with Git users about this problem, that depends on a very narrow definition of “losing history”.
Git encourages re-writing, and thereby losing prior versions of, the history of a branch. The commit information remains, but the history of how they link together is lost. That is a loss of information, which is not the case in the absence of such history re-writing.
Git users differ in whether they consider that information loss important; but it is, objectively, losing history information. So Ethan's impression is correct on this point.
It’s not lost, the only thing that’s “gone” is a pointer to the HEAD commit of that branch. Each commit points to it’s parent commit so if you find the HEAD and give it a name you’ll restore the branch. It just so happens inside the reflog you’ll see a list of the old HEADs of branches so you can get the old commit ID from the HEAD there. In addition depending on how you rewrote the branch and if you did anything else there is likely a reference to the old head at ORIG_HEAD. If you don’t have the reflog (this is per copy of the repository, so a different computer or deleting the repo and recreating it will lose it) and for similar reasons you don’t have the ORIG_HEAD, if you have any reference to the previous HEAD (email, commit messages, whatever) that’s enough to restore it assuming that the commits have not been garbage collected yet (which happens in 90 days or 30 days depending on what kind of unreferenced commit it is) you can restore it. The important thing to realize is that a “branch” isn’t anything special in git. All a branch does is act as a sort of symlink to a commit ID. Anything more beyond “what is the HEAD commit in this branch” is stored as part of the commits themselves and doesn’t rely on the branch to be named. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Donald Stufft <donald@stufft.io> writes:
It’s not lost, [… a long, presumably-accurate discourse of the many conditions that must be met before …] you can restore it.
This isn't the place to discuss the details of Git's internals, I think. I'm merely pointing out that:
The important thing to realize is that a “branch” isn’t anything special in git.
Because of that, Ethan's impression – that Git's default behaviour encourages losing history (by re-writing the history of commits to be other than what they were) is true, and “Git never loses history” simply isn't true. Whether that is a *problem* is a matter of debate, but the fact that Git's common workflow commonly discards information that some consider valuable, is a simple fact. If Ethan chooses to make that a factor in his decisions about Git, the facts are on his side. -- \ “One of the most important things you learn from the internet | `\ is that there is no ‘them’ out there. It's just an awful lot of | _o__) ‘us’.” —Douglas Adams | Ben Finney
On Nov 30, 2014, at 7:43 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
Donald Stufft <donald@stufft.io> writes:
It’s not lost, [… a long, presumably-accurate discourse of the many conditions that must be met before …] you can restore it.
This isn't the place to discuss the details of Git's internals, I think. I'm merely pointing out that:
The important thing to realize is that a “branch” isn’t anything special in git.
Because of that, Ethan's impression – that Git's default behaviour encourages losing history (by re-writing the history of commits to be other than what they were) is true, and “Git never loses history” simply isn't true.
Whether that is a *problem* is a matter of debate, but the fact that Git's common workflow commonly discards information that some consider valuable, is a simple fact.
If Ethan chooses to make that a factor in his decisions about Git, the facts are on his side.
Except it’s not true at all. That data is all still there if you want it to exist and it’s not a real differentiator between Mercurial and git because Mercurial has the ability to do the same thing. Never mind the fact that “lose” your history makes it sound accidental instead of on purpose. It’s like saying that ``rm foo.txt`` will “lose” the data in foo.txt. So either it was a misunderstanding in which case I wanted to point out that those operations don’t magically lose information or it’s a purposely FUDish statement in which case I want to point out that the statement is inaccurate. The only thing that is true is that git users are more likely to use the ability to rewrite history than Mercurial users are, but you’ll typically find that people generally don’t do this on public branches, only on private branches. Which again doesn’t make much sense in this context since generally currently the way people are using Mercurial with CPython you’re using patches to transfer the changes from the contributor to the committer so you’re “losing” that history regardless. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Can we please stop the hg-vs-git discussion? We've established earlier that the capabilities of the DVCS itself (hg or git) are not a differentiator, and further he-said-she-said isn't going to change anybody's opinion. What's left is preferences of core developers, possibly capabilities of the popular websites (though BitBucket vs. GitHub seems to be a wash as well), and preferences of contributors who aren't core developers (using popularity as a proxy). It seems the preferences of the core developers are mixed, while the preferences of non-core contributors are pretty clear, so we have a problem weighing these two appropriately. Also, let's not get distracted by the needs of the CPython repo, issue tracker, and code review tool. Arguments about core developers vs. contributors for CPython shouldn't affect the current discussion. Next, two of the three repos mentioned in Donald's PEP 481 are owned by Brett Cannon, according to the Contact column listed on hg.python.org. I propose to let Brett choose whether to keep these on hg.python.org, move to BitBucket, or move to GitHub. @Brett, what say you? (Apart from "I'm tired of the whole thread." :-) The third one is the peps repo, which has python-dev@python.org as Contact. It turns out that Nick is by far the largest contributor (he committed 215 of the most recent 1000 changes) so I'll let him choose. Finally, I'd like to get a few more volunteers for the PEP editors list, preferably non-core devs: the core devs are already spread too thinly, and I really shouldn't be the one who picks new PEP numbers and checks that PEPs are well-formed according to the rules of PEP 1. A PEP editor shouldn't have to pass judgment on the contents of a PEP (though they may choose to correct spelling and grammar). Knowledge of Mercurial is a plus. :-) On Sun, Nov 30, 2014 at 4:50 PM, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 7:43 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
Donald Stufft <donald@stufft.io> writes:
It’s not lost, [… a long, presumably-accurate discourse of the many conditions that must be met before …] you can restore it.
This isn't the place to discuss the details of Git's internals, I think. I'm merely pointing out that:
The important thing to realize is that a “branch” isn’t anything special in git.
Because of that, Ethan's impression – that Git's default behaviour encourages losing history (by re-writing the history of commits to be other than what they were) is true, and “Git never loses history” simply isn't true.
Whether that is a *problem* is a matter of debate, but the fact that Git's common workflow commonly discards information that some consider valuable, is a simple fact.
If Ethan chooses to make that a factor in his decisions about Git, the facts are on his side.
Except it’s not true at all.
That data is all still there if you want it to exist and it’s not a real differentiator between Mercurial and git because Mercurial has the ability to do the same thing. Never mind the fact that “lose” your history makes it sound accidental instead of on purpose. It’s like saying that ``rm foo.txt`` will “lose” the data in foo.txt. So either it was a misunderstanding in which case I wanted to point out that those operations don’t magically lose information or it’s a purposely FUDish statement in which case I want to point out that the statement is inaccurate.
The only thing that is true is that git users are more likely to use the ability to rewrite history than Mercurial users are, but you’ll typically find that people generally don’t do this on public branches, only on private branches. Which again doesn’t make much sense in this context since generally currently the way people are using Mercurial with CPython you’re using patches to transfer the changes from the contributor to the committer so you’re “losing” that history regardless.
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido)
On Nov 30, 2014, at 8:24 PM, Guido van Rossum <guido@python.org> wrote:
Can we please stop the hg-vs-git discussion? We've established earlier that the capabilities of the DVCS itself (hg or git) are not a differentiator, and further he-said-she-said isn't going to change anybody's opinion.
What's left is preferences of core developers, possibly capabilities of the popular websites (though BitBucket vs. GitHub seems to be a wash as well), and preferences of contributors who aren't core developers (using popularity as a proxy). It seems the preferences of the core developers are mixed, while the preferences of non-core contributors are pretty clear, so we have a problem weighing these two appropriately.
Also, let's not get distracted by the needs of the CPython repo, issue tracker, and code review tool. Arguments about core developers vs. contributors for CPython shouldn't affect the current discussion.
Next, two of the three repos mentioned in Donald's PEP 481 are owned by Brett Cannon, according to the Contact column listed on hg.python.org <http://hg.python.org/>. I propose to let Brett choose whether to keep these on hg.python.org <http://hg.python.org/>, move to BitBucket, or move to GitHub. @Brett, what say you? (Apart from "I'm tired of the whole thread." :-)
The third one is the peps repo, which has python-dev@python.org <mailto:python-dev@python.org> as Contact. It turns out that Nick is by far the largest contributor (he committed 215 of the most recent 1000 changes) so I'll let him choose.
Finally, I'd like to get a few more volunteers for the PEP editors list, preferably non-core devs: the core devs are already spread too thinly, and I really shouldn't be the one who picks new PEP numbers and checks that PEPs are well-formed according to the rules of PEP 1. A PEP editor shouldn't have to pass judgment on the contents of a PEP (though they may choose to correct spelling and grammar). Knowledge of Mercurial is a plus. :-)
I’m not sure if it got lost in the discussion or if it was purposely left out. However I did come up with another idea, where we enable people to make PRs against these repositories with PR integration within roundup. Using the fact that it’s trivial to turn a PR into a patch core contributors (and the “single source of truth”) for the repositories can remain Mercurial with core contributors needing to download a .patch file from Github instead of a .patch from from Roundup. This could allow non-committers to use git if they want, including PRs but without moving things around. The obvious cost is that since the committer side of things is still using the existing tooling there’s no “Merge button” or the other committer benefits of Github, it would strictly be enabling people who aren’t committing directly to the repository to use git and Github. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Sun, Nov 30, 2014 at 6:40 PM, Donald Stufft <donald@stufft.io> wrote:
I’m not sure if it got lost in the discussion or if it was purposely left out. However I did come up with another idea, where we enable people to make PRs against these repositories with PR integration within roundup. Using the fact that it’s trivial to turn a PR into a patch core contributors (and the “single source of truth”) for the repositories can remain Mercurial with core contributors needing to download a .patch file from Github instead of a .patch from from Roundup. This could allow non-committers to use git if they want, including PRs but without moving things around.
Hah. I just had a similar idea.
The obvious cost is that since the committer side of things is still using the existing tooling there’s no “Merge button” or the other committer benefits of Github, it would strictly be enabling people who aren’t committing directly to the repository to use git and Github.
This is not an added cost. It's just the status quo and something that can be addressed separately. -eric
On Sun Nov 30 2014 at 8:25:25 PM Guido van Rossum <guido@python.org> wrote:
Can we please stop the hg-vs-git discussion? We've established earlier that the capabilities of the DVCS itself (hg or git) are not a differentiator, and further he-said-she-said isn't going to change anybody's opinion.
+1 from me as well. I view this as a discussion of platforms and not DVCSs.
What's left is preferences of core developers, possibly capabilities of the popular websites (though BitBucket vs. GitHub seems to be a wash as well), and preferences of contributors who aren't core developers (using popularity as a proxy). It seems the preferences of the core developers are mixed, while the preferences of non-core contributors are pretty clear, so we have a problem weighing these two appropriately.
Also, let's not get distracted by the needs of the CPython repo, issue tracker, and code review tool. Arguments about core developers vs. contributors for CPython shouldn't affect the current discussion.
Next, two of the three repos mentioned in Donald's PEP 481 are owned by Brett Cannon, according to the Contact column listed on hg.python.org. I propose to let Brett choose whether to keep these on hg.python.org, move to BitBucket, or move to GitHub. @Brett, what say you? (Apart from "I'm tired of the whole thread." :-)
You do one or two nice things for python-dev and you end up being saddled with them for life. ;) Sure, I can handle the devguide and devinabox decisions since someone has to and it isn't going to be more "fun" for someone else compared to me.
The third one is the peps repo, which has python-dev@python.org as Contact. It turns out that Nick is by far the largest contributor (he committed 215 of the most recent 1000 changes) so I'll let him choose.
"Perk" of all those packaging PEPs.
Finally, I'd like to get a few more volunteers for the PEP editors list, preferably non-core devs: the core devs are already spread too thinly, and I really shouldn't be the one who picks new PEP numbers and checks that PEPs are well-formed according to the rules of PEP 1. A PEP editor shouldn't have to pass judgment on the contents of a PEP (though they may choose to correct spelling and grammar). Knowledge of Mercurial is a plus. :-)
And based on how Nick has been talking, will continue to be at least in the medium term. =) -Brett
On Sun, Nov 30, 2014 at 4:50 PM, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 7:43 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
Donald Stufft <donald@stufft.io> writes:
It’s not lost, [… a long, presumably-accurate discourse of the many conditions that must be met before …] you can restore it.
This isn't the place to discuss the details of Git's internals, I think. I'm merely pointing out that:
The important thing to realize is that a “branch” isn’t anything special in git.
Because of that, Ethan's impression – that Git's default behaviour encourages losing history (by re-writing the history of commits to be other than what they were) is true, and “Git never loses history” simply isn't true.
Whether that is a *problem* is a matter of debate, but the fact that Git's common workflow commonly discards information that some consider valuable, is a simple fact.
If Ethan chooses to make that a factor in his decisions about Git, the facts are on his side.
Except it’s not true at all.
That data is all still there if you want it to exist and it’s not a real differentiator between Mercurial and git because Mercurial has the ability to do the same thing. Never mind the fact that “lose” your history makes it sound accidental instead of on purpose. It’s like saying that ``rm foo.txt`` will “lose” the data in foo.txt. So either it was a misunderstanding in which case I wanted to point out that those operations don’t magically lose information or it’s a purposely FUDish statement in which case I want to point out that the statement is inaccurate.
The only thing that is true is that git users are more likely to use the ability to rewrite history than Mercurial users are, but you’ll typically find that people generally don’t do this on public branches, only on private branches. Which again doesn’t make much sense in this context since generally currently the way people are using Mercurial with CPython you’re using patches to transfer the changes from the contributor to the committer so you’re “losing” that history regardless.
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido)
As far as I'm concerned I'm just waiting for your decision now. On Mon, Dec 1, 2014 at 7:07 AM, Brett Cannon <brett@python.org> wrote:
On Sun Nov 30 2014 at 8:25:25 PM Guido van Rossum <guido@python.org> wrote:
Can we please stop the hg-vs-git discussion? We've established earlier that the capabilities of the DVCS itself (hg or git) are not a differentiator, and further he-said-she-said isn't going to change anybody's opinion.
+1 from me as well. I view this as a discussion of platforms and not DVCSs.
What's left is preferences of core developers, possibly capabilities of the popular websites (though BitBucket vs. GitHub seems to be a wash as well), and preferences of contributors who aren't core developers (using popularity as a proxy). It seems the preferences of the core developers are mixed, while the preferences of non-core contributors are pretty clear, so we have a problem weighing these two appropriately.
Also, let's not get distracted by the needs of the CPython repo, issue tracker, and code review tool. Arguments about core developers vs. contributors for CPython shouldn't affect the current discussion.
Next, two of the three repos mentioned in Donald's PEP 481 are owned by Brett Cannon, according to the Contact column listed on hg.python.org. I propose to let Brett choose whether to keep these on hg.python.org, move to BitBucket, or move to GitHub. @Brett, what say you? (Apart from "I'm tired of the whole thread." :-)
You do one or two nice things for python-dev and you end up being saddled with them for life. ;)
Sure, I can handle the devguide and devinabox decisions since someone has to and it isn't going to be more "fun" for someone else compared to me.
The third one is the peps repo, which has python-dev@python.org as Contact. It turns out that Nick is by far the largest contributor (he committed 215 of the most recent 1000 changes) so I'll let him choose.
"Perk" of all those packaging PEPs.
Finally, I'd like to get a few more volunteers for the PEP editors list, preferably non-core devs: the core devs are already spread too thinly, and I really shouldn't be the one who picks new PEP numbers and checks that PEPs are well-formed according to the rules of PEP 1. A PEP editor shouldn't have to pass judgment on the contents of a PEP (though they may choose to correct spelling and grammar). Knowledge of Mercurial is a plus. :-)
And based on how Nick has been talking, will continue to be at least in the medium term. =)
-Brett
On Sun, Nov 30, 2014 at 4:50 PM, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 7:43 PM, Ben Finney <ben+python@benfinney.id.au> wrote:
Donald Stufft <donald@stufft.io> writes:
It’s not lost, [… a long, presumably-accurate discourse of the many conditions that must be met before …] you can restore it.
This isn't the place to discuss the details of Git's internals, I think. I'm merely pointing out that:
The important thing to realize is that a “branch” isn’t anything special in git.
Because of that, Ethan's impression – that Git's default behaviour encourages losing history (by re-writing the history of commits to be other than what they were) is true, and “Git never loses history” simply isn't true.
Whether that is a *problem* is a matter of debate, but the fact that Git's common workflow commonly discards information that some consider valuable, is a simple fact.
If Ethan chooses to make that a factor in his decisions about Git, the facts are on his side.
Except it’s not true at all.
That data is all still there if you want it to exist and it’s not a real differentiator between Mercurial and git because Mercurial has the ability to do the same thing. Never mind the fact that “lose” your history makes it sound accidental instead of on purpose. It’s like saying that ``rm foo.txt`` will “lose” the data in foo.txt. So either it was a misunderstanding in which case I wanted to point out that those operations don’t magically lose information or it’s a purposely FUDish statement in which case I want to point out that the statement is inaccurate.
The only thing that is true is that git users are more likely to use the ability to rewrite history than Mercurial users are, but you’ll typically find that people generally don’t do this on public branches, only on private branches. Which again doesn’t make much sense in this context since generally currently the way people are using Mercurial with CPython you’re using patches to transfer the changes from the contributor to the committer so you’re “losing” that history regardless.
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido)
-- --Guido van Rossum (python.org/~guido)
On 2 December 2014 at 01:38, Guido van Rossum <guido@python.org> wrote:
As far as I'm concerned I'm just waiting for your decision now.
The RhodeCode team got in touch with me offline to suggest the possibility of using RhodeCode Enterprise as a self-hosted solution rather than a volunteer-supported installation of Kallithea. I'll be talking to them tomorrow, and if that discussion goes well, will update PEP 474 (and potentially PEP 462) accordingly. Given that that would take away the "volunteer supported" vs "commercially supported" distinction between self-hosting and using GitHub (as well as potentially building a useful relationship that may help us resolve other workflow issues in the future), I'd like us to hold off on any significant decisions regarding the fate of any of the repos until I've had a chance to incorporate the results of that discussion into my proposals. As described in PEP 474, I'm aware of the Mercurial team's concerns with RhodeCode's current licensing, but still consider it a superior alternative to an outright proprietary solution that doesn't get us any closer to solving the workflow problems with the main CPython repo. Regards, Nick. P.S. I'll also bring up some of the RFEs raised in this discussion around making it possible for folks to submit pull requests via GitHub/BitBucket, even if the master repositories are hosted on PSF infrastructure. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
So I was waiting for Nick to say what he wanted to do for the peps repo since I view it as I get 2/3 of the choices and he gets the other third. The way I view it, the options are: 1. Move to GitHub 2. Move to Bitbucket 3. Improve our current tooling (either through new hosting setup and/or adding first-world support for downloading PRs from GitHub/Bitbucket) Regardless of what we do, I think we should graduate the mirrors on GitHub and Bitbucket to "official" -- for the proposed repos and cpython -- and get their repos updating per-push instead of as a cron job. I also think we should also flip on any CI we can (e.g. turn on Travis for GitHub along with coveralls support using coverage.py's encodings trick <https://hg.python.org/devinabox/file/1eeb96fe98f1/README#l124>). This will get us the most accessible repo backups as well as the widest tool coverage for contributors to assist them in their contributions (heck, even if we just get regular coverage reports for Python that would be a great win out of all of this). Now as for whether we should move the repos, I see two possibilities to help make that decision. One is we end up with 3 PEPs corresponding to the 3 proposals outlined above, get them done before PyCon, and then we have a discussion at the language summit where we can either make a decision or see what the pulse at the conference and sprints then make a decision shortly thereafter (I can moderate the summit discussion to keep this on-task and minimize the rambling; if Guido wants I can even make the final call since I have already played the role of "villain" for our issue tracker and hg decisions). The other option is we take each one of the 3 proposed repos and pilot/experiment with them on a different platform. I would put peps on GitHub (as per Guido's comment of getting PRs from there already), the devguide on Bitbucket, and leave devinabox on hg.python.org but with the motivation of getting better tooling in place to contribute to it. We can then see if anything changes between now and PyCon and then discuss what occurred there (if we can't get the word out about this experiment and get new tooling up and going on the issue tracker in the next 4 months then that's another data point about how much people do/don't care about any of this). Obviously if we end up needing more time we don't *have* to make a decision at PyCon, but it's a good goal to have. I don't think we can cleanly replicate a single repo on all three solutions as I sure don't want to deal with that merging fun (unless someone comes forward to be basically a "release manager" for one of the repos to make that experiment happen). So do people want PEPs or experimentation first? On Tue Dec 02 2014 at 8:24:16 AM Nick Coghlan <ncoghlan@gmail.com> wrote:
On 2 December 2014 at 01:38, Guido van Rossum <guido@python.org> wrote:
As far as I'm concerned I'm just waiting for your decision now.
The RhodeCode team got in touch with me offline to suggest the possibility of using RhodeCode Enterprise as a self-hosted solution rather than a volunteer-supported installation of Kallithea. I'll be talking to them tomorrow, and if that discussion goes well, will update PEP 474 (and potentially PEP 462) accordingly.
Given that that would take away the "volunteer supported" vs "commercially supported" distinction between self-hosting and using GitHub (as well as potentially building a useful relationship that may help us resolve other workflow issues in the future), I'd like us to hold off on any significant decisions regarding the fate of any of the repos until I've had a chance to incorporate the results of that discussion into my proposals.
As described in PEP 474, I'm aware of the Mercurial team's concerns with RhodeCode's current licensing, but still consider it a superior alternative to an outright proprietary solution that doesn't get us any closer to solving the workflow problems with the main CPython repo.
Regards, Nick.
P.S. I'll also bring up some of the RFEs raised in this discussion around making it possible for folks to submit pull requests via GitHub/BitBucket, even if the master repositories are hosted on PSF infrastructure.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12/02/2014 11:50 AM, Brett Cannon wrote:
So do people want PEPs or experimentation first?
I'd vote for experimentation, to ground the discussion in actual practice. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iEYEARECAAYFAlR99X8ACgkQ+gerLs4ltQ7dpACgsGq7Rii7seJXHCOVMUymbOdL 2KQAn3qcOGWynKU4rd/H39hpBxwSsbk9 =93kJ -----END PGP SIGNATURE-----
On Tue, Dec 2, 2014 at 9:23 AM, Tres Seaver <tseaver@palladion.com> wrote:
I'd vote for experimentation, to ground the discussion in actual practice.
+1. There may be a number of practical gotchas that very well might not surface in PEPs and should be documented and planned for. Likewise with benefits.
Thanks for taking charge, Brett. I personally think this shouldn't be brought up at the summit -- it's likely to just cause lots of heat about git vs. hg, free vs. not-free, "loyalty" to free or open tools, the weighing of core committers' preferences vs. outside contributors' preferences, GitHub's diversity track record, with no new information added. Even if we *just* had a vote by show-of-hands at the summit that would just upset those who couldn't be present. But I'll leave that up to you. The only thing I ask you is not to give me the last word. I might just do something you regret. :-) --Guido On Tue, Dec 2, 2014 at 8:50 AM, Brett Cannon <brett@python.org> wrote:
So I was waiting for Nick to say what he wanted to do for the peps repo since I view it as I get 2/3 of the choices and he gets the other third.
The way I view it, the options are:
1. Move to GitHub 2. Move to Bitbucket 3. Improve our current tooling (either through new hosting setup and/or adding first-world support for downloading PRs from GitHub/Bitbucket)
Regardless of what we do, I think we should graduate the mirrors on GitHub and Bitbucket to "official" -- for the proposed repos and cpython -- and get their repos updating per-push instead of as a cron job. I also think we should also flip on any CI we can (e.g. turn on Travis for GitHub along with coveralls support using coverage.py's encodings trick <https://hg.python.org/devinabox/file/1eeb96fe98f1/README#l124>). This will get us the most accessible repo backups as well as the widest tool coverage for contributors to assist them in their contributions (heck, even if we just get regular coverage reports for Python that would be a great win out of all of this).
Now as for whether we should move the repos, I see two possibilities to help make that decision. One is we end up with 3 PEPs corresponding to the 3 proposals outlined above, get them done before PyCon, and then we have a discussion at the language summit where we can either make a decision or see what the pulse at the conference and sprints then make a decision shortly thereafter (I can moderate the summit discussion to keep this on-task and minimize the rambling; if Guido wants I can even make the final call since I have already played the role of "villain" for our issue tracker and hg decisions).
The other option is we take each one of the 3 proposed repos and pilot/experiment with them on a different platform. I would put peps on GitHub (as per Guido's comment of getting PRs from there already), the devguide on Bitbucket, and leave devinabox on hg.python.org but with the motivation of getting better tooling in place to contribute to it. We can then see if anything changes between now and PyCon and then discuss what occurred there (if we can't get the word out about this experiment and get new tooling up and going on the issue tracker in the next 4 months then that's another data point about how much people do/don't care about any of this). Obviously if we end up needing more time we don't *have* to make a decision at PyCon, but it's a good goal to have. I don't think we can cleanly replicate a single repo on all three solutions as I sure don't want to deal with that merging fun (unless someone comes forward to be basically a "release manager" for one of the repos to make that experiment happen).
So do people want PEPs or experimentation first?
On Tue Dec 02 2014 at 8:24:16 AM Nick Coghlan <ncoghlan@gmail.com> wrote:
On 2 December 2014 at 01:38, Guido van Rossum <guido@python.org> wrote:
As far as I'm concerned I'm just waiting for your decision now.
The RhodeCode team got in touch with me offline to suggest the possibility of using RhodeCode Enterprise as a self-hosted solution rather than a volunteer-supported installation of Kallithea. I'll be talking to them tomorrow, and if that discussion goes well, will update PEP 474 (and potentially PEP 462) accordingly.
Given that that would take away the "volunteer supported" vs "commercially supported" distinction between self-hosting and using GitHub (as well as potentially building a useful relationship that may help us resolve other workflow issues in the future), I'd like us to hold off on any significant decisions regarding the fate of any of the repos until I've had a chance to incorporate the results of that discussion into my proposals.
As described in PEP 474, I'm aware of the Mercurial team's concerns with RhodeCode's current licensing, but still consider it a superior alternative to an outright proprietary solution that doesn't get us any closer to solving the workflow problems with the main CPython repo.
Regards, Nick.
P.S. I'll also bring up some of the RFEs raised in this discussion around making it possible for folks to submit pull requests via GitHub/BitBucket, even if the master repositories are hosted on PSF infrastructure.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-- --Guido van Rossum (python.org/~guido)
On Tue Dec 02 2014 at 1:05:22 PM Guido van Rossum <guido@python.org> wrote:
Thanks for taking charge, Brett.
I personally think this shouldn't be brought up at the summit -- it's likely to just cause lots of heat about git vs. hg, free vs. not-free, "loyalty" to free or open tools, the weighing of core committers' preferences vs. outside contributors' preferences, GitHub's diversity track record, with no new information added. Even if we *just* had a vote by show-of-hands at the summit that would just upset those who couldn't be present.
Well, if I'm going to be the Great Decider on this then I can say upfront I'm taking a pragmatic view of preferring open but not mandating it, preferring hg over git but not ruling out a switch, preferring Python-based tools but not viewing it as a negative to not use Python, etc. I would like to think I have earned somewhat of a reputation of being level-headed and so none of this should really be a surprise to anyone. So if we did have a discussion at the summit and someone decided to argue for FLOSS vs. not as a key factor then I would politely cut them off and say that doesn't matter to me and move on. As I said, I would moderate the conversation to keep it on-task and not waste my time with points that have already been made and flagged by me and you as not deal-breakers. And any votes would be to gauge the feeling of the room and not as a binding decision; I assume either me or someone else is going to be the dictator on this and this won't be a majority decision.
But I'll leave that up to you. The only thing I ask you is not to give me the last word. I might just do something you regret. :-)
What about me doing something that *I* regret like taking this on? =) -Brett
--Guido
On Tue, Dec 2, 2014 at 8:50 AM, Brett Cannon <brett@python.org> wrote:
So I was waiting for Nick to say what he wanted to do for the peps repo since I view it as I get 2/3 of the choices and he gets the other third.
The way I view it, the options are:
1. Move to GitHub 2. Move to Bitbucket 3. Improve our current tooling (either through new hosting setup and/or adding first-world support for downloading PRs from GitHub/Bitbucket)
Regardless of what we do, I think we should graduate the mirrors on GitHub and Bitbucket to "official" -- for the proposed repos and cpython -- and get their repos updating per-push instead of as a cron job. I also think we should also flip on any CI we can (e.g. turn on Travis for GitHub along with coveralls support using coverage.py's encodings trick <https://hg.python.org/devinabox/file/1eeb96fe98f1/README#l124>). This will get us the most accessible repo backups as well as the widest tool coverage for contributors to assist them in their contributions (heck, even if we just get regular coverage reports for Python that would be a great win out of all of this).
Now as for whether we should move the repos, I see two possibilities to help make that decision. One is we end up with 3 PEPs corresponding to the 3 proposals outlined above, get them done before PyCon, and then we have a discussion at the language summit where we can either make a decision or see what the pulse at the conference and sprints then make a decision shortly thereafter (I can moderate the summit discussion to keep this on-task and minimize the rambling; if Guido wants I can even make the final call since I have already played the role of "villain" for our issue tracker and hg decisions).
The other option is we take each one of the 3 proposed repos and pilot/experiment with them on a different platform. I would put peps on GitHub (as per Guido's comment of getting PRs from there already), the devguide on Bitbucket, and leave devinabox on hg.python.org but with the motivation of getting better tooling in place to contribute to it. We can then see if anything changes between now and PyCon and then discuss what occurred there (if we can't get the word out about this experiment and get new tooling up and going on the issue tracker in the next 4 months then that's another data point about how much people do/don't care about any of this). Obviously if we end up needing more time we don't *have* to make a decision at PyCon, but it's a good goal to have. I don't think we can cleanly replicate a single repo on all three solutions as I sure don't want to deal with that merging fun (unless someone comes forward to be basically a "release manager" for one of the repos to make that experiment happen).
So do people want PEPs or experimentation first?
On Tue Dec 02 2014 at 8:24:16 AM Nick Coghlan <ncoghlan@gmail.com> wrote:
On 2 December 2014 at 01:38, Guido van Rossum <guido@python.org> wrote:
As far as I'm concerned I'm just waiting for your decision now.
The RhodeCode team got in touch with me offline to suggest the possibility of using RhodeCode Enterprise as a self-hosted solution rather than a volunteer-supported installation of Kallithea. I'll be talking to them tomorrow, and if that discussion goes well, will update PEP 474 (and potentially PEP 462) accordingly.
Given that that would take away the "volunteer supported" vs "commercially supported" distinction between self-hosting and using GitHub (as well as potentially building a useful relationship that may help us resolve other workflow issues in the future), I'd like us to hold off on any significant decisions regarding the fate of any of the repos until I've had a chance to incorporate the results of that discussion into my proposals.
As described in PEP 474, I'm aware of the Mercurial team's concerns with RhodeCode's current licensing, but still consider it a superior alternative to an outright proprietary solution that doesn't get us any closer to solving the workflow problems with the main CPython repo.
Regards, Nick.
P.S. I'll also bring up some of the RFEs raised in this discussion around making it possible for folks to submit pull requests via GitHub/BitBucket, even if the master repositories are hosted on PSF infrastructure.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-- --Guido van Rossum (python.org/~guido)
On Tue, Dec 2, 2014 at 10:21 AM, Brett Cannon <brett@python.org> wrote:
On Tue Dec 02 2014 at 1:05:22 PM Guido van Rossum <guido@python.org> wrote:
Thanks for taking charge, Brett.
I personally think this shouldn't be brought up at the summit -- it's likely to just cause lots of heat about git vs. hg, free vs. not-free, "loyalty" to free or open tools, the weighing of core committers' preferences vs. outside contributors' preferences, GitHub's diversity track record, with no new information added. Even if we *just* had a vote by show-of-hands at the summit that would just upset those who couldn't be present.
Well, if I'm going to be the Great Decider on this then I can say upfront I'm taking a pragmatic view of preferring open but not mandating it, preferring hg over git but not ruling out a switch, preferring Python-based tools but not viewing it as a negative to not use Python, etc. I would like to think I have earned somewhat of a reputation of being level-headed and so none of this should really be a surprise to anyone.
So if we did have a discussion at the summit and someone decided to argue for FLOSS vs. not as a key factor then I would politely cut them off and say that doesn't matter to me and move on. As I said, I would moderate the conversation to keep it on-task and not waste my time with points that have already been made and flagged by me and you as not deal-breakers. And any votes would be to gauge the feeling of the room and not as a binding decision; I assume either me or someone else is going to be the dictator on this and this won't be a majority decision.
But I'll leave that up to you. The only thing I ask you is not to give me the last word. I might just do something you regret. :-)
What about me doing something that *I* regret like taking this on? =)
I trust you more than myself in this issue, Brett. You'll do fine. I may just leave the room while it's being discussed. -- --Guido van Rossum (python.org/~guido)
On Tue, 02 Dec 2014 18:21:39 +0000 Brett Cannon <brett@python.org> wrote:
So if we did have a discussion at the summit and someone decided to argue for FLOSS vs. not as a key factor then I would politely cut them off and say that doesn't matter to me and move on. As I said, I would moderate the conversation to keep it on-task and not waste my time with points that have already been made and flagged by me and you as not deal-breakers. And any votes would be to gauge the feeling of the room and not as a binding decision; I assume either me or someone else is going to be the dictator on this and this won't be a majority decision.
Can we stop making decisions at summits where it's always the same people being present? Thanks Antoine.
On Tue Dec 02 2014 at 1:52:49 PM Antoine Pitrou <solipsis@pitrou.net> wrote:
On Tue, 02 Dec 2014 18:21:39 +0000 Brett Cannon <brett@python.org> wrote:
So if we did have a discussion at the summit and someone decided to argue for FLOSS vs. not as a key factor then I would politely cut them off and say that doesn't matter to me and move on. As I said, I would moderate
the
conversation to keep it on-task and not waste my time with points that have already been made and flagged by me and you as not deal-breakers. And any votes would be to gauge the feeling of the room and not as a binding decision; I assume either me or someone else is going to be the dictator on this and this won't be a majority decision.
Can we stop making decisions at summits where it's always the same people being present?
I already said I'm not going to make a decision there, but you have to admit having an in-person discussion is a heck of a lot easier than going back and forth in email and so I'm not willing to rule out at least talking about the topic at PyCon. I wouldn't hold it against a BDFAP talking about something at EuroPython and happening to make a decision while there and so I would expect the same courtesy. -Brett
Thanks
Antoine.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ brett%40python.org
On Dec 02, 2014, at 06:21 PM, Brett Cannon wrote:
Well, if I'm going to be the Great Decider on this then I can say upfront I'm taking a pragmatic view of preferring open but not mandating it, preferring hg over git but not ruling out a switch, preferring Python-based tools but not viewing it as a negative to not use Python, etc. I would like to think I have earned somewhat of a reputation of being level-headed and so none of this should really be a surprise to anyone.
I think it's equally important to describe what criteria you will use to make this decision. E.g. are you saying all these above points will be completely ignored, or all else being equal, they will help tip the balance? Cheers, -Barry
On Tue Dec 02 2014 at 1:59:20 PM Barry Warsaw <barry@python.org> wrote:
On Dec 02, 2014, at 06:21 PM, Brett Cannon wrote:
Well, if I'm going to be the Great Decider on this then I can say upfront I'm taking a pragmatic view of preferring open but not mandating it, preferring hg over git but not ruling out a switch, preferring Python-based tools but not viewing it as a negative to not use Python, etc. I would like to think I have earned somewhat of a reputation of being level-headed and so none of this should really be a surprise to anyone.
I think it's equally important to describe what criteria you will use to make this decision. E.g. are you saying all these above points will be completely ignored, or all else being equal, they will help tip the balance?
Considering Guido just gave me this position I have not exactly had a ton of time to think the intricacies out, but they are all positives and can help tip the balance or break ties (I purposely worded all of that with "prefer", etc.). For instance, if a FLOSS solution came forward that looked to be good and close enough to what would be a good workflow along with support commitments from the infrastructure team and folks to maintain the code -- and this will have to people several people as experience with the issue tracker has shown -- then that can help tip over the closed-source, hosted solution which might have some perks. As for Python over something else, that comes into play in open source more from a maintenance perspective, but for closed source it would be a tie-breaker only since it doesn't exactly influence the usability of the closed-source solution like it does an open-source one. Basically I'm willing to give brownie points for open source and Python stuff, but it is just that: points and not deal-breakers.
On Dec 2, 2014, at 2:09 PM, Brett Cannon <brett@python.org> wrote:
On Tue Dec 02 2014 at 1:59:20 PM Barry Warsaw <barry@python.org <mailto:barry@python.org>> wrote: On Dec 02, 2014, at 06:21 PM, Brett Cannon wrote:
Well, if I'm going to be the Great Decider on this then I can say upfront I'm taking a pragmatic view of preferring open but not mandating it, preferring hg over git but not ruling out a switch, preferring Python-based tools but not viewing it as a negative to not use Python, etc. I would like to think I have earned somewhat of a reputation of being level-headed and so none of this should really be a surprise to anyone.
I think it's equally important to describe what criteria you will use to make this decision. E.g. are you saying all these above points will be completely ignored, or all else being equal, they will help tip the balance?
Considering Guido just gave me this position I have not exactly had a ton of time to think the intricacies out, but they are all positives and can help tip the balance or break ties (I purposely worded all of that with "prefer", etc.). For instance, if a FLOSS solution came forward that looked to be good and close enough to what would be a good workflow along with support commitments from the infrastructure team and folks to maintain the code -- and this will have to people several people as experience with the issue tracker has shown -- then that can help tip over the closed-source, hosted solution which might have some perks. As for Python over something else, that comes into play in open source more from a maintenance perspective, but for closed source it would be a tie-breaker only since it doesn't exactly influence the usability of the closed-source solution like it does an open-source one.
Basically I'm willing to give brownie points for open source and Python stuff, but it is just that: points and not deal-breakers.
This sounds like a pretty reasonable attitude to take towards this. If we’re going to be experimenting/talking things over, should I withdraw my PEP? --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Tue Dec 02 2014 at 2:15:09 PM Donald Stufft <donald@stufft.io> wrote:
On Dec 2, 2014, at 2:09 PM, Brett Cannon <brett@python.org> wrote:
On Tue Dec 02 2014 at 1:59:20 PM Barry Warsaw <barry@python.org> wrote:
On Dec 02, 2014, at 06:21 PM, Brett Cannon wrote:
Well, if I'm going to be the Great Decider on this then I can say upfront I'm taking a pragmatic view of preferring open but not mandating it, preferring hg over git but not ruling out a switch, preferring Python-based tools but not viewing it as a negative to not use Python, etc. I would like to think I have earned somewhat of a reputation of being level-headed and so none of this should really be a surprise to anyone.
I think it's equally important to describe what criteria you will use to make this decision. E.g. are you saying all these above points will be completely ignored, or all else being equal, they will help tip the balance?
Considering Guido just gave me this position I have not exactly had a ton of time to think the intricacies out, but they are all positives and can help tip the balance or break ties (I purposely worded all of that with "prefer", etc.). For instance, if a FLOSS solution came forward that looked to be good and close enough to what would be a good workflow along with support commitments from the infrastructure team and folks to maintain the code -- and this will have to people several people as experience with the issue tracker has shown -- then that can help tip over the closed-source, hosted solution which might have some perks. As for Python over something else, that comes into play in open source more from a maintenance perspective, but for closed source it would be a tie-breaker only since it doesn't exactly influence the usability of the closed-source solution like it does an open-source one.
Basically I'm willing to give brownie points for open source and Python stuff, but it is just that: points and not deal-breakers.
This sounds like a pretty reasonable attitude to take towards this.
If we’re going to be experimenting/talking things over, should I withdraw my PEP?
No because only two people have said they like the experiment idea so that's not exactly enough to say it's worth the effort. =) Plus GitHub could be chosen in the end. Basically a PEP staying in draft is no big deal.
I should say I will take a few days to think about this and then I will start a new thread outlining what I think we should be aiming for to help frame the whole discussion and to give proponents something to target. On Tue Dec 02 2014 at 2:20:16 PM Brett Cannon <brett@python.org> wrote:
On Tue Dec 02 2014 at 2:15:09 PM Donald Stufft <donald@stufft.io> wrote:
On Dec 2, 2014, at 2:09 PM, Brett Cannon <brett@python.org> wrote:
On Tue Dec 02 2014 at 1:59:20 PM Barry Warsaw <barry@python.org> wrote:
On Dec 02, 2014, at 06:21 PM, Brett Cannon wrote:
Well, if I'm going to be the Great Decider on this then I can say upfront I'm taking a pragmatic view of preferring open but not mandating it, preferring hg over git but not ruling out a switch, preferring Python-based tools but not viewing it as a negative to not use Python, etc. I would like to think I have earned somewhat of a reputation of being level-headed and so none of this should really be a surprise to anyone.
I think it's equally important to describe what criteria you will use to make this decision. E.g. are you saying all these above points will be completely ignored, or all else being equal, they will help tip the balance?
Considering Guido just gave me this position I have not exactly had a ton of time to think the intricacies out, but they are all positives and can help tip the balance or break ties (I purposely worded all of that with "prefer", etc.). For instance, if a FLOSS solution came forward that looked to be good and close enough to what would be a good workflow along with support commitments from the infrastructure team and folks to maintain the code -- and this will have to people several people as experience with the issue tracker has shown -- then that can help tip over the closed-source, hosted solution which might have some perks. As for Python over something else, that comes into play in open source more from a maintenance perspective, but for closed source it would be a tie-breaker only since it doesn't exactly influence the usability of the closed-source solution like it does an open-source one.
Basically I'm willing to give brownie points for open source and Python stuff, but it is just that: points and not deal-breakers.
This sounds like a pretty reasonable attitude to take towards this.
If we’re going to be experimenting/talking things over, should I withdraw my PEP?
No because only two people have said they like the experiment idea so that's not exactly enough to say it's worth the effort. =) Plus GitHub could be chosen in the end.
Basically a PEP staying in draft is no big deal.
On 12/02/2014 11:21 AM, Brett Cannon wrote:
I should say I will take a few days to think about this and then I will start a new thread outlining what I think we should be aiming for to help frame the whole discussion and to give proponents something to target.
Thanks for taking this on, Brett. -- ~Ethan~
On Dec 02, 2014, at 07:20 PM, Brett Cannon wrote:
No because only two people have said they like the experiment idea so that's not exactly enough to say it's worth the effort. =) Plus GitHub could be chosen in the end.
Experimenting could be useful, although if the traffic is disproportionate (e.g. peps are updated way more often than devinabox) or folks don't interact with each of the repos, it might not be very representative. Still, I think it's better to get a visceral sense of how things actually work than to speculate about how they *might* work. Cheers, -Barry
On Tue Dec 02 2014 at 3:14:20 PM Barry Warsaw <barry@python.org> wrote:
On Dec 02, 2014, at 07:20 PM, Brett Cannon wrote:
No because only two people have said they like the experiment idea so that's not exactly enough to say it's worth the effort. =) Plus GitHub could be chosen in the end.
Experimenting could be useful, although if the traffic is disproportionate (e.g. peps are updated way more often than devinabox) or folks don't interact with each of the repos, it might not be very representative. Still, I think it's better to get a visceral sense of how things actually work than to speculate about how they *might* work.
That's my thinking. It's more about the workflow than measuring engagement on GitHub vs. Bitbucket (we already know how that skews). If I had my wish we would put the same repo in all three scenarios, but that is just asking for merge headaches. But I think if we go to the community and say, "help us test dev workflows by submitting spelling and grammar fixes" then we should quickly get an idea of the workflows (and I purposefully left devinabox out of a move since it is never touched after it essentially became a build script and a README and so represents our existing workflow; any benefit on our own infrastructure can go straight to cpython anyway which we can all experience).
Brett Cannon <brett@python.org> writes:
Well, if I'm going to be the Great Decider on this then I can say upfront I'm taking a pragmatic view of preferring open but not mandating it, preferring hg over git but not ruling out a switch, preferring Python-based tools but not viewing it as a negative to not use Python, etc.
(and you've later clarified that these will all be factors weighing in favour of a candidate.) Thanks for expressing your thoughts. Big thanks for taking on the role of consulting, evaluating, and deciding on this issue. -- \ “I think Western civilization is more enlightened precisely | `\ because we have learned how to ignore our religious leaders.” | _o__) —Bill Maher, 2003 | Ben Finney
On Tue, Dec 2, 2014 at 9:50 AM, Brett Cannon <brett@python.org> wrote:
So I was waiting for Nick to say what he wanted to do for the peps repo since I view it as I get 2/3 of the choices and he gets the other third.
The way I view it, the options are:
Move to GitHub Move to Bitbucket Improve our current tooling (either through new hosting setup and/or adding first-world support for downloading PRs from GitHub/Bitbucket)
I'd argue that option #3 here is somewhat orthogonal to switching hosting. It makes sense regardless unless we plan on ditching roundup and reitveld (to which I'd be opposed).
Regardless of what we do, I think we should graduate the mirrors on GitHub and Bitbucket to "official" -- for the proposed repos and cpython -- and get their repos updating per-push instead of as a cron job. I also think we should also flip on any CI we can (e.g. turn on Travis for GitHub along with coveralls support using coverage.py's encodings trick). This will get us the most accessible repo backups as well as the widest tool coverage for contributors to assist them in their contributions (heck, even if we just get regular coverage reports for Python that would be a great win out of all of this).
+1 to all of this. Doing this would allow us to move forward with GH/BB-roundup/reitveld integration (option #3) sooner rather than later, regardless of moving to other hosting.
So do people want PEPs or experimentation first?
+1 to PEPs. It's basically already happening. I'd like to see where 474/481/etc. end up, particularly with what Nick brought up earlier. Furthermore, I'm not sure how effectively we can experiment when it comes to moving hosting. There's overhead involved that biases the outcome and in part contributes to the momentum of the initial experimental conditions. I doubt any external solution is going to prove drastically better than another, making it harder to justify the effort to move yet again. -eric
Before anyone gets too excited about Rietveld (which I originally wrote as an APp Engine demo), AFAIK we're using a fork that only Martin von Loewis can maintain -- and it's a dead-end fork because the Rietveld project itself only supports App Engine, but Martin's fork runs on our own server infrastructure. These environments are *very* different (App Engine has its own unique noSQL API) and it took a major hack (not by MvL) to get it to work outside App Engine. That fork is not supported, and hence our Rietveld installation still has various bugs that have long been squashed in the main Rietveld repo. (And no, I don't have time to help with this -- my recommendation is to move off Rietveld to something supported.) On Tue, Dec 2, 2014 at 2:33 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:
On Tue, Dec 2, 2014 at 9:50 AM, Brett Cannon <brett@python.org> wrote:
So I was waiting for Nick to say what he wanted to do for the peps repo since I view it as I get 2/3 of the choices and he gets the other third.
The way I view it, the options are:
Move to GitHub Move to Bitbucket Improve our current tooling (either through new hosting setup and/or adding first-world support for downloading PRs from GitHub/Bitbucket)
I'd argue that option #3 here is somewhat orthogonal to switching hosting. It makes sense regardless unless we plan on ditching roundup and reitveld (to which I'd be opposed).
Regardless of what we do, I think we should graduate the mirrors on
and Bitbucket to "official" -- for the proposed repos and cpython -- and get their repos updating per-push instead of as a cron job. I also think we should also flip on any CI we can (e.g. turn on Travis for GitHub along with coveralls support using coverage.py's encodings trick). This will get us
GitHub the
most accessible repo backups as well as the widest tool coverage for contributors to assist them in their contributions (heck, even if we just get regular coverage reports for Python that would be a great win out of all of this).
+1 to all of this. Doing this would allow us to move forward with GH/BB-roundup/reitveld integration (option #3) sooner rather than later, regardless of moving to other hosting.
So do people want PEPs or experimentation first?
+1 to PEPs. It's basically already happening. I'd like to see where 474/481/etc. end up, particularly with what Nick brought up earlier.
Furthermore, I'm not sure how effectively we can experiment when it comes to moving hosting. There's overhead involved that biases the outcome and in part contributes to the momentum of the initial experimental conditions. I doubt any external solution is going to prove drastically better than another, making it harder to justify the effort to move yet again.
-eric
-- --Guido van Rossum (python.org/~guido)
On Dec 2, 2014, at 5:42 PM, Guido van Rossum <guido@python.org> wrote:
Before anyone gets too excited about Rietveld (which I originally wrote as an APp Engine demo), AFAIK we're using a fork that only Martin von Loewis can maintain -- and it's a dead-end fork because the Rietveld project itself only supports App Engine, but Martin's fork runs on our own server infrastructure. These environments are *very* different (App Engine has its own unique noSQL API) and it took a major hack (not by MvL) to get it to work outside App Engine. That fork is not supported, and hence our Rietveld installation still has various bugs that have long been squashed in the main Rietveld repo. (And no, I don't have time to help with this -- my recommendation is to move off Rietveld to something supported.)
It probably makes sense to include code reviews in the matrix of what tools we’re going to use then yea? Like Github/Bitbucket/etc have review built in. Other tools like Phabricator do as well but are self hosted instead. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 12/02/2014 02:47 PM, Donald Stufft wrote:
On Dec 2, 2014, at 5:42 PM, Guido van Rossum <guido@python.org <mailto:guido@python.org>> wrote:
Before anyone gets too excited about Rietveld (which I originally wrote as an APp Engine demo), AFAIK we're using a fork that only Martin von Loewis can maintain -- and it's a dead-end fork because the Rietveld project itself only supports App Engine, but Martin's fork runs on our own server infrastructure. These environments are *very* different (App Engine has its own unique noSQL API) and it took a major hack (not by MvL) to get it to work outside App Engine. That fork is not supported, and hence our Rietveld installation still has various bugs that have long been squashed in the main Rietveld repo. (And no, I don't have time to help with this -- my recommendation is to move off Rietveld to something supported.)
It probably makes sense to include code reviews in the matrix of what tools we’re going to use then yea?
Like Github/Bitbucket/etc have review built in. Other tools like Phabricator do as well but are self hosted instead.
I think the people/company behind phabricator are planning to offer an hosting solution. Could be worth poking at them to have and idea of what is the status of it. -- Pierre-Yves David
On 3 Dec 2014 08:47, "Donald Stufft" <donald@stufft.io> wrote:
On Dec 2, 2014, at 5:42 PM, Guido van Rossum <guido@python.org> wrote:
Before anyone gets too excited about Rietveld (which I originally wrote
as an APp Engine demo), AFAIK we're using a fork that only Martin von Loewis can maintain -- and it's a dead-end fork because the Rietveld project itself only supports App Engine, but Martin's fork runs on our own server infrastructure. These environments are *very* different (App Engine has its own unique noSQL API) and it took a major hack (not by MvL) to get it to work outside App Engine. That fork is not supported, and hence our Rietveld installation still has various bugs that have long been squashed in the main Rietveld repo. (And no, I don't have time to help with this -- my recommendation is to move off Rietveld to something supported.) Thanks Guido - I'd started thinking in that direction for PEP 462 (in terms of potentially using Kallithea/RhodeCode for the review component rather than Reitveld), so it's good to know you'd be OK with such a change.
It probably makes sense to include code reviews in the matrix of what tools we’re going to use then yea?
I'd suggest asking for discussion of a more general path forward for CPython workflow improvements. Not a "this must be included in the proposal", but rather answering the question, "if we choose this option for the support repos, how will it impact the future direction of CPython maintenance itself?". Cheers, Nick.
Like Github/Bitbucket/etc have review built in. Other tools like
Phabricator do as well but are self hosted instead.
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Tue, Dec 2, 2014 at 6:24 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
P.S. I'll also bring up some of the RFEs raised in this discussion around making it possible for folks to submit pull requests via GitHub/BitBucket, even if the master repositories are hosted on PSF infrastructure.
In case it helps with any GH/BB-to-roundup/reitveld integration we might do, I've already done something similar for GH-to-reviewboard at work. All the code is on-line: https://bitbucket.org/ericsnowcurrently/rb_webhooks_extension -eric
On 2014-12-01, 00:50 GMT, Donald Stufft wrote:
The only thing that is true is that git users are more likely to use the ability to rewrite history than Mercurial users are, but you’ll typically find that people generally don’t do this on public branches, only on private branches.
And I would add that any reasonable git repository manager (why we are talking only about GitHub as if there was no cgit, gitorious, gitlab, gitblit, etc.?) can forbid forced-push so the history could be as sacrosanct as with Mercurial. Matěj
Ben Finney writes:
Whether that is a *problem* is a matter of debate, but the fact that Git's common workflow commonly discards information that some consider valuable, is a simple fact.
It *was* a simple fact in git 0.99. Since the advent of reflogs (years ago), it is simply false. Worse, common hg workflows (as developed at the same time as your impression of "Git's common workflow") *also* discard information that some (namely, me) consider valuable, because it *never gets recorded*. Exploring the git reflog has taught me things about my workflow and skills (and lack thereof ;-) that I'd never learn from an hg or bzr branch. In the end, the logs look very similar. I can only conclude that the rebasing that I do in git is implicit in the process of composing a "coherent changeset" in hg or bzr. I also typically have a bunch of "loose commits" lying around in Mercurial queues or bzr stashes, which amount to rebases when reapplied.
If Ethan chooses to make that a factor in his decisions about Git, the facts are on his side.
Hardly. All he needs to do is pretend git is hg, and avoid rebase. The only thing that should matter is the annoyance of learning a new tool.
On 1 December 2014 01:17:02 CET, Ben Finney <ben+python@benfinney.id.au> wrote:
Donald Stufft <donald@stufft.io> writes:
I have never heard of git losing history.
In my experience talking with Git users about this problem, that depends on a very narrow definition of “losing history”.
Git encourages re-writing, and thereby losing prior versions of, the history of a branch. The commit information remains, but the history of how they link together is lost. That is a loss of information, which is not the case in the absence of such history re-writing.
"Losing data" is generally used in the sense that either the application or the filesystem accidentally deletes or overwrites data without the user's consent or knownledge. Rewriting and deleting (not "losing") history in git is explicitly done by the user, encouraged or not. -- Markus
On Sun, Nov 30, 2014 at 02:56:22PM -0500, Donald Stufft wrote:
As I mentioned in my other email, we’re already supporting two different tools, and it’s a hope of mine to use this as a sort of testbed to moving the other repositories as well.
If we go down this path, can we have some *concrete* and *objective* measures of success? If moving to git truly does improve things, then the move can be said to be a success. But if it makes no concrete difference, then we've wasted our time. In six months time, how will we know which it is? Can we have some concrete and objective measures of what would count as success, and some Before and After measurements? Just off the top of my head... if the number of documentation patches increases significiantly (say, by 30%) after six months, that's a sign the move was successful. It's one thing to say that using hg is discouraging contributors, and that hg is much more popular. It's another thing to say that moving to git will *actually make a difference*. Maybe all the would-be contributors using git are too busy writing kernel patches for Linus or using Node.js and wouldn't be caught dead with Python :-) With concrete and objective measures of success, you will have ammunition to suggest moving the rest of Python to git in a few years time. And without it, we'll also have good evidence that any further migration to git may be a waste of time and effort and we should focus our energy elsewhere rather than git vs hg holy wars. [...]
I also think it’s hard to look at a company like bitbucket, for example, and say they are *better* than Github just because they didn’t have a public and inflammatory event.
We can't judge companies on what they might be doing behind closed doors, only on what we can actually see of them. Anybody might be rotten bounders and cads in private, but how would we know? It's an imperfect world and we have imperfect knowledge but still have to make a decision as best we can.
Attempting to reduce the cognitive burden for contributing and aligning ourselves with the most popular tools allows us to take advantage of the network effects of these tools popularity. This can be the difference between someone with limited amount of time being able to contribute or not, which can make real inroads towards making it easier for under privileged people to contribute much more than refusing to use a product of one group of people over another just because the other group hasn’t had a public and inflammatory event.
In other contexts, that could be a pretty awful excuse for inaction against the most aggregiously bad behaviour. "Sure, Acme Inc might have adulterated baby food with arsenic, but other companies might have done worse things that we haven't found out about. So we should keep buying Acme's products, because they're cheaper and that's good for the poor." Not that I'm comparing GitHub's actions with poisoning babies. What GitHub did was much worse. *wink* -- Steven
On Mon, Dec 1, 2014 at 6:28 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
My issues with GitHub range from selfish to philosophical:
- (selfish) I don't want to learn git
This ties in directly with the popularity argument. How many people are there who know hg and don't know git? How many who know git and don't know hg? So while this is a git issue for you, it's going to be a Mercurial issue for a lot more people. (And even people like me who kinda know both are going to spend a lot more time with git than with hg, on average. My knowledge of git is fairly advanced - I use git hooks, have scripts that manage things for me, can repair a repository that's seen some mess in it, etc - but my Mercurial knowledge is basic - I can clone and pull, not even sure I can push as I have literally never done so, and I basically turn to a dev guide to figure out how to make a patch in the appropriate way.)
- (practical) from what I hear git can have issues with losing history -- in a project that has many volunteer and part-time developers, using a tool that can munge your data just doesn't seem very wise
It is possible to rewrite history in git, but - assuming you're not going to go to the ridiculous extent of SHA-1 cracking - it'll always be done by creating a new stream of commits, and git is very careful about confirming that you want to do this sort of thing. When you have a central server, you can simply configure it to reject any non-fast-forward pushes. (I'm not sure how to set receive.denyNonFastForwards on GitHub, but it's likely to be possible. In any case, you can always have an authoritative clone on python.org somewhere which mandates this kind of thing.) The git philosophy is: Your repository is yours. What you do with it is up to you. If that means rewriting history, git will confirm with you that you really want to do that, and then go right ahead and do what you ask. But then if your repo is a clone of someone else's, well, that someone else controls the other repo, so you might well not be allowed to merge changes in.
- (practical) supporting git and hg means learning two different workflows
Already a problem for a lot of people. Unless CPython is the only project you ever contribute to, chances are you're going to meet git somewhere. If CPython used some mindblowingly brilliant but utterly unheard-of DVCS, sure it might be possible for core devs to avoid using any of the popular systems, but nobody else could. Personally, and somewhat selfishly, I would love to see the PEPs repo move to GitHub. For the two PEPs that I wrote, I had to juggle my own personal PEP draft repo and the central Mercurial peps repo; making sure changes got deployed from one to the other was basically a manual job, without any tool support. If I could send PRs against a clone of the peps repo, I would work that way instead of creating a dedicated repo at all (at least until such time as I need other files; the PEP 463 repo grew a few other bits and bobs, but they could just as easily have been in an ancillary repo without the PEP text in it). There'll be easily as many people who selfishly want git as selfishly want hg :) ChrisA
On Sun, 30 Nov 2014 10:05:01 -0800 Guido van Rossum <guido@python.org> wrote:
I bring this up to emphasize that (unlike GNU software and the FSF) Python has no additional hidden agenda of bringing freedom to all software.
As far as GNU and the FSF are concerned, I don't think the agenda is "hidden" at all ;-)
Now some entirely practical points.
- I am basically the only remaining active PEP editor, so I see most PEP contributions by non-core-committers. Almost all of these uses github. Not bitbucket, not some other git host, but github. I spend a fair amount of time applying patches. It would most definitely be easier if I could get them to send me pull requests.
Are you sure that those contributors wouldn't want to use Bitbucket - or another hg-based hosting service? A PEP contributor is someone who is likely to contribute CPython code as well - so they would have to know hg. Regards Antoine.
On Nov 30, 2014, at 4:05 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Sun, 30 Nov 2014 10:05:01 -0800 Guido van Rossum <guido@python.org> wrote:
I bring this up to emphasize that (unlike GNU software and the FSF) Python has no additional hidden agenda of bringing freedom to all software.
As far as GNU and the FSF are concerned, I don't think the agenda is "hidden" at all ;-)
Now some entirely practical points.
- I am basically the only remaining active PEP editor, so I see most PEP contributions by non-core-committers. Almost all of these uses github. Not bitbucket, not some other git host, but github. I spend a fair amount of time applying patches. It would most definitely be easier if I could get them to send me pull requests.
Are you sure that those contributors wouldn't want to use Bitbucket - or another hg-based hosting service?
A PEP contributor is someone who is likely to contribute CPython code as well - so they would have to know hg.
Speaking as someone who has written a number of PEPS, plans to write a number more, and also has commit bit on hg.python.org and is maintaining code in CPython. I would greatly prefer using Github for PEPs than I would any hg hosting service I’ve seen to date. I can’t obviously speak for every single committer, however Guido also stated that through his work in committing patches of incoming PEPs he’s seen that a lot of the PEPs are being written on Github using git. I think it’s fair to say that those people would prefer PRs on Github over using Bitbucket as well since they were choosing Github over Bitbucket when there was no incentive to do so. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 11/30/2014 1:05 PM, Guido van Rossum wrote:
I don't feel it's my job to accept or reject this PEP, but I do have an opinion. ... - I am basically the only remaining active PEP editor, so I see most PEP contributions by non-core-committers. Almost all of these uses github. Not bitbucket, not some other git host, but github. I spend a fair amount of time applying patches. It would most definitely be easier if I could get them to send me pull requests.
The scope of the PEP is still apparently somewhat fluid. I said elsewhere that I think the principal maintainers of a specialized single-branch repository should have the principal say in where it lives. So I think you should be the one to decide on a PEP limited to moving the PEP repository. My understanding is that if the peps were moved to github, then I would be able to suggest changes via an online web form that would generate a pull request from edited text. If so, I would say go ahead and move them and see how it goes. To me, the multibranch CPython repository is a very different issue. I think it should stay where it is for now, especially with 2.7 support extended. I think for this we should better focus on better use of developer time and getting more developers active. -- Terry Jan Reedy
On 11/30/2014 09:09 AM, Donald Stufft wrote:
what Eric Raymond did with reposurgeon to convert from Bazaar to git. I fail to see how this is a reasonable argument to make at all since, as you mentioned, converting between two FLOSS tools can be an amazing amount of work. Realistically the amount of work is going to be predicated on whether or not
Even converting between two FLOSS tools is an amazing amount of work. Look at there is a tool that already handles the conversion for you. Assuming of course that the data is available to you at all.
The statement that switch a whole infra from one tool to another is "cheap" and predictable sounds extremely naive to me.
As a particularly relevant example, switching from Mercurial to Git is as easy as installing hg-git, creating a bookmark for master that tracks default, and then pushing to a git repository.
Migrating the DVCS content is usually easy. The hard part is then to find all the script, tools, doc that rely on the previous tools to upgrade them to the new one, sometimes trying to find feature parity. This is rarely cheap and never "predictable" -- Pierre-Yves David
On 2014-12-01, 02:12 GMT, Pierre-Yves David wrote:
Migrating the DVCS content is usually easy.
This is lovely mantra, but do you speak from your own experience? I did move rope from Bitbucket to https://github.com/python-rope and it was A LOT of work (particularly issues, existing pull requests, and other related stuff like many websites the projects holds). And rope is particularly simple (and almost dead so inactive) project. Best, Matěj
On Mon, 1 Dec 2014 08:46:46 +0100 Matěj Cepl <mcepl@cepl.eu> wrote:
On 2014-12-01, 02:12 GMT, Pierre-Yves David wrote:
Migrating the DVCS content is usually easy.
This is lovely mantra, but do you speak from your own experience? I did move rope from Bitbucket to https://github.com/python-rope and it was A LOT of work (particularly issues, existing pull requests, and other related stuff like many websites the projects holds).
He did say "DVCS content" (as in: stuff that's stored in git or hg), not ancillary data such as pull requests, issues, wikis and Web sites. But you're making his point: migrating a source code repository from hg to git or vice-versa is relatively easy, it's the surrounding stuff that's hard. Regards Antoine.
On Sun, Nov 30, 2014 at 10:55 AM, Barry Warsaw <barry@python.org> wrote:
On Nov 30, 2014, at 09:54 AM, Ian Cordasco wrote:
- Migrating "data" from GitHub is easy. There are free-as-in-freedom tools to do it and the only cost is the time it would take to monitor the process
*Extracting* data may be easy, but migrating it is a different story. As the Mailman project has seen in trying to migrate from Confluence to Moin, there is a ton of very difficult work involved after extracting the data. Parsing the data, ensuring that you have all the bits you need, fitting it into the new system's schema, working out the edge cases, adapting to semantic differences and gaps, ensuring that all the old links are redirected, and so on, were all exceedingly difficult[*].
The GitHub API is currently at Version 3. These may be useful references for the PEP: https://developer.github.com/v3/ https://developer.github.com/libraries/ https://github.com/jaxbot/github-issues.vim (:Gissues) https://developer.github.com/webhooks/ There are integrations for many platforms here: https://zapier.com/developer/documentation/ https://zapier.com/zapbook/apps/#sort=popular&filter=developer-tools
Even converting between two FLOSS tools is an amazing amount of work. Look at what Eric Raymond did with reposurgeon to convert from Bazaar to git.
It's a good thing that your data isn't locked behind a proprietary door, for now. That's only part of the story. But also, because github is a closed system, there's no guarantee that today's data-freeing APIs will still exist, continue to be functional for practical purposes, remain complete, or stay at parity with new features.
Cheers, -Barry
[*] And our huge gratitude goes to Paul Boddie for his amazing amount of work on the project. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
On Nov 30, 2014, at 12:06 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
Nick Coghlan <ncoghlan@gmail.com> writes:
1. I strongly believe that the long term sustainability of the overall open source community requires the availability and use of open source infrastructure.
I concur. This article <URL:http://mako.cc/writing/hill-free_tools.html> makes the arguments well, IMO.
2. I also feel that this proposal is far too cavalier in not even discussing the possibility of helping out the Mercurial team […] we'd prefer to switch to something else entirely rather than organising a sprint with them at PyCon to help ensure that our existing Mercurial based infrastructure is approachable for git & GitHub users?
Exactly. For such a core tool, instead of pushing proprietary platforms at the expense of software freedom, the sensible strategy for a project (Python) that hopes to be around in the long term is to use and improve the free software platforms.
I think there is a big difference here between using a closed source VCS or compiler and using a closed source code host. Namely in that the protocol is defined by git so switching from one host to another is easy. It’s akin to saying that if we chose to run the PyPI services on a Windows machine that it is somehow makes it less-free even though we could have chosen to run it on a “free” OS and we weren’t doing much, if anything, to tie us to that particular OS. If it makes people feel better we can continue to support the existing mechanisms of contribution, then people can choose between interacting with a “non free” host and “free” tooling. I suspect most people will choose the “non-free” tooling.
Specifically, which features are most ideal here? - [ ] Userbase - [ ] TTW editing only over SSL (see: Zope 2) - [ ] Pull Requests (see also: BitBucket, Torvalds rant) - [ ] Simple Issue Tagging - [ ] Pingbacks - [ ] CI Integration On Sat, Nov 29, 2014 at 11:27 PM, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 12:06 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
Nick Coghlan <ncoghlan@gmail.com> writes:
1. I strongly believe that the long term sustainability of the overall open source community requires the availability and use of open source infrastructure.
I concur. This article <URL:http://mako.cc/writing/hill-free_tools.html> makes the arguments well, IMO.
2. I also feel that this proposal is far too cavalier in not even discussing the possibility of helping out the Mercurial team […] we'd prefer to switch to something else entirely rather than organising a sprint with them at PyCon to help ensure that our existing Mercurial based infrastructure is approachable for git & GitHub users?
Exactly. For such a core tool, instead of pushing proprietary platforms at the expense of software freedom, the sensible strategy for a project (Python) that hopes to be around in the long term is to use and improve the free software platforms.
I think there is a big difference here between using a closed source VCS or compiler and using a closed source code host. Namely in that the protocol is defined by git so switching from one host to another is easy.
It’s akin to saying that if we chose to run the PyPI services on a Windows machine that it is somehow makes it less-free even though we could have chosen to run it on a “free” OS and we weren’t doing much, if anything, to tie us to that particular OS.
If it makes people feel better we can continue to support the existing mechanisms of contribution, then people can choose between interacting with a “non free” host and “free” tooling. I suspect most people will choose the “non-free” tooling. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
- [ ] Markdown - [ ] ReStructuredText - [ ] Review (why are these out of band?) On Sat, Nov 29, 2014 at 11:34 PM, Wes Turner <wes.turner@gmail.com> wrote:
Specifically, which features are most ideal here?
- [ ] Userbase - [ ] TTW editing only over SSL (see: Zope 2) - [ ] Pull Requests (see also: BitBucket, Torvalds rant) - [ ] Simple Issue Tagging - [ ] Pingbacks - [ ] CI Integration
On Sat, Nov 29, 2014 at 11:27 PM, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 12:06 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
Nick Coghlan <ncoghlan@gmail.com> writes:
1. I strongly believe that the long term sustainability of the overall open source community requires the availability and use of open source infrastructure.
I concur. This article <URL:http://mako.cc/writing/hill-free_tools.html
makes the arguments well, IMO.
2. I also feel that this proposal is far too cavalier in not even discussing the possibility of helping out the Mercurial team […] we'd prefer to switch to something else entirely rather than organising a sprint with them at PyCon to help ensure that our existing Mercurial based infrastructure is approachable for git & GitHub users?
Exactly. For such a core tool, instead of pushing proprietary platforms at the expense of software freedom, the sensible strategy for a project (Python) that hopes to be around in the long term is to use and improve the free software platforms.
I think there is a big difference here between using a closed source VCS or compiler and using a closed source code host. Namely in that the protocol is defined by git so switching from one host to another is easy.
It’s akin to saying that if we chose to run the PyPI services on a Windows machine that it is somehow makes it less-free even though we could have chosen to run it on a “free” OS and we weren’t doing much, if anything, to tie us to that particular OS.
If it makes people feel better we can continue to support the existing mechanisms of contribution, then people can choose between interacting with a “non free” host and “free” tooling. I suspect most people will choose the “non-free” tooling. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
- [ ] Stable URIs - [ ] Commit hashes On Sun, Nov 30, 2014 at 12:11 AM, Wes Turner <wes.turner@gmail.com> wrote:
- [ ] Markdown - [ ] ReStructuredText
- [ ] Review (why are these out of band?)
On Sat, Nov 29, 2014 at 11:34 PM, Wes Turner <wes.turner@gmail.com> wrote:
Specifically, which features are most ideal here?
- [ ] Userbase - [ ] TTW editing only over SSL (see: Zope 2) - [ ] Pull Requests (see also: BitBucket, Torvalds rant) - [ ] Simple Issue Tagging - [ ] Pingbacks - [ ] CI Integration
On Sat, Nov 29, 2014 at 11:27 PM, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 12:06 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
Nick Coghlan <ncoghlan@gmail.com> writes:
1. I strongly believe that the long term sustainability of the overall open source community requires the availability and use of open source infrastructure.
I concur. This article <URL: http://mako.cc/writing/hill-free_tools.html> makes the arguments well, IMO.
2. I also feel that this proposal is far too cavalier in not even discussing the possibility of helping out the Mercurial team […] we'd prefer to switch to something else entirely rather than organising a sprint with them at PyCon to help ensure that our existing Mercurial based infrastructure is approachable for git & GitHub users?
Exactly. For such a core tool, instead of pushing proprietary platforms at the expense of software freedom, the sensible strategy for a project (Python) that hopes to be around in the long term is to use and improve the free software platforms.
I think there is a big difference here between using a closed source VCS or compiler and using a closed source code host. Namely in that the protocol is defined by git so switching from one host to another is easy.
It’s akin to saying that if we chose to run the PyPI services on a Windows machine that it is somehow makes it less-free even though we could have chosen to run it on a “free” OS and we weren’t doing much, if anything, to tie us to that particular OS.
If it makes people feel better we can continue to support the existing mechanisms of contribution, then people can choose between interacting with a “non free” host and “free” tooling. I suspect most people will choose the “non-free” tooling. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
- [ ] Paste-able links On Sun, Nov 30, 2014 at 12:31 AM, Wes Turner <wes.turner@gmail.com> wrote:
- [ ] Stable URIs - [ ] Commit hashes
On Sun, Nov 30, 2014 at 12:11 AM, Wes Turner <wes.turner@gmail.com> wrote:
- [ ] Markdown - [ ] ReStructuredText
- [ ] Review (why are these out of band?)
On Sat, Nov 29, 2014 at 11:34 PM, Wes Turner <wes.turner@gmail.com> wrote:
Specifically, which features are most ideal here?
- [ ] Userbase - [ ] TTW editing only over SSL (see: Zope 2) - [ ] Pull Requests (see also: BitBucket, Torvalds rant) - [ ] Simple Issue Tagging - [ ] Pingbacks - [ ] CI Integration
On Sat, Nov 29, 2014 at 11:27 PM, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 12:06 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
Nick Coghlan <ncoghlan@gmail.com> writes:
1. I strongly believe that the long term sustainability of the overall open source community requires the availability and use of open source infrastructure.
I concur. This article <URL: http://mako.cc/writing/hill-free_tools.html> makes the arguments well, IMO.
2. I also feel that this proposal is far too cavalier in not even discussing the possibility of helping out the Mercurial team […] we'd prefer to switch to something else entirely rather than organising a sprint with them at PyCon to help ensure that our existing Mercurial based infrastructure is approachable for git & GitHub users?
Exactly. For such a core tool, instead of pushing proprietary platforms at the expense of software freedom, the sensible strategy for a project (Python) that hopes to be around in the long term is to use and improve the free software platforms.
I think there is a big difference here between using a closed source VCS or compiler and using a closed source code host. Namely in that the protocol is defined by git so switching from one host to another is easy.
It’s akin to saying that if we chose to run the PyPI services on a Windows machine that it is somehow makes it less-free even though we could have chosen to run it on a “free” OS and we weren’t doing much, if anything, to tie us to that particular OS.
If it makes people feel better we can continue to support the existing mechanisms of contribution, then people can choose between interacting with a “non free” host and “free” tooling. I suspect most people will choose the “non-free” tooling. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
Is there a kalithea celery task to mirror / SYNC with github, for pull requests and/or issues? https://pypi.python.org/pypi/Kallithea/ https://kallithea-scm.org/ https://kallithea-scm.org/repos/kallithea https://bitbucket.org/conservancy/kallithea http://pythonhosted.org//Kallithea http://kallithea.readthedocs.org/en/latest/ http://pypi.python.org/pypi/vcs http://pythonhosted.org//vcs https://github.com/codeinn/vcs On Sun, Nov 30, 2014 at 12:50 AM, Wes Turner <wes.turner@gmail.com> wrote:
- [ ] Paste-able links
On Sun, Nov 30, 2014 at 12:31 AM, Wes Turner <wes.turner@gmail.com> wrote:
- [ ] Stable URIs - [ ] Commit hashes
On Sun, Nov 30, 2014 at 12:11 AM, Wes Turner <wes.turner@gmail.com> wrote:
- [ ] Markdown - [ ] ReStructuredText
- [ ] Review (why are these out of band?)
On Sat, Nov 29, 2014 at 11:34 PM, Wes Turner <wes.turner@gmail.com> wrote:
Specifically, which features are most ideal here?
- [ ] Userbase - [ ] TTW editing only over SSL (see: Zope 2) - [ ] Pull Requests (see also: BitBucket, Torvalds rant) - [ ] Simple Issue Tagging - [ ] Pingbacks - [ ] CI Integration
On Sat, Nov 29, 2014 at 11:27 PM, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 12:06 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
Nick Coghlan <ncoghlan@gmail.com> writes:
> 1. I strongly believe that the long term sustainability of the overall > open source community requires the availability and use of open source > infrastructure.
I concur. This article <URL: http://mako.cc/writing/hill-free_tools.html> makes the arguments well, IMO.
> 2. I also feel that this proposal is far too cavalier in not even > discussing the possibility of helping out the Mercurial team […] we'd > prefer to switch to something else entirely rather than organising a > sprint with them at PyCon to help ensure that our existing Mercurial > based infrastructure is approachable for git & GitHub users?
Exactly. For such a core tool, instead of pushing proprietary platforms at the expense of software freedom, the sensible strategy for a project (Python) that hopes to be around in the long term is to use and improve the free software platforms.
I think there is a big difference here between using a closed source VCS or compiler and using a closed source code host. Namely in that the protocol is defined by git so switching from one host to another is easy.
It’s akin to saying that if we chose to run the PyPI services on a Windows machine that it is somehow makes it less-free even though we could have chosen to run it on a “free” OS and we weren’t doing much, if anything, to tie us to that particular OS.
If it makes people feel better we can continue to support the existing mechanisms of contribution, then people can choose between interacting with a “non free” host and “free” tooling. I suspect most people will choose the “non-free” tooling. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
https://kallithea-scm.org/repos/kallithea/files/tip/setup.py https://github.com/codeinn/vcs/blob/master/setup.py On Sun, Nov 30, 2014 at 1:16 AM, Wes Turner <wes.turner@gmail.com> wrote:
Is there a kalithea celery task to mirror / SYNC with github, for pull requests and/or issues?
https://pypi.python.org/pypi/Kallithea/ https://kallithea-scm.org/ https://kallithea-scm.org/repos/kallithea https://bitbucket.org/conservancy/kallithea
http://pythonhosted.org//Kallithea http://kallithea.readthedocs.org/en/latest/
http://pypi.python.org/pypi/vcs http://pythonhosted.org//vcs https://github.com/codeinn/vcs
On Sun, Nov 30, 2014 at 12:50 AM, Wes Turner <wes.turner@gmail.com> wrote:
- [ ] Paste-able links
On Sun, Nov 30, 2014 at 12:31 AM, Wes Turner <wes.turner@gmail.com> wrote:
- [ ] Stable URIs - [ ] Commit hashes
On Sun, Nov 30, 2014 at 12:11 AM, Wes Turner <wes.turner@gmail.com> wrote:
- [ ] Markdown - [ ] ReStructuredText
- [ ] Review (why are these out of band?)
On Sat, Nov 29, 2014 at 11:34 PM, Wes Turner <wes.turner@gmail.com> wrote:
Specifically, which features are most ideal here?
- [ ] Userbase - [ ] TTW editing only over SSL (see: Zope 2) - [ ] Pull Requests (see also: BitBucket, Torvalds rant) - [ ] Simple Issue Tagging - [ ] Pingbacks - [ ] CI Integration
On Sat, Nov 29, 2014 at 11:27 PM, Donald Stufft <donald@stufft.io> wrote:
> On Nov 30, 2014, at 12:06 AM, Ben Finney < ben+python@benfinney.id.au> wrote: > > Nick Coghlan <ncoghlan@gmail.com> writes: > >> 1. I strongly believe that the long term sustainability of the overall >> open source community requires the availability and use of open source >> infrastructure. > > I concur. This article <URL: http://mako.cc/writing/hill-free_tools.html> > makes the arguments well, IMO. > >> 2. I also feel that this proposal is far too cavalier in not even >> discussing the possibility of helping out the Mercurial team […] we'd >> prefer to switch to something else entirely rather than organising a >> sprint with them at PyCon to help ensure that our existing Mercurial >> based infrastructure is approachable for git & GitHub users? > > Exactly. For such a core tool, instead of pushing proprietary platforms > at the expense of software freedom, the sensible strategy for a project > (Python) that hopes to be around in the long term is to use and improve > the free software platforms.
I think there is a big difference here between using a closed source VCS or compiler and using a closed source code host. Namely in that the protocol is defined by git so switching from one host to another is easy.
It’s akin to saying that if we chose to run the PyPI services on a Windows machine that it is somehow makes it less-free even though we could have chosen to run it on a “free” OS and we weren’t doing much, if anything, to tie us to that particular OS.
If it makes people feel better we can continue to support the existing mechanisms of contribution, then people can choose between interacting with a “non free” host and “free” tooling. I suspect most people will choose the “non-free” tooling. _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
Donald Stufft <donald@stufft.io> writes:
I think there is a big difference here between using a closed source VCS or compiler and using a closed source code host. Namely in that the protocol is defined by git so switching from one host to another is easy.
GitHub deliberately encourages proprietary features that create valuable data that cannot be exported — the proprietary GitHub-specific pull requests being a prime example. So it is only true to say one can export the data to a different host if one entirely abandons all those features which create GitHub-proprietary data. If you want to re-write the PEP to be clear you are only talking about the repository hosting of GitHub, and *not* any of its extra features that make it so attractive to use their proprietary service, I'd be interested to see that. On the other hand, if you want to promote those proprietary features as part of the PEP, then it is disingenuous to claim the data can be easily exported to a different host. -- \ “Do not enter the lift backwards, and only when lit up.” | `\ —elevator, Leipzig | _o__) | Ben Finney
On Nov 30, 2014, at 6:18 AM, Ben Finney <ben+python@benfinney.id.au> wrote:
Donald Stufft <donald@stufft.io> writes:
I think there is a big difference here between using a closed source VCS or compiler and using a closed source code host. Namely in that the protocol is defined by git so switching from one host to another is easy.
GitHub deliberately encourages proprietary features that create valuable data that cannot be exported — the proprietary GitHub-specific pull requests being a prime example.
Once a Pull Request is merged there is little benefit to the PR itself, however if you want the diff you can create a .patch file by appending .diff or .patch to the end of the PR URL.
So it is only true to say one can export the data to a different host if one entirely abandons all those features which create GitHub-proprietary data.
That’s not true at all. There is an API that allows you to access *all* of the data that is stored there. Allowing you to replicate it to whatever system you wish.
If you want to re-write the PEP to be clear you are only talking about the repository hosting of GitHub, and *not* any of its extra features that make it so attractive to use their proprietary service, I'd be interested to see that.
On the other hand, if you want to promote those proprietary features as part of the PEP, then it is disingenuous to claim the data can be easily exported to a different host.
-- \ “Do not enter the lift backwards, and only when lit up.” | `\ —elevator, Leipzig | _o__) | Ben Finney
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 2014-11-30, 11:18 GMT, Ben Finney wrote:
Donald Stufft <donald@stufft.io> writes:
I think there is a big difference here between using a closed source VCS or compiler and using a closed source code host. Namely in that the protocol is defined by git so switching from one host to another is easy.
GitHub deliberately encourages proprietary features that create valuable data that cannot be exported — the proprietary GitHub-specific pull requests being a prime example.
What I really don’t understand is why this discussion is hg v. GitHub, when it should be hg v. git. Particular hosting is a secondary issue and it could be GitHub or git.python.org (with some FLOSS git hosting package ... cgit/gitolite, gitorious, gitlab, etc.) or python.gitorious.org (I believe Gitorious people might be happy to host you) or whatever else. Best, Matěj
M. Cepl asked:
What I really don't understand is why this discussion is hg v. GitHub, when it should be hg v. git. Particular hosting is a secondary issue
I think even the proponents concede that git isn't better enough to justify a switch in repositories. They do claim that GitHub (the whole environment; not just the hosting) is so much better that a switch to GitHub is justified. Github + hg offers far fewer benefits than Github + git, so also switching to git is part of the price. Whether that is an intolerable markup or a discount is disputed, as are the value of several other costs and benefits. -jJ -- If there are still threading problems with my replies, please email me with details, so that I can try to resolve them. -jJ
On Mon, Dec 1, 2014 at 12:37 PM, Jim J. Jewett <jimjjewett@gmail.com> wrote:
I think even the proponents concede that git isn't better enough to justify a switch in repositories.
There are also many who find the Bitbucket tools more usable than the Github tools. I'm not aware of any functional differences (though I don't often use Github myself), but the Bitbucket UIs have a much cleaner feel to them. -Fred -- Fred L. Drake, Jr. <fred at fdrake.net> "A storm broke loose in my mind." --Albert Einstein
hg vs Github
At best, this is apples to oranges in comparing a DVCS to a platform, or was the intention to change the subject to "hg vs git"? If so, then it's promoting a developer tool war in the same vein as the never ending vim vs emacs and will likely only result in continued dissension. IMHO, there's really no point in continuing this discussion past decisions to be made by a select few in the thread discussing PEP 481. On Mon, Dec 1, 2014 at 9:56 AM, Fred Drake <fred@fdrake.net> wrote:
On Mon, Dec 1, 2014 at 12:37 PM, Jim J. Jewett <jimjjewett@gmail.com> wrote:
I think even the proponents concede that git isn't better enough to justify a switch in repositories.
There are also many who find the Bitbucket tools more usable than the Github tools.
I'm not aware of any functional differences (though I don't often use Github myself), but the Bitbucket UIs have a much cleaner feel to them.
-Fred
-- Fred L. Drake, Jr. <fred at fdrake.net> "A storm broke loose in my mind." --Albert Einstein _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/demianbrecht%40gmail.com
-- Demian Brecht https://demianbrecht.github.io https://github.com/demianbrecht
On Mon, Dec 1, 2014 at 8:37 PM, Jim J. Jewett <jimjjewett@gmail.com> wrote:
What I really don't understand is why this discussion is hg v. GitHub, when it should be hg v. git. Particular hosting is a secondary issue
I think even the proponents concede that git isn't better enough to justify a switch in repositories.
They do claim that GitHub (the whole environment; not just the hosting) is so much better that a switch to GitHub is justified.
Github + hg offers far fewer benefits than Github + git, so also switching to git is part of the price. Whether that is an intolerable markup or a discount is disputed, as are the value of several other costs and benefits.
git - GitHub < hg + hgweb GitHub + hg < hg + hgweb GitHub + git > hg + hgweb GitHub + git <= hg + hgweb + Roundup GitHub + git < hg + hgweb + Roundup + Rietveld (GitHub + git) * Python = 0 (hg + hgweb) * Python = 1 There is also a question of support. Obviously Roundup will degrade over time if there is economic support for coordination action for people working on it, no matter how strong is their open source and community spirits. It was fun back in the days, but now time control and "human resource mining" techniques are so perfect that I hardly imagine someone to have free time to work on community open source projects. With no time to support our own dogfood, the community will have to switch to GitHub sooner or later, because that is the thing that saves time so valuable. GitLab could be a open source alternative if Python community chooses to be more tolerant to use Ruby tools dogfood in its ecosystem.
Links to OT (pending a mailman upgrade with auto-appended links): https://mail.python.org/pipermail/python-dev/2014-November/137283.html https://mail.python.org/pipermail/python-dev/2014-December/thread.html#13738... http://westurner.github.io/wiki/ideas#open-source-mailing-list-extractor (sphinx + github wiki repo) On Dec 1, 2014 11:37 AM, "Jim J. Jewett" <jimjjewett@gmail.com> wrote:
M. Cepl asked:
What I really don't understand is why this discussion is hg v. GitHub, when it should be hg v. git. Particular hosting is a secondary issue
I think even the proponents concede that git isn't better enough to justify a switch in repositories.
They do claim that GitHub (the whole environment; not just the hosting) is so much better that a switch to GitHub is justified.
Github + hg offers far fewer benefits than Github + git, so also switching to git is part of the price. Whether that is an intolerable markup or a discount is disputed, as are the value of several other costs and benefits.
-jJ
--
If there are still threading problems with my replies, please email me with details, so that I can try to resolve them. -jJ _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
I have some questions and/or issues with the PEP, but first I'm going to add something to Nick's comments: On Sun, Nov 30, 2014 at 11:12:17AM +1000, Nick Coghlan wrote:
Beyond that, GitHub is indeed the most expedient option. My two main reasons for objecting to taking the expedient path are:
1. I strongly believe that the long term sustainability of the overall open source community requires the availability and use of open source infrastructure. While I admire the ingenuity of the "free-as-in-beer" model for proprietary software companies fending off open source competition, I still know a proprietary platform play when I see one (and so do venture capitalists looking to extract monopoly rents from the industry in the future). (So yes, I regret relenting on this principle in previously suggesting the interim use of another proprietary hosted service)
2. I also feel that this proposal is far too cavalier in not even discussing the possibility of helping out the Mercurial team to resolve their documentation and usability issues rather than just yelling at them "your tool isn't popular enough for us, and we find certain aspects of it too hard to use, so we're switching to something else rather than working with you to address our concerns". We consider the Mercurial team a significant enough part of the Python ecosystem that Matt was one of the folks specifically invited to the 2014 language summit to discuss their concerns around the Python 3 transition. Yet we'd prefer to switch to something else entirely rather than organising a sprint with them at PyCon to help ensure that our existing Mercurial based infrastructure is approachable for git & GitHub users? (And yes, I consider some of the core Mercurial devs to be friends, so this isn't an entirely abstract concern for me)
Thanks Nick, I think these are excellent points, particularly the second. It would be a gross strawman to say that we should "only" use software developed in Python, but we should eat our own dogfood whenever practical and we should support and encourage the Python ecosystem, including Mercurial. Particularly since hg and git are neck and neck feature-wise, we should resist the tendency to jump on bandwagons. If git were clearly the superior product, then maybe there would be an argument for using the best tool for the job, but it isn't. As for the question of using Github hosting, there's another factor which has been conspicuous by its absence. Has GitHub's allegedly toxic and bullying culture changed since Julie Horvath quit in March? And if it has not, do we care? I'm not a saint, but I do try to choose ethical companies and institutions over unethical ones whenever it is possible and practical. I'm not looking for a witch-hunt against GitHub, but if the allegations made by Horvath earlier this year are true, and I don't believe anyone has denied them, then so long as GitHub's internal culture remains sexist and hostile to the degree reported, then I do not believe that we should use GitHub's services even if we shift some repos to git. I have serious doubts about GitHub's compatibility with the ideals expressed by the PSF. Even if our code of conduct does not explicitly forbid it, I think that it goes against the principles that we say we aspire to. Given Horvath's experiences, and the lack of clear evidence that anything has changed in GitHub, I would be deeply disappointed if Python lent even a smidgeon of legitimacy to their company, and I personally will not use their services. I acknowledge that it's hard to prove a negative, and GitHub may have difficulty proving to my satisfaction that they have changed. (My experience is that company culture rarely changes unless there is a change in management, and even then only slowly.) Particularly given GitHub's supposed egalitarian, non-hierarchical, and meritocratic structure, that nobody apparently saw anything wrong with the bullying of staff and workplace sexism until it became public knowledge suggests that it is not just a few bad apples but a problem all through the company. -- Steven
On Sat, Nov 29, 2014 at 3:27 PM, Donald Stufft <donald@stufft.io> wrote:
As promised in the "Move selected documentation repos to PSF BitBucket account?" thread I've written up a PEP for moving selected repositories from hg.python.org to Github.
FWIW, I'm a pretty solid -1 to this PEP. Don't get me wrong, I'm much more accustomed to git than I am hg, much prefer git's lightweight branching model and would love to see CPython and all ancillary repos migrated to git & Github. If that was what this PEP was after, I'd be a +1. What I don't like about it is the introduction of multiple tools that directly impact the barrier of entry to contributing. I think that splitting ancillary repos such as PEPs and docs out might be a little short sighted at an overall project level. In my mind, there are three major categories of contributors (and prospective contributors): 1. Those that use git on a daily basis 2. Those that use hg on a daily basis 3. Those who use neither and are more accustomed to Perforce, SVN and the like Let's say this PEP is approved and the suggested repos are moved to Github. For git users, life is suddenly made much easier when contributing to those projects for obvious reasons. However, they still have the same barrier of entry to contributing to CPython (I would imagine that this would be the goal for most users, but maybe I'm wrong about that). I would imagine that contributing to the ancillary repos would be great grounds to ramp up on using hg before hitting CPython with its multiple long lived branches and such. Making the switch as suggested by this PEP removes that. For hg users, you now add a barrier of entry for contributing to the repos now living on Github. In both cases, you've introduced the need to context switch when contributing to CPython and any of the other repos. Two tools that require quite different workflows. Then, for the third class of users, you've now introduced the requirement of learning two different sets of tools (if they want to do anything outside of using the "Edit" button through Github's UI). Now you're looking at conflated contributor documentation or project-specific documentation. IMHO, suboptimal either way you look at it. Personally, I don't think that there are any silver bullets to this problem. In no case is everyone going to be satisfied. In cases like that, I tend to think that no matter what the solution eventually agreed upon is, consistency is of the utmost importance. Moving a number of repos to Github breaks that consistency. What *would* be nice if CPython was to stay on mercurial though is perhaps moving those repos to Bitbucket. In that case, you get both the "Edit" feature that really by all account removes the initial bar of entry, but you still remain consistent with the rest of the project.
On 11/29/2014 10:14 PM, Demian Brecht wrote:
On Sat, Nov 29, 2014 at 3:27 PM, Donald Stufft <donald@stufft.io <mailto:donald@stufft.io>> wrote:
As promised in the "Move selected documentation repos to PSF BitBucket account?" thread I've written up a PEP for moving selected repositories from hg.python.org <http://hg.python.org> to Github.
FWIW, I'm a pretty solid -1 to this PEP.
Agreed. -1 for the same reasons (plus most of Nick's). -- ~Ethan~
On 29 November 2014 at 23:27, Donald Stufft <donald@stufft.io> wrote:
In previous years there was concern about how well supported git was on Windows in comparison to Mercurial. However git has grown to support Windows as a first class citizen. In addition to that, for Windows users who are not well aquanted with the Windows command line there are GUI options as well.
I have little opinion on the PEP as a whole, but is the above statement true? From the git website, version 2.2.0 is current, and yet the downloadable Windows version is still 1.9.4. That's a fairly significant version lag for a "first class citizen". I like git, and it has a number of windows-specific extensions that are really useful (more than Mercurial, AFAIK), but I wouldn't say that the core product supported Windows on an equal footing to Linux. Paul
On Nov 30, 2014, at 7:31 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 29 November 2014 at 23:27, Donald Stufft <donald@stufft.io> wrote:
In previous years there was concern about how well supported git was on Windows in comparison to Mercurial. However git has grown to support Windows as a first class citizen. In addition to that, for Windows users who are not well aquanted with the Windows command line there are GUI options as well.
I have little opinion on the PEP as a whole, but is the above statement true? From the git website, version 2.2.0 is current, and yet the downloadable Windows version is still 1.9.4. That's a fairly significant version lag for a "first class citizen".
I like git, and it has a number of windows-specific extensions that are really useful (more than Mercurial, AFAIK), but I wouldn't say that the core product supported Windows on an equal footing to Linux.
Paul
I think so yes. I may be wrong, however while 1.9.4 may be the latest downloadable version of git for Windows, there is no downloadable version of the Linux clients at all, they just tell you to go use your package manager which for instance is version 1.7 on Debian. On OS X the latest version is 2.0.1. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 30 November 2014 at 16:08, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 7:31 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 29 November 2014 at 23:27, Donald Stufft <donald@stufft.io> wrote:
In previous years there was concern about how well supported git was on Windows in comparison to Mercurial. However git has grown to support Windows as a first class citizen. In addition to that, for Windows users who are not well aquanted with the Windows command line there are GUI options as well.
I have little opinion on the PEP as a whole, but is the above statement true? From the git website, version 2.2.0 is current, and yet the downloadable Windows version is still 1.9.4. That's a fairly significant version lag for a "first class citizen".
I like git, and it has a number of windows-specific extensions that are really useful (more than Mercurial, AFAIK), but I wouldn't say that the core product supported Windows on an equal footing to Linux.
Paul
I think so yes. I may be wrong, however while 1.9.4 may be the latest downloadable version of git for Windows, there is no downloadable version of the Linux clients at all, they just tell you to go use your package manager which for instance is version 1.7 on Debian. On OS X the latest version is 2.0.1.
OTOH, presumably you can build your own copy of git from source on Linux/OSX. I haven't tried this on Windows but it looks pretty difficult (you start by downloading the msysgit development environment and go from there). Also, if it's easy to produce binaries for 2.2.0 on Windows, why haven't the msysgit project (still an external project, to an extent, AFAICT) done so? Paul
Can this discussion be split off into a separate discussion. It's tangential to the PEP and clearly not actively progressing so it doesn't seem productive. I don't care where it's taken, but I don't think this belongs here. Speculation on the actions of the msysgit project are not fair talk for this PEP. On Sun, Nov 30, 2014 at 12:14 PM, Paul Moore <p.f.moore@gmail.com> wrote:
On 30 November 2014 at 16:08, Donald Stufft <donald@stufft.io> wrote:
On Nov 30, 2014, at 7:31 AM, Paul Moore <p.f.moore@gmail.com> wrote:
On 29 November 2014 at 23:27, Donald Stufft <donald@stufft.io> wrote:
In previous years there was concern about how well supported git was on Windows in comparison to Mercurial. However git has grown to support Windows as a first class citizen. In addition to that, for Windows users who are not well aquanted with the Windows command line there are GUI options as well.
I have little opinion on the PEP as a whole, but is the above statement true? From the git website, version 2.2.0 is current, and yet the downloadable Windows version is still 1.9.4. That's a fairly significant version lag for a "first class citizen".
I like git, and it has a number of windows-specific extensions that are really useful (more than Mercurial, AFAIK), but I wouldn't say that the core product supported Windows on an equal footing to Linux.
Paul
I think so yes. I may be wrong, however while 1.9.4 may be the latest downloadable version of git for Windows, there is no downloadable version of the Linux clients at all, they just tell you to go use your package manager which for instance is version 1.7 on Debian. On OS X the latest version is 2.0.1.
OTOH, presumably you can build your own copy of git from source on Linux/OSX. I haven't tried this on Windows but it looks pretty difficult (you start by downloading the msysgit development environment and go from there). Also, if it's easy to produce binaries for 2.2.0 on Windows, why haven't the msysgit project (still an external project, to an extent, AFAICT) done so?
Paul _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/graffatcolmingov%40gmail....
On 11/30/2014 04:31 AM, Paul Moore wrote:
On 29 November 2014 at 23:27, Donald Stufft <donald@stufft.io> wrote:
In previous years there was concern about how well supported git was on Windows in comparison to Mercurial. However git has grown to support Windows as a first class citizen. In addition to that, for Windows users who are not well aquanted with the Windows command line there are GUI options as well.
I have little opinion on the PEP as a whole, but is the above statement true? From the git website, version 2.2.0 is current, and yet the downloadable Windows version is still 1.9.4. That's a fairly significant version lag for a "first class citizen".
I like git, and it has a number of windows-specific extensions that are really useful (more than Mercurial, AFAIK), but I wouldn't say that the core product supported Windows on an equal footing to Linux.
I'm curious about These useful extension. Can you elaborate? -- Pierre-Yves David
On 11/30/2014 04:31 AM, Paul Moore wrote:
On 29 November 2014 at 23:27, Donald Stufft<donald@stufft.io> wrote:
In previous years there was concern about how well supported git was on Windows in comparison to Mercurial. However git has grown to support Windows as a first class citizen. In addition to that, for Windows users who are not well aquanted with the Windows command line there are GUI options as well.
Mercurial have robust Windows support for a long time. This support is native (not using cygwin) and handle properly all kind of strange corner case. We have large scale ecosystem (http://unity3d.com/) using Mercurial on windows. We also have full featured GUI client http://tortoisehg.bitbucket.org/. It is actively developed by people who stay in touch with the Mercurial upstream so new feature tend to land in the GUI really fast. -- Pierre-Yves David
On 11/30/2014 8:19 PM, Pierre-Yves David wrote:
Mercurial have robust Windows support for a long time. This support is native (not using cygwin) and handle properly all kind of strange corner case. We have large scale ecosystem (http://unity3d.com/) using Mercurial on windows.
We also have full featured GUI client http://tortoisehg.bitbucket.org/.
Tortoisehg comes with Hg Workbench, which I use for everything I can. The only thing I do not use it for is file (versus repository/changeset) commands, like file annotation. I do that with right click in Windows Explorer. In my current physical state, having to do everything at a Windows command prompt would be more onerous. There exists a TortoiseGit, but it lacks the unifying Workbench GUI. There exists at least one 3rd party workbench that purports to work with both hg and git. I have not looked at it yet.
It is actively developed by people who stay in touch with the Mercurial upstream so new feature tend to land in the GUI really fast.
-- Terry Jan Reedy
One argument that keeps coming up is transferability of knowledge: knowing git and/or GitHub, as many seem to, it therefore becomes easier to commit to the Python ecosystem. What about the transferability of Python knowledge? Because I know Python, I can customize hg; because I know Python I can customize Roundup. I do not choose tools simply because they are written in Python -- I choose them because, being written in Python, I can work on them if I need to: I can enhance them, I can fix them, I can learn from them. -- ~Ethan~
On Dec 1, 2014, at 1:25 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
One argument that keeps coming up is transferability of knowledge: knowing git and/or GitHub, as many seem to, it therefore becomes easier to commit to the Python ecosystem.
What about the transferability of Python knowledge? Because I know Python, I can customize hg; because I know Python I can customize Roundup.
I do not choose tools simply because they are written in Python -- I choose them because, being written in Python, I can work on them if I need to: I can enhance them, I can fix them, I can learn from them.
-- ~Ethan~
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io
Git uses the idea of small individual commands that do small things, so you can write your own commands that work on text streams to extend git and you can even write those in Python. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 2014-12-01, 07:43 GMT, Donald Stufft wrote:
I do not choose tools simply because they are written in Python -- I choose them because, being written in Python, I I can work on them if I need to: I can enhance them, I can fix them, I can learn from them.
Git uses the idea of small individual commands that do small things, so you can write your own commands that work on text streams to extend git and you can even write those in Python.
I really really dislike this Mercurial propaganda for two reasons: a) obviously you are right ... git is a complete tool box for building your own tools in the best UNIX™ traditions. Each of has a ton of third-party (or our own) tools using git plumbing. (Is there a Mercurial equivalent of git-filter-branch? Can http://mercurial.selenic.com/wiki/ConvertExtension do the same as git-filter-branch?) b) it completely ignores existence of three (3) independent implementations of git format/protocol (also jgit and libgit2). How does VisualStudio/Eclipse/NetBeans/etc. support for hg works? Does it use a library or just runs hg binary in a subprocess (a thing which by the hg authors is Mercurial not designed to do)? Best, Matěj
On 12/01/2014 01:05 PM, Matěj Cepl wrote:
On 2014-12-01, 07:43 GMT, Donald Stufft wrote:
I do not choose tools simply because they are written in Python -- I choose them because, being written in Python, I I can work on them if I need to: I can enhance them, I can fix them, I can learn from them.
Git uses the idea of small individual commands that do small things, so you can write your own commands that work on text streams to extend git and you can even write those in Python.
I really really dislike this Mercurial propaganda for two reasons:
a) obviously you are right ... git is a complete tool box for building your own tools in the best UNIX™ traditions. Each of has a ton of third-party (or our own) tools using git plumbing. (Is there a Mercurial equivalent of git-filter-branch? Can http://mercurial.selenic.com/wiki/ConvertExtension do the same as git-filter-branch?) b) it completely ignores existence of three (3) independent implementations of git format/protocol (also jgit and libgit2). How does VisualStudio/Eclipse/NetBeans/etc. support for hg works? Does it use a library or just runs hg binary in a subprocess (a thing which by the hg authors is Mercurial not designed to do)?
Please at least try to get your facts right. """ For the vast majority of third party code, the best approach is to use Mercurial's published, documented, and stable API: the command line interface. """ http://mercurial.selenic.com/wiki/MercurialApi Georg
On Mon, Dec 1, 2014 at 12:25 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
One argument that keeps coming up is transferability of knowledge: knowing git and/or GitHub, as many seem to, it therefore becomes easier to commit to the Python ecosystem.
What about the transferability of Python knowledge? Because I know Python, I can customize hg; because I know Python I can customize Roundup.
I do not choose tools simply because they are written in Python -- I choose them because, being written in Python, I can work on them if I need to: I can enhance them, I can fix them, I can learn from them.
There are lots of Python tools written with Git: * https://pypi.python.org/pypi/vcs * https://pypi.python.org/pypi/dulwich * https://pypi.python.org/pypi/hg-git * http://docs.saltstack.com/en/latest/topics/tutorials/gitfs.html ("GitFS") * https://github.com/libgit2/pygit2 (C) * https://pypi.python.org/pypi/GitPython (Python) * https://pypi.python.org/pypi/pyrpo (subprocess wrapper for git, hg, bzr, svn)
Here's a roundup of tools links, to make sure we're all on the same page: Git HG Rosetta Stone =================== https://github.com/sympy/sympy/wiki/Git-hg-rosetta-stone#rosetta-stone BugWarrior =========== BugWarrior works with many issue tracker APIs https://warehouse.python.org/project/bugwarrior/ bugwarrior is a command line utility for updating your local taskwarrior
database from your forge issue trackers. It currently supports the following remote resources:
- github (api v3) - bitbucket - trac - bugzilla - megaplan - teamlab - redmine - jira - activecollab (2.x and 4.x) - phabricator
[...]
DVCS Interaction ================ Hg <-> Git ---------------- * https://warehouse.python.org/project/hg-git/ (dulwich) * hg-github https://github.com/stephenmcd/hg-github Git <-> Hg ------------------ * https://pypi.python.org/pypi/git-remote-hg/ * https://github.com/felipec/git-remote-hg Python <-> Hg ----------------------- | Wikipedia: https://en.wikipedia.org/wiki/Mercurial | Homepage: http://hg.selenic.org/ | Docs: http://mercurial.selenic.com/guide | Docs: http://hgbook.red-bean.com/ | Source: hg http://selenic.com/hg | Source: hg http://hg.intevation.org/mercurial/crew * http://evolution.experimentalworks.net/doc/user-guide.html * (growing list of included extensions) Python <-> Git ---------------------- * GitPython, pygit2 (libgit2), dulwich * https://github.com/libgit2/pygit2 (libgit2) * https://pythonhosted.org/GitPython/ (Python) * https://github.com/jelmer/dulwich (Python) * http://docs.saltstack.com/en/latest/topics/tutorials/gitfs.html#installing-d... GitHub -> BitBucket ----------------------------- * https://bitbucket.org/ZyX_I/gibiexport Sphinx Documentation ==================== * http://read-the-docs.readthedocs.org/en/latest/webhooks.html * https://github.com/yoloseem/awesome-sphinxdoc * changelogs, charts, csv, ipython, %doctest_mode Is there an issue ticket or a wiki page that supports Markdown/ReStructuredText, where I could put this? Which URI do we assign to this artifact? On Mon, Dec 1, 2014 at 8:37 AM, Wes Turner <wes.turner@gmail.com> wrote:
On Mon, Dec 1, 2014 at 12:25 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
One argument that keeps coming up is transferability of knowledge: knowing git and/or GitHub, as many seem to, it therefore becomes easier to commit to the Python ecosystem.
What about the transferability of Python knowledge? Because I know Python, I can customize hg; because I know Python I can customize Roundup.
I do not choose tools simply because they are written in Python -- I choose them because, being written in Python, I can work on them if I need to: I can enhance them, I can fix them, I can learn from them.
There are lots of Python tools written with Git:
* https://pypi.python.org/pypi/vcs * https://pypi.python.org/pypi/dulwich * https://pypi.python.org/pypi/hg-git * http://docs.saltstack.com/en/latest/topics/tutorials/gitfs.html ("GitFS") * https://github.com/libgit2/pygit2 (C) * https://pypi.python.org/pypi/GitPython (Python) * https://pypi.python.org/pypi/pyrpo (subprocess wrapper for git, hg, bzr, svn)
Hi! On Mon, Dec 01, 2014 at 10:42:16AM -0600, Wes Turner <wes.turner@gmail.com> wrote:
Here's a roundup of tools links, to make sure we're all on the same page:
Very nice!
Is there an issue ticket or a wiki page that supports Markdown/ReStructuredText, where I could put this? Which URI do we assign to this artifact?
There are already pages https://wiki.python.org/moin/Git and https://wiki.python.org/moin/Mercurial . You can create an additional page and reference it on that pages. Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
On 12/1/2014 11:42 AM, Wes Turner wrote:
Here's a roundup of tools links, to make sure we're all on the same page:
Git HG Rosetta Stone =================== https://github.com/sympy/sympy/wiki/Git-hg-rosetta-stone#rosetta-stone
BugWarrior =========== BugWarrior works with many issue tracker APIs
https://warehouse.python.org/project/bugwarrior/
bugwarrior is a command line utility for updating your local taskwarrior database from your forge issue trackers. It currently supports the following remote resources:
* github (api v3) * bitbucket * trac * bugzilla * megaplan * teamlab * redmine * jira * activecollab (2.x and 4.x) * phabricator
[...]
DVCS Interaction ================
Hg <-> Git ---------------- * https://warehouse.python.org/project/hg-git/ (dulwich) * hg-github https://github.com/stephenmcd/hg-github
Git <-> Hg ------------------ * https://pypi.python.org/pypi/git-remote-hg/ * https://github.com/felipec/git-remote-hg
Python <-> Hg ----------------------- | Wikipedia: https://en.wikipedia.org/wiki/Mercurial | Homepage: http://hg.selenic.org/ | Docs: http://mercurial.selenic.com/guide | Docs: http://hgbook.red-bean.com/ | Source: hg http://selenic.com/hg | Source: hg http://hg.intevation.org/mercurial/crew
* http://evolution.experimentalworks.net/doc/user-guide.html * (growing list of included extensions)
Python <-> Git ---------------------- * GitPython, pygit2 (libgit2), dulwich * https://github.com/libgit2/pygit2 (libgit2) * https://pythonhosted.org/GitPython/ (Python) * https://github.com/jelmer/dulwich (Python) * http://docs.saltstack.com/en/latest/topics/tutorials/gitfs.html#installing-d...
GitHub -> BitBucket ----------------------------- * https://bitbucket.org/ZyX_I/gibiexport
Sphinx Documentation ==================== * http://read-the-docs.readthedocs.org/en/latest/webhooks.html * https://github.com/yoloseem/awesome-sphinxdoc * changelogs, charts, csv, ipython, %doctest_mode
Is there an issue ticket or a wiki page that supports
Markdown/ReStructuredText,
whoops, I am not sure what moin uses.
where I could put this? Which URI do we assign to this artifact?
-- Terry Jan Reedy
On Mon, Dec 01, 2014 at 03:52:21PM -0500, Terry Reedy <tjreedy@udel.edu> wrote:
On 12/1/2014 11:42 AM, Wes Turner wrote:
Is there an issue ticket or a wiki page that supports
Markdown/ReStructuredText,
whoops, I am not sure what moin uses.
Let's see... https://wiki.python.org/moin/?action=raw Seems like reST. Oleg. -- Oleg Broytman http://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
participants (32)
-
Akira Li -
Alex Gaynor -
anatoly techtonik -
Antoine Pitrou -
Barry Warsaw -
Ben Finney -
Benjamin Peterson -
Brett Cannon -
Chris Angelico -
Cédric Krier -
Demian Brecht -
Donald Stufft -
Eric Snow -
Ethan Furman -
Fred Drake -
Georg Brandl -
Glyph -
Guido van Rossum -
Ian Cordasco -
Jim J. Jewett -
Larry Hastings -
Markus Unterwaditzer -
Matěj Cepl -
Nick Coghlan -
Oleg Broytman -
Paul Moore -
Pierre-Yves David -
Stephen J. Turnbull -
Steven D'Aprano -
Terry Reedy -
Tres Seaver -
Wes Turner