[Python-checkins] peps: Split software forge proposal into a new PEP

nick.coghlan python-checkins at python.org
Sat Jul 19 11:49:43 CEST 2014 

http://hg.python.org/peps/rev/0f5f7c61c522
changeset:   5500:0f5f7c61c522
user:        Nick Coghlan <ncoghlan at gmail.com>
date:        Sat Jul 19 19:49:30 2014 +1000
summary:
  Split software forge proposal into a new PEP

- moved the software forge proposal out to its own PEP
- switched forge proposal to Kallithea based on SFC concerns
  with the current RhodeCode licensing structure

files:
  pep-0462.txt |  161 +++-----------------------------------
  pep-0474.txt |  161 +++++++++++++++++++++++++++++++++++++++
  2 files changed, 175 insertions(+), 147 deletions(-)


diff --git a/pep-0462.txt b/pep-0462.txt
--- a/pep-0462.txt
+++ b/pep-0462.txt
@@ -21,11 +21,6 @@
 experience for other contributors that are reliant on the core team to get
 their changes incorporated.
 
-This PEP also proposes changes to the way certain supporting repositories
-(such as the repository for Python Enhancement Proposals) are managed to
-make them more accessible to new contributors, and easier to manage for
-core developers.
-
 
 Rationale for changes to the core development workflow
 ======================================================
@@ -158,23 +153,6 @@
   the case for OpenStack).
 
 
-Rationale for changes to source code repository management
-==========================================================
-
-Currently, hg.python.org hosts more than just the core CPython repository,
-it also hosts other repositories such as those for the CPython developer
-guide and for Python Enhancement Proposals, along with various "sandbox"
-repositories for core developer experimentation.
-
-These supporting repositories could benefit greatly from user the simple
-"pull request" style workflow made popular by code hosting sites like
-GitHub and BitBucket.
-
-This PEP proposes introducing a more sophisticated approach to repository
-management that includes more "self service" features, including support
-for pull requests.
-
-
 Current Tools
 =============
 
@@ -198,53 +176,8 @@
 Proposal
 ========
 
-This proposal consists of two key components:
-
-* Introducing RhodeCode for improved repository management
-* Introducing automated merge gating for the CPython project
-
-
-Improved repository management
-------------------------------
-
-The rise of free source code hosting sites like GitHub and BitBucket with a
-focus on improving the user experience have increased user expectations
-for the web based interface offered for source code repositories. This
-includes features like dynamic control of user permissions for repository
-administrators, online editing for documentation updates and similar small
-fixes, easy cloning and branching, etc.
-
-While GitHub and BitBucket are proprietary solutions to this problem that
-cannot be self-hosted, RhodeCode_ is a popular primarily open source solution
-that offers similar features to the major free hosting sites, while also
-allowing installation on your own servers. RhodeCode is also implemented
-in Python, allowing us to preserve our current situation of having our
-workflow tools being almost entirely self-hosting.
-
-There's nothing too complicated with this part of the proposal: it is
-essentially just a suggestion to convert hg.python.org to being backed
-by a RhodeCode Enterprise instance hosted on PSF infrastructure.
-
-All of the functional parts of RhodeCode Enterprise are open source under
-the GPLv3 license. The "look & feel" components are licensed under a
-`business source license`_ that is free for up to 20 users, as well as for
-organisations that qualify for one of RhodeCode's exemption categories
-(in the case of ``hg.python.org``, the relevant category is "public open
-source project").
-
-The RhodeCode developers have also expressed interest in helping out with
-ensuring a python.org RhodeCode deployment is successful.
-
-.. _RhodeCode: https://rhodecode.com/
-.. _business source license: https://rhodecode.com/licenses
-
-
-Automated merge gating for CPython
-----------------------------------
-
-The essence of this part of the proposal is that CPython aim to adopt a
-"core reviewer" development model, similar to that used by the OpenStack
-project.
+The essence of this proposal is that CPython aim to adopt a "core reviewer"
+development model, similar to that used by the OpenStack project.
 
 The workflow problems experienced by the CPython core development team are
 not unique. The OpenStack infrastructure team have come up with a well
@@ -255,7 +188,7 @@
 * patches never get merged without being tested relative to the current
   state of the branch
 * the main development branch always stays green. Patches that do not pass
-  the automated tests do not get merged.
+  the automated tests do not get merged
 
 If a core developer wants to tweak a patch prior to merging, they download
 it from the review tool, modify and *upload it back to the review tool*
@@ -360,25 +293,6 @@
 Perceived Benefits
 ==================
 
-Improved repository management
-------------------------------
-
-The primary benefit of using RhodeCode to manage hg.python.org is that
-supporting repositories such as the developer guide and the PEP repo could
-now be managed using pull requests and online editing. This would be
-much simpler then the current workflow which requires PEP editors and
-other core developers to act as intermediaries to apply updates suggested
-by other users.
-
-The richer administrative functionality would also make it substantially
-easier to grant users access to particular repositories for collaboration
-purposes, without having to grant them general access to the entire
-installation.
-
-
-Automated merge gating for CPython
-----------------------------------
-
 The benefits of this proposal accrue most directly to the core development
 team. First and foremost, it means that once we mark a patch as "Approved"
 in the updated code review system, *we're usually done*. The extra 20-30
@@ -438,35 +352,6 @@
 Zuul trigger and action plugins, and may require additional development
 in some of our existing tools.
 
-RhodeCode may also require some adjustment to fit in with existing
-infrastructure.
-
-
-User account management
------------------------
-
-Ideally we'd be able to offer a single account that spans all python.org
-services, including RhodeCode, Roundup/Rietveld, PyPI and the back end for
-the new python.org site, but actually implementing that would be a distinct
-infrastructure project, independent of this PEP.
-
-A potentially simpler near term solution would be if RhodeCode's user
-account management could be integrated with the existing account management
-in Roundup, similar to what was done for Rietveld. However, if that also
-turns out to be impractical in the near term, we would merely end up with
-another identity silo to be integrated at a later date, suggesting that
-this doesn't need to be considered a blocker for an initial RhodeCode
-deployment.
-
-
-Preserving existing SSH access and links for Mercurial repositories
--------------------------------------------------------------------
-
-Adopting RhodeCode may result in some changes to where repositories are
-located on the hg.python.org filesystem. We may want to ensure that
-existing links into the hg.python.org web service are preserved, and should
-definitely ensure that existing SSH based clones continue to work correctly.
-
 
 Rietveld/Roundup vs Gerrit
 --------------------------
@@ -512,18 +397,10 @@
 version control system.
 
 Zuul is also directly integrated with git for patch manipulation - as far
-as I am aware, this part of the design isn't pluggable.
-
-Rather than trying to adapt Zuul to work directly with Mercurial, it will
-likely be more practical to let Zuul continue to use git internally, and
-then use the hg-git Mercurial plugin to pull the output from Zuul into the
-master repo at hg.python.org. (While there are various plugins that are
-designed to let git push to Mercurial repos, the influence of GitHub is
-such that the hg-git plugin appears to be the most mature of the available
-options for hg-git interoperability).
-
-One advantage of adopting RhodeCode to manage the repository infrastructure
-is that it supports git repositories in addition to Mercurial repositories.
+as I am aware, this part of the design isn't pluggable. However, at PyCon
+US 2014, the Mercurial core developers at the sprints expressed some
+interest in collaborating with the core development team and the Zuul
+developers on enabling the use of Zuul with Mercurial in addition to git.
 
 
 Buildbot vs Jenkins
@@ -779,10 +656,9 @@
 Open Questions
 ==============
 
-Pretty much everything in the PEP. Do we want to adopt RhodeCode? Do we
-want to adopt merge gating and Zuul? Is Rietveld the right place to hook
-Zuul into our current workflows? How do we want to address the various
-technical challenges?
+Pretty much everything in the PEP. Do we want to adopt merge gating and
+Zuul? Is Rietveld the right place to hook Zuul into our current workflows?
+How do we want to address the various technical challenges?
 
 Assuming we do want to do it (or something like it), how is the work going
 to get done? Do we try to get it done solely as a volunteer effort? Do we
@@ -798,15 +674,10 @@
 Next Steps
 ==========
 
-The topic of CPython workflow automation is on the agenda for the Language
-Summit at PyCon US 2014 in Montreal, and we will be inviting additional
-participants (specifically Mercurial and Zuul developers) to be involved
-in the discussions (Guido van Rossum is the creator of Rietveld, and these
-workflow changes are not expected to require any significant changes in
-Roundup or Buildbot).
-
-Unfortunately, the lead RhodeCode developers aren't able to attend PyCon US
-this year, or we would have invited them, too.
+Unfortunately, we ran out of time at the PyCon 2014 language summit to really
+discuss these issues. However, the `core-workflow mailing list
+<https://mail.python.org/mailman/listinfo/core-workflow>`__ has now been set
+up to discuss workflow issues in general.
 
 
 Acknowledgements
@@ -817,10 +688,6 @@
 Taylor for additional technical feedback following publication of the
 initial draft.
 
-Thanks also to Marcin Kuzminski (CTO of RhodeCode) for getting in touch to
-express their interest in helping to ensure the success of a RhodeCode
-deployment if we choose to proceed down that path.
-
 
 Copyright
 =========
diff --git a/pep-0474.txt b/pep-0474.txt
new file mode 100644
--- /dev/null
+++ b/pep-0474.txt
@@ -0,0 +1,161 @@
+PEP: 474
+Title: Creating forge.python.org
+Version: $Revision$
+Last-Modified: $Date$
+Author: Nick Coghlan <ncoghlan at gmail.com>
+Status: Draft
+Type: Process
+Content-Type: text/x-rst
+Created: 19-Jul-2014
+Post-History: 19-Jul-2014
+
+
+Abstract
+========
+
+This PEP proposes setting up a new PSF provided resource, forge.python.org,
+as a location for maintaining various supporting repositories
+(such as the repository for Python Enhancement Proposals) in a way that is
+more accessible to new contributors, and easier to manage for core
+developers.
+
+This PEP does *not* propose any changes to the core development workflow
+for CPython itself (see PEP 462 in relation to that).
+
+
+Proposal
+========
+
+This PEP proposes that, once the Kallithea project has made an official
+release, that a Kallithea instance be deployed as "forge.python.org".
+
+Individual repositories (such as the developer guide or the PEPs repository)
+may then be migrated from the existing hg.python.org infrastructure to the
+new forge.python.org infrastructure on a case by case basis. Each migration
+will need to decide whether to retain a read-only mirror on hg.python.org,
+or whether to just migrate wholesale to the new location.
+
+This would not be a general purpose hosting site for arbitrary Python
+projects, but a more narrowly focused site akin to the existing
+hg.python.org.
+
+
+Rationale
+=========
+
+Currently, hg.python.org hosts more than just the core CPython repository,
+it also hosts other repositories such as those for the CPython developer
+guide and for Python Enhancement Proposals, along with various "sandbox"
+repositories for core developer experimentation.
+
+While the simple "pull request" style workflow made popular by code hosting
+sites like GitHub and BitBucket isn't adequate for the complex branching
+model needed for parallel maintenance and development of the various
+CPython releases, it's a good fit for several of the ancillary projects
+that surround CPython that we don't wish to move to a proprietary hosting
+site.
+
+The key requirements proposed for a PSF provided software forge are:
+
+* Must support self-hosting on PSF infrastructure without ongoing fees
+* Must support Mercurial (for consistency with existing tooling)
+* Must support simple "pull request" style workflows
+* Must support online editing for simple changes
+
+Ideally, the chosen solution would also be a fully open source application
+written in Python.
+
+The requirement for self-hosting without ongoing fees rules out the
+free-as-in-beer providers like GitHub and BitBucket, in addition to the
+various proprietary source code management offerings.
+
+The requirement for Mercurial support not only rules out GitHub, but also
+other Git-only solutions like GitLab and Gitorious.
+
+The requirement for online editing support rules out the Apache
+Allura/HgForge combination.
+
+That leaves two main contenders from a technical perspective:
+
+* `RhodeCode <https://code.rhodecode.com/rhodecode>`__
+* `Kallithea SCM <https://kallithea-scm.org/>`__
+
+The `legal uncertainty
+<http://sfconservancy.org/blog/2014/jul/15/why-kallithea/>`__ over the
+interaction between RhodeCode's current "Business Source" licensing model and
+the various GPL components it relies on currently make it unclear whether it
+is legally permissible to deploy it.
+
+By contrast, Kallithea is a full GPLv3 application (derived from the clearly
+and unambiguously GPLv3 licensed components of RhodeCode) that is being
+developed under the auspices of the `Software Freedom Conservancy
+<http://sfconservancy.org/news/2014/jul/04/kallithea-joins/>`__. The
+Conservancy has `affirmed
+<http://sfconservancy.org/blog/2014/jul/15/why-kallithea/>`__ that the
+Kallithea codebase is completely and validly licensed under GPLv3. In
+addition to their role in building the initial Kallithea community, the
+Conservancy is also the legal home of both the Mercurial and Git projects.
+Other SFC member projects that may be familiar to Python users include
+Twisted, Gevent, BuildBot and PyPy.
+
+
+Perceived Benefits
+==================
+
+The primary benefit of deploying Kallithea as forge.python.org is that
+supporting repositories such as the developer guide and the PEP repo could
+potentially be managed using pull requests and online editing. This would be
+much simpler than the current workflow which requires PEP editors and
+other core developers to act as intermediaries to apply updates suggested
+by other users.
+
+The richer administrative functionality would also make it substantially
+easier to grant users access to particular repositories for collaboration
+purposes, without having to grant them general access to the entire
+installation.
+
+
+Technical Challenges
+====================
+
+User account management
+-----------------------
+
+Ideally we'd be able to offer a single account that spans all python.org
+services, including Kallithea, Roundup/Rietveld, PyPI and the back end for
+the new python.org site, but actually implementing that would be a distinct
+infrastructure project, independent of this PEP.
+
+A potentially simpler near term solution would be if Kallithea's user
+account management could be integrated with the existing account management
+in Roundup, similar to what was done for Rietveld. However, if that also
+turns out to be impractical in the near term, we would merely end up with
+another identity silo to be integrated at a later date, suggesting that
+this doesn't need to be considered a blocker for an initial Kallithea
+deployment.
+
+
+Breaking existing SSH access and links for Mercurial repositories
+-----------------------------------------------------------------
+
+This PEP proposes leaving the existing hg.python.org installation alone,
+and setting up Kallithea on a new host. This approach minimises the risk
+of interfering with the development of CPython itself (and any other
+projects that don't migrate to the new software forge), but does make any
+migrations of existing repos more disruptive (since existing checkouts will
+break).
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+..
+   Local Variables:
+   mode: indented-text
+   indent-tabs-mode: nil
+   sentence-end-double-space: t
+   fill-column: 70
+   coding: utf-8
+   End:

-- 
Repository URL: http://hg.python.org/peps

More information about the Python-checkins mailing list