[Python-Dev] Move selected documentation repos to PSF BitBucket account?

Sun Nov 23 17:55:04 CET 2014

On Sun Nov 23 2014 at 6:18:46 AM Nick Coghlan <ncoghlan at gmail.com> wrote:

>
> On 23 Nov 2014 18:11, "Donald Stufft" <donald at stufft.io> wrote:
> > > On Nov 23, 2014, at 2:35 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > >
>
> > > In the absence of a proposal to change version control systems
> > > (again), the lack of Mercurial hosting on GitHub makes it rather a
> > > moot point. Given that we can barely muster up any enthusiasm for
> > > rehosting *without* changing version control systems (and the number
> > > of CI systems that integrate with hg.python.org repos other than the
> > > main CPython one is exactly zero), any proposal that involves doing
> > > even *more* work seems doubly doomed.
> > >
> >
> > I’d volunteer to do the work to get the PEPs, and possibly other
> repositories
> > onto Github if we so decided to do so. Don’t let the lack of volunteer
> stop
> > that because I will find the time to do it if need be.
>
> It's the other way around: someone would have to volunteer to make the
> case that switching version control systems will actually help in any way
> whatsoever with the core reviewer bandwidth problem.
>
> We do *not* have a shortage of people wanting to contribute. While ongoing
> outreach efforts are essential to promote increased diversity in the
> contributor base and to counter natural attrition, there is currently no
> major problem that needs solving on that front. CPython is high profile
> enough that folks are willing to do battle with the current complicated
> contribution process, so we're already one of the most active open source
> projects in the world, in *spite* of the problems with the existing
> workflow.
>

The *immediate* problem is making it easier to accept contributions from
people. The long-term, never-ending problem is making the whole process of
submitting a patch and getting it accepted as easy as possible for everyone
involved, contributor and committer alike. If the goal is to make it so we
can accept PRs for easier patch acceptances then that can be accomplished
on either Bitbucket or GitHub. But if we want to make it easier for people
to make contributions then GitHub is arguably better than Bitbucket,
whether it's through familiarity of GitHub for most people thanks to other
FLOSS projects or from the superior tooling around GitHub (both the
platform itself and the ecosystem that has sprung up around it).

>  This high level of activity also takes place in spite of the fact that
> direct corporate investment in paid contributions to the CPython runtime
> currently don't really reflect the key role that CPython holds in the
> enterprise Linux, OpenStack, data analysis, and education ecosystems (to
> name a few where I personally have some level of involvement either
> personally or through work).
>
> Where I believe we *do* have a problem is with failing to make the best
> possible use of core developer contribution time, as typos and other minor
> fixes to project (rather than product) documentation are managed through
> the same offline patch & upload process as the reference interpreter
> itself. (There are other issues as well, but they're out of scope for the
> current discussion, which is about the support repos, rather than CPython -
> the same problem exists there, but the solution is unlikely to be as
> straightforward).
>
> Patches getting held up in the review queue for weeks or months is a
> *huge* barrier to contribution, as it prevents the formation of the
> positive feedback cycle where having a contribution accepted feels good, so
> folks are more likely to want to contribute again.
>
> In that context, the useful features that a richer repo hosting
> application can potentially offer are single-click acceptance and merging
> of documentation changes that aren't directly linked to a specific CPython
> version, as well as the ability to make trivial fixes to that documentation
> (like fixing a typo) entirely online.
>
> Those features are readily accessible without changing the underlying
> version control system (whether self-hosted through Kallithea or externally
> hosted through BitBucket or RhodeCode). Thus the folks that want to change
> the version control system need to make the case that doing so will provide
> additional benefits that *can't* be obtained in a less disruptive way.
>

I guess my question is who and what is going to be disrupted if we go with
Guido's suggestion of switching to GitHub for code hosting? Contributors
won't be disrupted at all since most people are more familiar with GitHub
vs. Bitbucket (how many times have we all heard the fact someone has even
learned Mercurial just to contribute to Python?). Core developers might be
based on some learned workflow, but I'm willing to bet we all know git at
this point (and for those of us who still don't like it, myself included,
there are GUI apps to paper over it or hg-git for those that prefer a CLI).
Our infrastructure will need to be updated, but how much of it is that
hg-specific short of the command to checkout out the repo? Obviously
Bitbucket is much more minor by simply updating just a URL, but changing
`hg clone` to `git clone` isn't crazy either. Georg, Antoine, or Benjamin
can point out if I'm wrong on this, maybe Donald or someone in the
infrastructure committee.

Probably the biggest thing I can think of that would need updating is our
commit hooks. Once again Georg, Antoine, or Benjamin could say how
difficult it would be to update those hooks.

> From my perspective, swapping out Mercurial for git achieves exactly
> nothing in terms of alleviating the review bottleneck (since the core
> developers that strongly prefer the git UI will already be using an
> adapter), and is in fact likely to make it worse by putting the greatest
> burden in adapting to the change on the folks that are already under the
> greatest time pressure.
>

That's not entirely true. If you are pushing a PR shift in our patch
acceptance workflow then Bitbucket vs. GitHub isn't fundamentally any
different in terms of benefit, and I would honestly argue that GitHub's PR
experience is better. IOW either platform is of equal benefit.

> It's also worth keeping in mind that changing the underlying VCS means
> changing *all* the automation scripts, rather than just updating the
> configuration settings to reflect a new hosting URL.
>

What are the automation scripts there are that would require updating? I
would like to a list and to have the difficulty of moving them mentioned to
know what the impact would be.

> Orchestrating this kind of infrastructure enhancement for Red Hat *is* my
> day job, and you almost always want to go for the lowest impact, lowest
> risk approach that will alleviate the bottleneck you're worried about while
> changing the smallest possible number of elements in the overall workflow
> management system.
>

Sure, but I would never compare our infrastructure needs to Red Hat. =) You
also have to be conservative in order to minimize downtown and impact for
cost reasons. As an open source project we don't have those kinds of worry;
we just have to worry about keeping everyone happy.

> That underlying calculation doesn't really change much even when the units
> shift from budget dollars to volunteers' time and energy.
>
So here is what I want to know to focus this discussion:

First, what new workflow are you proposing regardless of repo hosting
provider? Are you proposing we maintain just mirrors and update the
devguide to tell people to fork on the hosting provider, make their
changes, generate a patch (which can be as simple as telling people how
find the raw diff on the hosting provider), and then upload the patch the
issue tracker just like today? Are you going farther and saying we have
people send PRs on the hosting site, have them point to their PR in the
issue tracker, and then we accept PRs (I'm going on the assumption we are
not dropping our issue tracker)?

Second, to properly gauge the impact of switching from git to hg from an
infrastructure perspective, what automation scripts do we have and how
difficult would it be to update them to use git instead of hg? This is
necessary simply to know where we would need to update URLs, let alone
change in DVCS.

Third, do our release managers care about hg vs. git strongly? They
probably use the DVCS the most directly and at a lower level by necessity
compared to anyone else.

Fourth, do any core developers feel strongly about not using GitHub? Now
please notice I said "GitHub" and not "git"; I think the proper way to
frame this whole discussion is we are deciding if we want to switch to
Bitbucket or GitHub who provide a low-level API for their version control
storage service through hg or git, respectively. I personally dislike git,
but I really like GitHub and I don't even notice git there since I use
GitHub's OS X app; as I said, I view this as choosing a platform and not
the underlying DVCS as I have happily chosen to access the GitHub hosting
service through an app that is not git (it's like accessing a web app
through it's web page or its REST API).

At least for me, until we get a clear understanding of what workflow
changes we are asking for both contributors and core developers and exactly
what work would be necessary to update our infrastructure for either
Bitbucket or GitHub we can't really have a reasonable discussion that isn't
going to be full of guessing.

And I'm still in support no matter what of breaking out the HOWTOs and the
tutorial into their own repos for easier updating (having to update the
Python porting HOWTO in three branches is a pain when it should be
consistent across Python releases).

-Brett

> Regards,
> Nick.
>
> >
> > ---
> > Donald Stufft
> > PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> >
>  _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141123/6a59f4f3/attachment-0001.html>