On Mon, 4 Jan 2016 at 09:50 Eli Bendersky eliben@gmail.com wrote:

On Fri, Jan 1, 2016 at 11:24 AM, Brett Cannon brett@python.org wrote:

...

If you want to read the reasons I chose GitHub over GitLab, see https://mail.python.org/pipermail/core-workflow/2016-January/000345.html . If you want to discuss the decision or help with the transition, please subscribe to the core-workflow mailing list.

Happy 2016 everyone, and here is to hoping we will have an easier developer workflow by the end of this year!

That sounds like a great idea.

I suppose you'll want to use https://github.com/python/cpython, which I'm currently maintaining as a read-only mirror. Let me know when you want to take control of that repo - I think since it belongs to the "python" Github org already, it should be easy to do.

I suspect we will want to use it, Eli, so thanks for the offer! I will let you know if we end up choosing to use it.

I have to admit that I'm not a big expert on Mercurial --> Git converters and the way I maintain this mirror may not be the best approach, so I encourage you folks to find a VCS guru to look into this for the "real" transition.

That will be one of the early steps of the transition. :)

I've found that hggit works very well - I used it to migrate my work project to github and still use it to avoid having to deal with git. (My intent is to keep using it for Python as well.)

Is the plan to migrate the entire history or just master?

Top-posted from my Windows Phone

-----Original Message----- From: "R. David Murray" rdmurray@bitdance.com Sent: ‎1/‎5/‎2016 7:34 To: "python-committers" python-committers@python.org Subject: Re: [python-committers] We will be moving to GitHub (hopefully) in2016

On Mon, 04 Jan 2016 18:18:02 +0000, Brett Cannon brett@python.org wrote:

On Mon, 4 Jan 2016 at 09:50 Eli Bendersky eliben@gmail.com wrote:

...

I have to admit that I'm not a big expert on Mercurial --> Git converters and the way I maintain this mirror may not be the best approach, so I encourage you folks to find a VCS guru to look into this for the "real" transition.

That will be one of the early steps of the transition. :)

Maybe The PSF could fund Eric Raymond to do this? He's got beyond-the-basics tooling, and a fair bit of experience converting large projects. He might even be able to clean up the older (svn) history along the way, although that might be too much to hope for ;)

(Oh, and let me mention this while I'm thinking about it: we're going to have to do some extra work to make the hash links work in the bug tracker, since I don't think there's any a priori way to distinguish between hg hashes and git hashes.)

--David

python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers

On Jan 4, 2016, at 4:11 PM, Brett Cannon brett@python.org wrote:

On Mon, 4 Jan 2016 at 13:08 Steve Dower mailto:steve.dower@python.org> wrote: I've found that hggit works very well - I used it to migrate my work project to github and still use it to avoid having to deal with git. (My intent is to keep using it for Python as well.)

Is the plan to migrate the entire history or just master?

TBD. Email beginning to outline the dependency graph for the transition forthcoming once I finish a code review at work that some co-worker handed to me when he went off to Australia for an extended holiday. ;)

It’s pretty easy to migrate the entire history (at least what’s in Hg) including all branches and tags.

Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

I'm not sure that you'd see much savings. You'd only get deltas that were never merged to master excluded. Point taken though.

Sent from my iPhone

On Jan 4, 2016, at 4:18 PM, Brett Cannon brett@python.org wrote:

...

On Mon, 4 Jan 2016 at 13:14 Donald Stufft donald@stufft.io wrote:

...

On Jan 4, 2016, at 4:11 PM, Brett Cannon brett@python.org wrote:

On Mon, 4 Jan 2016 at 13:08 Steve Dower steve.dower@python.org wrote:

...

I've found that hggit works very well - I used it to migrate my work project to github and still use it to avoid having to deal with git. (My intent is to keep using it for Python as well.)

Is the plan to migrate the entire history or just master?

TBD. Email beginning to outline the dependency graph for the transition forthcoming once I finish a code review at work that some co-worker handed to me when he went off to Australia for an extended holiday. ;)

It’s pretty easy to migrate the entire history (at least what’s in Hg) including all branches and tags.

It's not about the difficulty as the size of the clone. E.g., if we make Python 2 a separate repo does it buy us a lot of space savings (we need to remember not everyone has broadband, so there is some potential balance to be had between history vs. clone size)?

On Jan 04, 2016, at 02:09 PM, Guido van Rossum wrote:

I currently often rely on a single Hg clone containing all branches.

I do hope that a single repo will contain all the branches, though I wouldn't mind too much if we split Python 2 and 3 into separate repos. git worktree is a nice tool if you want separate file system directories for the different branches.

Cheers, -Barry

On Mon, Jan 4, 2016 at 2:56 PM, Brett Cannon brett@python.org wrote:

I'm not sure, which is why I'm asking what difference it would make if we separated out Python 2 branches into their own clone from Python 3 branches.

Irrespective of the size difference, separating Python2 repo from Python3 repo might be a good idea. It will help in reviewing patches and concentrating on Python 3 more. As a side effect, since GitHub is used by many developers, if we get many pull requests (or patches) against python 2 only, we will know about this from community participation.

-- Senthil

On Jan 04, 2016, at 03:33 PM, R. David Murray wrote:

Maybe The PSF could fund Eric Raymond to do this?

reposurgeon is the tool he developed to port Emacs's bzr repository to git, and it successfully handled all the previous vcses Emacs was ever developed under (after, IIRC, much tweaking).

http://www.catb.org/esr/reposurgeon/

Looks like it at least claims to support hg.

I once looked at it and decided it wasn't something I wanted to touch ;) so paying Eric to do it might not be a bad idea.

Cheers, -Barry

+1

Alex

On Mon, Jan 4, 2016 at 4:07 PM, Donald Stufft donald@stufft.io wrote:

...

On Jan 4, 2016, at 3:51 PM, Barry Warsaw barry@python.org wrote:

I once looked at it and decided it wasn't something I wanted to touch ;) so paying Eric to do it might not be a bad idea.

I’d prefer it if we didn’t financially support ESR since he likes to spout off racist and misogynistic garbage. I’m sure we can figure out how to successfully migrate from Hg to git. I’ve already done it once on the demo repo, if that’s not good enough I’ll work on it some more if need be.

---

Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

---

python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers

-- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084

On Tue, 05 Jan 2016 01:26:58 +0000, "Gregory P. Smith" greg@krypto.org wrote:

On Mon, Jan 4, 2016 at 12:34 PM R. David Murray rdmurray@bitdance.com wrote:

...

to have to do some extra work to make the hash links work in the bug tracker, since I don't think there's any a priori way to distinguish between hg hashes and git hashes.

Just ignore the remote possibility of short 32-bit hash prefix collisions (possible, but infrequent): the way to resolve that is when a hash lookup fails, to look it up in a translation index of former hg hashes. practical. good enough.

Yes, collision is not the problem, it's that you can't distinguish them without a lookup table or something. Which means we have to do more than just change the URL in the existing linkifier, which was my point.

--David

The linkifier converts old svn revision numbers to links to e.g. https://hg.python.org/lookup/r12345 , and this figures out the equivalent hg changeset and redirects to the corresponding hg.p.o page. The linkifier should just create links to https://hg.python.org/lookup/csid and let the page figure out if the csid is from hg or git and redirect where more appropriate.

Best Regards, Ezio Melotti

On Tue, Jan 5, 2016 at 3:37 AM, Alex Gaynor alex.gaynor@gmail.com wrote:

Probably the easiest thing is to point the linkifier at our own webservice that just does:

if hash not in cache: try: requests.head("github.com/hash") except requests.error: try: request.head("hg.python.org/hash") except request.error: return 404 else: cache[hash] = hg.python.org else: cache[hash] = github return cache[hash]

I'll send you my consulting bill :-) Alex

On Mon, Jan 4, 2016 at 8:33 PM, R. David Murray rdmurray@bitdance.com wrote:

...

On Tue, 05 Jan 2016 01:26:58 +0000, "Gregory P. Smith" greg@krypto.org wrote:

...

On Mon, Jan 4, 2016 at 12:34 PM R. David Murray rdmurray@bitdance.com wrote:

...

to have to do some extra work to make the hash links work in the bug tracker, since I don't think there's any a priori way to distinguish between hg hashes and git hashes.

Just ignore the remote possibility of short 32-bit hash prefix collisions (possible, but infrequent): the way to resolve that is when a hash lookup fails, to look it up in a translation index of former hg hashes. practical. good enough.

Yes, collision is not the problem, it's that you can't distinguish them without a lookup table or something. Which means we have to do more than just change the URL in the existing linkifier, which was my point.

--David

---

python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers

-- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084

---

python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers

On Tue, Jan 5, 2016 at 7:38 AM, Nick Coghlan ncoghlan@gmail.com wrote:

On 5 January 2016 at 11:33, R. David Murray rdmurray@bitdance.com wrote:

...

On Tue, 05 Jan 2016 01:26:58 +0000, "Gregory P. Smith" greg@krypto.org wrote:

...

On Mon, Jan 4, 2016 at 12:34 PM R. David Murray rdmurray@bitdance.com wrote:

...

to have to do some extra work to make the hash links work in the bug tracker, since I don't think there's any a priori way to distinguish between hg hashes and git hashes.

Just ignore the remote possibility of short 32-bit hash prefix collisions (possible, but infrequent): the way to resolve that is when a hash lookup fails, to look it up in a translation index of former hg hashes. practical. good enough.

Yes, collision is not the problem, it's that you can't distinguish them without a lookup table or something. Which means we have to do more than just change the URL in the existing linkifier, which was my point.

In the specific case of Roundup, does the linkifier have the "date of comment" info available? If so, it could fairly readily assume that hashes prior to the cutover date are hg ones, and later ones are for git.

By looking at the code I don't see it readily available, but it might be possible to retrieve it somehow. However I think it's better to keep the linkifier simple and stupid and just delegate to https://hg.python.org/lookup/ .

Best Regards, Ezio Melotti

Cheers, Nick.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 01/04/2016 10:59 PM, Serhiy Storchaka wrote:

Is it possible to keep the same hashes in both Mercurial and Git? Or at least the same short hashes?

tl;dr: it's impossible.

The hash is a SHA1 of the revision's "manifest", which contains all the metadata about the revision. To preserve the Mercurial hashes would mean doing something vile like rewriting these hashes after populating the repo. If this worked at all, it would be terribly fragile; my guess is that many git commands (e.g. "git fsck") would notice and view it as damage.

The short hashes are simply the unique leading part of the hash, so they would only be preserved if we could preserve the full hashes.

//arry/

This is now a moot point, but with generaldelta (active by default in hg 3.7) the hg clone size is at around 400 MB.

Georg

On 01/04/2016 11:53 PM, Alex Gaynor wrote:

My git clone is 350MB (after a make clean), a fresh hg clone is 650MB.

Alex

On Mon, Jan 4, 2016 at 5:38 PM, Georg Brandl mailto:g.brandl@gmx.net> wrote:

On 01/01/2016 08:24 PM, Brett Cannon wrote:
> If you want to read the reasons I chose GitHub over GitLab,
> see https://mail.python.org/pipermail/core-workflow/2016-January/000345.html .
> If you want to discuss the decision or help with the transition, please
> subscribe to the core-workflow mailing list.
>
> Happy 2016 everyone, and here is to hoping we will have an easier developer
> workflow by the end of this year!

Thanks for pioneering all the work and discussion needed to come to this
decision.

I think that in combination with a homu-style bot we can get a very nice
improved workflow going.

Georg

On Jan 4, 2016, at 7:06 PM, Trent Nelson trent@snakebite.org wrote:

Hey Brett, all,

I’m playing a bit of catch-up with e-mail, but it occurred to me some of the work I did getting PyParallel switched over to github could be of benefit. First thing that comes to mind is this wiki page where I tried to capture the steps I used for the conversion and subsequent keeping-in-sync:

https://github.com/pyparallel/pyparallel/wiki/Source-Control

They’re very rough notes but may prove useful. Should *hopefully* be repeatable. One issue I noticed after the fact is that a couple of the renames that happened when we went 2.x->3.x (like ConfigParser.py -> configparser.py) didn’t get picked up by the hg->git conversion so I had to manually fix them with commits like this: https://github.com/pyparallel/pyparallel/commit/d3654f13048c8d6185a4c4b7953c...

(I’ll be able to produce a proper list of the exact ones I had to fix… I just wanted to get this e-mail out there for now.)

You probably did this on OS X or another case insensitive filesystem yea? I had the same problem the first time I did the CPython demo repository, until I did it on Linux instead.

(That repo is sync’d up to around 3.5-ish (i.e. as of a few months ago)… all the PyParallel stuff lived in separate *-px branches that originated from 3.3.x-ish. Obviously we wouldn’t use that repo directly… or at least not without git filtering out my PyParallel stuff.)

I also made some changes to things like the buildinfo glue to work with git instead of hg:

https://github.com/pyparallel/pyparallel/blob/branches/3.3-px/diffs/PCbuild/...

I also updated the installer/msi.py to work with git but Steve’s since overhauled all this stuff so I’m not sure how useful it will be:

https://github.com/pyparallel/pyparallel/commit/ca64e60fd323875e2ca4af497d15...

What timeframe are we looking at? There were some mentions of PSF funding in later e-mails which I think this sort of work (once off but still needs high precision) is well suited for. I’ll throw my hat into the ring if the PSF<->Continuum want to formally book out some time. (I’m happy to help either way, but doing it formally at least guarantees time availability.)

Regards,

   Trent.

On Fri, Jan 01, 2016 at 07:24:53PM +0000, Brett Cannon wrote:

...

If you want to read the reasons I chose GitHub over GitLab, see https://mail.python.org/pipermail/core-workflow/2016-January/000345.html . If you want to discuss the decision or help with the transition, please subscribe to the core-workflow mailing list.

Happy 2016 everyone, and here is to hoping we will have an easier developer workflow by the end of this year!

...

---

python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers

---

python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers

Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA