Brett Cannon wrote:

On 7/12/07, "Martin v. Löwis" wrote:

...

I'm tasked with performing a number of merge operations across various Python branches. Can somebody please share a command line that I should use to continue with the merge tracking that has been used? Is that documented somewhere?

You mean using svnmerge? If so, see the dev FAQ: http://www.python.org/dev/faq/#how-do-i-merge-branches . If you are after something else then I don't know. =)

I do know, though, that Thomas kept talking about moving us over to Bazaar (or some distributed VCS) and instead of having a ton of svn branches we have distributed VCS branch for each feature in Py3K. That way the VCS's strong merge system would work in our favour for pulling in from the various Py3K branches and for eventual mainline merging.

-Brett

Hi all, While I'm generally just a silent listener to this list, I thought I'd share my experiences with distributed SCM - primarily because I think it's a great step in the right direction. So far I've used DARCS, Hg, and Git. And at this point Git is far and away the winner. While I can't claim to have spent alot of time with DARCS, my experience was that it took a fair whack of unintuitive pain to work out how to extract a patch that I could send upstream to be submitted to a project. I believe it also has a reputation for being rather slow. I've also noticed that repositories sometimes become "broken" and need to be re-checked out - but that I'm willing to put down to some other factor I'm not aware of. With Hg I went in fast and hard, and nearly got burned before I could bail out in time :-) It's very friendly to use, but we run a number of OpenBSD hosts for our core architecture, and it turns out Hg wraps calls to patch, and parses the text output from the call (assuming the version of patch is GnuPatch). The problem here is that under OpenBSD the output assumptions get violated, as reflected by the failure of lots of tests - including repository sanity checks. That meant Hg just wasn't going to be an option for us. I also found that having a new directory tree of files for each branch was rather onerous. Having bailed on Hg I found git to be fast, cross platform, and user-friendly (provided you understand the basic concepts of distributed SCM, and you're using git 1.5+ ;-) ). It also has some really cool features like "rebasing" for letting your branch actively track the trunk from which you branched it. I can't speak to how easily any of these cross over to the windows platform, although none of them seem to be overly windows friendly (YMMV). But I presume this would be one of the key problems facing a distributed versioning system by the python community. Cheers Dave

...

It also has some really cool features like "rebasing" for letting your branch actively track the trunk from which you branched it.

Unfortunately, rebasing doesn't seem to be stable yet. Sometimes it works for me, sometimes not. I don't know whether its because I don't know what I'm doing, bugs in git, or changes in the calling syntax.

Not to diverge py-dev too far into the depths of DRCS usage, but are you doing anything particularly complex ? While I wouldn't say I do it regularly, I certainly consider it part and parcel of general git usage, and I've always found it works well for me. Rebasing is one of the features that really convinced me git was a cut above the rest, obviously along with the ease of merging. For those that are interested the ultra brief explanation for git branching is that it stores all your branches in the one directory structure. You "git checkout" to switch between branches, which causes all the managed files in your directory to be changed to reflect the state branch you are changing to.

Dave Harrison wrote: [...]

I can't speak to how easily any of these cross over to the windows platform, although none of them seem to be overly windows friendly (YMMV). But I presume this would be one of the key problems facing a distributed versioning system by the python community.

We can probably assume that none of the Linux kernel team are developing on Windows. There is probably s a group with relevant experience somewhere. I'd Google for it, but I expect that the results would be dominated by British assertions that "you have to be a stupid git to run Windows". regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden --------------- Asciimercial ------------------ Get on the web: Blog, lens and tag the Internet Many services currently offer free registration ----------- Thank You for Reading -------------

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 13, 2007, at 4:04 PM, Giovanni Bajo wrote:

I can't speak of bzr.

I don't use or even have Windows, but this page says there are native Windows binaries available (yes Bazaar is pure Python): http://bazaar-vcs.org/BzrOnPureWindows There's also a Tortoise-like extension available: http://bazaar-vcs.org/TortoiseBzr - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBRpfckHEjvBPtnXfVAQKArwP/aS20z5EcNT7GJSMepCORo34tE5SpF1g/ OMvRTervNDuUmjulgAHM9w9Chi3lOibM6eX0nJ8kHHzoaPrFDQkzh1/Gc+OZqfSA FAgctJtVkT956o+F1+xFe90KqtTtH8gmwHMFMhr/a7BZvV1S1kpCyu8ygI3YEe2u 5JParA9UcO0= =5HLR -----END PGP SIGNATURE-----

Thomas Wouters wrote:

On 7/12/07, *Dave Harrison* mailto:dave@cryptohorizon.com> wrote:

So far I've used DARCS, Hg, and Git. And at this point Git is far and away the winner.

Let's not start a discussion on which DVCS is better, or I'd burn your ears off about all the ways each of those (as well as Bazaar, Arch/tla/bzr1, Arx, CodeVille, SVK, Monotone and BitKeeper) suck, suck badly or are the work of the devil. The official Python source repository will be Subversion for now (although it isn't officially my decision :). I encourage anyone to use a mirror of it in their own favourite VCS, and do their own development in it. 'tailor' is a nice tool if you care about having the full history (as I do) and you don't happen to hit bugs in it or the VCS. Do realize that the full history may be a burden, especially in DVCS solutions.

For what it's worth, rumour has it Subversion 1.5 or 2.0 will get actual branch tracking and full-history merging. If done properly (it's not done yet, so it's hard to say) it would reduce the advantage of the DVCS solutions by about half ;-P

While I don't necessarily agree (and await for that burning feeling to start in my ears ;-) ), I think the lack of a decent windows port for some, and general lack of real maturity in most, are the killer arguments against DRCS for Python at this point. My own DRCS of preference (git) has a method for pulling from subversion anyway, and I can't really claim to even contribute much directly _to_ Python's core, so I'm wary of claiming to be too invested in this. That said, there are always strong arguments in favour of the distributed model encouraging and fostering community dev participation - and code always speaks louder than words heh. Dave

Martin v. Löwis wrote:

...

That said, there are always strong arguments in favour of the distributed model encouraging and fostering community dev participation

Just for the record: Python's development model *is* distributed, and has been for a long time. We don't all work in the same office, or even in the same time zone. We rarely meet face-to-face (although the recent hiring wave at Google has changed some of that - but then, there also was ZC, BeOpen, CNRI before that).

Without wanting to have a discussion about the semantics of what it is to be "distributed", what it seems (to me at least) that you're saying is that the python community itself is distributed. That's doesn't mean that your versioning is distributed. The model you describe is very much centralised, where many people submit patches to a central source repository - otherwise SVN would already be a distributed RCS. The model you describe isn't quite the distributed method of operation where you have many repos that can be pulled and merged together - I won't attempt to rehash Linus' Google Tech Talk, See here : http://www.youtube.com/watch?v=4XpnKHJAok8) of the distributed model he uses, if only because so many python-dev members work for Google now ;-)

I really don't think that the non-D aspect of the VCS is a burden to contribution. Many patches are contributed from the released source, i.e. without bothering to check out anything, or even from the binary installation, i.e. without bothering to download and build the source.

No certainly not, I'm not by any means suggesting that the current way of handling revision control is having a negative impact. I also don't intend to evangelize. But there are arguments to be put that a DRCS can promote more involvement by allowing people to checkout and work on their own copy of the tree. This way they can cut code in a controlled environment using commits, reverts, version diffs etc without the necessity of setting up their own svn repo to commit to, or needing a sandbox in the python repo.

People who don't use subversion to get themselves the latest sources to contribute against won't start doing so if some other system was used (they are less likely to if they have to learn it first).

Generally I agree with you, for bugfixing etc it's unlikely that if they didn't participate before, an RCS change won't suddenly change their mind. That said, I think the number of people that know SVN beyond update and commit are few and far between as well - I'd suggest the real distinctions between distributed and non-distributed apps at this level are virtually nil for such simple use cases.

Instead, requiring contributors to use the Python VCS would place a burden, as does the requirement that we already have to contribute diff files, rather than an edited version of the original file. Also, having to register with the bugtracker is a burden for some contributors.

The learning curve is definitely one of the key impediments to any change to the RCS used. The last thing you want to do is cause unreasonable difficulty for the people who are already contributing to the project; it's not much use to have encouraged 20 new developers only to lose 20 existing ones. But it's worth considering the balance between costs and benefits, for example being able to branch virtually instantly without modifying the python.org repository is a big win I'd think. Being able to more easily modularise the maintenance and responsibility load for the source tree is another win, especially as the size of the tree grows and new efforts such as Py3k begin. With respect to bugtracking, well that's probably a separate issue really. However if there's a way to integrate a bugtracker with my RCS in a way that lets me create and resolve tickets when I'm offline, someone please tell me :-) Cheers Dave

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 12, 2007, at 5:54 PM, Brett Cannon wrote:

I do know, though, that Thomas kept talking about moving us over to Bazaar (or some distributed VCS) and instead of having a ton of svn branches we have distributed VCS branch for each feature in Py3K. That way the VCS's strong merge system would work in our favour for pulling in from the various Py3K branches and for eventual mainline merging.

Mailman switch to Bazaar on June 22. The transition went pretty smoothly (the hardest part of the process was shutting off write access to Subversion ;). I know some of us were talking about it at Pycon, but I've now been using it for 4 or 5 more months. It's really a top-notch dvcs and I think it would bring a lot of benefit to the Python community. I'll see if I can set up some public bzr mirrors of our svn repository for people to try it out. Or you could just use the bzr- svn plugin to get a local repository. There are lots of key benefits to using a dvcs, but IMO the most important is that it helps democratize the development process, and makes it much easier for the core to review and accept contributions. diffs are so 20th century. :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBRpeZYnEjvBPtnXfVAQL0ugQArei77bNsmv1hdrfs6RGinNiBdhNvSet8 vprjc3JgdV0iBltSX2KvHj+xOEd49ImDtKSws1uQNDLzIGMzv65/P03d3udwd2fy VrzBrO9Nlw+YdWtfJEcwCmRGT7Zj4HNUhYxQUB/V/3dytjLZoCp3m5xYAbfj44be FThPPbnCeDI= =CuEJ -----END PGP SIGNATURE-----

Barry> Silly me, the trunk is already available: Barry> https://code.launchpad.net/~vcs-imports/python/trunk Bazaar keeps this in sync with svn.python.org? Skip

Should the community ever decide to make the switch

The last time, I was asked to write a PEP. I would demand the same thing the next time such a switch is suggested. I would be -1 on any proposed switch to a system that hasn't made its 1.0 release yet (myself, I've used subversion since 0.21, and I certainly don't want to go back to these times). Regards, Martin

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 13, 2007, at 11:39 AM, skip@pobox.com wrote:

Barry> diffs are so 20th century. :)

How do you compare two versions of something without some sort of diff?

Well okay, you caught me being flippant. :) Sure, you visually compare with diffs, but you apply those changes with merges. This means the end of posting patches because instead what you would do is post the url to a branch that you published some place. It means that branch can be kept up-to-date as its parent branch changes, so a new feature candidate need never get stale. It also means your new feature candidate is a first class revision control branch, just as usable as the trunk, say. So it's much more powerful than trading patch files around. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBRpekD3EjvBPtnXfVAQLBqwP/RbJ3VhArWjijJ99/u3CzAy0xbcvnUIEv htlZ/2PBXGZ+ZZm0uQiQcnwlczVUVAiyopdPgTAUmerh9aYWZTP8rtC3SIG7gBDz 6QdNYMl2Rvh5AsvLNXo3HzTyVx74cKHEy91csUuoUWomBa1dR9DVRsT+CpbwM4U8 q11cm7kjXa4= =v4w0 -----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 13, 2007, at 12:24 PM, Facundo Batista wrote:

2007/7/13, Barry Warsaw :

...

with merges. This means the end of posting patches because instead what you would do is post the url to a branch that you published some place. It means that branch can be kept up-to-date as its parent branch changes, so a new feature candidate need never get stale. It also means your new feature candidate is a first class revision control branch, just as usable as the trunk, say. So it's much more powerful than trading patch files around.

More powerful, maybe, but also more limitating.

Do you still have the "patch" metodologie? How can you provide a patch if you don't have a place to publish the change?

You can still have a patch methodology if your internet connection really sucks. But if you have good enough access to the current Subversion repository, then you probably have good enough access to use a free branch hosting service like Launchpad. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBRpe3aHEjvBPtnXfVAQL1swQAuXP/QX5c/isttwQ3Et/N7fp6mG9a+sA+ HX7G6rSWxaYGtmYOjXTqjeou/QEoNlEtOqvZQcAJQ4iObMVg0mVIVFxixPln3JPQ u4XNt37FklJh7q0tbvi6VQeBi82beqWuL6+MTE6dMD1ruRAkJ/zpM9/ruiCBSmSB XO84dNkv/dY= =sUf+ -----END PGP SIGNATURE-----

Sure, you visually compare with diffs, but you apply those changes with merges. This means the end of posting patches because instead what you would do is post the url to a branch that you published some place.

But how do you publish stuff? Do you need to run your own web server on your dial-up machine? I simply cannot imagine that all contributors have access to some machine that is always online and willing to host bzr patches (which is necessary IIUC). Regards, Martin

On 7/12/07, Brett Cannon wrote:

I do know, though, that Thomas kept talking about moving us over to Bazaar (or some distributed VCS) and instead of having a ton of svn branches we have distributed VCS branch for each feature in Py3K. That way the VCS's strong merge system would work in our favour for pulling in from the various Py3K branches and for eventual mainline merging.

No. I never talked about moving us over. I do not believe and have never believed that to be feasible (in the foreseeable future.) It is certainly possible to host a project of Python's size, depth of history and number of committers in a DVCS (Monotone is my personal favourite, with Mercurial and Bazaar a shared second), and certain things would go a _lot_ better with proper branch tracking and merging, but switching over Python means an entirely new way of working, educating all the developers and working around the limitations of the chosen system. (I'm not saying they have more limitations than Subversion per se, but we are already used to working around Subversion's limitations.) What I talked about was converting the monolithic p3yk branch to a collection of separate feature-branches, which would then be merged together into "the py3k branch", as well as the trunk (for backported changes.) The p3yk branch contains many changes that will never be backported, mixed in with change we want in Python 2.6. We currently do manual merging from the trunk into p3yk. When we start backporting features, those features in the trunk will end up conflicting with the p3yk changes. Worse, we will need to manually track bugfixes to those features in the p3yk branch in order to apply them to the trunk, and take extra care when merging in bugfixes from the trunk, too. Because Subversion don't keep track of which chunk was already merged, you get a lot of spurious conflicts. It gets extremely tedious to do those merges manually, and the error rate raises significantly. (I already have an uncomfortable feeling that I missed some of the merges in the p3yk branch, leaving bugs in p3yk that were fixed in the trunk. Fortunately, tests catch them fairly quickly -- but we still don't have 100% test coverage, not even for newly fixed bugs.) I have been working on converting the p3yk branch into separate Bazaar[*] branches, by first converting the whole branch (with 'tailor') and then applying each changeset to an appropriate feature branch, creating them as needed (using a little script based on tailor, because cherrypicking doesn't quite do what I want.) The last half year of changes, though, gets so incredibly intertwined that it's quite hard to pull them apart. Furthermore, I would still have to merge them back together to create the original py3k branch, and do merges with the trunk, too. I've been forced to decide it's not worth it, at this point. If we'd started with separate feature-branches earlier, it would have been. Now, it's easier and less error prone to just suck it up and deal with the numerous conflicts backported features will create. (Since I will be doing much backporting myself, I'm not just letting someone else deal with the mess :-) Some-branch-of-me'ly y'rs, [*] Bazaar was chosen for a number of unimportant reasons that are entirely moot at this point. It doesn't reflect my personal preference or constitute any form of official endorsement :-) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread!