PEP: Migrating the Python CVS to Subversion

I'd like to see the Python source be stored in Subversion instead of CVS, and on python.org instead of sf.net. To facilitate discussion, I have drafted a PEP describing the rationale for doing so, and the technical procedure to be performed. This is for discussion on python-dev and eventual BDFL pronouncement; if you see a reason not to make such a change, or if you would prefer a different procedure, please speak up. Encouragement and support is welcome, as well :-) Regards, Martin PEP: XXX Title: Migrating the Python CVS to Subversion Version: $Revision $ Last-Modified: $Date$ Author: Martin v. Löwis <martin@v.loewis.de> Discussions-To: <python-dev@python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 14-Jul-2004 Post-History: 14-Jul-2004 Abstract ======== The Python source code is currently managed in a CVS repository on sourceforge.net. This PEP proposes to move it to a subversion repository on svn.python.org. Rationale ========= This change has two aspects: moving from CVS to subversion, and moving from SourceForge to python.org. For each, a rationale will be given. Moving to Subversion -------------------- CVS has a number of limitations that have been elimintation by Subversion. For the development of Python, the most notable improvements are: - ability to rename files and directories, and to remove directories, while keeping the history of these files. - support for change sets (sets of correlated changes to multiple files) through global revision numbers. - support for offline diffs, which is useful when creating patches. Moving to python.org -------------------- SourceForge has kindly provided an important infrastructure for the past years. Unfortunately, the attention that SF received has also caused repeated overload situations in the past, to which the SF operators could not always respond in a timely manner. In particular, for CVS, they had to reduce the load on the primary CVS server by introducing a second, read-only CVS server for anonymous access. This server is regularly synchronized, but behind the the read-write CVS repository between synchronizations. As a result, users without commit access can see recent changes to the repository only with a delay. On python.org, it would be possible to make the repository accessible for anonymous access. Migration Procedure =================== To move the Python CVS repository, the following steps need to be executed. The steps are elaborated in more detail in the next sections. 1. Assign passwords for all current committers for use on svn.python.org. User names on SF and svn.python.org should be identical, unless some committer requests a different user name. 2. At the beginning of the migration, announce that the repository on SourceForge closed. 3. 24 hours after the last commit, download the CVS repository. 4. Convert the CVS repository into two subversion repositories, one for distutils and one for Python. 5. Publish the repositories for write access for all committers, and anonymous read access. 6. Disable CVS access on SF. Assign Passwords ================ Currently, access to Subversion on svn.python.org uses WebDAV over https, using basic authentication. For this to work, authorized users need to provide a password. This mechanism should be used, atleast initially, for the Python CVS as well, since various committers also have a username/password pair for the www SVN repository already. Alternatives to password-based access include: - Subversion over SSH, using SSH key pairs. This would require to give committers accounts on the machine, which currently is ruled out by the administration policy of svn.python.org. - Subversion over WebDAV, using SSL client certificates. This would work, but would require to administrate a certificate authority. Downloading the CVS Repository ============================== The CVS repository can be downloaded from http://cvs.sourceforge.net/cvstarballs/python-cvsroot.tar.bz2 Since this tarball is generated only once a day, some time after the repository freeze must pass before the tarball can be picked up. It should be verified that the last commit, as recorded on the python-commits mailing list, is indeed included in the tarball. Converting the CVS Repository ============================= The Python CVS repository contains two modules: distutils and python. Keeping them together will produce quite long repository URLs, so it is more convenient if the Python CVS and the distutils CVS are converted into two separate repositories. As the repository format, fsfs should be used (requires Subversion 1.1). fsfs has the advantage of being more backup-friendly, as it allows to backup a repository incrementally, without requiring to run any dump commands. The conversion should be done using cvs2svn utility, available e.g. in the cvs2svn Debian package. The command for converting the Python repository is cvs2svn -q --encoding=latin1 --force-branch=cnri-16-start --force-branch=descr-branch --force-branch=release152p1-patches --force-tag=r16b1 --fs-type=fsfs -s py.svn.new python/python The command to convert the distutils repository is cvs2svn -q --encoding=latin1 --fs-type=fsfs -s dist.svn.new python/distutils Sample results of this conversion are available at http://www.dcl.hpi.uni-potsdam.de/python/ http://www.dcl.hpi.uni-potsdam.de/distutils/ Publish the Repositories ======================== The repositories should be published at https://svn.python.org/python and https://svn.python.org/distutils. Read-write should be granted through basic authentication to all current SF committers; read-only access should be granted anonymously. As an option, websvn (available e.g. from the Debian websvn package) could be provided. The current SF project admins should get write access to the password file, in order to create or delete new users. Disable CVS =========== It appears that CVS cannot be disabled entirely. Only the user interface can be removed from the project page; the repository itself remains available. If desired, write access to the python and distutils modules can be disabled through a commitinfo entry.

On 7/28/05, "Martin v. Löwis" <martin@v.loewis.de> wrote:
In principle I'm +1 on this (both aspects). I don't know enough to understand all the subtleties. I hope we're correctly estimating the effort required to manage the server and the users. Managing users is especially important -- if a user is compromised (as has happened in the past for python.org users) the whole repository is compromised. Now this could happen to SF users too, but I'm not sure that we know all the tricks in the book to prevent attacks; SF has been doing this for years and that's an aspect of SF that I trust (I think I've heard that they have even modified their SSH server to be stricter). -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Jul 28, 2005, at 4:20 PM, Guido van Rossum wrote:
If you use the fsfs storage mechanism for subversion, it is somewhat simpler to verify that the repository is not compromised. Each commit is represented as a separate file, and thus old commits are never modified. Only new files are appended to the directory. If you have a filesystem that allows "append-only" permissions on a directory, you can enforce this directly. Additionally, it is possible in your backup script to verify that only new files were added and nothing else changed. Then at least you know how much you need to examine instead of having to treat the entire repository as possibly contaminated. James

On Thu, 2005-07-28 at 16:20, Guido van Rossum wrote:
I hope we're correctly estimating the effort required to manage the server and the users.
Yah, me too! ;) We are building some experience with this though, having moved many of the system files, and all of the web pages, to svn. So far, the management overhead has been almost nil (um, None :). We'll have a bit of ongoing work to add users, but the infrastructure team is also building up some community knowledge about how to do that.
James has a very interesting idea for mitigating this. Presumably <heh>, we'll have backups of everything. I'll feel better when we have coverage from maybe 6 admins spanning as many timezones as possible. -Barry

I theory Subversion should allow you to be more secure. CVS has a very limited concept of security and for the most part you need to rely on SSH. Subversion makes use of Apache as one of its server options. Any authentication method you can use in Apache 2 you can use for Subversion. Once Apache does the authentication, Subversion allows fairly fine grained access control. SSL can be used for encrypting communication. Note that there are sites like Sourceforge that do provide Subversion hosting to open source projects. I don't know enough about them to be able to recommend any. -Chris On Thu, Jul 28, 2005 at 01:20:14PM -0700, Guido van Rossum wrote:

Guido van Rossum wrote:
There are three issues that I see: - management of access control. This is actually something that we do on SF as well; if the pydotorg admins get overloaded, I'm sure the current projects/python admins would be willing to help there. - assignment of passwords. This I don't like about the current pydotorg setup - there should be a way to chose your own password; perhaps without involving an administrator. I could imagine a web form for password change, and administrator interaction in case of a lost password. - compromised passwords. The only tricky question then is: was the repository altered? Fortunately, for Subversion, there should be an easy way to tell: in fsfs, files never change (only new files are added). So we could generate md5sums of all files in the repository, and download these to an offsite place. If the md5sum of an immutable file changes, we were compromised (there are, of course, a few files that do change regularly). Of course, we also need regular backups of the entire data so we can restore them if they got compromised. Regards, Martin

On Fri, 2005-07-29 at 00:44, "Martin v. Löwis" wrote:
I disagree. By reserving password generation to the pydotorg admins, we can better insure the passwords are more robust against dictionary attacks. See my previous message. I actually /don't/ want individuals to be able to set their own passwords. In practice, you only have to know your password once, because svn caches the authentication (yes, that opens up opportunities for compromise, but that's how svn works).
+1 to all that. -Barry

Barry Warsaw wrote:
See Michael's (I think) message: that is a much greater risk than the one of a brute-force attack. In our environment, a determined student could easily read out my home directory, and get at my pydotorg password (if I would allow svn to cache it). They would have to break all kinds of rules in doing so; yet, it would be technically possible - so I just can't turn on this svn setting, and have to type the password every time. This is surely inconvenient, as I cannot even remember the password. Regards, Martin

On Thu, Jul 28, 2005 at 10:00:00PM +0200, "Martin v. L?wis" wrote:
I'd like to see the Python source be stored in Subversion instead of CVS, and on python.org instead of sf.net.
+1, +1.
-- transactional operation - a changeset is either committed or rolled back at once; -- very effective (both in terms of speed and memory) tagging and branching; tags and branches are very easy to understand and use in SVN. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

On Thu, 2005-07-28 at 16:00, "Martin v. Löwis" wrote:
I'd like to see the Python source be stored in Subversion instead of CVS
+1
, and on python.org instead of sf.net.
+0 I know that SF has promised svn access to projects for a long time now, but I haven't heard anything from them in a long time. It's listed under their "Strategic Projects" but the last update to that news item was back in April. Question: do we wait for SF to make the service available (after getting more up-to-date status and a realistic timetable), or do we go straight to svn.python.org?
Thanks for writing this PEP Martin!
We've been going with firstname.lastname (with some exceptions -- hi Aahz! :) for the current svn access. Is it better to stay with that convention or to maintain consistency with SF's CVS committer names? Maybe the latter for revision history consistency.
2. At the beginning of the migration, announce that the repository on SourceForge closed.
We can probably play games with the various CVS hooks to disable all checkins during this time. We might also want to disable the u/i access to CVS at the same time.
4. Convert the CVS repository into two subversion repositories, one for distutils and one for Python.
Do we also want to split off nondist and encodings? IWBNI the Python source code proper weren't buried too deep in the directory structure. Note that we might want to provide different access permission to different parts of the repository (but I think we can do that even if we don't split those off into separate repos).
Definitely +1 on fsfs. Again, thanks Martin! -Barry

On 7/28/05, Barry Warsaw <barry@python.org> wrote:
+1 from me as well; single commit numbers for commits across multiple files will be wonderful.
I say forget SF and we move it. Of course I won't be involved with the migration so me saying this doesn't mean too much. =)
I say go with the first.last naming. While this might put committer names out of sync, we could keep a mapping of SF names to the new names in developers.txt for easy referencing. But it would be handy to have actual name references since I know I don't always remember who is whom on SF since some people go with nicks that are not based on their name at all. [SNIP]
Seems like a reasonable thing. Would make it easier for occasional committers as well as people who check out the code just for generating a patch. -Brett

Barry Warsaw wrote:
My proposal is to go straight to svn.python.org, for two reasons: - It might well be 2006 before they update the status and provide a realistic timetable. It's not that I lost faith into SF, but they seem to be continually overworked (as anybody), so anything that is not *really* essential to the operation of the service (such as support for subversion) has to wait "until we have time" -- which means it has to wait forever. Remember the plan to provide shell access to the project admins on the CVS server? - They currently have a separate anonymous access for CVS which is behind the real CVS, for load sharing reasons. They also had severe performance issues in the past. It might be that we also get performance problems, but we only need to support a single project (or two if you count pydotorg) on Subversion, not thousands of them. So our load evolution is much more predictable.
It's also a convenience issue, and has social aspects. For example, firstname.lastname does not work quite well for me, either (martin.v.loewis or martin.von.loewis would work better; likewise guido.van.rossum?), other users prefer not to use their "true" real name (e.g. tim_one vs. tim.peters). Another issue is password assignment - currently, users cannot choose their passwords on svn.python.org, right?
That should be tested in advance, of course.
Well, encodings is empty right now, so that might be a good idea. Technically, I would just move the files in the CVS repository before conversion. As for nondist: how precisely would you do that? Make it a separate repository? If so, where? I could envision a "/projects" repository, that has the current nondist.
I don't know how cvs2svn supports it, but one option would be to revert the trunk/ structure in /projects, so you get http://www.python.org/projects/peps/trunk http://www.python.org/projects/peps/tags http://www.python.org/projects/peps/branches http://www.python.org/projects/sandbox/trunk http://www.python.org/projects/sandbox/tags http://www.python.org/projects/sandbox/branches Then you could give different access to peps and to sandbox. Perhaps there isn't even a need for tags/branches on PEPs? Or we put everything in nondist into /projects, and then put tags + branches as siblings (so only people with write access to tags could create them)? Regards, Martin

On Fri, 2005-07-29 at 00:35, "Martin v. Löwis" wrote:
Yep, we would use guido.van.rossum. I think it would be fine for you to use martin.v.loewis or martin.von.loewis, just as MAL could use m.a.lemburg or marc.andre.lemberg. Our general concern was people hiding behind obscure nicknames like 'pumpichank' and no one really knowing who's behind that <wink>.
Another issue is password assignment - currently, users cannot choose their passwords on svn.python.org, right?
Correct, which I think is a feature. :)
Then you could give different access to peps and to sandbox. Perhaps there isn't even a need for tags/branches on PEPs?
Probably so. -Barry

"Martin v. Löwis" wrote:
If I understand things correctly, one project/one repo creates a 'hard' barrier for moving code across projects (while retaining history, so done via an svn command). Is the 'long url' really the only argument for this, and is it significant enough? Instead of: https://svn.python.org/python https://svn.python.org/distutils you could have https://svn.python.org/main/python https://svn.python.org/main/distutils or something similar. It's an extra few chars, and it would give a convenient way to branch off pieces of the main code into their own subprojects in the future if needed. For more experimental things, you can always have other repos: https://svn.python.org/someotherrepo/... But maybe the issue of moving code isn't too important, I'm certainly no expert on svn. Cheers, f

On Thursday 28 July 2005 20:07, Fernando Perez wrote:
More interestingly, keeping it in a single repository makes it easier to merge projects, or parts of projects, together, without losing the history. This would be useful when developing packages that may be considered for the standard library, but which also need to continue separate releases to support older versions of Python. We've found it very handy to keep multiple projects in a single repository for zope.org. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

On 7/29/05, Fred L. Drake, Jr. <fdrake@acm.org> wrote:
I thought I would chime in on this with some more experience. We use SVN (migated from CVS about 2 years ago) for everything, and we keep it all in one repository (even though there's several "products") for exactly this reason. The major "downside" that I, and some others, find is changeset explosion. When you start getting into 5 digit changeset numbers it can become a bit unwieldy to remember to type them all, but otherwise it works well. We also moved from BerkeleyDB-based to fsfs recently, and it seems to have fixed some intermittent problems we had had in the past. Another thing to look at would be Trac, which we are in the process of moving to from the horrendous nightmare of Bugzilla. It's integration with SVN as well as Wiki is quite amazing. Chris -- | Christopher Petrilli | petrilli@gmail.com

On Fri, 2005-07-29 at 00:50, Christopher Petrilli wrote:
Now's the time I pipe in to remind everyone that Atlassian has offered free (as in beer) versions of Jira and Confluence for the Python project (actually any open source project). If you haven't used these, they're definitely worth a look. Jira is the issue tracker, Confluence the wiki. They're extremely professional, usable, well-integrated, and you can integrate them with Subversion. We've used them at work for maybe a year now and I've been very happy with them. Jira is definitely one of the better issue trackers, free or not free, that I've used. www.atlassian.com They're not Python and they're not open source, so perhaps it's legitimate to dismiss them because of that. But they are also definitely cool. At the Atlassian folks are very cool too, and fans of FOSS. -Barry

Barry Warsaw wrote:
I've also used Confluence at work, and been very impressed. A very nice feature which I haven't seen anywhere else is that the Wiki pages allow comments as well as direct editing - it allows discussion *about* the page to occur in a normal answer/response fashion, possibly leading to eventual update of the page text itself.
I'd hope we wouldn't dismiss them out of hand - one of the things that appeals to me about Python is the philosophy that open-source and protected-source groups can both benefit from working together. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

"Martin v. Löwis" wrote:
For ipython, which recently went through cvs2svn, I found that moving over to a project/trunk structure was a few minutes worth of work. Since svn has moving commands, it was just a matter of making the extra project/ directory and moving things one level down the hierarchy. Even if cvs2svn doesn't quite create things the way you want them in the long run, svn is flexible enough that a few manual tweaks should be quite easy to perform. Cheers, f

Martin v. Löwis wrote:
So do you use project/trunk or trunk/project? If the former, I would need to get instructions on how to do the conversion from CVS.
project/trunk/ On Friday 29 July 2005 02:12, Fernando Perez wrote:
Yes, this will work. I vaguely recall that Jim converted the zope.org repository one project at a time, so that made it easier to keep them separate. We didn't decommission the CVS entirely, though; Zope 2.7 is still maintained in CVS since we didn't want to risk worrying about the branch structure too much; cvs2svn still had a few issues with the complex branch structure combined with the use of symlinks within the repository (one of the reasons we were so keen to move to Subversion). I'm sure Jim can provide more details of what he had to do; I was only slightly involved in the discussions. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

"Martin v. Löwis" wrote:
To be honest, I don't really know the details, but it seems to work fine. A quick look at ipython: planck[IPython]> svn update At revision 661. planck[IPython]> svn diff -r 10 genutils.py | tail - - Deprecated: this function has been superceded by timing() which has better - fucntionality.""" - - rng = range(reps) - start = time.clock() - for dummy in rng: func(*args,**kw) - return time.clock()-start - -#*************************** end of file <genutils.py> ********************** Revision 10 was most certainly back in the early CVS days, and the wholesale renaming happened when I started using svn, which was around revision 600 or so. There may be other subtleties I'm missing, but so far I haven't experienced any problems. Cheers, f

Martin v. Löwis wrote:
I'm definitely positive to a migration to Subversion, but I'd be really concerned about using plain text authentication mechanisms. SSH obviously is much prefered, and clearly there are ways to limit the "accounts" on the svn.python.org, many other projects does this already for cvs. E.g thor (17:11) 350/0 $ ssh -l 'username' cvs.mozilla.org To use anonymous CVS install the latest version of CVS on your local machine. Then set your CVSROOT environment variable to the following value: cvsuser@megalon:/cvsroot Connection to cvs.mozilla.org closed. We should have enough man powers to come up with some secure solution here :). -- Leif

On Thu, 2005-07-28 at 20:15, Leif Hedstrom wrote:
I'm definitely positive to a migration to Subversion, but I'd be really concerned about using plain text authentication mechanisms.
We won't use plain text, but we may (or, we currently do) use basic auth over ssl. The security then is in the passwords, so we have to make sure they're generated securely.
We should have enough man powers to come up with some secure solution here :).
Volunteers (i.e. those with actual free time to give on implementing any solution) are definitely welcome. -Barry

On Fri, 2005-07-29 at 01:00, "Martin v. Löwis" wrote:
Actually, the passwords are still hashed in the file, so they wouldn't be able to extract the plain text password. They definitely are vulnerable to brute force attack, though probably not to a dictionary attack. In practice I've been using a password generated based on os.urandom() -- we generate the password and get it to the Subversion user via a "secure route" <heh>. I'd be happy to share my password generation script with anybody who wants to audit it. Public/private keys would be better, and if anybody knows how to set up a Subversion server to use these without having to create accounts for everyone, I think we (the pythong.org admins) would love your help. -Barry

Barry Warsaw wrote:
I'll play around with it some this weekend, see what's possible, and was isn't. I'm thinking we ought to be able to use SSH's configuration to only allow one specific command, e.g. something like this in the authorized_keys: no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command="/usr/bin/svnserve" -- Leif

At 06:39 PM 7/29/2005 -0400, Barry Warsaw wrote:
But that would still require us to create accounts for every committer, right?
No. Just one account. You can have more than one key listed in authorized_keys, using svnserve --tunnel-user and sshd will select the right command based on the public key the client authenticates with.

Barry Warsaw wrote:
Nah. Somebody who takes over svn.python.org can replace Apache, and that will receive plain text passwords over the wire (in case you wonder: modules/aaa/mod_auth.c:authenticate_real_user - you can even write an Apache module that gets hold of the sent password). An intruder would have to wait some time before the password come in, instead of being able to read them all from a file at once - that's true.
Ok. Since this falls in my research interest, I definitely want to give it a try. I think I would set up PyCA to let users generate their private keys in the browser. Regards, Martin

At 05:54 PM 7/29/2005 -0400, Barry Warsaw wrote:
From the svnserve man page: -t, --tunnel Causes svnserve to run in tunnel mode, which is just like the inetd mode of operation (serve one connection over stdin/stdout) except that the connection is considered to be pre-authenticated with the username of the current uid. This flag is selected by the client when running over a tunnel agent. --tunnel-user=username When combined with --tunnel, overrides the pre-authenticated username with the supplied username. This is useful in combina- tion with the ssh authorized_key file's "command" directive to allow a single system account to be used by multiple committers, each having a distinct ssh identity. So, it looks like you'd just need to set up public keys for each user, and list them in authorized_keys. Presumably doing something like this: command="/usr/bin/svnserve --root=/svnroot -t --tunnel-user=username",no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa [key info here] would therefore do the trick. I've used a similar arrangement for my own CVS repository, but haven't tried it for SVN yet.

[Martin v. Löwis]
Encouragement and support on SVN, undecided on moving to python.org (don't know when SF intends to support SVN, don't have a feel for the state of-- or propsects for ongoing --python.org volunteerism).
I'm sending this to Jim Fulton because he did the conversion of Zope Corp's code base to SVN. Unfortunately, Jim will soon be out of touch for several weeks. Jim, do you have time to summarize the high bits of the problems you hit? IIRC, you didn't find any conversion script at the time capable of doing the whole job as-is. If that's wrong, it would be good to know that too. Other than that, I'd just like to see an explicit mention in the PEP of a plan to preserve the pre-conversion CVS tarball forever.

On Thursday 28 July 2005 07:21 pm, Tim Peters wrote:
<snip>
The conversion script isn't perfect and does fail on complex CVS arrangements or where there is extensive history to migate. However it appears above that Martin has already tried the script out, with success. BTW, re SSH access on python.org, using Apache's SSL support re https would provide as good of security without the risk of giving out shell accounts. SSL would encrypt the link and require a password or permit cert auth instead, same as SSH. Cert admin needn't be hard if only a single server cert is used, with client passwords, instead of client certs. -Jeff

[Jeff Rush]
I'd still like to hear from Jim, as I don't believe all serious problems were identified by eyeball at once. That said, the Python project has made relatively very little use of complex (for CVS) concepts like branching, but in Zopeland it wasn't unusual, over long stretches, for someone to make a new branch every day. Ah, before I forget, "single repository" has worked very well for Zope (which includes top-level Zope2, Zope3, ZODB, ZConfig, zdaemon, ... projects): http://svn.zope.org/ Long URLs don't really get in the way in practice (rarely a need to type one after initial checkout; even "svn switch" is usually just a tail-end cmdline edit starting from a copy+paste of "svn info" output).

On Thu, 2005-07-28 at 22:14, Tim Peters wrote:
It depends. In my use of svn, I do a lot of cross-branch merging and repo-side tagging. Those are done with urls and in those cases, long urls can suck. But we may not do a ton of that with the Python project, and besides it might not be important enough to split the directories. -Barry

[Tim]
[Barry]
It depends. In my use of svn, I do a lot of cross-branch merging and repo-side tagging.
Yup, me too -- between the two of us, we don't have enough fingers to count how many trunks, branches, and tags of ZODB and Zope I have to fiddle with.
Those are done with urls and in those cases, long urls can suck.
They're all still copy, paste, tail-edit for me, and-- indeed! --having them all in the same repository is what makes just-the-tail editing possible. Merges I do from the cmdline, but repo-side tagging I do with the TortoiseSVN GUI, and the latter gives easy-to-copy/paste/edit URL fields. So switch to Windows for that part ;-)
But we may not do a ton of that with the Python project, and besides it might not be important enough to split the directories.
Ya, in Python we make a branch about once per release, + once per 5 years when Jeremy underestimates how long it will take to finish a crusade <wink>.

"BAW" == Barry Warsaw <barry@python.org> writes:
BAW> So are you saying that moving to svn will let us do more long BAW> lived branches? Yay! Yes, but you still have to be disciplined about it. svn is not much better than cvs about detecting and ignoring spurious conflicts due to code that gets merged from branch A to branch B, then back to branch A. Unrestricted cherry-picking is still out. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

On Sun, 2005-07-31 at 23:54, Stephen J. Turnbull wrote:
Yeah. IMHO the sadest thing about SVN is it doesn't do branch/merge properly. All the other cool stuff like renames etc is kinda undone by that. For a definition of properly, see; http://prcs.sourceforge.net/merge.html This is why I don't bother migrating any existing CVS projects to SVN; the benefits don't yet outweigh the pain of migrating. For new projects sure, SVN is a better choice than CVS. -- Donovan Baarda <abo@minkirri.apana.org.au>

"Donovan" == Donovan Baarda <abo@minkirri.apana.org.au> writes:
Donovan> Yeah. IMHO the sadest thing about SVN is it doesn't do Donovan> branch/merge properly. All the other cool stuff like Donovan> renames etc is kinda undone by that. [...] This is why Donovan> I don't bother migrating any existing CVS projects to Donovan> SVN; the benefits don't yet outweigh the pain of Donovan> migrating. FWIW, XEmacs just had this discussion, and we basically came to the conclusion that for a multi-developer project it's _definitely_ worth the effort if it can be done by cvs2svn (which for us it probably can't, due to some black magic we did on the CVS repository a few years ago :-( ). For the record, I was opposed for exactly the reason you give, but changed my mind. The point is that with several developers there's almost surely someone enthusiastic enough about svn to bear the burden of fooling with the script for a couple of hours to see if it works, a fascist policy about migrating account names makes that almost trivial, and after that it's all gravy: the administration does not look any worse, the security issues are similar, and the change is likely to incite only a few people to press for account name changes after the move. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

George V. Neville-Neil wrote:
No. The PEP is only about Subversion. Why should we be looking at Per Force? Only because Python is Open Source? I think anything but Subversion is ruled out because: - there is no offer to host that anywhere (for subversion, there is already svn.python.org) - there is no support for converting a CVS repository (for subversion, there is cvs2svn) Regards, Martin

[Martin von Löwis]
The PEP is only about Subversion. I think anything but Subversion is ruled out because:
- there is no offer to host that anywhere (for subversion, there is already svn.python.org)
- there is no support for converting a CVS repository (for subversion, there is cvs2svn)
I quickly discussed Subversion with a few friends. While some say Subversion is the most reasonable avenue nowadays, others them told me they found something more appealing than Subversion: http://www.venge.net/monotone/ The hosting paradigm is fairly different, and for a few weeks now, they have a CVS repository converter. In my very naive eyes, the centralised aspects of Python development are be better represented with Subversion. It is notable also that Subversion if more Python-friendly than Monotone, with its Lua-based scripting. I did not deepen why, but at first glance, Monotone does not seduce me. On the other hand, the two guys saying good about Monotone are well informed (and also well known), so I would not dismiss their opinion so lightly. So, it might be worth at least a quick look? :-) -- François Pinard http://pinard.progiciels-bpi.ca

[Raymond Hettinger]
The current release is 0.21 which suggests that it is not ready for primetime.
It suggests it, yes, and to me as well. On the other hand, there is a common prejudice that something requires many releases, or frequent releases, to be qualified as good. While it might be true on average, this is not necessarily true: some packages need not so many steps for becoming very usable, mature or stable. (Note that I'm not asserting anything about Monotone, here.) We should merely keep an open mind. -- François Pinard http://pinard.progiciels-bpi.ca

On Tue, 2005-08-02 at 09:06, François Pinard wrote:
It is true that some well designed/developed software becomes reliable very quicky. However, it still takes heavy use over time to prove that. You don't want to be the guy who finds out that this is not one of those bits of software. IMHO you need maturity for revision control software... you are relying on it for history. The only open source options worth considering for Python are CVS and SVN, and even SVN is questionable (see bdb backend issues). -- Donovan Baarda <abo@minkirri.apana.org.au>

[Donovan Baarda]
It is true that some well designed/developed software becomes reliable very quicky. However, it still takes heavy use over time to prove that.
There is wisdom in your say! :-) -- François Pinard http://pinard.progiciels-bpi.ca

On 8/2/05, "Martin v. Löwis" <martin@v.loewis.de> wrote:
Perforce is a commercial product, but it can be had for free for verified Open Source projects, which Python shouldn't have any problem with. There are other problems, like you have to renew the agreement every year, but it might be worth considering, given the fact that it's an excellent system.
We could host a Perforce repository just as easily, I would think.
- there is no support for converting a CVS repository (for subversion, there is cvs2svn)
I'd put $20 on the fact that cvs2svn will *not* work out of the box for converting the python repository. Just call it a hunch. In any case, the Perforce-supplied cvs2p4 should work at least as well. -- Nick

On Wed, Aug 03, 2005, Nicholas Bastin wrote:
Maybe. OTOH, I went to a CVS->SVN talk today at OSCON, and I'd be suspicious of claims that Python's repository is more difficult to convert than others that have successfully made the switch (such as KDE). I'd rather not rely on licensing of a closed-source system; one of the points made during the talk was that the Linux project had to scramble when they lost their Bitkeeper license (but they didn't switch to SVN because they wanted a distributed model -- one of things I appreciated about this talk was the lack of One True Way-ism). -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything.

"aahz" == aahz <aahz@pythoncraft.com> writes:
aahz> I'd rather not rely on licensing of a closed-source system; aahz> one of the points made during the talk was that the Linux aahz> project had to scramble when they lost their Bitkeeper aahz> license Python is unlikely to throw away its license in the same way, I should think. For additional security, you could try to negotiate a perpetual license on a particular version, or a license that required substantial notice (say, six months) for termination. I would imagine you could get them; the only reason for the vendor not to give them would be spite. The problem with both of those options is the one that Martin already pointed out: negotiation takes effort. There are several good open source alternatives, one of which (svn) is well-established and gets excellent reviews for those goals it sets itself, which happen to be solving the problems (as opposed to missing features) of CVS. Why spend effort on negotiating licenses and preparing for potential vendor relationship problems, unless there's acknowledged need for features svn doesn't provide? -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

Nicholas Bastin wrote:
So we should consider it because it is an excellent system... I don't know what that means, in precise, day-to-day usage terms (i.e. what precisely would it do for us that, say, Subversion can't do).
Interesting offer. I'll add this to the PEP - who is "we" in this context?
You could have read the PEP before losing that money :-) It did work out of the box. Regards, Martin

On 8/4/05, "Martin v. Löwis" <martin@v.loewis.de> wrote:
It's a mature product. I would hope that that would count for something. I've had enough corrupted subversion repositories that I'm not crazy about the thought of using it in a production system. I know I'm not the only person with this experience. Sure, you can keep backups, and not really lose any work, but we're moving over because we have uptime and availability problems, so lets not just create them again.
Uh, the Python community. Which is currently hosting a subversion repository, so it doesn't seem like a stretch to imagine that p4.python.org could exist just as easily.
Pardon me if I don't feel that I'd like to see a system in production for a few weeks before we declare victory. The problems with this kind of conversion can be very subtle, and very painful. I'm not saying we shouldn't do this, I'm just saying that we should take an appropriate measure of how much greener the grass really is on the other side, and how much work we're willing to put in to make it that way. -- Nick

On Sun, 2005-08-07 at 21:52, Nicholas Bastin wrote:
Has anyone experienced svn corruptions with the fsfs backend? I haven't, across quite a few repositories.
Unfortunately, I don't think "we" (meaning specifically the collective python.org admins) have much if any operational experience with Perforce. We do with Subversion though and that's a big plus. If "we" were to host a Perforce repository, we'd need significant commitments from several somebodies to get things set up, keep it running, and help socialize long-term institutional knowledge amongst the other admins. We'd also have to teach the current crop of developers how to use the client tools effectively. I think it's fairly simple to teach a CVS user how to use Subversion, but have no idea if translating CVS experience to Perforce is as straightforward. -Barry

On Sun, Aug 07, 2005 at 11:51:49PM -0400, Barry Warsaw wrote:
Has anyone experienced svn corruptions with the fsfs backend? I haven't, across quite a few repositories.
I haven't. But I must admit that the repositories I'm working with aren't big. The bigest is at svn.colorstudy.com (I am working on SQLObject) and since the time Ian has switched from dbfs to fsfs I don't remember any problems with the repo. Speaking of merge. SVN relived much pain that CVS had gave me. With CVS I had a lot of conflicts - if the code to be merged is already there (had been merged from another branch) one got conflict. If the code contains CVS keywords (__version__ = "$Id$") cvs merge always produced conflicts. SVN fixed both problems so now I see only real conflicts. SVN just ignores the code to be merged if it has ben already merged; and SVN convert keywords internally to its default form ($Id$ instead of $Id: python.c 42 phd $) before merging. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

Barry Warsaw <barry@python.org> writes:
Also (from someone who is on the fringes of the pydotorg admin set): I don't know that much about subversion administration. But, if it proves necessary, as it's an open source project and all, I'm willing to put some time into learning about it. I'm *much* less likely to do this for a closed source package unless someone is paying me to do it. Maybe I'm the only person who thinks this way, but if not, it's something to think about. Cheers, mwh -- 42. You can measure a programmer's perspective by noting his attitude on the continuing vitality of FORTRAN. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

On Sun, Aug 07, 2005, Barry Warsaw wrote:
The impression I got from Alex Martelli is that it's not particularly straightforward. (Google apparently uses Perforce.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything.

[Aahz wrote]
Agreed.
(Google apparently uses Perforce.)
We do at ActiveState as well. *The* Perl source code repository is a Perforce one (hosted separately here at ActiveState as well). Microsoft licenses the Perforce code and uses it (with some slight modifications I hear) internally. Trent -- Trent Mick TrentM@ActiveState.com

Nicholas Bastin wrote:
It's a mature product. I would hope that that would count for something.
Sure. But so is subversion.
I've had enough corrupted subversion repositories that I'm not crazy about the thought of using it in a production system.
I had the last corrupted repository with subversion 0.23. It has matured since then. Even then, invoking db_recover would restore the operation, without losing data (i.e. I did not need to go to backup).
Ah. But these people have no expertise with Perforce, and there is no Debian Perforce package, so it *is* a stretch assuming that they could also host a perforce directory. So I should then remove your offer to host a perforce installation, as you never made such an offer, right?
Yes. That's what this PEP is for. So I guess you are -1 on the PEP. Regards, Martin

On 8/8/05, "Martin v. Löwis" <martin@v.loewis.de> wrote:
I will then assume that you and I have different ideas of what 'mature' means.
So I should then remove your offer to host a perforce installation, as you never made such an offer, right?
Correct. .
Yes. That's what this PEP is for. So I guess you are -1 on the PEP.
Not completely. More like -0 at the moment. We need a better system, but I think we shouldn't just pick a system because it's the one the PEP writer preferred - there should be some sort of effort to test a few systems (including bug trackers). I know this is work, but this isn't just something we can change easily again later. -- Nick

On Mon, 2005-08-15 at 12:27 -0400, Nicholas Bastin wrote:
Bigger projects than Python use it and consider it mature for real use (All the Apache projects, all of KDE, GNOME is planning on switching soon, etc). I've never seen a corrupted FSFS repo, only corrupted BDB repos, and I will happily grant that using BDB ended up being a big mistake for Subversion. Not one that could have easily been foreseen at the time, but such is life. But this is why FSFS is the default for 1.2+ I've never seen you post about a corrupted repository to svn-users or svn-dev or file a bug, so i can't say why you see corrupted repositories if they are FSFS ones. --Dan

Nicholas Bastin wrote:
But that's how the PEP process works: the PEP author is supposed to collect feedback from the community in a fair way, but he is not required to implement every suggestion that the community makes. People who strongly disagree that the entire approach should be taken should write an alternative ("counter") PEP, proposing their strategy. In the end, the BDFL will pronounce which approach (if any) should be implemented. In the specific case, I'm personally not willing to discuss every SCM system out there. If somebody manages to make me curious (as Guido did with the bazaar posts), I will try it out, if I can find an easy way to do so. Your comments about (what was the name again) did not make me curious. As for bug trackers: this PEP is specifically *not* about bug trackers at all. If you think the SourceForge bugtracker should be replaced with something else, write a PEP. I really don't see a reasonable alternative to the SF bugtracker.
I know this is work, but this isn't just something we can change easily again later.
I don't bother asking who "we" is, here: apparently not you. Regards, Martin

Nicholas Bastin wrote:
compared to Perforce, SVN is extremely fragile. I've used both for years, and I've never had Perforce repository break down on me. our SVN repositories are relatively stable these days, but the clients are still buggy as hell (mostly along the "I don't feel like doing this today, despite the fact that it worked yesterday, and I don't feel like telling you what's wrong either" lines. having to nuke workspaces from time to time gets boring, quickly.) in contrast, Perforce just runs and runs and runs. the clients always do what you tell them. and server maintenance is trivial; just make sure that the server starts when the host computer boots, and if you have enough disk, just leave it running. if you're tight on disk space, trim away some log files now and then. that's it. but despite this, if all you need is a better CVS, I'd say SVN is good enough for today's python-dev. I'd still think that a more distributed, mail-driven system (built on top of Mercurial, Bazaar-NG, or some such (*)) would speed up both development and patch processing, and also make it a lot easier for "casual contributors" and "drive-by developers" to help develop Python, but that's another story. </F> *) being able to ship a fully working Python-powered SCM with the Python source code would be an extra coolness bonus, of course.

On 8/10/05, Fredrik Lundh <fredrik@pythonware.com> wrote:
We've used P4 at Elemental for two years now; I mostly agree with this assessment, although occasionally the server becomes unbearably slow and a sysadmin does some painful magic to rescue it. Maybe that's just because the box is underpowered. More troublesome is that I've seen a few client repositories getting out of sync; one developer spent a lot of time tracking down mysterious compilation errors that went away after forced resync'ing. We never figured out the cause, but (since he swears he didn't touch the affected files) most likely hitting ^C during a previous sync could've broken some things. Another problem with P4 is that local operation is lousy -- if you can't reach the server, you can't do *anything* -- while svn always lets you edit and diff. Also, P4 has *no* command to tell you which files you've created without adding them to the repository yet -- so the most frequent build breakage is caused by missing new files. Finally, while I hear that P4's branching support is superior over SVN's, I find it has a steep learning curve -- almost every developer needs some serious hand-holding before they understand P4 branches correctly. I'm intrigued by Linus Torvald's preference for extremely distributed source control, but I have no experience and it seems a bit, um, experimental. Someone should contact Steve Alexander, who I believe is excited about Bazaar-NG. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <gvanrossum@gmail.com> wrote:
"git", which is Linus' home-grown replacement for BitKeeper, quickly attracted a development community and has grown into a reasonably full-featured distributed RCS. It is apparently already stable enough for serious use. If I was trying to pick an RCS for a large, distributed project, I would at least investigate it as a possibility. Charles -- ----------------------------------------------------------------------- Charles Cazabon <python@discworld.dyndns.org> GPL'ed software available at: http://pyropus.ca/software/ -----------------------------------------------------------------------

[Guido van Rossum wrote]
This one is a frequent complaint from CVS-heads here at ActiveState. I have a p4 wrapper called "px" that extends some p4 commands (and adds a couple). One of the commands that it extends is "diff" to add a "-sn" (new) option similar to the "-se" (edit), "-sd" (delete). $ px help diff ...the usual 'p4 help diff'... new px options: [-sn -c changelist#] Px adds another -s<flag> option: -sn Local files not in the p4 client. Px also adds the --skip option (which only makes sense together with -sn) to specify that regularly skipped file (CVS control files, *~) should be skipped. The '-c' option can be used to limit diff'ing to files in the given changelist. '-c' cannot be used with any of the '-s' options. 'px' should grow a "px status" a la "svn|cvs status" to give a quick summary of local differences. Other additions: $ px help px 'px' entensions to 'p4': px --help Add px-specific help output to the usual 'p4 -h' and 'p4 -?'. See 'px help usage'. px -V, --version Print px-specific version information in addition to the usage 'p4 -V' output. See 'px help usage'. px -g ... Format input/output as *un*marshalled Python objects. Compare to the usual 'p4 -G ...'. See 'px help usage'. px annotate ... Identify last change to each line in given file, like 'cvs annotate' or 'p4pr.pl'. See 'px help annotate'. px backout ... Provide all the general steps for rolling back a perforce change as described in Perforce technote 14. See 'px help backout'. px changes -d ... Print the full 'p4 describe -du' output for each listed change. See 'px help changes'. px diff -sn --skip ... List local files not in the p4 depot. Useful for importing new files into a depot via 'px diff -sn --skip ./... | px -x - add'. See 'px help diff'. px diff -c <change> ... Limit diffing to files opened in the given pending change. See 'px help diff'. px genpatch [<change>] Generate a patch (usable by the GNU 'patch' program) from a pending or submitted chagelist. See 'px help genpatch'. Available here: http://starship.python.net/~tmick/#px Pure python. Works on Python >=2.2. Windows, Linux, Mac OS X, Unix. Trent -- Trent Mick TrentM@ActiveState.com

Perforce offers free licensing to open source projects.
There *is* support for converting a CVS repository to Perforce [1]. Perforce is very good, stable, solid, reliable, good tools, etc. etc. but I'd tend to support SVN over Perforce for Python development. Perforce usage is quite different than CVS (would be a painful re-learning for old CVS-hands) and SVN tends to better support highly distributed development: most operations don't need to talk to the server, with Perforce (aka p4), almost *all* operations talk to the server. This can be somewhat mitigated with "p4proxy" (a tool that Perforce also provides) but people would be happier with SVN, I'd bet. [1] It is a project called VCP. Some details here (I didn't look too hard): http://www.cpan.org/modules/by-module/LWP/AUTRIJUS/VCP-autrijus-snapshot-0.9... Trent -- Trent Mick TrentM@ActiveState.com

On Mon, Aug 08, 2005, Trent Mick wrote:
So did BitKeeper. Linux got bitten by that. We'd need a strong incentive to consider Perforce over Subversion just because of that issue. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything.

Trent Mick wrote:
Ok, so I now got "it's mature", and "it would be without charges". Given that it is now running against Subversion, I would be still interested in advantages that it offers compared to svn. The biggest disadvantage, to me, is that few people know how to use it (myself included). I don't trust tools I've never used. So for me, as the author of this PEP, usage of the revsion control software is non-negotiable (selection of hoster, to a limited degree, is). If you want to see Perforce used for the Python development, you should write a counter-PEP, so we could let the BDFL decide. [This is a theoretical "you" here, since you then explain that you would still prefer to use subversion] Regards, Martin

One feature I like in Perforce (which Subversion doesn't have) is the ability to have pending changesets. A changeset is, as with subversion, something you check-in atomically. Pending changesets in Perforce allow you to (1) group related files in a source tree where you might be working on multiple things at once to ensure and (2) to build a change description as you go. In a large source tree this can be useful for separating chunks of work. There are other little things, like not being able to trim the check-in filelist when editing the check-in message. For example, say you have 10 files checked out scattered around the Python source tree and you want to check 9 of those in. Currently with svn you have to manually specify those 9 to be sure to not include the remaining one. With p4 you just say to check-in the whole tree and then remove that one from the list give you in your editor with entering the check-in message. Not that big of a deal. [Martin v. L?wis on Perforce]
The biggest disadvantage, to me, is that few people know how to use it (myself included).
Granted. For that reason and for a couple of others I mentioned (SVN will probably work better for offline and distributed developers) I think Subversion wins over Perforce. That is presuming, of course, that we find Subversion to be acceptibly stable/robust/manageble. Trent -- Trent Mick TrentM@ActiveState.com

[Trent Mick]
This seems dubious, since you're not checking in the state you actually have locally, and you were careful to run the full Python test suite with your full local state ;-)
As a purely theoretical exercise <wink>, the last time I faced that under SVN, I opened the single file I didn't want to check-in in my editor, did "svn revert" on it from the cmdline, checked in the whole tree, and then hit the editor's "save" button. This doesn't scale well to skipping 25 of 50, but it's effective enough for 1 or 2.

On Mon, 2005-08-08 at 19:29, Tim Peters wrote:
Or one could use a decent client, like say psvn under XEmacs <wink> which presents you a list of all modified files and lets you select which ones you want to commit. The one thing I dislike about svn (in my day-to-day use of it) is that it can take a VERY long time to do updates at the roots of very large trees. I once tried to check out the root of our dev tree, which contains all branches and tags. Of course the initial checkout took forever. But an update at the root made this approach unusable. svn would sit there, seemingly idle for 30-45 minutes and then take another 30-45 minutes updating the changes, which typically consisted of maybe 50 files out of thousands. And this on a gig LAN with fast h/w all around (and for Tim's sake I won't even complain about how some operating systems appear to perform much worse than others :). The smaller you can keep your working copies, the better. -Barry

On Mon, 2005-08-08 at 15:49, Trent Mick wrote:
This seems like a poor workaround for crappy branch/merge support. I'm new to perforce, but the pending changesets seem dodgey to me... you are accumulating changes gradually without recording any history during the process... ie, no checkins until the end. Even worse, perforce seems to treat clients like "unversioned branches", allowing you to review and test pending changesets in other clients. This supposedly allows people to review/test each others changes before they are committed. The problem is, since these changes are not committed, there is no firm history of what what was reviewed/tested vs what gets committed... ie they could be different. Having multiple different pending changesets in one large source tree also feels like a workaround for high client overheads. Trying to develop and test a mixture of different changes in one source tree is asking for trouble... they can interact. Maybe I just haven't grokked perforce yet... which might be considered a black mark against it's learning curve :-) For me, the logical way to group a collection of changes is in a branch. This allows you to commit and track history of the collection of changes. You check out each branch into different directories and develop/test them independantly. The branch can then be reviewed and merged when it is complete. -- Donovan Baarda <abo@minkirri.apana.org.au>

Who made me the Perforce-bitch? Here I am screaming "Subversion! Subversion!" and y'all think I just using that as cover for a p4 lover affair. :) [Donovan Baarda wrote]
More like a pretty nice independent self-organizing feature that was necessitated as a workaround for a crappy solution (clientspecs) for huge data trees.
You want to do checkins of code in a consisten state. Some large changes take a couple of days to write. During which one may have to do a couple minor things in unrelated sections of a project. Having some mechanism to capture some thoughts and be able to say "these are the relevant source files for this work" is handy. Creating a branch for something that takes a couple of days is overkill. Perforce branching is pretty good in my experience. For very long projects one can easily create a branch.
Even worse, perforce seems to treat clients like "unversioned branches", allowing you to review and test pending changesets in other clients.
I'm not sure what you are talking about here. Yes, client information is stored on the server, but the *changes* (i.e. the diffs) on the client aren't so you must be talking about some other tool. Actually, if there *were* such a feature that would be quite handy. I'd love to be able to easily transfer my diffs developed on my Windows box to my Linux or Mac OS X box to quickly test changes there before checking in.
The alternative being either that you have separate branches for everything (can be a pain) or just check-in for review (possibly breaking the build or functionality for other developers until the review is done). Actually the Perl guys working on PureMessage downstairs have two branches going in Perforce: one for checking into right away and then a cleaner tree to which only reviewed check-ins from the first are integrated. I'm not saying I am awash in pending changelists here. Nor that they should be used for what is better handled with branching. It is a tool (and a minor one).
Trying to develop and test a mixture of different changes in one source tree is asking for trouble... they can interact.
...within reason. Trent -- Trent Mick TrentM@ActiveState.com

On Mon, 2005-08-08 at 17:51, Trent Mick wrote: [...]
I don't need to checkin in a consitent state if I'm working on a seperate branch. I can checkin any time I want to record a development checkpoint... I can capture the thoughts in the version history of the branch.
It all comes down to how painless branch/merge is. Many esoteric "features" of version control systems feel like they are there to workaround the absence of proper branch/merge histories. Note: SVN doesn't have branch/merge histories either. -- Donovan Baarda <abo@minkirri.apana.org.au>

"Donovan" == Donovan Baarda <abo@minkirri.apana.org.au> writes:
Donovan> It all comes down to how painless branch/merge is. Many Donovan> esoteric "features" of version control systems feel like Donovan> they are there to workaround the absence of proper Donovan> branch/merge histories. It's not that simple. I've followed both the Arch and the darcs lists---they handle a lot more branch/merge scenarios than Subversion does, but you still can't get away with zero discipline. On the other hand, for the purpose of the main repository for a well-factored project with disciplined workflow like Python, it's not obvious to me that the middle-complexity scenarios are that important. Furthermore, taking good advantage of the more complex branch/merge scenarios will require a change to Python workflow (for example, push- to-tracker will no longer be a convenient way to submit patches for most developers); that will be costly. More important, since none of the core Python people have spoken up strongly in favor of an advanced system, I would guess there's little experience to support planning a new workflow. (Cf. the Linux case, where Linus opted to roll his own.) I know many people in the Emacs communities who are successfully using CVS for the main repositories and various advanced systems (prcs, darcs, arch at least) for local branching and small group project communication. It seems fairly easy to automate that (much easier than extracting changeset information from CVS!) I think that as developers find they have need for such capabilities, the practice will grow in Python too, and then there may be a case to be built for moving the main repository to such a system. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

Trent Mick wrote:
One feature I like in Perforce (which Subversion doesn't have) is the ability to have pending changesets.
That sounds useful.
Depends on the client also. With Tortoise SVN, you do get a checkbox list where you can exclude files from the checkin. Regards, Martin

Donovan Baarda <abo@minkirri.apana.org.au> writes:
This is why I don't bother migrating any existing CVS projects to SVN; the benefits don't yet outweigh the pain of migrating.
I think they do. I was on dialup for a while, and would have _loved_ Python to be using SVN then -- and given how long diffs can take even over my broadband connection... Cheers, mwh PS: Wot, noone's suggested git yet? :) -- C++ is a siren song. It *looks* like a HLL in which you ought to be able to write an application, but it really isn't. -- Alain Picard, comp.lang.lisp

Martin v. Löwis wrote:
maybe it's changed since I last looked at it, but last time I looked SVN didn't track merge histories. From the svnbook; "Unfortunately, Subversion is not such a system. Like CVS, Subversion 1.0 does not yet record any information about merge operations. When you commit local modifications, the repository has no idea whether those changes came from running svn merge, or from just hand-editing the files." What this means is SVN has no way of automatically identifying the common version. An svn merge requires you to manually identify and specify the "last common point" where the branch was created or last merged. PRCS automatically finds the common version from the branch/merge history, and even remembers the merge/replace/nothing/delete decision you make for each file as the default to use for future merges. You can see this in the command line differences. For subversion; # create and checkout branch my-calc-branch $ svn copy http://svn.example.com/repos/calc/trunk \ http://svn.example.com/repos/calc/branches/my-calc-branch \ -m "Creating a private branch of /calc/trunk." $ svn checkout http://svn.example.com/repos/calc/branches/my-calc-branch # merge and commit changes from trunk $ svn merge -r 341:HEAD http://svn.example.com/repos/calc/trunk $ svn commit -m "Merged trunc changes to my-calc-branch." # merge and commit more changes from trunk $ svn merge -r 345:HEAD http://svn.example.com/repos/calc/trunk $ svn commit -m "Merged trunc changes to my-calc-branch." Note that 341 and 345 are "magic" version numbers which correspond to the trunc version at the time of branch and first merge respectively. It is up to the user to figure out these versions using either meticulous use of tags or svn logs. In PRCS; # create and checkout branch my-calc-branch $ prcs checkout calc -r 0 $ prcs checkin -r my-calc-branch -m "Creating my-calc-branch" # merge and commit changes from trunk $ prcs merge -r 0 $ prcs checkin -m " merged changes from trunk" # merge and commit more changes from trunk $ prcs merge -r 0 $ prcs checkin -m " merged changes from trunk" Note that "-R 0" means "HEAD of trunk branch", and "-r my-calc-branch" means "HEAD of my-calc-branch". There is no need to figure out what versions of those branches to use as the "changes from" point, because PRCS figures it out for you. Not only that, but if you chose to ignore changes in certain files during the first merge, the second merge will remember that as the default action for the second merge. -- Donovan Baarda

Donovan Baarda wrote:
What this means is SVN has no way of automatically identifying the common version.
Ah, ok. That's true. It doesn't mean you can't do proper merging with subversion - it only means that it is harder, as you need to figure out the revision range that you want to merge. If this is too painful, you can probably use subversion to store the relevant information. For example, you could define a custom property on the directory, last_merge_from_trunk, which you would always update after you have done a merge operation. Then you don't have to look through history to find out when you last merged. Regards, Martin

On Sunday 07 August 2005 15:33, Martin v. Löwis wrote:
This is what I do with shtoom - I have properties branchURI and branchRev on the root of the branch. I can then use these when landing the branch. It seems to work well enough for me. Anthony -- Anthony Baxter <anthony@interlink.com.au> It's never too late to have a happy childhood.

On Thu, Jul 28, 2005, Barry Warsaw wrote:
Why can't you write a Python script to generate the URLs? <0.3 wink> -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything.

Tim Peters wrote:
That would indeed give conversion problems: cvs2svn automatically generates tags/trunk/branches in the root, with the original CVS modules below; there is a single space for tags and branches (as is in the original CVS repository). I would see two possible conversion procedures: 1. convert the modules one-by-one, adding to the same repository. I.e. python would get all the small revision numbers, then peps, then sandbox, and so on. Cross-module tags would not be supported (but likely don't exist in the first place), and the revision number would not increase in historical order. 2. convert the entire CVS, then rearrange things through svn move. This would be tedious to do (atleast for tags/branches), and it would cause all files to be renamed right from the scratch (some svn clients fail to display history past moves). So for all who would prefer to see a single repository, could somebody please 1. state how precisely the repository should be organized (with exact URLs on svn.python.org - eg. which of the non-dist directories becomes toplevel, which ones get their own tags/branches/trunk, etc). 2. state how the conversion should be performed Regards, Martin

Martin v. Löwis wrote:
Nah
You reminded me of another reason why I used a custom conversion script. :) I did convert projects individually. I told cvs2svn to just create dump files. I then used svnload to load the dump files myself so that I could make each project a top-level directory with it's own trunk, branches and tags. I'd be happy to share my scrips, although there's nothing all that complicated about them.
I'm not close enough to the Python repo to offer much of an opinion other than: - IMO, a single repository is good - The repository should be orgabized by "projects", where separate projects reflect more or less independent development efforts with their own teams and schedules. (Maybe Python doesn't have many of these.)
2. state how the conversion should be performed
Once you decide what the projects are, I suggest converting each project separately. I'd be happy to share my scrips and experenice, although, as Tim noted I'll be off-line for two or three weeks starting in the next few days. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

Jim Fulton wrote:
If that's how it worked, I'm sure I can cook my own. Just for confirmation: the svn revision numbers don't increase chronologically across 'modules', right: i.e. the first revision of the module that was converted second has a higher revision than the last revision of the first module, even though historically, the order should have been reverse. Not that this worries me much, but I'd like to confirm there is no other way. Regards, Martin

Martin v. Löwis wrote:
Right. The revision numbers reflect the order in which they are added to the svn repo. I'm pretty sure the revision meta data, in particular the date, was retained. This is an advantage advantage of using the low-level dump/load mechanism. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

Tim Peters wrote:
I'm not aware of any errors in the conversion. ...
Ah, before I forget, "single repository" has worked very well for Zope
Yup. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

Jeff Rush wrote:
That is the currently-proposed setup. However, with the current subversion clients, you will have to save your password to disk, or type it in every time. This is the real security disk: if somebody attacks the client machine, they get access to the python source repository. Regards, Martin

Tim Peters wrote:
[Martin v. Löwis]
[...]
If you hit any snags, you may be interested in contacting the admin for scipy.org. The scipy CVS repo choked cvs2svn pretty badly a while ago, but recent efforts eventually prevailed. This afternoon an email arrived from him: -------- Original Message -------- Subject: [SciPy-dev] SciPy CVS to Subversion migration Date: Thu, 28 Jul 2005 20:02:59 -0500 From: Joe Cooper <joe@enthought.com> Reply-To: SciPy Developers List <scipy-dev@scipy.net> To: SciPy Dev List <scipy-dev@scipy.net> Hi all, The issues with converting our CVS repository to Subversion have been resolved, and so I'd like to make the switch tomorrow (Friday) afternoon. [...] ------------------------------------------------------------ I know Joe was in contact with the SVN devs to work on this, so perhaps he's using a patched version of cvs2svn, I simply don't know. But I mention it in case it proves useful to the python.org conversion. cheers, f

Fernando Perez wrote:
Thanks for the pointer. It turns out that I could resolve all my conversion problems myself (following Jim Fulton's suggestion of creating dump files). I found that somebody created a patch to support different structures in cvs2svn directly, but these patches have not been integrated yet. Regards, Martin

Tim Peters wrote: ...
I had two problems, one of which was zope specific: 1. We were making extensive use of symbolic links to share packages among various Zope projects. This requires special care and was the main reason I wrote my own scrips. I don't expect that this would be an issue for Python. 2. I initially tried to conver our entire repository, including all branches. This would have taken days. I finally decided to just convert our trunk, which took several hours. The main time sink was in the load step of the conversion process. I suspect that much of the time hit was due to the Berkely DB back end. I think I've heard that the file-system-based back end is much faster in general. We're using the BDB back end because that's all that was available at the time we converted. Every few weeks, the database gets wedged and I have to do a recovery operation. Every few months, the machine gets wedged and required a reboot. I'm pretty sure the later is due to locking issues. I definately recommend the file-system back-end, even though I haven't tried it myself. I haven't had the time to do the conversion myself. :( I'll also say that, on balance, I'm *very* *very* happy with subversion. I recommend it highly.
Other than that, I'd just like to see an explicit mention in the PEP of a plan to preserve the pre-conversion CVS tarball forever.
+1 Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

On Friday 29 July 2005 09:17, Jim Fulton wrote:
I know of three symlinks in the Python repository, all from the distutils project. There's one for the package and one for each of the documents.
This might be a possibility for Python as well, though we have a much less complex branching structure, so the conversion may not be so difficult. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

Jim Fulton wrote:
I don't expect that this would be an issue for Python.
Right.
Dunno. For the Python CVS (which translates into some 40,000 revisions), on the machine which I was originally using (1GHz Pentium), conversion took 7h. On my current workstation, it takes a little over an hour. Regards, Martin

Martin v. Löwis wrote:
Was this with the file-system back end? What is your current system? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

"Martin v. Löwis" <martin@v.loewis.de> writes:
Would it work/how much risk would it be to create accounts with shell /bin/false? Cheers, mwh (still faintly bothered by ~/.subversion/auth/svn.simple/*...) -- If trees could scream, would we be so cavalier about cutting them down? We might, if they screamed all the time, for no good reason. -- Jack Handey

Michael Hudson wrote:
Would it work/how much risk would it be to create accounts with shell /bin/false?
It seems that the pydotorg admins are worried about such a prospect. I believe this alone either won't work or won't be good enough (not sure which one): If you have /bin/false as login shell, and still manage to invoke /usr/bin/svnserve remotely, you can likely also invoke /usr/bin/cat /etc/passwd remotely (or download and build the root exploit via ssh). So you would have restrict the set of valid programs to *only* svnserve. This is possible, but difficult to manage (AFAIK).
(still faintly bothered by ~/.subversion/auth/svn.simple/*...)
Indeed. I personally would prefer SSL client certificates. This is still tricky (where do you get the passphrase for the private key from (*)), but slightly better. Regards, Martin (*) In case you wonder, I'm personally using the following techniques here: - on windows, remove the passphrase from the private key/.p12 file, and encrypt the file through NTFS encryption. Then you don't need to enter a passphrase, but still nobody can steal the private key from your disk (as long as you are not logged in; the same holds for ssh-agent) - on Linux, my issue is that .subversion is on NFS, so any root user in our net can connect to the file. Therefore, I copy the .p12 file to /tmp/private_dir, and remove the passphrase there. No other machine can read the file (as /tmp is not exported), and the file goes away after machine shutdown latest (as tmp is cleaned on reboot). - again on windows, I'm working on teaching subversion the Microsoft certificate store.

Martin v. Löwis wrote:
I'd like to see the Python source be stored in Subversion instead of CVS,
+1
Not sure about the move to svn.python.org. This would bind additional developer resources for doing administration work. If SF is such a show-stopper, would it be reasonable to look for managed alternatives, such as eg. CollabNet (who funded Subversion development) ? Perhaps we could get a good deal from them given that the PSF is a non-profit organization. Greg Stein could probably provide contacts to the right people to talk to. http://www.collab.net/products/team_edition/index.html Just an idea, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 29 2005)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

On Friday 29 July 2005 06:34, M.-A. Lemburg wrote:
docutils has been using berlios.de for Subversion; that might be a reasonable option. I'm not sure if there are any political issues about that, but I think everyone actively developing on that project has been happy with the move. (Only the berlios.de Subversion is being used; everything else remains at SourceForge IIRC.) -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

M.-A. Lemburg wrote:
True. OTOH, there already is a subversion on svn.python.org, and administrative work is likely only about creating new accounts. I guess the current SF project admins would help out if help is needed.
If SF is such a show-stopper
It is at the moment, atleast: there currently is not svn support on sf.net.
Perhaps. Somebody would need to research the precise migration procedure. I once tried to move the Python CVS to Sunsite (or its successors), and gave up after half a year of getting nowhere; I'm personally not keen on repeating such an experience. However, if somebody stepped forward to do the actual work (perhaps based on the draft PEP), I would happily hand this project over (it would be good if that volunteer would set a deadline for negotiations with the new host). Regards, Martin

On Fri, 2005-07-29 at 16:59, "Martin v. Löwis" wrote:
I'm with you. SF is a devil we know. If we don't stick with them (and it looks like that's not an option if we want to switch to svn soon), then I say we move it to svn.python.org, where we already have a server set up and running. If we need more volunteers, well the community has always stepped up before and I'm sure it will when we come asking again. Actually, it's a good idea to /always/ be soliciting for help. People get burned out and busy and I'd love to have a bullpen of volunteers that gets constantly refreshed. -Barry

Barry Warsaw wrote:
I guess my point in suggesting a company to do the hosting was to overcome any issues with people getting burned or tired of administration. The PSF does have a reasonable budget, so why not use it to maintain the infrastructure needed for Python development and let a company do the administration of the needed servers and the importing of the CSV and tracker items into their systems ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 30 2005)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

Martin v. Löwis wrote:
True, but if we never ask, we'll never know :-) My question was: Would asking a professional hosting company be a reasonable approach ?
From the answers, I take it that there's not much trust in these offers, so I guess there's not much desire to PSF money into this.
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 02 2005)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

M.-A. Lemburg wrote:
It would be an option, yes, of course. It's not an approach that *I* would be willing to implement, though.
From the answers, I take it that there's not much trust in these offers, so I guess there's not much desire to PSF money into this.
I haven't received any offers to make a qualified statement. I only know that I would oppose an approach to ask somebody but our volunteers to do it for free, and I also know that I don't want to spend my time researching commercial alternatives (although I wouldn't mind if you spent your time). Regards, Martin

Martin v. Löwis wrote:
Fair enough.
I don't quite understand what you meant here: are you opposing spending PSF money on a hosting company if and only if volunteers who take on the job don't get paid ? I've done a bit of research on the subject and so far only found CollabNet and VA offering commercial services in this area. VA hosts SourceForge so that's a non-option, I guess :-) I know that Greg Stein worked for CollabNet, so thought it might be a good idea to ask him about the idea to move things to CollabNet. Of course, before taking this route, I wanted to get a feeling for the general attitude towards a commercial approach, which is why I tossed in the idea. Other non-commercial alternatives are Berlios and Savannah, but I'm not sure whether they'd offer Subversion support. BTW, have you considered using Trac as issue tracker on svn.python.org ? They have a very good subversion integration, it's easy to use, comes with a wiki and looks great. Oh, and it's written in Python :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 03 2005)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

M.-A. Lemburg wrote:
No. I'm opposed to approaching somebody to do it for free, except the somebody are the pydotorg volunteers (IOW, I won't take gifts from anybody else in this matter).
It's not that I dislike VA - I personally think they are doing a great job with SourceForge, and I like SourceForge a lot. There are just some issues with it (like that they offer no Subversion). The question would be: what precisely is the commercial offering from VA: does it provide subversion? how is the user management done? etc.
Ok - I expect that the project might be *done* before we even have a single commercial offer, with a precise service description, and a precise price tag. That makes commercial offers so difficult: that it is so time expensive to use them, that you might spend less time doing it yourself.
Other non-commercial alternatives are Berlios and Savannah, but I'm not sure whether they'd offer Subversion support.
For me, they fall into the "I won't take gifts" category.
BTW, have you considered using Trac as issue tracker on svn.python.org ?
You mean, me personally? I quite like the Subversion tracker, and don't want to trade it for anything else. I know Guido wants to use Roundup (which is also written in Python), and obviously so does Richard Jones. The main questions are the same as with this PEP: how to do the migration from SF (without losing data), and how to do the ongoing maintenance. It's just that finding answers to these questions is so much harder, therefore, this PEP is *only* about CVS. Regards, Martin

Martin v. Löwis wrote:
Ok.
I guess this was a misunderstanding on my part: VA doesn't offer their commercial solution in an ASP-like way. Their product, called SourceForge Enterprise, is a J2EE application which we'd have to install and run. They do mention Subversion as being supported by the Enterprise edition.
For (more or less) simple things like setting up SVN, I'd agree, but for hosting a complete development system, I have my doubts - things start to get rather complicated and integration of various different tools tends to be very time consuming. Sysadmin tasks like doing backups, emergency recovery, etc. also get more complicated once you have to deal with many different ways of data storage deployed by such tools, e.g. many of them require use of special tools to do hot backups.
Ok, I'll drop the idea.
Ok. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 04 2005)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

M.-A. Lemburg wrote:
Ah, ok. I don't think I want to operate such a software (and, strictly speaking, this is out of the scope of the PEP). I had the "pleasure" once of having to maintain a SourceForge installation (before SourceForge became closed source), and it was a nightmare to operate.
I guess Python's development process is very simple then. We use mailing lists, CVS, newsgroups, web servers, and bug trackers, but these don't have to integrate. Many of these services are already on pydotorg, and I propose to add an additional one (revision control).
We are doing quite well here. XS4ALL kindly does disk backup for us, and, in the specific case of Subversion's fsfs, this is all that is needed. For Postgres, we backup to disk, which then gets picked up by the disk backup. Regards, Martin

Martin v. Löwis wrote:
With J2EE I doubt that things got any easier to maintain... (assuming that you had to run the version of the software which is used on SF.net).
Sounds like you have everything under control, which is good :-) BTW, in one of your replies I read that you had a problem with how cvs2svn handles trunk, branches and tags. In reality, this is no problem at all, since Subversion is very good at handling moves within the repository: you can easily change the repository layout after the import to whatevery layout you see fit - without losing any of the version history. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 04 2005)
2005-07-18: Released mxODBC.Zope.DA for Zope 2.8 ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

At 08:28 PM 8/4/2005 +0200, M.-A. Lemburg wrote:
Yeah, in my use of SVN I find that this is more theoretical than actual for certain use cases. You can see the history of a file including the history of any file it was copied from. However, if you want to try to look at the whole layout, you can't easily get to the old locations. This can be a royal pain, whereas at least in CVS you can use viewcvs to show you the "attic". Subversion doesn't have an attic, which makes looking at structural history very difficult. That having been said, I generally like Subversion, I just know that when I moved my projects to it I felt it was worth taking extra care to convert them in a way that didn't require me to reorganize the repository immediately thereafter, because I didn't want a sudden discontinuity, beyond which history would be difficult to follow. Therefore, I'm saying that taking some care with the conversion process to get things the way we like them would be a good idea.

Phillip J. Eby wrote:
Hmm, I usually create a tag before doing such changes in our Subversion repo. This makes it very easy to look at layouts before a restructuring. And because Subversion doesn't really care whether you do a tag, branch, or some other form of diverting versions into different namespaces (it's all just copying data), you can easily create a directory called "attic" for just this purpose and copy your structural change tags in there :-)
Still very true indeed. The fact that cvs2svn is written in Python should make this even easier. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 04 2005)
2005-07-18: Released mxODBC.Zope.DA for Zope 2.8 ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

Phillip J. Eby wrote:
I guess this is a client issue also; in websvn, you can browse an older revision to see what the structure looked at that point. If you made tags, you can also browse the tags through the standard HTTP interface. I don't know a client, off-hand, which would answer the question "which files have been moved since tag xyz?". Regards, Martin

M.-A. Lemburg wrote:
Yes, however, I recall that some clients have problems with displaying history across renames (in particular, I believe viewcvs has this problem); also, it becomes difficult to refer to an old version by path name, since the old versions had all different path names. Jim Fulton has suggested a different approach: cvs2svn can create a dump file, and svnadmin load accepts a parent directory. Then, no renames are necessary. Regards, Martin

Martin v. Löwis wrote:
Since I only use trac to view the source code (which doesn't have this problem), I can't comment on this.
Good idea. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 07 2005)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

On Wednesday 03 August 2005 15:01, M.-A. Lemburg wrote:
Other non-commercial alternatives are Berlios and Savannah, but I'm not sure whether they'd offer Subversion support.
Berlios does offer Subversion; the docutils project is using the Berlios Subversion and SourceForge for everything else. I don't know whether Savannah is offering Subversion right now, but the last time I looked at it, it appeared nearly un-maintained. But that may just be the understated nature of that community. :-) -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

"M" == "M.-A. Lemburg" <mal@egenix.com> writes:
M> Other non-commercial alternatives are Berlios and Savannah, but M> I'm not sure whether they'd offer Subversion support. Savannah doesn't offer great reliability or support, at least to judge by the frequency with which the GNU Emacs and GNU Arch projects have been unable to access various services on Savannah, including mailing lists and CVS. I also wonder if Savannah poses security risks. They've been successfully cracked (ISTR more than once) in the last couple of years, and took 6-10 weeks to get back to normal. This makes them reluctant to make minor variations in their established procedures for the convenience of projects. For example, it took a couple of months for GNU Arch to arrange sftp access so that they could host the Arch project in an Arch repository (Arch can use sftp but not plain ssh as a transport). SunSITE.dk does provide reliable service and timely support. XEmacs has been very happy with it. But Martin v. Loewis apparently hasn't had the same good experience with negotiating with them, and at least some negotiation and relationship maintenance is necessary---it's a closer, more personal relationship than with SF or Savannah. In particular for Subversion support (I was told they allow it on a case by case basis, and once success is demonstrated they plan to offer it in general). As I say, we've been happy with SunSITE, but the amount of effort is basically the same as if we ran our own repository, just directed more toward "vendor relations" and away from "sys admin" (which suits us). FWIW, XEmacs has moved or reorganized CVS repositories five times since 1999. Although it's not all in the PEP, if you add the discussion on this list Martin has covered the important issues we encountered or worried about. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

On Thursday 28 July 2005 13:00, Martin v. Löwis wrote:
I'd like to see the Python source be stored in Subversion instead of CVS,
I'm +1 on this, assuming we use the fsfs backend, and not the berkeley DB one. I'm -1 if we're using the bdb backend (I've had nothing but pain from it).
- tagging for releases will no longer cause the release manager to experience fits of burning rage (personal record was something like 1h45m for 'cvs tag' to finish, from memory). My only concern is that we have sufficient volunteers to manage the system. I'm happy to be one of these, but that's assuming we have other people also volunteering. . . Anthony -- Anthony Baxter <anthony@interlink.com.au> It's never too late to have a happy childhood.

On 7/28/05, "Martin v. Löwis" <martin@v.loewis.de> wrote:
In principle I'm +1 on this (both aspects). I don't know enough to understand all the subtleties. I hope we're correctly estimating the effort required to manage the server and the users. Managing users is especially important -- if a user is compromised (as has happened in the past for python.org users) the whole repository is compromised. Now this could happen to SF users too, but I'm not sure that we know all the tricks in the book to prevent attacks; SF has been doing this for years and that's an aspect of SF that I trust (I think I've heard that they have even modified their SSH server to be stricter). -- --Guido van Rossum (home page: http://www.python.org/~guido/)

On Jul 28, 2005, at 4:20 PM, Guido van Rossum wrote:
If you use the fsfs storage mechanism for subversion, it is somewhat simpler to verify that the repository is not compromised. Each commit is represented as a separate file, and thus old commits are never modified. Only new files are appended to the directory. If you have a filesystem that allows "append-only" permissions on a directory, you can enforce this directly. Additionally, it is possible in your backup script to verify that only new files were added and nothing else changed. Then at least you know how much you need to examine instead of having to treat the entire repository as possibly contaminated. James

On Thu, 2005-07-28 at 16:20, Guido van Rossum wrote:
I hope we're correctly estimating the effort required to manage the server and the users.
Yah, me too! ;) We are building some experience with this though, having moved many of the system files, and all of the web pages, to svn. So far, the management overhead has been almost nil (um, None :). We'll have a bit of ongoing work to add users, but the infrastructure team is also building up some community knowledge about how to do that.
James has a very interesting idea for mitigating this. Presumably <heh>, we'll have backups of everything. I'll feel better when we have coverage from maybe 6 admins spanning as many timezones as possible. -Barry

I theory Subversion should allow you to be more secure. CVS has a very limited concept of security and for the most part you need to rely on SSH. Subversion makes use of Apache as one of its server options. Any authentication method you can use in Apache 2 you can use for Subversion. Once Apache does the authentication, Subversion allows fairly fine grained access control. SSL can be used for encrypting communication. Note that there are sites like Sourceforge that do provide Subversion hosting to open source projects. I don't know enough about them to be able to recommend any. -Chris On Thu, Jul 28, 2005 at 01:20:14PM -0700, Guido van Rossum wrote:

Guido van Rossum wrote:
There are three issues that I see: - management of access control. This is actually something that we do on SF as well; if the pydotorg admins get overloaded, I'm sure the current projects/python admins would be willing to help there. - assignment of passwords. This I don't like about the current pydotorg setup - there should be a way to chose your own password; perhaps without involving an administrator. I could imagine a web form for password change, and administrator interaction in case of a lost password. - compromised passwords. The only tricky question then is: was the repository altered? Fortunately, for Subversion, there should be an easy way to tell: in fsfs, files never change (only new files are added). So we could generate md5sums of all files in the repository, and download these to an offsite place. If the md5sum of an immutable file changes, we were compromised (there are, of course, a few files that do change regularly). Of course, we also need regular backups of the entire data so we can restore them if they got compromised. Regards, Martin

On Fri, 2005-07-29 at 00:44, "Martin v. Löwis" wrote:
I disagree. By reserving password generation to the pydotorg admins, we can better insure the passwords are more robust against dictionary attacks. See my previous message. I actually /don't/ want individuals to be able to set their own passwords. In practice, you only have to know your password once, because svn caches the authentication (yes, that opens up opportunities for compromise, but that's how svn works).
+1 to all that. -Barry

Barry Warsaw wrote:
See Michael's (I think) message: that is a much greater risk than the one of a brute-force attack. In our environment, a determined student could easily read out my home directory, and get at my pydotorg password (if I would allow svn to cache it). They would have to break all kinds of rules in doing so; yet, it would be technically possible - so I just can't turn on this svn setting, and have to type the password every time. This is surely inconvenient, as I cannot even remember the password. Regards, Martin

On Thu, Jul 28, 2005 at 10:00:00PM +0200, "Martin v. L?wis" wrote:
I'd like to see the Python source be stored in Subversion instead of CVS, and on python.org instead of sf.net.
+1, +1.
-- transactional operation - a changeset is either committed or rolled back at once; -- very effective (both in terms of speed and memory) tagging and branching; tags and branches are very easy to understand and use in SVN. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

On Thu, 2005-07-28 at 16:00, "Martin v. Löwis" wrote:
I'd like to see the Python source be stored in Subversion instead of CVS
+1
, and on python.org instead of sf.net.
+0 I know that SF has promised svn access to projects for a long time now, but I haven't heard anything from them in a long time. It's listed under their "Strategic Projects" but the last update to that news item was back in April. Question: do we wait for SF to make the service available (after getting more up-to-date status and a realistic timetable), or do we go straight to svn.python.org?
Thanks for writing this PEP Martin!
We've been going with firstname.lastname (with some exceptions -- hi Aahz! :) for the current svn access. Is it better to stay with that convention or to maintain consistency with SF's CVS committer names? Maybe the latter for revision history consistency.
2. At the beginning of the migration, announce that the repository on SourceForge closed.
We can probably play games with the various CVS hooks to disable all checkins during this time. We might also want to disable the u/i access to CVS at the same time.
4. Convert the CVS repository into two subversion repositories, one for distutils and one for Python.
Do we also want to split off nondist and encodings? IWBNI the Python source code proper weren't buried too deep in the directory structure. Note that we might want to provide different access permission to different parts of the repository (but I think we can do that even if we don't split those off into separate repos).
Definitely +1 on fsfs. Again, thanks Martin! -Barry

On 7/28/05, Barry Warsaw <barry@python.org> wrote:
+1 from me as well; single commit numbers for commits across multiple files will be wonderful.
I say forget SF and we move it. Of course I won't be involved with the migration so me saying this doesn't mean too much. =)
I say go with the first.last naming. While this might put committer names out of sync, we could keep a mapping of SF names to the new names in developers.txt for easy referencing. But it would be handy to have actual name references since I know I don't always remember who is whom on SF since some people go with nicks that are not based on their name at all. [SNIP]
Seems like a reasonable thing. Would make it easier for occasional committers as well as people who check out the code just for generating a patch. -Brett

Barry Warsaw wrote:
My proposal is to go straight to svn.python.org, for two reasons: - It might well be 2006 before they update the status and provide a realistic timetable. It's not that I lost faith into SF, but they seem to be continually overworked (as anybody), so anything that is not *really* essential to the operation of the service (such as support for subversion) has to wait "until we have time" -- which means it has to wait forever. Remember the plan to provide shell access to the project admins on the CVS server? - They currently have a separate anonymous access for CVS which is behind the real CVS, for load sharing reasons. They also had severe performance issues in the past. It might be that we also get performance problems, but we only need to support a single project (or two if you count pydotorg) on Subversion, not thousands of them. So our load evolution is much more predictable.
It's also a convenience issue, and has social aspects. For example, firstname.lastname does not work quite well for me, either (martin.v.loewis or martin.von.loewis would work better; likewise guido.van.rossum?), other users prefer not to use their "true" real name (e.g. tim_one vs. tim.peters). Another issue is password assignment - currently, users cannot choose their passwords on svn.python.org, right?
That should be tested in advance, of course.
Well, encodings is empty right now, so that might be a good idea. Technically, I would just move the files in the CVS repository before conversion. As for nondist: how precisely would you do that? Make it a separate repository? If so, where? I could envision a "/projects" repository, that has the current nondist.
I don't know how cvs2svn supports it, but one option would be to revert the trunk/ structure in /projects, so you get http://www.python.org/projects/peps/trunk http://www.python.org/projects/peps/tags http://www.python.org/projects/peps/branches http://www.python.org/projects/sandbox/trunk http://www.python.org/projects/sandbox/tags http://www.python.org/projects/sandbox/branches Then you could give different access to peps and to sandbox. Perhaps there isn't even a need for tags/branches on PEPs? Or we put everything in nondist into /projects, and then put tags + branches as siblings (so only people with write access to tags could create them)? Regards, Martin

On Fri, 2005-07-29 at 00:35, "Martin v. Löwis" wrote:
Yep, we would use guido.van.rossum. I think it would be fine for you to use martin.v.loewis or martin.von.loewis, just as MAL could use m.a.lemburg or marc.andre.lemberg. Our general concern was people hiding behind obscure nicknames like 'pumpichank' and no one really knowing who's behind that <wink>.
Another issue is password assignment - currently, users cannot choose their passwords on svn.python.org, right?
Correct, which I think is a feature. :)
Then you could give different access to peps and to sandbox. Perhaps there isn't even a need for tags/branches on PEPs?
Probably so. -Barry

"Martin v. Löwis" wrote:
If I understand things correctly, one project/one repo creates a 'hard' barrier for moving code across projects (while retaining history, so done via an svn command). Is the 'long url' really the only argument for this, and is it significant enough? Instead of: https://svn.python.org/python https://svn.python.org/distutils you could have https://svn.python.org/main/python https://svn.python.org/main/distutils or something similar. It's an extra few chars, and it would give a convenient way to branch off pieces of the main code into their own subprojects in the future if needed. For more experimental things, you can always have other repos: https://svn.python.org/someotherrepo/... But maybe the issue of moving code isn't too important, I'm certainly no expert on svn. Cheers, f

On Thursday 28 July 2005 20:07, Fernando Perez wrote:
More interestingly, keeping it in a single repository makes it easier to merge projects, or parts of projects, together, without losing the history. This would be useful when developing packages that may be considered for the standard library, but which also need to continue separate releases to support older versions of Python. We've found it very handy to keep multiple projects in a single repository for zope.org. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

On 7/29/05, Fred L. Drake, Jr. <fdrake@acm.org> wrote:
I thought I would chime in on this with some more experience. We use SVN (migated from CVS about 2 years ago) for everything, and we keep it all in one repository (even though there's several "products") for exactly this reason. The major "downside" that I, and some others, find is changeset explosion. When you start getting into 5 digit changeset numbers it can become a bit unwieldy to remember to type them all, but otherwise it works well. We also moved from BerkeleyDB-based to fsfs recently, and it seems to have fixed some intermittent problems we had had in the past. Another thing to look at would be Trac, which we are in the process of moving to from the horrendous nightmare of Bugzilla. It's integration with SVN as well as Wiki is quite amazing. Chris -- | Christopher Petrilli | petrilli@gmail.com

On Fri, 2005-07-29 at 00:50, Christopher Petrilli wrote:
Now's the time I pipe in to remind everyone that Atlassian has offered free (as in beer) versions of Jira and Confluence for the Python project (actually any open source project). If you haven't used these, they're definitely worth a look. Jira is the issue tracker, Confluence the wiki. They're extremely professional, usable, well-integrated, and you can integrate them with Subversion. We've used them at work for maybe a year now and I've been very happy with them. Jira is definitely one of the better issue trackers, free or not free, that I've used. www.atlassian.com They're not Python and they're not open source, so perhaps it's legitimate to dismiss them because of that. But they are also definitely cool. At the Atlassian folks are very cool too, and fans of FOSS. -Barry

Barry Warsaw wrote:
I've also used Confluence at work, and been very impressed. A very nice feature which I haven't seen anywhere else is that the Wiki pages allow comments as well as direct editing - it allows discussion *about* the page to occur in a normal answer/response fashion, possibly leading to eventual update of the page text itself.
I'd hope we wouldn't dismiss them out of hand - one of the things that appeals to me about Python is the philosophy that open-source and protected-source groups can both benefit from working together. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com

"Martin v. Löwis" wrote:
For ipython, which recently went through cvs2svn, I found that moving over to a project/trunk structure was a few minutes worth of work. Since svn has moving commands, it was just a matter of making the extra project/ directory and moving things one level down the hierarchy. Even if cvs2svn doesn't quite create things the way you want them in the long run, svn is flexible enough that a few manual tweaks should be quite easy to perform. Cheers, f

Martin v. Löwis wrote:
So do you use project/trunk or trunk/project? If the former, I would need to get instructions on how to do the conversion from CVS.
project/trunk/ On Friday 29 July 2005 02:12, Fernando Perez wrote:
Yes, this will work. I vaguely recall that Jim converted the zope.org repository one project at a time, so that made it easier to keep them separate. We didn't decommission the CVS entirely, though; Zope 2.7 is still maintained in CVS since we didn't want to risk worrying about the branch structure too much; cvs2svn still had a few issues with the complex branch structure combined with the use of symlinks within the repository (one of the reasons we were so keen to move to Subversion). I'm sure Jim can provide more details of what he had to do; I was only slightly involved in the discussions. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

"Martin v. Löwis" wrote:
To be honest, I don't really know the details, but it seems to work fine. A quick look at ipython: planck[IPython]> svn update At revision 661. planck[IPython]> svn diff -r 10 genutils.py | tail - - Deprecated: this function has been superceded by timing() which has better - fucntionality.""" - - rng = range(reps) - start = time.clock() - for dummy in rng: func(*args,**kw) - return time.clock()-start - -#*************************** end of file <genutils.py> ********************** Revision 10 was most certainly back in the early CVS days, and the wholesale renaming happened when I started using svn, which was around revision 600 or so. There may be other subtleties I'm missing, but so far I haven't experienced any problems. Cheers, f

Martin v. Löwis wrote:
I'm definitely positive to a migration to Subversion, but I'd be really concerned about using plain text authentication mechanisms. SSH obviously is much prefered, and clearly there are ways to limit the "accounts" on the svn.python.org, many other projects does this already for cvs. E.g thor (17:11) 350/0 $ ssh -l 'username' cvs.mozilla.org To use anonymous CVS install the latest version of CVS on your local machine. Then set your CVSROOT environment variable to the following value: cvsuser@megalon:/cvsroot Connection to cvs.mozilla.org closed. We should have enough man powers to come up with some secure solution here :). -- Leif

On Thu, 2005-07-28 at 20:15, Leif Hedstrom wrote:
I'm definitely positive to a migration to Subversion, but I'd be really concerned about using plain text authentication mechanisms.
We won't use plain text, but we may (or, we currently do) use basic auth over ssl. The security then is in the passwords, so we have to make sure they're generated securely.
We should have enough man powers to come up with some secure solution here :).
Volunteers (i.e. those with actual free time to give on implementing any solution) are definitely welcome. -Barry

On Fri, 2005-07-29 at 01:00, "Martin v. Löwis" wrote:
Actually, the passwords are still hashed in the file, so they wouldn't be able to extract the plain text password. They definitely are vulnerable to brute force attack, though probably not to a dictionary attack. In practice I've been using a password generated based on os.urandom() -- we generate the password and get it to the Subversion user via a "secure route" <heh>. I'd be happy to share my password generation script with anybody who wants to audit it. Public/private keys would be better, and if anybody knows how to set up a Subversion server to use these without having to create accounts for everyone, I think we (the pythong.org admins) would love your help. -Barry

Barry Warsaw wrote:
I'll play around with it some this weekend, see what's possible, and was isn't. I'm thinking we ought to be able to use SSH's configuration to only allow one specific command, e.g. something like this in the authorized_keys: no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command="/usr/bin/svnserve" -- Leif

At 06:39 PM 7/29/2005 -0400, Barry Warsaw wrote:
But that would still require us to create accounts for every committer, right?
No. Just one account. You can have more than one key listed in authorized_keys, using svnserve --tunnel-user and sshd will select the right command based on the public key the client authenticates with.

Barry Warsaw wrote:
Nah. Somebody who takes over svn.python.org can replace Apache, and that will receive plain text passwords over the wire (in case you wonder: modules/aaa/mod_auth.c:authenticate_real_user - you can even write an Apache module that gets hold of the sent password). An intruder would have to wait some time before the password come in, instead of being able to read them all from a file at once - that's true.
Ok. Since this falls in my research interest, I definitely want to give it a try. I think I would set up PyCA to let users generate their private keys in the browser. Regards, Martin

At 05:54 PM 7/29/2005 -0400, Barry Warsaw wrote:
From the svnserve man page: -t, --tunnel Causes svnserve to run in tunnel mode, which is just like the inetd mode of operation (serve one connection over stdin/stdout) except that the connection is considered to be pre-authenticated with the username of the current uid. This flag is selected by the client when running over a tunnel agent. --tunnel-user=username When combined with --tunnel, overrides the pre-authenticated username with the supplied username. This is useful in combina- tion with the ssh authorized_key file's "command" directive to allow a single system account to be used by multiple committers, each having a distinct ssh identity. So, it looks like you'd just need to set up public keys for each user, and list them in authorized_keys. Presumably doing something like this: command="/usr/bin/svnserve --root=/svnroot -t --tunnel-user=username",no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa [key info here] would therefore do the trick. I've used a similar arrangement for my own CVS repository, but haven't tried it for SVN yet.

[Martin v. Löwis]
Encouragement and support on SVN, undecided on moving to python.org (don't know when SF intends to support SVN, don't have a feel for the state of-- or propsects for ongoing --python.org volunteerism).
I'm sending this to Jim Fulton because he did the conversion of Zope Corp's code base to SVN. Unfortunately, Jim will soon be out of touch for several weeks. Jim, do you have time to summarize the high bits of the problems you hit? IIRC, you didn't find any conversion script at the time capable of doing the whole job as-is. If that's wrong, it would be good to know that too. Other than that, I'd just like to see an explicit mention in the PEP of a plan to preserve the pre-conversion CVS tarball forever.

On Thursday 28 July 2005 07:21 pm, Tim Peters wrote:
<snip>
The conversion script isn't perfect and does fail on complex CVS arrangements or where there is extensive history to migate. However it appears above that Martin has already tried the script out, with success. BTW, re SSH access on python.org, using Apache's SSL support re https would provide as good of security without the risk of giving out shell accounts. SSL would encrypt the link and require a password or permit cert auth instead, same as SSH. Cert admin needn't be hard if only a single server cert is used, with client passwords, instead of client certs. -Jeff

[Jeff Rush]
I'd still like to hear from Jim, as I don't believe all serious problems were identified by eyeball at once. That said, the Python project has made relatively very little use of complex (for CVS) concepts like branching, but in Zopeland it wasn't unusual, over long stretches, for someone to make a new branch every day. Ah, before I forget, "single repository" has worked very well for Zope (which includes top-level Zope2, Zope3, ZODB, ZConfig, zdaemon, ... projects): http://svn.zope.org/ Long URLs don't really get in the way in practice (rarely a need to type one after initial checkout; even "svn switch" is usually just a tail-end cmdline edit starting from a copy+paste of "svn info" output).

On Thu, 2005-07-28 at 22:14, Tim Peters wrote:
It depends. In my use of svn, I do a lot of cross-branch merging and repo-side tagging. Those are done with urls and in those cases, long urls can suck. But we may not do a ton of that with the Python project, and besides it might not be important enough to split the directories. -Barry

[Tim]
[Barry]
It depends. In my use of svn, I do a lot of cross-branch merging and repo-side tagging.
Yup, me too -- between the two of us, we don't have enough fingers to count how many trunks, branches, and tags of ZODB and Zope I have to fiddle with.
Those are done with urls and in those cases, long urls can suck.
They're all still copy, paste, tail-edit for me, and-- indeed! --having them all in the same repository is what makes just-the-tail editing possible. Merges I do from the cmdline, but repo-side tagging I do with the TortoiseSVN GUI, and the latter gives easy-to-copy/paste/edit URL fields. So switch to Windows for that part ;-)
But we may not do a ton of that with the Python project, and besides it might not be important enough to split the directories.
Ya, in Python we make a branch about once per release, + once per 5 years when Jeremy underestimates how long it will take to finish a crusade <wink>.

"BAW" == Barry Warsaw <barry@python.org> writes:
BAW> So are you saying that moving to svn will let us do more long BAW> lived branches? Yay! Yes, but you still have to be disciplined about it. svn is not much better than cvs about detecting and ignoring spurious conflicts due to code that gets merged from branch A to branch B, then back to branch A. Unrestricted cherry-picking is still out. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

On Sun, 2005-07-31 at 23:54, Stephen J. Turnbull wrote:
Yeah. IMHO the sadest thing about SVN is it doesn't do branch/merge properly. All the other cool stuff like renames etc is kinda undone by that. For a definition of properly, see; http://prcs.sourceforge.net/merge.html This is why I don't bother migrating any existing CVS projects to SVN; the benefits don't yet outweigh the pain of migrating. For new projects sure, SVN is a better choice than CVS. -- Donovan Baarda <abo@minkirri.apana.org.au>

"Donovan" == Donovan Baarda <abo@minkirri.apana.org.au> writes:
Donovan> Yeah. IMHO the sadest thing about SVN is it doesn't do Donovan> branch/merge properly. All the other cool stuff like Donovan> renames etc is kinda undone by that. [...] This is why Donovan> I don't bother migrating any existing CVS projects to Donovan> SVN; the benefits don't yet outweigh the pain of Donovan> migrating. FWIW, XEmacs just had this discussion, and we basically came to the conclusion that for a multi-developer project it's _definitely_ worth the effort if it can be done by cvs2svn (which for us it probably can't, due to some black magic we did on the CVS repository a few years ago :-( ). For the record, I was opposed for exactly the reason you give, but changed my mind. The point is that with several developers there's almost surely someone enthusiastic enough about svn to bear the burden of fooling with the script for a couple of hours to see if it works, a fascist policy about migrating account names makes that almost trivial, and after that it's all gravy: the administration does not look any worse, the security issues are similar, and the change is likely to incite only a few people to press for account name changes after the move. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.

George V. Neville-Neil wrote:
No. The PEP is only about Subversion. Why should we be looking at Per Force? Only because Python is Open Source? I think anything but Subversion is ruled out because: - there is no offer to host that anywhere (for subversion, there is already svn.python.org) - there is no support for converting a CVS repository (for subversion, there is cvs2svn) Regards, Martin

[Martin von Löwis]
The PEP is only about Subversion. I think anything but Subversion is ruled out because:
- there is no offer to host that anywhere (for subversion, there is already svn.python.org)
- there is no support for converting a CVS repository (for subversion, there is cvs2svn)
I quickly discussed Subversion with a few friends. While some say Subversion is the most reasonable avenue nowadays, others them told me they found something more appealing than Subversion: http://www.venge.net/monotone/ The hosting paradigm is fairly different, and for a few weeks now, they have a CVS repository converter. In my very naive eyes, the centralised aspects of Python development are be better represented with Subversion. It is notable also that Subversion if more Python-friendly than Monotone, with its Lua-based scripting. I did not deepen why, but at first glance, Monotone does not seduce me. On the other hand, the two guys saying good about Monotone are well informed (and also well known), so I would not dismiss their opinion so lightly. So, it might be worth at least a quick look? :-) -- François Pinard http://pinard.progiciels-bpi.ca

[Raymond Hettinger]
The current release is 0.21 which suggests that it is not ready for primetime.
It suggests it, yes, and to me as well. On the other hand, there is a common prejudice that something requires many releases, or frequent releases, to be qualified as good. While it might be true on average, this is not necessarily true: some packages need not so many steps for becoming very usable, mature or stable. (Note that I'm not asserting anything about Monotone, here.) We should merely keep an open mind. -- François Pinard http://pinard.progiciels-bpi.ca

On Tue, 2005-08-02 at 09:06, François Pinard wrote:
It is true that some well designed/developed software becomes reliable very quicky. However, it still takes heavy use over time to prove that. You don't want to be the guy who finds out that this is not one of those bits of software. IMHO you need maturity for revision control software... you are relying on it for history. The only open source options worth considering for Python are CVS and SVN, and even SVN is questionable (see bdb backend issues). -- Donovan Baarda <abo@minkirri.apana.org.au>

[Donovan Baarda]
It is true that some well designed/developed software becomes reliable very quicky. However, it still takes heavy use over time to prove that.
There is wisdom in your say! :-) -- François Pinard http://pinard.progiciels-bpi.ca

On 8/2/05, "Martin v. Löwis" <martin@v.loewis.de> wrote:
Perforce is a commercial product, but it can be had for free for verified Open Source projects, which Python shouldn't have any problem with. There are other problems, like you have to renew the agreement every year, but it might be worth considering, given the fact that it's an excellent system.
We could host a Perforce repository just as easily, I would think.
- there is no support for converting a CVS repository (for subversion, there is cvs2svn)
I'd put $20 on the fact that cvs2svn will *not* work out of the box for converting the python repository. Just call it a hunch. In any case, the Perforce-supplied cvs2p4 should work at least as well. -- Nick

On Wed, Aug 03, 2005, Nicholas Bastin wrote:
Maybe. OTOH, I went to a CVS->SVN talk today at OSCON, and I'd be suspicious of claims that Python's repository is more difficult to convert than others that have successfully made the switch (such as KDE). I'd rather not rely on licensing of a closed-source system; one of the points made during the talk was that the Linux project had to scramble when they lost their Bitkeeper license (but they didn't switch to SVN because they wanted a distributed model -- one of things I appreciated about this talk was the lack of One True Way-ism). -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything.

"aahz" == aahz <aahz@pythoncraft.com> writes:
aahz> I'd rather not rely on licensing of a closed-source system; aahz> one of the points made during the talk was that the Linux aahz> project had to scramble when they lost their Bitkeeper aahz> license Python is unlikely to throw away its license in the same way, I should think. For additional security, you could try to negotiate a perpetual license on a particular version, or a license that required substantial notice (say, six months) for termination. I would imagine you could get them; the only reason for the vendor not to give them would be spite. The problem with both of those options is the one that Martin already pointed out: negotiation takes effort. There are several good open source alternatives, one of which (svn) is well-established and gets excellent reviews for those goals it sets itself, which happen to be solving the problems (as opposed to missing features) of CVS. Why spend effort on negotiating licenses and preparing for potential vendor relationship problems, unless there's acknowledged need for features svn doesn't provide? -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software.
participants (32)
-
"Martin v. Löwis"
-
Aahz
-
Anthony Baxter
-
Barry Warsaw
-
Benji York
-
Bob Ippolito
-
Brett Cannon
-
Charles Cazabon
-
Chris Lambacher
-
Christopher Petrilli
-
Daniel Berlin
-
Donovan Baarda
-
Fernando Perez
-
François Pinard
-
Fred L. Drake, Jr.
-
Fredrik Lundh
-
George V. Neville-Neil
-
Guido van Rossum
-
James Y Knight
-
Jeff Rush
-
Jim Fulton
-
Leif Hedstrom
-
M.-A. Lemburg
-
Michael Hudson
-
Nicholas Bastin
-
Nick Coghlan
-
Oleg Broytmann
-
Phillip J. Eby
-
Raymond Hettinger
-
Stephen J. Turnbull
-
Tim Peters
-
Trent Mick