Re: [python-committers] History lost in svn to hg conversion
![](https://secure.gravatar.com/avatar/db5f70d2f2520ef725839f046bdc32fb.jpg?s=120&d=mm&r=g)
Sorry, reply initially sent off-list.
-------- Message transféré -------- De: Antoine Pitrou <solipsis@pitrou.net> À: Alexander Belopolsky <alexander.belopolsky@gmail.com> Sujet: Re: [python-committers] History lost in svn to hg conversion Date: Mon, 25 Apr 2011 19:58:50 +0200
When I committed Lib/datetime.py to svn, I made sure that it was done in a way that preserved the history of that file going back to 2002.
Can you be more precise? How did you do that?
However, hg log of this file starts with my commit about a year ago. Is it possible to re-add the old history?
We cannot "add" any past history in any case.
The history of datetime.py is very helpful for understanding various design decisions implemented in that module.
I would advise to document design decisions in source code comments or in a separate file. Having to dig in VCS logs, or read many tracker entries, is IMO too tedious.
![](https://secure.gravatar.com/avatar/2c69ccb9eb83c7ef2ba155a850df68a9.jpg?s=120&d=mm&r=g)
On Apr 25, 2011, at 2:00 PM, Antoine Pitrou wrote:
The history of datetime.py is very helpful for understanding various design decisions implemented in that module.
I would advise to document design decisions in source code comments or in a separate file. Having to dig in VCS logs, or read many tracker entries, is IMO too tedious.
Great advice for the future, Antoine. Now can you help with the Hg issue?
I had understood that *all* history was going to be retained. Did I misremember or was I incorrectly advised?
regards Steve
Steve Holden steve@holdenweb.com
![](https://secure.gravatar.com/avatar/db5f70d2f2520ef725839f046bdc32fb.jpg?s=120&d=mm&r=g)
Le lundi 25 avril 2011 à 14:29 -0400, Steve Holden a écrit :
Great advice for the future, Antoine. Now can you help with the Hg issue?
As far as I understand it, this is not so much an hg issue than a migration issue.
I had understood that *all* history was going to be retained. Did I misremember or was I incorrectly advised?
All mainline history has been kept, as well as "active" feature branches (feature branches someone asked to be kept) (*). I don't know if Alexander's changesets are part of that, since he didn't precise where they were made in the SVN repos (if they were made in the sandbox, chances are they weren't).
(*) Look at "history management" for more information: http://www.python.org/dev/peps/pep-0385/#history-management
Also, there are some kind of things that are possible in SVN land (for example, copying some files from one branch/revision, other files from another one, etc.) which cannot be expressed in hg terms. The migration would have converted these changesets to a "dumber" form - without losing the actual contents of the files.
Regards
Antoine.
![](https://secure.gravatar.com/avatar/2c69ccb9eb83c7ef2ba155a850df68a9.jpg?s=120&d=mm&r=g)
On Apr 25, 2011, at 2:47 PM, Antoine Pitrou wrote:
Le lundi 25 avril 2011 à 14:29 -0400, Steve Holden a écrit :
Great advice for the future, Antoine. Now can you help with the Hg issue?
As far as I understand it, this is not so much an hg issue than a migration issue.
I had understood that *all* history was going to be retained. Did I misremember or was I incorrectly advised?
All mainline history has been kept, as well as "active" feature branches (feature branches someone asked to be kept) (*). I don't know if Alexander's changesets are part of that, since he didn't precise where they were made in the SVN repos (if they were made in the sandbox, chances are they weren't).
(*) Look at "history management" for more information: http://www.python.org/dev/peps/pep-0385/#history-management
Also, there are some kind of things that are possible in SVN land (for example, copying some files from one branch/revision, other files from another one, etc.) which cannot be expressed in hg terms. The migration would have converted these changesets to a "dumber" form - without losing the actual contents of the files.
Thanks for your detailed comments and feedback. I was clearly over-simplifying the situation.
regards Steve
Steve Holden steve@holdenweb.com
![](https://secure.gravatar.com/avatar/d6b9415353e04ffa6de5a8f3aaea0553.jpg?s=120&d=mm&r=g)
Alexander's changesets are part of that, since he didn't precise where
For the benefit of people who are not native-English speakers and who wish to write literate English: The English word 'precise' is only an adjective, and not a verb, so the above does not work as an English sentence.
This mistake, which I have seen before, has an understandable reason. 'Precise' is derived (borrowed) from the French "pre'cis" which is at least a verb and noun. "Pre'cis" comes from the Latin 'praecisus' and 'praecidere'. Spanish has the same verb in the form 'precisar'. So native Romance speakers have a tendency to over-generalize the usage of this restricted English cognate. English speakers learning other languages so the same sort of thing. The closest English verbs to "pre'cis" are 'abstract', 'summarize', and 'specify'.
Terry
![](https://secure.gravatar.com/avatar/db5f70d2f2520ef725839f046bdc32fb.jpg?s=120&d=mm&r=g)
Le lundi 25 avril 2011 à 15:44 -0400, Terry Reedy a écrit :
Alexander's changesets are part of that, since he didn't precise where
For the benefit of people who are not native-English speakers and who wish to write literate English: The English word 'precise' is only an adjective, and not a verb, so the above does not work as an English sentence.
This mistake, which I have seen before, has an understandable reason. 'Precise' is derived (borrowed) from the French "pre'cis" which is at least a verb and noun. "Pre'cis" comes from the Latin 'praecisus' and 'praecidere'. Spanish has the same verb in the form 'precisar'. So native Romance speakers have a tendency to over-generalize the usage of this restricted English cognate. English speakers learning other languages so the same sort of thing. The closest English verbs to "pre'cis" are 'abstract', 'summarize', and 'specify'.
Thanks. "Specify" would do the trick :)
Regards
Antoine.
![](https://secure.gravatar.com/avatar/3acb8bae5a2b5a28f6fe522a4ea9b873.jpg?s=120&d=mm&r=g)
For the benefit of people who are not native-English speakers and who wish to write literate English: The English word 'precise' is only an adjective, and not a verb, so the above does not work as an English sentence.
This mistake, which I have seen before, has an understandable reason. 'Precise' is derived (borrowed) from the French "pre'cis" which is at least a verb and noun. "Pre'cis" comes from the Latin 'praecisus' and 'praecidere'. Spanish has the same verb in the form 'precisar'.
FWIW, we have "präzisieren", supposedly imported from the French word in the 19th century. Too bad the Englishmen failed to accept that import :-( The best the dictionaries come up with (besides "to specify") is "to state more precisely", "to render more precisely".
Regards, Martin
![](https://secure.gravatar.com/avatar/25ef0a6698317c91220d6a1a89543df3.jpg?s=120&d=mm&r=g)
On Mon, Apr 25, 2011 at 6:29 PM, "Martin v. Löwis" <martin@v.loewis.de> wrote:
For the benefit of people who are not native-English speakers and who wish to write literate English: The English word 'precise' is only an adjective, and not a verb, so the above does not work as an English sentence.
I find it peculiar that in international forums "literate English" is not always the most effective form of communication. I had no problem understanding what Antoine wrote. In fact, Russian, just as French and German, has a verb form of the word "precise".
I still appreciate Terry's and other native speakers' comments on English usage. As Edsger Dijkstra once wrote, "Besides a mathematical inclination, an exceptionally good mastery of one's native tongue is the most vital asset of a competent programmer."
http://www.cs.utexas.edu/users/EWD/transcriptions/EWD04xx/EWD498.html
This mistake, which I have seen before, has an understandable reason. 'Precise' is derived (borrowed) from the French "pre'cis" which is at least a verb and noun. "Pre'cis" comes from the Latin 'praecisus' and 'praecidere'. Spanish has the same verb in the form 'precisar'.
FWIW, we have "präzisieren", supposedly imported from the French word in the 19th century. Too bad the Englishmen failed to accept that import :-( The best the dictionaries come up with (besides "to specify") is "to state more precisely", "to render more precisely".
Regards, Martin
python-committers mailing list python-committers@python.org http://mail.python.org/mailman/listinfo/python-committers
![](https://secure.gravatar.com/avatar/66974ed16b5ad5293bac2d23005ff9a9.jpg?s=120&d=mm&r=g)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 25/04/11 21:44, Terry Reedy wrote:
This mistake, which I have seen before, has an understandable reason. 'Precise' is derived (borrowed) from the French "pre'cis" which is at
Thanks for the nice explanation. I find interesting that english, being so used to interchange verbs, adjetives and nouns, is so picky here :).
For instance, in Spanish we have a translation for "searcher", but not for "finder". It is a non-word in spanish :). English is usually so liberal, in comparison... :)
Don't ask me for the plural of "virus" in different lenguajes :-p.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTbYEBplgi5GaxT1NAQL44QP8DeRyJlRjTcoUeWGSpb1JlGNihK0pH7RP VenF5t9WlhzjiEyYhhtVsr0wNV9sg9TmWqDfgWkHozYsKrK7A5aydKUc1BxGkOkf GXVebe5uUL5ZR/BwGAYAURi+B0lpkCLlButi9XEA6W76kGrjF/UI+Y5RWU/b7aCQ EqYAVuQavks= =KOEH -----END PGP SIGNATURE-----
![](https://secure.gravatar.com/avatar/f3ba3ecffd20251d73749afbfa636786.jpg?s=120&d=mm&r=g)
On Tue, Apr 26, 2011 at 9:30 AM, Jesus Cea <jcea@jcea.es> wrote:
Thanks for the nice explanation. I find interesting that english, being so used to interchange verbs, adjetives and nouns, is so picky here :).
I have a T-shirt that says "English doesn't borrow from other languages. English follows other languages down dark alleys, knocks them down and goes through their pockets for loose grammar." (it's a paraphrase of an older quote, but I forget the original). When native English speakers view our language that way, I'm constantly amazed that non-native speakers manage to figure it out was well as they do :)
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
![](https://secure.gravatar.com/avatar/e2cb0083d4d61cebb1325281f88f3843.jpg?s=120&d=mm&r=g)
On Tue, Apr 26, 2011 at 02:01:02PM +1000, Nick Coghlan wrote:
I have a T-shirt that says "English doesn't borrow from other languages. English follows other languages down dark alleys, knocks them down and goes through their pockets for loose grammar." (it's a paraphrase of an older quote, but I forget the original). When native
The original is from 1990 on Usenet: http://en.wikipedia.org/wiki/James_Nicoll#.22The_Purity_of_the_English_Langu...
--amk
![](https://secure.gravatar.com/avatar/25ef0a6698317c91220d6a1a89543df3.jpg?s=120&d=mm&r=g)
On Mon, Apr 25, 2011 at 2:47 PM, Antoine Pitrou <solipsis@pitrou.net> wrote: ..
All mainline history has been kept, as well as "active" feature branches (feature branches someone asked to be kept) (*). I don't know if Alexander's changesets are part of that, since he didn't precise where they were made in the SVN repos (if they were made in the sandbox, chances are they weren't).
(*) Look at "history management" for more information: http://www.python.org/dev/peps/pep-0385/#history-management
It is possible that I did not pay enough attention during hg migration discussions, but I don't remember seeing any call for feature branches to be preserved. Did anyone ever post a list of feature branches to be dropped during hg migration? How would developers know that they would need to "opt-in" for their work to be preserved? Given the unforgiving nature of hg when it comes to altering history, I don't think sufficient notice was given when the decision to trim the history was made.
![](https://secure.gravatar.com/avatar/f9c4ab38a9ced1923ff1bf6e3553a029.jpg?s=120&d=mm&r=g)
On Mon, 25 Apr 2011 17:00:21 -0400, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
On Mon, Apr 25, 2011 at 2:47 PM, Antoine Pitrou <solipsis@pitrou.net> wrote: ..
All mainline history has been kept, as well as "active" feature branches (feature branches someone asked to be kept) (*). I don't know if Alexander's changesets are part of that, since he didn't precise where they were made in the SVN repos (if they were made in the sandbox, chances are they weren't).
(*) Look at "history management" for more information: http://www.python.org/dev/peps/pep-0385/#history-management
It is possible that I did not pay enough attention during hg migration discussions, but I don't remember seeing any call for feature branches to be preserved. Did anyone ever post a list of feature branches to be dropped during hg migration? How would developers know that they would need to "opt-in" for their work to be preserved? Given the unforgiving nature of hg when it comes to altering history, I don't think sufficient notice was given when the decision to trim the history was made.
I remember a (brief) discussion and at least one call for nominating branches to preserve (I believe there was more than one call), which included a list that the poster (Antoine? Djirkan?) was planning to keep and the kinds of things he was planning to drop. I think it happened on python-dev rather than here, though. (Since I didn't have any branches at the time I didn't pay much attention to it.) I'm pretty sure it was mentioned again just pre-conversion, but I don't think any details were given at that time.
-- R. David Murray http://www.bitdance.com
![](https://secure.gravatar.com/avatar/db5f70d2f2520ef725839f046bdc32fb.jpg?s=120&d=mm&r=g)
Le lundi 25 avril 2011 à 17:43 -0400, R. David Murray a écrit :
On Mon, 25 Apr 2011 17:00:21 -0400, Alexander Belopolsky <alexander.belopolsky@gmail.com> wrote:
On Mon, Apr 25, 2011 at 2:47 PM, Antoine Pitrou <solipsis@pitrou.net> wrote: ..
All mainline history has been kept, as well as "active" feature branches (feature branches someone asked to be kept) (*). I don't know if Alexander's changesets are part of that, since he didn't precise where they were made in the SVN repos (if they were made in the sandbox, chances are they weren't).
(*) Look at "history management" for more information: http://www.python.org/dev/peps/pep-0385/#history-management
It is possible that I did not pay enough attention during hg migration discussions, but I don't remember seeing any call for feature branches to be preserved. Did anyone ever post a list of feature branches to be dropped during hg migration? How would developers know that they would need to "opt-in" for their work to be preserved? Given the unforgiving nature of hg when it comes to altering history, I don't think sufficient notice was given when the decision to trim the history was made.
I remember a (brief) discussion and at least one call for nominating branches to preserve (I believe there was more than one call), which included a list that the poster (Antoine? Djirkan?) was planning to keep and the kinds of things he was planning to drop. I think it happened on python-dev rather than here, though.
A quick search found the following message by Dirkjan, but it is likely earlier messages on the subject had been posted too: http://mail.python.org/pipermail/python-dev/2009-July/090325.html
By the way, the "pymigr" repository where the "all-branches.txt" file is stored is now only accessible through the ssh:// URLs, as someone complained that unmangled e-mail addresses of former committers were given out by the Web UI (in the "author-map" file).
Regardless, since Alexander's previous work was in the sandbox repo (not the python repo), I don't think it would have been possible to integrate it during the hg migration.
Regards
Antoine.
![](https://secure.gravatar.com/avatar/3acb8bae5a2b5a28f6fe522a4ea9b873.jpg?s=120&d=mm&r=g)
Regardless, since Alexander's previous work was in the sandbox repo (not the python repo), I don't think it would have been possible to integrate it during the hg migration.
See the message I just sent - it's *not* a separate repository (else subversion would not be able to reconstruct the full history). So it might have been possible to deal with it, had it been noticed.
Regards, Martin
![](https://secure.gravatar.com/avatar/25ef0a6698317c91220d6a1a89543df3.jpg?s=120&d=mm&r=g)
On Mon, Apr 25, 2011 at 5:56 PM, Antoine Pitrou <solipsis@pitrou.net> wrote: ..
A quick search found the following message by Dirkjan, but it is likely earlier messages on the subject had been posted too: http://mail.python.org/pipermail/python-dev/2009-July/090325.html
No wonder I missed that. I assume you are referring to this part of a multi-page post:
"""
- Get agreement on branch strategy and branch processing (list of branches + proposed handling at http://hg.python.org/pymigr/file/tip/all-branches.txt) <--- PLEASE REVIEW """
Even now, this does not sound to me like "WARNING: We are going to drop substantial chunks of history during hg migration. If you want to see the history of work you did in your feature branches preserved, please speak up."
By the way, the "pymigr" repository where the "all-branches.txt" file is stored is now only accessible through the ssh:// URLs, as someone complained that unmangled e-mail addresses of former committers were given out by the Web UI (in the "author-map" file).
How do I access this file now? I tried
$ hg cat ssh://hg@hg.python.org/pymigr/file/tip/all-branches.txt ssh:/hg@hg.python.org/pymigr/file/tip/all-branches.txt: No such file in rev 39047f8bd1d1
Regardless, since Alexander's previous work was in the sandbox repo (not the python repo),
See issue7989 for the details on how Lib/datetime.py was developed.
I don't think it would have been possible to integrate it during the hg migration.
Since I understand that the current plan is to preserve read-only SVN repository indefinitely, I don't think anything needs to be done other than making tracker smarter a Martin suggested. However, if maintaining an SVN server becomes a burden, maybe the complete SVN history should be converted to an Hg instance using some lossless process.
![](https://secure.gravatar.com/avatar/db5f70d2f2520ef725839f046bdc32fb.jpg?s=120&d=mm&r=g)
Le lundi 25 avril 2011 à 18:25 -0400, Alexander Belopolsky a écrit :
On Mon, Apr 25, 2011 at 5:56 PM, Antoine Pitrou <solipsis@pitrou.net> wrote: ..
A quick search found the following message by Dirkjan, but it is likely earlier messages on the subject had been posted too: http://mail.python.org/pipermail/python-dev/2009-July/090325.html
No wonder I missed that. I assume you are referring to this part of a multi-page post:
Ok, I've found a later message: http://mail.python.org/pipermail/python-dev/2009-August/090959.html
How do I access this file now? I tried
$ hg cat ssh://hg@hg.python.org/pymigr/file/tip/all-branches.txt ssh:/hg@hg.python.org/pymigr/file/tip/all-branches.txt: No such file in rev 39047f8bd1d1
Apparently "hg cat" can't work on remote URLs. Just "hg clone ssh://hg@hg.python.org/pymigr" to get the whole repo (it's small).
Regards
Antoine.
![](https://secure.gravatar.com/avatar/25ef0a6698317c91220d6a1a89543df3.jpg?s=120&d=mm&r=g)
On Mon, Apr 25, 2011 at 6:33 PM, Antoine Pitrou <solipsis@pitrou.net> wrote: ..
Ok, I've found a later message: http://mail.python.org/pipermail/python-dev/2009-August/090959.html
This post precedes the creation of py3k-datetime branch, so no wonder that it was not mentioned. I wonder, though why sandbox was not mentioned. Tim Peters' work was in sandbox/datetime and I don't think it could be preserved without preserving the sandbox.
![](https://secure.gravatar.com/avatar/3acb8bae5a2b5a28f6fe522a4ea9b873.jpg?s=120&d=mm&r=g)
Can you be more precise? How did you do that?
See the example that he gave, e.g. r82065. The changes are in /sandbox/branches/py3k-datetime.
However, hg log of this file starts with my commit about a year ago. Is it possible to re-add the old history?
We cannot "add" any past history in any case.
I propose that the redirector (hg.../lookup) redirects to the subversion viewer if it can't find a matching hg revision.
Regards, Martin
![](https://secure.gravatar.com/avatar/2c69ccb9eb83c7ef2ba155a850df68a9.jpg?s=120&d=mm&r=g)
On Apr 25, 2011, at 2:57 PM, Martin v. Löwis wrote:
I propose that the redirector (hg.../lookup) redirects to the subversion viewer if it can't find a matching hg revision.
That sounds like an extremely user-friendly solution.
regards Steve
Steve Holden steve@holdenweb.com
![](https://secure.gravatar.com/avatar/25ef0a6698317c91220d6a1a89543df3.jpg?s=120&d=mm&r=g)
On Mon, Apr 25, 2011 at 2:57 PM, "Martin v. Löwis" <martin@v.loewis.de> wrote:
Can you be more precise? How did you do that?
See the example that he gave, e.g. r82065. The changes are in /sandbox/branches/py3k-datetime.
Martin mostly answered Antoine's question for me. You can get more details by comparing the history of Lib/datetime.py in hg and svn.
The annotated view is completely inaccurate as a result:
$ svn blame -v Lib/datetime.py | awk '{print $2}'| sort | uniq -c 664 alexander.belopolsky 2 bcannon 319 gvanrossum 1123 tim_one
$ hg blame -vu Lib/datetime.py | awk '{print $1,$2}'| sort | uniq -c 2112 Alexander Belopolsky
It looks like this history survived the cvs to svn migration, but was lost in migration to hg.
![](https://secure.gravatar.com/avatar/3acb8bae5a2b5a28f6fe522a4ea9b873.jpg?s=120&d=mm&r=g)
It looks like this history survived the cvs to svn migration, but was lost in migration to hg.
It's not there anymore, but it isn't lost, either - you can still get it from svn if you want to (and should still be able to do so twenty years from now if you care - just as you can still get the original CVS repository if you want to).
In any case, this part of the history has not been converted, and there is no way to fix it now - less so in Mercurial than there might have been in CVS. Changing the history would require to come up with completely new revision "numbers", which would be unrelated to the revision numbers in all the clones out there. So anybody pushing changes would bring back all these "incorrect" revisions. That's why people say that changing history is futile in DVCSs one the genie is out of the bottle.
So just accept that apparently, you wrote datetime.py all on your own :-)
If you wonder what precisely has gone wrong: you shouldn't have created a branch in /sandbox, but in python/branches. /sandbox shouldn't have been used for anything whose history is of any interest. The hg conversion didn't consider anything outside /python, so it couldn't track the copying of the file.
It might be that the svn-to-hg conversion algorithm could do better with files that are copied from within the same repository, but from outside the converted subtree. This is tricky to implement, though, since you can't give reasonable file path names to such files. OTOH, our approach to svn (i.e. multiple projects in one repository) is so uncommon that the Mercurial people are likely to ignore problems arising out of it.
Regards, Martin
participants (9)
-
"Martin v. Löwis"
-
A.M. Kuchling
-
Alexander Belopolsky
-
Antoine Pitrou
-
Jesus Cea
-
Nick Coghlan
-
R. David Murray
-
Steve Holden
-
Terry Reedy