Create a Mercurial repository with a branch per issue
Hi,
When a patch is attached to an issue, it is usually a patch for the last stable release or to trunk (2.7 or 3.2 trunk). The problem is that latter the patch doesn't apply to trunk anymore and the author have to update its patch. Sometimes, we ask to update a patch six months or one year after the creation of the patch. It is difficult to update big patches.
It would be nice to create a branch for each issue (or each attached to an issue?), so we can test the patch in the branch and work on the patch without having to update the branch to trunk. If the patch has many versions, we keep the history of the changes, and at the end, it is easier to update the patch to trunk (e.g. rebase the branch).
If the buildbots are able to checkout this branch: we can also test directly the patch and automatically post the result on the bug tracker. If it is a problem with security (a patch can execute arbitrary commands), this service may be reserved to core developer and the action would be manual.
I hope that it will help to manage issues like "Regexp 2.7 (modifications to current re 2.2.2)" (issue #2636) which has ~73 files attached, most of them are new versions of the huge regex patch, on an history of 2 years... But I am not sure that the author of the 73 files will use Mercurial, so we should so something to simplify the usage of Mercurial in this case.
I heard somewhere that some projects are already doing exactly that (create a branch per issue), but I don't remember which projects. We should learn from these projects.
Victor
(I prefer to discuss here because launching a possible flamewar on python-dev)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 12:52, Victor Stinner wrote:
It would be nice to create a branch for each issue (or each attached to an issue?)
Change "branch" to "temporal clone".
I support this, but implementation would be non trivial.
I have missed forever about being able to check a fix before "officially" committing it. Your request would provide it.
The problem is how to cope with security issues (a patch could execute arbitrary code in the buildbots) and how to change the buildbot reporting tools to NOT mix the results from different changeset.
Currently when I try to identify the changeset that introduces a bug, and I do a bisection search, "everybody" can see that "experiment" in the public buildbots. That is not very sensible. For instance, you can see "red" buildbots, when those failing builds are not for HEAD and so, not relevant to anybody else. I can sympathize with each core developer just committing a patch and getting a rush because she sees red buildbots... not related to the patch. More, she has to wait until I finish before she can see results relevant to her.
Rebasing is good before committing to the "real" repository, but anybody with a checkout clone will suffer. I guess the clone should be automatically destroyed as soon as the clone is merged back to the real repository.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUvsNJlgi5GaxT1NAQI2xQP/U+zdReBfUe8Fu8AmFZMd+3To+mQsP9Oe IleFmkZ896Nz1CeNzSGCYM3vsytU+c4kd0CN7R+y1mFFGbuZ4oHySf96k/FAMnyH gO9RjnkRhKYi6BPKYi9nPYMwPd+t07F4bKPewFjlwbCa9EULtkQ2i2qyXl1ozMmn Th1aP/deTIw= =S8JV -----END PGP SIGNATURE-----
Le vendredi 04 février 2011 à 13:08 +0100, Jesus Cea a écrit :
I support this, but implementation would be non trivial.
Yes, it is non trivial, but it would be helpful :-)
The problem is how to cope with security issues
As I wrote: if security matters, only core developers should be able to run buildbots on such branch.
how to change the buildbot reporting tools to NOT mix the results from different changeset.
Hey, the buildbots are already able to run on a specific SVN branch, it is not something new. This service is already reserved to code developer... Hum, I am not sure...
That is not very sensible. For instance, you can see "red" buildbots, when those failing builds are not for HEAD and so, not relevant to anybody else.
Yeah, it should be improved, but I can be fixed later.
Rebasing is good before committing to the "real" repository, but anybody with a checkout clone will suffer.
Why?
Victor
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 13:15, Victor Stinner wrote:
Rebasing is good before committing to the "real" repository, but anybody with a checkout clone will suffer.
Why?
Rebasing alters history. It messes with the "inmmutable" nature of history. Anybody with a clone must do it. If you don't do, when you resync, you will see a new head and your clone will be "different" of the one doing the rebase.
That is, suppose:
- -->1-->2-->3-->4
Now you rebase and collapse changesets 2-4, to get ready for merging with the real respository. You have:
- -->1-->5
in your repository.
Anybody that had cloned from you, when pull, will have in her repository:
- -->1-->2-->3-->4
\-->5
Two heads, "different" repository views. If this second person has a patch in his own repository, and you try to merge back, you will get again the original changeset list, two heads, etc.
As a general rule, rebase should be done ONLY in personal clones not shared by anybody. Or, in this particular case, a clone should be automatically destroyed after merging to main development line.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUv2EJlgi5GaxT1NAQIrAQP/ZQAVu+AxIC5TGiU8MP6XfXihyNdA+qwL vmmSRLXb8jifAKtbbyk9z8Ucn6rzQW5vbeNJStvt4hpPDM224CwMDgUu2XFBums+ j8e7SEUvXmNleiwmwUsDAEqFYWRw3/rUmZIUvt/FWhy1lhrgOyIwFObeDMxjYdSG 1JFJrM5DHUo= =yB4X -----END PGP SIGNATURE-----
On Fri, Feb 4, 2011 at 1:50 PM, Jesus Cea <jcea@jcea.es> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 13:15, Victor Stinner wrote:
Rebasing is good before committing to the "real" repository, but anybody with a checkout clone will suffer.
Why?
Rebasing alters history. It messes with the "inmmutable" nature of history. Anybody with a clone must do it. If you don't do, when you resync, you will see a new head and your clone will be "different" of the one doing the rebase.
That is, suppose:
- -->1-->2-->3-->4
Now you rebase and collapse changesets 2-4, to get ready for merging with the real respository. You have:
- -->1-->5
in your repository.
Anybody that had cloned from you, when pull, will have in her repository:
- -->1-->2-->3-->4
\-->5Two heads, "different" repository views. If this second person has a patch in his own repository, and you try to merge back, you will get again the original changeset list, two heads, etc.
As a general rule, rebase should be done ONLY in personal clones not shared by anybody. Or, in this particular case, a clone should be automatically destroyed after merging to main development line.
That's why I think it's much cleaner to work with mq to build a clean single-commit patch, even if a clone may be used for temporary states and sharing.
We are experiencing merge hell right now in Distutils2, as the contributor list grows, because of the way people work with clones. We are polluting history with a lot of "merge" commits because it's the most simple way to work w/ mercurial.
I don't want to fork this thread but I think we should set up a "mercurial good practice" guide somewhere for hg.python.org could be awesome, in particular since the number of indirect contributors is going to grow with Python under a DVCS
Cheers Tarek
-- Tarek Ziadé | http://ziade.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 14:31, Tarek Ziadé wrote:
As a general rule, rebase should be done ONLY in personal clones not shared by anybody. Or, in this particular case, a clone should be automatically destroyed after merging to main development line.
That's why I think it's much cleaner to work with mq to build a clean single-commit patch, even if a clone may be used for temporary states and sharing.
Well, MQ are not easily shared. If you don't share your clone anyway, you can freely commit to it and then collapse all your changes via rebase, before pushing.
We are experiencing merge hell right now in Distutils2, as the contributor list grows, because of the way people work with clones. We are polluting history with a lot of "merge" commits because it's the most simple way to work w/ mercurial.
I have tendency to commit to my clones constantly. I can do 100 commits per day, in my private clone. When pushing to the main repository, you have the option to push all the tiny changesets, or collapse all of them via rebase.
I am not sure what way is better. Keeping ALL the history would be interesting anyway, but most of my tiny commits would break the build (I commit everytime I stop for thinking, pee, whatever. It is like autosave in my text editor), so things like "bisect" would be difficult to use. And reviewing 200 stupid patches when patch n+20 undo buggy patch n+1, is not pleasant.
This is a social issue.
I don't want to fork this thread but I think we should set up a "mercurial good practice" guide somewhere for hg.python.org could be awesome, in particular since the number of indirect contributors is going to grow with Python under a DVCS
+1. But first we need the mercurial switchover to be done and to get some first hand experience before writing down the best practices *wiki* :).
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUwDyZlgi5GaxT1NAQJGoAP/bvWOUNoc5DKfjt3K3T2uMes8A8Agsawr RlbQmsBhlPDeaye4oe5zBgue2xIguhtVazGjd5jC9SOBYHSIVMTlpabdxoFoUisO eili9X8zHwdEVDhmfg7Gp8BeGMwS7mOx35WEY96Xbsw1WtKbTSP8J4WyZwMC9TW0 ZDXdNqDo9mE= =Fdwj -----END PGP SIGNATURE-----
On Fri, Feb 4, 2011 at 2:48 PM, Jesus Cea <jcea@jcea.es> wrote: ..
I am not sure what way is better. Keeping ALL the history would be interesting anyway, but most of my tiny commits would break the build (I commit everytime I stop for thinking, pee, whatever. It is like autosave in my text editor), so things like "bisect" would be difficult to use. And reviewing 200 stupid patches when patch n+20 undo buggy patch n+1, is not pleasant.
This is a social issue.
Let's take the use case of Python development since it's what we discuss here.
You are building a patch and submit it to reviews (in roundup or in a clone or retvield etc). You get feedback and you refine the patch.
I don't think you want to keep that history, e.g. "changing this and that in my patch after feedback from MrX about Y"
The main line history is what counts imo, not the history of how your patch was built, refactored etc. before it got merged.
I don't want to fork this thread but I think we should set up a "mercurial good practice" guide somewhere for hg.python.org could be awesome, in particular since the number of indirect contributors is going to grow with Python under a DVCS
+1. But first we need the mercurial switchover to be done and to get some first hand experience before writing down the best practices *wiki* :).
I'd rather have a best practice guide *now* and refine it later. e.g. "how to contribute to python via mercurial."
I think we have people here with a decent expertise of Mercurial that can come up w/ this, even if it's changed after.
Cheers Tarek
-- Tarek Ziadé | http://ziade.org
Tarek Ziadé wrote:
You are building a patch and submit it to reviews (in roundup or in a clone or retvield etc). You get feedback and you refine the patch.
I don't think you want to keep that history, e.g. "changing this and that in my patch after feedback from MrX about Y"
The main line history is what counts imo, not the history of how your patch was built, refactored etc. before it got merged.
+1
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Feb 04 2011)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Feb 04, 2011, at 04:35 PM, M.-A. Lemburg wrote:
Tarek Ziadé wrote:
You are building a patch and submit it to reviews (in roundup or in a clone or retvield etc). You get feedback and you refine the patch.
I don't think you want to keep that history, e.g. "changing this and that in my patch after feedback from MrX about Y"
The main line history is what counts imo, not the history of how your patch was built, refactored etc. before it got merged.
+1
I guess this is just a Mercurial quirk I'll have to get used to. It's not something I ever bother with in Bazaar because, while those intermediate commits are still part of the merged branch's history, it's generally hidden from you when you dump the log or bisect, unless you specifically request to see them.
I wholeheartedly agree with the suggestion that we need a best practices guide before the migration is operational, even if it changes over time. Let's get everyone starting on the same page.
-Barry
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 16:58, Barry Warsaw wrote:
I guess this is just a Mercurial quirk I'll have to get used to. It's not something I ever bother with in Bazaar because, while those intermediate commits are still part of the merged branch's history, it's generally hidden from you when you dump the log or bisect, unless you specifically request to see them.
""" $ hg help log [...]
- --follow-first only follow the first parent of merge changesets [...]
- -m --only-merges show only merges [...]
- -M --no-merges do not show merges [...] """
Would be interesting that "bisect" would be "improved" to follow only the first parent in a merge.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUxC8Jlgi5GaxT1NAQJ0tQP/dHk0izxHp2NLFMiTao5+bDdE5vADcj4+ aSyAZGCiw6ozLu/KZFTFKujcsnzshXwaHWacUk3RMVzUNctRVHSCnRwMnwII+SUL 8a4uMqY31Xc4nqzIycFjYeU7gSQGhbOq0z0FFU2LW1BChVQ8YUwQbUpeSN7pgopS 1enrdo6U9ls= =HilB -----END PGP SIGNATURE-----
On Fri, Feb 4, 2011 at 11:57 PM, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
I'd rather have a best practice guide *now* and refine it later. e.g. "how to contribute to python via mercurial."
I think we have people here with a decent expertise of Mercurial that can come up w/ this, even if it's changed after.
Yes, please. Having a best practices document *in advance* will help keep things productive, especially with the Pycon sprints coming up shortly after the anticipated switchover.
Aside from the basics (how to checkout, how to generate a patch file), things I'd personally like advice on: porting model for bug fixing
- how best to work with multiple branches to support the forward
- how to clean a history ready for commit
- how to set up a feature branch for collaborative development
Policy decisions on release management can be deferred for now, since they will be largely in the hands of the Python 3.3 Release Manager.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Le vendredi 04 février 2011 à 14:48 +0100, Jesus Cea a écrit :
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 14:31, Tarek Ziadé wrote:
As a general rule, rebase should be done ONLY in personal clones not shared by anybody. Or, in this particular case, a clone should be automatically destroyed after merging to main development line.
That's why I think it's much cleaner to work with mq to build a clean single-commit patch, even if a clone may be used for temporary states and sharing.
Well, MQ are not easily shared. If you don't share your clone anyway, you can freely commit to it and then collapse all your changes via rebase, before pushing.
About rebase: I forgot to mention that I don't publish my repository, so I am free to rebase it as much as I want :-)
Victor
On Fri, Feb 4, 2011 at 14:31, Tarek Ziadé <ziade.tarek@gmail.com> wrote:
That's why I think it's much cleaner to work with mq to build a clean single-commit patch, even if a clone may be used for temporary states and sharing.
We are experiencing merge hell right now in Distutils2, as the contributor list grows, because of the way people work with clones. We are polluting history with a lot of "merge" commits because it's the most simple way to work w/ mercurial.
Yeah, it seems like you might want to ask more people to use mq or rebase (either way works -- or just pull and update just before your commit).
IMO people should always rebase locally before committing, so that merges aren't generally required for that. Only if their work has gone really stale (some large refactoring landed in the mean time) it might be worth it to explicitly record the merge.
I think extra clones per issue could work for some larger issues that require lots of work, but I don't think the every-small-feature-on-a-branch model (as practiced in Twisted, IIRC) is particularly helpful. On the other hand, I do think it's helpful to publish changes as multiple commits (where the test suite passes at each separate stage of the larger change).
Cheers,
Dirkjan
On Feb 4, 2011, at 9:44 AM, Dirkjan Ochtman wrote:
I think extra clones per issue could work for some larger issues that require lots of work, but I don't think the every-small-feature-on-a-branch model (as practiced in Twisted, IIRC) is particularly helpful. On the other hand, I do think it's helpful to publish changes as multiple commits (where the test suite passes at each separate stage of the larger change).
+1
S
participants (8)
-
Barry Warsaw -
Dirkjan Ochtman -
Jesus Cea -
M.-A. Lemburg -
Nick Coghlan -
Steve Holden -
Tarek Ziadé -
Victor Stinner