Code Simplicity » Open Source Community, Simplified
I imagine many of you will have seen this, but it's worth over-posting to make sure everyone gets to see it who is interested.
http://www.codesimplicity.com/post/open-source-community-simplified/
regards Steve
On 02/02/2011 13:25, Steve Holden wrote:
I imagine many of you will have seen this, but it's worth over-posting to make sure everyone gets to see it who is interested.
http://www.codesimplicity.com/post/open-source-community-simplified/
Thanks Steve.
One issue it raises is the difficulties caused by freezing the trunk for releases. Instead they advocate creating the release branch at the point of the release candidate instead of freezing trunk.
There are issues I currently *can't* work on because trunk is frozen and would personally prefer to see us use the branch-on-release-candidate process.
All the best,
Michael Foord
regards Steve
PSF-Members mailing list PSF-Members@python.org http://mail.python.org/mailman/listinfo/psf-members
-- http://www.voidspace.org.uk/
May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
Great article, thank you Steve!
Le mercredi 02 février 2011 à 13:31 +0000, Michael Foord a écrit :
On 02/02/2011 13:25, Steve Holden wrote:
I imagine many of you will have seen this, but it's worth over-posting to make sure everyone gets to see it who is interested.
http://www.codesimplicity.com/post/open-source-community-simplified/
Thanks Steve.
One issue it raises is the difficulties caused by freezing the trunk for releases. Instead they advocate creating the release branch at the point of the release candidate instead of freezing trunk.
There are issues I currently *can't* work on because trunk is frozen and would personally prefer to see us use the branch-on-release-candidate process.
+11. This is one thing that makes me dread another release candidate. First, we did a survey of all our past developers who had left the project, asking them why they had left. This was just a free-form survey, allowing people to answer any way they wanted.
Should we make our own survey of past contributors?
Regards
Antoine.
On Feb 02, 2011, at 01:31 PM, Michael Foord wrote:
One issue it raises is the difficulties caused by freezing the trunk for releases. Instead they advocate creating the release branch at the point of the release candidate instead of freezing trunk. There are issues I currently *can't* work on because trunk is frozen and would personally prefer to see us use the branch-on-release-candidate process.
This is one of the primary problems solved by a dvcs. You can *always* and *easily* work on new features, publish them for comment and review by others, make continual progress regardless of the release status of the official branches, and easily track movement in those official branches.
Pycon 2011 will mark the second anniversary of Guido's pronouncement. If we do nothing else in Atlanta, can we *please* *please* *please* come away from there with the conversion operationally completed?
-Barry
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/02/11 17:47, Barry Warsaw wrote:
This is one of the primary problems solved by a dvcs. You can *always* and *easily* work on new features, publish them for comment and review by others, make continual progress regardless of the release status of the official branches, and easily track movement in those official branches.
That being true, I am interested in the buildbots. Publishing a private repository is not helpful, if you can't use the buildbots.
I guess the best workflow would be for the Release Manager to create a clone, keeping the development repository open while the RM checks the clone and "cherry pick" changesets from the development branch if needed.
When the release is complete, the clone can be tagged and merged back to the main development.
So, nobody has to wait...
Pycon 2011 will mark the second anniversary of Guido's pronouncement. If we do nothing else in Atlanta, can we *please* *please* *please* come away from there with the conversion operationally completed?
+inf!.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUmTwplgi5GaxT1NAQIRbQP9FoUx5bcuwqiBVQczC2QBD7ad6ikVwN5e uf7ghaXkMltzUSgC5Bf3q+3yB/JVkfQ+gIK3TCvlQE+Kh7FlsKRd0MeKK62JDXwD U5YhKpX7CPGc14pM4+rEqaFuh5i/pbXNE8idNPd/kfk02CLt/+KKEoBwXmomm2cg Q/wUBlPrQZo= =M8ad -----END PGP SIGNATURE-----
Le mercredi 02 février 2011 à 18:26 +0100, Jesus Cea a écrit :
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/02/11 17:47, Barry Warsaw wrote:
This is one of the primary problems solved by a dvcs. You can *always* and *easily* work on new features, publish them for comment and review by others, make continual progress regardless of the release status of the official branches, and easily track movement in those official branches.
That being true, I am interested in the buildbots. Publishing a private repository is not helpful, if you can't use the buildbots.
I guess the best workflow would be for the Release Manager to create a clone, keeping the development repository open while the RM checks the clone and "cherry pick" changesets from the development branch if needed.
That sounds like a full-time job for the poor Release Manager. We really need to merge changesets *ourselves*. It is irresponsible to expect someone else to do the grunt job of merging stuff.
Regards
Antoine.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/02/11 18:38, Antoine Pitrou wrote:
I guess the best workflow would be for the Release Manager to create a clone, keeping the development repository open while the RM checks the clone and "cherry pick" changesets from the development branch if needed.
That sounds like a full-time job for the poor Release Manager. We really need to merge changesets *ourselves*. It is irresponsible to expect someone else to do the grunt job of merging stuff.
The merge here is mostly automatic. In fact, if the RM doesn't change his/her clone at all, the merge is "null", even if devel repository has evolved a lot in the meantime.
The cost of the merge will be, usually, proportional to the "private" changesets done in the clone. If the only patches applied there are "backports" from the main development line (that is, backport to candidate release from the mainline), those "doesn't count" to the complexity.
Ideally, a release candidate clone would have very few patches, and most of them coming from the main development (or main branch line).
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUmgk5lgi5GaxT1NAQKOIQQAnNQv+kol8bTr8nP6lM/29MQn7tz37QZv uWo2jBajMgpFWujawLEzxdW2hELjwplR8LdZc90hBbzXW182ePDQJJOZWUxKJPTW cLfYhMHKL8/CVTvEsOLsUr38ylvSd0spgch/7pUTc6Or/Xa9F9QM++D3bcAx9ndZ SmtaylMbb+k= =mSBR -----END PGP SIGNATURE-----
Le mercredi 02 février 2011 à 19:21 +0100, Jesus Cea a écrit :
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/02/11 18:38, Antoine Pitrou wrote:
I guess the best workflow would be for the Release Manager to create a clone, keeping the development repository open while the RM checks the clone and "cherry pick" changesets from the development branch if needed.
That sounds like a full-time job for the poor Release Manager. We really need to merge changesets *ourselves*. It is irresponsible to expect someone else to do the grunt job of merging stuff.
The merge here is mostly automatic. In fact, if the RM doesn't change his/her clone at all, the merge is "null", even if devel repository has evolved a lot in the meantime.
By merge I meant the cherry picking operation itself ("svnmerge").
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/02/11 19:28, Antoine Pitrou wrote:
The merge here is mostly automatic. In fact, if the RM doesn't change his/her clone at all, the merge is "null", even if devel repository has evolved a lot in the meantime.
By merge I meant the cherry picking operation itself ("svnmerge").
To be concrete, how many patches went inside 2.7.0 after cutting the "rc1"?. Ideally, the answer should be "a handful".
AFAIK, there are issues when you cherry pick patches and then you try to merge back (same patch present in both lines, but no metadata saying so). But that seems an issue with current HG/cherrypick extension, not with an abstract mercurial inability. In fact, python could be a push to improve that in HG.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUmkz5lgi5GaxT1NAQI3HAP9GJ5bKP6LNkzve9j/VIJM8JDUNecEJ8w7 Sqz6WSG4SLan15kFF/IyMbZoCkGDdDqmm8Ut0WpV5tIjByt4SgIJeLk8FyBM4WgW rEbHocufkF8STL3B/skgODUd10jgnKK1DZtZfv/zYVWsKBlXq8P1pSUTN8EegWwp qlFYAP+slrA= =Ckcz -----END PGP SIGNATURE-----
Le mercredi 02 février 2011 à 19:39 +0100, Jesus Cea a écrit :
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/02/11 19:28, Antoine Pitrou wrote:
The merge here is mostly automatic. In fact, if the RM doesn't change his/her clone at all, the merge is "null", even if devel repository has evolved a lot in the meantime.
By merge I meant the cherry picking operation itself ("svnmerge").
To be concrete, how many patches went inside 2.7.0 after cutting the "rc1"?. Ideally, the answer should be "a handful".
I don't think we are talking about branching after rc1 but after beta1, so that the feature branch can continue receive non-bugfix patches. That's quite many changesets to review.
On Feb 2, 2011, at 1:47 PM, Antoine Pitrou wrote:
Le mercredi 02 février 2011 à 19:39 +0100, Jesus Cea a écrit :
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/02/11 19:28, Antoine Pitrou wrote:
The merge here is mostly automatic. In fact, if the RM doesn't change his/her clone at all, the merge is "null", even if devel repository has evolved a lot in the meantime.
By merge I meant the cherry picking operation itself ("svnmerge").
To be concrete, how many patches went inside 2.7.0 after cutting the "rc1"?. Ideally, the answer should be "a handful".
I don't think we are talking about branching after rc1 but after beta1, so that the feature branch can continue receive non-bugfix patches. That's quite many changesets to review. The paper I referenced talks about branching after tagging the release candidate. It seemed trivially obvious to me that you wouldn't want to branch while the feature set code is still being modified.
That way the only things that (might) need merging would be late-stage and hopefully minor fixes from the released branch to the main trunk. With release candidates of sufficiently high quality there should be nothing to do.
It seems so much simpler. To me, at least. But I could be missing something.
regards Steve
Le mercredi 02 février 2011 à 13:59 -0500, Steve Holden a écrit :
On Feb 2, 2011, at 1:47 PM, Antoine Pitrou wrote:
Le mercredi 02 février 2011 à 19:39 +0100, Jesus Cea a écrit :
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/02/11 19:28, Antoine Pitrou wrote:
The merge here is mostly automatic. In fact, if the RM doesn't change his/her clone at all, the merge is "null", even if devel repository has evolved a lot in the meantime.
By merge I meant the cherry picking operation itself ("svnmerge").
To be concrete, how many patches went inside 2.7.0 after cutting the "rc1"?. Ideally, the answer should be "a handful".
I don't think we are talking about branching after rc1 but after beta1, so that the feature branch can continue receive non-bugfix patches. That's quite many changesets to review. The paper I referenced talks about branching after tagging the release candidate. It seemed trivially obvious to me that you wouldn't want to branch while the feature set code is still being modified.
Ok, my bad. Perhaps they have different rules about what goes in between beta and rc, though? I don't know the Bugzilla project.
That way the only things that (might) need merging would be late-stage and hopefully minor fixes from the released branch to the main trunk. With release candidates of sufficiently high quality there should be nothing to do.
True.
Regards
Antoine.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/02/11 19:47, Antoine Pitrou wrote:
I don't think we are talking about branching after rc1 but after beta1, so that the feature branch can continue receive non-bugfix patches. That's quite many changesets to review.
A beta is like any other branch. The "canonical way" (TM) in mercurial is to write the patch for the "oldest" supported branch (in this case, the beta branch), and "up-port" it to the open devel repository.
So, RM could say something like "beta branch is here. Commit there only things that MUST be in 3.3.0. For general development, commit to "py3k"". Anybody can merge from "beta" *TO* "py3k", for "up porting".
The trick here is to patch the oldest supported version, and "up port" later, via mercurial merge.
Just exactly the reverse of current workflow.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUm8M5lgi5GaxT1NAQIgPAQAk4O26PQqlIi8S9S/3PVB09HinWTYJOhZ 0pD5ul6jkOO506ngZvjN6a652wsTq5CHscTnoyNHRRlq38Pn5Y6m4vZOYvIaH6P1 LGJ7b5bv97nrxSzHnfmMIyi7jGPbv7Jgv8otgov7UIJE3YeMtSjJrVZaG98Sv2rq Xdlv5DJwBb4= =f6/e -----END PGP SIGNATURE-----
On Feb 02, 2011, at 09:18 PM, Jesus Cea wrote:
On 02/02/11 19:47, Antoine Pitrou wrote:
I don't think we are talking about branching after rc1 but after beta1, so that the feature branch can continue receive non-bugfix patches. That's quite many changesets to review.
A beta is like any other branch. The "canonical way" (TM) in mercurial is to write the patch for the "oldest" supported branch (in this case, the beta branch), and "up-port" it to the open devel repository.
So, RM could say something like "beta branch is here. Commit there only things that MUST be in 3.3.0. For general development, commit to "py3k"". Anybody can merge from "beta" *TO* "py3k", for "up porting".
The trick here is to patch the oldest supported version, and "up port" later, via mercurial merge.
Just exactly the reverse of current workflow.
And for good reason (IMO). It's often much less clear exactly how far back a specific patch should be committed when it's first being developed. It makes much more sense to me to fix a problem in the current development branch first, and then determine if, how, and where the patch should be back ported.
-Barry
On Thu, Feb 3, 2011 at 6:30 AM, Barry Warsaw barry@python.org wrote:
Just exactly the reverse of current workflow.
And for good reason (IMO). It's often much less clear exactly how far back a specific patch should be committed when it's first being developed. It makes much more sense to me to fix a problem in the current development branch first, and then determine if, how, and where the patch should be back ported.
Indeed. Mainline is the only branch where we *know* most patches need to be applied. It also means that people can contribute while maintaining only a single checkout, rather than necessarily needing all active versions.
I suspect this problem with the preferred DVCS workflow is going to cut fairly heavily into the number of bug fixes applied to the maintenance branches.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Feb 03, 2011, at 08:54 AM, Nick Coghlan wrote:
I suspect this problem with the preferred DVCS workflow is going to cut fairly heavily into the number of bug fixes applied to the maintenance branches.
I'd be really surprised if it *has* to be that way. Just how painful is it in Mercurial to apply to newest branch first and back port?
-Barry
Barry Warsaw wrote:
On Feb 03, 2011, at 08:54 AM, Nick Coghlan wrote:
I suspect this problem with the preferred DVCS workflow is going to cut fairly heavily into the number of bug fixes applied to the maintenance branches.
I'd be really surprised if it *has* to be that way. Just how painful is it in Mercurial to apply to newest branch first and back port?
Another question: Wouldn't it be easier to apply the patch to the mainline branch and the apply it to all release branches from there (and regardless of the order) ?
I don't think "up-porting" patches works well for Python, since you usually have to adapt new patches to make them fit older releases (not the other way around).
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Feb 03 2011)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Thu, Feb 3, 2011 at 9:16 AM, Barry Warsaw barry@python.org wrote:
On Feb 03, 2011, at 08:54 AM, Nick Coghlan wrote:
I suspect this problem with the preferred DVCS workflow is going to cut fairly heavily into the number of bug fixes applied to the maintenance branches.
I'd be really surprised if it *has* to be that way. Just how painful is it in Mercurial to apply to newest branch first and back port?
I have no idea. I just know that the first response from veteran hg users when asked how to implement our current workflow in Mercurial is "don't do that" (followed by reluctant explanations of tools you might use to do it, if you insisted on doing things "the wrong way").
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Feb 2, 2011, at 6:29 PM, Nick Coghlan wrote:
On Thu, Feb 3, 2011 at 9:16 AM, Barry Warsaw barry@python.org wrote:
On Feb 03, 2011, at 08:54 AM, Nick Coghlan wrote:
I suspect this problem with the preferred DVCS workflow is going to cut fairly heavily into the number of bug fixes applied to the maintenance branches.
I'd be really surprised if it *has* to be that way. Just how painful is it in Mercurial to apply to newest branch first and back port?
I have no idea. I just know that the first response from veteran hg users when asked how to implement our current workflow in Mercurial is "don't do that" (followed by reluctant explanations of tools you might use to do it, if you insisted on doing things "the wrong way").
Call me unadventurous, but i'd quite like to see how we get on ding things the *right* way first.
regards Steve
On Thu, Feb 3, 2011 at 9:35 AM, Steve Holden steve@holdenweb.com wrote:
Call me unadventurous, but i'd quite like to see how we get on ding things the *right* way first.
Call me pessimistic, but I'm predicting that quite a few people (including me) will continue their existing workflow, which is to work primarily on the main development branch. Without a reasonable backporting workflow, the end result will simply be stuff not being incorporated into maintenance branches.
The backport approach allows the backport to be handled by someone other than the original committer, and defers the decision on whether or not a change is suitable for a maintenance branch until a more reasonable time in the process (i.e. after you have tried it out on the developmental branch first). The recommended Mercurial way asks people to make the decision of which branch to target *far* too early in the process.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
(I should note that I don't consider this a dealbreaker for the switch to Mercurial - I'm just predicting a drop in maintenance patches for an extended period of time after the switch)
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 02/02/2011 23:35, Steve Holden wrote:
On Feb 2, 2011, at 6:29 PM, Nick Coghlan wrote:
On Thu, Feb 3, 2011 at 9:16 AM, Barry Warsawbarry@python.org wrote:
On Feb 03, 2011, at 08:54 AM, Nick Coghlan wrote:
I suspect this problem with the preferred DVCS workflow is going to cut fairly heavily into the number of bug fixes applied to the maintenance branches. I'd be really surprised if it *has* to be that way. Just how painful is it in Mercurial to apply to newest branch first and back port? I have no idea. I just know that the first response from veteran hg users when asked how to implement our current workflow in Mercurial is "don't do that" (followed by reluctant explanations of tools you might use to do it, if you insisted on doing things "the wrong way").
Call me unadventurous, but i'd quite like to see how we get on ding things the *right* way first.
Well, I'm with Nick. Doing development on main and backporting *some* fixes is the right way for a development process.
Michael
regards Steve
python-committers mailing list python-committers@python.org http://mail.python.org/mailman/listinfo/python-committers
-- http://www.voidspace.org.uk/
May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On Wed, Feb 2, 2011 at 7:22 PM, Michael Foord mfoord@python.org wrote:
On 02/02/2011 23:35, Steve Holden wrote:
On Feb 2, 2011, at 6:29 PM, Nick Coghlan wrote:
On Thu, Feb 3, 2011 at 9:16 AM, Barry Warsawbarry@python.org wrote:
On Feb 03, 2011, at 08:54 AM, Nick Coghlan wrote:
I suspect this problem with the preferred DVCS workflow is going to cut fairly heavily into the number of bug fixes applied to the maintenance branches.
I'd be really surprised if it *has* to be that way. Just how painful is it in Mercurial to apply to newest branch first and back port?
I have no idea. I just know that the first response from veteran hg users when asked how to implement our current workflow in Mercurial is "don't do that" (followed by reluctant explanations of tools you might use to do it, if you insisted on doing things "the wrong way").
Call me unadventurous, but i'd quite like to see how we get on ding things the *right* way first.
Call me naive, but isn't the planned hg workflow described in the PEP?
http://www.python.org/dev/peps/pep-0385/#proposed-workflow
No, I have not read it. :-)
Am 03.02.2011 00:16, schrieb Barry Warsaw:
On Feb 03, 2011, at 08:54 AM, Nick Coghlan wrote:
I suspect this problem with the preferred DVCS workflow is going to cut fairly heavily into the number of bug fixes applied to the maintenance branches.
I'd be really surprised if it *has* to be that way. Just how painful is it in Mercurial to apply to newest branch first and back port?
IIUC, you lose tracking if you let patches flow in the opposite direction (i.e. in the way it is currently).
With the new workflow, you can do "hg pull release32-maint" any time in the trunk, and it will integrate all pending patches. You don't have to specify any revision number there; it will just pick all missing changes from the maintenance branch and integrate them into the trunk.
If you let patches flow in the other direction, I understand you lose all support from hg wrt. history tracking. There may be ways to still do this through Mercurial commands (instead of generating a patch and manually applying it), but the backported patch will be different from the patch in the trunk, since it will have different parent versions (IIUC, the command is called "transplant").
I don't know whether hg keeps track of what patches have been transplanted, but I doubt it.
In any case, this discussion reiterates my point that this project (Mercurial migration) really needs a project lead, who then will also pronounce on usage policies. Otherwise, it's doomed to fail.
Regards, Martin
Indeed. Mainline is the only branch where we *know* most patches need to be applied. It also means that people can contribute while maintaining only a single checkout, rather than necessarily needing all active versions.
Notice that you'll automatically will have all active versions available, if PEP 385 is implemented. A single clone will have all maintenance branches as named branches inside, so integrating a bug fix should be a sequence like
hg update release32-maint patch hg commit hg update default hg merge release32-maint hg commit -m merged hg push
Sprinkle test runs into this to your taste.
Regards, Martin
On Sat, 05 Feb 2011 02:07:52 +0100, martin@v.loewis.de wrote:
Indeed. Mainline is the only branch where we *know* most patches need to be applied. It also means that people can contribute while maintaining only a single checkout, rather than necessarily needing all active versions.
Notice that you'll automatically will have all active versions available, if PEP 385 is implemented. A single clone will have all maintenance branches as named branches inside, so integrating a bug fix should be a sequence like
hg update release32-maint patch hg commit hg update default hg merge release32-maint hg commit -m merged hg push
Sprinkle test runs into this to your taste.
What about compiles? And perhaps even make distclean/configure? With our current workflow I have separate checkouts for 2.7, 3.1, and 3.2, and run make when I see a c file go by after an svn up (and make distclean if I see an update to a .in file, though I know that isn't always necessary).
Is there a way to translate that bit of the workflow to hg, or do I need to run make after each update, and make distclean if things fail to work as expected? Will running make after an update always do the right thing (ignoring those issues for which make distclean is currently needed)?
We really really really need some to document the expected best practices. Even if there isn't agreement among the hg veterans yet, someone should document *something* that we can iterate on to reach a preliminary consensus so us newbies have something to work from when the switchover is made.
I'm with Nick here; I don't have a project that uses hg, and until I do no amount of reading about it is going to do any good. And even after I'm going to need help...I tried using bzr for email6 and I *still* don't understand how to use it properly. Of course, nobody had written a best practices guide for me for my project, which is why I'm joining the chorus asking for one for Python :)
-- R. David Murray www.bitdance.com
Just to help settle this, I will write something and work with the appropriate people to get something worked out. I will most likely branch the devguide with an hg_transition branch and work in there so people can still publicly participate but not have it affect the published site.
On Sat, Feb 5, 2011 at 10:08, R. David Murray rdmurray@bitdance.com wrote:
On Sat, 05 Feb 2011 02:07:52 +0100, martin@v.loewis.de wrote:
Indeed. Mainline is the only branch where we *know* most patches need to be applied. It also means that people can contribute while maintaining only a single checkout, rather than necessarily needing all active versions.
Notice that you'll automatically will have all active versions available, if PEP 385 is implemented. A single clone will have all maintenance branches as named branches inside, so integrating a bug fix should be a sequence like
hg update release32-maint patch hg commit hg update default hg merge release32-maint hg commit -m merged hg push
Sprinkle test runs into this to your taste.
What about compiles? And perhaps even make distclean/configure? With our current workflow I have separate checkouts for 2.7, 3.1, and 3.2, and run make when I see a c file go by after an svn up (and make distclean if I see an update to a .in file, though I know that isn't always necessary).
Is there a way to translate that bit of the workflow to hg, or do I need to run make after each update, and make distclean if things fail to work as expected? Will running make after an update always do the right thing (ignoring those issues for which make distclean is currently needed)?
We really really really need some to document the expected best practices. Even if there isn't agreement among the hg veterans yet, someone should document *something* that we can iterate on to reach a preliminary consensus so us newbies have something to work from when the switchover is made.
I'm with Nick here; I don't have a project that uses hg, and until I do no amount of reading about it is going to do any good. And even after I'm going to need help...I tried using bzr for email6 and I *still* don't understand how to use it properly. Of course, nobody had written a best practices guide for me for my project, which is why I'm joining the chorus asking for one for Python :)
-- R. David Murray www.bitdance.com
python-committers mailing list python-committers@python.org http://mail.python.org/mailman/listinfo/python-committers
I started the branch, so any commits to the devguide mentiong hg, you can rest assured they are not going to the live site. =)
I will get a workflow written up and then email the people I know who are already heavy hg users (Antoine, Georg, and Dirkjan; if you want to be included in the discussion let me know) to read it over. Once we have agreed that the docs as written are accurate and we think they are best practices for what we want, I will let python-committers know and people can have a look to make sure everything is clear and easy to understand for when the transition occurs.
This will also give Nick a chance to make sure the dev FAQ is ready to go since he wanted it kept alive. =)
On Sat, Feb 5, 2011 at 11:44, Brett Cannon brett@python.org wrote:
Just to help settle this, I will write something and work with the appropriate people to get something worked out. I will most likely branch the devguide with an hg_transition branch and work in there so people can still publicly participate but not have it affect the published site.
On Sat, Feb 5, 2011 at 10:08, R. David Murray rdmurray@bitdance.com wrote:
On Sat, 05 Feb 2011 02:07:52 +0100, martin@v.loewis.de wrote:
Indeed. Mainline is the only branch where we *know* most patches need to be applied. It also means that people can contribute while maintaining only a single checkout, rather than necessarily needing all active versions.
Notice that you'll automatically will have all active versions available, if PEP 385 is implemented. A single clone will have all maintenance branches as named branches inside, so integrating a bug fix should be a sequence like
hg update release32-maint patch hg commit hg update default hg merge release32-maint hg commit -m merged hg push
Sprinkle test runs into this to your taste.
What about compiles? And perhaps even make distclean/configure? With our current workflow I have separate checkouts for 2.7, 3.1, and 3.2, and run make when I see a c file go by after an svn up (and make distclean if I see an update to a .in file, though I know that isn't always necessary).
Is there a way to translate that bit of the workflow to hg, or do I need to run make after each update, and make distclean if things fail to work as expected? Will running make after an update always do the right thing (ignoring those issues for which make distclean is currently needed)?
We really really really need some to document the expected best practices. Even if there isn't agreement among the hg veterans yet, someone should document *something* that we can iterate on to reach a preliminary consensus so us newbies have something to work from when the switchover is made.
I'm with Nick here; I don't have a project that uses hg, and until I do no amount of reading about it is going to do any good. And even after I'm going to need help...I tried using bzr for email6 and I *still* don't understand how to use it properly. Of course, nobody had written a best practices guide for me for my project, which is why I'm joining the chorus asking for one for Python :)
-- R. David Murray www.bitdance.com
python-committers mailing list python-committers@python.org http://mail.python.org/mailman/listinfo/python-committers
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 06/02/11 04:21, Brett Cannon wrote:
I will get a workflow written up and then email the people I know who are already heavy hg users (Antoine, Georg, and Dirkjan; if you want to be included in the discussion let me know) to read it over.
Could I read it too?. I have been using HG for the last three years, at least.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTU4feZlgi5GaxT1NAQIJmQP+IdVCtga5lt+qFxnGJZMXdi4nB2YIuc/4 ZIJypIFzm0W3WDzxG5THysI+h0CtmWgUgXnbL4Tshm68sAJpcsNY0FTXvtlHQtsG 7JvDRa/9vsC2sBEnHdhxm+HZusCgesd45cS8tFF1fRuO/WSGI38vjM46YNU2m1kH EHhkSzFbmeE= =dZQQ -----END PGP SIGNATURE-----
On Sun, Feb 6, 2011 at 2:11 PM, Jesus Cea jcea@jcea.es wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 06/02/11 04:21, Brett Cannon wrote:
I will get a workflow written up and then email the people I know who are already heavy hg users (Antoine, Georg, and Dirkjan; if you want to be included in the discussion let me know) to read it over.
Could I read it too?. I have been using HG for the last three years, at least.
I don't believe the formatted HTML is being published anywhere at this stage, but the raw ReST files are available from the Hg repository: http://hg.python.org/devguide/branches
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, Feb 6, 2011 at 1:21 PM, Brett Cannon brett@python.org wrote:
This will also give Nick a chance to make sure the dev FAQ is ready to go since he wanted it kept alive. =)
Sure, I can do that. If there is anything I can't figure out I'll leave a placeholder for someone with greater hg-fu to fill in.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
hg update release32-maint patch hg commit hg update default hg merge release32-maint hg commit -m merged hg push
Sprinkle test runs into this to your taste.
What about compiles?
Good question; this is beyond my hg-FU. Approaches that might be worth trying are:
have separate build directories, all referring to the same source directory. Python currently doesn't fully support that (i.e. there are cases when the source directory is changed), but people have been asking that this gets supported for a long time.
share repositories across multiple checkouts, using symlinks. Not sure whether this is supported.
Make local clones, one per branch, and then push across them. E.g. the flow might change to
cd 32 hg pull -u upstream patch build hg commit cd ../trunk hg pull 32 hg merge build hg commit -m merge hg push upstream
Notice that the push and the pull come from different clones: you would pull into the 3.2 clone, and eventually push from the trunk clone.
As for the "32" references above: the first one is a directory; the second is a named source (from ..hg/hgrc). Not sure whether "hg pull ../32" would work as well.
And perhaps even make distclean/configure? With our current workflow I have separate checkouts for 2.7, 3.1, and 3.2, and run make when I see a c file go by after an svn up (and make distclean if I see an update to a .in file, though I know that isn't always necessary).
At least the third one should be identical to your current setup, except that a) changes flow into the other direction, and b) there is no server communication to move patches across branches (whereas in subversion, you commit to the server, then svnmerge from the server).
Regards, Martin
On Sat, Feb 5, 2011 at 12:12 PM, "Martin v. Löwis" martin@v.loewis.dewrote:
- Make local clones, one per branch, and then push across them.
Like Victor, I have been using git to talk to the Python SVN servers for some time now. I have been using the strategy Martin proposes here with great success.
I seem to recall reading that hg makes cheap clones by using hard links for files that are the same in both clones (i.e., the common revision history before they branched).
-- Daniel Stutzbach
Le lundi 07 février 2011 à 09:25 -0800, Daniel Stutzbach a écrit :
On Sat, Feb 5, 2011 at 12:12 PM, "Martin v. Löwis" martin@v.loewis.de wrote: - Make local clones, one per branch, and then push across them.
Like Victor, I have been using git to talk to the Python SVN servers for some time now. I have been using the strategy Martin proposes here with great success.
I seem to recall reading that hg makes cheap clones by using hard links for files that are the same in both clones (i.e., the common revision history before they branched).
It does. And since CPython is fast to compile, it is quite a reasonable method. Anyone wanting to know more about hg should try to read (at least part of) http://hgbook.red-bean.com/ .
Also, I've finished converting (after Nick started it) the developer's FAQ to display instructions for hg. Reviewing it is welcome; I put a build here: http://potrou.net/hgdevguide/faq.html
Regards
Antoine.
On Tue, Feb 8, 2011 at 3:29 AM, Antoine Pitrou solipsis@pitrou.net wrote:
Also, I've finished converting (after Nick started it) the developer's FAQ to display instructions for hg. Reviewing it is welcome; I put a build here: http://potrou.net/hgdevguide/faq.html
Thanks for that - it definitely gives us a solid base to start from. If we get any repetitive questions after the transition happens, we can add the answers to the FAQ then.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/02/11 21:30, Barry Warsaw wrote:
Just exactly the reverse of current workflow.
And for good reason (IMO). It's often much less clear exactly how far back a specific patch should be committed when it's first being developed. It makes much more sense to me to fix a problem in the current development branch first, and then determine if, how, and where the patch should be back ported.
While I do agree with your point, when I take over a bug I test all supported versions to see which ones are buggy. Now you can choose to patch the latest and backport, or to patch the oldest and up-port. It is a choice. Mercurial is designed for up-porting.
In fact, "up-porting" is usually better, because you don't have to think if you must downport or not. Versión "n+1" is always a superset of versión "n". So you "up-port" *ALWAYS*, automatically (mostly) via merge.
If you choose the backporting way, you must take a decision about what to backport, for every single changeset. And god forbids you forget something in the process...
I agree you have a point when you write a new functionality for the latest version and LATER somebody decides that a particular changeset must be backported. Or a fix not planed for old versions is re-scheduled.
Would be very nice if the HG cherrypick extension would be more clever. Of if mercurial would recognize that a patch is already manually committed in both lines, and cope with that gracefully when merging. I am sure we are not alone with these issues, and I hope somebody is actually doing something about it.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUn4zZlgi5GaxT1NAQLitQP/cWcs8zuR/bsEkLLzM0BKLKnvt+U/5EI9 EGfqTz+SOkB9g6Yvrh35r5LcyXREQGJz9jtvOQFo+6L5UOlhpDWDwBiyFW37sdzi SVrnzsog63DqAOpGVrOqwSV+0/pwgl7Kst5z9w+i42PD/atZsq2z31ze2qWP4QNM nvoR2K8HSSc= =cQ6Z -----END PGP SIGNATURE-----
Am 03.02.2011 01:37, schrieb Jesus Cea:
On 02/02/11 21:30, Barry Warsaw wrote:
Just exactly the reverse of current workflow.
And for good reason (IMO). It's often much less clear exactly how far back a specific patch should be committed when it's first being developed. It makes much more sense to me to fix a problem in the current development branch first, and then determine if, how, and where the patch should be back ported.
While I do agree with your point, when I take over a bug I test all supported versions to see which ones are buggy. Now you can choose to patch the latest and backport, or to patch the oldest and up-port. It is a choice. Mercurial is designed for up-porting.
In fact, "up-porting" is usually better, because you don't have to think if you must downport or not. Versión "n+1" is always a superset of versión "n". So you "up-port" *ALWAYS*, automatically (mostly) via merge.
If you choose the backporting way, you must take a decision about what to backport, for every single changeset. And god forbids you forget something in the process...
That is the biggest problem.
If we do adopt the backporting approach (which I dislike as well), I will continue work on my extended transplant extension that tracks backported changesets and supports blocking, as I think it will be essential to have something like svnmerge in that case.
I agree you have a point when you write a new functionality for the latest version and LATER somebody decides that a particular changeset must be backported. Or a fix not planed for old versions is re-scheduled.
Would be very nice if the HG cherrypick extension would be more clever. Of if mercurial would recognize that a patch is already manually committed in both lines, and cope with that gracefully when merging.
It does, if the diff is the same. If the diff is different, how should Mercurial know that the two changesets are the same patch?
Georg
On Thu, Feb 3, 2011 at 10:37 AM, Jesus Cea jcea@jcea.es wrote:
In fact, "up-porting" is usually better, because you don't have to think if you must downport or not. Versión "n+1" is always a superset of versión "n". So you "up-port" *ALWAYS*, automatically (mostly) via merge.
As I said, I'm happy to roll with the proposed workflow as documented in the PEP, based on the advice from experts that Mercurial doesn't really suit our current work flow. I'm just noting that I expect the result will be fewer fixes in maintenance branches, since it is harder to divide the tasks of implementing a fix for the main line of development and applying that fix to the maintenance branches (unlike the way it often happens now).
The worse possibility is that fixes may be applied to maintenance branches and not to the main line of development, leading to regressions after upgrades (since the associated tests wouldn't be forward ported either).
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Le jeudi 03 février 2011 à 22:04 +1000, Nick Coghlan a écrit :
On Thu, Feb 3, 2011 at 10:37 AM, Jesus Cea jcea@jcea.es wrote:
In fact, "up-porting" is usually better, because you don't have to think if you must downport or not. Versión "n+1" is always a superset of versión "n". So you "up-port" *ALWAYS*, automatically (mostly) via merge.
As I said, I'm happy to roll with the proposed workflow as documented in the PEP, based on the advice from experts that Mercurial doesn't really suit our current work flow. I'm just noting that I expect the result will be fewer fixes in maintenance branches, since it is harder to divide the tasks of implementing a fix for the main line of development and applying that fix to the maintenance branches (unlike the way it often happens now).
I share the concern that it will make things harder for contributors (right now they only have to care about the feature branch).
Also, the fact that we have several bugfix branches (2.x, 3.x, not counting security-only branches) makes the whole thing much fuzzier.
Regards
Antoine.
On Thu, Feb 3, 2011 at 10:20 PM, Antoine Pitrou solipsis@pitrou.net wrote:
Le jeudi 03 février 2011 à 22:04 +1000, Nick Coghlan a écrit :
On Thu, Feb 3, 2011 at 10:37 AM, Jesus Cea jcea@jcea.es wrote:
In fact, "up-porting" is usually better, because you don't have to think if you must downport or not. Versión "n+1" is always a superset of versión "n". So you "up-port" *ALWAYS*, automatically (mostly) via merge.
As I said, I'm happy to roll with the proposed workflow as documented in the PEP, based on the advice from experts that Mercurial doesn't really suit our current work flow. I'm just noting that I expect the result will be fewer fixes in maintenance branches, since it is harder to divide the tasks of implementing a fix for the main line of development and applying that fix to the maintenance branches (unlike the way it often happens now).
I share the concern that it will make things harder for contributors (right now they only have to care about the feature branch).
One useful thing I have personally gotten out of this discussion is to identify the core reason I feel backporting is the "right" way to handle maintenance branches: it ensures that the *tests* that confirm a bug has been fixed are always applied to the *latest* branch first. This means that there should never be regressions of tested bug fixes, even after a major version upgrade. When fixes and their associated tests are applied to the oldest branch first, the only thing ensuring the forward port isn't forgotten is manual review, thus opening the door to regressions following a major version upgrade.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/02/11 15:07, Nick Coghlan wrote:
One useful thing I have personally gotten out of this discussion is to identify the core reason I feel backporting is the "right" way to handle maintenance branches: it ensures that the *tests* that confirm a bug has been fixed are always applied to the *latest* branch first. This means that there should never be regressions of tested bug fixes, even after a major version upgrade. When fixes and their associated tests are applied to the oldest branch first, the only thing ensuring the forward port isn't forgotten is manual review, thus opening the door to regressions following a major version upgrade.
I am not sure I understand correctly. It is 4:15AM in Spain and I need to sleep badly.
"Up-porting" CAN'T be forgotten because it is done "automagically" vía mercurial merges. That is the point...
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUtv4plgi5GaxT1NAQIoBQP+OATzNITrX9vsDLW3o01vH2VSbi1EN0+5 x+oYWyu8vkg+IenrF7TS3nfo+P72EC7AzqTCA5Nlhcc/HUsayrc0D73DVuUpeyac 5BsSDq2AbRPkQcMd8fhaAKxsWL5p4NIiYsqGO2Odhfu8ZBO+KKIdJZbsU2//cZAi hmr7/4tbiE4= =xoOv -----END PGP SIGNATURE-----
On Fri, Feb 4, 2011 at 1:17 PM, Jesus Cea jcea@jcea.es wrote:
"Up-porting" CAN'T be forgotten because it is done "automagically" vía mercurial merges. That is the point...
So developer A checks in a fix on 2.7, then gets sidetracked before forward porting it.
When does it make it to 3.2 or the main development branch?
Does everyone doing a forward merge from the maintenance branches run the risk of being landed with the task of doing a bulk merge of any forgotten forward ports before they can forward port the fix they're actually trying to implement?
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 10:54, Nick Coghlan wrote:
On Fri, Feb 4, 2011 at 1:17 PM, Jesus Cea jcea@jcea.es wrote:
"Up-porting" CAN'T be forgotten because it is done "automagically" vía mercurial merges. That is the point...
So developer A checks in a fix on 2.7, then gets sidetracked before forward porting it.
When does it make it to 3.2 or the main development branch?
Does everyone doing a forward merge from the maintenance branches run the risk of being landed with the task of doing a bulk merge of any forgotten forward ports before they can forward port the fix they're actually trying to implement?
This is a social problem, not a tech one. In my opinion, the "owner" of a fix can not be considered "done" until the patch is merged everywhere.
In fact, if all the branches were in the same repository (I do not advocate it), you can check it at push time.
Failing to comply would be as bad as committing a patch leaving all buildbots in red, or unbuildable code.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUvkVplgi5GaxT1NAQJlUAQAlX4V4H4kpSNn95fjNJVMgR6SwJzsYrUe SGeIuqMM5TALMxx/f9t5V1rE10JEV7nzNyKtdSnYDVom3RJL0fDNyvIsxrSzvTJM jJB+sTnMDI5cJ7m767B/Fz+xQ2Z7dpdjBJ3e/dZbqJmas4GkXWW3W5fjkU41Kvqu mqhpleBPpuo= =Y27E -----END PGP SIGNATURE-----
Am 04.02.2011 10:54, schrieb Nick Coghlan:
On Fri, Feb 4, 2011 at 1:17 PM, Jesus Cea jcea@jcea.es wrote:
"Up-porting" CAN'T be forgotten because it is done "automagically" vía mercurial merges. That is the point...
So developer A checks in a fix on 2.7, then gets sidetracked before forward porting it.
When does it make it to 3.2 or the main development branch?
Does everyone doing a forward merge from the maintenance branches run the risk of being landed with the task of doing a bulk merge of any forgotten forward ports before they can forward port the fix they're actually trying to implement?
You picked the wrong example, but: yes.
Your example is wrong, because porting between 2.7 and 3.2 won't be using any Mercurial support, and will rely on cherry-picking.
It's really the 3.2->trunk merging (and possibly 3.1->3.2->trunk merging) that uses the Mercurial merge tracking.
If somebody commits to 3.1, and forgets to forward-merge, the next time somebody commits to 3.1 and then wants to merge will have to merge both changes. In the 3.2->trunk case, this should be easy, since merges happen often, so there won't be much pending changes (and the original committer will still remember, and might be responsive to do the merge himself). In the 3.1->3.2 case, if merging is forgotten, it may take a while until somebody notices, in which case 3.2 will have evolved, so merging becomes more difficult.
I guess it would be possible to send a daily report on pending merges (no report if no pending merges), either to python-dev or to the original committer (it might be also possible to determine the original pusher, and send mail to him instead).
Regards, Martin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 03/02/11 13:04, Nick Coghlan wrote:
On Thu, Feb 3, 2011 at 10:37 AM, Jesus Cea jcea@jcea.es wrote:
In fact, "up-porting" is usually better, because you don't have to think if you must downport or not. Versión "n+1" is always a superset of versión "n". So you "up-port" *ALWAYS*, automatically (mostly) via merge.
As I said, I'm happy to roll with the proposed workflow as documented in the PEP, based on the advice from experts that Mercurial doesn't really suit our current work flow. I'm just noting that I expect the result will be fewer fixes in maintenance branches, since it is harder to divide the tasks of implementing a fix for the main line of development and applying that fix to the maintenance branches (unlike the way it often happens now).
The problem now is that patches in the development branch are "forgotten" and not backported when appropiate, and nobody cares about "blocking" versions to "not backport". Now running "svnmerge" to do a real merge is very risky, because the "blocking" is not actually maintained. People uses "svnmerge" backporting patch by patch, manually. Automatic mode is a disaster, because nobody cares about "blocking" in "svnmerge".
If we up-port, no patch is forgotten. The rule should be: "patches in n+1 are a SUPERSET of patches in n". With this rule, mercurial takes care of everything (a patch in n+1 can 'undo' a patch up-ported from n, if needed, keeping the rule).
The worse possibility is that fixes may be applied to maintenance branches and not to the main line of development, leading to regressions after upgrades (since the associated tests wouldn't be forward ported either).
That is not going to happen, because the mercurial merging between maintenance and development. This is the up-porting side, and merging should be automatic, you don't need to track anything, it is automagically done by mercurial.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUtuJ5lgi5GaxT1NAQL9OwP/ScnQ6Ahcgqdu73H01VGe4XPeJv8mDGss DokVIB/UrqnhtdeE+ky1uWTVSvcdcsClWsLMHPtW1KyTqWuf83fT9J1UWdXm5ol9 +QR1I8Vgot/WYt0XJ4ktP8VbN3SsUJoT+cJfEdGYI9I3Cc5UrXVOuOwpioANjyvE 5EBgj6ro1QY= =slHU -----END PGP SIGNATURE-----
Jesus Cea wrote:
On 03/02/11 13:04, Nick Coghlan wrote:
On Thu, Feb 3, 2011 at 10:37 AM, Jesus Cea jcea@jcea.es wrote:
In fact, "up-porting" is usually better, because you don't have to think if you must downport or not. Versión "n+1" is always a superset of versión "n". So you "up-port" *ALWAYS*, automatically (mostly) via merge.
As I said, I'm happy to roll with the proposed workflow as documented in the PEP, based on the advice from experts that Mercurial doesn't really suit our current work flow. I'm just noting that I expect the result will be fewer fixes in maintenance branches, since it is harder to divide the tasks of implementing a fix for the main line of development and applying that fix to the maintenance branches (unlike the way it often happens now).
The problem now is that patches in the development branch are "forgotten" and not backported when appropiate, and nobody cares about "blocking" versions to "not backport". Now running "svnmerge" to do a real merge is very risky, because the "blocking" is not actually maintained. People uses "svnmerge" backporting patch by patch, manually. Automatic mode is a disaster, because nobody cares about "blocking" in "svnmerge".
What's wrong about manually using svnmerge to backport those things that you actually do want to backport, rather than automatically merging back changes that you don't want to be backported ?
Note that blocking checkins from backports to certain branches adds more work to the developers, than simply not relying on automatic merges at all. The reason is that many new features will not get backported, only fixes to problems will.
If we up-port, no patch is forgotten. The rule should be: "patches in n+1 are a SUPERSET of patches in n". With this rule, mercurial takes care of everything (a patch in n+1 can 'undo' a patch up-ported from n, if needed, keeping the rule).
Mercurial is still a mystery to me, so please bare with me...
Is there a technical requirement for having to use this workflow:
- Patch -> Branch 3.1 -> Branch 3.2 -> Mainline
Why can't we use our current workflow:
- Patch -> Mainline (optionally) -> Branch 3.2 (optionally) -> Branch 3.1
Or perhaps a slightly modified one (if it suits Mercurial better):
- Patch -> Mainline (optionally) -> Branch 3.2 (optionally) -> Branch 3.1
(Legend to the diagrams: "->" means "merge patch into")
Note that 2) could just as well be done like this:
- Patch -> Mainline (optionally) -> Branch 3.1 (optionally) -> Branch 3.2
i.e. the order of the backports doesn't matter.
I don't understand why the developers have to follow the tools and not the tools the developers. If a tool doesn't fit, it's the wrong tool, IMHO, unless there are good reasons for changing you workflow because it allows the *workflow* to improve (rather than just permitting you to work with a specific tool). In other words: change is good if it improves things, not if it only changes things :-)
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Feb 04 2011)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 11:19, M.-A. Lemburg wrote:
I don't understand why the developers have to follow the tools and not the tools the developers.
Well, the tool is designed for a particular workflow. If you do something else, you are fighting the tool and not taking advantage of its bigger strenght.
I am advocating a particular workflow (reverse to current) because it is "natural" to mercurial and I find it sensible. That said, the fact is that current workflow can not be used comfy with current mercurial. That is a shameful limitation, I do agree.
Some people is working in solving this limitations. But we have to consider if using a "non natural" workflow would alienate new coders familiar with "natural" mercurial workflow. Since one of the objectives of HG migration is to ease external contributions, doing thing "the right way" would help too.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUvm/plgi5GaxT1NAQJwegP/f5N5PzUpwchEoh1ZeJGccuoGSVj7crTx 96hDNaQEcd9GG7V8IQ2+LXkjwF/HkCshkuyO2XMPFBVahzKOidlG5zEv4bAzDdvY lpiqo70sJYJt7doTwTxQx4v9lbuqYD0gHiv+3Sw06887gqyR05YqCdhcm8NgoNFg +YfLwmrwvu8= =iIIh -----END PGP SIGNATURE-----
The problem now is that patches in the development branch are "forgotten" and not backported when appropiate
Sorry, do you have real examples of this?
If we up-port, no patch is forgotten. The rule should be: "patches in n+1 are a SUPERSET of patches in n". With this rule, mercurial takes care of everything (a patch in n+1 can 'undo' a patch up-ported from n, if needed, keeping the rule).
That's a theoretical and IMO naïve point of view. In practice, there are many changesets that will not "up-port" cleanly and will need manual work. The work will not be much less than with down-porting.
Regards
Antoine.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 13:25, Antoine Pitrou wrote:
The problem now is that patches in the development branch are "forgotten" and not backported when appropiate
Sorry, do you have real examples of this?
None I can show just now :(. I do remember discussion about some patches not being backported because people were trying to speed up py3k.
If we up-port, no patch is forgotten. The rule should be: "patches in n+1 are a SUPERSET of patches in n". With this rule, mercurial takes care of everything (a patch in n+1 can 'undo' a patch up-ported from n, if needed, keeping the rule).
That's a theoretical and IMO naïve point of view. In practice, there are many changesets that will not "up-port" cleanly and will need manual work. The work will not be much less than with down-porting.
I don't understand how "backporting" is easy, and "upporting" is difficult. You have to potentially manually tweak in both cases.
Let's see.
We have these branches:
Old releases, still supported (security, critical fixes). These releases are mostly static. Any patch there would "naturally" need to be applied too in more modern branches. That is, "up porting".
Released and supported releases: These are bug fix only branches. Traffic should be low. Since only fixes go in, every changeset should be applied too to more modern branches. That is, "up porting".
Not yet, but about to be released branch. Currently this "freezes" the trunk, or patches are very restricted. For instance, "no new features". This status can be pretty lenghtly now, months. I suggest to "clone" the trunk here. This will be the beta/rc/final/maintenance clone. Trunk remains fully open. This clone has a restricting patch policy (no new features in the beginning, fixes only later). Any patch here, bug fix, documentation clean up, etc. should be "up ported" too to trunk. Trunk is open all the time.
Some fellows said that this would disincentivate contributions to the beta branch. In fact, coders would love to see their fixes/code/whatever they do being released in the next python version, three months for now, instead in the n+1 version, two years away. So I guess people would rush to commit to the beta clone (just like now!). But coders with long term plans can keep working in the trunk, freely. For instance, new stdlib modules.
The only issue in this plan would be trunk evolving so fast that up-porting changesets from the beta to trunk (merging) would be non automatic. The beta being a few months long would help. Anyway, we could have a social rule to "avoid" doing heavy incompatible work for a while, until the release. Anyway, we already have the issue now, because if trunk is wildy diverging, we already have issues when backporting from trunk to the maintenance branches. In fact this situation disincentivate "back porting", since it would be costly. If you "up port", you can't left behind any patch.
My guess is that most coders would concentrate in the beta clone, because they want their work released as fast as possible, and people working in trunk would be guys adding new functionality. I bet that overlapping changes would be pretty rare, during the beta/rc time.
Something to note is that mercurial merge is pretty clever, and can follow things like file renaming/moving, line number offset, etc. It is a "three way merge", far more powerful that simply trying to apply a patch to a file. Mercurial do a history evaluation and know how to modify your changeset to be applied cleanly, most of the time. For instance, if the file was modified in trunk to add a 2000 line patch, mercurial knows how to "offset" your changeset before trying to automatically apply.
PS: When a branch moves to a *temporal* restricted commit state, a new clone should be created, for people to keep working in the new minor release. That is, we are now in 3.2.0 release candidate. Patching is very restricted. If we had a 3.2.0 release candidate clone, MUST SHIP changes would be in the 3.2.0rc repository, but patches planned for 3.2.1 would be being already being committed the clone. When 3.2.0 is released and the branch is marked as "maintenance", the 3.2.1 clone would merge into the 3.2 maintenance and destroyed.
The point is... to never temporally freeze any repository.
The only frozen repositories would be dead ones.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUv/kplgi5GaxT1NAQLWHQQAhOlRmla8vfLFJekN7pa51rDex2aJ8a+X XtUucgBrzHjxvjCdlmMFexPvY52tq2/re4UkSso79ANzdn6B7wvaVLW+HZvcIrhF uWptP765gdgSIPb1bwuN8ToJDK77CjjXQvcO1gO50oTe6P+I6l703kwHWMC499al NA1x17si8xk= =4qaS -----END PGP SIGNATURE-----
On Fri, 04 Feb 2011 13:25:54 +0100, Antoine Pitrou solipsis@pitrou.net wrote:
The problem now is that patches in the development branch are "forgotten" and not backported when appropiate
Sorry, do you have real examples of this?
I can't point you at specific examples, but I remember reading issues where the comment was made that a fix had been "forgotten". Sometimes it gets into a point release, sometimes it gets forgotten until after the minor version goes into security mode.
I think this happened most when people were relying on other people doing mass merges, when the mass merges stopped happening, but I'm not sure.
If we up-port, no patch is forgotten. The rule should be: "patches in n+1 are a SUPERSET of patches in n". With this rule, mercurial takes care of everything (a patch in n+1 can 'undo' a patch up-ported from n, if needed, keeping the rule).
That's a theoretical and IMO naïve point of view. In practice, there are many changesets that will not "up-port" cleanly and will need manual work. The work will not be much less than with down-porting.
As Nick pointed it, it is much worse to forget to forward port a patch than to forget to back port it. We have had a few of the former even with our current workflow, and it is really embarrassing when it happens[*].
-- R. David Murray www.bitdance.com
[*] And I can reference you to one mistake, where the Mercurial-style workflow was used: at one point the version number of the email package was bumped in a release candidate branch but the bump wasn't forward ported to the mainline.
On Sat, Feb 5, 2011 at 12:30 AM, R. David Murray rdmurray@bitdance.com wrote:
That's a theoretical and IMO naïve point of view. In practice, there are many changesets that will not "up-port" cleanly and will need manual work. The work will not be much less than with down-porting.
As Nick pointed it, it is much worse to forget to forward port a patch than to forget to back port it. We have had a few of the former even with our current workflow, and it is really embarrassing when it happens[*].
On the other hand, if *any* forward port naturally picks up all the missed forward ports, then the Mercurial perspective starts to make more sense (especially if the merge is able to exploit the DAG in order to make fewer mistakes).
I'd definitely like to see some specific guidance from the Mercurial veterans on how they handle developing against multiple branches, though. From the sound of it, there's going to be a lot more hopping around between branches for different activities once it goes live.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, Feb 4, 2011 at 16:49, Nick Coghlan ncoghlan@gmail.com wrote:
On the other hand, if *any* forward port naturally picks up all the missed forward ports, then the Mercurial perspective starts to make more sense (especially if the merge is able to exploit the DAG in order to make fewer mistakes).
That's exactly the idea of the Mercurial way. You type hg merge <branch> and it will merge everything from the other branch that hasn't been merged yet (where both "blocking", in svnmerge terminology, and merging count as having been merged).
Cheers,
Dirkjan
Le vendredi 04 février 2011 à 17:25 +0100, Dirkjan Ochtman a écrit :
On Fri, Feb 4, 2011 at 16:49, Nick Coghlan ncoghlan@gmail.com wrote:
On the other hand, if *any* forward port naturally picks up all the missed forward ports, then the Mercurial perspective starts to make more sense (especially if the merge is able to exploit the DAG in order to make fewer mistakes).
That's exactly the idea of the Mercurial way. You type hg merge <branch> and it will merge everything from the other branch that hasn't been merged yet (where both "blocking", in svnmerge terminology, and merging count as having been merged).
How do you "block"?
On Fri, Feb 4, 2011 at 17:31, Antoine Pitrou solipsis@pitrou.net wrote:
That's exactly the idea of the Mercurial way. You type hg merge <branch> and it will merge everything from the other branch that hasn't been merged yet (where both "blocking", in svnmerge terminology, and merging count as having been merged).
How do you "block"?
By reverting changes during the merge (after hg merge, before hg commit).
Cheers,
Dirkjan
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 17:31, Antoine Pitrou wrote:
How do you "block"?
What patches in "3.2" you don't want in "3.3"?.
Remember the rule: "Patches in n+1 are a SUPERSET of patches in n". But a patch in "n+1" can undo a patch in "n", keeping this rule true always.
The usual approach is to do a merge just before the patch you don't want, and then a "null merge" just after the patch you don't want. "null merge" = a merge metadata update without actually bringing any new code.
I can't think any reason you want this, beside avoiding the "change 3.2 to 3.2.1 strings" to propagate to trunk.
In this particular situation, the merge would conflict (because "3.2->3.2.1" conflicts with "3.3", no "3.2" anywhere in trunk). Solving this conflict is trivial (just drop that change), and now you can retry merging again, this time successfully.
Ideally, when you push to the repository, you push your patches *AND* the merges, all in an atomic operation. So you don't move the merge burden to somebody else. I just wrote about this in a just sent email.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUxIPJlgi5GaxT1NAQK2IQP/d8+ILx4EukYf65m6OYn749TJ5jvejKgI 4m+O2CAnJIORiOo9jYhp12GUanCeziBDVcdVNx9sFRAHB2QNowVhxbJv+B50/WSF NXIlGxJaQcbCC4TlniJtnWezPzCr/cfNmxlUFv10qLErl2gEfMUCQMLC0/Zh9BDg f+8gpoaiBt0= =FVbK -----END PGP SIGNATURE-----
On 2/4/2011 1:41 PM, Jesus Cea wrote:
Remember the rule: "Patches in n+1 are a SUPERSET of patches in n".
This is simply not true of 2.7 versus 3.2. 3.x is a fork off of 2.x at about 2.6. There are many 2.7-only bugs whose patches should not be forward ported, either 'by hand' or by auto merge.
a patch in "n+1" can undo a patch in "n", keeping this rule true always.
The usual approach is to do a merge just before the patch you don't want, and then a "null merge" just after the patch you don't want. "null merge" = a merge metadata update without actually bringing any new code.
Seems like a lot of work to satisfy an artificial rule.
At one time, things were simple. There was an active maintenance branch and trunk. When trunk was released as 2.k final and a new, 2nd maintenance branch created, the older maintenance branch was released as 2.(k-1).m rc1 and (hopefully) a week later, 2.(k-1).m final and retired, thus restoring the number of maintenance branches to one. Whether bug patches went forwards or backwards between maintenance and trunk did not matter too much.
It was later decided to put maintenance branches into security-fix mode for a few years before freezing them, but in practice, this has meant little difference.
The problem now is that we have a second, increasingly divergent maintenance branch, and will for several years. Python is literally a two-headed giant (monster?-). So I do not think forcing it into a single-head model will work very well. (Note that from a temporal viewpoint, 2.7 -> 3.1 -> 3.2 is backwards, then forwards.)
As a user of 3.x (and not 2.x), I will work on bug fixes for 3.x. If the bug involves 2.7 also and the fix transports easily to 2.7, fine. If not, someone who cares about 2.7 can do the additional work.
Terry Jan Reedy
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 20:55, Terry Reedy wrote:
Remember the rule: "Patches in n+1 are a SUPERSET of patches in n".
This is simply not true of 2.7 versus 3.2. 3.x is a fork off of 2.x at about 2.6. There are many 2.7-only bugs whose patches should not be forward ported, either 'by hand' or by auto merge.
Would be nice to know how many patches, currently, are traveling back and forth between 2.7 and 3.x. I guess not many, and it will be less every day.
I remember some decision, time ago, about not messing with 2.7 workflow, because it is a dead branch. Do not pays to mess with it, tools, integration, whatever, if its air supply is restricted to minimum. I don't remember the details, sorry.
Anyway I can't tell how many patches applied to 2.7 are not in 3.x too. Checking the last commits in 2.7, it seems we have quite a big overlap, so "up porting" would be sensible.
Let's see:
""" changeset: 45834:7849bb687f46 branch: release27-maint tag: tip user: antoine.pitrou date: Fri Feb 04 21:17:53 2011 +0100 summary: [svn r88336] Merged revisions 88334 via svnmerge from
changeset: 45833:4e7d4386cb5c branch: release27-maint user: eric.araujo date: Thu Feb 03 01:12:18 2011 +0100 summary: [svn r88328] Merged revisions 86236,86240,86332,86340,87271,87273,87447 via svnmerge from
changeset: 45832:7e802a9f182c branch: release27-maint user: eric.araujo date: Wed Feb 02 18:03:38 2011 +0100 summary: [svn r88320] Fix typo: BadZipFile exists in 3.2+ only, not older versions.
changeset: 45831:9dc5b5c3e429 branch: release27-maint user: raymond.hettinger date: Wed Feb 02 09:37:11 2011 +0100 summary: [svn r88318] Issue 11089: Fix performance bug in ConfigParser that impaired its """
The merges are coming from somewhere. I assume they are backportings. They could be equally developed in 2.7 and then up-ported via mercurial merges.
svn r88320: It is a single line fix I can't see what is changed, actually. What is the change?. Judging ONLY from the comment, it seems it would be need to be applied to 3.1 branch. So it could be applied to 2.7, up-ported to 3.1 and possibly "null merged" in 3.2 (or mercurial could consider it is not needed at all, because the code already has that line!). This "null merge" would be a NOP when merging from 3.2 to py3k trunk.
I think this bug was introduced when backporting (to 2.7 and 3.1) from 3.2. It would be not there if the original patch were developed in 2.7 and up-ported via mercurial merge. Kudos to the backporting workflow :).
svn r88318: http://bugs.python.org/issue11089 - This is a fix to apply to 2.7, 3.1, 3.2 and py3k trunk. So, developing in 2.7 and up-porting would be appropiate. too.
So, the last few patches applied to 2.7.x were applied too to 3.1.x, 3.2.x. So, up-porting seems a sensible approach. I don't see your point. Do you want to check a few more patches, just to be sure that IN GENERAL, patches in "n+1" are, and SHOULD BE, a superset of patches in "n"?. My small survey is not a proof, but confirms that.
(Note that from a temporal viewpoint, 2.7 -> 3.1 -> 3.2 is backwards, then forwards.)
In my book, 3.x came from 2.6, but 3.1 and 2.7 were basically developed in sync, so considering 2.7 -> 3.1 -> 3.2 could not be the actual thing, but it is a realistic assumption, for the new (hypothetical) new workflow.
Look at issue 11089. It is applied in 2.7 and 3.1, but it can't be applied to 3.2 because we are in RC state now. Now, somebody *MUST* remember to apply this patch when the 3.2 branch in open again. That is a waste of mental energy for nothing.
If we don't close branches, and the beta/rc/final/maintenance is done in clones, the thing would be:
Somebody opens issue 11089.
Somebody test versions affected. 2.7, 3.1 and 3.2 RC are affected.
A patch is developed *FIRST* *FOR* 2.7. Tested. Reviewed. Committed to 2.7.
The patch propagate to 3.1 via a merge. This would be automatic, or almost, with mercurial.
The patch should be applied to 3.2, but it is in RC already, so it is applied to 3.2.1 clone, vía merge. Automatically done by mercurial. The 3.2.1 clone is done when the 3.2 clone moves to Release Candidate. 3.2.1 clone accumulates patches to be released in 3.2.1 and that can't be applied to 3.2 because it is in RC state.
The patch is propagated to trunk (from 3.2.1) via another mercurial merge.
This should be basically automatic, unless the patch need manual tweak when merging. In this particular case, the code seems pretty trivial to adjust, if necessary. This is not different that using svnmerge backward, with the difference of keeping the history right and using the mercurial machinery (three way merging and history tracking) to automatically solve a lot of small details like file renaming, offsets changings, whatever.
In the meantime, any patch to 3.2 clone (in RC state, so they should be only a handful) is propagated to 3.2.1 clone and trunk clone, via mercurial merge.
When 3.2(.0) is published, the 3.2 clone is open again. We (atomatically) merge 3.2.1 clone to 3.2, and destroy 3.2.1. When time come for 3.2.1 beta, we clone a 3.2.2 from 3.2 branch, and repeat.
In this schema, no developer has to WAIT for anything/anybody/BETA/RC/Whatever.
I can see that having a 3.2.1 clone while 3.2 is closed complicates things, but it has the huge advantage of not delaying any forward progress. Anyway, this part is optional. You can forget the 3.2.1 clone and simply forbid merging to 3.2 while is is in RC state. When 3.2 is released, 3.1 is merged to 3.2, bringing all delayed patches. And people waiting for 3.2 opening to commit new code, can do it now.
During 3.2 RC, you can merge to trunk from 3.1, for people working on trunk to get the fixes. When you merge from 3.1 to 3.2, you get the fixes there too. When merging 3.2 to trunk, later, Mercurial is clever enough to skip over the changesets merged vía 3.1 and coming again thru 3.2. This is something you can't do with SVN.
As a user of 3.x (and not 2.x), I will work on bug fixes for 3.x. If the bug involves 2.7 also and the fix transports easily to 2.7, fine. If not, someone who cares about 2.7 can do the additional work.
If what you are saying is that you don't want to waste your time porting your 3.x fixes to 2.7, of even worse, developing your patches in 2.7 and "up-porting to 3.x"... That is an entirely different topic.
With current python-dev commitment, any bug discovered in 3.1 that affect 2.7, should be solved in both. In fact, 2.7 fixing should more prioritary that 3.1, since 2.7.x is going to survive far longer that 3.1, 3.2, 3.3 and maybe 3.4...
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUyKhZlgi5GaxT1NAQL6NgP/SY6grnzbDL520mDmeeZv37UEkVleAU/z 6JTBoD8XLOJ3hqB1YkRbMTHIt5ySi8laOU5fl/4XltIuHwCfh5LFuFADJlnnno3q /RFwlu8ol3wCb0IcjvnfeTJCoQmsq2BNmnwbaZi+eOKKljVfCibVQrrydHl8J74w O6lbP7EtgJw= =5guh -----END PGP SIGNATURE-----
Look at issue 11089. It is applied in 2.7 and 3.1, but it can't be applied to 3.2 because we are in RC state now. Now, somebody *MUST* remember to apply this patch when the 3.2 branch in open again. That is a waste of mental energy for nothing.
That's because Raymond chose to break the usual workflow of fixing in 3.2 first. If he had waited for 3.2 to unfreeze first, there would be no "waste of mental energy for nothing".
That said, I don't think it's useful to discuss hg workflows at length. We certainly already did so in the past, and nothing came out (otherwise we wouldn't have this discussion again). Someone could sit and produce a written proposal evaluating the various possibilities, and we can iterate from that.
Regards
Antoine.
On Feb 4, 2011, at 6:36 PM, Antoine Pitrou wrote:
Look at issue 11089. It is applied in 2.7 and 3.1, but it can't be applied to 3.2 because we are in RC state now. Now, somebody *MUST* remember to apply this patch when the 3.2 branch in open again. That is a waste of mental energy for nothing.
That's because Raymond chose to break the usual workflow of fixing in 3.2 first. If he had waited for 3.2 to unfreeze first, there would be no "waste of mental energy for nothing".
That said, I don't think it's useful to discuss hg workflows at length. We certainly already did so in the past, and nothing came out (otherwise we wouldn't have this discussion again). Someone could sit and produce a written proposal evaluating the various possibilities, and we can iterate from that.
When known workflows emerge we can ask Brett to include them int he developer documentation (which I had hoped would include Hg -- hope springs eternal.
regards Steve
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 05/02/11 00:36, Antoine Pitrou wrote:
Look at issue 11089. It is applied in 2.7 and 3.1, but it can't be applied to 3.2 because we are in RC state now. Now, somebody *MUST* remember to apply this patch when the 3.2 branch in open again. That is a waste of mental energy for nothing.
That's because Raymond chose to break the usual workflow of fixing in 3.2 first. If he had waited for 3.2 to unfreeze first, there would be no "waste of mental energy for nothing".
In this case the issue should be open for the length of 3.2 RC state. In three weeks time your triage will vanish from memory and when 3.2 is open, "somebody" has to go back to this bug, refresh details already forgotten, write a patch for py3k trunk, "svnmerge" to 3.2, 3.1 and 2.7.
If I discover a bug, triage it and can write a patch NOW, when all the details are fresh in my mind, I should do. Waiting because a branch is "closed" is a waste. If I can commit NOW and I know that patch propagation will be automatic via mercurial merges when the branch is open again, I rather prefer to solve it now, commit, and move on.
People now must backport via "svnmerge", manually. Up-porting is automatic, via mercurial merges. You can't "leave a patch behind" because you forgot. It is simply not possible (you can do explicitly if a patch must not be up-ported, but that is the exception, not the rule).
That said, I don't think it's useful to discuss hg workflows at length. We certainly already did so in the past, and nothing came out (otherwise we wouldn't have this discussion again). Someone could sit and produce a written proposal evaluating the various possibilities, and we can iterate from that.
I do agree. I am getting the feeling that (some not small group of) python core devs are not familiar with mercurial at all. That is a bit scary, considering the two years migration patch (and counting) and that to discuss workflow you must know the abilities of the tools... Really scary :).
Do not consider my comment an attack. We are all busy people and change is painful. I sympathize, actually. But two years has been long enough...
As a developer I think that "up porting" is the way to go. And I think that the "non block" clone philosophy should be something to aim. Actually the main problem I see there is buildbot infraestructure: if you keep more clones open (2.7, 2.7.x, 3.1, 3.1.x, 3.2, 3.2.x, trunk), buildbots should support that. More complicated yet if we want the option to include arbitrary repositories around in the buildbot infraestructure for, let say, developing long term features.
2.7: maintenance branch. 2.7.x: clone of 2.7 to host patches for 2.7.x when 2.7 is closed (RC state) getting ready for releasing 2.7.(x-1).
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUyTHplgi5GaxT1NAQKNlwP/U7F2PrAJ9/+tKiWg3wemKlxiPDlb42dV mpn5vLNVzZfj6p/nLkNroJiPce9CBL5pfGukD1Gqr+DG3IMh++xEXHk8EtZOPvPP Q/UUjXpHXDSnXtjugXL1nq4329oXUSJZSTIrCHRD6At2ykBBV5xRWDJvplU6zFw5 aMVTFAG3xVI= =nmyY -----END PGP SIGNATURE-----
Le samedi 05 février 2011 à 01:00 +0100, Jesus Cea a écrit :
Someone could sit and produce a written proposal evaluating the various possibilities, and we can iterate from that.
I do agree.
Can you try writing said document?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 05/02/11 01:07, Antoine Pitrou wrote:
Le samedi 05 février 2011 à 01:00 +0100, Jesus Cea a écrit :
Someone could sit and produce a written proposal evaluating the various possibilities, and we can iterate from that.
I do agree.
Can you try writing said document?
I could write down what I already explained this last days. But I have the feeling that it is a losing position. Anyway, anybody has an alternative that doesn't fight against the strenghts of Mercurial?.
I would like to read the opinion of the people actually doing the HG migration work.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUylgplgi5GaxT1NAQIutAP/S2XtCWVeaaHxRjI0rWF607idcm5Jq4GG yGBlJ14wrC96ELgAodrbAYEzSmFxlfuYP5hmngBSVR0JfUvxXRVPSll7k6QJ8RC8 2STyu9+bser2n+/JX7wNZJby99Gs1P9Q6Pt+xAJn3kwBPPhYQ2Oz14R8+L39Hl+x 1bxRwSEis8E= =LqsQ -----END PGP SIGNATURE-----
Le samedi 05 février 2011 à 02:18 +0100, Jesus Cea a écrit :
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 05/02/11 01:07, Antoine Pitrou wrote:
Le samedi 05 février 2011 à 01:00 +0100, Jesus Cea a écrit :
Someone could sit and produce a written proposal evaluating the various possibilities, and we can iterate from that.
I do agree.
Can you try writing said document?
I could write down what I already explained this last days. But I have the feeling that it is a losing position.
Well, that would certainly be more constructive than sending 15+ messages on python-committers to explain us how we should work ;)
I would like to read the opinion of the people actually doing the HG migration work.
Both Dirkjan and Georg already answered in this thread.
Regards
Antoine.
On Sat, Feb 5, 2011 at 10:00 AM, Jesus Cea jcea@jcea.es wrote:
I do agree. I am getting the feeling that (some not small group of) python core devs are not familiar with mercurial at all. That is a bit scary, considering the two years migration patch (and counting) and that to discuss workflow you must know the abilities of the tools... Really scary :).
Work = CVS, Python = SVN, ??? = Hg. I don't believe you can genuinely learn any VCS until you use it for serious work, and there's simply nothing I work on currently that stresses the abilities of Hg. (The extent of my Hg usage is playing with the commit hooks and devguide repositories on hg.python.org).
I've read plenty about it, and have a reasonable idea of how it works, but a lot of the things I've read about the recommended workflows simply don't line up with the way we have historically done things. As I've said, I'm happy to roll with that, but I consider the onus to be on the champions of the Mercurial switchover to document the best workflows they can collectively come up with for the benefit of those of us that are Mercurial novices. What we have with SVN certainly has its problems, but it mostly works and we're used to it.
A wildly divergent 2.7 branch, with a 3.3 development branch, a 3.2 maintenance branch, 2.6 and 3.1 security branches and assorted feature branches is going to be plenty to be going on with, even before we get to the question of managing the 3.3 release in mid-to-late 2012.
Consider the day of the switchover, when SVN has gone read-only and Hg is opened up for pushes. What should I have checked out on my machine in advance in order to work on:
- PEPs
- The dev guide
- New 3.x features
- 3.x only bug fixes
- Bug fixes that also apply to 2.7
- Security fixes that apply to 3.1/2.6
- A feature-specific branch
I'm happy to be guided by the Hg veterans here, *but the Hg veterans need to be willing to document their advice in a common location and come to consensus on it*.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Am 04.02.2011 19:41, schrieb Jesus Cea:
On 04/02/11 17:31, Antoine Pitrou wrote:
How do you "block"?
What patches in "3.2" you don't want in "3.3"?.
It happens in more cases than you think.
I sometimes develop two versions of a patch, one for the maintenance branch which preserves backwards compatibility, and another version that fixes it in the right way. I think I regularly contributes two or three such changes to every feature release of Python. I think a few other committers have also written such changes over time.
Regards, Martin
On Feb 04, 2011, at 05:25 PM, Dirkjan Ochtman wrote:
On Fri, Feb 4, 2011 at 16:49, Nick Coghlan ncoghlan@gmail.com wrote:
On the other hand, if *any* forward port naturally picks up all the missed forward ports, then the Mercurial perspective starts to make more sense (especially if the merge is able to exploit the DAG in order to make fewer mistakes).
That's exactly the idea of the Mercurial way. You type hg merge <branch> and it will merge everything from the other branch that hasn't been merged yet (where both "blocking", in svnmerge terminology, and merging count as having been merged).
Doesn't that mean that the person doing the forward port will potentially have to review a lot more code than the fix they specifically want to make?
IOW, if I apply a patch to 3.1 and forget to forward port it to 3.2 and the trunk, then you also apply a patch to 3.1 and *do* the forward port, you could end up with conflicts resulting from my merge. That change might or might not make sense to you, but in a way, you shouldn't even have to worry about it because it interrupts the work you're doing on your patch.
What if my fix for 3.1 isn't applicable to 3.2 or the trunk?
Wouldn't it be better for you to have to only deal with your change, and then admonish me for not completing my patch by making sure it is correctly committed to all branches in which it applies?
-Barry
On Fri, Feb 4, 2011 at 17:36, Barry Warsaw barry@python.org wrote:
Doesn't that mean that the person doing the forward port will potentially have to review a lot more code than the fix they specifically want to make?
Potentially, yes. I would imagine in many cases (let's not count 2.x -> 3.x for now) the merge wouldn't be that hard, and most fixes should be easy to recognize as applicable.
But if not, this should incentivize people to forward-port their own changes, which is a good thing, IMO.
Cheers,
Dirkjan
On Feb 04, 2011, at 05:46 PM, Dirkjan Ochtman wrote:
But if not, this should incentivize people to forward-port their own changes, which is a good thing, IMO.
Sure. I guess my question is, what do *you* do in that case? Are you blocked because I didn't do my job properly? Can you tell your merge to ignore my change so you can keep making progress, complete your patch, and send me a nastygram to finish my work? :)
-Barry
On Fri, Feb 4, 2011 at 17:51, Barry Warsaw barry@python.org wrote:
Sure. I guess my question is, what do *you* do in that case? Are you blocked because I didn't do my job properly? Can you tell your merge to ignore my change so you can keep making progress, complete your patch, and send me a nastygram to finish my work? :)
You could:
- merge the other's stuff and tell them to check if you merged it correctly
- don't merge the other's stuff and tell them to patch it in again
- backout the other's stuff, proceed as planned (optionally re-commit other's stuff)
In general: sure, there are all kinds of ways to mangle history and make it work.
Cheers,
Dirkjan
Am 04.02.2011 18:03, schrieb Dirkjan Ochtman:
On Fri, Feb 4, 2011 at 17:51, Barry Warsaw barry@python.org wrote:
Sure. I guess my question is, what do *you* do in that case? Are you blocked because I didn't do my job properly? Can you tell your merge to ignore my change so you can keep making progress, complete your patch, and send me a nastygram to finish my work? :)
You could:
- merge the other's stuff and tell them to check if you merged it correctly
- don't merge the other's stuff and tell them to patch it in again
- backout the other's stuff, proceed as planned (optionally re-commit other's stuff)
In general: sure, there are all kinds of ways to mangle history and make it work.
I think the easiest way would be to
- base your changes onto a revision that doesn't contain the other's unmerged change, and merge that one.
Georg
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 17:51, Barry Warsaw wrote:
Sure. I guess my question is, what do *you* do in that case? Are you blocked because I didn't do my job properly? Can you tell your merge to ignore my change so you can keep making progress, complete your patch, and send me a nastygram to finish my work? :)
You can, but is not trivial neither automatic.
There are quite a few strategies. For instance, I can move my patch BEFORE your patch (vía rebase), creating a new head, and merging my head with trunk. My head is solved. Yours is still hanging around.
Or you can force this via a push "hook": no heads left behind! :-)
I just wrote an email about this.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUxJxJlgi5GaxT1NAQJejwQAmrR9PuJmvL/FcKHYirsKk5A0gvoebG8v Sd7baUi3dfcWR4HEsJa5SLc1knY+f/wlFyDzd0fYx102+QMgYUiipp8n9r3+yXfN PSR1tREix93Zbb2dlfkSQe6L0+pfz6hXZRNHMBCgN9EFpIspcD05/yZVISwsBHyG 80EKgV1+FGg= =2D3a -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 16:49, Nick Coghlan wrote:
On the other hand, if *any* forward port naturally picks up all the missed forward ports, then the Mercurial perspective starts to make more sense (especially if the merge is able to exploit the DAG in order to make fewer mistakes).
A mercurial merge "brings" "automatically" (if can be done) any patch in a side, to the other side. Basically "joins" the two sides making a single one.
clone 1:
A->B->C->D
Clone 2:
A->E->F->G->H
When you merge you have:
A->B->C->D--------->I \ / \->E->F->G->H-/
Code in "I" contains all "B", "C", "D", "E", "F", "G" and "H".
This merging is "automatic" (if there are no conflicts!). That is the magic in Mercurial. The merge machinery. It is a huge improvement over subversion, even in 1.6.*.
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUw/eZlgi5GaxT1NAQJjdwP/dF6H2K9sdOUrXJ9jwJxwwgZzrGDQfM0p OG1sKTUCnn7WKC0II2Ep56BdVMC7HvF5PT9fQDOMKwyDz2WN02YgTIxvrHwh4wHz 90zXvDB7VslH5k1shXm7v++mqgutWDMOUHRJ73ZjKuS6rSnf69m4yCN7PK0stGcX 1uHaPiMJI+8= =p9Zg -----END PGP SIGNATURE-----
Am 04.02.2011 13:25, schrieb Antoine Pitrou:
The problem now is that patches in the development branch are "forgotten" and not backported when appropiate
Sorry, do you have real examples of this?
If you really want to know, I can dig them out. I'm fairly sure I owe 10 or 20 backports to somebody. Some of them may have been backported by somebody else meanwhile.
In many cases, the bug tracker will indicate that a backport is still pending, but sometimes, it's really just forgotten "for good".
I personally don't see that as a problem. If somebody really wants a certain bug fixed, they can open a new bug report and request a backport.
If we up-port, no patch is forgotten. The rule should be: "patches in n+1 are a SUPERSET of patches in n". With this rule, mercurial takes care of everything (a patch in n+1 can 'undo' a patch up-ported from n, if needed, keeping the rule).
That's a theoretical and IMO naïve point of view. In practice, there are many changesets that will not "up-port" cleanly and will need manual work. The work will not be much less than with down-porting.
I'm really with Nick here - I don't view *that* as the real problem with the "natural" workflow. Instead, I also predict that some committers just won't bother with backporting (even if they currently do backport in svn). So with the new workflow, even more patches will be forgotten wrt. backporting.
It would still be possible that somebody else backports for them, but that will be more tedious than it is now with svnmerge (since you first have to transplant, and then merge back the transplanted patch, which should come out as a no-op - but the merge must be recorded).
Regards, Martin
On Feb 04, 2011, at 04:10 AM, Jesus Cea wrote:
If we up-port, no patch is forgotten. The rule should be: "patches in n+1 are a SUPERSET of patches in n". With this rule, mercurial takes care of everything (a patch in n+1 can 'undo' a patch up-ported from n, if needed, keeping the rule).
I'm not totally convinced, but I'm willing to suspend my disbelieve and embrace The Mercurial Way to see how well it works in practice. ;)
The worse possibility is that fixes may be applied to maintenance branches and not to the main line of development, leading to regressions after upgrades (since the associated tests wouldn't be forward ported either).
That is not going to happen, because the mercurial merging between maintenance and development. This is the up-porting side, and merging should be automatic, you don't need to track anything, it is automagically done by mercurial.
Given that this workflow is a social one, encouraged but not imposed by the technology, how will we respond when things are done The Wrong Way? What are the effects if someone forgets and commits a patch to trunk first? Have we hosed the branches or is it just a PITA to recover?
-Barry
Am 04.02.2011 17:03, schrieb Barry Warsaw:
The worse possibility is that fixes may be applied to maintenance branches and not to the main line of development, leading to regressions after upgrades (since the associated tests wouldn't be forward ported either).
That is not going to happen, because the mercurial merging between maintenance and development. This is the up-porting side, and merging should be automatic, you don't need to track anything, it is automagically done by mercurial.
Given that this workflow is a social one, encouraged but not imposed by the technology, how will we respond when things are done The Wrong Way? What are the effects if someone forgets and commits a patch to trunk first? Have we hosed the branches or is it just a PITA to recover?
The patch will be committed to maintenance separately and thus occur twice in the changeset history.
At the next merging of maintenance into trunk, Mercurial will notice that both changesets have in the same result and it's fine. If the maintenance commit is different to the trunk commit, there will be a merge conflict to be resolved by ignoring the changes from maintenance.
Georg
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/02/11 17:03, Barry Warsaw wrote:
Given that this workflow is a social one, encouraged but not imposed by the technology, how will we respond when things are done The Wrong Way? What are the effects if someone forgets and commits a patch to trunk first? Have we hosed the branches or is it just a PITA to recover?
The obvious approach would be to use "transplant" to copy the patch to the "old" clone. If the patch in both places is the same, mercurial recognize the fact. If they are not, you have a false conflict, to be resolved in the merge (usually you do this just now, that you have the details fresh in mind, and if you don't do, somebody has to do it).
Other option is to export the patch from the "new" clone, backout the patch ("hg backout" command), apply the patch to the "old" clone and then do a regular merge.
As said, this is a social problem. For instance, if I write a patch to "new" and do not backport, somebody else can do it "eventually". This is practical, but you can "forget" or decide you don't want to spend your time maintaining, let's say, 2.7.*.
But using "up-porting", when somebody does a merge from "old" to "new", the operation brings all patches from "old". If the patches doesn't apply cleanly (mercurial merge machinery is clever, but not everything can be done automatically), the guy doing a merge is going to curse badly.
Ideally, every changeset committed to "old" should be "merged" inmediatelly to "new", by the patch maker. I don't consider an issue to be "closed" until this is done.
Not sure if this should be a real rule, but if "old" and "new" reside in the same repository (for instance, named branches), you can check this in the push "hook" living in the server. That is, the hook check that the changesets you are pushing don't create a new head in "old" (your patch create a head, but your merge "joins" heads in both "old" and "new". When you push, you push both changesets at the same time, and the repository is verified and updated atomically).
Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQCVAwUBTUxFRJlgi5GaxT1NAQJmpAP/bmdgoK4J4O/hS+DNrC59sVxOU0Y5gH4I gW7SnU/FDejszMeDJ8fi6/3/v4jcV+Mfy9MOSr4lCmfnmXXXUyHoMBL5UEmDvuGR LNN3QXEKZeNo5op4UQWa1rRp9qjYL95R/dBiVPgGY6Ia+V406fjdgYFXVn331KhY 0qpf5/teZ4c= =FYnu -----END PGP SIGNATURE-----
Am 02.02.2011 19:47, schrieb Antoine Pitrou:
Le mercredi 02 février 2011 à 19:39 +0100, Jesus Cea a écrit :
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/02/11 19:28, Antoine Pitrou wrote:
The merge here is mostly automatic. In fact, if the RM doesn't change his/her clone at all, the merge is "null", even if devel repository has evolved a lot in the meantime.
By merge I meant the cherry picking operation itself ("svnmerge").
To be concrete, how many patches went inside 2.7.0 after cutting the "rc1"?. Ideally, the answer should be "a handful".
I don't think we are talking about branching after rc1 but after beta1, so that the feature branch can continue receive non-bugfix patches. That's quite many changesets to review.
I don't think branching off at beta1 is helpful. Developers will happily continue adding features to trunk (because that's what's fun, isn't it), and not care much about the release branch. Obviously we want the opposite, and we want them to concentrate on bug fixing.
Georg
On Feb 03, 2011, at 09:01 AM, Georg Brandl wrote:
I don't think branching off at beta1 is helpful. Developers will happily continue adding features to trunk (because that's what's fun, isn't it), and not care much about the release branch. Obviously we want the opposite, and we want them to concentrate on bug fixing.
Aside from when we branch for the next release, I think we can still impose trunk freezes when we feel we need them (either by convention or technology). This will prevent folks from *landing* new features on the trunk, but it will not prevent folks from *developing* new features, and even publishing, reviewing and sharing them. That's a big advantage over a centralized vcs like Subversion.
-Barry
Le jeudi 03 février 2011 à 09:01 +0100, Georg Brandl a écrit :
I don't think branching off at beta1 is helpful. Developers will happily continue adding features to trunk (because that's what's fun, isn't it), and not care much about the release branch. Obviously we want the opposite, and we want them to concentrate on bug fixing.
I'm obviously not in a position to judge, but I find it interesting that you're arguing precisely for what the article says is better abandoned (according, apparently, to a numerical study of community activity):
Graph analysis showed very clearly that every time we would
freeze, the community would shrink drastically and it would take
several months after we un-froze for the size of the community
to recover. It happened uniformly, every single time we would
freeze, over many years and many releases.
[...]
We addressed this issue by *never* freezing the trunk. [...] We
are also doing feature development on the trunk *simultaneously*
with those bug fixes. However, we’ve found that not only does
the community expand more rapidly this way, but we also actually
get our releases out more quickly than we used to. So it’s a
win-win situation.(emphasis mine)
Regards
Antoine.
Am 03.02.2011 22:39, schrieb Antoine Pitrou:
Le jeudi 03 février 2011 à 09:01 +0100, Georg Brandl a écrit :
I don't think branching off at beta1 is helpful. Developers will happily continue adding features to trunk (because that's what's fun, isn't it), and not care much about the release branch. Obviously we want the opposite, and we want them to concentrate on bug fixing.
I'm obviously not in a position to judge, but I find it interesting that you're arguing precisely for what the article says is better abandoned (according, apparently, to a numerical study of community activity):
Well, I hope that nobody thinks the article to be eternal truth that can't be argued about. I also haven't read it yet, so I'm just stating what I think.
Graph analysis showed very clearly that every time we would freeze, the community would shrink drastically and it would take several months after we un-froze for the size of the community to recover. It happened uniformly, every single time we would freeze, over many years and many releases. [...] We addressed this issue by *never* freezing the trunk. [...] We are also doing feature development on the trunk *simultaneously* with those bug fixes. However, we’ve found that not only does the community expand more rapidly this way, but we also actually get our releases out more quickly than we used to. So it’s a win-win situation.
It's unclear to me what is meant here by "the community would shrink". The amount of core developers certainly doesn't shrink during feature freezes, from what I've experienced. Those who are active before remain active, those who contribute sporadically don't really care.
But as Barry says, this is all a bit different with DVCS, and everyone is welcome to develop new features in their own branches. I still think that -- even if only for workload reasons -- the official trunk should be closed to features during beta-rc phases.
Georg
Am 02.02.2011 17:47, schrieb Barry Warsaw:
On Feb 02, 2011, at 01:31 PM, Michael Foord wrote:
One issue it raises is the difficulties caused by freezing the trunk for releases. Instead they advocate creating the release branch at the point of the release candidate instead of freezing trunk. There are issues I currently *can't* work on because trunk is frozen and would personally prefer to see us use the branch-on-release-candidate process.
This is one of the primary problems solved by a dvcs.
Exactly. This is why I didn't do it that way it during the 3.2 releases; I was always waiting for the hg switch to happen, and until then I thought it best not to change the process.
Georg
On Feb 03, 2011, at 08:58 AM, Georg Brandl wrote:
Exactly. This is why I didn't do it that way it during the 3.2 releases; I was always waiting for the hg switch to happen, and until then I thought it best not to change the process.
Smart man. :)
We have a perfect storm approaching for the migration though. 3.2 will be released very soon so we have a natural cut-over point. The sprints will see most of the stakeholders in physical proximity to work together to make it happen. We'll also have lots of non-committers around for some high bandwidth tutorials getting the new workflows evangelized.
Seriously, if we can't leave Atlanta with the migration operational, we should just admit defeat and stick with Subversion.
-Barry
Le jeudi 03 février 2011 à 16:36 -0500, Barry Warsaw a écrit :
On Feb 03, 2011, at 08:58 AM, Georg Brandl wrote:
Exactly. This is why I didn't do it that way it during the 3.2 releases; I was always waiting for the hg switch to happen, and until then I thought it best not to change the process.
Smart man. :)
We have a perfect storm approaching for the migration though. 3.2 will be released very soon so we have a natural cut-over point. The sprints will see most of the stakeholders in physical proximity to work together to make it happen. We'll also have lots of non-committers around for some high bandwidth tutorials getting the new workflows evangelized.
Seriously, if we can't leave Atlanta with the migration operational, we should just admit defeat and stick with Subversion.
I'm not sure all of us will be *physically* in Atlanta - although there's no doubt we'll think about you :)
Am 03.02.2011 22:36, schrieb Barry Warsaw:
On Feb 03, 2011, at 08:58 AM, Georg Brandl wrote:
Exactly. This is why I didn't do it that way it during the 3.2 releases; I was always waiting for the hg switch to happen, and until then I thought it best not to change the process.
Smart man. :)
We have a perfect storm approaching for the migration though. 3.2 will be released very soon so we have a natural cut-over point. The sprints will see most of the stakeholders in physical proximity to work together to make it happen. We'll also have lots of non-committers around for some high bandwidth tutorials getting the new workflows evangelized.
I'm still hoping to get the conversion done *before* PyCon, so that everyone can already get familiar with the new system, in order to be productive at the sprints. But of course, I'm also counting on PyCon sprints as the "last line of defence" should all other attempts fail.
Seriously, if we can't leave Atlanta with the migration operational, we should just admit defeat and stick with Subversion.
Hah! I don't believe a word you say about sticking with Subversion. I think you're just silently waiting for an occasion to uncover the ready and polished bzr repositories...
<wink> Georg
On Feb 03, 2011, at 10:54 PM, Georg Brandl wrote:
I'm still hoping to get the conversion done *before* PyCon, so that everyone can already get familiar with the new system, in order to be productive at the sprints. But of course, I'm also counting on PyCon sprints as the "last line of defence" should all other attempts fail.
That would be awesome.
Seriously, if we can't leave Atlanta with the migration operational, we should just admit defeat and stick with Subversion.
Hah! I don't believe a word you say about sticking with Subversion. I think you're just silently waiting for an occasion to uncover the ready and polished bzr repositories...
No comment. :)
But seriously though, if we can get to hg before pycon, that will be great.
-Barry
Le mercredi 02 février 2011 à 11:47 -0500, Barry Warsaw a écrit :
One issue it raises is the difficulties caused by freezing the trunk for releases. Instead they advocate creating the release branch at the point of the release candidate instead of freezing trunk. There are issues I currently *can't* work on because trunk is frozen and would personally prefer to see us use the branch-on-release-candidate process.
This is one of the primary problems solved by a dvcs. You can *always* and *easily* work on new features, publish them for comment and review by others, make continual progress regardless of the release status of the official branches, and easily track movement in those official branches.
I am already using a DVCS (git-svn) to work on Python, especially for my work on Unicode. It is just impossible to work on an huge change on trunk, it changes too fast. A SVN branch should be created or a DVCS tool should be used. I prefer DVCS over SVN: it's faster, it's easier to do small commits, merge/remove commits, etc.
But my last problem is that I don't know how to publish my DVCS repository. It would be possible to host it at home, but I am to lazy to setup my server for that. The problem is also that the repository is huge (254 MB): publish it to Internet would be slow (I upload at 80 KB/sec) if I have to publish all files (and the history) the first time. It would be faster if I can fork an existing repository on a server (download is always faster than upload).
I suppose that bitbucket will have a miror of Python (because they already proposed to host the official repository), and so it will be trivial to fork Python repository with just a bitbucket account. That will be great, because it will easier to share a branch, without having to be a Python core developer.
We might also host forks on our server, but it implies to have to manager user accounts (with permissions), so do the same job than Bitbucket and other hosting websites.
I don't use Mercurial miror today, because I cannot be used to commit into Subversion (if I'm wrong, please tell me how to do that!), and I'm also using git-svn to commit into Python.
Victor
On Feb 04, 2011, at 12:35 PM, Victor Stinner wrote:
I am already using a DVCS (git-svn) to work on Python, especially for my work on Unicode. It is just impossible to work on an huge change on trunk, it changes too fast. A SVN branch should be created or a DVCS tool should be used. I prefer DVCS over SVN: it's faster, it's easier to do small commits, merge/remove commits, etc.
I did all of my PEP work for 3.2 in a Bazaar mirror of the py3k svn branch. I made zillions of small commits as I was working things out, and it was very easy for me to generate diffs for attaching to the bug tracker. It was also very easy for me to track changes in the trunk. Of course, conflicts happen occasionally, but that's unavoidable and IME were easily resolved.
For the actual commits to trunk though, I generated a patch and applied it to the official svn branch. I probably could have used the bzr-svn plugin to do the commit directly, but for vague reasons I didn't.
But my last problem is that I don't know how to publish my DVCS repository. It would be possible to host it at home, but I am to lazy to setup my server for that. The problem is also that the repository is huge (254 MB): publish it to Internet would be slow (I upload at 80 KB/sec) if I have to publish all files (and the history) the first time. It would be faster if I can fork an existing repository on a server (download is always faster than upload).
I suppose that bitbucket will have a miror of Python (because they already proposed to host the official repository), and so it will be trivial to fork Python repository with just a bitbucket account. That will be great, because it will easier to share a branch, without having to be a Python core developer.
We should most definitely host branches of works-in-progress for core committers, just as we do for the official branches. The fact that there are free code hosting services available should make it easy for anybody to publish their branches, anywhere they want.
We might also host forks on our server, but it implies to have to manager user accounts (with permissions), so do the same job than Bitbucket and other hosting websites.
We already have to manage user accounts for core committers, so for them, there should be little additional work. In fact, two years ago, we did set up personal code hosting on python.org servers for both the Mercurial and Bazaar experiments.
It's definitely more work to manage user accounts for non-core committers, so I'm not suggesting we do that from the start, but we should be open to that option if we have the volunteers to maintain that.
-Barry
участники (16)
-
"Martin v. Löwis" -
Alexander Belopolsky -
Antoine Pitrou -
Barry Warsaw -
Brett Cannon -
Daniel Stutzbach -
Dirkjan Ochtman -
Georg Brandl -
Jesus Cea -
M.-A. Lemburg -
Michael Foord -
Nick Coghlan -
R. David Murray -
Steve Holden -
Terry Reedy -
Victor Stinner