Proposed modification to YTEP-1776: bugfixes to the stable branch, code freezes
Hi all, Before making a PR to the YTEP repository, I wanted to propose a modification to our development workflow to the mailing list. In particular, I think we need to be more careful about what changes go to the yt branch and the stable branch. Right now we suggest all new changes, including bug fixes, API changes, and new features, go to the "yt" branch. Periodically, we merge from the "yt" branch into the "stable" branch, tag the merge with a release number, and upload to pypi. This means that we are restricted in making bugfix releases that include no breaking changes. As soon as a new feature gets merged into the "yt" branch, by all rights the next merge-to-stable and release should be a minor version change instead of a patch version change (i.e. we should be releasing yt 3.1 instead of yt 3.0.1 after merging in a new feature or API change, see http://semver.org/ for more details on this nomenclature). Generally, people expect patch version bumps to only include bugfixes. I'd like to propose a modification to our development workflow that should make it easier to make sure bugfixes can be released without breaking unrelated things, while still allowing us to temporarily break stuff on the dev branch. I think rather than merging from the "yt" branch into "stable" *every* time we do a release, we should instead issue patch releases from the current state of the stable branch. Instead of merging from "yt" into "stable" with every release, we should only do so before a minor or major version release, not patch releases. This means that bugfixes will need to go to the "stable" branch instead of the "yt" branch. It also means that the "stable" branch will need to be merged into the "yt" branch with some regularity to make sure bugfixes make their way to the dev version of the code. It will also take some degree of manual care to ensure that the "@" bookmark always points at the tip of the stable branch so fresh clones get bugfixes. Additionally, bugfixes may need to be backported or cherrypicked from the "yt" branch to the "stable" branch if a bugfix comes in as part of some bigger change or if a contributor isn't aware that bugfixes should go to the stable branch. This workflow sort of matches what's done in the mercurial community. I think it would be a good idea to follow the mercurial community in another way, we should institute a "code freeze" for the two weeks prior to a non-bugfix release. This means that all developers switch to the tip of the "yt" branch, and only PRs that fix bugs, documentation issues, or improve error messages can be merged. Just before release, the "yt" branch is merged into "stable". See http://mercurial.selenic.com/wiki/TimeBasedReleasePlan for more details. Instituting such a testing period should help us understand the scope of a release a few weeks out, keep pre-release documentation updates bounded, and hopefully help suss out bugs since most developers will be testing a sync'd up-to-date version of the code that people will actually be using. What do you all think about these proposals? If there's broad agreement that this is a good idea, I'll put this proposal together as a patch for YTEP-1776. If not I'll hold off for a while until we can come to a consensus. Best, Nathan
Hey Nathan, I think in general this is a great idea. The only concern I have is that it may not always be easy to separate bugfixes for the stable branch vs bugfixes which may only work on code in the dev branch (where the API may be different or there may be new functionality). I'm not saying this is impossible, but it may be challenging to do right. I suppose if we have less time between releases, this will be minimized, since dev and stable will be more in sync with each other. An example of this that comes to mind is the ProjectionPlot's 'method' attribute. In yt 3.0, this attribute was 'style', but I submitted a PR which changed it to 'method' recently. Since this is an API change, it shouldn't be incorporated into patch releases, but it would only go into yt 3.1, right? But John Zuhone has a PR right now with lots of bugfixes, including one that enables the 'method' attribute to work with 'sum's in the OffAxisProjection. Under the proposed infrastructure, how would he submit this bugfix? Would he submit it as a bugfix to the 'style' attribute in the old stable branch (prior to the acceptance of my PR?), which would later change to the 'method' attribute in the next point release? Or would he hold off on it until the next point release? It seems unclear to me, but maybe I'm just being daft. I do really like the idea of shutting off substantive PRs 2 weeks before release, though, and asking everyone to do their work on the tip. I'm very much in favor of that. Cameron On Mon, Nov 3, 2014 at 4:43 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi all,
Before making a PR to the YTEP repository, I wanted to propose a modification to our development workflow to the mailing list.
In particular, I think we need to be more careful about what changes go to the yt branch and the stable branch.
Right now we suggest all new changes, including bug fixes, API changes, and new features, go to the "yt" branch. Periodically, we merge from the "yt" branch into the "stable" branch, tag the merge with a release number, and upload to pypi.
This means that we are restricted in making bugfix releases that include no breaking changes. As soon as a new feature gets merged into the "yt" branch, by all rights the next merge-to-stable and release should be a minor version change instead of a patch version change (i.e. we should be releasing yt 3.1 instead of yt 3.0.1 after merging in a new feature or API change, see http://semver.org/ for more details on this nomenclature). Generally, people expect patch version bumps to only include bugfixes.
I'd like to propose a modification to our development workflow that should make it easier to make sure bugfixes can be released without breaking unrelated things, while still allowing us to temporarily break stuff on the dev branch.
I think rather than merging from the "yt" branch into "stable" *every* time we do a release, we should instead issue patch releases from the current state of the stable branch. Instead of merging from "yt" into "stable" with every release, we should only do so before a minor or major version release, not patch releases.
This means that bugfixes will need to go to the "stable" branch instead of the "yt" branch. It also means that the "stable" branch will need to be merged into the "yt" branch with some regularity to make sure bugfixes make their way to the dev version of the code. It will also take some degree of manual care to ensure that the "@" bookmark always points at the tip of the stable branch so fresh clones get bugfixes. Additionally, bugfixes may need to be backported or cherrypicked from the "yt" branch to the "stable" branch if a bugfix comes in as part of some bigger change or if a contributor isn't aware that bugfixes should go to the stable branch.
This workflow sort of matches what's done in the mercurial community.
I think it would be a good idea to follow the mercurial community in another way, we should institute a "code freeze" for the two weeks prior to a non-bugfix release. This means that all developers switch to the tip of the "yt" branch, and only PRs that fix bugs, documentation issues, or improve error messages can be merged. Just before release, the "yt" branch is merged into "stable".
See http://mercurial.selenic.com/wiki/TimeBasedReleasePlan for more details.
Instituting such a testing period should help us understand the scope of a release a few weeks out, keep pre-release documentation updates bounded, and hopefully help suss out bugs since most developers will be testing a sync'd up-to-date version of the code that people will actually be using.
What do you all think about these proposals? If there's broad agreement that this is a good idea, I'll put this proposal together as a patch for YTEP-1776. If not I'll hold off for a while until we can come to a consensus.
Best,
Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
On Mon, Nov 3, 2014 at 3:56 PM, Cameron Hummels <chummels@gmail.com> wrote:
Hey Nathan,
I think in general this is a great idea. The only concern I have is that it may not always be easy to separate bugfixes for the stable branch vs bugfixes which may only work on code in the dev branch (where the API may be different or there may be new functionality). I'm not saying this is impossible, but it may be challenging to do right. I suppose if we have less time between releases, this will be minimized, since dev and stable will be more in sync with each other.
An example of this that comes to mind is the ProjectionPlot's 'method' attribute. In yt 3.0, this attribute was 'style', but I submitted a PR which changed it to 'method' recently. Since this is an API change, it shouldn't be incorporated into patch releases, but it would only go into yt 3.1, right? But John Zuhone has a PR right now with lots of bugfixes, including one that enables the 'method' attribute to work with 'sum's in the OffAxisProjection. Under the proposed infrastructure, how would he submit this bugfix? Would he submit it as a bugfix to the 'style' attribute in the old stable branch (prior to the acceptance of my PR?), which would later change to the 'method' attribute in the next point release? Or would he hold off on it until the next point release? It seems unclear to me, but maybe I'm just being daft.
The API change would go in the next minor release, so all of these changes would end up in yt-3.1. If there's a bugfix for "style" then yes technically it should be backported to the stable branch, so that means the fix might need to happen twice. I think this is ok - we're working in the best interests of our users to ensure that everyone gets fixes for bugs. In practice it might be difficult to backport all fixes. We can try our best, though.... It's true that having PRs go to two different branches makes life more difficult. Some fixes will need to be manually cherrypicked to stable. Then again, having a culture of "bugfix to stable" may encourage people to keep bugfixes more atomic and small, and generally apply them to the stable branch. This is definitely a trade-off. We'd be making the lives of developers a bit more complicated in the interest of not surprising our users with breaking changes in a "bugfix" release.
I do really like the idea of shutting off substantive PRs 2 weeks before release, though, and asking everyone to do their work on the tip. I'm very much in favor of that.
Cameron
On Mon, Nov 3, 2014 at 4:43 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi all,
Before making a PR to the YTEP repository, I wanted to propose a modification to our development workflow to the mailing list.
In particular, I think we need to be more careful about what changes go to the yt branch and the stable branch.
Right now we suggest all new changes, including bug fixes, API changes, and new features, go to the "yt" branch. Periodically, we merge from the "yt" branch into the "stable" branch, tag the merge with a release number, and upload to pypi.
This means that we are restricted in making bugfix releases that include no breaking changes. As soon as a new feature gets merged into the "yt" branch, by all rights the next merge-to-stable and release should be a minor version change instead of a patch version change (i.e. we should be releasing yt 3.1 instead of yt 3.0.1 after merging in a new feature or API change, see http://semver.org/ for more details on this nomenclature). Generally, people expect patch version bumps to only include bugfixes.
I'd like to propose a modification to our development workflow that should make it easier to make sure bugfixes can be released without breaking unrelated things, while still allowing us to temporarily break stuff on the dev branch.
I think rather than merging from the "yt" branch into "stable" *every* time we do a release, we should instead issue patch releases from the current state of the stable branch. Instead of merging from "yt" into "stable" with every release, we should only do so before a minor or major version release, not patch releases.
This means that bugfixes will need to go to the "stable" branch instead of the "yt" branch. It also means that the "stable" branch will need to be merged into the "yt" branch with some regularity to make sure bugfixes make their way to the dev version of the code. It will also take some degree of manual care to ensure that the "@" bookmark always points at the tip of the stable branch so fresh clones get bugfixes. Additionally, bugfixes may need to be backported or cherrypicked from the "yt" branch to the "stable" branch if a bugfix comes in as part of some bigger change or if a contributor isn't aware that bugfixes should go to the stable branch.
This workflow sort of matches what's done in the mercurial community.
I think it would be a good idea to follow the mercurial community in another way, we should institute a "code freeze" for the two weeks prior to a non-bugfix release. This means that all developers switch to the tip of the "yt" branch, and only PRs that fix bugs, documentation issues, or improve error messages can be merged. Just before release, the "yt" branch is merged into "stable".
See http://mercurial.selenic.com/wiki/TimeBasedReleasePlan for more details.
Instituting such a testing period should help us understand the scope of a release a few weeks out, keep pre-release documentation updates bounded, and hopefully help suss out bugs since most developers will be testing a sync'd up-to-date version of the code that people will actually be using.
What do you all think about these proposals? If there's broad agreement that this is a good idea, I'll put this proposal together as a patch for YTEP-1776. If not I'll hold off for a while until we can come to a consensus.
Best,
Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Nathan, I’ll echo Cameron’s concerns that it’s not always obvious how to disentangle API changes from pure bugfixes. I’ve just submitted a PR that fixes a bug *by* changing the API. In this case it’s backwards compatible (the API has only grown), but I’m not sure yt has a sufficiently stable API to protect. If we allow the policy to be broken from time to time, and simply reflect recommended practice, I think it’s a good idea. Douglas Rudd Scientific Computing Consultant Research Computing Center drudd@uchicago.edu<mailto:drudd@uchicago.edu> On Nov 3, 2014, at 5:56 PM, Cameron Hummels <chummels@gmail.com<mailto:chummels@gmail.com>> wrote: Hey Nathan, I think in general this is a great idea. The only concern I have is that it may not always be easy to separate bugfixes for the stable branch vs bugfixes which may only work on code in the dev branch (where the API may be different or there may be new functionality). I'm not saying this is impossible, but it may be challenging to do right. I suppose if we have less time between releases, this will be minimized, since dev and stable will be more in sync with each other. An example of this that comes to mind is the ProjectionPlot's 'method' attribute. In yt 3.0, this attribute was 'style', but I submitted a PR which changed it to 'method' recently. Since this is an API change, it shouldn't be incorporated into patch releases, but it would only go into yt 3.1, right? But John Zuhone has a PR right now with lots of bugfixes, including one that enables the 'method' attribute to work with 'sum's in the OffAxisProjection. Under the proposed infrastructure, how would he submit this bugfix? Would he submit it as a bugfix to the 'style' attribute in the old stable branch (prior to the acceptance of my PR?), which would later change to the 'method' attribute in the next point release? Or would he hold off on it until the next point release? It seems unclear to me, but maybe I'm just being daft. I do really like the idea of shutting off substantive PRs 2 weeks before release, though, and asking everyone to do their work on the tip. I'm very much in favor of that. Cameron On Mon, Nov 3, 2014 at 4:43 PM, Nathan Goldbaum <nathan12343@gmail.com<mailto:nathan12343@gmail.com>> wrote: Hi all, Before making a PR to the YTEP repository, I wanted to propose a modification to our development workflow to the mailing list. In particular, I think we need to be more careful about what changes go to the yt branch and the stable branch. Right now we suggest all new changes, including bug fixes, API changes, and new features, go to the "yt" branch. Periodically, we merge from the "yt" branch into the "stable" branch, tag the merge with a release number, and upload to pypi. This means that we are restricted in making bugfix releases that include no breaking changes. As soon as a new feature gets merged into the "yt" branch, by all rights the next merge-to-stable and release should be a minor version change instead of a patch version change (i.e. we should be releasing yt 3.1 instead of yt 3.0.1 after merging in a new feature or API change, see http://semver.org/ for more details on this nomenclature). Generally, people expect patch version bumps to only include bugfixes. I'd like to propose a modification to our development workflow that should make it easier to make sure bugfixes can be released without breaking unrelated things, while still allowing us to temporarily break stuff on the dev branch. I think rather than merging from the "yt" branch into "stable" *every* time we do a release, we should instead issue patch releases from the current state of the stable branch. Instead of merging from "yt" into "stable" with every release, we should only do so before a minor or major version release, not patch releases. This means that bugfixes will need to go to the "stable" branch instead of the "yt" branch. It also means that the "stable" branch will need to be merged into the "yt" branch with some regularity to make sure bugfixes make their way to the dev version of the code. It will also take some degree of manual care to ensure that the "@" bookmark always points at the tip of the stable branch so fresh clones get bugfixes. Additionally, bugfixes may need to be backported or cherrypicked from the "yt" branch to the "stable" branch if a bugfix comes in as part of some bigger change or if a contributor isn't aware that bugfixes should go to the stable branch. This workflow sort of matches what's done in the mercurial community. I think it would be a good idea to follow the mercurial community in another way, we should institute a "code freeze" for the two weeks prior to a non-bugfix release. This means that all developers switch to the tip of the "yt" branch, and only PRs that fix bugs, documentation issues, or improve error messages can be merged. Just before release, the "yt" branch is merged into "stable". See http://mercurial.selenic.com/wiki/TimeBasedReleasePlan for more details. Instituting such a testing period should help us understand the scope of a release a few weeks out, keep pre-release documentation updates bounded, and hopefully help suss out bugs since most developers will be testing a sync'd up-to-date version of the code that people will actually be using. What do you all think about these proposals? If there's broad agreement that this is a good idea, I'll put this proposal together as a patch for YTEP-1776. If not I'll hold off for a while until we can come to a consensus. Best, Nathan _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org<mailto:yt-dev@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org -- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org<http://chummels.org/> _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org<mailto:yt-dev@lists.spacepope.org> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
I'm +1 on the manifesto, but share the concerns about disentangling PRs that need to put in the right places - could be challenging, but I'm not sure there is a straightforward fix. On Mon, Nov 3, 2014 at 5:05 PM, Douglas Harvey Rudd <drudd@uchicago.edu> wrote:
Hi Nathan,
I’ll echo Cameron’s concerns that it’s not always obvious how to disentangle API changes from pure bugfixes. I’ve just submitted a PR that fixes a bug *by* changing the API. In this case it’s backwards compatible (the API has only grown), but I’m not sure yt has a sufficiently stable API to protect.
If we allow the policy to be broken from time to time, and simply reflect recommended practice, I think it’s a good idea.
Douglas Rudd Scientific Computing Consultant Research Computing Center drudd@uchicago.edu
On Nov 3, 2014, at 5:56 PM, Cameron Hummels <chummels@gmail.com> wrote:
Hey Nathan,
I think in general this is a great idea. The only concern I have is that it may not always be easy to separate bugfixes for the stable branch vs bugfixes which may only work on code in the dev branch (where the API may be different or there may be new functionality). I'm not saying this is impossible, but it may be challenging to do right. I suppose if we have less time between releases, this will be minimized, since dev and stable will be more in sync with each other.
An example of this that comes to mind is the ProjectionPlot's 'method' attribute. In yt 3.0, this attribute was 'style', but I submitted a PR which changed it to 'method' recently. Since this is an API change, it shouldn't be incorporated into patch releases, but it would only go into yt 3.1, right? But John Zuhone has a PR right now with lots of bugfixes, including one that enables the 'method' attribute to work with 'sum's in the OffAxisProjection. Under the proposed infrastructure, how would he submit this bugfix? Would he submit it as a bugfix to the 'style' attribute in the old stable branch (prior to the acceptance of my PR?), which would later change to the 'method' attribute in the next point release? Or would he hold off on it until the next point release? It seems unclear to me, but maybe I'm just being daft.
I do really like the idea of shutting off substantive PRs 2 weeks before release, though, and asking everyone to do their work on the tip. I'm very much in favor of that.
Cameron
On Mon, Nov 3, 2014 at 4:43 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi all,
Before making a PR to the YTEP repository, I wanted to propose a modification to our development workflow to the mailing list.
In particular, I think we need to be more careful about what changes go to the yt branch and the stable branch.
Right now we suggest all new changes, including bug fixes, API changes, and new features, go to the "yt" branch. Periodically, we merge from the "yt" branch into the "stable" branch, tag the merge with a release number, and upload to pypi.
This means that we are restricted in making bugfix releases that include no breaking changes. As soon as a new feature gets merged into the "yt" branch, by all rights the next merge-to-stable and release should be a minor version change instead of a patch version change (i.e. we should be releasing yt 3.1 instead of yt 3.0.1 after merging in a new feature or API change, see http://semver.org/ for more details on this nomenclature). Generally, people expect patch version bumps to only include bugfixes.
I'd like to propose a modification to our development workflow that should make it easier to make sure bugfixes can be released without breaking unrelated things, while still allowing us to temporarily break stuff on the dev branch.
I think rather than merging from the "yt" branch into "stable" *every* time we do a release, we should instead issue patch releases from the current state of the stable branch. Instead of merging from "yt" into "stable" with every release, we should only do so before a minor or major version release, not patch releases.
This means that bugfixes will need to go to the "stable" branch instead of the "yt" branch. It also means that the "stable" branch will need to be merged into the "yt" branch with some regularity to make sure bugfixes make their way to the dev version of the code. It will also take some degree of manual care to ensure that the "@" bookmark always points at the tip of the stable branch so fresh clones get bugfixes. Additionally, bugfixes may need to be backported or cherrypicked from the "yt" branch to the "stable" branch if a bugfix comes in as part of some bigger change or if a contributor isn't aware that bugfixes should go to the stable branch.
This workflow sort of matches what's done in the mercurial community.
I think it would be a good idea to follow the mercurial community in another way, we should institute a "code freeze" for the two weeks prior to a non-bugfix release. This means that all developers switch to the tip of the "yt" branch, and only PRs that fix bugs, documentation issues, or improve error messages can be merged. Just before release, the "yt" branch is merged into "stable".
See http://mercurial.selenic.com/wiki/TimeBasedReleasePlan for more details.
Instituting such a testing period should help us understand the scope of a release a few weeks out, keep pre-release documentation updates bounded, and hopefully help suss out bugs since most developers will be testing a sync'd up-to-date version of the code that people will actually be using.
What do you all think about these proposals? If there's broad agreement that this is a good idea, I'll put this proposal together as a patch for YTEP-1776. If not I'll hold off for a while until we can come to a consensus.
Best,
Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Sounds like this falls into the "not 'rules'... More like 'guidelines'..." category. John ZuHone Kavli Center for Astrophysics and Space Research Massachusetts Institute of Technology 77 Massachusetts Ave., 37-582G Cambridge, MA 02139 (w) 617-253-2354 (m) 781-708-5004 jzuhone@space.mit.edu jzuhone@gmail.com http://www.jzuhone.com
On Nov 3, 2014, at 7:15 PM, Chris Malone <chris.m.malone@gmail.com> wrote:
I'm +1 on the manifesto, but share the concerns about disentangling PRs that need to put in the right places - could be challenging, but I'm not sure there is a straightforward fix.
On Mon, Nov 3, 2014 at 5:05 PM, Douglas Harvey Rudd <drudd@uchicago.edu> wrote: Hi Nathan,
I’ll echo Cameron’s concerns that it’s not always obvious how to disentangle API changes from pure bugfixes. I’ve just submitted a PR that fixes a bug *by* changing the API. In this case it’s backwards compatible (the API has only grown), but I’m not sure yt has a sufficiently stable API to protect.
If we allow the policy to be broken from time to time, and simply reflect recommended practice, I think it’s a good idea.
Douglas Rudd Scientific Computing Consultant Research Computing Center drudd@uchicago.edu
On Nov 3, 2014, at 5:56 PM, Cameron Hummels <chummels@gmail.com> wrote:
Hey Nathan,
I think in general this is a great idea. The only concern I have is that it may not always be easy to separate bugfixes for the stable branch vs bugfixes which may only work on code in the dev branch (where the API may be different or there may be new functionality). I'm not saying this is impossible, but it may be challenging to do right. I suppose if we have less time between releases, this will be minimized, since dev and stable will be more in sync with each other.
An example of this that comes to mind is the ProjectionPlot's 'method' attribute. In yt 3.0, this attribute was 'style', but I submitted a PR which changed it to 'method' recently. Since this is an API change, it shouldn't be incorporated into patch releases, but it would only go into yt 3.1, right? But John Zuhone has a PR right now with lots of bugfixes, including one that enables the 'method' attribute to work with 'sum's in the OffAxisProjection. Under the proposed infrastructure, how would he submit this bugfix? Would he submit it as a bugfix to the 'style' attribute in the old stable branch (prior to the acceptance of my PR?), which would later change to the 'method' attribute in the next point release? Or would he hold off on it until the next point release? It seems unclear to me, but maybe I'm just being daft.
I do really like the idea of shutting off substantive PRs 2 weeks before release, though, and asking everyone to do their work on the tip. I'm very much in favor of that.
Cameron
On Mon, Nov 3, 2014 at 4:43 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote: Hi all,
Before making a PR to the YTEP repository, I wanted to propose a modification to our development workflow to the mailing list.
In particular, I think we need to be more careful about what changes go to the yt branch and the stable branch.
Right now we suggest all new changes, including bug fixes, API changes, and new features, go to the "yt" branch. Periodically, we merge from the "yt" branch into the "stable" branch, tag the merge with a release number, and upload to pypi.
This means that we are restricted in making bugfix releases that include no breaking changes. As soon as a new feature gets merged into the "yt" branch, by all rights the next merge-to-stable and release should be a minor version change instead of a patch version change (i.e. we should be releasing yt 3.1 instead of yt 3.0.1 after merging in a new feature or API change, see http://semver.org/ for more details on this nomenclature). Generally, people expect patch version bumps to only include bugfixes.
I'd like to propose a modification to our development workflow that should make it easier to make sure bugfixes can be released without breaking unrelated things, while still allowing us to temporarily break stuff on the dev branch.
I think rather than merging from the "yt" branch into "stable" *every* time we do a release, we should instead issue patch releases from the current state of the stable branch. Instead of merging from "yt" into "stable" with every release, we should only do so before a minor or major version release, not patch releases.
This means that bugfixes will need to go to the "stable" branch instead of the "yt" branch. It also means that the "stable" branch will need to be merged into the "yt" branch with some regularity to make sure bugfixes make their way to the dev version of the code. It will also take some degree of manual care to ensure that the "@" bookmark always points at the tip of the stable branch so fresh clones get bugfixes. Additionally, bugfixes may need to be backported or cherrypicked from the "yt" branch to the "stable" branch if a bugfix comes in as part of some bigger change or if a contributor isn't aware that bugfixes should go to the stable branch.
This workflow sort of matches what's done in the mercurial community.
I think it would be a good idea to follow the mercurial community in another way, we should institute a "code freeze" for the two weeks prior to a non-bugfix release. This means that all developers switch to the tip of the "yt" branch, and only PRs that fix bugs, documentation issues, or improve error messages can be merged. Just before release, the "yt" branch is merged into "stable".
See http://mercurial.selenic.com/wiki/TimeBasedReleasePlan for more details.
Instituting such a testing period should help us understand the scope of a release a few weeks out, keep pre-release documentation updates bounded, and hopefully help suss out bugs since most developers will be testing a sync'd up-to-date version of the code that people will actually be using.
What do you all think about these proposals? If there's broad agreement that this is a good idea, I'll put this proposal together as a patch for YTEP-1776. If not I'll hold off for a while until we can come to a consensus.
Best,
Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hi Nathan, +1 on the two-week dev freeze prior to releases. That is a great idea. As for the other thing, I'm ok with doing our best to adhere to this policy whenever we can and allowing for the occasional exception. I think this plan is complicated enough that it needs a named position in charge of making it happen. This person's duties would be to keep track of states of the two branches, make sure fixes are applied where they should be, etc., etc. Without that, I think this may turn chaotic before too long. Perhaps this person can be the same person who calls the development freeze leading up to a release, in which case this is just the release manage. If so, we need to lay these details out explicitly in the YTEP. Britton On Tue, Nov 4, 2014 at 12:27 AM, John ZuHone <jzuhone@gmail.com> wrote:
Sounds like this falls into the "not 'rules'... More like 'guidelines'..." category.
John ZuHone Kavli Center for Astrophysics and Space Research Massachusetts Institute of Technology 77 Massachusetts Ave., 37-582G Cambridge, MA 02139 (w) 617-253-2354 (m) 781-708-5004 jzuhone@space.mit.edu jzuhone@gmail.com http://www.jzuhone.com
On Nov 3, 2014, at 7:15 PM, Chris Malone <chris.m.malone@gmail.com> wrote:
I'm +1 on the manifesto, but share the concerns about disentangling PRs that need to put in the right places - could be challenging, but I'm not sure there is a straightforward fix.
On Mon, Nov 3, 2014 at 5:05 PM, Douglas Harvey Rudd <drudd@uchicago.edu> wrote:
Hi Nathan,
I’ll echo Cameron’s concerns that it’s not always obvious how to disentangle API changes from pure bugfixes. I’ve just submitted a PR that fixes a bug *by* changing the API. In this case it’s backwards compatible (the API has only grown), but I’m not sure yt has a sufficiently stable API to protect.
If we allow the policy to be broken from time to time, and simply reflect recommended practice, I think it’s a good idea.
Douglas Rudd Scientific Computing Consultant Research Computing Center drudd@uchicago.edu
On Nov 3, 2014, at 5:56 PM, Cameron Hummels <chummels@gmail.com> wrote:
Hey Nathan,
I think in general this is a great idea. The only concern I have is that it may not always be easy to separate bugfixes for the stable branch vs bugfixes which may only work on code in the dev branch (where the API may be different or there may be new functionality). I'm not saying this is impossible, but it may be challenging to do right. I suppose if we have less time between releases, this will be minimized, since dev and stable will be more in sync with each other.
An example of this that comes to mind is the ProjectionPlot's 'method' attribute. In yt 3.0, this attribute was 'style', but I submitted a PR which changed it to 'method' recently. Since this is an API change, it shouldn't be incorporated into patch releases, but it would only go into yt 3.1, right? But John Zuhone has a PR right now with lots of bugfixes, including one that enables the 'method' attribute to work with 'sum's in the OffAxisProjection. Under the proposed infrastructure, how would he submit this bugfix? Would he submit it as a bugfix to the 'style' attribute in the old stable branch (prior to the acceptance of my PR?), which would later change to the 'method' attribute in the next point release? Or would he hold off on it until the next point release? It seems unclear to me, but maybe I'm just being daft.
I do really like the idea of shutting off substantive PRs 2 weeks before release, though, and asking everyone to do their work on the tip. I'm very much in favor of that.
Cameron
On Mon, Nov 3, 2014 at 4:43 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi all,
Before making a PR to the YTEP repository, I wanted to propose a modification to our development workflow to the mailing list.
In particular, I think we need to be more careful about what changes go to the yt branch and the stable branch.
Right now we suggest all new changes, including bug fixes, API changes, and new features, go to the "yt" branch. Periodically, we merge from the "yt" branch into the "stable" branch, tag the merge with a release number, and upload to pypi.
This means that we are restricted in making bugfix releases that include no breaking changes. As soon as a new feature gets merged into the "yt" branch, by all rights the next merge-to-stable and release should be a minor version change instead of a patch version change (i.e. we should be releasing yt 3.1 instead of yt 3.0.1 after merging in a new feature or API change, see http://semver.org/ for more details on this nomenclature). Generally, people expect patch version bumps to only include bugfixes.
I'd like to propose a modification to our development workflow that should make it easier to make sure bugfixes can be released without breaking unrelated things, while still allowing us to temporarily break stuff on the dev branch.
I think rather than merging from the "yt" branch into "stable" *every* time we do a release, we should instead issue patch releases from the current state of the stable branch. Instead of merging from "yt" into "stable" with every release, we should only do so before a minor or major version release, not patch releases.
This means that bugfixes will need to go to the "stable" branch instead of the "yt" branch. It also means that the "stable" branch will need to be merged into the "yt" branch with some regularity to make sure bugfixes make their way to the dev version of the code. It will also take some degree of manual care to ensure that the "@" bookmark always points at the tip of the stable branch so fresh clones get bugfixes. Additionally, bugfixes may need to be backported or cherrypicked from the "yt" branch to the "stable" branch if a bugfix comes in as part of some bigger change or if a contributor isn't aware that bugfixes should go to the stable branch.
This workflow sort of matches what's done in the mercurial community.
I think it would be a good idea to follow the mercurial community in another way, we should institute a "code freeze" for the two weeks prior to a non-bugfix release. This means that all developers switch to the tip of the "yt" branch, and only PRs that fix bugs, documentation issues, or improve error messages can be merged. Just before release, the "yt" branch is merged into "stable".
See http://mercurial.selenic.com/wiki/TimeBasedReleasePlan for more details.
Instituting such a testing period should help us understand the scope of a release a few weeks out, keep pre-release documentation updates bounded, and hopefully help suss out bugs since most developers will be testing a sync'd up-to-date version of the code that people will actually be using.
What do you all think about these proposals? If there's broad agreement that this is a good idea, I'll put this proposal together as a patch for YTEP-1776. If not I'll hold off for a while until we can come to a consensus.
Best,
Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (6)
-
Britton Smith -
Cameron Hummels -
Chris Malone -
Douglas Harvey Rudd -
John ZuHone -
Nathan Goldbaum