Backporting bugfixes and possible changes to development practices
Hi all, I spent some time this week coming up with a technical solution to our social problem of doing bugfix releases. This is embodied in a new script, "pr_backport.py", which I've created a pull request for here to include with the yt repository. https://bitbucket.org/yt_analysis/yt/pull-requests/1717/add-pr-backport-scri... Briefly, the script creates a temporary clone of yt_analysis/yt, figures out which commits on the "yt" branch need to be included on the stable branch, and then assigns those commits to a pull request, and then interactively suggests mercurial commands to run in the temporary repository to backport individual pull requests to the stable branch. This is all done "manually" by copy/pasting mercurial commands suggested by the script. I don't think this should be fully automated since merge conflicts can and likely will be triggered, particularly if the stable and yt branches begin to substantially diverge. My hope is that this will make the task of backporting bugfixes, which up until this point has been associated with such a high cognitive barrier that no one has wanted to sit down and acutally do it, much easier. I might be biased since I wrote the script, but using the final version that I pull requested it took me about 10 minutes or so to go through the list of pull requests and backport them to create this pull request: https://bitbucket.org/yt_analysis/yt/pull-requests/1716/backporting-bugfixes... Being able to backport bugfixes like this allows us to possibly make some changes to our development workflow and suggested best practices. 1. We can now suggest most users stay on the stable branch rather than switching to the dev branch to get bugfixes. We can also issue bugfix releases on a much quicker cadence (monthly? biweekly?). 2. Since the "stable" branch is more stable and less buggy and the dev branch is basically now just for new features, perhaps we can allow more experimental development on the main yt branch without an onerous requirement of documentation and review. This will help unblock the development of major features like support for unstructured mesh data and the new volume rendering interface. 3. Since there is less pressure to do a feature release just to get bugfixes out, we might be able to have longer intervals between feature releases, giving more time to shake out issues. I'm more or less just throwing these thoughts out there to get feedback from the development community. Does the easy availability of bugfix releases constitute a significant change for our development and release mindset? Thanks all for your time and input! -Nathan
Hi Nathan, This is really incredible. What we may want to do is conclude each PR Triage hangout with a request for someone to step up and volunteer to run this. Then we'd have the "stable" branch up to date, and we could start encouraging people to be *on* stable. In fact, if this is successful, we may want to encourage developers to start doing non-breaking development and whatnot on stable. I'd like to see us get to a point where we view the stable as stable, not just "released", and we can start to do the more experimental work, like you suggested, in the "yt" branch. But that's a bigger topic. Thank you for doing this! -Matt On Fri, Aug 21, 2015 at 2:32 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi all,
I spent some time this week coming up with a technical solution to our social problem of doing bugfix releases. This is embodied in a new script, "pr_backport.py", which I've created a pull request for here to include with the yt repository.
https://bitbucket.org/yt_analysis/yt/pull-requests/1717/add-pr-backport-scri...
Briefly, the script creates a temporary clone of yt_analysis/yt, figures out which commits on the "yt" branch need to be included on the stable branch, and then assigns those commits to a pull request, and then interactively suggests mercurial commands to run in the temporary repository to backport individual pull requests to the stable branch. This is all done "manually" by copy/pasting mercurial commands suggested by the script. I don't think this should be fully automated since merge conflicts can and likely will be triggered, particularly if the stable and yt branches begin to substantially diverge.
My hope is that this will make the task of backporting bugfixes, which up until this point has been associated with such a high cognitive barrier that no one has wanted to sit down and acutally do it, much easier.
I might be biased since I wrote the script, but using the final version that I pull requested it took me about 10 minutes or so to go through the list of pull requests and backport them to create this pull request:
https://bitbucket.org/yt_analysis/yt/pull-requests/1716/backporting-bugfixes...
Being able to backport bugfixes like this allows us to possibly make some changes to our development workflow and suggested best practices.
1. We can now suggest most users stay on the stable branch rather than switching to the dev branch to get bugfixes. We can also issue bugfix releases on a much quicker cadence (monthly? biweekly?).
2. Since the "stable" branch is more stable and less buggy and the dev branch is basically now just for new features, perhaps we can allow more experimental development on the main yt branch without an onerous requirement of documentation and review. This will help unblock the development of major features like support for unstructured mesh data and the new volume rendering interface.
3. Since there is less pressure to do a feature release just to get bugfixes out, we might be able to have longer intervals between feature releases, giving more time to shake out issues.
I'm more or less just throwing these thoughts out there to get feedback from the development community. Does the easy availability of bugfix releases constitute a significant change for our development and release mindset?
Thanks all for your time and input!
-Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Hear hear--this is awesome, Nathan. I like the idea of having someone at the end of the PR Triage agree to run all of this. Great job! Cameron On Fri, Aug 21, 2015 at 2:01 PM, Matthew Turk <matthewturk@gmail.com> wrote:
Hi Nathan,
This is really incredible. What we may want to do is conclude each PR Triage hangout with a request for someone to step up and volunteer to run this. Then we'd have the "stable" branch up to date, and we could start encouraging people to be *on* stable.
In fact, if this is successful, we may want to encourage developers to start doing non-breaking development and whatnot on stable. I'd like to see us get to a point where we view the stable as stable, not just "released", and we can start to do the more experimental work, like you suggested, in the "yt" branch. But that's a bigger topic.
Thank you for doing this!
-Matt
On Fri, Aug 21, 2015 at 2:32 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi all,
I spent some time this week coming up with a technical solution to our social problem of doing bugfix releases. This is embodied in a new script, "pr_backport.py", which I've created a pull request for here to include with the yt repository.
https://bitbucket.org/yt_analysis/yt/pull-requests/1717/add-pr-backport-scri...
Briefly, the script creates a temporary clone of yt_analysis/yt, figures out which commits on the "yt" branch need to be included on the stable branch, and then assigns those commits to a pull request, and then interactively suggests mercurial commands to run in the temporary repository to backport individual pull requests to the stable branch. This is all done "manually" by copy/pasting mercurial commands suggested by the script. I don't think this should be fully automated since merge conflicts can and likely will be triggered, particularly if the stable and yt branches begin to substantially diverge.
My hope is that this will make the task of backporting bugfixes, which up until this point has been associated with such a high cognitive barrier that no one has wanted to sit down and acutally do it, much easier.
I might be biased since I wrote the script, but using the final version that I pull requested it took me about 10 minutes or so to go through the list of pull requests and backport them to create this pull request:
https://bitbucket.org/yt_analysis/yt/pull-requests/1716/backporting-bugfixes...
Being able to backport bugfixes like this allows us to possibly make some changes to our development workflow and suggested best practices.
1. We can now suggest most users stay on the stable branch rather than switching to the dev branch to get bugfixes. We can also issue bugfix releases on a much quicker cadence (monthly? biweekly?).
2. Since the "stable" branch is more stable and less buggy and the dev branch is basically now just for new features, perhaps we can allow more experimental development on the main yt branch without an onerous requirement of documentation and review. This will help unblock the development of major features like support for unstructured mesh data and the new volume rendering interface.
3. Since there is less pressure to do a feature release just to get bugfixes out, we might be able to have longer intervals between feature releases, giving more time to shake out issues.
I'm more or less just throwing these thoughts out there to get feedback from the development community. Does the easy availability of bugfix releases constitute a significant change for our development and release mindset?
Thanks all for your time and input!
-Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org
participants (3)
-
Cameron Hummels -
Matthew Turk -
Nathan Goldbaum