the Misc/NEWS problem
If we ever want to have a nice workflow where we can automate as much as possible, we need to figure out a way deal with our most common merge conflict: Misc/NEWS. Thanks to shifts in the format between different minor versions the file is pretty much guaranteed to have a conflict when doing a merge up a version. So how do we solve this? I can remember 3 possible solutions that have been proposed previously: 1. A single file per entry 2. A single file per release version of Python 3. Automating it based on commit messages I personally prefer #3 as I hate repeating myself since I just copy and paste the first line(s) of my commits to Misc/NEWS as it is anyway (basically up to the first pair of newlines). We would need a way to signal that the commit message contains nothing useful for the to-be-generated NEWS file when it's simply a fix for a previous commit (probably some marker that is somewhat inconspicuous like a dash on its own line or something). In terms of the section of the NEWS file that a commit belongs, that can once again be a marker or honestly something we drop or infer based on what files were edited in the commit.
On Thursday, August 6, 2015, Brett Cannon <bcannon@gmail.com> wrote:
If we ever want to have a nice workflow where we can automate as much as possible, we need to figure out a way deal with our most common merge conflict: Misc/NEWS. Thanks to shifts in the format between different minor versions the file is pretty much guaranteed to have a conflict when doing a merge up a version.
So how do we solve this? I can remember 3 possible solutions that have been proposed previously:
1. A single file per entry 2. A single file per release version of Python 3. Automating it based on commit messages
I personally prefer #3 as I hate repeating myself since I just copy and paste the first line(s) of my commits to Misc/NEWS as it is anyway (basically up to the first pair of newlines). We would need a way to signal that the commit message contains nothing useful for the to-be-generated NEWS file when it's simply a fix for a previous commit (probably some marker that is somewhat inconspicuous like a dash on its own line or something). In terms of the section of the NEWS file that a commit belongs, that can once again be a marker or honestly something we drop or infer based on what files were edited in the commit.
See also http://bugs.python.org/issue18967, which even has a couple of sample implementations. -- Zach (On an iPad) -- Sent from Gmail Mobile
On 8/5/15 10:43 PM, Zachary Ware wrote:
On Thursday, August 6, 2015, Brett Cannon <bcannon@gmail.com <mailto:bcannon@gmail.com>> wrote:
If we ever want to have a nice workflow where we can automate as much as possible, we need to figure out a way deal with our most common merge conflict: Misc/NEWS. Thanks to shifts in the format between different minor versions the file is pretty much guaranteed to have a conflict when doing a merge up a version.
So how do we solve this? I can remember 3 possible solutions that have been proposed previously:
1. A single file per entry 2. A single file per release version of Python 3. Automating it based on commit messages
I personally prefer #3 as I hate repeating myself since I just copy and paste the first line(s) of my commits to Misc/NEWS as it is anyway (basically up to the first pair of newlines). We would need a way to signal that the commit message contains nothing useful for the to-be-generated NEWS file when it's simply a fix for a previous commit (probably some marker that is somewhat inconspicuous like a dash on its own line or something). In terms of the section of the NEWS file that a commit belongs, that can once again be a marker or honestly something we drop or infer based on what files were edited in the commit.
See also http://bugs.python.org/issue18967, which even has a couple of sample implementations.
Thanks Zach. This issue has interesting reading. The crux of that issue's discussion balances: a) the desire for automation of generating the NEWS file with b) the desire to provide useful information to the users. Something similar to Firefox's approach https://www.mozilla.org/en-US/firefox/39.0/releasenotes/ may be reasonable. Putting implementation and display aside for the moment, the Firefox approach gives: 1) user friendly info on a subset of news/release items and 2) a link to a comprehensive list of changes. Even if one wanted a text file similar to the current NEWS file, one could still take the Firefox approach. Put user friendly highlights for a subset of key issues (which would require some manual intervention though much less than now) and follow that with a comprehensive list of changes using one of the proposed options based on commit messages. Carol
-- Zach (On an iPad)
-- Sent from Gmail Mobile
_______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
-- *Carol Willing* Developer | Willing Consulting https://willingconsulting.com
On 06.08.2015 07:30, Brett Cannon wrote:
If we ever want to have a nice workflow where we can automate as much as possible, we need to figure out a way deal with our most common merge conflict: Misc/NEWS. Thanks to shifts in the format between different minor versions the file is pretty much guaranteed to have a conflict when doing a merge up a version.
So how do we solve this? I can remember 3 possible solutions that have been proposed previously:
1. A single file per entry 2. A single file per release version of Python 3. Automating it based on commit messages
I had mentioned a fourth one: 4. Add a NEWS field to the issue tracker and use it's content for the file entry This is how eGenix does it and it's been working great so far. We simply have our issue tracker generate a report and use this as basis for the change log when cutting a release. The advantage over checkin messages is that you are not stuck if you have multiple checkins for a single issue which should only have a single entry in the change log. Without using the issue tracker, this is similar to approach #2 you mentioned above, but with only creating that file once during the release process instead of patching it up with every single checkin.
I personally prefer #3 as I hate repeating myself since I just copy and paste the first line(s) of my commits to Misc/NEWS as it is anyway (basically up to the first pair of newlines). We would need a way to signal that the commit message contains nothing useful for the to-be-generated NEWS file when it's simply a fix for a previous commit (probably some marker that is somewhat inconspicuous like a dash on its own line or something). In terms of the section of the NEWS file that a commit belongs, that can once again be a marker or honestly something we drop or infer based on what files were edited in the commit.
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 06 2015)
Python Projects, Coaching and Consulting ... http://www.egenix.com/ mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 6 August 2015 at 19:59, M.-A. Lemburg <mal@egenix.com> wrote:
I had mentioned a fourth one:
4. Add a NEWS field to the issue tracker and use it's content for the file entry
This is how eGenix does it and it's been working great so far. We simply have our issue tracker generate a report and use this as basis for the change log when cutting a release.
The advantage over checkin messages is that you are not stuck if you have multiple checkins for a single issue which should only have a single entry in the change log.
Without using the issue tracker, this is similar to approach #2 you mentioned above, but with only creating that file once during the release process instead of patching it up with every single checkin.
This is fairly similar to the way Red Hat does it for downstream products - the docs team figure out the Errata text for end users based on the bug, then that's mapped to the right places accordingly. Eventually pursuing that approach would give us a few rich tiers of information: * full commit logs * commit logs filtered to just be those which mention bug numbers * issue tracker NEWS items mentioned in commit entries * the What's New documentation However, given that we're always going to be able to generate full commit logs, and the filtered logs that mention issues, I think that makes for a good place to start, even if we end up adding a new field to the tracker later. After all, we'll always need to write commit messages regardless of what else we do. Something I *would* like to address up front is how we're going to deal with errors in commit messages (especially cases where we've failed to mention an issue number, or have mentioned the wrong one). However, having a commit log based generator offers a relatively decent way to deal with that: a Misc/NEWS.overrides directory, with filename prefixes based on the commit hashes to be overridden. Dealing with the forward merges will also be necessary - existing projects https://pypi.python.org/pypi/gitchangelog may provide useful ideas on that front. If we wanted to get particularly fancy, commits that added versionadded and versionchanged tags to the documentation could also be automatically highlighted. Regardless, I think one thing I'm figuring out from this discussion is that there *shouldn't be* a NEWS file in the repo itself - the repo will contain the source data, but the rendering can happen separately rather than being done incrementally on each commit. This means that the renderer can be developed in parallel with the current process, with some seed data used to capture the state of the NEWS file as of a particular commit. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Thu, Aug 6, 2015 at 4:07 AM Nick Coghlan <ncoghlan@gmail.com> wrote:
On 6 August 2015 at 19:59, M.-A. Lemburg <mal@egenix.com> wrote:
I had mentioned a fourth one:
4. Add a NEWS field to the issue tracker and use it's content for the file entry
This is how eGenix does it and it's been working great so far. We simply have our issue tracker generate a report and use this as basis for the change log when cutting a release.
The advantage over checkin messages is that you are not stuck if you have multiple checkins for a single issue which should only have a single entry in the change log.
Without using the issue tracker, this is similar to approach #2 you mentioned above, but with only creating that file once during the release process instead of patching it up with every single checkin.
This is fairly similar to the way Red Hat does it for downstream products - the docs team figure out the Errata text for end users based on the bug, then that's mapped to the right places accordingly.
Eventually pursuing that approach would give us a few rich tiers of information:
* full commit logs * commit logs filtered to just be those which mention bug numbers * issue tracker NEWS items mentioned in commit entries * the What's New documentation
However, given that we're always going to be able to generate full commit logs, and the filtered logs that mention issues, I think that makes for a good place to start, even if we end up adding a new field to the tracker later. After all, we'll always need to write commit messages regardless of what else we do.
What is "that"? You forgot to specify which alternative you were proposing to use as an interim to adding a field to the issue tracker.
Something I *would* like to address up front is how we're going to deal with errors in commit messages (especially cases where we've failed to mention an issue number, or have mentioned the wrong one).
Does this mean you are talking about proposal #3 (generating from commit message)?
However, having a commit log based generator offers a relatively decent way to deal with that: a Misc/NEWS.overrides directory, with filename prefixes based on the commit hashes to be overridden.
This is making me prefer MAL's #4 solution.
Dealing with the forward merges will also be necessary - existing projects https://pypi.python.org/pypi/gitchangelog may provide useful ideas on that front.
I would simply ignore merge commits. It's implied that anything fixed in the previous minor release applies to the default branch. Once again, MAL's approach helps with this as we would simply check the Versions field.
If we wanted to get particularly fancy, commits that added versionadded and versionchanged tags to the documentation could also be automatically highlighted.
Regardless, I think one thing I'm figuring out from this discussion is that there *shouldn't be* a NEWS file in the repo itself - the repo will contain the source data, but the rendering can happen separately rather than being done incrementally on each commit. This means that the renderer can be developed in parallel with the current process, with some seed data used to capture the state of the NEWS file as of a particular commit.
After this email I'm now supporting MAL's idea #4. We can make the field required when the Resolution is set to Fixed. I would even potentially argue for a dropdown for What's New inclusion to flag whether a What's New mention is warranted. By making this a required field to set it would greatly simplify the writing of that doc later by having a generated list of all issues that require a mention, effectively creating a TODO list instead of having to hunt through all commits or the entire changelog (and I don't think we can ask non-core contributors to provide What's New text if we want to keep the quality of that doc high).
On Thu, 06 Aug 2015 18:16:53 -0000, Brett Cannon <bcannon@gmail.com> wrote:
However, having a commit log based generator offers a relatively decent way to deal with that: a Misc/NEWS.overrides directory, with filename prefixes based on the commit hashes to be overridden.
This is making me prefer MAL's #4 solution.
I'm liking this solution as well...there's programming work to be done regardless, and adding a tracker field isn't *that* hard. However, the problem Nick points out above is really still an issue we ought to address at some point even if we don't take NEWS info from the commit messages, because it would be nice to be able to do some automated checks about the relationships between issues and commits, and to do that we have to have some way to "edit" the commit messages where we specify the wrong issue number. Could there be some sort of Mercurial solution for that problem? In addition to the "NEWS relevant changes *must* have a tracker issue" (which I agree is good anyway), another thing to keep in mind for the tracker based proposal is that it will make the versions and resolution fields more meaningful, and thus more care must be taken with those fields. I don't see this as a bad thing either, but it is something we need to recognize and deal with if we deploy this solution. Oh, and it does mean that the technique we have sometimes used of removing versions from the list once the fix is committed to that version would have to be changed. So that involves some retraining as well. --David
On Thu, Aug 6, 2015 at 12:46 PM R. David Murray <rdmurray@bitdance.com> wrote:
On Thu, 06 Aug 2015 18:16:53 -0000, Brett Cannon <bcannon@gmail.com> wrote:
However, having a commit log based generator offers a relatively decent way to deal with that: a Misc/NEWS.overrides directory, with filename prefixes based on the commit hashes to be overridden.
This is making me prefer MAL's #4 solution.
I'm liking this solution as well...there's programming work to be done regardless, and adding a tracker field isn't *that* hard.
However, the problem Nick points out above is really still an issue we ought to address at some point even if we don't take NEWS info from the commit messages, because it would be nice to be able to do some automated checks about the relationships between issues and commits, and to do that we have to have some way to "edit" the commit messages where we specify the wrong issue number.
Could there be some sort of Mercurial solution for that problem?
Short or an errata file for commit messages I don't think there is thanks to Mercurial's strong stance on immutable commit data. Does git even allow editing the commit message after the fact?
In addition to the "NEWS relevant changes *must* have a tracker issue" (which I agree is good anyway), another thing to keep in mind for the tracker based proposal is that it will make the versions and resolution fields more meaningful, and thus more care must be taken with those fields. I don't see this as a bad thing either, but it is something we need to recognize and deal with if we deploy this solution.
Yep.
Oh, and it does mean that the technique we have sometimes used of removing versions from the list once the fix is committed to that version would have to be changed. So that involves some retraining as well.
Yes, it would mean either a second field to list what versions the fix has been applied to (eww), or simply having to read through the messages of the issue to keep track of where a fix still need to be applied and not considering it fixed until all versions have received the fix.
On Aug 6, 2015, at 4:06 PM, Brett Cannon <bcannon@gmail.com> wrote:
On Thu, Aug 6, 2015 at 12:46 PM R. David Murray <rdmurray@bitdance.com <mailto:rdmurray@bitdance.com>> wrote: On Thu, 06 Aug 2015 18:16:53 -0000, Brett Cannon <bcannon@gmail.com <mailto:bcannon@gmail.com>> wrote:
However, having a commit log based generator offers a relatively decent way to deal with that: a Misc/NEWS.overrides directory, with filename prefixes based on the commit hashes to be overridden.
This is making me prefer MAL's #4 solution.
I'm liking this solution as well...there's programming work to be done regardless, and adding a tracker field isn't *that* hard.
However, the problem Nick points out above is really still an issue we ought to address at some point even if we don't take NEWS info from the commit messages, because it would be nice to be able to do some automated checks about the relationships between issues and commits, and to do that we have to have some way to "edit" the commit messages where we specify the wrong issue number.
Could there be some sort of Mercurial solution for that problem?
Short or an errata file for commit messages I don't think there is thanks to Mercurial's strong stance on immutable commit data. Does git even allow editing the commit message after the fact?
You can edit it sure, but it will create a new commit and divergent histories so it’s not going to do something generally useful for already pushed commits.
In addition to the "NEWS relevant changes *must* have a tracker issue" (which I agree is good anyway), another thing to keep in mind for the tracker based proposal is that it will make the versions and resolution fields more meaningful, and thus more care must be taken with those fields. I don't see this as a bad thing either, but it is something we need to recognize and deal with if we deploy this solution.
Yep.
Oh, and it does mean that the technique we have sometimes used of removing versions from the list once the fix is committed to that version would have to be changed. So that involves some retraining as well.
Yes, it would mean either a second field to list what versions the fix has been applied to (eww), or simply having to read through the messages of the issue to keep track of where a fix still need to be applied and not considering it fixed until all versions have received the fix. _______________________________________________ core-workflow mailing list core-workflow@python.org <mailto:core-workflow@python.org> https://mail.python.org/mailman/listinfo/core-workflow <https://mail.python.org/mailman/listinfo/core-workflow> This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct <https://www.python.org/psf/codeofconduct>
On 7 August 2015 at 05:46, R. David Murray <rdmurray@bitdance.com> wrote:
On Thu, 06 Aug 2015 18:16:53 -0000, Brett Cannon <bcannon@gmail.com> wrote:
However, having a commit log based generator offers a relatively decent way to deal with that: a Misc/NEWS.overrides directory, with filename prefixes based on the commit hashes to be overridden.
This is making me prefer MAL's #4 solution.
I'm liking this solution as well...there's programming work to be done regardless, and adding a tracker field isn't *that* hard.
However, the problem Nick points out above is really still an issue we ought to address at some point even if we don't take NEWS info from the commit messages, because it would be nice to be able to do some automated checks about the relationships between issues and commits, and to do that we have to have some way to "edit" the commit messages where we specify the wrong issue number.
One of the interesting aspects of Gerrit's workflow is that you get to review the commit message in addition to the change itself, and I think GitHub/BitBucket PRs allow that as well. This means that by the time a change is merged, the correct issue reference will almost always be there. For Beaker, that "the commit message is covered by the review process" aspect let us write https://git.beaker-project.org/cgit/beaker-administrivia/tree/checkbugs.py to audit the Gerrit review state against the corresponding Bugzilla state, and investigate any anomalies to see whether it was the BZ metadata or the Gerrit change proposal commit message that needed updating. (Side note: creating a cpython-administrivia repo for easier discoverability of workflow management utility scripts may not be a bad idea, especially as we start making use of the REST API support being added to Roundup) In the Roundup+Mercurial case, I think we can deal with this more comprehensively by actually storing our canonical issue->commit reference in *Roundup*, and mentioning an issue number in a commit would just be a way of creating those associations automatically. That way, if we get it wrong at commit time (mentioning the wrong issue, not mentioning an issue at all, creating an issue after we've already done the work), then we can edit the commit associations on the affected issues *in Roundup* to ensure the appropriate change info is reported. If multiple commits were made referencing the same issue, then we'd still only get one news entry for that issue, and if one commit addresses multiple issues, then we'd get multiple news entries (one per issue). Since Mercurial associates commits with named branches, this would also mean we'd be able to use the issue->commit tracking to determine where a change had been applied, rather than the existing versions field, so we'd be free to keep using the latter to keep track of branches that still require updating. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Thu, Aug 6, 2015 at 7:17 PM Nick Coghlan <ncoghlan@gmail.com> wrote:
On 7 August 2015 at 05:46, R. David Murray <rdmurray@bitdance.com> wrote:
On Thu, 06 Aug 2015 18:16:53 -0000, Brett Cannon <bcannon@gmail.com> wrote:
However, having a commit log based generator offers a relatively decent way to deal with that: a Misc/NEWS.overrides directory, with filename prefixes based on the commit hashes to be overridden.
This is making me prefer MAL's #4 solution.
I'm liking this solution as well...there's programming work to be done regardless, and adding a tracker field isn't *that* hard.
However, the problem Nick points out above is really still an issue we ought to address at some point even if we don't take NEWS info from the commit messages, because it would be nice to be able to do some automated checks about the relationships between issues and commits, and to do that we have to have some way to "edit" the commit messages where we specify the wrong issue number.
One of the interesting aspects of Gerrit's workflow is that you get to review the commit message in addition to the change itself, and I think GitHub/BitBucket PRs allow that as well. This means that by the time a change is merged, the correct issue reference will almost always be there.
Yes, they both let you edit the commit message before accepting a PR.
For Beaker, that "the commit message is covered by the review process" aspect let us write https://git.beaker-project.org/cgit/beaker-administrivia/tree/checkbugs.py to audit the Gerrit review state against the corresponding Bugzilla state, and investigate any anomalies to see whether it was the BZ metadata or the Gerrit change proposal commit message that needed updating. (Side note: creating a cpython-administrivia repo for easier discoverability of workflow management utility scripts may not be a bad idea, especially as we start making use of the REST API support being added to Roundup)
In the Roundup+Mercurial case, I think we can deal with this more comprehensively by actually storing our canonical issue->commit reference in *Roundup*, and mentioning an issue number in a commit would just be a way of creating those associations automatically. That way, if we get it wrong at commit time (mentioning the wrong issue, not mentioning an issue at all, creating an issue after we've already done the work), then we can edit the commit associations on the affected issues *in Roundup* to ensure the appropriate change info is reported. If multiple commits were made referencing the same issue, then we'd still only get one news entry for that issue, and if one commit addresses multiple issues, then we'd get multiple news entries (one per issue).
So are you saying you want a field to paste in the commit numbers to do the tracking? I assume you would want commit messages listing an issue to automatically append to that field but have it editable to fix any typos in the commit message.
Since Mercurial associates commits with named branches, this would also mean we'd be able to use the issue->commit tracking to determine where a change had been applied, rather than the existing versions field, so we'd be free to keep using the latter to keep track of branches that still require updating.
Yes, and depending on how fancy we got, we could even decorate the Versions field with some marker to show when a commit had been applied to a branch.
On 8 August 2015 at 08:25, Brett Cannon <bcannon@gmail.com> wrote:
On Thu, Aug 6, 2015 at 7:17 PM Nick Coghlan <ncoghlan@gmail.com> wrote:
In the Roundup+Mercurial case, I think we can deal with this more comprehensively by actually storing our canonical issue->commit reference in *Roundup*, and mentioning an issue number in a commit would just be a way of creating those associations automatically. That way, if we get it wrong at commit time (mentioning the wrong issue, not mentioning an issue at all, creating an issue after we've already done the work), then we can edit the commit associations on the affected issues *in Roundup* to ensure the appropriate change info is reported. If multiple commits were made referencing the same issue, then we'd still only get one news entry for that issue, and if one commit addresses multiple issues, then we'd get multiple news entries (one per issue).
So are you saying you want a field to paste in the commit numbers to do the tracking? I assume you would want commit messages listing an issue to automatically append to that field but have it editable to fix any typos in the commit message.
Yep, exactly. Longer term, once we have a repo hosting solution that *also* has a REST API, then the UX could be updated to better integrate the two (e.g. displaying the summary line from the commit in the Roundup UI, display the titles of all linked issues in the repo hosting UI), but the simplest possible way to get started is with a free text comma separated field (akin to the nosy list, dependency and superceder fields) that we can audit with an offline script and set from the existing Roundup notification commit hook.
Since Mercurial associates commits with named branches, this would also mean we'd be able to use the issue->commit tracking to determine where a change had been applied, rather than the existing versions field, so we'd be free to keep using the latter to keep track of branches that still require updating.
Yes, and depending on how fancy we got, we could even decorate the Versions field with some marker to show when a commit had been applied to a branch.
I was more thinking in terms of listing the branches against the displayed commits, but yeah, marking up the versions field might be a nice way to go, too. In my "Can I have a pony?" moments, I start wondering how we might be able to work https://taiga.io/ into the mix to help coordinate more complex activities (like the release process, or some of our infrastructure updates). Fedora now have an instance of that up at http://taiga.cloud.fedoraproject.org/ (that instance requires a Fedora account to log in, but those are freely available: https://admin.fedoraproject.org/accounts/user/new). Baby steps, though, baby steps... :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Thu, Aug 6, 2015 at 7:17 PM Nick Coghlan <ncoghlan@gmail.com> wrote:
In the Roundup+Mercurial case, I think we can deal with this more comprehensively by actually storing our canonical issue->commit reference in *Roundup*, and mentioning an issue number in a commit would just be a way of creating those associations automatically. That way, if we get it wrong at commit time (mentioning the wrong issue, not mentioning an issue at all, creating an issue after we've already done the work), then we can edit the commit associations on the affected issues *in Roundup* to ensure the appropriate change info is reported. If multiple commits were made referencing the same issue, then we'd still only get one news entry for that issue, and if one commit addresses multiple issues, then we'd get multiple news entries (one per issue).
So are you saying you want a field to paste in the commit numbers to do
On Fri, Aug 7, 2015, 20:13 Nick Coghlan <ncoghlan@gmail.com> wrote: On 8 August 2015 at 08:25, Brett Cannon <bcannon@gmail.com> wrote: the
tracking? I assume you would want commit messages listing an issue to automatically append to that field but have it editable to fix any typos in the commit message.
Yep, exactly. Longer term, once we have a repo hosting solution that *also* has a REST API, then the UX could be updated to better integrate the two (e.g. displaying the summary line from the commit in the Roundup UI, display the titles of all linked issues in the repo hosting UI), but the simplest possible way to get started is with a free text comma separated field (akin to the nosy list, dependency and superceder fields) that we can audit with an offline script and set from the existing Roundup notification commit hook. OK, assuming David's in agreement then I think this approach wins with the comma-separated field for commits that the hg hook for Roundup auto-appends to and of course the field to enter the NEWS entry. Now the next question is how easy/hard is it to implement this, how long will it take, and who is willing to do the work? With this in hand we can propose it to python-committers for 3.6 since the NEWS file should be easy enough to back-fill to this approach while its still small. -Brett
Since Mercurial associates commits with named branches, this would also mean we'd be able to use the issue->commit tracking to determine where a change had been applied, rather than the existing versions field, so we'd be free to keep using the latter to keep track of branches that still require updating.
Yes, and depending on how fancy we got, we could even decorate the Versions field with some marker to show when a commit had been applied to a branch.
I was more thinking in terms of listing the branches against the displayed commits, but yeah, marking up the versions field might be a nice way to go, too. In my "Can I have a pony?" moments, I start wondering how we might be able to work https://taiga.io/ into the mix to help coordinate more complex activities (like the release process, or some of our infrastructure updates). Fedora now have an instance of that up at http://taiga.cloud.fedoraproject.org/ (that instance requires a Fedora account to log in, but those are freely available: https://admin.fedoraproject.org/accounts/user/new). Baby steps, though, baby steps... :) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, 08 Aug 2015 17:44:04 -0000, Brett Cannon <bcannon@gmail.com> wrote:
OK, assuming David's in agreement then I think this approach wins with the comma-separated field for commits that the hg hook for Roundup auto-appends to and of course the field to enter the NEWS entry.
Now the next question is how easy/hard is it to implement this, how long will it take, and who is willing to do the work? With this in hand we can propose it to python-committers for 3.6 since the NEWS file should be easy enough to back-fill to this approach while its still small.
Yes I agree this is the best approach, assuming we can get it implemented. The advantage of #4, though, is that Ezio already did the work. I'm *willing* to do the roundup work, but I don't know as I have the required time, at least for the next month. Part of the trick is the need to get a test instance set up...there was work done at PyCon and after at making a tracker-in-a-box, so I'd need to find out where that was at and learn how to use it (or finish it, if needed). The code itself is probably a half-day job, probably including enhancing the hook to update the commits field. But together with getting a working test instance we're talking a couple days work at least. Writing the NEWS generating script is a not exactly trivial job, but probably wants to wait until we have the REST API. So we'd have upgrading our Roundup to that when it lands on the list as well, with a 3.6 Beta 1 deadline on getting it all done. --David
On Sat, Aug 8, 2015 at 1:18 PM R. David Murray <rdmurray@bitdance.com> wrote:
OK, assuming David's in agreement then I think this approach wins with
On Sat, 08 Aug 2015 17:44:04 -0000, Brett Cannon <bcannon@gmail.com> wrote: the
comma-separated field for commits that the hg hook for Roundup auto-appends to and of course the field to enter the NEWS entry.
Now the next question is how easy/hard is it to implement this, how long will it take, and who is willing to do the work? With this in hand we can propose it to python-committers for 3.6 since the NEWS file should be easy enough to back-fill to this approach while its still small.
Yes I agree this is the best approach, assuming we can get it implemented. The advantage of #4, though, is that Ezio already did the work.
I'm *willing* to do the roundup work, but I don't know as I have the required time, at least for the next month. Part of the trick is the need to get a test instance set up...there was work done at PyCon and after at making a tracker-in-a-box, so I'd need to find out where that was at and learn how to use it (or finish it, if needed). The code itself is probably a half-day job, probably including enhancing the hook to update the commits field. But together with getting a working test instance we're talking a couple days work at least.
Perk of getting the tracker-in-a-box working is it's a one-time cost that will be beneficial long term. I also don't think there is a rush since we still have to convince python-committers that this is the right solution. I plan to propose it on Monday to the list to make sure we have buy-in.
Writing the NEWS generating script is a not exactly trivial job, but probably wants to wait until we have the REST API. So we'd have upgrading our Roundup to that when it lands on the list as well, with a 3.6 Beta 1 deadline on getting it all done.
Yeah, so we have a bit of time to worry about the generation script.
I have told python-committers of our plan to make sure it won't lead to a revolt. On Sat, 8 Aug 2015 at 14:25 Brett Cannon <bcannon@gmail.com> wrote:
On Sat, Aug 8, 2015 at 1:18 PM R. David Murray <rdmurray@bitdance.com> wrote:
OK, assuming David's in agreement then I think this approach wins with
On Sat, 08 Aug 2015 17:44:04 -0000, Brett Cannon <bcannon@gmail.com> wrote: the
comma-separated field for commits that the hg hook for Roundup auto-appends to and of course the field to enter the NEWS entry.
Now the next question is how easy/hard is it to implement this, how long will it take, and who is willing to do the work? With this in hand we can propose it to python-committers for 3.6 since the NEWS file should be easy enough to back-fill to this approach while its still small.
Yes I agree this is the best approach, assuming we can get it implemented. The advantage of #4, though, is that Ezio already did the work.
I'm *willing* to do the roundup work, but I don't know as I have the required time, at least for the next month. Part of the trick is the need to get a test instance set up...there was work done at PyCon and after at making a tracker-in-a-box, so I'd need to find out where that was at and learn how to use it (or finish it, if needed). The code itself is probably a half-day job, probably including enhancing the hook to update the commits field. But together with getting a working test instance we're talking a couple days work at least.
Perk of getting the tracker-in-a-box working is it's a one-time cost that will be beneficial long term.
I also don't think there is a rush since we still have to convince python-committers that this is the right solution. I plan to propose it on Monday to the list to make sure we have buy-in.
Writing the NEWS generating script is a not exactly trivial job, but probably wants to wait until we have the REST API. So we'd have upgrading our Roundup to that when it lands on the list as well, with a 3.6 Beta 1 deadline on getting it all done.
Yeah, so we have a bit of time to worry about the generation script.
I forgot to follow up here that there was nary a peep from python-committers about the proposal of keeping NEWS entries in the issue tracker, so we have a green light to implement this idea. On Mon, Aug 10, 2015, 15:39 Brett Cannon <bcannon@gmail.com> wrote:
I have told python-committers of our plan to make sure it won't lead to a revolt.
On Sat, 8 Aug 2015 at 14:25 Brett Cannon <bcannon@gmail.com> wrote:
On Sat, Aug 8, 2015 at 1:18 PM R. David Murray <rdmurray@bitdance.com> wrote:
OK, assuming David's in agreement then I think this approach wins with
On Sat, 08 Aug 2015 17:44:04 -0000, Brett Cannon <bcannon@gmail.com> wrote: the
comma-separated field for commits that the hg hook for Roundup auto-appends to and of course the field to enter the NEWS entry.
Now the next question is how easy/hard is it to implement this, how long will it take, and who is willing to do the work? With this in hand we can propose it to python-committers for 3.6 since the NEWS file should be easy enough to back-fill to this approach while its still small.
Yes I agree this is the best approach, assuming we can get it implemented. The advantage of #4, though, is that Ezio already did the work.
I'm *willing* to do the roundup work, but I don't know as I have the required time, at least for the next month. Part of the trick is the need to get a test instance set up...there was work done at PyCon and after at making a tracker-in-a-box, so I'd need to find out where that was at and learn how to use it (or finish it, if needed). The code itself is probably a half-day job, probably including enhancing the hook to update the commits field. But together with getting a working test instance we're talking a couple days work at least.
Perk of getting the tracker-in-a-box working is it's a one-time cost that will be beneficial long term.
I also don't think there is a rush since we still have to convince python-committers that this is the right solution. I plan to propose it on Monday to the list to make sure we have buy-in.
Writing the NEWS generating script is a not exactly trivial job, but probably wants to wait until we have the REST API. So we'd have upgrading our Roundup to that when it lands on the list as well, with a 3.6 Beta 1 deadline on getting it all done.
Yeah, so we have a bit of time to worry about the generation script.
On Sun, Aug 9, 2015 at 12:25 AM, Brett Cannon <bcannon@gmail.com> wrote:
On Sat, Aug 8, 2015 at 1:18 PM R. David Murray <rdmurray@bitdance.com> wrote:
On Sat, 08 Aug 2015 17:44:04 -0000, Brett Cannon <bcannon@gmail.com> wrote:
OK, assuming David's in agreement then I think this approach wins with the comma-separated field for commits that the hg hook for Roundup auto-appends to and of course the field to enter the NEWS entry.
Now the next question is how easy/hard is it to implement this, how long will it take, and who is willing to do the work? With this in hand we can propose it to python-committers for 3.6 since the NEWS file should be easy enough to back-fill to this approach while its still small.
Yes I agree this is the best approach, assuming we can get it implemented. The advantage of #4, though, is that Ezio already did the work.
I'm *willing* to do the roundup work, but I don't know as I have the required time, at least for the next month. Part of the trick is the need to get a test instance set up...there was work done at PyCon and after at making a tracker-in-a-box, so I'd need to find out where that was at and learn how to use it (or finish it, if needed). The code itself is probably a half-day job, probably including enhancing the hook to update the commits field. But together with getting a working test instance we're talking a couple days work at least.
Perk of getting the tracker-in-a-box working is it's a one-time cost that will be beneficial long term.
FYI Shiyao (one of the GSoC students) has been working on https://bitbucket.org/introom/docker-b.p.o/src . Best Regards, Ezio Melotti
I also don't think there is a rush since we still have to convince python-committers that this is the right solution. I plan to propose it on Monday to the list to make sure we have buy-in.
Writing the NEWS generating script is a not exactly trivial job, but probably wants to wait until we have the REST API. So we'd have upgrading our Roundup to that when it lands on the list as well, with a 3.6 Beta 1 deadline on getting it all done.
Yeah, so we have a bit of time to worry about the generation script.
On Tue, 11 Aug 2015 07:12:38 +0300, Ezio Melotti <ezio.melotti@gmail.com> wrote:
On Sun, Aug 9, 2015 at 12:25 AM, Brett Cannon <bcannon@gmail.com> wrote:
On Sat, Aug 8, 2015 at 1:18 PM R. David Murray <rdmurray@bitdance.com> wrote:
On Sat, 08 Aug 2015 17:44:04 -0000, Brett Cannon <bcannon@gmail.com> wrote:
OK, assuming David's in agreement then I think this approach wins with the comma-separated field for commits that the hg hook for Roundup auto-appends to and of course the field to enter the NEWS entry.
Now the next question is how easy/hard is it to implement this, how long will it take, and who is willing to do the work? With this in hand we can propose it to python-committers for 3.6 since the NEWS file should be easy enough to back-fill to this approach while its still small.
Yes I agree this is the best approach, assuming we can get it implemented. The advantage of #4, though, is that Ezio already did the work.
I'm *willing* to do the roundup work, but I don't know as I have the required time, at least for the next month. Part of the trick is the need to get a test instance set up...there was work done at PyCon and after at making a tracker-in-a-box, so I'd need to find out where that was at and learn how to use it (or finish it, if needed). The code itself is probably a half-day job, probably including enhancing the hook to update the commits field. But together with getting a working test instance we're talking a couple days work at least.
Perk of getting the tracker-in-a-box working is it's a one-time cost that will be beneficial long term.
Yes. I just have to find time to get over that initial hump :)
FYI Shiyao (one of the GSoC students) has been working on https://bitbucket.org/introom/docker-b.p.o/src .
And this will help, I hope! --David
Hi,
Yes. I just have to find time to get over that initial hump :)
FYI Shiyao (one of the GSoC students) has been working on https://bitbucket.org/introom/docker-b.p.o/src .
And this will help, I hope!
Let me see if I can do anything to help on this. Take care that the oic package introduces some troublesome package and the roundup will not work. Regards. -- 吾輩は猫である。ホームーページはhttps://introo.me。
On 8/11/15 7:36 AM, Shiyao Ma wrote:
Hi,
Yes. I just have to find time to get over that initial hump :)
FYI Shiyao (one of the GSoC students) has been working on https://bitbucket.org/introom/docker-b.p.o/src . And this will help, I hope! Let me see if I can do anything to help on this.
Take care that the oic package introduces some troublesome package and the roundup will not work.
Regards.
Shiyao, Keep up the good work for the home stretch of GSoC. I would be interested in reading your end of project report. Ezio, Thanks for mentoring GSoC. I'm also mentoring a student for another organization. Please let me know if you need any help with docs of these projects after GSoC wraps up. Warmly, Carol
On Thu, Aug 6, 2015 at 2:59 AM M.-A. Lemburg <mal@egenix.com> wrote:
On 06.08.2015 07:30, Brett Cannon wrote:
If we ever want to have a nice workflow where we can automate as much as possible, we need to figure out a way deal with our most common merge conflict: Misc/NEWS. Thanks to shifts in the format between different minor versions the file is pretty much guaranteed to have a conflict when doing a merge up a version.
So how do we solve this? I can remember 3 possible solutions that have been proposed previously:
1. A single file per entry 2. A single file per release version of Python 3. Automating it based on commit messages
I had mentioned a fourth one:
4. Add a NEWS field to the issue tracker and use it's content for the file entry
This is how eGenix does it and it's been working great so far. We simply have our issue tracker generate a report and use this as basis for the change log when cutting a release.
It also takes care of categorization since we can just grab that from the Components field.
The advantage over checkin messages is that you are not stuck if you have multiple checkins for a single issue which should only have a single entry in the change log.
Without using the issue tracker, this is similar to approach #2 you mentioned above, but with only creating that file once during the release process instead of patching it up with every single checkin.
The only drawback I see to this is it will require an issue for every change that should go into the changelog. Now I actually think that's a good thing, but I can see some pushback from python-dev by those who don't want the hassle. But I would argue that any change significant enough to warrant an entry in the changelog warrants an issue to track it. -Brett
I personally prefer #3 as I hate repeating myself since I just copy and paste the first line(s) of my commits to Misc/NEWS as it is anyway (basically up to the first pair of newlines). We would need a way to signal that the commit message contains nothing useful for the to-be-generated NEWS file when it's simply a fix for a previous commit (probably some marker that is somewhat inconspicuous like a dash on its own line or something). In terms of the section of the NEWS file that a commit belongs, that can once again be a marker or honestly something we drop or infer based on what files were edited in the commit.
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Aug 06 2015)
Python Projects, Coaching and Consulting ... http://www.egenix.com/ mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 08/05/2015 10:30 PM, Brett Cannon wrote:
If we ever want to have a nice workflow where we can automate as much as possible, we need to figure out a way deal with our most common merge conflict: Misc/NEWS. Thanks to shifts in the format between different minor versions the file is pretty much guaranteed to have a conflict when doing a merge up a version.
So how do we solve this? I can remember 3 possible solutions that have been proposed previously:
1. A single file per entry 2. A single file per release version of Python 3. Automating it based on commit messages
I personally prefer #3 as I hate repeating myself since I just copy and paste the first line(s) of my commits to Misc/NEWS as it is anyway (basically up to the first pair of newlines). We would need a way to signal that the commit message contains nothing useful for the to-be-generated NEWS file when it's simply a fix for a previous commit (probably some marker that is somewhat inconspicuous like a dash on its own line or something). In terms of the section of the NEWS file that a commit belongs, that can once again be a marker or honestly something we drop or infer based on what files were edited in the commit.
(I'm a bit late to the party here, sorry). You have another solution (5) having a dedicated merge script for misc/NEWS that knows how to parse the format and generate a good merge. It should not be too complicated to achieve. I've successfully used such script for similarish file in the past. The main issue of (3) is that you will make mistake. Either wrong commit message or change flagged as News worthy but later backed out or changed so much than the initial message is irrelevant. (Mercurial will have not problem editing commit message as long as they did not made it to the main public repo, however be sure that you will let mistake slip through). -- Pierre-Yves David
On Sat, Aug 8, 2015 at 8:25 AM, Pierre-Yves David <pierre-yves.david@ens-lyon.org> wrote:
On 08/05/2015 10:30 PM, Brett Cannon wrote:
If we ever want to have a nice workflow where we can automate as much as possible, we need to figure out a way deal with our most common merge conflict: Misc/NEWS. Thanks to shifts in the format between different minor versions the file is pretty much guaranteed to have a conflict when doing a merge up a version.
So how do we solve this? I can remember 3 possible solutions that have been proposed previously:
1. A single file per entry 2. A single file per release version of Python 3. Automating it based on commit messages
I personally prefer #3 as I hate repeating myself since I just copy and paste the first line(s) of my commits to Misc/NEWS as it is anyway (basically up to the first pair of newlines). We would need a way to signal that the commit message contains nothing useful for the to-be-generated NEWS file when it's simply a fix for a previous commit (probably some marker that is somewhat inconspicuous like a dash on its own line or something). In terms of the section of the NEWS file that a commit belongs, that can once again be a marker or honestly something we drop or infer based on what files were edited in the commit.
(I'm a bit late to the party here, sorry).
You have another solution (5) having a dedicated merge script for misc/NEWS that knows how to parse the format and generate a good merge. It should not be too complicated to achieve. I've successfully used such script for similarish file in the past.
A merge script is already attached to the aforementioned issue: see http://bugs.python.org/issue18967#msg217079 and http://bugs.python.org/file35032/newsmerge.py I think you were the one who convinced and helped me implementing it :) Best Regards, Ezio Melotti
The main issue of (3) is that you will make mistake. Either wrong commit message or change flagged as News worthy but later backed out or changed so much than the initial message is irrelevant.
(Mercurial will have not problem editing commit message as long as they did not made it to the main public repo, however be sure that you will let mistake slip through).
-- Pierre-Yves David
On 8 August 2015 at 22:05, Ezio Melotti <ezio.melotti@gmail.com> wrote:
On Sat, Aug 8, 2015 at 8:25 AM, Pierre-Yves David <pierre-yves.david@ens-lyon.org> wrote:
You have another solution (5) having a dedicated merge script for misc/NEWS that knows how to parse the format and generate a good merge. It should not be too complicated to achieve. I've successfully used such script for similarish file in the past.
A merge script is already attached to the aforementioned issue: see http://bugs.python.org/issue18967#msg217079 and http://bugs.python.org/file35032/newsmerge.py I think you were the one who convinced and helped me implementing it :)
Right, the merge script would be a good option if we were otherwise happy with the checked-in NEWS file as the data entry format for user facing messages. We aren't particularly happy with that though, so it makes sense to tackle the larger problem. That said, the merge script might still be a good tactical solution in the interim to alleviate some of the current irritations in merging forward bug fixes. In relation to the bugs.python.org based approach, it occurs to me that we could move patch contributor acknowledgements for merged patches there as well, so they can be more readily queried programmatically than they are today. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, Aug 8, 2015, 07:40 Nick Coghlan <ncoghlan@gmail.com> wrote: On 8 August 2015 at 22:05, Ezio Melotti <ezio.melotti@gmail.com> wrote:
On Sat, Aug 8, 2015 at 8:25 AM, Pierre-Yves David <pierre-yves.david@ens-lyon.org> wrote:
You have another solution (5) having a dedicated merge script for misc/NEWS that knows how to parse the format and generate a good merge. It should not be too complicated to achieve. I've successfully used such script for similarish file in the past.
A merge script is already attached to the aforementioned issue: see http://bugs.python.org/issue18967#msg217079 and http://bugs.python.org/file35032/newsmerge.py I think you were the one who convinced and helped me implementing it :)
Right, the merge script would be a good option if we were otherwise happy with the checked-in NEWS file as the data entry format for user facing messages. We aren't particularly happy with that though, so it makes sense to tackle the larger problem. That said, the merge script might still be a good tactical solution in the interim to alleviate some of the current irritations in merging forward bug fixes. In relation to the bugs.python.org based approach, it occurs to me that we could move patch contributor acknowledgements for merged patches there as well, so they can be more readily queried programmatically than they are today. One thing at a time, but yes. Once we have push button merges we can easily record who contributed the patch to make sure ACKS is always up-to-date.
participants (10)
-
Brett Cannon -
Carol Willing -
Donald Stufft -
Ezio Melotti -
M.-A. Lemburg -
Nick Coghlan -
Pierre-Yves David -
R. David Murray -
Shiyao Ma -
Zachary Ware