Roundup improvements use-cases

Hello, in the last few weeks I have been working on defining use-cases which will lead the improvement of the Roundup tracker. As this is very important, I would like your valuable input in form of comments, criticism and advices. Use-cases in-lined: ============== Daniel is a great hacker, but the only time he has time to hack is when his child is asleep. And his child wakes up every now and then, pretty often actually. So when it starts crying, Daniel commits fix to that silly bug that has been hunting us for ages, but forgets to close down the bug so it stays open. Looking over bug tracker, Jane finds the same bug and starts working on the same bug. Wasted time. Daniel should be able to format commit message in such a way that would automatically close the bug. Maria is a student, and got involved in FOSS recently. With all the studies she has to do, she would prefer not to spend her time changing bug status. She should be able to format commit message in such a way that would change bug status to something other then the current status. Technical talk: USE CASE A: Integrate issue property editing with Mercurial, including: * USE CASE A1: Allow users to close issues via tokens embedded into commit messages. * USE CASE A2: Allow users to change issue status via tokens embedded into commit messages. Ronny is about to fix a torny bug, so he has a public branch for that. He is making great progress, but the issue that tracks the bug only contains a very out-of-date patch, prompting other developers to try to fix the same bug. Ronny should be able to tell Roundup about where his code lives, so users can get up-to-date patches/diffs automatically. This also allows other users to know all the code that changed to fix a given bug. Technical talk: Integrate branch and rich patch handling into Roundup * USE CASE B: Track all changesets that relate to a given issue: o USE CASE B1: Using tokens embedded into commit messages. (Post commit) o USE CASE B2: Using named branches, bookmarks. (Pre or post commit) o USE CASE B3: Using patchsets, bundles or whatchacallit for fat Mercurial patches. (Pre or post commit) Brett wants to fix a couple of related issues and has a local Mercurial branch for that. He would like his commit messages to include useful information for when his patch/branch lands in the Python repository. Besides the Mercurial->Roundup integration, a Roundup->Mercurial one that would allow one to fetch issue details and existing patches/branches with metadata would make Brett's work more valuable. Technical talk: USE CASE C: Add a CLI interface for Roundup so VCSs can query the tracker. * USE CASE C1: Automatically fetch issue data. * USE CASE C2: Pre-format output for greater usefulness in commit messages: o USE CASE C2.1: Setting issue properties. o USE CASE C2.2: Grouping changesets. * USE CASE C3: Fetch branch information for local cloning. * USE CASE C4: Add a Mercurial extension to exercize the CLI client. USE CASE D: Antoine is merging lots of branches and applying lots of patches to his local branch of the Python repository. He goes to each issue with a patch/branch and tells people whether their patches apply cleanly, just to avoid merge issues in the main branch. While he could use Mercurial Patch Queues (mq), Roundup would need to learn to both listen to and to submit patches to mq in order to completely replace Antoine's work with automated tools. Having a quick 'check if this patch applies cleanly' button would make triaging issues much easier. USE CASE E: David is checking the python-commits list for avoiding bad code from landing and nesting in the Python code base. He only sees the patches: while informative, it requires a bit of mental gymanstics to see how it would merge with the surrounding code. Whenever he thinks some code has tabs and spaces or lines that are too long, he runs pylint and reindent.py locally. He can only raise concerns about the code after it lands on the main repository. It should be easier to see the code changes in context. Having a way to check the code for mistakes in the tracker would make David's life a lot easier. USE CASE F: Van Lindberg is concerned about code submissions from non-core developers: how can the PSF re-license this code in the future without talking to each contributor, whether the PSF is safe from litigation based on copyrights of these contributions and related questions are always bugging him. While the PSF has Contributor Agreements exactly for these cases, it would be great to have the issue tracker and the VCS talk to each other, so they could ask contributors to sign (or declare to have already signed) the CAs when necessary. USE CASE G: Use Transplant/Patch Branch to generate patches from branches linked from Roundup. USE CASE J: Integrate the code/commits navigation interface with Roundup, so changesets, branches, etc., can be easily linked/browsed (starting) from the Roundup UI and issues can be created/linked to commits (starting) from the navigation tool UI. USE CASE K: For a given issue, add per patched file links for RSS logs (see http://selenic.com/hg/rss-log/tip/mercurial/hgweb/hgweb_mod.py). USE CASE M: Besides links to files, allow adding links to files at given versions/tags/branchs, links to tarballs and easy to clone links to branches and repositories. USE CASE V: Handle small branches (and maybe suggest using them for small patches?) generated using the convert extension with --filemap. ============== Improvements to the use cases will follow on the wiki page: http://www.roundup-tracker.org/cgi-bin/moin.cgi/GSoC09 Thanks for your attention and time. Respectfully, Mario

Mario wrote:
USE CASE F:
Van Lindberg is concerned about code submissions from non-core developers: how can the PSF re-license this code in the future without talking to each contributor, whether the PSF is safe from litigation based on copyrights of these contributions and related questions are always bugging him. While the PSF has Contributor Agreements exactly for these cases, it would be great to have the issue tracker and the VCS talk to each other, so they could ask contributors to sign (or declare to have already signed) the CAs when necessary.
+1. This is exactly right.

Hi Mario, thanks for sending these use-cases out. Let me give some feedback: Mario wrote:
Technical talk: USE CASE A: Integrate issue property editing with Mercurial, including:
* USE CASE A1: Allow users to close issues via tokens embedded into commit messages. * USE CASE A2: Allow users to change issue status via tokens embedded into commit messages.
This is the most fundamental one, and it is implementable without even touching Roundup itself. All that is required is a 'post-commit-hook' (whatever that is spelled for a particular VCS) that sends a mail to a tracker, if some tokens were found in the commit message. While in theory this is all up for customization, I think it would be nice to set up some conventions to guide users in how to do this (the 'token grammar', mostly), just for convenience.
Ronny is about to fix a torny bug, so he has a public branch for that. He is making great progress, but the issue that tracks the bug only contains a very out-of-date patch, prompting other developers to try to fix the same bug. Ronny should be able to tell Roundup about where his code lives, so users can get up-to-date patches/diffs automatically. This also allows other users to know all the code that changed to fix a given bug.
Technical talk: Integrate branch and rich patch handling into Roundup
* USE CASE B: Track all changesets that relate to a given issue: o USE CASE B1: Using tokens embedded into commit messages. (Post commit) o USE CASE B2: Using named branches, bookmarks. (Pre or post commit) o USE CASE B3: Using patchsets, bundles or whatchacallit for fat Mercurial patches. (Pre or post commit)
Hmm, I have difficulties mapping your prosaic description of the use case above to the 'technical talk'. The latter almost sounds like an implementation strategy to me. This is a very interesting use case, driving us at the core of the 'VCS integration'. We need to customize a schema to provide a notion for 'repository' and 'branch', and also for 'patch'. Then we need to somehow endow the patch type with knowledge of what it means to be 'applied' to a branch, etc. (more on this below).
Brett wants to fix a couple of related issues and has a local Mercurial branch for that. He would like his commit messages to include useful information for when his patch/branch lands in the Python repository. Besides the Mercurial->Roundup integration, a Roundup->Mercurial one that would allow one to fetch issue details and existing patches/branches with metadata would make Brett's work more valuable.
Technical talk: USE CASE C: Add a CLI interface for Roundup so VCSs can query the tracker.
* USE CASE C1: Automatically fetch issue data. * USE CASE C2: Pre-format output for greater usefulness in commit messages: o USE CASE C2.1: Setting issue properties. o USE CASE C2.2: Grouping changesets. * USE CASE C3: Fetch branch information for local cloning. * USE CASE C4: Add a Mercurial extension to exercize the CLI client.
This set of use cases ('C') is what I have most difficulties with. It clearly is about 'VCS <-> Roundup integration', but it's in the other direction. And thus, this doesn't seem to concern Roundup itself, or does it ? Presumably, everything needed already exists. We now have an XMLRPC interface, so it is possible to write (remote) Roundup clients. I think writing such a 'Roundup client library' would be useful, but I think somewhat out of scope with respect to this GSoC project.
USE CASE D:
Antoine is merging lots of branches and applying lots of patches to his local branch of the Python repository. He goes to each issue with a patch/branch and tells people whether their patches apply cleanly, just to avoid merge issues in the main branch. While he could use Mercurial Patch Queues (mq), Roundup would need to learn to both listen to and to submit patches to mq in order to completely replace Antoine's work with automated tools. Having a quick 'check if this patch applies cleanly' button would make triaging issues much easier.
Let's drill down here a little, as this may be more than one use-case: D1: provide an interface through which users can attempt to apply a submitted patch to a given branch. This requires a 'working copy' of the code (Roundup may be configured to create that if it doesn't exist yet), and a simple interface to the 'patch' tool. If we create a new 'Patch' type, it may include metadata that help Roundup to figure out what repo / branch URL to use to fetch the code from against which the patch is to be applied against. D2: While D1 is about applying patches, this may not be enough to figure out whether the patch is good. The user may also want to attempt to build the code (and even run tests). While I believe this to be outside the realm of a bug / patch tracker, I can see good use in integration with external build tools. In fact, it is a use case such as this that I had in mind when alluding to bridging with buildbot. I think it would be nice to think a little more about it, without actually implementing it. Just making it possible for future work in this direction ought to be enough.
USE CASE E:
David is checking the python-commits list for avoiding bad code from landing and nesting in the Python code base. He only sees the patches: while informative, it requires a bit of mental gymanstics to see how it would merge with the surrounding code. Whenever he thinks some code has tabs and spaces or lines that are too long, he runs pylint and reindent.py locally. He can only raise concerns about the code after it lands on the main repository. It should be easier to see the code changes in context. Having a way to check the code for mistakes in the tracker would make David's life a lot easier.
Yes, if we create a new 'Patch' type, we also want to add to the templating module to provide rendering functions that can render the patch in different ways (diffs between the target branch, with and without the patch applied, etc.) Again, 'checking the code for mistakes' can be a big task, and so goes into the D2 case above.
USE CASE F:
Van Lindberg is concerned about code submissions from non-core developers: how can the PSF re-license this code in the future without talking to each contributor, whether the PSF is safe from litigation based on copyrights of these contributions and related questions are always bugging him. While the PSF has Contributor Agreements exactly for these cases, it would be great to have the issue tracker and the VCS talk to each other, so they could ask contributors to sign (or declare to have already signed) the CAs when necessary.
Sounds good. Nothing new here (besides what we already described in the use cases above).
USE CASE G: Use Transplant/Patch Branch to generate patches from branches linked from Roundup.
This sounds like accessing a VCS-specific GUI through Roundup. While I can see this being useful, I'm not sure how much this relates directly to Roundup.
USE CASE J:
Integrate the code/commits navigation interface with Roundup, so changesets, branches, etc., can be easily linked/browsed (starting) from the Roundup UI and issues can be created/linked to commits (starting) from the navigation tool UI.
Similarly here.
USE CASE K: For a given issue, add per patched file links for RSS logs (see http://selenic.com/hg/rss-log/tip/mercurial/hgweb/hgweb_mod.py).
This sounds like a small addition to case A above: Once I collect (references to) changesets in an issue, I can use them to backlink into a repository UI.
USE CASE M: Besides links to files, allow adding links to files at given versions/tags/branchs, links to tarballs and easy to clone links to branches and repositories.
Right. Easy enough once our schema knows about 'repo', 'branch' and 'revision' (We still need a per VCS definition of how to construct such a link from this tuple.
USE CASE V:
Handle small branches (and maybe suggest using them for small patches?) generated using the convert extension with --filemap.
I have no idea what this is about, sorry. :-)
==============
Improvements to the use cases will follow on the wiki page: http://www.roundup-tracker.org/cgi-bin/moin.cgi/GSoC09
Great, thanks. I suggest that we keep discussions here on the list(s), and then capture any outcome on that (and related) wiki page(s).
Thanks for your attention and time.
Thanks for working on this ! Stefan -- ...ich hab' noch einen Koffer in Berlin...

Stefan Seefeld <seefeld@sympatico.ca> writes:
While in theory [implementing a hook for closing bugs via VCS commit messages] is all up for customization, I think it would be nice to set up some conventions to guide users in how to do this (the 'token grammar', mostly), just for convenience.
A reference implementation that I think works quite well is the system used by the Debian project for detecting, from the changelog entries, which bugs to close when a particular release of a package is uploaded. <URL:http://www.debian.org/doc/manuals/developers-reference/pkgs.html#upload-bugf...> Note also that this system doesn't just close the bug, it makrs it as fixed in a specific version of the package: the package version that included the close request in its changelog entry. It would be valuable if analogous behaviour could occur with Roundup, which requires allowing for it in the specification of the workflow.
This is a very interesting use case, driving us at the core of the 'VCS integration'. We need to customize a schema to provide a notion for 'repository' and 'branch', and also for 'patch'. Then we need to somehow endow the patch type with knowledge of what it means to be 'applied' to a branch, etc. (more on this below).
All this would apply to the above workflow for “mark that bug as fixed by this specific branch”. (good sigmonster, have a cookie) -- \ “That's all very good in practice, but how does it work in | `\ *theory*?” —anonymous | _o__) | Ben Finney

Stefan Seefeld <seefeld@sympatico.ca> writes:
This set of use cases ('C') is what I have most difficulties with. It clearly is about 'VCS <-> Roundup integration', but it's in the other direction. And thus, this doesn't seem to concern Roundup itself, or does it ? Presumably, everything needed already exists. We now have an XMLRPC interface, so it is possible to write (remote) Roundup clients. I think writing such a 'Roundup client library' would be useful, but I think somewhat out of scope with respect to this GSoC project.
Note that there are several attempts in the wild at “BTS database as versioned data in the VCS repository”, allowing the tightest integration between the two and ensuring that VCS commits and BTS updates are inherently tied to each other. An article at Linux Weekly News <URL:http://lwn.net/Articles/281849/> describes these systems. It might be instructive for those looking to tightly integrate VCS and BTS to seek inspiration from these designs. -- \ “The userbase for strong cryptography declines by half with | `\ every additional keystroke or mouseclick required to make it | _o__) work.” —Carl Ellison | Ben Finney
participants (4)
-
Ben Finney
-
Mario
-
Stefan Seefeld
-
VanL