
Hi, I added a new "stats" page to the bug tracker: http://bugs.python.org/issue?@template=stats The page can be reached from the sidebar of the bug tracker: Summaries -> Stats The data are updated once a week, together with the Summary of Python tracker issues. Best Regards, Ezio Melotti

Two questions: how hard would be to add (or enhance) a chart with the “open issues type enhancement” and “open issues type bug” info ? In the summaries there is a link to “Issues with patch”, means that the ones not listed there are in “needs patch” or “new” status? Regards, francis

On Mon, 23 Jun 2014 17:52:33 +0200, francis <francismb@email.de> wrote:
The stats graphs are based on the data generated for the weekly issue report. I have a patched version of that report that adds the bug/enhancement info. I'll try to dig it up this week; someone ping me if I forget :) It think the patch will need to be updated based on Ezio's changes. --David

On Mon, Jun 23, 2014 at 04:12:24PM -0400, R. David Murray wrote:
After PyCon, I started working on a scraper that would produce a bunch of different lists and charts. My ideas were: * pie charts of issues by status and type. * list or histogram of open library issues by module, perhaps limited to the top N modules * list of N oldest issues with no subsequent activity (the unreviewed ones) * list of N people with the most open issues assigned to them The idea is to provide charts that help us direct effort to particular subsets of bugs. --amk

On Tue, Jun 24, 2014 at 1:25 AM, A.M. Kuchling <amk@amk.ca> wrote:
We don't have module-specific tags yet (see the core-workflow ML for discussions about that), but I have other scripts that analyze all the patches and divide them by module. I didn't have time to integrate this in the tracker though.
* list of N oldest issues with no subsequent activity (the unreviewed ones)
You can search for issues with only one message: http://bugs.python.org/issue?%40sort0=activity&%40sort1=&%40group0=&%40group1=&%40columns=title%2Cid%2Cactivity%2Cstatus&%40filter=status%2Cmessage_count&status=1&message_count=1&%40pagesize=50&%40startwith=0
* list of N people with the most open issues assigned to them
And then poke them with a goad until they fix them? :)
The idea is to provide charts that help us direct effort to particular subsets of bugs.
If someone wants to experiment with and/or improve the tracker stats, this is how it works: 1) The roundup-summary script [0] analyzes the issues once a week and produce the weekly report and a static JSON file [1]; 2) The stats page [2] request the JSON file and uses the data to generate the charts client-side. Now there are two ways to improve it: 1) the easy way is just to use the roundup-summary script to expose more of its data or to find new ones and add them to the JSON file (and possibly to the summary too); 2) the hard way is to decouple the roundup-summary and the stats page and either make another weekly (or daily/hourly) script to generate the JSON file, or a template page that generates the data in real-time. Once the data are in the JSON file is quite easy to use jqPlot [4] to make any kind of charts. Keep in mind that some things are trivial to get out from the DB (e.g. number of issues for each status/type), but other things are a bit more complicated (e.g. things involving specific periods of time) and currently the roundup-summary takes a few minutes to analyze all the issues. I also tried to include just a few useful charts on the stats page -- at first I had several more charts but then I removed them. Feel free to ping me on IRC (#python-dev@Freenode) if you have questions. Best Regards, Ezio Melotti [0]: http://hg.python.org/tracker/python-dev/file/default/scripts/roundup-summary [1]: http://bugs.python.org/@@file/issue.stats.json [2]: http://hg.python.org/tracker/python-dev/file/bbbe6c190a99/html/issue.stats.h... [3]: http://www.jqplot.com/tests/
--amk

On Mon, Jun 23, 2014 at 6:52 PM, francis <francismb@email.de> wrote:
Not particularly hard, but I won't have time to get back to this project for a while (contributions are welcomed though!).
That summary lists all the issues with the "patch" keyword, and the ones not listed simply don't have it. The keyword is added automatically whenever an attachment is added to the issue, so there might be false positives (e.g. if the attachment is a script to reproduce the issue rather than a patch, or if the available patches are outdated). The might also be issues with patches that are not included in the summary (e.g. if someone accidentally removed the keyword), but that shouldn't be very common.
Best Regards, Ezio Melotti
Regards, francis

On Tue, 24 Jun 2014 20:43:41 +0200, francis <francismb@email.de> wrote:
Anyone can review patches, in case that isn't clear.
more committers: ?? automate!! :-)
That's a goal of the python-workflow interest group. Unfortunately between billable work and GSOC mentoring I haven't had time to do much there lately. Our first goal is to make the review step easier to manage (know which patches really need review, be able to list patches where community review is thought to be complete) by improving the tracker, then we'll look at creating the patch gating system Nick has talked about previously. Still needs a committer to approve the patch, but it should increase the throughput considerably. In the meantime, something that would help would be if people would do reviews and say on the issue "I think this is commit ready" and have the issue moved to 'commit review' stage. Do that a few times where people who are already triagers/committers agree with you, and you'll get triage privileges on the tracker. --David

Two questions: how hard would be to add (or enhance) a chart with the “open issues type enhancement” and “open issues type bug” info ? In the summaries there is a link to “Issues with patch”, means that the ones not listed there are in “needs patch” or “new” status? Regards, francis

On Mon, 23 Jun 2014 17:52:33 +0200, francis <francismb@email.de> wrote:
The stats graphs are based on the data generated for the weekly issue report. I have a patched version of that report that adds the bug/enhancement info. I'll try to dig it up this week; someone ping me if I forget :) It think the patch will need to be updated based on Ezio's changes. --David

On Mon, Jun 23, 2014 at 04:12:24PM -0400, R. David Murray wrote:
After PyCon, I started working on a scraper that would produce a bunch of different lists and charts. My ideas were: * pie charts of issues by status and type. * list or histogram of open library issues by module, perhaps limited to the top N modules * list of N oldest issues with no subsequent activity (the unreviewed ones) * list of N people with the most open issues assigned to them The idea is to provide charts that help us direct effort to particular subsets of bugs. --amk

On Tue, Jun 24, 2014 at 1:25 AM, A.M. Kuchling <amk@amk.ca> wrote:
We don't have module-specific tags yet (see the core-workflow ML for discussions about that), but I have other scripts that analyze all the patches and divide them by module. I didn't have time to integrate this in the tracker though.
* list of N oldest issues with no subsequent activity (the unreviewed ones)
You can search for issues with only one message: http://bugs.python.org/issue?%40sort0=activity&%40sort1=&%40group0=&%40group1=&%40columns=title%2Cid%2Cactivity%2Cstatus&%40filter=status%2Cmessage_count&status=1&message_count=1&%40pagesize=50&%40startwith=0
* list of N people with the most open issues assigned to them
And then poke them with a goad until they fix them? :)
The idea is to provide charts that help us direct effort to particular subsets of bugs.
If someone wants to experiment with and/or improve the tracker stats, this is how it works: 1) The roundup-summary script [0] analyzes the issues once a week and produce the weekly report and a static JSON file [1]; 2) The stats page [2] request the JSON file and uses the data to generate the charts client-side. Now there are two ways to improve it: 1) the easy way is just to use the roundup-summary script to expose more of its data or to find new ones and add them to the JSON file (and possibly to the summary too); 2) the hard way is to decouple the roundup-summary and the stats page and either make another weekly (or daily/hourly) script to generate the JSON file, or a template page that generates the data in real-time. Once the data are in the JSON file is quite easy to use jqPlot [4] to make any kind of charts. Keep in mind that some things are trivial to get out from the DB (e.g. number of issues for each status/type), but other things are a bit more complicated (e.g. things involving specific periods of time) and currently the roundup-summary takes a few minutes to analyze all the issues. I also tried to include just a few useful charts on the stats page -- at first I had several more charts but then I removed them. Feel free to ping me on IRC (#python-dev@Freenode) if you have questions. Best Regards, Ezio Melotti [0]: http://hg.python.org/tracker/python-dev/file/default/scripts/roundup-summary [1]: http://bugs.python.org/@@file/issue.stats.json [2]: http://hg.python.org/tracker/python-dev/file/bbbe6c190a99/html/issue.stats.h... [3]: http://www.jqplot.com/tests/
--amk

On Mon, Jun 23, 2014 at 6:52 PM, francis <francismb@email.de> wrote:
Not particularly hard, but I won't have time to get back to this project for a while (contributions are welcomed though!).
That summary lists all the issues with the "patch" keyword, and the ones not listed simply don't have it. The keyword is added automatically whenever an attachment is added to the issue, so there might be false positives (e.g. if the attachment is a script to reproduce the issue rather than a patch, or if the available patches are outdated). The might also be issues with patches that are not included in the summary (e.g. if someone accidentally removed the keyword), but that shouldn't be very common.
Best Regards, Ezio Melotti
Regards, francis

On Tue, 24 Jun 2014 20:43:41 +0200, francis <francismb@email.de> wrote:
Anyone can review patches, in case that isn't clear.
more committers: ?? automate!! :-)
That's a goal of the python-workflow interest group. Unfortunately between billable work and GSOC mentoring I haven't had time to do much there lately. Our first goal is to make the review step easier to manage (know which patches really need review, be able to list patches where community review is thought to be complete) by improving the tracker, then we'll look at creating the patch gating system Nick has talked about previously. Still needs a committer to approve the patch, but it should increase the throughput considerably. In the meantime, something that would help would be if people would do reviews and say on the issue "I think this is commit ready" and have the issue moved to 'commit review' stage. Do that a few times where people who are already triagers/committers agree with you, and you'll get triage privileges on the tracker. --David
participants (6)
-
A.M. Kuchling
-
Ethan Furman
-
Ezio Melotti
-
francis
-
R. David Murray
-
Raymond Hettinger