Tracker workflow proposal
OK, wall of text time. Sorry about this :) CPython Tracker Workflow ======================== Framing the Discussion ---------------------- This document is intended as a starting point for discussion. I won't be offended if we throw it out and start over. Even if you like the general idea, I'm sure I've made some mistakes and have not managed to correctly think through all the issues. I also realize that what I'm talking about here is a non-trivial amount of work. But if we want to have a 'best in class' workflow infrastructure (and as Nick has said, we *need* one), we are looking at a non-trivial amount of work no mater how we achieve it. Discussions of alternate ways to achieve the same effect are welcome, but I'm betting that in the long run we really do want to tailor this stuff to *our* workflow, rather than try to shoehorn ourselves into someone else's workflow. Which doesn't mean we shouldn't leverage existing tools (like Roundup :), but does mean there's a lot of tailoring to do to get this right. That said, feedback from people who have actual experience with tools that support similar workflows is valuable. The ones I've worked with were not like what I'm trying to create here, they were either like what we have now (bug report systems with little to no workflow support) or pure peer-review systems (that is, there were no non-committers). I'm volunteering to be the coordinator for the work, and I'm also volunteering to do as much of it as necessary. That is, I'm planning to make this the focus of most of the "python time" I have available. I think that there is a way to implement this workflow or whatever workflow we decide on as a "new interface" while keeping the existing interface available, allowing us to test and refine (like, maybe do some usability testing? :). Roundup has a way to dynamically specify which page template is used to render a particular page type, and I think we can leverage that to have two parallel UIs. I could be wrong, though, in which case we'd need to set up a test tracker instead, which I can do. (Assuming it does work, there might need to be a bit of glue code to keep things in sync. And there would need to be some changes in the data values available in the current UI, but I don't think that would be a bad idea anyway.) The scope of this discussion is the workflow for an "issue", which currently means an entry created by someone in the bug tracker at bugs.python.org, and goes from the creation of the issue to the resolution of the issue, which can but does not necessarily include committing something. I won't be discussing the tooling for Zuul-like patch gating at this stage, I'll just assume we can figure out how to implement that. It would also be possible to build a patch gating system without initially integrating it with our other tools, and if we decide that's more important than the tracker workflow at this stage, we should switch our conversation to what that would look like. Or perhaps we can work on the two in parallel. My own feeling is that in order to get maximum benefit out of patch gating, we need more clarity and utility in our *tracker* workflow first, but I do realize that how long it currently takes to commit a patch is a significant pain point, and it might be better to address that first. Goals ----- In suggesting improvements to the existing workflow support, I'm starting from the fundamental idea that anything we do in the issue tracker and/or patch gating system should have either *operational* implications or *important* informational implications. The former should be more heavily weighted than the latter, in my opinion. What I mean by this is that there should be no "busy work" setting of attributes in our workflow: when you change the state of an issue, it should mean that something new is *going* to happen to that issue. Any purely informational attribute setting should be in support of that new action, and wherever possible that information should *mean* something to some part of our tooling, not *just* to the humans looking at the issue. In general I want there to be as few clicks as practical involved in updating an issue. However, I'm not addressing the actual UI design here (although I have ideas); I assume we are going to iterate on it for a bit until we have something we really like. NB: This proposal includes a number of ideas from the "desired tracker features" (http://wiki.python.org/moin/DesiredTrackerFeatures), but by no means all of them. Some of the others are worth implementing, but in this document I'm focusing on primary workflow, which I think should happen first. Roles ----- Conceptual Roles ~~~~~~~~~~~~~~~~ In thinking about our workflow, I identify the following roles: original reporter commenter triager patch-producer reviewer core reviewer committer These are in the order that the role is involved in the issue (roughly; variations are possible depending on the issue), and obviously a single person can take on multiple roles during the lifetime of a patch. I originally thought this list would have operational implications, but it turned out to be only an aid in thinking about the problem. I'm leaving it in here for exactly that purpose...it helps when thinking about the states and state transitions. I'd also like to add something that we currently only have informally, but which has been requested as a feature more than once: at every stage, I'd like there to be the possibility of there being a 'responsible party'. This is sort of like 'assigned to', except that it could be anyone with a tracker account, and the assignment would have a limited lifetime: either until the issue's state changes, or until the issue has been without update for some number of days (off the cuff I'd suggest 14, but it might also vary by state depending on what states it actually got used in). The idea behind this is that we have eager contributors who either wind up stepping on each other's toes, or rush to create and submit a patch before someone else does, and as a consequence of rushing, do not produce as good a patch as they are capable of, which actually slows down the issue resolution. The ability to "take" a task and know it is "yours" is an important part of the new contributor process, and having a "responsible party" field would support that. Tracker Roles ~~~~~~~~~~~~~ The tracker roles important to the workflow are: User Triager ('Developer') Committer Although the tracker calls the Triager role 'Developer', I'm going to refer to it as Triager throughout this document, for clarity as to its intent. The tracker does not currently have a role equivalent to 'committer', but we may not need to add one explicitly, since the account of a committer is marked as such. The important thing to understand about these roles in the context of this document is that anything a user can do, a triager or committer can do, and anything a triager can do, a committer can do. Information Fields ------------------ An issue has an issue number and a title, and those wouldn't change. It also has a state, which is the subject of the second half of this document, and would likely be a new field in order to be able to have both UIs available (this field is where the glue code would be needed). Beyond those fields, I suggest several changes to the issue metadata; I will cover each one separately. Versions ~~~~~~~~ Currently our versions field does double duty: we set it to the versions we want to fix the bug in, and then deselect versions as we fix them. However, I noticed when doing the "What's New" document that this makes things really confusing if you *do* look at the ticket later. You can't tell which versions got the fix without reading through the entire issue. So I'd like to split this into two fields: should be fixed in: [list of versions] has been fixed in: [list of versions] I'd also like patch set links to be displayed next to the versions for 'has been fixed'. When a commit references an issue, it should appear next to the appropriate version, but it should *not* automatically change the version state to fixed. That should require separate action, since we sometimes apply patches that are relevant to the issue but do not fix it. When an issue is transitioned to closed, there should be an easy way to say "mark all versions with changesets as fixed". It would probably be appropriate for that to be the default. These fields have direct operational meaning: they indicate a task to be performed or signal the completion of a task. Type ~~~~ documentation python bug interpreter crash security enhancement request I add 'documentation' here based on PyCon feedback from Jessica McKeller and Selena Deckelmann. None of the existing types makes sense to a beginner for a documentation bug, and the resulting confusion can lead someone to abandon the effort to contribute. It also has operational implications (see below). I rename 'behavior' to 'python bug' for a similar reason, but I don't have any usability data to back that feeling, so I'm not strongly advocating that change. I rename 'crash' to 'interpreter crash' in an attempt to reduce the chances that someone will report a python traceback as a crash, something that otherwise happens very regularly. I'm sure we'll never completely eliminate those. I drop 'compiler error', 'resource usage' and 'performance'. All of these are bugs in the minds of the reporters, and classifying them separately in the 'type' field does not, as far as I can see, lead to any operational difference in the way the issue is treated. What I mean by that: most bugs are either 'documentation' or 'python bug' or 'enhancement request'. Differentiating these is important, as doc bugs are handled very differently from python bugs (documentation fixes do not get NEWS entries, for example), and bug vs enhancement determines which versions we fix things in. 'compiler error', 'resource usage', and 'performance', on the other hand, are all handled with the same workflow that applies to bugs. You might think that there is a difference for resource usage and performance, in that we don't in general backport those fixes. The key there, though, is "in general". The decision as to which versions to apply the fix is made based on the magnitude of the bug, and there really are only two cases: we mostly don't backport, as is the case for enhancements, but we sometimes do backport, just like we would a bug fix. The same applies in reverse to compile bugs: we mostly fix those in all versions, but we don't always. So the only *operational* effect of having these as distinct types would be to confuse things, since we couldn't tell from the type whether or not this was something that should be applied to all versions or only default. Instead we should set (or, more likely, triage) them as either 'python bug' or 'enhancement', which have the correct operational implications. There *is* an operational reason for having security and interpreter crash as separate types. In both of these cases the versions we fix it in is always the same (all active for crashers, all active + all security-only for security bugs), and the issue priority should default to either high or release blocker. In addition, security bugs should be automatically reported to the PSRT, and arguably the report should be be hidden from all but the PSRT and the original reporter. Priority ~~~~~~~~ low normal high deferred blocker release blocker The only change here is to eliminate 'critical'. I'm not wedded to that, but if we have both 'critical' and 'high' priorities, 'high' ends up getting treated as pretty much equivalent to either 'normal' or 'critical', depending on the person. I would argue that anything that is severe enough to be marked 'critical' should in fact be a release blocker, and anything that is not a release blocker is effectively only 'high' priority. The priority is currently operational only in that one can sort issues by priority. I propose that we make them much more operational, by posting a count and/or list of the bugs with more than normal priority somewhere public on a regular basis, such as python-dev, python-committers, and/or #python-dev. (Well, definitely #python-dev.) I'd like to see us strive keep those queues clear, so that promoting a bug to high or release blocker means it *will* get acted on reasonably promptly. (This raises the issue of what to do about bugs we currently mark as "release blocker" as a *reminder* to the release manager. I don't have a proposal for that at the current time, as release management is out of scope for this document, but we'll need an answer if we are going to implement this.) Component ~~~~~~~~~ I propose that we completely change the nature of this item. I think we should make it a text field that is filled in by autocomplete like the nosy field can be, and that the value be any of the components listed in the experts file. This would further respect how the experts have listed themselves in that file by autonosying those that are willing for that to happen. The field should still be a multilink (that is, can contain more than one value). This change would mean that it would be possible to search for issues based on module, which is an often-requested feature. Patch Status ~~~~~~~~~~~~ This is a new set of fields that records information about the patch: unit test fix documentation changes news entry what's new entry commit message For each of these, the value could be 'needs work', 'complete', or 'not applicable'. For issues of type 'documentation', all lines except 'documentation changes' and 'commit message' would be set to NA by default, otherwise they will be set to 'needs work' initially, except for "what's new", which will be set to NA for anything except type enhancement. Note that the inclusion of 'news entry' and 'commit message' assume two things: that we retool the NEWS file so that NEWS entries can be added automatically without conflicts, and that we change our patch workflow to use mercurial's capability to accept 'hg export' patches and/or some other sort of pull request. This part is one place we get into non-roundup territory, so that may be a phase two addition. Mercurial's 'evolve' feature, which we saw demoed at PyCon and which I understand is currently available as a plugin, makes this capability much more useful. When the non-core committer syncs the master repository after their patch has been committed, the right thing happens to the history in their repository to make their local commit disappear in favor of the commit made to master. The contributor will also be able to essentially *edit* the patch in their local repository based on the review feedback, without ending up with a chain of commits in their local repo that they have to deal with. Evolve should also facilitate DVCS-based collaboration between core and non-core developers, as well as between non-core developers. (I'm assuming all this works with exported changesets, since in the demo they talked about starting with a patch received via email, which is their normal workflow. But we should double check this.) These fields allow us to represent all the parts that a patch must have to be complete, and they act as a checklist for making sure the parts are all there. This is similar in intent to patchcheck, but since which pieces are needed is ultimately determined by a human, it is more accurate than that part of patchcheck and therefore more useful. Stage ~~~~~ This goes away, subsumed into state. Status ~~~~~~ This either holds the state information, or goes away in favor of a new 'state' field. Keywords ~~~~~~~~ buildbot easy stuck buildbot and easy are the only two existing, non-redundant tags that I can see a way to make operational. 'easy' of course has to be set manually. The system for listing easy issues should list the issues only when there is an action that someone could take on the issue (clarify issue, write patch, review patch). (NB: it might be possible to come up with a better name for this tag.) For buildbot, those issues should be displayed in a report or dashboard, and we should try to keep this queue empty. This becomes even more important when we have a patch gating system online. All of the other values are informational-only, and in some cases redundant. If we want informational keywords, IMO we should do a full user-accessible tagging system. That would be a separate proposal, though. Note: I claim the regression tags are redundant because I think regression issues should be release blockers. (They are also ambiguous: is '3.3regression' a bug in 3.3 that is a regression relative to 3.2, or a 3.3.x bug relative to 3.3.x-1, or a 3.4 bug that is a regression relative to 3.3? Better to just explain exactly what is going on in the issue comments, IMO.) The purpose of the 'stuck' tag is to label issues that we agree are real issues that we don't want to close as "won't fix", but that we can't for one reason or another currently move forward. Operationally, stuck issues are either not displayed in work queues (see below) or are displayed at the end of the queue. I think the keywords should be settable by anyone, if they aren't already. Issue States and State Transitions ---------------------------------- Here are the states that I think an issue can be in: new needs information needs decision needs decision from committer needs patch needs review needs patch update needs commit review closed/fixed closed/wont fix closed/duplicate closed/insufficient information closed/postponed closed/not a bug closed/third party I discuss each of these, and the possible state transitions, below. All legal state transitions should be displayed in a particular area in the UI, and each transition should be a single radio button. Which possible transitions will be displayed will depend on the user's roundup role, and occasionally other state in the issue. The default transition is "no change". Note that the names of these states are, unlike most of the other names I've used so far, not the ones I necessarily expect us to use in the UI. They are descriptive of the state from the workflow POV, not necessarily the labels that should be used for the state *transitions*. New ~~~ The issue has been created by the original reporter. The information collected at this stage should be as simple as possible while gathering the information we need. I propose, based on conversations at PyCon with people who work with new users, that the only information we *need* is the title and first comment. The rest would ideally be hidden/revealed via javascript, with a per-user or browser-sticky setting that defaults to 'collapsed' for new users. The 'advanced' section would allow the setting of type, component, and nosy. One could argue that it might be useful to collect 'found in version' and perhaps other information such as os/version, but in my own use of trackers I find that this is more often confusing than it is helpful, and generally more detailed questions about the user's environment are required in any case, making the up front collection attempt of dubious utility. A new issue can transition to: needs information user needs decision user needs patch triage needs decision from committer triage closed triage, original reporter needs review triage, if there is already a patch needs commit review committer, if there is already a patch That is, any user can indicate that more information is needed or request that triage decide whether to accept or reject the issue, but only triage can accept the issue as one that should be worked on (needs patch), or close it, or request a committer to make that decision. Operationally, new bugs get sent to the 'new bugs' list, and appear on the weekly report in the new bugs section. We also have 'new bugs without response' in the weekly report. The list of bugs in this state constitute a "work queue", and should be available as a standard search query and/or as a dashboard feature. Triagers are primarily responsible for draining this queue, but anyone can help. needs information ~~~~~~~~~~~~~~~~~ We are waiting for more information from the original reporter, or from anyone else who manages to reproduce the problem. If there is no additional activity for some period of time (one month?) the issue is automatically moved to "closed/insufficient information". A report of autoclosed issues will be sent to appropriate places (python-dev? new-bugs-announce? Anyone with triage privileges? Accessible from the dashboard UI?). The possible transitions are: needs decision user needs patch triage needs decision from committer triage closed triage, original reporter, autoclose needs review triage, if there is already a patch needs commit review committer, if there is already a patch Bugs in this state are *not* a work queue. needs decision ~~~~~~~~~~~~~~ A user has expressed their opinion about what should be done with the bug, and confirmation is needed from triage. Note that an experienced user can submit a bug and immediately move it to 'needs decision'. There is nothing wrong with that, but it isn't necessary, since triage will review bugs in new status just as soon (or sooner) than they will review bugs in 'needs decision' status. The possible transitions are: needs information user needs patch triage needs decision from committer triage closed triage, original reporter needs review triage, if there is already a patch needs commit review committer, if there is already a patch This is a work queue and it is triage's responsibility to drain it. We should strive to keep this queue empty. Issues where we can't make a decision should marked stuck. needs patch ~~~~~~~~~~~ The issue has been accepted as something we would like to fix. Now a patch implementing the change is needed. The possible transitions are: needs review user needs information user needs decision user needs decision from committer triage needs commit review triage closed triage, original reporter This is a work queue, and it is something that everyone can help with. It is the responsibility of the *community* to drain this queue, and triager's or committers who write patches are doing so as part of the general community, not in their specific empowered roles. This is the primary place where someone might want to set themselves as 'responsible party'. needs review ~~~~~~~~~~~~ There is a patch, and it is ready to be reviewed. The possible transitions are: needs information user needs decision user needs patch update user needs commit review triage needs decision from committer triage closed triage, original reporter An issue cannot transition to commit review if any of the patch status items is set to 'needs work'. Setting these can be done by any user. It is always possible to upload a patch, regardless of state. It is always possible to adjust the patch checkboxes regardless of state. Anyone can do these things. So, a patch can get all the way to being *ready* for commit review without getting beyond 'needs decision' state (that is, without a triage person taking action), but it cannot get into a commit review state without triage getting involved. The theory behind this is that patch work is not *blocked* by triage/committer non-action, but by the same token a patch can't get into the commit review queue without one of those two actively deciding it should be. Like patch generation, this queue is also a community work queue. needs patch update ~~~~~~~~~~~~~~~~~~ The patch isn't perfect yet, but has had at least one review. It might be nice for the reviewer to be able to easily restore the 'responsible party' to point to the patch submitter when moving the issue to this state, or perhaps even for that to happen automatically. The possible transitions are: needs information user needs decision user needs review user needs commit review triage needs decision from committer triage closed triage, original reporter This is a community work queue. It is distinct from "needs patch" so that contributors who want to find a new problem to work on can look at the 'needs patch' queue. needs decision from committer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A question has come up that triage doesn't feel qualified to make the call on. I envision this mostly as a training mechanism for triage: the committer who responds should explain *why* they make the decision they do. A triager who continues to reach for this button instead of starting to make the calls themselves isn't a good candidate for further responsibility. Note that if committers are already nosy on an issue, there is normally little reason to use this state, but it does move the issue into a queue the committers are directly responsible for, so it can occasionally be appropriate. There is another use for this state: asking for a decision from the maintainer of the module when there is an active maintainer. The possible transitions are: needs information user needs review user needs patch update user, if there is a patch needs patch triage needs commit review triage closed triage, original reporter This is a committer work queue. Note that the issue can be transitioned out of 'needs decision from committer' by anyone, so that if someone with more experience comes along it doesn't have to stay in the committer work queue if that isn't in fact necessary. needs commit review ~~~~~~~~~~~~~~~~~~~ Triage agrees that the patch is ready to be committed. This issue now appears in a search/dashboard display for committers. Ideally this queue should be kept close to empty, but in reality it probably won't be until we've gotten the bug list reduced and more committers online. This state represents the transition to the patch gating system. Until an automated system is on-line, it would be the current manual system, and patches could easily sit in this state for a while. Once we've figured out the automated patch gating system, either the issue transitions to that system until it is committed, at which point the state in the tracker transitions to closed/fixed, or we will be adding additional states and controls to roundup to implement control flow between the tracker and the gating system. The possible transitions are: needs information user needs decision user needs patch update user needs decision from committer triage closed triage, original reporter If any of the patch status checkboxes are changed to 'needs work', the 'needs patch update' next-state should automatically be checked. This is a committer work queue. Closed ~~~~~~ An issue can make a transition to any of the closed states from any other state (even fixed...there are times an issue is 'fixed' by another issue, and so an issue can be closed as fixed even without a patch having been committed). You can also view the state as "closed-with-reason", with the reason being the 'resolution' as currently managed by the tracker. However, I think the UI for this should indicate both the close and the reason with the same "gesture", instead of having to manipulate two different dropdowns the way we do now. (And it's currently two only if you ignore stage.) Triage or higher role is required to close an issue, except that the original reporter can also close an issue (as they can now). A closed issue can transition to: needs decision user needs information triage needs patch triage needs decision from committer triage needs review triage, if there is already a patch needs commit review committer, if there is already a patch I'm not sure if allowing any user to move a closed issue to 'needs decision' is a good idea or not; currently closed issues can only be opened by the original reporter or triage. But that has worked fine for the most part, and the overhead of dealing with the people who have abused it hasn't been much higher than those people generate anyway. I'm sure those folks will be able to find some other part of this design to abuse if we don't allow 'needs decision' here. Dashboard --------- The dashboard would be a new feature, a new landing page and linked from the left hand menu bar. It would list the first N (adjustable) items in each queue relevant to the user for which they are the responsible party, followed by those on which the user is nosy, followed by those on which they are not nosy. A general user would see the 'new', 'needs patch', 'needs patch update', and, 'needs review' queues. A triager would see, above the preceding, the 'needs decision' queue. A committer would see, above the preceding, the 'needs decision from committer' and 'needs commit review' queues. In all queues, issues would be sorted in priority order followed by most recent activity, with color coding for the priority. 'stuck' issues would appear last, and there should be something to visually differentiate issues on which the user is responsible, just nosy, or not nosy. As with everything else, this is of course subject to discussion. It would also be lovely if the user could configure their dashboard, but that would probably fall into the category of advanced features we come back to later. Big Picture ----------- My goal here is to facilitate the involvement of the wider community in our workflow as much as possible. The structure above is designed to allow the community to do as much work as possible, and the "trusted individuals" to act as gatekeepers to insure quality. It is theoretically possible for a patch to get all the way to 'needs commit review' without any more higher level involvement than a triage person moving the issue to 'patch needed'. Of course in reality much more involvement from core will be needed, since we need to transfer knowledge to the community, and especially the newcomers, about our standards and procedures, Not to mention the code. And committers are going to *want* to do general commentary and triage level activities on issues anyway. But, for those committers who have less time, I'd like to think that this system would allow them to focus their time on just the stuff only they can do, and thus perhaps draw more contributions from some of our less active or currently-inactive committers. The key to this is that instead of issues getting lost, core can watch certain key queues: 'needs decision', 'needs committer decision', and 'needs commit review'. When issues get to those stages, the people with the experience to do something know that it is *their* responsibility to do it: the issues are to a point where the general community can't help without core input. If we can keep those queues to a manageable size, I think we can increase the amount of energy coming our way. The other important goal is that via these explicit state transitions, especially the 'needs decision' and 'needs commit review' transitions, we get a much clearer picture of who in the community *has* absorbed the core standards and procedures and is ready to be "promoted" to the next level of responsibility. And the more people we get moved up, the more we can get done. (Note: making that information *easily* accessible requires additional tooling, but it should be possible and should be done.) Note also that I tried to engineer this so that the structure does not handicap us or add bureaucracy: even though the basic structure is that you get triage signoff first and then committer signoff, if the person doing the triage is a committer, they can just move the issue to wherever it needs to be. So the only time two levels of signoff will be *required* is when the committers are too busy to drain the queues. And work on an issue should never be *blocked* by the system, the purpose of the queues is to draw attention to issues that are "ready" for decision. Thus the two levels help us manage the load when we need the help, and don't get in our way otherwise. That's my intent, anyway. You tell me if you think I'm headed in the right direction.
Hi, On lun., 2014-04-21 at 12:04 -0400, R. David Murray wrote:
I drop 'compiler error', 'resource usage' and 'performance'. All of these are bugs in the minds of the reporters, and classifying them separately in the 'type' field does not, as far as I can see, lead to any operational difference in the way the issue is treated.
But they are easily searchable that way, which is a big advantage (at least for "resource usage" and "performance"). As for "compiler error", I find it interesting to categorize build-time problems separately from runtime problems. Note that these could become "components".
I propose that we completely change the nature of this item. I think we should make it a text field that is filled in by autocomplete like the nosy field can be, and that the value be any of the components listed in the experts file. This would further respect how the experts have listed themselves in that file by autonosying those that are willing for that to happen. The field should still be a multilink (that is, can contain more than one value).
Note that the UI then becomes less accessible to beginners (the current list is readily displayed, having to start typing something is intimidating when you don't know what to type).
Patch Status ~~~~~~~~~~~~
This is a new set of fields that records information about the patch:
unit test fix documentation changes news entry what's new entry commit message
Hmm, in principle I'd rather avoid us adding new fields (i.e.: interaction is already complex enough).
These fields allow us to represent all the parts that a patch must have to be complete, and they act as a checklist for making sure the parts are all there. This is similar in intent to patchcheck, but since which pieces are needed is ultimately determined by a human, it is more accurate than that part of patchcheck and therefore more useful.
But is it *really* useful? You said you wanted additions to be operational, not informational :-)
The purpose of the 'stuck' tag is to label issues that we agree are real issues that we don't want to close as "won't fix", but that we can't for one reason or another currently move forward. Operationally, stuck issues are either not displayed in work queues (see below) or are displayed at the end of the queue.
Is it a tag or a specific issue status, then?
new needs information needs decision needs decision from committer needs patch needs review needs patch update needs commit review closed/fixed closed/wont fix closed/duplicate closed/insufficient information closed/postponed closed/not a bug closed/third party
Up front, this looks like too much formalization (IMHO).
A new issue can transition to:
needs information user needs decision user needs patch triage needs decision from committer triage closed triage, original reporter needs review triage, if there is already a patch needs commit review committer, if there is already a patch
That is, any user can indicate that more information is needed or request that triage decide whether to accept or reject the issue, but only triage can accept the issue as one that should be worked on (needs patch), or close it, or request a committer to make that decision.
Is there any reason to restrict actions by role like this? I don't think we've had any significant amount of tracker abuse in the past. (also, I'll remind you that we don't really have any official triagers, instead letting everyone act as would-be triagers :-))
We are waiting for more information from the original reporter, or from anyone else who manages to reproduce the problem. If there is no additional activity for some period of time (one month?) the issue is automatically moved to "closed/insufficient information".
I have never felt automatic closing was a good idea. A bug is a bug, even if noone was able/willing to give more precise information. Also, keeping a bug open can help other people find it later and bring confirmation or information. Regards Antoine.
On Mon, 21 Apr 2014 18:53:41 +0200, Antoine Pitrou <antoine@python.org> wrote:
On lun., 2014-04-21 at 12:04 -0400, R. David Murray wrote:
I drop 'compiler error', 'resource usage' and 'performance'. All of these are bugs in the minds of the reporters, and classifying them separately in the 'type' field does not, as far as I can see, lead to any operational difference in the way the issue is treated.
But they are easily searchable that way, which is a big advantage (at least for "resource usage" and "performance").
As for "compiler error", I find it interesting to categorize build-time problems separately from runtime problems.
Note that these could become "components".
Yes, I think that would be good and desireable.
I propose that we completely change the nature of this item. I think we should make it a text field that is filled in by autocomplete like the nosy field can be, and that the value be any of the components listed in the experts file. This would further respect how the experts have listed themselves in that file by autonosying those that are willing for that to happen. The field should still be a multilink (that is, can contain more than one value).
Note that the UI then becomes less accessible to beginners (the current list is readily displayed, having to start typing something is intimidating when you don't know what to type).
Well, the beginners wouldn't be setting it, someone more experienced would be setting. When the beginner knows what to do with that field, they become one of the people who uses the 'uncollapsed' version of the UI and sets it.
Patch Status ~~~~~~~~~~~~
This is a new set of fields that records information about the patch:
unit test fix documentation changes news entry what's new entry commit message
Hmm, in principle I'd rather avoid us adding new fields (i.e.: interaction is already complex enough).
These fields allow us to represent all the parts that a patch must have to be complete, and they act as a checklist for making sure the parts are all there. This is similar in intent to patchcheck, but since which pieces are needed is ultimately determined by a human, it is more accurate than that part of patchcheck and therefore more useful.
But is it *really* useful? You said you wanted additions to be operational, not informational :-)
It would not break my heart to drop this :). What it provides is a way to see at a glance what is missing, which in principle makes it possible for a new contributor to, say, add documentation to an otherwise complete patch. But that may well not be enough of an advantage to be worth it. Probably my main motivation for adding it, actually, was to make sure we get NEWS items and What's New items when appropriate, something we miss on a fairly regular basis.
The purpose of the 'stuck' tag is to label issues that we agree are real issues that we don't want to close as "won't fix", but that we can't for one reason or another currently move forward. Operationally, stuck issues are either not displayed in work queues (see below) or are displayed at the end of the queue.
Is it a tag or a specific issue status, then?
A tag. An issue can get stuck in any status in principle, though in practice I think it is most likely to happen in either 'needs patch' (ie: we don't know *how* to fix it) or one of the 'needs decision' states (we can't *decide* how to fix it).
new needs information needs decision needs decision from committer needs patch needs review needs patch update needs commit review closed/fixed closed/wont fix closed/duplicate closed/insufficient information closed/postponed closed/not a bug closed/third party
Up front, this looks like too much formalization (IMHO).
It's not about formalization, though. It's about making the list of bugs *manageable*. The structure above is what provides the actionable work queues. *That* is the point of the exercise, and not control :)
A new issue can transition to:
needs information user needs decision user needs patch triage needs decision from committer triage closed triage, original reporter needs review triage, if there is already a patch needs commit review committer, if there is already a patch
That is, any user can indicate that more information is needed or request that triage decide whether to accept or reject the issue, but only triage can accept the issue as one that should be worked on (needs patch), or close it, or request a committer to make that decision.
Is there any reason to restrict actions by role like this? I don't think we've had any significant amount of tracker abuse in the past.
Well, at this stage probably not. I can imagine us needing it if people (probably newcommers to the community) start pushing for 'needs patch' or 'commit review' before a bug is confirmed as a bug or before a patch is really ready, but we could change the permissions if and when that happened. The permissions I proposed is mostly about scaleability, and while we are growing rapidly you are probably right that we aren't there yet.
(also, I'll remind you that we don't really have any official triagers, instead letting everyone act as would-be triagers :-))
Yes, except that many would-be triagers *can't* adjust the fields. We do have a few (ex: I've given Developer privs to Berker Peksag and Jessica McKeller, to name two who are currently exercising the triage role only. Both of whom I hope will be committers before long :) Still, I don't myself have any objection to letting anybody do such updates and see how it goes (it will probably go just fine).
We are waiting for more information from the original reporter, or from anyone else who manages to reproduce the problem. If there is no additional activity for some period of time (one month?) the issue is automatically moved to "closed/insufficient information".
I have never felt automatic closing was a good idea. A bug is a bug, even if noone was able/willing to give more precise information. Also, keeping a bug open can help other people find it later and bring confirmation or information.
What if we made that another work queue in the triage dashboard, but issues would only appear there if they were more than N days old? In other words, someone would be prompted to make a decision to close the bug. I don't think keeping bugs we can't reproduce open serves anyone. Our default search searches both open and closed bugs for a reason. Given the ability for anyone to reopen a bug, someone finding a closed "insufficient information" bug and reactivating it is easy. However, if people feel strongly that they should be left open, it's not a big deal; we'd just leave 'needs information' as not being a work queue, and those open issues would get ignored (except in our total open issue count). --David
On lun., 2014-04-21 at 13:49 -0400, R. David Murray wrote:
I don't think keeping bugs we can't reproduce open serves anyone. Our default search searches both open and closed bugs for a reason.
Hmm, usually the first thing I do when searching for bugs is change the radio box from "all" to "open" :-) Open but lingering bugs are really a different status from closed bugs. I don't have any hard feelings about it but I think the current setup is more useful. Regards Antoine.
On Mon, 21 Apr 2014 19:54:59 +0200, Antoine Pitrou <antoine@python.org> wrote:
On lun., 2014-04-21 at 13:49 -0400, R. David Murray wrote:
I don't think keeping bugs we can't reproduce open serves anyone. Our default search searches both open and closed bugs for a reason.
Hmm, usually the first thing I do when searching for bugs is change the radio box from "all" to "open" :-)
Well, but we don't want newcommers to do that. Often they are reporting a bug that has already been fixed, so we want them to see the closed bugs as well.
Open but lingering bugs are really a different status from closed bugs. I don't have any hard feelings about it but I think the current setup is more useful.
OK, this is a good point. I'm focusing on bugs where we don't have enough information to know if it is a bug (IMO, those should get closed). But the way I structured it, 'needs information' could get used for other things, like asking the submitter to try the proposed fix. In *those* cases, the issue should indeed stay open. So I'd propose that we make 'needs information' a work queue as I proposed, and have whoever checks it at the N day mark either closes it (we don't even know if it is a bug) or tags it as stuck (we need a response and haven't gotten one). That way once we've taken an action on a stale 'needs information' bug it is either gone or has moved to the bottom of the work queue and is labeled as stuck, so we know we can ignore it. --David
On Apr 21, 2014 8:49 PM, "R. David Murray" <rdmurray@bitdance.com> wrote:
On Mon, 21 Apr 2014 18:53:41 +0200, Antoine Pitrou <antoine@python.org>
wrote:
On lun., 2014-04-21 at 12:04 -0400, R. David Murray wrote:
I drop 'compiler error', 'resource usage' and 'performance'. All of these are bugs in the minds of the reporters, and classifying them separately in the 'type' field does not, as far as I can see, lead to any operational difference in the way the issue is treated.
But they are easily searchable that way, which is a big advantage (at least for "resource usage" and "performance").
As for "compiler error", I find it interesting to categorize build-time problems separately from runtime problems.
Note that these could become "components".
Yes, I think that would be good and desireable.
I propose that we completely change the nature of this item. I think we should make it a text field that is filled in by autocomplete like the nosy field can be, and that the value be any of the components listed in the experts file. This would further respect how the experts have listed themselves in that file by autonosying those that are willing for that to happen. The field should still be a multilink (that is, can contain more than one value).
Note that the UI then becomes less accessible to beginners (the current list is readily displayed, having to start typing something is intimidating when you don't know what to type).
Well, the beginners wouldn't be setting it, someone more experienced would be setting. When the beginner knows what to do with that field, they become one of the people who uses the 'uncollapsed' version of the UI and sets it.
Patch Status ~~~~~~~~~~~~
This is a new set of fields that records information about the patch:
unit test fix documentation changes news entry what's new entry commit message
Hmm, in principle I'd rather avoid us adding new fields (i.e.: interaction is already complex enough).
These fields allow us to represent all the parts that a patch must have to be complete, and they act as a checklist for making sure the parts are all there. This is similar in intent to patchcheck, but since which pieces are needed is ultimately determined by a human, it is more accurate than that part of patchcheck and therefore more useful.
But is it *really* useful? You said you wanted additions to be operational, not informational :-)
It would not break my heart to drop this :). What it provides is a way to see at a glance what is missing, which in principle makes it possible for a new contributor to, say, add documentation to an otherwise complete patch.
But that may well not be enough of an advantage to be worth it.
Probably my main motivation for adding it, actually, was to make sure we get NEWS items and What's New items when appropriate, something we miss on a fairly regular basis.
The purpose of the 'stuck' tag is to label issues that we agree are real issues that we don't want to close as "won't fix", but that we can't for one reason or another currently move forward. Operationally, stuck issues are either not displayed in work queues (see below) or are displayed at the end of the queue.
Is it a tag or a specific issue status, then?
A tag. An issue can get stuck in any status in principle, though in practice I think it is most likely to happen in either 'needs patch' (ie: we don't know *how* to fix it) or one of the 'needs decision' states (we can't *decide* how to fix it).
new needs information needs decision needs decision from committer needs patch needs review needs patch update needs commit review closed/fixed closed/wont fix closed/duplicate closed/insufficient information closed/postponed closed/not a bug closed/third party
Up front, this looks like too much formalization (IMHO).
It's not about formalization, though. It's about making the list of bugs *manageable*. The structure above is what provides the actionable work queues. *That* is the point of the exercise, and not control :)
A new issue can transition to:
needs information user needs decision user needs patch triage needs decision from committer triage closed triage, original reporter needs review triage, if there is already a
I think components are/were mostly used to indicate different dirs/packages (e.g. Doc/, Modules/, Lib/, Liberty/test/, Lib/json/), but now also contain Unicode, Windows, and even cross-build. patch
needs commit review committer, if there is
already a patch
That is, any user can indicate that more information is needed or request that triage decide whether to accept or reject the issue, but only triage can accept the issue as one that should be worked on (needs patch), or close it, or request a committer to make that
decision.
Is there any reason to restrict actions by role like this? I don't think we've had any significant amount of tracker abuse in the past.
Well, at this stage probably not. I can imagine us needing it if people (probably newcommers to the community) start pushing for 'needs patch' or 'commit review' before a bug is confirmed as a bug or before a patch is really ready, but we could change the permissions if and when that happened. The permissions I proposed is mostly about scaleability, and while we are growing rapidly you are probably right that we aren't there yet.
(also, I'll remind you that we don't really have any official triagers, instead letting everyone act as would-be triagers :-))
Yes, except that many would-be triagers *can't* adjust the fields. We do have a few (ex: I've given Developer privs to Berker Peksag and Jessica McKeller, to name two who are currently exercising the triage role only. Both of whom I hope will be committers before long :)
Still, I don't myself have any objection to letting anybody do such updates and see how it goes (it will probably go just fine).
We are waiting for more information from the original reporter, or from anyone else who manages to reproduce the problem. If there is no additional activity for some period of time (one month?) the issue is automatically moved to "closed/insufficient information".
I have never felt automatic closing was a good idea. A bug is a bug, even if noone was able/willing to give more precise information. Also, keeping a bug open can help other people find it later and bring confirmation or information.
What if we made that another work queue in the triage dashboard, but issues would only appear there if they were more than N days old? In other words, someone would be prompted to make a decision to close the bug.
I usually set these as pending while asking for more info, so that they are easy to find later. Closing them automatically could be OK. Of course we won't close automatically issues with patches or some kind of progress, even if slow.
I don't think keeping bugs we can't reproduce open serves anyone. Our default search searches both open and closed bugs for a reason. Given the ability for anyone to reopen a bug, someone finding a closed "insufficient information" bug and reactivating it is easy. However, if people feel strongly that they should be left open, it's not a big deal; we'd just leave 'needs information' as not being a work queue, and those open issues would get ignored (except in our total open issue count).
--David _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
On Apr 21, 2014, at 12:04 PM, R. David Murray wrote:
Priority ~~~~~~~~
low normal high deferred blocker release blocker
I'd like to see us strive keep those queues clear, so that promoting a bug to high or release blocker means it *will* get acted on reasonably promptly. (This raises the issue of what to do about bugs we currently mark as "release blocker" as a *reminder* to the release manager. I don't have a proposal for that at the current time, as release management is out of scope for this document, but we'll need an answer if we are going to implement this.)
This is definitely an area of friction that I think needs addressing. The problem is that a release blocker for one version may not be a release blocker for another. Worse, it's the responsibility of the release managers - which may be different per release - to manage these. Generalizing, it may be in fact that different active versions of Python might assign different priorities to the same bug. I'd like to at least keep on the table the option to assign a priority per active Python version. Cheers, -Barry
On Apr 21, 2014, at 02:35 PM, Barry Warsaw wrote:
Generalizing, it may be in fact that different active versions of Python might assign different priorities to the same bug. I'd like to at least keep on the table the option to assign a priority per active Python version.
Although it's used somewhat differently in Launchpad (which can track bugs that affect multiple projects, multiple versions of those projects, and multiple Ubuntu releases those versions appear in), the concept of a "bug task" may be informative here. Bug tasks are a way of providing optional finer grained "applicability" to a single issue. Let's say you find a bug in project "foo". Generally you just submit the bug and this creates a single issue. Now you realize that the bug also affects project "bar". You can open a second bug task that tracks the status, priority, assignment, and a few other things, of that bug as it relates to bar. So let's say Alice fixes the bug in foo; she would only close the foo bug task. A little later, Bob fixes the bug in bar, at which time the second bug task would be closed. It's important to note that both bug tasks share the same issue number, and in fact are part of the same issue. Of course, bug tasks can be added and deleted as necessary. (E.g. Bob decides that the bug actually doesn't affect bar.) In our case, we generally have just one project, but bug tasks may still be an interesting idea to handle a bug that affects multiple versions of Python. There could be a bug task for issue 12345 in Python 3.3 assigned to Alice, and another bug tasks for issue 12345 in Python 3.5 assigned to Bob. This may also be a way to handle different values of release blocker for different versions of Python. Another interesting twist that Launchpad adds is the notion of fix-committed vs. fix-released. Let's say you walk up to bug 12345 and you see that it is fix-committed in Python 3.3, but fix-released in Python 3.4. This means that the bug will not be present in the latest download of the Python 3.4 release tarball, but it *will* still be present in latest download of the Python 3.3 release tarball. Of course in both cases, the bug will have been fixed in both hg branches. Cheers, -Barry
On Mon, 21 Apr 2014 15:00:27 -0400, Barry Warsaw <barry@python.org> wrote:
On Apr 21, 2014, at 02:35 PM, Barry Warsaw wrote:
Generalizing, it may be in fact that different active versions of Python might assign different priorities to the same bug. I'd like to at least keep on the table the option to assign a priority per active Python version.
Although it's used somewhat differently in Launchpad (which can track bugs that affect multiple projects, multiple versions of those projects, and multiple Ubuntu releases those versions appear in), the concept of a "bug task" may be informative here.
Bug tasks are a way of providing optional finer grained "applicability" to a single issue. Let's say you find a bug in project "foo". Generally you just submit the bug and this creates a single issue.
Now you realize that the bug also affects project "bar". You can open a second bug task that tracks the status, priority, assignment, and a few other things, of that bug as it relates to bar. So let's say Alice fixes the bug in foo; she would only close the foo bug task. A little later, Bob fixes the bug in bar, at which time the second bug task would be closed.
It's important to note that both bug tasks share the same issue number, and in fact are part of the same issue. Of course, bug tasks can be added and deleted as necessary. (E.g. Bob decides that the bug actually doesn't affect bar.)
I have to say that as an outsider looking in, this was *totally* confusing. I signed up to see 'python' bugs, and most of the traffic on the tickets seemed to be about changes to things that looked like other tickets but I guess were bug tasks. Our case would be more straightforward...but I'm still not sure that the extra complexity would be worth it. --David
On Apr 21, 2014, at 04:29 PM, R. David Murray wrote:
I have to say that as an outsider looking in, this was *totally* confusing. I signed up to see 'python' bugs, and most of the traffic on the tickets seemed to be about changes to things that looked like other tickets but I guess were bug tasks.
Our case would be more straightforward...but I'm still not sure that the extra complexity would be worth it.
Right. It makes a lot more sense when you're working on multiple packages across the distro instead of primarily on a single package. It may not make sense for the Python tracker, where you're essentially working on a single project, although I wonder if adding alternative implementations, or subprojects (e.g. idle) might make it more relevant. Anyway, I was mostly just throwing that out there to think about. -Barry P.S. I'll mostly leave the blocker/deferred discussion to active RMs, since they're most affected by it. But I think it probably does make sense to have a release-manager role with certain additional privileges.
If there is considerable work in porting something between branches you might have a task per branch. (This might also provide clarity about which branches are being targeted.) Or sometimes you might have separate tasks for code, tests, docs. Then again, it does sound pretty marginal, and I'm not saying that our current process is broken where these specific flows are concerned. On Mon, Apr 21, 2014 at 1:42 PM, Barry Warsaw <barry@python.org> wrote:
On Apr 21, 2014, at 04:29 PM, R. David Murray wrote:
I have to say that as an outsider looking in, this was *totally* confusing. I signed up to see 'python' bugs, and most of the traffic on the tickets seemed to be about changes to things that looked like other tickets but I guess were bug tasks.
Our case would be more straightforward...but I'm still not sure that the extra complexity would be worth it.
Right. It makes a lot more sense when you're working on multiple packages across the distro instead of primarily on a single package.
It may not make sense for the Python tracker, where you're essentially working on a single project, although I wonder if adding alternative implementations, or subprojects (e.g. idle) might make it more relevant. Anyway, I was mostly just throwing that out there to think about.
-Barry
P.S. I'll mostly leave the blocker/deferred discussion to active RMs, since they're most affected by it. But I think it probably does make sense to have a release-manager role with certain additional privileges. _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
-- --Guido van Rossum (python.org/~guido)
On Mon, 21 Apr 2014 14:35:48 -0400, Barry Warsaw <barry@python.org> wrote:
On Apr 21, 2014, at 12:04 PM, R. David Murray wrote:
Priority ~~~~~~~~
low normal high deferred blocker release blocker
I'd like to see us strive keep those queues clear, so that promoting a bug to high or release blocker means it *will* get acted on reasonably promptly. (This raises the issue of what to do about bugs we currently mark as "release blocker" as a *reminder* to the release manager. I don't have a proposal for that at the current time, as release management is out of scope for this document, but we'll need an answer if we are going to implement this.)
This is definitely an area of friction that I think needs addressing. The problem is that a release blocker for one version may not be a release blocker for another. Worse, it's the responsibility of the release managers - which may be different per release - to manage these.
Generalizing, it may be in fact that different active versions of Python might assign different priorities to the same bug. I'd like to at least keep on the table the option to assign a priority per active Python version.
Well, one way to approach this might be via the 'fixed in version' field, which is more-or-less how we handle it now. To make this work, we need another role, the release manager, who gets a new work queue for each release for which they are responsible, and it only shows bugs that are priority 'release blocker' or 'deferred blocker' and have *not* been fixed in the version the queue is for. Another possibility would be to remove the blocker priorities, and make them version-specific tags (release-blocker-3.4, deferred-blocker-3.4, etc). This approach would also allow those tags to be used for the 'reminder' function. It makes for a lot of tags, though. Which wouldn't be bad if we had a full tag system such as Barry mentions in a later message. I have a feeling that per-release priorities only apply to release management. Would blocker/deferred be enough priority distinction to serve? --David
On Apr 21, 2014, at 12:04 PM, R. David Murray wrote:
All of the other values are informational-only, and in some cases redundant. If we want informational keywords, IMO we should do a full user-accessible tagging system. That would be a separate proposal, though.
Launchpad has a nice approach here. Tags are free-form and anybody can enter whatever they want, but a particular project[*] can define "official" tags that show up boldfaced when autocompleted. It allows a project to assert that certain tags have meaning to them, while preserving the freedom for more global searches based on free-form tags. Often, tags have a direct effect on workflow process and tooling, e.g. duplicate-check-needed, fix-verified, ci-passes, etc. (And of course, any tag can be promoted to or demoted from being official.) [*] I'm not sure the concept of per-project tags makes sense when we have a tracker essentially devoted to a single project, but a global official/unofficial designation on tags could still make sense. Cheers, -Barry
On Mon, Apr 21, 2014 at 12:04 PM, R. David Murray <rdmurray@bitdance.com>wrote:
interpreter crash
I think this would attract a higher proportion of 'correct' classifications if this was named "internal error". I'm not sure if it's worth keeping this separate, though, despite the apparent inherent higher priority. -- Thomas Wouters <thomas@python.org> Hi! I'm an email virus! Think twice before sending your email to help me spread!
On Mon, Apr 21, 2014 at 11:04 AM, R. David Murray <rdmurray@bitdance.com> wrote:
OK, wall of text time. Sorry about this :)
You weren't kidding :P
Tracker Roles ~~~~~~~~~~~~~
The tracker roles important to the workflow are:
User Triager ('Developer') Committer
Although the tracker calls the Triager role 'Developer', I'm going to refer to it as Triager throughout this document, for clarity as to its intent.
The tracker does not currently have a role equivalent to 'committer', but we may not need to add one explicitly, since the account of a committer is marked as such.
The important thing to understand about these roles in the context of this document is that anything a user can do, a triager or committer can do, and anything a triager can do, a committer can do.
I think it may be worthwhile to actually implement these roles as named, and promote people to Triager a lot earlier in their Python career than we currently promote to committer. I think it would be worthwhile and help with retention to "reward" (with more work) people who have made a couple of non-trivial contributions and seem interested in doing more; people like, for example, Vajrasky Kok, Nikolaus Rath, and Todd Rovito who if I'm not mistaken only have the User role currently.
Versions ~~~~~~~~
Currently our versions field does double duty: we set it to the versions we want to fix the bug in, and then deselect versions as we fix them. However, I noticed when doing the "What's New" document that this makes things really confusing if you *do* look at the ticket later. You can't tell which versions got the fix without reading through the entire issue. So I'd like to split this into two fields:
should be fixed in: [list of versions] has been fixed in: [list of versions]
I'd also like patch set links to be displayed next to the versions for 'has been fixed'. When a commit references an issue, it should appear next to the appropriate version, but it should *not* automatically change the version state to fixed. That should require separate action, since we sometimes apply patches that are relevant to the issue but do not fix it. When an issue is transitioned to closed, there should be an easy way to say "mark all versions with changesets as fixed". It would probably be appropriate for that to be the default.
These fields have direct operational meaning: they indicate a task to be performed or signal the completion of a task.
As may become apparent as this message goes on, I'm a fan of checkboxes. I don't much like the current method for selecting multiple versions or components, which requires both keyboard and mouse to make the selections in most cases. Since it's not an extremely common form element on the modern web, some users (not necessarily just Users) may not even be familiar with how to pick and choose specific ones--I suspect this may be part of why we have so many issues created with every version selected, the user attempted to choose 2.7 and 3.current but didn't know how to only select the two. So, my suggestion here (also colored by Barry's thoughts on release blocking) is to have a series of checkboxes for each version, "affected", "blocks release", and "fixed". Or even just "affected" and "fixed", with a priority box per version.
Type ~~~~ documentation python bug interpreter crash security enhancement request
I add 'documentation' here based on PyCon feedback from Jessica McKeller and Selena Deckelmann. None of the existing types makes sense to a beginner for a documentation bug, and the resulting confusion can lead someone to abandon the effort to contribute. It also has operational implications (see below).
I rename 'behavior' to 'python bug' for a similar reason, but I don't have any usability data to back that feeling, so I'm not strongly advocating that change.
I rename 'crash' to 'interpreter crash' in an attempt to reduce the chances that someone will report a python traceback as a crash, something that otherwise happens very regularly. I'm sure we'll never completely eliminate those.
Perhaps it could be made even clearer by naming them "python bug or traceback" and "interpreter crash (segfault)", or even just calling "interpreter crash" "segfault/abort".
I drop 'compiler error', 'resource usage' and 'performance'. All of these are bugs in the minds of the reporters, and classifying them separately in the 'type' field does not, as far as I can see, lead to any operational difference in the way the issue is treated.
I think it would be good to keep these as "components", as Antoine suggested.
Component ~~~~~~~~~
I propose that we completely change the nature of this item. I think we should make it a text field that is filled in by autocomplete like the nosy field can be, and that the value be any of the components listed in the experts file. This would further respect how the experts have listed themselves in that file by autonosying those that are willing for that to happen. The field should still be a multilink (that is, can contain more than one value).
This change would mean that it would be possible to search for issues based on module, which is an often-requested feature.
I like this idea quite a bit, especially changing the type of input widget.
Patch Status ~~~~~~~~~~~~
This is a new set of fields that records information about the patch:
unit test fix documentation changes news entry what's new entry commit message
For each of these, the value could be 'needs work', 'complete', or 'not applicable'. For issues of type 'documentation', all lines except 'documentation changes' and 'commit message' would be set to NA by default, otherwise they will be set to 'needs work' initially, except for "what's new", which will be set to NA for anything except type enhancement.
Instead of having 3 textual options for each field, I'd prefer checkboxes (or perhaps a series of checkboxes). For each field, have three checkboxes, "needed", "present" and "ready". "needed" would be visible only to Triager+; unchecked fields are not needed for a complete patch and that field is hidden entirely from Users. In the majority of cases, "present" could be set by sniffing through a patch when it is attached; if a patch contains "^diff.+Lib/.*test.*\.py$", chances are very good that it has a unit test. The "present" checkbox should probably just be a graphic for Users, but available to Triager+ to clean things up where automation fails. "ready" would be checked by a reviewer who approves that particular part of a patch.
Keywords ~~~~~~~~ <snip> I think the keywords should be settable by anyone, if they aren't already.
They're not, unless it's changed since November.
Issue States and State Transitions ----------------------------------
Here are the states that I think an issue can be in:
new needs information needs decision needs decision from committer needs patch needs review needs patch update needs commit review closed/fixed closed/wont fix closed/duplicate closed/insufficient information closed/postponed closed/not a bug closed/third party
I discuss each of these, and the possible state transitions, below.
All legal state transitions should be displayed in a particular area in the UI, and each transition should be a single radio button. Which possible transitions will be displayed will depend on the user's roundup role, and occasionally other state in the issue.
The default transition is "no change".
Note that the names of these states are, unlike most of the other names I've used so far, not the ones I necessarily expect us to use in the UI. They are descriptive of the state from the workflow POV, not necessarily the labels that should be used for the state *transitions*.
I like this scheme, and I think the split of abilities between roles is reasonable, as long as we promote people to Triager a lot faster than we currently do.
Dashboard --------- <snip>
I like this too, it's a vastly superior extension to a few saved queries that I use.
Big Picture ----------- <snip> That's my intent, anyway. You tell me if you think I'm headed in the right direction.
Looks like a good general direction to me. -- Zach
On Mon, Apr 21, 2014 at 11:42 PM, Zachary Ware <zachary.ware+pydev@gmail.com> wrote:
On Mon, Apr 21, 2014 at 11:04 AM, R. David Murray <rdmurray@bitdance.com> wrote:
OK, wall of text time. Sorry about this :)
You weren't kidding :P
Tracker Roles ~~~~~~~~~~~~~
The tracker roles important to the workflow are:
User Triager ('Developer') Committer
Although the tracker calls the Triager role 'Developer', I'm going to refer to it as Triager throughout this document, for clarity as to its intent.
The tracker does not currently have a role equivalent to 'committer', but we may not need to add one explicitly, since the account of a committer is marked as such.
The important thing to understand about these roles in the context of this document is that anything a user can do, a triager or committer can do, and anything a triager can do, a committer can do.
I think it may be worthwhile to actually implement these roles as named, and promote people to Triager a lot earlier in their Python career than we currently promote to committer. I think it would be worthwhile and help with retention to "reward" (with more work) people who have made a couple of non-trivial contributions and seem interested in doing more; people like, for example, Vajrasky Kok, Nikolaus Rath, and Todd Rovito who if I'm not mistaken only have the User role currently.
I think we are quite liberal in giving away the "Developer" role on the tracker. The main issue that it might take a core dev notices that someone has been actively contributing for a while and decide to check if they are Developers or not. If people request it on IRC they can generally get it sooner.
Versions ~~~~~~~~
Currently our versions field does double duty: we set it to the versions we want to fix the bug in, and then deselect versions as we fix them. However, I noticed when doing the "What's New" document that this makes things really confusing if you *do* look at the ticket later. You can't tell which versions got the fix without reading through the entire issue. So I'd like to split this into two fields:
should be fixed in: [list of versions] has been fixed in: [list of versions]
I'd also like patch set links to be displayed next to the versions for 'has been fixed'. When a commit references an issue, it should appear next to the appropriate version, but it should *not* automatically change the version state to fixed. That should require separate action, since we sometimes apply patches that are relevant to the issue but do not fix it. When an issue is transitioned to closed, there should be an easy way to say "mark all versions with changesets as fixed". It would probably be appropriate for that to be the default.
These fields have direct operational meaning: they indicate a task to be performed or signal the completion of a task.
As may become apparent as this message goes on, I'm a fan of checkboxes.
Note that we shouldn't limited ourself to basic form elements. In theory we could use any fancy js-powered widget (e.g. jqueryui, the one I used for the nosy autocomplete), as long as it improves usability and that the page is still usable when js is not available. This is an example I found googling: http://www.erichynds.com/examples/jquery-ui-multiselect-widget/demos/
I don't much like the current method for selecting multiple versions or components, which requires both keyboard and mouse to make the selections in most cases. Since it's not an extremely common form element on the modern web, some users (not necessarily just Users) may not even be familiar with how to pick and choose specific ones--I suspect this may be part of why we have so many issues created with every version selected, the user attempted to choose 2.7 and 3.current but didn't know how to only select the two. So, my suggestion here (also colored by Barry's thoughts on release blocking) is to have a series of checkboxes for each version, "affected", "blocks release", and "fixed". Or even just "affected" and "fixed", with a priority box per version.
Type ~~~~ documentation python bug interpreter crash security enhancement request
I add 'documentation' here based on PyCon feedback from Jessica McKeller and Selena Deckelmann. None of the existing types makes sense to a beginner for a documentation bug, and the resulting confusion can lead someone to abandon the effort to contribute. It also has operational implications (see below).
I rename 'behavior' to 'python bug' for a similar reason, but I don't have any usability data to back that feeling, so I'm not strongly advocating that change.
I rename 'crash' to 'interpreter crash' in an attempt to reduce the chances that someone will report a python traceback as a crash, something that otherwise happens very regularly. I'm sure we'll never completely eliminate those.
Perhaps it could be made even clearer by naming them "python bug or traceback" and "interpreter crash (segfault)", or even just calling "interpreter crash" "segfault/abort".
I drop 'compiler error', 'resource usage' and 'performance'. All of these are bugs in the minds of the reporters, and classifying them separately in the 'type' field does not, as far as I can see, lead to any operational difference in the way the issue is treated.
I think it would be good to keep these as "components", as Antoine suggested.
Component ~~~~~~~~~
I propose that we completely change the nature of this item. I think we should make it a text field that is filled in by autocomplete like the nosy field can be, and that the value be any of the components listed in the experts file. This would further respect how the experts have listed themselves in that file by autonosying those that are willing for that to happen. The field should still be a multilink (that is, can contain more than one value).
This change would mean that it would be possible to search for issues based on module, which is an often-requested feature.
I like this idea quite a bit, especially changing the type of input widget.
Patch Status ~~~~~~~~~~~~
This is a new set of fields that records information about the patch:
unit test fix documentation changes news entry what's new entry commit message
For each of these, the value could be 'needs work', 'complete', or 'not applicable'. For issues of type 'documentation', all lines except 'documentation changes' and 'commit message' would be set to NA by default, otherwise they will be set to 'needs work' initially, except for "what's new", which will be set to NA for anything except type enhancement.
Instead of having 3 textual options for each field, I'd prefer checkboxes (or perhaps a series of checkboxes). For each field, have three checkboxes, "needed", "present" and "ready". "needed" would be visible only to Triager+; unchecked fields are not needed for a complete patch and that field is hidden entirely from Users. In the majority of cases, "present" could be set by sniffing through a patch when it is attached; if a patch contains "^diff.+Lib/.*test.*\.py$", chances are very good that it has a unit test.
This can be done, but I would prefer a better approach where the exact file names are extracted and normalized from each patch like I did for http://wolfprojects.altervista.org/issues.html
The "present" checkbox should probably just be a graphic for Users, but available to Triager+ to clean things up where automation fails. "ready" would be checked by a reviewer who approves that particular part of a patch.
Keywords ~~~~~~~~ <snip> I think the keywords should be settable by anyone, if they aren't already.
They're not, unless it's changed since November.
Issue States and State Transitions ----------------------------------
Here are the states that I think an issue can be in:
new needs information needs decision needs decision from committer needs patch needs review needs patch update needs commit review closed/fixed closed/wont fix closed/duplicate closed/insufficient information closed/postponed closed/not a bug closed/third party
I discuss each of these, and the possible state transitions, below.
All legal state transitions should be displayed in a particular area in the UI, and each transition should be a single radio button. Which possible transitions will be displayed will depend on the user's roundup role, and occasionally other state in the issue.
The default transition is "no change".
Note that the names of these states are, unlike most of the other names I've used so far, not the ones I necessarily expect us to use in the UI. They are descriptive of the state from the workflow POV, not necessarily the labels that should be used for the state *transitions*.
I like this scheme, and I think the split of abilities between roles is reasonable, as long as we promote people to Triager a lot faster than we currently do.
As I mentioned above, we aren't slow because we don't want to promote people, but just because the promotion only happens when a core dev happens to notice that someone deserving that role still doesn't have it (and knows how to upgrade the role). Maybe showing the role more clearly (with an icon like the committer one) would make it easier to notice when someone is not a Triager yet. Best Regards, Ezio Melotti
Dashboard --------- <snip>
I like this too, it's a vastly superior extension to a few saved queries that I use.
Big Picture ----------- <snip> That's my intent, anyway. You tell me if you think I'm headed in the right direction.
Looks like a good general direction to me.
-- Zach
On Tue, Apr 22, 2014 at 8:27 AM, Ezio Melotti <ezio.melotti@gmail.com> wrote:
As I mentioned above, we aren't slow because we don't want to promote people, but just because the promotion only happens when a core dev happens to notice that someone deserving that role still doesn't have it (and knows how to upgrade the role). Maybe showing the role more clearly (with an icon like the committer one) would make it easier to notice when someone is not a Triager yet.
Sorry, I didn't mean to imply that we don't want to promote people (or that we do it slower than we should). The only real data I have behind my view is my own experience, where I went straight from a plain User on the tracker to a full-blown committer with nothing in-between and basically assumed it's pretty much the same for most. I think I may have benefited from having the Developer role earlier on, but I didn't want to ask for it (for fear of seeming greedy or something) and didn't really know who should be asked, anyway. I think an icon showing what role people have would be a good thing; I for one don't know how to tell who has what role (and don't know how to upgrade someone's role, either). -- Zach
On Apr 22, 2014 6:27 PM, "Zachary Ware" <zachary.ware+pydev@gmail.com> wrote:
On Tue, Apr 22, 2014 at 8:27 AM, Ezio Melotti <ezio.melotti@gmail.com>
wrote:
As I mentioned above, we aren't slow because we don't want to promote people, but just because the promotion only happens when a core dev happens to notice that someone deserving that role still doesn't have it (and knows how to upgrade the role). Maybe showing the role more clearly (with an icon like the committer one) would make it easier to notice when someone is not a Triager yet.
Sorry, I didn't mean to imply that we don't want to promote people (or that we do it slower than we should).
No need to apology, I was just pointing out why we don't promote people more actively.
The only real data I have behind my view is my own experience, where I went straight from a plain User on the tracker to a full-blown committer with nothing in-between and basically assumed it's pretty much the same for most.
Usually this should happen before, but it's easy to miss it. It's also likely that at some point people assumed you must have been a triager already since you were very active and didn't bother double-checking. I was added to Misc/ACKS after becoming a committer because everyone thought I was there already :)
I think I may have benefited from having the Developer role earlier on, but I didn't want to ask for it (for fear of seeming greedy or something) and didn't really know who should be asked, anyway.
IRC is good for this.
I think an icon showing what role people have would be a good thing; I for one don't know how to tell who has what role (and don't know how to upgrade someone's role, either).
It can be seen from the user page, but it might require some privileges to change it. Best Regards, Ezio Melotti
-- Zach
On Tue, Apr 22, 2014 at 11:44 AM, Ezio Melotti <ezio.melotti@gmail.com> wrote:
On Apr 22, 2014 6:27 PM, "Zachary Ware" <zachary.ware+pydev@gmail.com> wrote:
I think an icon showing what role people have would be a good thing; I for one don't know how to tell who has what role (and don't know how to upgrade someone's role, either).
It can be seen from the user page, but it might require some privileges to change it.
Hmmm, I'm not seeing it. Looking at your user page, I see fields for Name, Login Name, Organisation, Timezone, Homepage, Contributor Form Received, Is Committer, E-mail address, Alternate E-mail, and Associate OpenID, but nothing about Roles. On my own page, the only additions are password fields, though I can see in the History section (which is only visible on my own page) "2013-11-02 10:24:29 brett.cannon set roles: User -> User,Developer". Other non-privileged users' pages look the same as yours to me, though if they have an associated OpenID I have a "Drop this OpenID" button. -- Zach
On Tue, Apr 22, 2014 at 8:05 PM, Zachary Ware <zachary.ware+pydev@gmail.com> wrote:
On Tue, Apr 22, 2014 at 11:44 AM, Ezio Melotti <ezio.melotti@gmail.com> wrote:
On Apr 22, 2014 6:27 PM, "Zachary Ware" <zachary.ware+pydev@gmail.com> wrote:
I think an icon showing what role people have would be a good thing; I for one don't know how to tell who has what role (and don't know how to upgrade someone's role, either).
It can be seen from the user page, but it might require some privileges to change it.
Hmmm, I'm not seeing it. Looking at your user page, I see fields for Name, Login Name, Organisation, Timezone, Homepage, Contributor Form Received, Is Committer, E-mail address, Alternate E-mail, and Associate OpenID, but nothing about Roles. On my own page, the only additions are password fields, though I can see in the History section (which is only visible on my own page) "2013-11-02 10:24:29 brett.cannon set roles: User -> User,Developer". Other non-privileged users' pages look the same as yours to me, though if they have an associated OpenID I have a "Drop this OpenID" button.
Apparenlty only Coordinators can see/edit it (there are a few core devs that are also coordinators). I don't think there are any problems to make the roles visible to everyone, however I don't know if there are problems to make the roles editable (if we make it editable for developers maybe they could make themselves coordinators by changing their role?). Either way we could investigate it and fix it if we need to. Best Regards, Ezio Melotti
-- Zach
On Tue, Apr 22, 2014 at 12:22 PM, Ezio Melotti <ezio.melotti@gmail.com> wrote:
Apparenlty only Coordinators can see/edit it (there are a few core devs that are also coordinators). I don't think there are any problems to make the roles visible to everyone, however I don't know if there are problems to make the roles editable (if we make it editable for developers maybe they could make themselves coordinators by changing their role?). Either way we could investigate it and fix it if we need to.
I definitely think making roles visible to all (like the Committer icon, not just buried in the user page) would be a good thing. I agree that allowing Developers to change roles has the potential for causing more trouble than it is worth. -- Zach
Hi David, Thanks for taking the time to write this up. Aside from some small concerns, for example around auto-closing tickets, which others have already voiced, I just wanted to say that I'm enthusiastic about this outline. In particular, I'm enthusiastic about making it clearer what tasks are available for new contributors, triagers, and committers. I think this will have a big impact on getting and retaining contributors.
'easy' of course has to be set manually. The system for listing easy issues should list the issues only when there is an action that someone could take on the issue (clarify issue, write patch, review patch). (NB: it might be possible to come up with a better name for this tag.)
Some folks get discouraged by "easy" tickets not seeming that easy. A common alternative tag is "bitesized". Straying off topic to talk about workflow documentation: The easy tag is often not set consistently by projects (or it's forgotten until a dash to comb through tickets before sprints to add it). Making the easy tag a triage checklist item could help with that. I think new contributors would also appreciate the clarity of a patch submission checklist. I am in general a big fan of checklists. :) I also volunteer myself to help update the devguide as the workflow gets updated. -Jessica
Hi, On Mon, Apr 21, 2014 at 7:04 PM, R. David Murray <rdmurray@bitdance.com> wrote:
I'm volunteering to be the coordinator for the work, and I'm also volunteering to do as much of it as necessary. That is, I'm planning to make this the focus of most of the "python time" I have available.
My Python time lately is very limited (it might get better during/after the summer), but as one of the main maintainer of the tracker I'll try to follow and give my input wherever is possible. If someone wants to work on the tracker I can also provide guidance/help (you can ping me on IRC).
I think that there is a way to implement this workflow or whatever workflow we decide on as a "new interface" while keeping the existing interface available, allowing us to test and refine (like, maybe do some usability testing? :). Roundup has a way to dynamically specify which page template is used to render a particular page type, and I think we can leverage that to have two parallel UIs. I could be wrong, though, in which case we'd need to set up a test tracker instead, which I can do. (Assuming it does work, there might need to be a bit of glue code to keep things in sync. And there would need to be some changes in the data values available in the current UI, but I don't think that would be a bad idea anyway.)
This should be doable. We can have a separate page with a completely different template and layout (and things like js folding/autocomplete/etc), and as long as the values (for e.g. status, resolution, etc.) are the same everything should work fine. If you want to change the values then it gets a bit more tricky. Roundup also supports Jinja templates now, so we could use them if it makes things easier (the alternative is TAL). Matching the www.python.org style (and update to html5) can also be done, but that's a different issue.
The scope of this discussion is the workflow for an "issue", which currently means an entry created by someone in the bug tracker at bugs.python.org, and goes from the creation of the issue to the resolution of the issue, which can but does not necessarily include committing something. I won't be discussing the tooling for Zuul-like patch gating at this stage, I'll just assume we can figure out how to implement that.
It would also be possible to build a patch gating system without initially integrating it with our other tools, and if we decide that's more important than the tracker workflow at this stage, we should switch our conversation to what that would look like. Or perhaps we can work on the two in parallel.
My own feeling is that in order to get maximum benefit out of patch gating, we need more clarity and utility in our *tracker* workflow first,
Having better integration between roundup and rietveld would be a good step in that direction. Having Roundup be more aware of patches (if they are patches or other attachments, what files they touch, on what branches they apply cleanly) would be another good step. I did some work on this (especially on analyzing patches) that would be good to be integrated with the tracker.
but I do realize that how long it currently takes to commit a patch is a significant pain point, and it might be better to address that first.
Goals -----
In suggesting improvements to the existing workflow support, I'm starting from the fundamental idea that anything we do in the issue tracker and/or patch gating system should have either *operational* implications or *important* informational implications. The former should be more heavily weighted than the latter, in my opinion.
What I mean by this is that there should be no "busy work" setting of attributes in our workflow: when you change the state of an issue, it should mean that something new is *going* to happen to that issue. Any purely informational attribute setting should be in support of that new action, and wherever possible that information should *mean* something to some part of our tooling, not *just* to the humans looking at the issue.
In general I want there to be as few clicks as practical involved in updating an issue. However, I'm not addressing the actual UI design here (although I have ideas); I assume we are going to iterate on it for a bit until we have something we really like.
NB: This proposal includes a number of ideas from the "desired tracker features" (http://wiki.python.org/moin/DesiredTrackerFeatures), but by no means all of them. Some of the others are worth implementing, but in this document I'm focusing on primary workflow, which I think should happen first.
Roles -----
Conceptual Roles ~~~~~~~~~~~~~~~~
In thinking about our workflow, I identify the following roles:
original reporter commenter triager patch-producer reviewer core reviewer committer
These are in the order that the role is involved in the issue (roughly; variations are possible depending on the issue), and obviously a single person can take on multiple roles during the lifetime of a patch.
I originally thought this list would have operational implications, but it turned out to be only an aid in thinking about the problem. I'm leaving it in here for exactly that purpose...it helps when thinking about the states and state transitions.
I'd also like to add something that we currently only have informally, but which has been requested as a feature more than once: at every stage, I'd like there to be the possibility of there being a 'responsible party'. This is sort of like 'assigned to', except that it could be anyone with a tracker account, and the assignment would have a limited lifetime: either until the issue's state changes, or until the issue has been without update for some number of days (off the cuff I'd suggest 14, but it might also vary by state depending on what states it actually got used in).
I believe twisted uses a similar workflow, after a review the reviewer assigns the issue to whoever made the patch, and when the patch is updated the issue is re-assigned to the reviewer. This also applies if e.g. more information are needed. The ideas is that the "assigned to" field should always point to the person responsible for the next step that will move the issue forward.
The idea behind this is that we have eager contributors who either wind up stepping on each other's toes, or rush to create and submit a patch before someone else does, and as a consequence of rushing, do not produce as good a patch as they are capable of, which actually slows down the issue resolution. The ability to "take" a task and know it is "yours" is an important part of the new contributor process, and having a "responsible party" field would support that.
Is stepping on each other's toes a common issue? Note that "taking" a task might have the opposite effect (it's not difficult to find weeks/months-old messages that state "they will work on a patch soon"). Having two patches might be better than having zero.
Tracker Roles ~~~~~~~~~~~~~
The tracker roles important to the workflow are:
User Triager ('Developer') Committer
Although the tracker calls the Triager role 'Developer', I'm going to refer to it as Triager throughout this document, for clarity as to its intent.
The tracker does not currently have a role equivalent to 'committer', but we may not need to add one explicitly, since the account of a committer is marked as such.
The important thing to understand about these roles in the context of this document is that anything a user can do, a triager or committer can do, and anything a triager can do, a committer can do.
Information Fields ------------------
An issue has an issue number and a title, and those wouldn't change. It also has a state, which is the subject of the second half of this document, and would likely be a new field in order to be able to have both UIs available (this field is where the glue code would be needed).
Beyond those fields, I suggest several changes to the issue metadata; I will cover each one separately.
Versions ~~~~~~~~
Currently our versions field does double duty: we set it to the versions we want to fix the bug in, and then deselect versions as we fix them.
Most of the issues are fixed on all branches at the same time, so deselecting is IME a rare occurence.
However, I noticed when doing the "What's New" document that this makes things really confusing if you *do* look at the ticket later. You can't tell which versions got the fix without reading through the entire issue. So I'd like to split this into two fields:
should be fixed in: [list of versions] has been fixed in: [list of versions]
FWIW while triaging is sometimes select versions that are /likely/ to be affected by a bug, without actually checking, so "should be fixed in" might not always be accurate (unless we decide that it should be and leave version unselected until we have verified that they are affected).
I'd also like patch set links to be displayed next to the versions for 'has been fixed'. When a commit references an issue, it should appear next to the appropriate version, but it should *not* automatically change the version state to fixed.
It could also work the other way around: after (or before) the list of patches, we can have a list of related changesets that is updated automatically by the hg hook (or detected from the python-dev messages). Next to the changeset we could put the version, and possibly a checkbox to indicate that the changeset fixed the issue -- however I don't think this is necessary. If the issue has some related changesets and it's open, then more work is required (either because not all the versions are fixed, or because it was a preliminary set of changes that will be followed by another set). If it's closed, the listed changesets fixed the issue. This is to say, I'm not sure that separating versions in two different lists is a good idea, since it adds more fields and more work for the triagers/committers.
That should require separate action, since we sometimes apply patches that are relevant to the issue but do not fix it. When an issue is transitioned to closed, there should be an easy way to say "mark all versions with changesets as fixed". It would probably be appropriate for that to be the default.
This could be a solution to the "more work for triagers/committers". We could also have a single list of versions and transition from affected to fixed.
These fields have direct operational meaning: they indicate a task to be performed or signal the completion of a task.
Type ~~~~ documentation python bug interpreter crash security enhancement request
I add 'documentation' here based on PyCon feedback from Jessica McKeller and Selena Deckelmann. None of the existing types makes sense to a beginner for a documentation bug, and the resulting confusion can lead someone to abandon the effort to contribute. It also has operational implications (see below).
This seems reasonable to me.
I rename 'behavior' to 'python bug' for a similar reason, but I don't have any usability data to back that feeling, so I'm not strongly advocating that change.
Why not just bug?
I rename 'crash' to 'interpreter crash' in an attempt to reduce the chances that someone will report a python traceback as a crash, something that otherwise happens very regularly. I'm sure we'll never completely eliminate those.
I drop 'compiler error', 'resource usage' and 'performance'. All of these are bugs in the minds of the reporters, and classifying them separately in the 'type' field does not, as far as I can see, lead to any operational difference in the way the issue is treated.
What I mean by that: most bugs are either 'documentation' or 'python bug' or 'enhancement request'.
Just floating an idea: what if we only keep these 3 (or maybe just bug/enhancements) and use tags for things like performance, security, crash, documentation, etc? We could have a crash that can turn in a security issue, or resource usage that causes bad performances and with tags this could be described easily (and we could use them instead of components too). This is quite a big change, but other bug tracking systems use them quite successfully (to the point of having tags (possibly divided in categories and using different color codes) tracking versions, types, priority, etc.).
Differentiating these is important, as doc bugs are handled very differently from python bugs (documentation fixes do not get NEWS entries, for example), and bug vs enhancement determines which versions we fix things in. 'compiler error', 'resource usage', and 'performance', on the other hand, are all handled with the same workflow that applies to bugs. You might think that there is a difference for resource usage and performance, in that we don't in general backport those fixes. The key there, though, is "in general". The decision as to which versions to apply the fix is made based on the magnitude of the bug, and there really are only two cases: we mostly don't backport, as is the case for enhancements, but we sometimes do backport, just like we would a bug fix. The same applies in reverse to compile bugs: we mostly fix those in all versions, but we don't always. So the only *operational* effect of having these as distinct types would be to confuse things, since we couldn't tell from the type whether or not this was something that should be applied to all versions or only default. Instead we should set (or, more likely, triage) them as either 'python bug' or 'enhancement', which have the correct operational implications.
There *is* an operational reason for having security and interpreter crash as separate types. In both of these cases the versions we fix it in is always the same (all active for crashers, all active + all security-only for security bugs), and the issue priority should default to either high or release blocker. In addition, security bugs should be automatically reported to the PSRT, and arguably the report should be be hidden from all but the PSRT and the original reporter.
See http://psf.upfronthosting.co.za/roundup/meta/issue393 and http://psf.upfronthosting.co.za/roundup/meta/file275/issue393.png ) for a meta-tracker patch that warns about the PSRT while reporting security issues.
Priority ~~~~~~~~
low normal high deferred blocker release blocker
The only change here is to eliminate 'critical'. I'm not wedded to that, but if we have both 'critical' and 'high' priorities, 'high' ends up getting treated as pretty much equivalent to either 'normal' or 'critical', depending on the person. I would argue that anything that is severe enough to be marked 'critical' should in fact be a release blocker, and anything that is not a release blocker is effectively only 'high' priority.
SGTM.
The priority is currently operational only in that one can sort issues by priority. I propose that we make them much more operational, by posting a count and/or list of the bugs with more than normal priority somewhere public on a regular basis, such as python-dev, python-committers, and/or #python-dev. (Well, definitely #python-dev.)
I'd like to see us strive keep those queues clear, so that promoting a bug to high or release blocker means it *will* get acted on reasonably promptly. (This raises the issue of what to do about bugs we currently mark as "release blocker" as a *reminder* to the release manager. I don't have a proposal for that at the current time, as release management is out of scope for this document, but we'll need an answer if we are going to implement this.)
The release managers of the affected versions are already automatically added to the nosy list of the issue if the priority is set to release blocker (I think the script should be updated for 3.5).
Component ~~~~~~~~~
I propose that we completely change the nature of this item. I think we should make it a text field that is filled in by autocomplete like the nosy field can be, and that the value be any of the components listed in the experts file. This would further respect how the experts have listed themselves in that file by autonosying those that are willing for that to happen. The field should still be a multilink (that is, can contain more than one value).
This sounds similar to the tag system I mentioned above.
This change would mean that it would be possible to search for issues based on module, which is an often-requested feature.
I'm still not sure having a "tag" for each module would be a good idea. Taking them from the experts index would at least solve the problem of modules that are not available on specific versions or that have been renamed. One thing I would like to do is getting the affected module from the patch (see http://wolfprojects.altervista.org/issues.html ), but that only work when a patch is there. If we add per-module tags, we could add them automatically when a patch is submitted.
Patch Status ~~~~~~~~~~~~
This is a new set of fields that records information about the patch:
unit test fix documentation changes news entry what's new entry commit message
For each of these, the value could be 'needs work', 'complete', or 'not applicable'. For issues of type 'documentation', all lines except 'documentation changes' and 'commit message' would be set to NA by default, otherwise they will be set to 'needs work' initially, except for "what's new", which will be set to NA for anything except type enhancement.
This sounds like a lot of new field / UI clutter / work for the triagers, even though this could be automated somehow by looking at the files touched by the patch.
Note that the inclusion of 'news entry' and 'commit message' assume two things: that we retool the NEWS file so that NEWS entries can be added automatically without conflicts, and that we change our patch workflow to use mercurial's capability to accept 'hg export' patches and/or some other sort of pull request. This part is one place we get into non-roundup territory, so that may be a phase two addition.
Mercurial's 'evolve' feature, which we saw demoed at PyCon and which I understand is currently available as a plugin, makes this capability much more useful. When the non-core committer syncs the master repository after their patch has been committed, the right thing happens to the history in their repository to make their local commit disappear in favor of the commit made to master. The contributor will also be able to essentially *edit* the patch in their local repository based on the review feedback, without ending up with a chain of commits in their local repo that they have to deal with. Evolve should also facilitate DVCS-based collaboration between core and non-core developers, as well as between non-core developers. (I'm assuming all this works with exported changesets, since in the demo they talked about starting with a patch received via email, which is their normal workflow. But we should double check this.)
These fields allow us to represent all the parts that a patch must have to be complete, and they act as a checklist for making sure the parts are all there. This is similar in intent to patchcheck, but since which pieces are needed is ultimately determined by a human, it is more accurate than that part of patchcheck and therefore more useful.
Maybe this could be implemented in the patch list, as a series of ✓ added automatically: File name |fix|tst|doc|nws| ------------+---+---+---+---+ patch1.diff | ✓ | | | | patch2.diff | ✓ | ✓ | | | patch3.diff | ✓ | ✓ | ✓ | ✓ |
Stage ~~~~~
This goes away, subsumed into state.
Status ~~~~~~
This either holds the state information, or goes away in favor of a new 'state' field.
Keywords ~~~~~~~~
buildbot easy stuck
buildbot and easy are the only two existing, non-redundant tags that I can see a way to make operational.
'easy' of course has to be set manually. The system for listing easy issues should list the issues only when there is an action that someone could take on the issue (clarify issue, write patch, review patch). (NB: it might be possible to come up with a better name for this tag.)
Having an icon for issues with patches in the issue list (similar to the attachment icon in the mail list) would easily show what issues still don't have a patch.
For buildbot, those issues should be displayed in a report or dashboard, and we should try to keep this queue empty. This becomes even more important when we have a patch gating system online.
First we should turn all the buildbot green and have email notification to python-checkins whenever a stable buildbot turns red.
All of the other values are informational-only, and in some cases redundant. If we want informational keywords, IMO we should do a full user-accessible tagging system. That would be a separate proposal, though.
Do you mean that users can create their own tags on the fly? I would rather define a fixed set of available tags if we go down this route.
Note: I claim the regression tags are redundant because I think regression issues should be release blockers. (They are also ambiguous: is '3.3regression' a bug in 3.3 that is a regression relative to 3.2, or a 3.3.x bug relative to 3.3.x-1, or a 3.4 bug that is a regression relative to 3.3? Better to just explain exactly what is going on in the issue comments, IMO.)
The purpose of the 'stuck' tag is to label issues that we agree are real issues that we don't want to close as "won't fix", but that we can't for one reason or another currently move forward. Operationally, stuck issues are either not displayed in work queues (see below) or are displayed at the end of the queue.
Is this a replacement for languishing?
I think the keywords should be settable by anyone, if they aren't already.
They should be, I think only the stage is not settable by everyone (and it should be IMHO).
Issue States and State Transitions ----------------------------------
Here are the states that I think an issue can be in:
new needs information needs decision needs decision from committer needs patch needs review needs patch update needs commit review closed/fixed closed/wont fix closed/duplicate closed/insufficient information closed/postponed closed/not a bug closed/third party
I discuss each of these, and the possible state transitions, below.
These are a lot of values to add, and the workflow you suggest seems quite complex. OTOH it seems to have operational advantages, so it might be worth giving it a chance, especially if it has a good UI,
All legal state transitions should be displayed in a particular area in the UI, and each transition should be a single radio button.
A radio button is not a good UI :) Maybe something like "This issue | needs information |▼|.", with the dropdown showing all the possible states that can follow "needs informations".
[... list of all the states ...]
Dashboard ---------
The dashboard would be a new feature, a new landing page and linked from the left hand menu bar. It would list the first N (adjustable) items in each queue relevant to the user for which they are the responsible party, followed by those on which the user is nosy, followed by those on which they are not nosy.
A general user would see the 'new', 'needs patch', 'needs patch update', and, 'needs review' queues.
A triager would see, above the preceding, the 'needs decision' queue.
A committer would see, above the preceding, the 'needs decision from committer' and 'needs commit review' queues.
In all queues, issues would be sorted in priority order followed by most recent activity, with color coding for the priority. 'stuck' issues would appear last, and there should be something to visually differentiate issues on which the user is responsible, just nosy, or not nosy.
As with everything else, this is of course subject to discussion. It would also be lovely if the user could configure their dashboard, but that would probably fall into the category of advanced features we come back to later.
What happens to the other pages?
Big Picture -----------
My goal here is to facilitate the involvement of the wider community in our workflow as much as possible. The structure above is designed to allow the community to do as much work as possible, and the "trusted individuals" to act as gatekeepers to insure quality. It is theoretically possible for a patch to get all the way to 'needs commit review' without any more higher level involvement than a triage person moving the issue to 'patch needed'. Of course in reality much more involvement from core will be needed, since we need to transfer knowledge to the community, and especially the newcomers, about our standards and procedures, Not to mention the code. And committers are going to *want* to do general commentary and triage level activities on issues anyway. But, for those committers who have less time, I'd like to think that this system would allow them to focus their time on just the stuff only they can do, and thus perhaps draw more contributions from some of our less active or currently-inactive committers.
The key to this is that instead of issues getting lost, core can watch certain key queues: 'needs decision', 'needs committer decision', and 'needs commit review'. When issues get to those stages, the people with the experience to do something know that it is *their* responsibility to do it: the issues are to a point where the general community can't help without core input. If we can keep those queues to a manageable size, I think we can increase the amount of energy coming our way.
What if we can't keep them manageable? There's a fine line between managing to handle all the incoming issues and having issues start leaking in and giving up on the goal of keeping the queues empty. When your queue has 3 issues, well, you could just take a look now and make it empty, but you don't quite have time right now for looking at 5 issues... Maybe later (or tomorrow, or during the weekend) you will have more time. I think many of you had something similar happening with your mail inbox (or new year resolutions, or similar things). I'll also suggest another related (and "controversial") idea. People like to reach goals: if they address the 3 issues in their queue they have reached the "empty queue" goal. Addressing 3 of the 5 issues isn't quite the same thing. I've seen this concept being exploited in three main ways: 1) badges/trophies/achievements; 2) competitions; 3) streaks; The first means that the user can get a badge because they closed their 10th issues, or triaged their 50th, or submitted 5 patches, being the first to reply on an issue for the 10th time, or whatever. Even if fixing 3 out of 5 issues won't make you reach the "empty queue" goal, maybe you can reach the "10 closed issues". An example of this are StackOverflow badges (e.g. http://stackoverflow.com/users/95810/alex-martelli?tab=badges ). The second includes "leaderboards" or "awards" for being above average. Examples of this are Twisted high score (http://twistedmatrix.com/highscores/) or charts like http://www.ohloh.net/p/python/contributors (at some point I was the most active contributor and was trying to keep my contributions going, but then Serhiy became a contributor... :). Something similar could be done by mentioning "exceptional" results in the weekly summary report (e.g. people who fixed/contributed to the most issues). The third is about perseverance. Every day/week you have a goal to meet, if you reach it you streak counter increases, if you miss it the counter starts again from zero. Once you start building up a high count, you really don't want to miss your goal and start everything from scratch. Here the goal might be close 3 issues per week, or something similar, and could have associate badges ("contribute every day for a month", "close 3 issues per week for 3 weeks in a row", etc) While I understand that probably most of the core devs would be against similar things, this might motivate new users and make them "addicted" to the tracker, while making their experience more enjoyable, and the example I linked show that similar things exist even in these environments (and not only on the micro-transaction based smartphone games :). People who don't care about this (different people are more or less competitive) could just ignore it. OTOH this might have a negative side-effect if users start closing issues randomly just to get the "100 closed issues" badge, but this is not difficult to avoid.
The other important goal is that via these explicit state transitions, especially the 'needs decision' and 'needs commit review' transitions, we get a much clearer picture of who in the community *has* absorbed the core standards and procedures and is ready to be "promoted" to the next level of responsibility. And the more people we get moved up, the more we can get done. (Note: making that information *easily* accessible requires additional tooling, but it should be possible and should be done.)
Note also that I tried to engineer this so that the structure does not handicap us or add bureaucracy: even though the basic structure is that you get triage signoff first and then committer signoff, if the person doing the triage is a committer, they can just move the issue to wherever it needs to be. So the only time two levels of signoff will be *required* is when the committers are too busy to drain the queues. And work on an issue should never be *blocked* by the system, the purpose of the queues is to draw attention to issues that are "ready" for decision. Thus the two levels help us manage the load when we need the help, and don't get in our way otherwise.
That's my intent, anyway. You tell me if you think I'm headed in the right direction.
I would like to see a summary/mock-up of what the end result would be after all these fields are added/removed. This will give a better idea about the complexity of the new workflow, but there are definitely good suggestions in there (even though it's a lot of work). Best Regards, Ezio Melotti
On 22 Apr 2014 00:06, "Ezio Melotti" <ezio.melotti@gmail.com> wrote:
What if we can't keep them manageable? There's a fine line between managing to handle all the incoming issues and having issues start leaking in and giving up on the goal of keeping the queues empty. When your queue has 3 issues, well, you could just take a look now and make it empty, but you don't quite have time right now for looking at 5 issues... Maybe later (or tomorrow, or during the weekend) you will have more time. I think many of you had something similar happening with your mail inbox (or new year resolutions, or similar things).
Then that becomes data folks like me, Toshio Kuratomi, Jesse Noller and Van Lindberg can take to our respective management chains to say "Hey, we have this critical upstream dependency that needs more developer time to keep up with its workload, can we have a hiring req, please". That's the other advantage of setting up core development as clearly being a service role for the rest of the community - it's a way to bring full or part time paid development into the picture without generating resentment amongst volunteer contributors, by specifically tasking the paid developers with ensuring that roadblocks are cleared out of the paths of volunteers. Where Tim Peters used to say "we read Knuth so you don't have to", the ideal of paid open source development for me is "paid staff do the boring-but-necessary bits so volunteers don't have to". (I think payment is also appropriate for essentially "on call" roles like the security response team) Cheers, Nick.
On Tue, Apr 22, 2014 at 7:06 AM, Ezio Melotti <ezio.melotti@gmail.com> wrote:
I'll also suggest another related (and "controversial") idea. People like to reach goals: if they address the 3 issues in their queue they have reached the "empty queue" goal. Addressing 3 of the 5 issues isn't quite the same thing. I've seen this concept being exploited in three main ways: 1) badges/trophies/achievements; 2) competitions; 3) streaks; The first means that the user can get a badge because they closed their 10th issues, or triaged their 50th, or submitted 5 patches, being the first to reply on an issue for the 10th time, or whatever. Even if fixing 3 out of 5 issues won't make you reach the "empty queue" goal, maybe you can reach the "10 closed issues". An example of this are StackOverflow badges (e.g. http://stackoverflow.com/users/95810/alex-martelli?tab=badges ). The second includes "leaderboards" or "awards" for being above average. Examples of this are Twisted high score (http://twistedmatrix.com/highscores/) or charts like http://www.ohloh.net/p/python/contributors (at some point I was the most active contributor and was trying to keep my contributions going, but then Serhiy became a contributor... :). Something similar could be done by mentioning "exceptional" results in the weekly summary report (e.g. people who fixed/contributed to the most issues). The third is about perseverance. Every day/week you have a goal to meet, if you reach it you streak counter increases, if you miss it the counter starts again from zero. Once you start building up a high count, you really don't want to miss your goal and start everything from scratch. Here the goal might be close 3 issues per week, or something similar, and could have associate badges ("contribute every day for a month", "close 3 issues per week for 3 weeks in a row", etc)
While I understand that probably most of the core devs would be against similar things, this might motivate new users and make them "addicted" to the tracker, while making their experience more enjoyable, and the example I linked show that similar things exist even in these environments (and not only on the micro-transaction based smartphone games :). People who don't care about this (different people are more or less competitive) could just ignore it. OTOH this might have a negative side-effect if users start closing issues randomly just to get the "100 closed issues" badge, but this is not difficult to avoid.
See also the "Gamification" section of https://en.wikipedia.org/wiki/Overjustification_effect#Applications .
On Tue, 22 Apr 2014 07:06:53 +0300 Ezio Melotti <ezio.melotti@gmail.com> wrote:
I'll also suggest another related (and "controversial") idea. People like to reach goals: if they address the 3 issues in their queue they have reached the "empty queue" goal. Addressing 3 of the 5 issues isn't quite the same thing. I've seen this concept being exploited in three main ways: 1) badges/trophies/achievements; 2) competitions; 3) streaks;
[...]
While I understand that probably most of the core devs would be against similar things, this might motivate new users and make them "addicted" to the tracker, while making their experience more enjoyable, and the example I linked show that similar things exist even in these environments (and not only on the micro-transaction based smartphone games :). People who don't care about this (different people are more or less competitive) could just ignore it. OTOH this might have a negative side-effect if users start closing issues randomly just to get the "100 closed issues" badge, but this is not difficult to avoid.
Not difficult how? In any gamification system, people will work towards getting new rewards / awards, not towards making meaningful contributions. I think something like the Twisted high scores is acceptable (since it's quite un-serious), but starting displaying awards will really bias how people contribute (with a definite emphasis on quantity over quality, IMO). (it's the same reason I'm rather ambiguous on the whole idea of sprints) I think trying to ensure we actually *thank* people goes a long way towards achieving the same goal, but without the bias. Regards Antoine.
On 22 April 2014 19:51, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Tue, 22 Apr 2014 07:06:53 +0300 Ezio Melotti <ezio.melotti@gmail.com> wrote: Not difficult how? In any gamification system, people will work towards getting new rewards / awards, not towards making meaningful contributions. I think something like the Twisted high scores is acceptable (since it's quite un-serious), but starting displaying awards will really bias how people contribute (with a definite emphasis on quantity over quality, IMO).
While I think gamification done right is actually a good way to build community, I also think we have a lot more fundamental issues just smoothing the path for the people that *already* want to contribute. For example, different people have different expectations regarding the cycle times where their efforts will have an impact - whether they want to make a difference in a few weeks, in a few months or in a few years. We're actually in a position to start channelling people more appropriately on that front, since we do different things on all those time scales (weeks: documentation, infrastructure & core workflow tools; months: CPython bug fixes, alternative interpreter development and packaging & distribution tools; years: Python language design and new feature development). Yet we don't currently make it clear that if people "want to help improve Python", there are actually several different ways to go about it according to their skills and inclinations. It's an ever-evolving process, just as the language and standard library will continue to evolve in response to the changes in the world around us.
(it's the same reason I'm rather ambiguous on the whole idea of sprints)
For me, sprints are mostly useful from the perspective of having high bandwidth feedback opportunities, as well as personalising the experience of contribution in a way that isn't easy over IRC, email or the issue tracker.
I think trying to ensure we actually *thank* people goes a long way towards achieving the same goal, but without the bias.
Good metrics are actually a useful way to know who to thank, though. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On mar., 2014-04-22 at 22:17 -0400, Nick Coghlan wrote:
I think trying to ensure we actually *thank* people goes a long way towards achieving the same goal, but without the bias.
Good metrics are actually a useful way to know who to thank, though.
I'm talking about spontaneous thanks by a committer when someone makes a welcome contribution, not institutional / bureaucratic thanks à la PSF awards. One of the strong points of our community is that it's still rather human, despite its growth. Regards Antoine.
On Wed, Apr 23, 2014 at 5:17 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 22 April 2014 19:51, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Tue, 22 Apr 2014 07:06:53 +0300 Ezio Melotti <ezio.melotti@gmail.com> wrote: Not difficult how? In any gamification system, people will work towards getting new rewards / awards, not towards making meaningful contributions. I think something like the Twisted high scores is acceptable (since it's quite un-serious), but starting displaying awards will really bias how people contribute (with a definite emphasis on quantity over quality, IMO).
While I think gamification done right is actually a good way to build community, I also think we have a lot more fundamental issues just smoothing the path for the people that *already* want to contribute.
Sure, I'm just floating an idea, and this is clearly low-priority compared to many of changes David suggested. It's also quite a bit of work to implement it.
For example, different people have different expectations regarding the cycle times where their efforts will have an impact - whether they want to make a difference in a few weeks, in a few months or in a few years. We're actually in a position to start channelling people more appropriately on that front, since we do different things on all those time scales
I never thought about it as "I want to do something that will improve Python within a week/month/year", and I'm not sure other do. People work on the documentation or in the infrastructure because they can and know how to improve them, not because their efforts will be rewarded sooner. The two point of confusions about timescale I see most often are: 1) features can only go on "default", but some people expect them to be available for the next 2.7 release too (and sometime this kills their motivation); 2) docs are not updated in real time, so some contributors ask "why I still see the old docs even if the fix has just been committed?".
(weeks: documentation, infrastructure & core workflow tools; months: CPython bug fixes, alternative interpreter development and packaging & distribution tools; years: Python language design and new feature development). Yet we don't currently make it clear that if people "want to help improve Python", there are actually several different ways to go about it according to their skills and inclinations.
This is very true. I'm sure there's plenty of talented web developers that could help out with the bug tracker. Same goes for many other parts of the infrastructure. Unfortunately most of these things happen behind the scenes and are hidden even to core-developers. (For a long time I wanted to improve things on the (old) website, but the fact that I didn't know where the code was, that there was no public bug tracker for it, and that almost none of the core devs were involved in it, prevented me to do so. With the new website things are better, but IMHO it would be even easier if the repo was in hg.python.org with the others -- I understand this has other implications though, and anyway it's getting off-topic.)
It's an ever-evolving process, just as the language and standard library will continue to evolve in response to the changes in the world around us.
(it's the same reason I'm rather ambiguous on the whole idea of sprints)
For me, sprints are mostly useful from the perspective of having high bandwidth feedback opportunities, as well as personalising the experience of contribution in a way that isn't easy over IRC, email or the issue tracker.
I agree, but there is usually an expectation that as many issues as possible should be closed during the sprint. I think most uninformed people would consider it a failure if only, say, 8 issues were closed during the PyCon sprints. What they don't consider is that during the sprint people were able to discuss better solutions, work on patches and proof of concepts, find out how to finally get around a showstopper, and that this will (hopefully) lead to a number of issue being closed in the following days/weeks. Rushing the commits to make the number higher might work from a marketing point of view ("the sprint was successful, we closed 83 issues!") , but it has other negative side-effects.
I think trying to ensure we actually *thank* people goes a long way towards achieving the same goal, but without the bias.
Good metrics are actually a useful way to know who to thank, though.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Wed, 23 Apr 2014 15:53:39 +0300, Ezio Melotti <ezio.melotti@gmail.com> wrote:
On Wed, Apr 23, 2014 at 5:17 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 22 April 2014 19:51, Antoine Pitrou <solipsis@pitrou.net> wrote:
(it's the same reason I'm rather ambiguous on the whole idea of sprints)
For me, sprints are mostly useful from the perspective of having high bandwidth feedback opportunities, as well as personalising the experience of contribution in a way that isn't easy over IRC, email or the issue tracker.
I agree, but there is usually an expectation that as many issues as possible should be closed during the sprint. I think most uninformed people would consider it a failure if only, say, 8 issues were closed during the PyCon sprints. What they don't consider is that during the sprint people were able to discuss better solutions, work on patches and proof of concepts, find out how to finally get around a showstopper, and that this will (hopefully) lead to a number of issue being closed in the following days/weeks. Rushing the commits to make the number higher might work from a marketing point of view ("the sprint was successful, we closed 83 issues!") , but it has other negative side-effects.
You might notice that I haven't issued an "N bugs closed" report for the PyCon sprint, nor (as far as I know) was there any rushing of commits to make the closed count higher. There may have been some commits that happened faster than normal, but mostly that was because there were multiple people to review right there in one place. We may have lost the benefit of some "think time" letting an issue sit might have had, but I'm willing to accept that tradeoff. I expect many of the other issues we worked on to get closed, but to me the significant part of that is the individual contributors seeing their issues get closed, not the absolute count. I may generate a report after I've gone over the whole list of bugs looking for things to commit, but that will take a while. Anyone who wants to help, the list is here: http://bit.ly/bitesized-python-tickets
I think trying to ensure we actually *thank* people goes a long way towards achieving the same goal, but without the bias.
Good metrics are actually a useful way to know who to thank, though.
I'd like us to have good metrics, that was one of my sub-goals in the proposal. But as others have pointed out, we need to be careful what we measure, because what (and how) we measure will affect what outcomes we get. Speaking of which, I'll see if I can go over all of the discussion and post a summary this weekend. But it feels like we have a reasonable degree of agreement. --David
On Wed, Apr 23, 2014 at 2:51 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Tue, 22 Apr 2014 07:06:53 +0300 Ezio Melotti <ezio.melotti@gmail.com> wrote:
I'll also suggest another related (and "controversial") idea. People like to reach goals: if they address the 3 issues in their queue they have reached the "empty queue" goal. Addressing 3 of the 5 issues isn't quite the same thing. I've seen this concept being exploited in three main ways: 1) badges/trophies/achievements; 2) competitions; 3) streaks;
[...]
While I understand that probably most of the core devs would be against similar things, this might motivate new users and make them "addicted" to the tracker, while making their experience more enjoyable, and the example I linked show that similar things exist even in these environments (and not only on the micro-transaction based smartphone games :). People who don't care about this (different people are more or less competitive) could just ignore it. OTOH this might have a negative side-effect if users start closing issues randomly just to get the "100 closed issues" badge, but this is not difficult to avoid.
Not difficult how? In any gamification system, people will work towards getting new rewards / awards, not towards making meaningful contributions.
By having badges/trophies that are not easily abused. For example having "Submit 10 patches in one day" is unrealistic and will likely push whoever wants to get this to either hold off his patches and submit them at once or to produce 10 poor-quality patches. Having "Triage 100 issues" is less prone to abusing IMHO, because even if someone is doing some extra work to try to reach the goal, I don't think this will affect the quality of the work.
I think something like the Twisted high scores is acceptable (since it's quite un-serious), but starting displaying awards will really bias how people contribute (with a definite emphasis on quantity over quality, IMO).
Note that I'm not suggesting this to be a pervasive thing. On the tracker, badges should be relegated to a specific page and won't be visible elsewhere. Users will be able to check their badges and the badges of other devs and there might be a leaderboard page showing who has the most badges or something similar. We can also include a section in the weekly report that says something like "Congratulations to: John Smith for contributed his 100th patch, Jane Smith who triaged 50 issues, etc.", but that's pretty much it. If you want to make it less game-like, we could turn this into a more "serious" user stats page (and a global stats for the "leaderboard" page) showing user activity, number of issues they opened/closed/triaged/commented on/assigned to, patches contributed, patches accepted. Adding graphs and deltas with the previous week/month/year, or with the average/best contributor might add motivation and replace to some extent the need for badges, but it's also less appealing (at least for some people). Ohloh has similar stats/graphs (that we could maybe embed).
(it's the same reason I'm rather ambiguous on the whole idea of sprints)
I think trying to ensure we actually *thank* people goes a long way towards achieving the same goal, but without the bias.
The two are not mutually exclusive :)
Regards
Antoine.
On mer., 2014-04-23 at 15:27 +0300, Ezio Melotti wrote:
By having badges/trophies that are not easily abused. For example having "Submit 10 patches in one day" is unrealistic and will likely push whoever wants to get this to either hold off his patches and submit them at once or to produce 10 poor-quality patches. Having "Triage 100 issues" is less prone to abusing IMHO, because even if someone is doing some extra work to try to reach the goal, I don't think this will affect the quality of the work.
Why don't you think so? We actually all know how easy it is to be extremely dumb at triaging, since there has been a well-known example on the tracker (I won't mention his name out of charity, but he thought he was being very productive). You should read this before you think awards (badges, trophies...) don't have any negative effects: http://www.joelonsoftware.com/articles/fog0000000070.html (well, of course, the author, Joel Spolsky, is also the co-founder of Stack Overflow... the other co-founder shortly explains his take on gamification here: http://blog.codinghorror.com/the-gamification/ ; as that article shows, he was solving a different problem than ours, though: namely, that web forums usually have a poor S/N ratio) Regards Antoine.
On Wed, Apr 23, 2014 at 3:42 PM, Antoine Pitrou <antoine@python.org> wrote:
On mer., 2014-04-23 at 15:27 +0300, Ezio Melotti wrote:
By having badges/trophies that are not easily abused. For example having "Submit 10 patches in one day" is unrealistic and will likely push whoever wants to get this to either hold off his patches and submit them at once or to produce 10 poor-quality patches. Having "Triage 100 issues" is less prone to abusing IMHO, because even if someone is doing some extra work to try to reach the goal, I don't think this will affect the quality of the work.
Why don't you think so? We actually all know how easy it is to be extremely dumb at triaging, since there has been a well-known example on the tracker (I won't mention his name out of charity, but he thought he was being very productive).
There might be exceptions, but I think most of the contributors (especially the ones promoted to triagers) would be reasonable and won't "cheat" just to obtain a badge. If they do, we can just tell them and/or remove their triager privileges (or award them the "bad triager" badge :).
You should read this before you think awards (badges, trophies...) don't have any negative effects:
I am myself somewhat skeptical that badges are a good way to appeal to more contributors (see also my link about overjustification), but, as Nick says, if we decide to do it and we do it right, it might work. Asking new contributors if they think this would motivate them might also give us an idea.
http://www.joelonsoftware.com/articles/fog0000000070.html
(well, of course, the author, Joel Spolsky, is also the co-founder of Stack Overflow... the other co-founder shortly explains his take on gamification here: http://blog.codinghorror.com/the-gamification/ ; as that article shows, he was solving a different problem than ours, though: namely, that web forums usually have a poor S/N ratio)
Thanks for the links! The goal is indeed somewhat different, namely appealing to new (and possibly younger) developers and having them sticking around. I know some core-devs that already have a "self-imposed" goal of fixing at least N bugs per week -- badges would propose a similar challenge to everyone and provide an easy way to track it and a "reward" for completing it successfully.
Regards
Antoine.
On Mon, Apr 21, 2014 at 7:04 PM, R. David Murray <rdmurray@bitdance.com> wrote:
Priority ~~~~~~~~
low normal high deferred blocker release blocker
The only change here is to eliminate 'critical'. I'm not wedded to that, but if we have both 'critical' and 'high' priorities, 'high' ends up getting treated as pretty much equivalent to either 'normal' or 'critical', depending on the person. I would argue that anything that is severe enough to be marked 'critical' should in fact be a release blocker, and anything that is not a release blocker is effectively only 'high' priority.
The priority is currently operational only in that one can sort issues by priority. I propose that we make them much more operational, by posting a count and/or list of the bugs with more than normal priority somewhere public on a regular basis, such as python-dev, python-committers, and/or #python-dev. (Well, definitely #python-dev.)
The Django project uses a development dashboard to display numbers like "release blockers", "patches needing review", etc: https://dashboard.djangoproject.com/ It would be good if there is a dashboard like that for CPython (by using Roundup's XML-RPC API: http://roundup.sourceforge.net/docs/xmlrpc.html#client-api ) --Berker
The Django project uses a development dashboard to display numbers like "release blockers", "patches needing review", etc: https://dashboard.djangoproject.com/ It would be good if there is a dashboard like that for CPython (by using Roundup's XML-RPC API: http://roundup.sourceforge.net/docs/xmlrpc.html#client-api )
Nice Dashboard: +1 for this :-) BTW. from a Non-Dev point of view: I just would like to be able to know how far or ready is a patch for manually review and then commit. The precondition for the review should IMHO be that it passes all tests, pep8, bot-integration-tests, and that the patch still applies without rework on the 2.7, 3.X tip(s)... (some of those step are already doable now, but checks automation would be nice to have). Regards, francis
On Thu, 24 Apr 2014 14:16:48 +0200, francis <francismb@email.de> wrote:
The Django project uses a development dashboard to display numbers like "release blockers", "patches needing review", etc: https://dashboard.djangoproject.com/ It would be good if there is a dashboard like that for CPython (by using Roundup's XML-RPC API: http://roundup.sourceforge.net/docs/xmlrpc.html#client-api )
Nice Dashboard:
+1 for this :-)
We should definitely experiment with some formats for dashboards. Maybe we can have more than one :)
BTW. from a Non-Dev point of view:
I just would like to be able to know how far or ready is a patch for manually review and then commit.
Yes, that's part of the goal of my making the state of an issue more fine grained.
The precondition for the review should IMHO be that it passes all tests, pep8, bot-integration-tests, and that the patch still applies without rework on the 2.7, 3.X tip(s)... (some of those step are already doable now, but checks automation would be nice to have).
Still applies is an interesting one...currently rietveld applies against the parent revision if it knows it, and default if it doesn't, so we'd need to add some tooling to check it periodically (or on request) against the tip of the branches it is supposed to be applied to (2.7 and current maintenance for bugs, default for enhancements) and report the results. For tests, the patch has to be approved for testing by a committer even with a patch gating system, and yes, we will want additional feedback into the tracker about the results of the tests runs. The interfaces are all there, I think, to figure that out, but we'll need a bunch of code to tie it all together. --David
BTW. from a Non-Dev point of view:
I just would like to be able to know how far or ready is a patch for manually review and then commit.
Yes, that's part of the goal of my making the state of an issue more fine grained.
IMHO it would also help to define what kind of work is "external" or "internal" (from the core-dev-commiter point view). Then that external steps should be, as far as possible, done (e.g. "ready for review", "ready for commit") before the core-dev looks at it (she/he should just do "internals": review, commit or reject :-)).
[...] The interfaces are all there, I think, to figure that out, but we'll need a bunch of code to tie it all together.
and python is perfect for that ;-) Regards, francis
On Thu, 24 Apr 2014 23:06:41 +0200, francis <francismb@email.de> wrote:
BTW. from a Non-Dev point of view:
I just would like to be able to know how far or ready is a patch for manually review and then commit.
Yes, that's part of the goal of my making the state of an issue more fine grained.
IMHO it would also help to define what kind of work is "external" or "internal" (from the core-dev-commiter point view). Then that external steps should be, as far as possible, done (e.g. "ready for review", "ready for commit") before the core-dev looks at it (she/he should just do "internals": review, commit or reject :-)).
Yep, if you read it, that's what my stage structure is designed to do. There are only two gating points: will we fix this, and is this committable. Everything else non-committers can decide (though core will no doubt chime in as we do now), and even the first of those can be triage[*]. --David [*] Or, as Antoine has advocated, anyone...though I worry that that will leave people feeling like they did work for nothing if a veto on commit comes from core later.
I'm a week later than I expected, but I've added my notes to the discussion summary started by Ezio at: https://wiki.python.org/moin/TrackerDevelopmentPlanning Based on the feedback, I propose two major changes compared to my initial proposal. The first is to combine 'keywords' and 'component' into a single tags field, which will include all the labels from the experts index. Types is reduced to *just* documentation/bug/enhancement, everything else is a tag. Likewise priority is reduced to just high/normal/low, and release-blocker and deferred-blocker become per-release tags. I think the tags should at least initially be a fixed set (that is, normal users can't add tags). The second major change is that I think we should defer the decision about how to manage patches until we can experiment with some possibilities. In particular, Ezio has some preliminary code to do content analysis on patches, and I think we should experiment with incorporating that, and try out a couple of different ways of managing patches once we have that automated analysis working. Ezio made some notes about this on the document. Although this wasn't discussed directly, I also removed 'committer decision needed' from the list of states, leaving just 'decision needed'. The various 'closed/xxx' items can at least for now (and probably forever) continue to be a 'closed' state plus the (simplified) 'resolution'. Given this, it would be quite practical to simply change 'stage' list from its current value to what I propose for the new state. That means it would look like this: - no selection - (that is: new) information needed decision needed patch needed review needed patch update needed commit review needed resolved We can then add reactors (and javascript) such that changing an issue stage to 'resolved' also closes it, and that changing an issue status to 'closed' resolves it, and vice versa (re-opening an issue goes to, say, 'patch needed' if no specific stage is set; changing the stage reopens the issue). This will make it easy to implement the new state scheme and build the UI for the queues &c, and when we are ready to cut over to the new UI, we can retire 'status'. If this is deemed acceptable, then I would propose that we (a) go ahead and make that change, (b) make the change to the 'assigned to' field, (which we are currently not using much at all), (c) add code to the existing UI to allow regular users to make the state (stage) transitions as outlined in my proposal, and (d) add the dashboard UI for getting optimal access to the resulting queues. These changes are really the heart of my proposal, and would get us a lot of the benefit, even before we optimize the UI. --David
Hello David, I would like to offer my thoughts as a new Python contributor. I may be new as a Python contributor but I've been involved in programming, managing, and teaching a variety of computer languages and operating systems since the mid-1970s. As I have not met many of you personally, I would like to thank each of you for your contributions to Python. It's truly my pleasure to use Python. I have a couple of brief comments on Tracker planning from the perspective of a new contributor (though someone with an understanding of software development processes). Three different pieces of information are most valuable to me as a brand new contributor (i.e. someone without any special privileges to change status or fields in an issue): 1. Current state 2. Next action step 3. Next action step claimed by:___________ and on what date_________ Ideally, these three pieces of information will answer the new person's questions: 1. What is this issue's current status? 2. What next step could I take to move this issue along to the next desired state? 3. Is the next step something that is free to work on (unclaimed) or stalled? What can I do to claim the next action step? For example, an issue might have a current state of "Reproduced issue successfully" and the next action step might be "Update documentation" or "Create patch". Here's a write up of how I found the contribution process and how I would explain the process to others based on my experience: http://pastebin.com/iKhL5RvS I'm not asking you to change or modify the process as it stands, but merely to consider the process from the perspective of someone who would like to contribute but is new to Python's specific workflow. Thanks, Carol Willing -- Carol Willing Developer Willing Consulting
On 6 May 2014 08:42, "Carol Willing" <willingc@willingconsulting.com> wrote:
Here's a write up of how I found the contribution process and how I would explain the process to others based on my experience: http://pastebin.com/iKhL5RvS
Thanks Carol, that's a great write-up and definitely includes some good ideas for improvement. For folks with greater Roundup-fu than me: how hard does the "personal tags" idea sound? Alternatively, the front end could be more like a personal set of "issue lists". Cheers, Nick.
I'm not asking you to change or modify the process as it stands, but
merely to consider the process from the perspective of someone who would like to contribute but is new to Python's specific workflow.
Thanks, Carol Willing
-- Carol Willing Developer Willing Consulting
_______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct:
On Tue, May 6, 2014 at 1:56 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 6 May 2014 08:42, "Carol Willing" <willingc@willingconsulting.com> wrote:
Here's a write up of how I found the contribution process and how I would explain the process to others based on my experience: http://pastebin.com/iKhL5RvS
Thanks Carol, that's a great write-up and definitely includes some good ideas for improvement.
For folks with greater Roundup-fu than me: how hard does the "personal tags" idea sound? Alternatively, the front end could be more like a personal set of "issue lists".
I haven't read Carol's link yet, but if with "personal tags" you mean tags hat a single user can add for himself to an issue and that are not visible to others, then I think it might be somewhat difficult. You could store them in a user field (e.g. user.personal_tags = {issue123: [tag1, tag2], issue124: [tag3]}) but then it might not be too easy to access them (e.g. while searching). Same goes if we keep them in a separate table. In addition, we also have to write the code to handle addition/deletion of both tags in a personal list, and on the issue. A possible compromise might be to add a way to "star" interesting issues. I think there are 3 "levels", and we currently support only two: 1) If I know I'm going to fix an issue, I assign it to myself; 2) if I know how to fix an issue, or I would like to work on it, I have to mark it on my mail client; 3) if I left a comment or a review or even if an issue is "interesting", I add myself to nosy; If you were referring to "a way to easily tag issues to return to as promising possibilities", then a star/favorite feature might cover case 2) and solve the issue, and even if it's not as fine grained as a full personal tags system, it should be simpler to implement.
Cheers, Nick.
I'm not asking you to change or modify the process as it stands, but merely to consider the process from the perspective of someone who would like to contribute but is new to Python's specific workflow.
Thanks, Carol Willing
-- Carol Willing Developer Willing Consulting
_______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
_______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
Ezio and Nick, Thanks for the feedback on the tracker. A couple of more thoughts on "personal tags" which hopefully will clarify and save work and time. FWIW, I believe the data to determine easily the state of an issue, the next action required on the issue, and if the issue is free to work on are most important to a new contributor. A simple "star/favorite" issue toggle was really the extent of what I was thinking would have been helpful.
For folks with greater Roundup-fu than me: how hard does the "personal tags" idea sound? Alternatively, the front end could be more like a personal set of "issue lists". "a way to easily tag issues to return to as promising possibilities", then a star/favorite feature might cover case 2) and solve the issue, and even if it's not as fine grained as a full personal tags system, it should be simpler to implement. Ezio's links to his experiments with alternate displays of information reminded me of something that I thought about when searching for a suitable new contributor issue.
http://wolfprojects.altervista.org/patches.html (requires js) When I was looking for an issue to work on, I tried a number of search criteria (easy, new, documentation, testing). It's possible that my "search"-fu is lacking, yet I had a difficult time identifying a "bite sized" issue related to documentation, testing, or development in either C or Python that didn't appear to be actively worked on or very old. I suspect there were a number of suitable issues yet it felt like searching for a needle in a very large haystack. At the time I wondered if it just wouldn't be easier to grab the entire "csv" file of issues and use ipython notebook with numpy/pandas to run interactive searches for issues. Instead, I wound up just randomly reading issues from the tracker which was why a "star/tag" would have been helpful. In hindsight, I suspect if I used the entire csv file with appropriate search criteria in numpy/pandas that I would have greatly simplified my search for an issue. I may do this now so I can see if I can find search criteria that would be useful for new contributors to identify suitable issues for: * documentation * increasing test coverage[pulling from the coverage site] * triaging[identifying specific tasks that someone without developer status can do to add value]). Thanks, Carol -- Carol Willing Developer Willing Consulting
On Tue, May 6, 2014 at 5:33 PM, Carol Willing <willingc@willingconsulting.com> wrote:
Ezio and Nick,
Thanks for the feedback on the tracker. A couple of more thoughts on "personal tags" which hopefully will clarify and save work and time. FWIW, I believe the data to determine easily the state of an issue, the next action required on the issue, and if the issue is free to work on are most important to a new contributor.
A simple "star/favorite" issue toggle was really the extent of what I was thinking would have been helpful.
For me it would be OK to add it, since I needed it myself more than once.
For folks with greater Roundup-fu than me: how hard does the "personal tags" idea sound? Alternatively, the front end could be more like a personal set of "issue lists".
"a way to easily tag issues to return to as promising possibilities", then a star/favorite feature might cover case 2) and solve the issue, and even if it's not as fine grained as a full personal tags system, it should be simpler to implement.
Ezio's links to his experiments with alternate displays of information reminded me of something that I thought about when searching for a suitable new contributor issue.
http://wolfprojects.altervista.org/patches.html (requires js)
When I was looking for an issue to work on, I tried a number of search criteria (easy, new, documentation, testing). It's possible that my "search"-fu is lacking, yet I had a difficult time identifying a "bite sized" issue related to documentation, testing, or development in either C or Python that didn't appear to be actively worked on or very old. I suspect there were a number of suitable issues yet it felt like searching for a needle in a very large haystack.
The haystack is very large (we are approaching 5000 open issues). hopefully tags (including the "bite sized" one proposed by David) will make search easier. My experiment, if properly implemented, will make search faster (it's possible to search for individual columns too if we want). This will allow to refine searches until you find something interesting.
At the time I wondered if it just wouldn't be easier to grab the entire "csv" file of issues and use ipython notebook with numpy/pandas to run interactive searches for issues. Instead, I wound up just randomly reading issues from the tracker which was why a "star/tag" would have been helpful.
The short answer is probably no. The main problem is that the csv only includes ids for the different fields. So instead of having "open", "closed", "pending" in the status column you have e.g. "1", "2", "3". You can find the corresponding value or each id, but then you have to replace them and it starts taking more time. The second link I posted (http://wolfprojects.altervista.org/issue.index.html) is generated from the csv (as it was just a proof of concept I put together in 5 minutes), so it only has the ids and it's not updated with the latest issues. The csv might also not contain all the information you need, even though you can decide which columns should be included by fiddling with the URL.
In hindsight, I suspect if I used the entire csv file with appropriate search criteria in numpy/pandas that I would have greatly simplified my search for an issue. I may do this now so I can see if I can find search criteria that would be useful for new contributors to identify suitable issues for:
documentation increasing test coverage[pulling from the coverage site] triaging[identifying specific tasks that someone without developer status can do to add value]).
If you still want to try, you should be able to see the corresponding values for each id at the following URLs: http://bugs.python.org/component ("documentation" is 4) http://bugs.python.org/stage ("need test" is 2, "need patch" is 3, "no value" should be -1)
Thanks, Carol
-- Carol Willing Developer Willing Consulting
Hi, On Mon, May 5, 2014 at 6:24 PM, R. David Murray <rdmurray@bitdance.com> wrote:
I'm a week later than I expected, but I've added my notes to the discussion summary started by Ezio at:
https://wiki.python.org/moin/TrackerDevelopmentPlanning
Based on the feedback, I propose two major changes compared to my initial proposal.
The first is to combine 'keywords' and 'component' into a single tags field, which will include all the labels from the experts index.
A few questions: 1) should we merge the keywords into components, do the opposite, or create a brand new "tags" field? 2) how are we going to migrate metadata from the old format to the new one? 3) with "all the labels from the experts index" you mean every single entry? There are quite a lot of entries (all the modules, platforms, interest areas: https://docs.python.org/devguide/experts.html ); 4) how should this look without js? similar to the nosy list (input box, without autocomplete), or select/option like the one we currently have for components/keywords (entries can be grouped, but they are still a lot); And a few comments: 1) tags/labels should be color-coded so that each group has a different color (module, platform, component, general, release, etc..), if they are not too many they could be showed in the issue list too (think about gmail labels); 2) the autonosy should be revised once we do this. One idea is to suggest people to add (Something along the lines of 'You selected the module tag "html", suggested dev: [ezio.melotti]'). The other idea is to handle this automatically -- on the expert index with have a * to allow assignment, we might do something for nosy too and use this to handle autonosy/autoassign. Otherwise we can just edit autonosy/autoassign like we currently do (manually from roundup), but with more available tags;
Types is reduced to *just* documentation/bug/enhancement, everything else is a tag. Likewise priority is reduced to just high/normal/low, and release-blocker and deferred-blocker become per-release tags.
I think the tags should at least initially be a fixed set (that is, normal users can't add tags).
+1 for fixed set. Free tags might lead to redundant or misspelled tags (and possibly things like #ibrokepython #lolsegfault #yolo). We could add tags if we think they are useful, but if we want to add free tags they should be in a separate category with their own color code and an easy way to moderate them.
The second major change is that I think we should defer the decision about how to manage patches until we can experiment with some possibilities. In particular, Ezio has some preliminary code to do content analysis on patches, and I think we should experiment with incorporating that, and try out a couple of different ways of managing patches once we have that automated analysis working. Ezio made some notes about this on the document.
My latest experiment is: http://wolfprojects.altervista.org/patches.html (requires js). This includes a few ideas: 1) detecting if the patch has tests/docs/news/acks (see last column on the right); 2) detecting the files it affects (second last column on the right; could be used for module auto-tagging); 3) having a simple way to filter/inspect patches in real time with js; 1 and 2 should be integrated on the issue page, 3 could be done on a separate page. Since 3 looked like an interesting idea (it makes search a lot easier -- you can just filter/group for user name, files, issue, etc on the fly), I also put together a proof of concept for a similar interface that could replace the main issue list (and/or be used in the dashboard): http://wolfprojects.altervista.org/issue.index.html To implement this we just need a fixed page and feed issues to it via ajax/json. With a bit of caching we can make the whole thing very efficient (no need to build the page server-side every time, less data moving on the network, real-time client-side filtering/grouping/searching, no need to go back and forth from the search page to refine queries) so the idea has lot of potential. While this is not too difficult to implement, it still requires some work, so for now it's low priority.
Although this wasn't discussed directly, I also removed 'committer decision needed' from the list of states, leaving just 'decision needed'.
Are there limitations on who can change the stage from "decision needed" to something else? ISTM that if a user provides information he can move the stage to "decision needed" or if he provides a patch he can move the stage to "review needed", but do we want them to add their opinion and move it from "decision needed" to something else? Also is it expected that we move back to previous stages (e.g. going back to "decision needed" after a few different patches have been proposed) or you were thinking about a more linear evolution?
The various 'closed/xxx' items can at least for now (and probably forever) continue to be a 'closed' state plus the (simplified) 'resolution'. Given this, it would be quite practical to simply change 'stage' list from its current value to what I propose for the new state. That means it would look like this:
- no selection - (that is: new) information needed decision needed patch needed review needed patch update needed commit review needed resolved
We can then add reactors (and javascript) such that changing an issue stage to 'resolved' also closes it, and that changing an issue status to 'closed' resolves it, and vice versa (re-opening an issue goes to, say, 'patch needed' if no specific stage is set; changing the stage reopens the issue). This will make it easy to implement the new state scheme and build the UI for the queues &c, and when we are ready to cut over to the new UI, we can retire 'status'.
If this is deemed acceptable, then I would propose that we (a) go ahead and make that change,
that == the list of stages you suggested?
(b) make the change to the 'assigned to' field, (which we are currently not using much at all),
So make it possible to assign the issue to everyone and then start using it like this?
(c) add code to the existing UI to allow regular users to make the state (stage) transitions as outlined in my proposal,
For this we need to change the schema too, since currently users are not allowed to change the stage. Regarding the UI changes, what exactly do you want to change?
and (d) add the dashboard UI for getting optimal access to the resulting queues.
Adding a new page for this can be done quite easily, getting the queries right might be a bit more complicated (I think we shouldn't use sql directly, and not all the values are easily accessible from the Roundup API).
These changes are really the heart of my proposal, and would get us a lot of the benefit, even before we optimize the UI.
It would be good to set up an online test tracker where we can push/test changes live, get some feedback, and then apply the change that works best to the real tracker. This would make things much easier, so I would give higher priority to this. Having some tentative mockups for the new interface would also help get a better idea, especially if we can get some designer to help us out (that could be done later once we decided the fields we want though).
--David
On Mon May 05 2014 at 11:54:18 AM, R. David Murray <rdmurray@bitdance.com> wrote:
I'm a week later than I expected, but I've added my notes to the discussion summary started by Ezio at:
https://wiki.python.org/moin/TrackerDevelopmentPlanning
Based on the feedback, I propose two major changes compared to my initial proposal.
The first is to combine 'keywords' and 'component' into a single tags field, which will include all the labels from the experts index. Types is reduced to *just* documentation/bug/enhancement, everything else is a tag. Likewise priority is reduced to just high/normal/low, and release-blocker and deferred-blocker become per-release tags.
I think the tags should at least initially be a fixed set (that is, normal users can't add tags).
SGTM. I know Ezio pointed out the list will be long, but since triagers will be the ones typically setting the field it shouldn't be an issue.
The second major change is that I think we should defer the decision about how to manage patches until we can experiment with some possibilities. In particular, Ezio has some preliminary code to do content analysis on patches, and I think we should experiment with incorporating that, and try out a couple of different ways of managing patches once we have that automated analysis working. Ezio made some notes about this on the document.
Although this wasn't discussed directly, I also removed 'committer decision needed' from the list of states, leaving just 'decision needed'. The various 'closed/xxx' items can at least for now (and probably forever) continue to be a 'closed' state plus the (simplified) 'resolution'. Given this, it would be quite practical to simply change 'stage' list from its current value to what I propose for the new state. That means it would look like this:
- no selection - (that is: new) information needed decision needed patch needed review needed patch update needed commit review needed resolved
We can then add reactors (and javascript) such that changing an issue stage to 'resolved' also closes it, and that changing an issue status to 'closed' resolves it, and vice versa (re-opening an issue goes to, say, 'patch needed' if no specific stage is set; changing the stage reopens the issue). This will make it easy to implement the new state scheme and build the UI for the queues &c, and when we are ready to cut over to the new UI, we can retire 'status'.
Also SGTM. -Brett
If this is deemed acceptable, then I would propose that we (a) go ahead and make that change, (b) make the change to the 'assigned to' field, (which we are currently not using much at all), (c) add code to the existing UI to allow regular users to make the state (stage) transitions as outlined in my proposal, and (d) add the dashboard UI for getting optimal access to the resulting queues. These changes are really the heart of my proposal, and would get us a lot of the benefit, even before we optimize the UI.
--David _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
I'm wondering if "decision needed" might be more accurately named "triage needed"? Looking at David's well documented proposals and other mail comments, "triage needed" more specifically describes the 'state'. A few thoughts: 1. "Triage needed" would raise the importance and visibility of the triage contributor role. A positive for onboarding and growing development talent. 2. "Triage needed" is more descriptive and clearer than "decision needed" especially for those users that do not read documentation or understand the development workflow. "Decision needed" implies that a decision will be made to include or not include in a release. Realistically, decisions are made throughout the remainder of the development process based on time, resources, etc. Thanks, Carol On 5/7/14, 6:39 AM, Brett Cannon wrote:
On Mon May 05 2014 at 11:54:18 AM, R. David Murray <rdmurray@bitdance.com <mailto:rdmurray@bitdance.com>> wrote:
Given this, it would be quite practical to simply change 'stage' list from its current value to what I propose for the new state. That means it would look like this:
- no selection - (that is: new) information needed decision needed patch needed review needed patch update needed commit review needed resolved
-- Carol Willing Developer Willing Consulting
On Wed, 07 May 2014 08:09:03 -0700, Carol Willing <willingc@willingconsulting.com> wrote:
I'm wondering if "decision needed" might be more accurately named "triage needed"?
Looking at David's well documented proposals and other mail comments, "triage needed" more specifically describes the 'state'.
A few thoughts:
1. "Triage needed" would raise the importance and visibility of the triage contributor role. A positive for onboarding and growing development talent.
2. "Triage needed" is more descriptive and clearer than "decision needed" especially for those users that do not read documentation or understand the development workflow. "Decision needed" implies that a decision will be made to include or not include in a release. Realistically, decisions are made throughout the remainder of the development process based on time, resources, etc.
I'll be interested in what others think, but to me "decision needed" is closer than "triage needed". That is, the state means that someone other than the person moving the issue to that state needs to make a decision. That decision can be "Is this something we consider a bug? What releases can we fix this in given our backward compatibility requirements? Is this an acceptable enhancement? And any other decision that needs to be made before the issue can move forward. All of these *can* be "triage" decisions, but to my ear it is the word "triage" that is more about deciding where to allocate resources ("which release"), whereas we generally don't work that way. We just decide if it can go in or not, and if the patch is ready before the next release, it can go in. More specifically, because I removed 'committer decision needed', 'decision needed' covers the case of needing a committer decision, which is by definition not a triage decision :) Perhaps 'committer decision needed' should be kept after all? --David
On 8 May 2014 01:59, "R. David Murray" <rdmurray@bitdance.com> wrote:
On Wed, 07 May 2014 08:09:03 -0700, Carol Willing <
willingc@willingconsulting.com> wrote:
I'm wondering if "decision needed" might be more accurately named "triage needed"?
Looking at David's well documented proposals and other mail comments, "triage needed" more specifically describes the 'state'.
A few thoughts:
1. "Triage needed" would raise the importance and visibility of the triage contributor role. A positive for onboarding and growing development talent.
2. "Triage needed" is more descriptive and clearer than "decision needed" especially for those users that do not read documentation or understand the development workflow. "Decision needed" implies that a decision will be made to include or not include in a release. Realistically, decisions are made throughout the remainder of the development process based on time, resources, etc.
I'll be interested in what others think, but to me "decision needed" is closer than "triage needed". That is, the state means that someone other than the person moving the issue to that state needs to make a decision. That decision can be "Is this something we consider a bug? What releases can we fix this in given our backward compatibility requirements? Is this an acceptable enhancement? And any other decision that needs to be made before the issue can move forward.
All of these *can* be "triage" decisions, but to my ear it is the word "triage" that is more about deciding where to allocate resources ("which release"), whereas we generally don't work that way. We just decide if it can go in or not, and if the patch is ready before the next release, it can go in.
More specifically, because I removed 'committer decision needed', 'decision needed' covers the case of needing a committer decision, which is by definition not a triage decision :)
Perhaps 'committer decision needed' should be kept after all?
From a work queue perspective, two separate states likely makes sense, since "Triage needed" & "Committer decision needed" are aimed at slightly different groups of people (with the latter being a subset of the former). That way, "Committer decision needed" becomes a clear avenue for escalation by triagers to the core developers when they need a design decision or risk assessment on a particular approach.
That more structured mechanism should nicely complement the option of punting decisions to the collective wisdom (hah!) of python-dev & python-ideas. Cheers, Nick.
--David _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct:
On Thu, May 8, 2014 at 1:58 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 8 May 2014 01:59, "R. David Murray" <rdmurray@bitdance.com> wrote:
On Wed, 07 May 2014 08:09:03 -0700, Carol Willing <willingc@willingconsulting.com> wrote:
I'm wondering if "decision needed" might be more accurately named "triage needed"?
Looking at David's well documented proposals and other mail comments, "triage needed" more specifically describes the 'state'.
A few thoughts:
1. "Triage needed" would raise the importance and visibility of the triage contributor role. A positive for onboarding and growing development talent.
2. "Triage needed" is more descriptive and clearer than "decision needed" especially for those users that do not read documentation or understand the development workflow. "Decision needed" implies that a decision will be made to include or not include in a release. Realistically, decisions are made throughout the remainder of the development process based on time, resources, etc.
I'll be interested in what others think, but to me "decision needed" is closer than "triage needed". That is, the state means that someone other than the person moving the issue to that state needs to make a decision. That decision can be "Is this something we consider a bug? What releases can we fix this in given our backward compatibility requirements? Is this an acceptable enhancement? And any other decision that needs to be made before the issue can move forward.
I agree. To me "triaging" mostly means updating fields/metadata and it's something that is done once the issue is reported. This also includes adding people to the nosy list so that they can comment and start dealing with the issues, but I don't consider this "triaging" anymore. IOW "triage needed" would correspond to "- no selection -". OTOH, "decision needed" means that the people working on the issue need to reach an agreement. A good example of this is http://bugs.python.org/issue18967. Here no triaging is needed IMHO.
All of these *can* be "triage" decisions, but to my ear it is the word "triage" that is more about deciding where to allocate resources ("which release"), whereas we generally don't work that way. We just decide if it can go in or not, and if the patch is ready before the next release, it can go in.
More specifically, because I removed 'committer decision needed', 'decision needed' covers the case of needing a committer decision, which is by definition not a triage decision :)
Perhaps 'committer decision needed' should be kept after all?
I don't think the distinction is useful. Anyone can contribute to the discussion, as long as they don't just give their opinion and change the stage directly. For example in http://bugs.python.org/issue18967 a Mercurial dev could give his opinion, and if the others agree, the stage can be updated (to "needs patch" or "needs review" if a patch is already present).
From a work queue perspective, two separate states likely makes sense, since "Triage needed" & "Committer decision needed" are aimed at slightly different groups of people (with the latter being a subset of the former). That way, "Committer decision needed" becomes a clear avenue for escalation by triagers to the core developers when they need a design decision or risk assessment on a particular approach.
I would rather keep the list of stages short, i.e.: - no selection - information needed decision needed patch needed review needed commit (review) needed resolved with the following meanings - no selection -: issue hasn't been triaged/looked at yet information needed: something needs to be confirmed/clarified before proceeding decision needed: an agreement should be reached before taking action patch needed: we know what to do, we need the code review needed: we have the code, we need someone to review it commit (review) needed: we have the code, we need a committer to commit it resolved: issue is now closed These would be useful to (triagers also includes committers): - no selection -: triagers (searching for untriaged issues) information needed: original poster (probably not useful in searches) decision needed: committers (and possibly triagers) patch needed: everyone (people searching for issues that need patches) review needed: triagers (searching for issues that need a review) commit (review) needed: committers (searching for issues that are ready to go in) resolved: everyone (people marveling at the outstanding work of our team) Also think how these stages would affect the dashboards (e.g. "information needed" should be prominent on the original poster's dashboard). To illustrate the possible evolution of an issue I made a couple of flow charts: http://imgur.com/a/UgJBJ The first is without "patch update needed" the second includes it. Also note that is possible to go from basically every state to "resolved", however I omitted those arrows, since they would just clutter the flow chart. I'm not sure if this should be enforced (probably not), but at least it should provided a clearer picture. I left "decision needed", removed "patch update needed", and possibly removed the "review" from "commit review needed": 1) "decision needed" means that the problem is clear, but we have to discuss about the best solution. To me, "triage needed" mostly means that the fields/metadata are not updated; 2) "patch update needed" seems redundant to me, and can be replaced with "patch needed" + "assigned to <previous patch author>". I'm not strongly opposed about removing it though; 3) "commit review needed" seems useful to signal to core devs that someone reviewed the patch and it is now ready to be committed. This assumes that the committer will do a further review, but if we already passed the "review needed" stage, there are good chances that the patch is ready to go in (if not we can always go back to "patch needed"). This can also be simply called "commit needed";
That more structured mechanism should nicely complement the option of punting decisions to the collective wisdom (hah!) of python-dev & python-ideas.
Cheers, Nick.
--David
On Thu May 08 2014 at 1:44:21 AM, Ezio Melotti <ezio.melotti@gmail.com> wrote:
On Thu, May 8, 2014 at 1:58 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 8 May 2014 01:59, "R. David Murray" <rdmurray@bitdance.com> wrote:
On Wed, 07 May 2014 08:09:03 -0700, Carol Willing <willingc@willingconsulting.com> wrote:
I'm wondering if "decision needed" might be more accurately named "triage needed"?
Looking at David's well documented proposals and other mail comments, "triage needed" more specifically describes the 'state'.
A few thoughts:
1. "Triage needed" would raise the importance and visibility of the triage contributor role. A positive for onboarding and growing development talent.
2. "Triage needed" is more descriptive and clearer than "decision needed" especially for those users that do not read documentation or understand the development workflow. "Decision needed" implies that a decision will be made to include or not include in a release. Realistically, decisions are made throughout the remainder of the development process based on time, resources, etc.
I'll be interested in what others think, but to me "decision needed" is closer than "triage needed". That is, the state means that someone
other
than the person moving the issue to that state needs to make a decision. That decision can be "Is this something we consider a bug? What releases can we fix this in given our backward compatibility requirements? Is this an acceptable enhancement? And any other decision that needs to be made before the issue can move forward.
I agree. To me "triaging" mostly means updating fields/metadata and it's something that is done once the issue is reported. This also includes adding people to the nosy list so that they can comment and start dealing with the issues, but I don't consider this "triaging" anymore. IOW "triage needed" would correspond to "- no selection -". OTOH, "decision needed" means that the people working on the issue need to reach an agreement. A good example of this is http://bugs.python.org/issue18967. Here no triaging is needed IMHO.
This is exactly the way I thought as well: the initial default state is "triage needed" and "decision needed" means someone higher up has to make a call on a question. Can we simply name the "no selection" state have the text "triage needed"? -Brett
All of these *can* be "triage" decisions, but to my ear it is the word "triage" that is more about deciding where to allocate resources ("which release"), whereas we generally don't work that way. We just decide if it can go in or not, and if the patch is ready before the next release, it can go in.
More specifically, because I removed 'committer decision needed', 'decision needed' covers the case of needing a committer decision, which is by definition not a triage decision :)
Perhaps 'committer decision needed' should be kept after all?
I don't think the distinction is useful. Anyone can contribute to the discussion, as long as they don't just give their opinion and change the stage directly. For example in http://bugs.python.org/issue18967 a Mercurial dev could give his opinion, and if the others agree, the stage can be updated (to "needs patch" or "needs review" if a patch is already present).
From a work queue perspective, two separate states likely makes sense, since "Triage needed" & "Committer decision needed" are aimed at slightly different groups of people (with the latter being a subset of the former). That way, "Committer decision needed" becomes a clear avenue for escalation by triagers to the core developers when they need a design decision or risk assessment on a particular approach.
I would rather keep the list of stages short, i.e.: - no selection - information needed decision needed patch needed review needed commit (review) needed resolved
with the following meanings - no selection -: issue hasn't been triaged/looked at yet information needed: something needs to be confirmed/clarified before proceeding decision needed: an agreement should be reached before taking action patch needed: we know what to do, we need the code review needed: we have the code, we need someone to review it commit (review) needed: we have the code, we need a committer to commit it resolved: issue is now closed
These would be useful to (triagers also includes committers): - no selection -: triagers (searching for untriaged issues) information needed: original poster (probably not useful in searches) decision needed: committers (and possibly triagers) patch needed: everyone (people searching for issues that need patches) review needed: triagers (searching for issues that need a review) commit (review) needed: committers (searching for issues that are ready to go in) resolved: everyone (people marveling at the outstanding work of our team)
Also think how these stages would affect the dashboards (e.g. "information needed" should be prominent on the original poster's dashboard).
To illustrate the possible evolution of an issue I made a couple of flow charts: http://imgur.com/a/UgJBJ
The first is without "patch update needed" the second includes it. Also note that is possible to go from basically every state to "resolved", however I omitted those arrows, since they would just clutter the flow chart. I'm not sure if this should be enforced (probably not), but at least it should provided a clearer picture.
I left "decision needed", removed "patch update needed", and possibly removed the "review" from "commit review needed": 1) "decision needed" means that the problem is clear, but we have to discuss about the best solution. To me, "triage needed" mostly means that the fields/metadata are not updated; 2) "patch update needed" seems redundant to me, and can be replaced with "patch needed" + "assigned to <previous patch author>". I'm not strongly opposed about removing it though; 3) "commit review needed" seems useful to signal to core devs that someone reviewed the patch and it is now ready to be committed. This assumes that the committer will do a further review, but if we already passed the "review needed" stage, there are good chances that the patch is ready to go in (if not we can always go back to "patch needed"). This can also be simply called "commit needed";
That more structured mechanism should nicely complement the option of punting decisions to the collective wisdom (hah!) of python-dev & python-ideas.
Cheers, Nick.
--David
core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
This is exactly the way I thought as well: the initial default state is "triage needed" IMHO, "triage" has somewhat different connotations to each of us. It's
and "decision needed" means someone higher up has to make a call on a question. My apologies in advance if my ability to communicate in writing my next
Can we simply name the "no selection" state have the text "triage needed"? Naming the "no selection" state has merit. Perhaps it auto defaults to
Ezio, Brett, Nick, and David, Your thoughts and additional information added some helpful insights and improvements. Some thoughts in-line below. Here's a link to a document that helps on-board people and demonstrates the many aspects a tracker communicates information to others in the community: https://wiki.openstack.org/wiki/How_To_Contribute [It illustrates fairly concretely that the tracker supports several different workflows and many different community members.] I'm going to go out on a limb and say that we're in agreement on a macro level. FWIW, a few thoughts below on 'state' naming. State ----- "Initial State" (aka No Selection, Triage needed) [I can't think of a great word for this state. What comes to mind is Harry Potter's Sorting Hat to determine where the issue should go] Information Please (aka Information needed) Decision Point (aka decision needed) Patch Development (aka Patch needed) Patch Review Commit Review Resolved probably best not to use it in the tracker as a state. This would allow documentation to treat "triage" as a process that may differ on the type of task or who is doing the task. My apologies for suggesting it as a state, but it has been interesting seeing the diversity in perspectives of what "triage" means. point is below par. Before I say what I am feeling, I want to be clear that I have strong respect for the core developers and committers based on their dedication to and passion for Python. I also have great respect for others that contribute in many different ways and the talents that they bring to a team. I can't quite put my finger on why, but my gut reaction to "decision needed" when coupled with words like "higher ups", "more experienced", "control", "hierarchy", "proven" is unwelcoming and a little patronizing. Oddly, simply stating "decision point" or "decision" as a state does not invoke the same reaction. I think the word "needed" subtly adds the implication that I am not trusted, capable enough, or proven my intelligence to make a decision, and I must wait to get permission from an "authority figure". Not a huge sticking point if consensus wants to keep it as "decision needed". I have actually found folks to be helpful and collaborative. I'm merely throwing my reaction out there because I trust you to consider it. the "first step" unless overridden by submitter.
I would rather keep the list of stages short, i.e.:
Agree that simple is better than complex.
with the following meanings - no selection -: issue hasn't been triaged/looked at yet
information needed: something needs to be confirmed clarified before proceeding decision needed: an agreement should be reached before taking action patch needed: we know what to do, we need the code review needed: we have the code, we need someone to review it commit (review) needed: we have the code, we need a committer to commit it resolved: issue is now closed
I like these descriptions of the states.
To illustrate the possible evolution of an issue I made a couple of flow charts: http://imgur.com/a/UgJBJ
Thanks for the visuals. They help support understanding the workflow :-)
Also note that is possible to go from basically every state to "resolved", however I omitted those arrows, since they would just clutter the flow chart.
Excellent point. Thanks! Carol P.S. Happy to pitch in with documentation or tracker work as you move forward :-) -- Carol Willing Developer Willing Consulting
On Thu, May 8, 2014 at 8:42 PM, Carol Willing <willingc@willingconsulting.com> wrote:
Ezio, Brett, Nick, and David,
Your thoughts and additional information added some helpful insights and improvements.
Some thoughts in-line below.
Here's a link to a document that helps on-board people and demonstrates the many aspects a tracker communicates information to others in the community: https://wiki.openstack.org/wiki/How_To_Contribute [It illustrates fairly concretely that the tracker supports several different workflows and many different community members.]
I'm going to go out on a limb and say that we're in agreement on a macro level. FWIW, a few thoughts below on 'state' naming.
State ----- "Initial State" (aka No Selection, Triage needed) [I can't think of a great word for this state. What comes to mind is Harry Potter's Sorting Hat to determine where the issue should go] Information Please (aka Information needed) Decision Point (aka decision needed) Patch Development (aka Patch needed) Patch Review Commit Review Resolved
This is exactly the way I thought as well: the initial default state is "triage needed"
IMHO, "triage" has somewhat different connotations to each of us. It's probably best not to use it in the tracker as a state. This would allow documentation to treat "triage" as a process that may differ on the type of task or who is doing the task. My apologies for suggesting it as a state, but it has been interesting seeing the diversity in perspectives of what "triage" means.
FWIW when I started contributing I didn't even know the meaning of the word "triage" (not even in medical context), and I don't think there is any corresponding word in my native language.
and "decision needed" means someone higher up has to make a call on a question.
My apologies in advance if my ability to communicate in writing my next point is below par. Before I say what I am feeling, I want to be clear that I have strong respect for the core developers and committers based on their dedication to and passion for Python. I also have great respect for others that contribute in many different ways and the talents that they bring to a team.
I can't quite put my finger on why, but my gut reaction to "decision needed" when coupled with words like "higher ups", "more experienced", "control", "hierarchy", "proven" is unwelcoming and a little patronizing. Oddly, simply stating "decision point" or "decision" as a state does not invoke the same reaction. I think the word "needed" subtly adds the implication that I am not trusted, capable enough, or proven my intelligence to make a decision, and I must wait to get permission from an "authority figure".
Maybe we could use "agreement" or "consensus" (or something equivalent) instead. For simple issues any core dev could take the decision directly (especially if the issue is related to releases or other policies that contributors might not be aware of), but otherwise taking the decision is a group process where anyone can contribute (much like what we are doing here).
Not a huge sticking point if consensus wants to keep it as "decision needed". I have actually found folks to be helpful and collaborative. I'm merely throwing my reaction out there because I trust you to consider it.
Can we simply name the "no selection" state have the text "triage needed"?
Naming the "no selection" state has merit. Perhaps it auto defaults to the "first step" unless overridden by submitter.
I wonder if only triagers/committers should have the stage field visible while creating a new issue. That would help making sure that, if the stage is different from "no selection", at least a triager looked at the issue. If the field is available to anyone, contributors might set it correctly, but leave other important fields unset, and triagers might end up ignoring the issues because it looks already triaged.
I would rather keep the list of stages short, i.e.:
Agree that simple is better than complex.
with the following meanings - no selection -: issue hasn't been triaged/looked at yet
information needed: something needs to be confirmed clarified before
proceeding decision needed: an agreement should be reached before taking action patch needed: we know what to do, we need the code review needed: we have the code, we need someone to review it commit (review) needed: we have the code, we need a committer to commit it resolved: issue is now closed
I like these descriptions of the states.
To illustrate the possible evolution of an issue I made a couple of flow charts: http://imgur.com/a/UgJBJ
Thanks for the visuals. They help support understanding the workflow :-)
Also note that is possible to go from basically every state to "resolved", however I omitted those arrows, since they would just clutter the flow chart.
Excellent point.
Thanks! Carol
P.S. Happy to pitch in with documentation or tracker work as you move forward :-)
-- Carol Willing Developer Willing Consulting
On 9 May 2014 04:32, "Carol Willing" <willingc@willingconsulting.com> wrote:
and "decision needed" means someone higher up has to make a call on a
question.
My apologies in advance if my ability to communicate in writing my next
point is below par. Before I say what I am feeling, I want to be clear that I have strong respect for the core developers and committers based on their dedication to and passion for Python. I also have great respect for others that contribute in many different ways and the talents that they bring to a team.
I can't quite put my finger on why, but my gut reaction to "decision
needed" when coupled with words like "higher ups", "more experienced", "control", "hierarchy", "proven" is unwelcoming and a little patronizing. Oddly, simply stating "decision point" or "decision" as a state does not invoke the same reaction. I think the word "needed" subtly adds the implication that I am not trusted, capable enough, or proven my intelligence to make a decision, and I must wait to get permission from an "authority figure".
Not a huge sticking point if consensus wants to keep it as "decision
needed". I have actually found folks to be helpful and collaborative. I'm merely throwing my reaction out there because I trust you to consider it. "Consensus/decision needed" may be a more accurate name for the state - an autocratic decision from a core developer or module maintainer is a last resort to get an issue moving again, rather than the preferred approach. "Rough consensus and running code" (or "readable text" in the docs case) is a much better option when it works, but that can occasionally stall with no way to move forward (or, worse, descend into bitter acrimony over a potentially trivial issue) in the absence of an escalation procedure of some kind. So core devs (& particularly module maintainers) really do have decision making authority. I don't think we'd be doing anyone any favours by pretending that hierarchy doesn't exist instead of trying to make it as transparent as possible - including the fact that gaining more responsibility (for those that want it) is mostly a case of "the reward for doing work well is being offered the chance to do more work". On the other hand, Ezio's also right that our decision making authority is essentially limited getting to define how rough "rough consensus" can be in any given situation, and even if a core dev or module maintainer is needed to make the final call, we're often in the situation of catching up on a discussion that may have been going on for a while (see the couple of paragraphs in http://www.joelonsoftware.com/articles/fog0000000072.htmlthat start with "At Microsoft, management was extremely hands-off.") In those cases, *anyone* can take it upon themselves to: * write a comment on the issue tracker summarising the competing perspectives and their pros and cons * escalating the issue to python-dev for broader discussion (preferably with a summary like the one above) * for more radical/controversial notions, start a thread on python-ideas to request help in building a compelling case for the proposal Sometimes those acts will allow consensus to emerge without an explicit decision being needed from anyone. Cheers, Nick.
Nick and Ezio, Thanks! We're in sync. I appreciate your thoughtful responses regarding the state label "decision needed". I'm +1 on transparency and clarity around consensus building and do not disagree with "final decision" authority rests with the module maintainer or core dev which is far better than languishing issues. I believe it would be helpful for some of Nick and Ezio's thoughts making it into some documentation (dev guide?): "...an autocratic decision from a core developer or module maintainer is a last resort to get an issue moving again, rather than the preferred approach...decision making authority is essentially limited getting to define how rough "rough consensus" can be in any given situation, and even if a core dev or module maintainer is needed to make the final call, we're often in the situation of catching up on a discussion that may have been going on for a while." Any of these names (Green light needed, consensus/decision needed, or decision needed) will work well when combined with greater community education (via docs and other communication mechanisms). Hopefully, this upfront dialogue will help the workflow tracker enable more people to contribute most effectively and in turn will reward core devs with more "meaningful" work not just more work. Regards, Carol P.S. Antoine, Point taken. If anyone is aware of a dynamic mailing list message "diff" organizer other than brute force, I would love to know. Cheers! On 5/8/14, 3:07 PM, Nick Coghlan wrote:
On 9 May 2014 04:32, "Carol Willing" <willingc@willingconsulting.com <mailto:willingc@willingconsulting.com>> wrote:
and "decision needed" means someone higher up has to make a call on
a question.
My apologies in advance if my ability to communicate in writing my
next point is below par. Before I say what I am feeling, I want to be clear that I have strong respect for the core developers and committers based on their dedication to and passion for Python. I also have great respect for others that contribute in many different ways and the talents that they bring to a team.
I can't quite put my finger on why, but my gut reaction to "decision
needed" when coupled with words like "higher ups", "more experienced", "control", "hierarchy", "proven" is unwelcoming and a little patronizing. Oddly, simply stating "decision point" or "decision" as a state does not invoke the same reaction. I think the word "needed" subtly adds the implication that I am not trusted, capable enough, or proven my intelligence to make a decision, and I must wait to get permission from an "authority figure".
Not a huge sticking point if consensus wants to keep it as "decision
needed". I have actually found folks to be helpful and collaborative. I'm merely throwing my reaction out there because I trust you to consider it.
"Consensus/decision needed" may be a more accurate name for the state - an autocratic decision from a core developer or module maintainer is a last resort to get an issue moving again, rather than the preferred approach. "Rough consensus and running code" (or "readable text" in the docs case) is a much better option when it works, but that can occasionally stall with no way to move forward (or, worse, descend into bitter acrimony over a potentially trivial issue) in the absence of an escalation procedure of some kind.
So core devs (& particularly module maintainers) really do have decision making authority. I don't think we'd be doing anyone any favours by pretending that hierarchy doesn't exist instead of trying to make it as transparent as possible - including the fact that gaining more responsibility (for those that want it) is mostly a case of "the reward for doing work well is being offered the chance to do more work".
On the other hand, Ezio's also right that our decision making authority is essentially limited getting to define how rough "rough consensus" can be in any given situation, and even if a core dev or module maintainer is needed to make the final call, we're often in the situation of catching up on a discussion that may have been going on for a while (see the couple of paragraphs in http://www.joelonsoftware.com/articles/fog0000000072.html that start with "At Microsoft, management was extremely hands-off.")
In those cases, *anyone* can take it upon themselves to:
* write a comment on the issue tracker summarising the competing perspectives and their pros and cons * escalating the issue to python-dev for broader discussion (preferably with a summary like the one above) * for more radical/controversial notions, start a thread on python-ideas to request help in building a compelling case for the proposal
Sometimes those acts will allow consensus to emerge without an explicit decision being needed from anyone.
Cheers, Nick.
-- Carol Willing Developer Willing Consulting
To everyone participating: the discussion has become IMO completely impossible to follow. There is a single monster-thread discussing all issues at once, without any subject change. I think issues should be broken into distinct threads with distinct mail subjects, to make things readable again. Thanks & regards Antoine. On Thu, 08 May 2014 14:31:35 +0000 Brett Cannon <bcannon@gmail.com> wrote:
On Thu May 08 2014 at 1:44:21 AM, Ezio Melotti <ezio.melotti@gmail.com> wrote:
On Thu, May 8, 2014 at 1:58 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 8 May 2014 01:59, "R. David Murray" <rdmurray@bitdance.com> wrote:
On Wed, 07 May 2014 08:09:03 -0700, Carol Willing <willingc@willingconsulting.com> wrote:
I'm wondering if "decision needed" might be more accurately named "triage needed"?
Looking at David's well documented proposals and other mail comments, "triage needed" more specifically describes the 'state'.
A few thoughts:
1. "Triage needed" would raise the importance and visibility of the triage contributor role. A positive for onboarding and growing development talent.
2. "Triage needed" is more descriptive and clearer than "decision needed" especially for those users that do not read documentation or understand the development workflow. "Decision needed" implies that a decision will be made to include or not include in a release. Realistically, decisions are made throughout the remainder of the development process based on time, resources, etc.
I'll be interested in what others think, but to me "decision needed" is closer than "triage needed". That is, the state means that someone
other
than the person moving the issue to that state needs to make a decision. That decision can be "Is this something we consider a bug? What releases can we fix this in given our backward compatibility requirements? Is this an acceptable enhancement? And any other decision that needs to be made before the issue can move forward.
I agree. To me "triaging" mostly means updating fields/metadata and it's something that is done once the issue is reported. This also includes adding people to the nosy list so that they can comment and start dealing with the issues, but I don't consider this "triaging" anymore. IOW "triage needed" would correspond to "- no selection -". OTOH, "decision needed" means that the people working on the issue need to reach an agreement. A good example of this is http://bugs.python.org/issue18967. Here no triaging is needed IMHO.
This is exactly the way I thought as well: the initial default state is "triage needed" and "decision needed" means someone higher up has to make a call on a question. Can we simply name the "no selection" state have the text "triage needed"?
-Brett
All of these *can* be "triage" decisions, but to my ear it is the word "triage" that is more about deciding where to allocate resources ("which release"), whereas we generally don't work that way. We just decide if it can go in or not, and if the patch is ready before the next release, it can go in.
More specifically, because I removed 'committer decision needed', 'decision needed' covers the case of needing a committer decision, which is by definition not a triage decision :)
Perhaps 'committer decision needed' should be kept after all?
I don't think the distinction is useful. Anyone can contribute to the discussion, as long as they don't just give their opinion and change the stage directly. For example in http://bugs.python.org/issue18967 a Mercurial dev could give his opinion, and if the others agree, the stage can be updated (to "needs patch" or "needs review" if a patch is already present).
From a work queue perspective, two separate states likely makes sense, since "Triage needed" & "Committer decision needed" are aimed at slightly different groups of people (with the latter being a subset of the former). That way, "Committer decision needed" becomes a clear avenue for escalation by triagers to the core developers when they need a design decision or risk assessment on a particular approach.
I would rather keep the list of stages short, i.e.: - no selection - information needed decision needed patch needed review needed commit (review) needed resolved
with the following meanings - no selection -: issue hasn't been triaged/looked at yet information needed: something needs to be confirmed/clarified before proceeding decision needed: an agreement should be reached before taking action patch needed: we know what to do, we need the code review needed: we have the code, we need someone to review it commit (review) needed: we have the code, we need a committer to commit it resolved: issue is now closed
These would be useful to (triagers also includes committers): - no selection -: triagers (searching for untriaged issues) information needed: original poster (probably not useful in searches) decision needed: committers (and possibly triagers) patch needed: everyone (people searching for issues that need patches) review needed: triagers (searching for issues that need a review) commit (review) needed: committers (searching for issues that are ready to go in) resolved: everyone (people marveling at the outstanding work of our team)
Also think how these stages would affect the dashboards (e.g. "information needed" should be prominent on the original poster's dashboard).
To illustrate the possible evolution of an issue I made a couple of flow charts: http://imgur.com/a/UgJBJ
The first is without "patch update needed" the second includes it. Also note that is possible to go from basically every state to "resolved", however I omitted those arrows, since they would just clutter the flow chart. I'm not sure if this should be enforced (probably not), but at least it should provided a clearer picture.
I left "decision needed", removed "patch update needed", and possibly removed the "review" from "commit review needed": 1) "decision needed" means that the problem is clear, but we have to discuss about the best solution. To me, "triage needed" mostly means that the fields/metadata are not updated; 2) "patch update needed" seems redundant to me, and can be replaced with "patch needed" + "assigned to <previous patch author>". I'm not strongly opposed about removing it though; 3) "commit review needed" seems useful to signal to core devs that someone reviewed the patch and it is now ready to be committed. This assumes that the committer will do a further review, but if we already passed the "review needed" stage, there are good chances that the patch is ready to go in (if not we can always go back to "patch needed"). This can also be simply called "commit needed";
That more structured mechanism should nicely complement the option of punting decisions to the collective wisdom (hah!) of python-dev & python-ideas.
Cheers, Nick.
--David
core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
participants (14)
-
Antoine Pitrou -
Antoine Pitrou -
Barry Warsaw -
Berker Peksağ -
Brett Cannon -
Carol Willing -
Ezio Melotti -
francis -
Guido van Rossum -
Jessica McKellar -
Nick Coghlan -
R. David Murray -
Thomas Wouters -
Zachary Ware