interpretation of the draft governance document (was Re: Governance model request)
Hi Travis, On Tue, Sep 22, 2015 at 3:08 AM, Travis Oliphant <travis@continuum.io> wrote:
By "all" I just meant "all of us who were there" (which was a majority of the active maintainers + a number of other interested parties -- the list of attendees is in the meeting notes if you're curious). In general I totally agree with your concern about only including a subset of the community. That's why we followed up by posting to the list a full set of notes on tentative-decisions-made, and the draft governance document in particular, for further discussion. We've already had multiple threads talking about it, even before this one. And it's pretty explicit in the document itself that no non-trivial decision can be considered final unless it's *at least* been posted on the mailing list. We didn't try to legislate the exact review requirements for every decision, because it's impossible to have a set of rules that scales from trivial typos (which just get merged, only github subscribers even know it happened) to foundational discussions like this one. This means that one of the things we trust contributors (esp. senior contributors) to do is to use their knowledge of the project to make judgement calls about how risky or controversial a given change will be, or if there's some particular expertise that should be consulted. (E.g. we might make sure to ping Robert Kern if there's some np.random change being discussed; I'm hesitant to finalize the PyUFunc ABI changes being discussed in the other thread until Ralf gets back, because I know that among the core maintainers he's particularly critical of the idea of breaking ABI.) And if we new information later comes to light then a decision can always be revisited -- people may get grumpy if you try to re-open an issue that's been considered settled for a year, but if you have a good reason and nothing irrevocable has happened (e.g. a veto obviously can't remove code from an existing release), then, well, it's annoying but what can you do, let's hear your reason. It's could certainly happen that sometimes the steering council + mailing list readers will all miss something important. But this is unavoidable in any system -- we're obviously not going to, like, institute a one month waiting period on every single decision or something. Ultimately you have to trust the core maintainers to have good judgement about which changes to accept, together with good meta-judgement about how controversial or broad-reaching any given change is likely to be, and then hope that the rest of the community will also supplement as they can.
See above. For regular discussions, there are some rough guidelines (uncontroversial bug fixes can just be merged; substantive API changes need at least a few days review on the mailing list). This governance discussion has been left open for a few weeks, and: "worst case, if a change is more controversial than expected, or a crucial critique is delayed because someone was on vacation, then it's no big deal: we apologize for misjudging the situation, [back up, and sort things out](http://producingoss.com/en/producingoss.html#version-control-relaxation)." (I think we all thought the governance discussion was done, actually. But then you posted, and so now we're talking about it some more. No worries; if there's an issue, we'd rather know, right?) For formal council votes (the ones we expect will rarely if every happen), then we do have a slightly more formal rule: that the vote "should be held open for long enough to give all interested Council Members a chance to respond -- at least one week." The words "at least" are in there to emphasize that the goal is to get an honest read of the council; e.g. it's not legitimate to play games by scheduling a vote when you know someone is on vacation.
In the draft governance document, anyone who cares enough to pay attention effectively has a seat on this advisory council. I assume this is a superset of the people that you would nominate?
Oh good, I'm glad when things turn out to be misunderstandings, because those are (relatively) easy to solve :-).
I see, right, slight miscommunication here -- I was thinking about the decision of "what process do we [in general] use to decide who gets a commit bit", not the decision "should [this person] get a commit bit". Currently, yeah, the general process is that commit bits get given out by a somewhat informal private discussion among active committers (or, presumably, the "the steering council" if it does get formalized). But if someone wants to suggest that we switch to some other process instead, or formalize the current process, then anything like that would be proposed and debated on the mailing list. (One of my favorite wacky policies is that there are projects like Rubinius which automatically grant commit bits to everyone who submits a successful patch. I guess the theory is that worst case, they go and merge something they shouldn't have, more senior folks see it going by and they revert it again, no biggie! Usually the problem is just the opposite -- there are never enough reviewers. But I have not quite had the guts to seriously propose this for numpy ;-).) -n -- Nathaniel J. Smith -- http://vorpus.org
Hi Nathaniel, Thanks for the clarifications. Is the governance document committed to the repository? I keep looking for it and have a hard time finding it --- I think I read it last in an email. In this way, I could make Pull Requests to the governance document if there are concrete suggestions for change, and then have them reviewed in the standard way. I'm hopeful that a few tweaks to the document would satisfy all my concerns. Thanks, -Travis On Wed, Sep 23, 2015 at 1:04 PM, Nathaniel Smith <njs@pobox.com> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On Wed, Sep 23, 2015 at 11:59 AM, Travis Oliphant <travis@continuum.io> wrote:
Indeed, sorry -- getting it into the repo has been on my todo list, though somewhat on hold given the uncertainty the last few days. For reference in the mean time, the original posting is here: http://thread.gmane.org/gmane.comp.python.numeric.general/61106 and I'll paste it again at the bottom of this email. (The only difference between the version in this email and the previous should be the addition of Jaime to fill in the empty slot on the tentative NumFOCUS subcommittee.)
In this way, I could make Pull Requests to the governance document if there
are concrete suggestions for change, and then have them reviewed in the standard way.
While this makes sense, I think the "standard way" for reviewing substantive changes (i.e., not just simple wording changes) here would be via discussion on the mailing list :-). We're still trying to figure out the best way to balance github versus mailing list... it sounds like things are at least converging somewhat, though, so I'll work on getting a PR together soon in any case. -n ------ The purpose of this document is to formalize the governance process used by the NumPy project in both ordinary and extraordinary situations, and to clarify how decisions are made and how the various elements of our community interact, including the relationship between open source collaborative development and work that may be funded by for-profit or non-profit entities. Summary ======= NumPy is a community-owned and community-run project. To the maximum extent possible, decisions about project direction are made by community consensus (but note that "consensus" here has a somewhat technical meaning that might not match everyone's expectations -- see below). Some members of the community additionally contribute by serving on the NumPy steering council, where they are responsible for facilitating the establishment of community consensus, for stewarding project resources, and -- in extreme cases -- for making project decisions if the normal community-based process breaks down. The Project =========== The NumPy Project (The Project) is an open source software project affiliated with the 501(c)3 NumFocus Foundation. The goal of The Project is to develop open source software for array-based computing in Python, and in particular the `numpy` package, along with related software such as `f2py` and the NumPy Sphinx extensions. The Software developed by The Project is released under the BSD (or similar) open source license, developed openly and hosted on public GitHub repositories under the `numpy` GitHub organization. The Project is developed by a team of distributed developers, called Contributors. Contributors are individuals who have contributed code, documentation, designs or other work to the Project. Anyone can be a Contributor. Contributors can be affiliated with any legal entity or none. Contributors participate in the project by submitting, reviewing and discussing GitHub Pull Requests and Issues and participating in open and public Project discussions on GitHub, mailing lists, and other channels. The foundation of Project participation is openness and transparency. Here is a list of the current Contributors to the main NumPy repository: [ https://github.com/numpy/numpy/graphs/contributors](https://github.com/numpy/numpy/graphs/contributors) The Project Community consists of all Contributors and Users of the Project. Contributors work on behalf of and are responsible to the larger Project Community and we strive to keep the barrier between Contributors and Users as low as possible. The Project is formally affiliated with the 501(c)3 NumFOCUS Foundation ([ http://numfocus.org](http://numfocus.org)), which serves as its fiscal sponsor, may hold project trademarks and other intellectual property, helps manage project donations and acts as a parent legal entity. NumFOCUS is the only legal entity that has a formal relationship with the project (see Institutional Partners section below). Governance ========== This section describes the governance and leadership model of The Project. The foundations of Project governance are: - Openness & Transparency - Active Contribution - Institutional Neutrality Consensus-based decision making by the community ------------------------------------------------ Normally, all project decisions will be made by consensus of all interested Contributors. The primary goal of this approach is to ensure that the people who are most affected by and involved in any given change can contribute their knowledge in the confidence that their voices will be heard, because thoughtful review from a broad community is the best mechanism we know of for creating high-quality software. The mechanism we use to accomplish this goal may be unfamiliar for those who are not experienced with the cultural norms around free/open-source software development. We provide a summary here, and highly recommend that all Contributors additionally read [Chapter 4: Social and Political Infrastructure]( http://producingoss.com/en/producingoss.html#social-infrastructure) of Karl Fogel's classic *Producing Open Source Software*, and in particular the section on [Consensus-based Democracy]( http://producingoss.com/en/producingoss.html#consensus-democracy), for a more detailed discussion. In this context, consensus does *not* require: - that we wait to solicit everybody's opinion on every change, - that we ever hold a vote on anything, - or that everybody is happy or agrees with every decision. For us, what consensus means is that we entrust *everyone* with the right to veto any change if they feel it necessary. While this may sound like a recipe for obstruction and pain, this is not what happens. Instead, we find that most people take this responsibility seriously, and only invoke their veto when they judge that a serious problem is being ignored, and that their veto is necessary to protect the project. And in practice, it turns out that such vetoes are almost never formally invoked, because their mere possibility ensures that Contributors are motivated from the start to find some solution that everyone can live with -- thus accomplishing our goal of ensuring that all interested perspectives are taken into account. How do we know when consensus has been achieved? In principle, this is rather difficult, since consensus is defined by the absence of vetos, which requires us to somehow prove a negative. In practice, we use a combination of our best judgement (e.g., a simple and uncontroversial bug fix posted on GitHub and reviewed by a core developer is probably fine) and best efforts (e.g., all substantive API changes must be posted to the mailing list in order to give the broader community a chance to catch any problems and suggest improvements; we assume that anyone who cares enough about NumPy to invoke their veto right should be on the mailing list). If no-one bothers to comment on the mailing list after a few days, then it's probably fine. And worst case, if a change is more controversial than expected, or a crucial critique is delayed because someone was on vacation, then it's no big deal: we apologize for misjudging the situation, [back up, and sort things out]( http://producingoss.com/en/producingoss.html#version-control-relaxation). If one does need to invoke a formal veto, then it should consist of: - an unambiguous statement that a veto is being invoked, - an explanation of why it is being invoked, and - a description of what conditions (if any) would convince the vetoer to withdraw their veto. If all proposals for resolving some issue are vetoed, then the status quo wins by default. In the worst case, if a Contributor is genuinely misusing their veto in an obstructive fashion to the detriment of the project, then they can be ejected from the project by consensus of the Steering Council -- see below. Steering Council ---------------- The Project will have a Steering Council that consists of Project Contributors who have produced contributions that are substantial in quality and quantity, and sustained over at least one year. The overall role of the Council is to ensure, with input from the Community, the long-term well-being of the project, both technically and as a community. During the everyday project activities, council members participate in all discussions, code review and other project activities as peers with all other Contributors and the Community. In these everyday activities, Council Members do not have any special power or privilege through their membership on the Council. However, it is expected that because of the quality and quantity of their contributions and their expert knowledge of the Project Software and Services that Council Members will provide useful guidance, both technical and in terms of project direction, to potentially less experienced contributors. The Steering Council and its Members play a special role in certain situations. In particular, the Council may, if necessary: - Make decisions about the overall scope, vision and direction of the project. - Make decisions about strategic collaborations with other organizations or individuals. - Make decisions about specific technical issues, features, bugs and pull requests. They are the primary mechanism of guiding the code review process and merging pull requests. - Make decisions about the Services that are run by The Project and manage those Services for the benefit of the Project and Community. - Update policy documents such as this one. - Make decisions when regular community discussion doesn’t produce consensus on an issue in a reasonable time frame. However, the Council's primary responsibility is to facilitate the ordinary community-based decision making procedure described above. If we ever have to step in and formally override the community for the health of the Project, then we will do so, but we will consider reaching this point to indicate a failure in our leadership. ### Council decision making If it becomes necessary for the Steering Council to produce a formal decision, then they will use a form of the [Apache Foundation voting process](https://www.apache.org/foundation/voting.html). This is a formalized version of consensus, in which +1 votes indicate agreement, -1 votes are vetoes (and must be accompanied with a rationale, as above), and one can also vote fractionally (e.g. -0.5, +0.5) if one wishes to express an opinion without registering a full veto. These numeric votes are also often used informally as a way of getting a general sense of people's feelings on some issue, and should not normally be taken as formal votes. A formal vote only occurs if explicitly declared, and if this does occur then the vote should be held open for long enough to give all interested Council Members a chance to respond -- at least one week. In practice, we anticipate that for most Steering Council decisions (e.g., voting in new members) a more informal process will suffice. ### Council membership To become eligible to join the Steering Council, an individual must be a Project Contributor who has produced contributions that are substantial in quality and quantity, and sustained over at least one year. Potential Council Members are nominated by existing Council members and voted upon by the existing Council after asking if the potential Member is interested and willing to serve in that capacity. The Council will be initially formed from the set of existing Core Developers who, as of late 2015, have been significantly active over the last year. When considering potential Members, the Council will look at candidates with a comprehensive view of their contributions. This will include but is not limited to code, code review, infrastructure work, mailing list and chat participation, community help/building, education and outreach, design work, etc. We are deliberately not setting arbitrary quantitative metrics (like “100 commits in this repo”) to avoid encouraging behavior that plays to the metrics rather than the project’s overall well-being. We want to encourage a diverse array of backgrounds, viewpoints and talents in our team, which is why we explicitly do not define code as the sole metric on which council membership will be evaluated. If a Council member becomes inactive in the project for a period of one year, they will be considered for removal from the Council. Before removal, inactive Member will be approached to see if they plan on returning to active participation. If not they will be removed immediately upon a Council vote. If they plan on returning to active participation soon, they will be given a grace period of one year. If they don’t return to active participation within that time period they will be removed by vote of the Council without further grace period. All former Council members can be considered for membership again at any time in the future, like any other Project Contributor. Retired Council members will be listed on the project website, acknowledging the period during which they were active in the Council. The Council reserves the right to eject current Members, if they are deemed to be actively harmful to the project’s well-being, and attempts at communication and conflict resolution have failed. This requires the consensus of the remaining Members. [We also have to decide on the initial membership for the Council. While the above text makes pains to distinguish between "committer" and "Council Member", in the past we've pretty much treated them as the same. So to keep things simple and deterministic, I propose that we seed the Council with everyone who has reviewed/merged a pull request since Jan 1, 2014, and move those who haven't used their commit bit in >1.5 years to the emeritus list. Based on the output of git log --grep="^Merge pull request" --since 2014-01-01 | grep Author: | sort -u I believe this would give us an initial Steering Council of: Sebastian Berg, Jaime Fernández del Río, Ralf Gommers, Alex Griffing, Charles Harris, Nathaniel Smith, Julian Taylor, and Pauli Virtanen (assuming everyone on that list is interested/willing to serve).] ### Conflict of interest It is expected that the Council Members will be employed at a wide range of companies, universities and non-profit organizations. Because of this, it is possible that Members will have conflict of interests. Such conflict of interests include, but are not limited to: - Financial interests, such as investments, employment or contracting work, outside of The Project that may influence their work on The Project. - Access to proprietary information of their employer that could potentially leak into their work with the Project. All members of the Council shall disclose to the rest of the Council any conflict of interest they may have. Members with a conflict of interest in a particular issue may participate in Council discussions on that issue, but must recuse themselves from voting on the issue. ### Private communications of the Council Unless specifically required, all Council discussions and activities will be public and done in collaboration and discussion with the Project Contributors and Community. The Council will have a private mailing list that will be used sparingly and only when a specific matter requires privacy. When private communications and decisions are needed, the Council will do its best to summarize those to the Community after eliding personal/private/sensitive information that should not be posted to the public internet. ### Subcommittees The Council can create subcommittees that provide leadership and guidance for specific aspects of the project. Like the Council as a whole, subcommittees should conduct their business in an open and public manner unless privacy is specifically called for. Private subcommittee communications should happen on the main private mailing list of the Council unless specifically called for. ### NumFOCUS Subcommittee The Council will maintain one narrowly focused subcommittee to manage its interactions with NumFOCUS. - The NumFOCUS Subcommittee is comprised of 5 persons who manage project funding that comes through NumFOCUS. It is expected that these funds will be spent in a manner that is consistent with the non-profit mission of NumFOCUS and the direction of the Project as determined by the full Council. - This Subcommittee shall NOT make decisions about the direction, scope or technical direction of the Project. - This Subcommittee will have 5 members, 4 of whom will be current Council Members and 1 of whom will be external to the Steering Council. No more than 2 Subcommitee Members can report to one person through employment or contracting work (including the reportee, i.e. the reportee + 1 is the max). This avoids effective majorities resting on one person. [Initially, the NumFOCUS subcommittee will consist of: Chuck Harris, Ralf Gommers, Nathaniel Smith, Jaime Fernández del Río as internal members, and Thomas Caswell as external member.] Institutional Partners and Funding ================================== The Steering Council are the primary leadership for the project. No outside institution, individual or legal entity has the ability to own, control, usurp or influence the project other than by participating in the Project as Contributors and Council Members. However, because institutions can be an important funding mechanism for the project, it is important to formally acknowledge institutional participation in the project. These are Institutional Partners. An Institutional Contributor is any individual Project Contributor who contributes to the project as part of their official duties at an Institutional Partner. Likewise, an Institutional Council Member is any Project Steering Council Member who contributes to the project as part of their official duties at an Institutional Partner. With these definitions, an Institutional Partner is any recognized legal entity in the United States or elsewhere that employs at least 1 Institutional Contributor of Institutional Council Member. Institutional Partners can be for-profit or non-profit entities. Institutions become eligible to become an Institutional Partner by employing individuals who actively contribute to The Project as part of their official duties. To state this another way, the only way for a Partner to influence the project is by actively contributing to the open development of the project, in equal terms to any other member of the community of Contributors and Council Members. Merely using Project Software in institutional context does not allow an entity to become an Institutional Partner. Financial gifts do not enable an entity to become an Institutional Partner. Once an institution becomes eligible for Institutional Partnership, the Steering Council must nominate and approve the Partnership. If an existing Institutional Partner no longer has a contributing employee, they will be given a 1 year grace period for remaining employees to begin contributing. An Institutional Partner is free to pursue funding for their work on The Project through any legal means. This could involve a non-profit organization raising money from private foundations and donors or a for-profit company building proprietary products and services that leverage Project Software and Services. Funding acquired by Institutional Partners to work on The Project is called Institutional Funding. However, no funding obtained by an Institutional Partner can override the Steering Council. If a Partner has funding to do NumPy work and the Council decides to not pursue that work as a project, the Partner is free to pursue it on their own. However in this situation, that part of the Partner’s work will not be under the NumPy umbrella and cannot use the Project trademarks in a way that suggests a formal relationship. Institutional Partner benefits are: - Acknowledgement on the NumPy websites, in talks and T-shirts. - Ability to acknowledge their own funding sources on the NumPy websites, in talks and T-shirts. - Ability to influence the project through the participation of their Council Member. - Council Members invited to NumPy Developer Meetings. Existing Institutional Partners: - UC Berkeley (Nathaniel Smith) Acknowledgements ================ Substantial portions of this document were ~~inspired~~ stolen wholesale from the Jupyter/IPython project's governance document, [IPEP 29]( https://github.com/ipython/ipython/wiki/IPEP-29:-Project-Governance). -- Nathaniel J. Smith -- http://vorpus.org
One of the problems with this list is that my contributions to the project are extremely under-represented because the large majority of my commitment of code happened in 2005 to 2006 before github was used. So, using this as a list of the contributors is quite misleading --- and there are a lot of people now looking only at lists like this one and it might confuse them why I care so much. So, if you are going to make this list public in a governance document like this, then I think some acknowledgement of the source of the original code and the contributors to that needs to also be made --- or you could just also point to the THANKS document which lists people up to about 2008. Between 2008 and 2010 we will lose contributions, still and this can be acknowledged.
Concretely, I'm asking to be included in this initial council so a simple "along with Travis Oliphant who is the original author of NumPy". If other long-time contributors to the code-base also want to be on this initial seed council, I think it would make sense as well.
I would just remove quite a bit of this language and just say that members who are inactive on the Council may be asked to step down if they are not planning on returning to active participation.
I would ask to be on this initial council by having the rule include "original contributors of the code" which would basically include Robert Kern and Pearu Peterson in addition to Chuck Harris (though Pearu's main contribution was numpy.distutils and f2py and he may not be interested any longer in the rest of it). Thanks, -Travis
Travis -- have you included all your email addresses in your GitHub profile? When I type git shortlog -ne, I see 2063 commits from your Continuum address that seem to be missing from the contributors page on github. Generally speaking, the git logs tend to be more reliable for these counts. On Wed, Sep 23, 2015 at 3:12 PM, Travis Oliphant <travis@continuum.io> wrote:
On Wed, Sep 23, 2015 at 3:12 PM, Travis Oliphant <travis@continuum.io> wrote:
I'm actually not even sure why this link is even here -- it's another thing that just got copied from Jupyter/IPython :-) It looks like the reason your commits are massively undercounted on that page is that github uses emails to match up commits to github accounts. Unfortunately it looks like almost all your old commits used <oliphant@enthought.com> for your email address, and apparently the @teoliphant github account doesn't have that listed as one of its emails. I guess that's probably unfixable on your end, and while git has a standard way to solve these issues (.mailmap -- that's why Stephan got the right answer from git shortlog), github is not clever enough to use it: https://github.com/holman/ama/issues/551#issuecomment-46129792 So it sounds like that page is basically unfixable, and if it's misleading then we should just delete it from the doc. We definitely don't want to diminish anyone's contributions! [snip]
I'll leave this discussion for the other thread that's specifically about the council membership, so as not to confuse matters.
I think it's important to have *some* kind of relatively objective/impartial criteria here, because if the rules force somebody to be the bad guy before anyone can get pruned, then the council will never get pruned, and over time we'll end up with an official council membership list that gets more and more out of touch with reality. Which is the kind of "social debt" we don't need IMO -- we already have enough technical debt to sort out :-). That doesn't mean it necessarily has to be these particular criteria though... probably the decision about what exactly they should be is another thing to defer to the other thread (or at least put on hold until we have some decision on the initial membership question). I'm feeling good -- it sounds like we're narrowing things down quite a bit now. -n -- Nathaniel J. Smith -- http://vorpus.org
On Thu, Sep 24, 2015 at 12:22 AM, Nathaniel Smith <njs@pobox.com> wrote:
I just posted a pull request version of the governance document: https://github.com/numpy/numpy/pull/6352 It's definitely still a draft, but now that we're making changes I wanted to get it into a form where people can at least see what those are :-). In interests of making that easy, the PR is structured as a ReST-converted version of the document originally posted, plus clearly-separated commits on top of that. The only non-trivial changes are: - I split the lists of people out into a separate document: https://github.com/njsmith/numpy/blob/governance/doc/source/dev/governance/p... (currently filled with the original seed council proposal for the reasons mentioned above) - I removed the link to the inaccurate github "contributors" page -n -- Nathaniel J. Smith -- http://vorpus.org
On Do, 2015-09-24 at 00:22 -0700, Nathaniel Smith wrote:
Well, I think we can at least learn one thing from this directly. I think we should include some relation to historic contributions into the document. I think what might be nice to do would be for example to also "seed" the emeritus list (if it is not too difficult). It is somewhat unrelated to governance, but since we want it to be a prominent document, it should not overshadow the past. - Sebastian <snip>
-n
[We also have to decide on the initial membership for the Council. While
On Wed, Sep 23, 2015 at 11:12 PM, Travis Oliphant <travis@continuum.io> wrote: the above text makes pains to distinguish between "committer" and "Council Member", in the past we've pretty much treated them as the same. So to keep things simple and deterministic, I propose that we seed the Council with everyone who has reviewed/merged a pull request since Jan 1, 2014, and move those who haven't used their commit bit in >1.5 years to the emeritus list. Based on the output of that list is interested/willing to serve).]
I would ask to be on this initial council by having the rule include
"original contributors of the code" which would basically include Robert Kern and Pearu Peterson in addition to Chuck Harris (though Pearu's main contribution was numpy.distutils and f2py and he may not be interested any longer in the rest of it). Were I to want such a thing, I would simply review and merge one of the many languishing PRs. I have the power to add myself to this list without asking for special exemptions to the general rule, merely a token bit of effort on my part. -- Robert Kern
Hi Nathaniel, Thanks for the clarifications. Is the governance document committed to the repository? I keep looking for it and have a hard time finding it --- I think I read it last in an email. In this way, I could make Pull Requests to the governance document if there are concrete suggestions for change, and then have them reviewed in the standard way. I'm hopeful that a few tweaks to the document would satisfy all my concerns. Thanks, -Travis On Wed, Sep 23, 2015 at 1:04 PM, Nathaniel Smith <njs@pobox.com> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On Wed, Sep 23, 2015 at 11:59 AM, Travis Oliphant <travis@continuum.io> wrote:
Indeed, sorry -- getting it into the repo has been on my todo list, though somewhat on hold given the uncertainty the last few days. For reference in the mean time, the original posting is here: http://thread.gmane.org/gmane.comp.python.numeric.general/61106 and I'll paste it again at the bottom of this email. (The only difference between the version in this email and the previous should be the addition of Jaime to fill in the empty slot on the tentative NumFOCUS subcommittee.)
In this way, I could make Pull Requests to the governance document if there
are concrete suggestions for change, and then have them reviewed in the standard way.
While this makes sense, I think the "standard way" for reviewing substantive changes (i.e., not just simple wording changes) here would be via discussion on the mailing list :-). We're still trying to figure out the best way to balance github versus mailing list... it sounds like things are at least converging somewhat, though, so I'll work on getting a PR together soon in any case. -n ------ The purpose of this document is to formalize the governance process used by the NumPy project in both ordinary and extraordinary situations, and to clarify how decisions are made and how the various elements of our community interact, including the relationship between open source collaborative development and work that may be funded by for-profit or non-profit entities. Summary ======= NumPy is a community-owned and community-run project. To the maximum extent possible, decisions about project direction are made by community consensus (but note that "consensus" here has a somewhat technical meaning that might not match everyone's expectations -- see below). Some members of the community additionally contribute by serving on the NumPy steering council, where they are responsible for facilitating the establishment of community consensus, for stewarding project resources, and -- in extreme cases -- for making project decisions if the normal community-based process breaks down. The Project =========== The NumPy Project (The Project) is an open source software project affiliated with the 501(c)3 NumFocus Foundation. The goal of The Project is to develop open source software for array-based computing in Python, and in particular the `numpy` package, along with related software such as `f2py` and the NumPy Sphinx extensions. The Software developed by The Project is released under the BSD (or similar) open source license, developed openly and hosted on public GitHub repositories under the `numpy` GitHub organization. The Project is developed by a team of distributed developers, called Contributors. Contributors are individuals who have contributed code, documentation, designs or other work to the Project. Anyone can be a Contributor. Contributors can be affiliated with any legal entity or none. Contributors participate in the project by submitting, reviewing and discussing GitHub Pull Requests and Issues and participating in open and public Project discussions on GitHub, mailing lists, and other channels. The foundation of Project participation is openness and transparency. Here is a list of the current Contributors to the main NumPy repository: [ https://github.com/numpy/numpy/graphs/contributors](https://github.com/numpy/numpy/graphs/contributors) The Project Community consists of all Contributors and Users of the Project. Contributors work on behalf of and are responsible to the larger Project Community and we strive to keep the barrier between Contributors and Users as low as possible. The Project is formally affiliated with the 501(c)3 NumFOCUS Foundation ([ http://numfocus.org](http://numfocus.org)), which serves as its fiscal sponsor, may hold project trademarks and other intellectual property, helps manage project donations and acts as a parent legal entity. NumFOCUS is the only legal entity that has a formal relationship with the project (see Institutional Partners section below). Governance ========== This section describes the governance and leadership model of The Project. The foundations of Project governance are: - Openness & Transparency - Active Contribution - Institutional Neutrality Consensus-based decision making by the community ------------------------------------------------ Normally, all project decisions will be made by consensus of all interested Contributors. The primary goal of this approach is to ensure that the people who are most affected by and involved in any given change can contribute their knowledge in the confidence that their voices will be heard, because thoughtful review from a broad community is the best mechanism we know of for creating high-quality software. The mechanism we use to accomplish this goal may be unfamiliar for those who are not experienced with the cultural norms around free/open-source software development. We provide a summary here, and highly recommend that all Contributors additionally read [Chapter 4: Social and Political Infrastructure]( http://producingoss.com/en/producingoss.html#social-infrastructure) of Karl Fogel's classic *Producing Open Source Software*, and in particular the section on [Consensus-based Democracy]( http://producingoss.com/en/producingoss.html#consensus-democracy), for a more detailed discussion. In this context, consensus does *not* require: - that we wait to solicit everybody's opinion on every change, - that we ever hold a vote on anything, - or that everybody is happy or agrees with every decision. For us, what consensus means is that we entrust *everyone* with the right to veto any change if they feel it necessary. While this may sound like a recipe for obstruction and pain, this is not what happens. Instead, we find that most people take this responsibility seriously, and only invoke their veto when they judge that a serious problem is being ignored, and that their veto is necessary to protect the project. And in practice, it turns out that such vetoes are almost never formally invoked, because their mere possibility ensures that Contributors are motivated from the start to find some solution that everyone can live with -- thus accomplishing our goal of ensuring that all interested perspectives are taken into account. How do we know when consensus has been achieved? In principle, this is rather difficult, since consensus is defined by the absence of vetos, which requires us to somehow prove a negative. In practice, we use a combination of our best judgement (e.g., a simple and uncontroversial bug fix posted on GitHub and reviewed by a core developer is probably fine) and best efforts (e.g., all substantive API changes must be posted to the mailing list in order to give the broader community a chance to catch any problems and suggest improvements; we assume that anyone who cares enough about NumPy to invoke their veto right should be on the mailing list). If no-one bothers to comment on the mailing list after a few days, then it's probably fine. And worst case, if a change is more controversial than expected, or a crucial critique is delayed because someone was on vacation, then it's no big deal: we apologize for misjudging the situation, [back up, and sort things out]( http://producingoss.com/en/producingoss.html#version-control-relaxation). If one does need to invoke a formal veto, then it should consist of: - an unambiguous statement that a veto is being invoked, - an explanation of why it is being invoked, and - a description of what conditions (if any) would convince the vetoer to withdraw their veto. If all proposals for resolving some issue are vetoed, then the status quo wins by default. In the worst case, if a Contributor is genuinely misusing their veto in an obstructive fashion to the detriment of the project, then they can be ejected from the project by consensus of the Steering Council -- see below. Steering Council ---------------- The Project will have a Steering Council that consists of Project Contributors who have produced contributions that are substantial in quality and quantity, and sustained over at least one year. The overall role of the Council is to ensure, with input from the Community, the long-term well-being of the project, both technically and as a community. During the everyday project activities, council members participate in all discussions, code review and other project activities as peers with all other Contributors and the Community. In these everyday activities, Council Members do not have any special power or privilege through their membership on the Council. However, it is expected that because of the quality and quantity of their contributions and their expert knowledge of the Project Software and Services that Council Members will provide useful guidance, both technical and in terms of project direction, to potentially less experienced contributors. The Steering Council and its Members play a special role in certain situations. In particular, the Council may, if necessary: - Make decisions about the overall scope, vision and direction of the project. - Make decisions about strategic collaborations with other organizations or individuals. - Make decisions about specific technical issues, features, bugs and pull requests. They are the primary mechanism of guiding the code review process and merging pull requests. - Make decisions about the Services that are run by The Project and manage those Services for the benefit of the Project and Community. - Update policy documents such as this one. - Make decisions when regular community discussion doesn’t produce consensus on an issue in a reasonable time frame. However, the Council's primary responsibility is to facilitate the ordinary community-based decision making procedure described above. If we ever have to step in and formally override the community for the health of the Project, then we will do so, but we will consider reaching this point to indicate a failure in our leadership. ### Council decision making If it becomes necessary for the Steering Council to produce a formal decision, then they will use a form of the [Apache Foundation voting process](https://www.apache.org/foundation/voting.html). This is a formalized version of consensus, in which +1 votes indicate agreement, -1 votes are vetoes (and must be accompanied with a rationale, as above), and one can also vote fractionally (e.g. -0.5, +0.5) if one wishes to express an opinion without registering a full veto. These numeric votes are also often used informally as a way of getting a general sense of people's feelings on some issue, and should not normally be taken as formal votes. A formal vote only occurs if explicitly declared, and if this does occur then the vote should be held open for long enough to give all interested Council Members a chance to respond -- at least one week. In practice, we anticipate that for most Steering Council decisions (e.g., voting in new members) a more informal process will suffice. ### Council membership To become eligible to join the Steering Council, an individual must be a Project Contributor who has produced contributions that are substantial in quality and quantity, and sustained over at least one year. Potential Council Members are nominated by existing Council members and voted upon by the existing Council after asking if the potential Member is interested and willing to serve in that capacity. The Council will be initially formed from the set of existing Core Developers who, as of late 2015, have been significantly active over the last year. When considering potential Members, the Council will look at candidates with a comprehensive view of their contributions. This will include but is not limited to code, code review, infrastructure work, mailing list and chat participation, community help/building, education and outreach, design work, etc. We are deliberately not setting arbitrary quantitative metrics (like “100 commits in this repo”) to avoid encouraging behavior that plays to the metrics rather than the project’s overall well-being. We want to encourage a diverse array of backgrounds, viewpoints and talents in our team, which is why we explicitly do not define code as the sole metric on which council membership will be evaluated. If a Council member becomes inactive in the project for a period of one year, they will be considered for removal from the Council. Before removal, inactive Member will be approached to see if they plan on returning to active participation. If not they will be removed immediately upon a Council vote. If they plan on returning to active participation soon, they will be given a grace period of one year. If they don’t return to active participation within that time period they will be removed by vote of the Council without further grace period. All former Council members can be considered for membership again at any time in the future, like any other Project Contributor. Retired Council members will be listed on the project website, acknowledging the period during which they were active in the Council. The Council reserves the right to eject current Members, if they are deemed to be actively harmful to the project’s well-being, and attempts at communication and conflict resolution have failed. This requires the consensus of the remaining Members. [We also have to decide on the initial membership for the Council. While the above text makes pains to distinguish between "committer" and "Council Member", in the past we've pretty much treated them as the same. So to keep things simple and deterministic, I propose that we seed the Council with everyone who has reviewed/merged a pull request since Jan 1, 2014, and move those who haven't used their commit bit in >1.5 years to the emeritus list. Based on the output of git log --grep="^Merge pull request" --since 2014-01-01 | grep Author: | sort -u I believe this would give us an initial Steering Council of: Sebastian Berg, Jaime Fernández del Río, Ralf Gommers, Alex Griffing, Charles Harris, Nathaniel Smith, Julian Taylor, and Pauli Virtanen (assuming everyone on that list is interested/willing to serve).] ### Conflict of interest It is expected that the Council Members will be employed at a wide range of companies, universities and non-profit organizations. Because of this, it is possible that Members will have conflict of interests. Such conflict of interests include, but are not limited to: - Financial interests, such as investments, employment or contracting work, outside of The Project that may influence their work on The Project. - Access to proprietary information of their employer that could potentially leak into their work with the Project. All members of the Council shall disclose to the rest of the Council any conflict of interest they may have. Members with a conflict of interest in a particular issue may participate in Council discussions on that issue, but must recuse themselves from voting on the issue. ### Private communications of the Council Unless specifically required, all Council discussions and activities will be public and done in collaboration and discussion with the Project Contributors and Community. The Council will have a private mailing list that will be used sparingly and only when a specific matter requires privacy. When private communications and decisions are needed, the Council will do its best to summarize those to the Community after eliding personal/private/sensitive information that should not be posted to the public internet. ### Subcommittees The Council can create subcommittees that provide leadership and guidance for specific aspects of the project. Like the Council as a whole, subcommittees should conduct their business in an open and public manner unless privacy is specifically called for. Private subcommittee communications should happen on the main private mailing list of the Council unless specifically called for. ### NumFOCUS Subcommittee The Council will maintain one narrowly focused subcommittee to manage its interactions with NumFOCUS. - The NumFOCUS Subcommittee is comprised of 5 persons who manage project funding that comes through NumFOCUS. It is expected that these funds will be spent in a manner that is consistent with the non-profit mission of NumFOCUS and the direction of the Project as determined by the full Council. - This Subcommittee shall NOT make decisions about the direction, scope or technical direction of the Project. - This Subcommittee will have 5 members, 4 of whom will be current Council Members and 1 of whom will be external to the Steering Council. No more than 2 Subcommitee Members can report to one person through employment or contracting work (including the reportee, i.e. the reportee + 1 is the max). This avoids effective majorities resting on one person. [Initially, the NumFOCUS subcommittee will consist of: Chuck Harris, Ralf Gommers, Nathaniel Smith, Jaime Fernández del Río as internal members, and Thomas Caswell as external member.] Institutional Partners and Funding ================================== The Steering Council are the primary leadership for the project. No outside institution, individual or legal entity has the ability to own, control, usurp or influence the project other than by participating in the Project as Contributors and Council Members. However, because institutions can be an important funding mechanism for the project, it is important to formally acknowledge institutional participation in the project. These are Institutional Partners. An Institutional Contributor is any individual Project Contributor who contributes to the project as part of their official duties at an Institutional Partner. Likewise, an Institutional Council Member is any Project Steering Council Member who contributes to the project as part of their official duties at an Institutional Partner. With these definitions, an Institutional Partner is any recognized legal entity in the United States or elsewhere that employs at least 1 Institutional Contributor of Institutional Council Member. Institutional Partners can be for-profit or non-profit entities. Institutions become eligible to become an Institutional Partner by employing individuals who actively contribute to The Project as part of their official duties. To state this another way, the only way for a Partner to influence the project is by actively contributing to the open development of the project, in equal terms to any other member of the community of Contributors and Council Members. Merely using Project Software in institutional context does not allow an entity to become an Institutional Partner. Financial gifts do not enable an entity to become an Institutional Partner. Once an institution becomes eligible for Institutional Partnership, the Steering Council must nominate and approve the Partnership. If an existing Institutional Partner no longer has a contributing employee, they will be given a 1 year grace period for remaining employees to begin contributing. An Institutional Partner is free to pursue funding for their work on The Project through any legal means. This could involve a non-profit organization raising money from private foundations and donors or a for-profit company building proprietary products and services that leverage Project Software and Services. Funding acquired by Institutional Partners to work on The Project is called Institutional Funding. However, no funding obtained by an Institutional Partner can override the Steering Council. If a Partner has funding to do NumPy work and the Council decides to not pursue that work as a project, the Partner is free to pursue it on their own. However in this situation, that part of the Partner’s work will not be under the NumPy umbrella and cannot use the Project trademarks in a way that suggests a formal relationship. Institutional Partner benefits are: - Acknowledgement on the NumPy websites, in talks and T-shirts. - Ability to acknowledge their own funding sources on the NumPy websites, in talks and T-shirts. - Ability to influence the project through the participation of their Council Member. - Council Members invited to NumPy Developer Meetings. Existing Institutional Partners: - UC Berkeley (Nathaniel Smith) Acknowledgements ================ Substantial portions of this document were ~~inspired~~ stolen wholesale from the Jupyter/IPython project's governance document, [IPEP 29]( https://github.com/ipython/ipython/wiki/IPEP-29:-Project-Governance). -- Nathaniel J. Smith -- http://vorpus.org
One of the problems with this list is that my contributions to the project are extremely under-represented because the large majority of my commitment of code happened in 2005 to 2006 before github was used. So, using this as a list of the contributors is quite misleading --- and there are a lot of people now looking only at lists like this one and it might confuse them why I care so much. So, if you are going to make this list public in a governance document like this, then I think some acknowledgement of the source of the original code and the contributors to that needs to also be made --- or you could just also point to the THANKS document which lists people up to about 2008. Between 2008 and 2010 we will lose contributions, still and this can be acknowledged.
Concretely, I'm asking to be included in this initial council so a simple "along with Travis Oliphant who is the original author of NumPy". If other long-time contributors to the code-base also want to be on this initial seed council, I think it would make sense as well.
I would just remove quite a bit of this language and just say that members who are inactive on the Council may be asked to step down if they are not planning on returning to active participation.
I would ask to be on this initial council by having the rule include "original contributors of the code" which would basically include Robert Kern and Pearu Peterson in addition to Chuck Harris (though Pearu's main contribution was numpy.distutils and f2py and he may not be interested any longer in the rest of it). Thanks, -Travis
Travis -- have you included all your email addresses in your GitHub profile? When I type git shortlog -ne, I see 2063 commits from your Continuum address that seem to be missing from the contributors page on github. Generally speaking, the git logs tend to be more reliable for these counts. On Wed, Sep 23, 2015 at 3:12 PM, Travis Oliphant <travis@continuum.io> wrote:
On Wed, Sep 23, 2015 at 3:12 PM, Travis Oliphant <travis@continuum.io> wrote:
I'm actually not even sure why this link is even here -- it's another thing that just got copied from Jupyter/IPython :-) It looks like the reason your commits are massively undercounted on that page is that github uses emails to match up commits to github accounts. Unfortunately it looks like almost all your old commits used <oliphant@enthought.com> for your email address, and apparently the @teoliphant github account doesn't have that listed as one of its emails. I guess that's probably unfixable on your end, and while git has a standard way to solve these issues (.mailmap -- that's why Stephan got the right answer from git shortlog), github is not clever enough to use it: https://github.com/holman/ama/issues/551#issuecomment-46129792 So it sounds like that page is basically unfixable, and if it's misleading then we should just delete it from the doc. We definitely don't want to diminish anyone's contributions! [snip]
I'll leave this discussion for the other thread that's specifically about the council membership, so as not to confuse matters.
I think it's important to have *some* kind of relatively objective/impartial criteria here, because if the rules force somebody to be the bad guy before anyone can get pruned, then the council will never get pruned, and over time we'll end up with an official council membership list that gets more and more out of touch with reality. Which is the kind of "social debt" we don't need IMO -- we already have enough technical debt to sort out :-). That doesn't mean it necessarily has to be these particular criteria though... probably the decision about what exactly they should be is another thing to defer to the other thread (or at least put on hold until we have some decision on the initial membership question). I'm feeling good -- it sounds like we're narrowing things down quite a bit now. -n -- Nathaniel J. Smith -- http://vorpus.org
On Thu, Sep 24, 2015 at 12:22 AM, Nathaniel Smith <njs@pobox.com> wrote:
I just posted a pull request version of the governance document: https://github.com/numpy/numpy/pull/6352 It's definitely still a draft, but now that we're making changes I wanted to get it into a form where people can at least see what those are :-). In interests of making that easy, the PR is structured as a ReST-converted version of the document originally posted, plus clearly-separated commits on top of that. The only non-trivial changes are: - I split the lists of people out into a separate document: https://github.com/njsmith/numpy/blob/governance/doc/source/dev/governance/p... (currently filled with the original seed council proposal for the reasons mentioned above) - I removed the link to the inaccurate github "contributors" page -n -- Nathaniel J. Smith -- http://vorpus.org
On Do, 2015-09-24 at 00:22 -0700, Nathaniel Smith wrote:
Well, I think we can at least learn one thing from this directly. I think we should include some relation to historic contributions into the document. I think what might be nice to do would be for example to also "seed" the emeritus list (if it is not too difficult). It is somewhat unrelated to governance, but since we want it to be a prominent document, it should not overshadow the past. - Sebastian <snip>
-n
[We also have to decide on the initial membership for the Council. While
On Wed, Sep 23, 2015 at 11:12 PM, Travis Oliphant <travis@continuum.io> wrote: the above text makes pains to distinguish between "committer" and "Council Member", in the past we've pretty much treated them as the same. So to keep things simple and deterministic, I propose that we seed the Council with everyone who has reviewed/merged a pull request since Jan 1, 2014, and move those who haven't used their commit bit in >1.5 years to the emeritus list. Based on the output of that list is interested/willing to serve).]
I would ask to be on this initial council by having the rule include
"original contributors of the code" which would basically include Robert Kern and Pearu Peterson in addition to Chuck Harris (though Pearu's main contribution was numpy.distutils and f2py and he may not be interested any longer in the rest of it). Were I to want such a thing, I would simply review and merge one of the many languishing PRs. I have the power to add myself to this list without asking for special exemptions to the general rule, merely a token bit of effort on my part. -- Robert Kern
participants (5)
-
Nathaniel Smith -
Robert Kern -
Sebastian Berg -
Stephan Hoyer -
Travis Oliphant