Hi everyone, There have been some wonderfully vigorous discussions over the past few months that have made it clear that we need some clarity about how decisions will be made in the NumPy community. When we were a smaller bunch of people it seemed easier to come to an agreement and things pretty much evolved based on (mostly) consensus and who was available to actually do the work. There is a need for a more clear structure so that we know how decisions will get made and so that code can move forward while paying attention to the current user-base. There has been a "steering committee" structure for SciPy in the past, and I have certainly been prone to lump both NumPy and SciPy together given that I have a strong interest in and have spent a great amount of time working on both projects. Others have also spent time on both projects. However, I think it is critical at this stage to clearly separate the projects and define a governing structure that is fair and agreeable for NumPy. SciPy has multiple modules and will probably need structure around each module independently. For now, I wanted to open up a discussion to see what people thought about NumPy's governance. My initial thoughts: * discussions happen as they do now on the mailing list * a small group of developers (5-11) constitute the "board" and major decisions are made by vote of that group (not just simple majority --- needs at least 2/3 +1 votes). * votes are +1/+0/-0/-1 * if a topic is difficult to resolve it is moved off the main list and discussed on a separate "board" mailing list --- these should be rare, but parts of the NA discussion would probably qualify * This board mailing list is "publically" viewable but only board members may post. * The board is renewed and adjusted each year --- based on nomination and 2/3 vote of the current board until board is at 11. * The chairman of the board is voted by a majority of the board and has veto power unless over-ridden by 3/4 of the board. * Petitions to remove people off the board can be made by 50+ independent reverse nominations (hopefully people will just withdraw if they are no longer active). All of these points are open for discussion. I just thought I would start the conversation. I will be much more active this next year with NumPy and will be very interested in the direction NumPy is taking. I'm hoping to discern by this conversation, who else is very interested in the direction of NumPy so that the first board can be formally constituted. Best regards, -Travis
Hi Travis, On Sat, Dec 3, 2011 at 6:18 PM, Travis Oliphant <teoliphant@gmail.com> wrote:
Hi everyone,
There have been some wonderfully vigorous discussions over the past few months that have made it clear that we need some clarity about how decisions will be made in the NumPy community.
When we were a smaller bunch of people it seemed easier to come to an agreement and things pretty much evolved based on (mostly) consensus and who was available to actually do the work.
There is a need for a more clear structure so that we know how decisions will get made and so that code can move forward while paying attention to the current user-base. There has been a "steering committee" structure for SciPy in the past, and I have certainly been prone to lump both NumPy and SciPy together given that I have a strong interest in and have spent a great amount of time working on both projects. Others have also spent time on both projects.
However, I think it is critical at this stage to clearly separate the projects and define a governing structure that is fair and agreeable for NumPy. SciPy has multiple modules and will probably need structure around each module independently. For now, I wanted to open up a discussion to see what people thought about NumPy's governance.
My initial thoughts:
* discussions happen as they do now on the mailing list * a small group of developers (5-11) constitute the "board" and major decisions are made by vote of that group (not just simple majority --- needs at least 2/3 +1 votes). * votes are +1/+0/-0/-1 * if a topic is difficult to resolve it is moved off the main list and discussed on a separate "board" mailing list --- these should be rare, but parts of the NA discussion would probably qualify * This board mailing list is "publically" viewable but only board members may post. * The board is renewed and adjusted each year --- based on nomination and 2/3 vote of the current board until board is at 11. * The chairman of the board is voted by a majority of the board and has veto power unless over-ridden by 3/4 of the board. * Petitions to remove people off the board can be made by 50+ independent reverse nominations (hopefully people will just withdraw if they are no longer active).
Thanks very much for starting this discussion. You have probably seen that my preference would be for all discussions to be public - in the sense that all can contribute. So, it seems reasonable to me to have 'board' as you describe, but that the board should vote on the same mailing list as the rest of the discussion. Having a separate mailing list for discussion makes the separation overt between those with a granted voice and those without, and I would hope for a structure which emphasized discsussion in an open forum. Put another way, what advantage would having a separate public mailing list have? How does this governance compare to that of - say - Linux or Python or Debian? My worry will be that it will be too tempting to terminate discussions and proceed to resolve by vote, when voting (as Karl Vogel describes) may still do harm. What will be the position - maybe I mean your position - on consensus as Nathaniel has described it? I feel the masked array discussion would have been more productive (an maybe shorter and more to the point) if there had been some rule-of-thumb that every effort is made to reach consensus before proceeding to implementation - or a vote. For example, in the masked array discussion, I would have liked to be able to say 'hold on, we have a rule that we try our best to reach consensus; I do not feel we have done that yet'. See you, Matthew I guess that the
I like the idea of trying to reach consensus first. The only point of having a board is to have someway to resolve issues should consensus not be reachable. Believe me, I'm not that excited about a separate mailing list. It would be great if we could resolve everything on a single list. -Travis On Dec 3, 2011, at 9:42 PM, Matthew Brett wrote:
Hi Travis,
On Sat, Dec 3, 2011 at 6:18 PM, Travis Oliphant <teoliphant@gmail.com> wrote:
Hi everyone,
There have been some wonderfully vigorous discussions over the past few months that have made it clear that we need some clarity about how decisions will be made in the NumPy community.
When we were a smaller bunch of people it seemed easier to come to an agreement and things pretty much evolved based on (mostly) consensus and who was available to actually do the work.
There is a need for a more clear structure so that we know how decisions will get made and so that code can move forward while paying attention to the current user-base. There has been a "steering committee" structure for SciPy in the past, and I have certainly been prone to lump both NumPy and SciPy together given that I have a strong interest in and have spent a great amount of time working on both projects. Others have also spent time on both projects.
However, I think it is critical at this stage to clearly separate the projects and define a governing structure that is fair and agreeable for NumPy. SciPy has multiple modules and will probably need structure around each module independently. For now, I wanted to open up a discussion to see what people thought about NumPy's governance.
My initial thoughts:
* discussions happen as they do now on the mailing list * a small group of developers (5-11) constitute the "board" and major decisions are made by vote of that group (not just simple majority --- needs at least 2/3 +1 votes). * votes are +1/+0/-0/-1 * if a topic is difficult to resolve it is moved off the main list and discussed on a separate "board" mailing list --- these should be rare, but parts of the NA discussion would probably qualify * This board mailing list is "publically" viewable but only board members may post. * The board is renewed and adjusted each year --- based on nomination and 2/3 vote of the current board until board is at 11. * The chairman of the board is voted by a majority of the board and has veto power unless over-ridden by 3/4 of the board. * Petitions to remove people off the board can be made by 50+ independent reverse nominations (hopefully people will just withdraw if they are no longer active).
Thanks very much for starting this discussion.
You have probably seen that my preference would be for all discussions to be public - in the sense that all can contribute. So, it seems reasonable to me to have 'board' as you describe, but that the board should vote on the same mailing list as the rest of the discussion. Having a separate mailing list for discussion makes the separation overt between those with a granted voice and those without, and I would hope for a structure which emphasized discsussion in an open forum.
Put another way, what advantage would having a separate public mailing list have?
How does this governance compare to that of - say - Linux or Python or Debian?
My worry will be that it will be too tempting to terminate discussions and proceed to resolve by vote, when voting (as Karl Vogel describes) may still do harm.
What will be the position - maybe I mean your position - on consensus as Nathaniel has described it? I feel the masked array discussion would have been more productive (an maybe shorter and more to the point) if there had been some rule-of-thumb that every effort is made to reach consensus before proceeding to implementation - or a vote.
For example, in the masked array discussion, I would have liked to be able to say 'hold on, we have a rule that we try our best to reach consensus; I do not feel we have done that yet'.
See you,
Matthew
I guess that the _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
--- Travis Oliphant Enthought, Inc. oliphant@enthought.com 1-512-536-1057 http://www.enthought.com
On Sat, Dec 3, 2011 at 7:18 PM, Travis Oliphant <teoliphant@gmail.com>wrote:
Hi everyone,
There have been some wonderfully vigorous discussions over the past few months that have made it clear that we need some clarity about how decisions will be made in the NumPy community.
When we were a smaller bunch of people it seemed easier to come to an agreement and things pretty much evolved based on (mostly) consensus and who was available to actually do the work.
There is a need for a more clear structure so that we know how decisions will get made and so that code can move forward while paying attention to the current user-base. There has been a "steering committee" structure for SciPy in the past, and I have certainly been prone to lump both NumPy and SciPy together given that I have a strong interest in and have spent a great amount of time working on both projects. Others have also spent time on both projects.
However, I think it is critical at this stage to clearly separate the projects and define a governing structure that is fair and agreeable for NumPy. SciPy has multiple modules and will probably need structure around each module independently. For now, I wanted to open up a discussion to see what people thought about NumPy's governance.
My initial thoughts:
* discussions happen as they do now on the mailing list * a small group of developers (5-11) constitute the "board" and major decisions are made by vote of that group (not just simple majority --- needs at least 2/3 +1 votes). * votes are +1/+0/-0/-1 * if a topic is difficult to resolve it is moved off the main list and discussed on a separate "board" mailing list --- these should be rare, but parts of the NA discussion would probably qualify * This board mailing list is "publically" viewable but only board members may post. * The board is renewed and adjusted each year --- based on nomination and 2/3 vote of the current board until board is at 11. * The chairman of the board is voted by a majority of the board and has veto power unless over-ridden by 3/4 of the board. * Petitions to remove people off the board can be made by 50+ independent reverse nominations (hopefully people will just withdraw if they are no longer active).
All of these points are open for discussion. I just thought I would start the conversation. I will be much more active this next year with NumPy and will be very interested in the direction NumPy is taking. I'm hoping to discern by this conversation, who else is very interested in the direction of NumPy so that the first board can be formally constituted.
If the purpose of the board is to resolve controversies, the 2/3 requirement is going to cause problems. The reason majority votes are usually used and that committees are set up with an odd number of members is that nothing gets resolved otherwise. Doing nothing is not a solution to missing consensus. Furthermore, at the current time, I don't think there are 5 active developers, let alone 11. With hard work you might scrape together two or three. Having 5 or 11 people making decisions for the two or three actually doing the work isn't going to go over well. I would propose a technical board of one or three people who can step in if an issue look like it needs outside intervention. And I would suggest at least one of the members be someone from the outside but familiar with the project, say someone like Fernando. The one member model is if we decide to go with a benevolent dictator. Note that for the smaller boards both the 2/3'rds and majority votes would be the same number of people ;) Chuck
On 12/4/2011 1:43 AM, Charles R Harris wrote:
I don't think there are 5 active developers, let alone 11. With hard work you might scrape together two or three. Having 5 or 11 people making decisions for the two or three actually doing the work isn't going to go over well.
Very true! But you might consider including on any board a developer or two from important projects that are very NumPy dependent. (E.g., Matplotlib.) One other thing: how about starting with a "board" of 3 and a rule that says any active developer can request to join, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority? (Fixing the board size in advance for a project we all hope will grow substantially seems odd.) fwiw, Alan Isaac
On Sun, Dec 4, 2011 at 6:16 AM, Alan G Isaac <alan.isaac@gmail.com> wrote:
On 12/4/2011 1:43 AM, Charles R Harris wrote:
I don't think there are 5 active developers, let alone 11. With hard work you might scrape together two or three. Having 5 or 11 people making decisions for the two or three actually doing the work isn't going to go over well.
Very true! But you might consider including on any board a developer or two from important projects that are very NumPy dependent. (E.g., Matplotlib.)
That's a good idea.
One other thing: how about starting with a "board" of 3 and a rule that says any active developer can request to join, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority? (Fixing the board size in advance for a project we all hope will grow substantially seems odd.)
I'm thinking of a board for resolving technical conflicts. Involving more people for larger discussions about direction and coordination with other projects would be the right thing to do. I think that can mostly be left informal at the moment but having an official get together at an event might be useful. Chuck
Great points. My initial suggestion of 5-11 was more about current board size rather than trying to fix it. I agree that having someone represent from major downstream projects would be a great thing. -Travis On Dec 4, 2011, at 7:16 AM, Alan G Isaac wrote:
On 12/4/2011 1:43 AM, Charles R Harris wrote:
I don't think there are 5 active developers, let alone 11. With hard work you might scrape together two or three. Having 5 or 11 people making decisions for the two or three actually doing the work isn't going to go over well.
Very true! But you might consider including on any board a developer or two from important projects that are very NumPy dependent. (E.g., Matplotlib.)
One other thing: how about starting with a "board" of 3 and a rule that says any active developer can request to join, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority? (Fixing the board size in advance for a project we all hope will grow substantially seems odd.)
fwiw, Alan Isaac
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
--- Travis Oliphant Enthought, Inc. oliphant@enthought.com 1-512-536-1057 http://www.enthought.com
I'm not sure I'm crazy about leaving final decision making for a board. A board may be a good way of carefully considering the issues, and it could make it's own recommendation (with a sufficient majority). But in the end I think one person needs to decide (and that decision may go against the board consensus, presumably only rarely). Why shouldn't that person be you? Perry On Dec 4, 2011, at 11:32 PM, Travis Oliphant wrote:
Great points. My initial suggestion of 5-11 was more about current board size rather than trying to fix it.
I agree that having someone represent from major downstream projects would be a great thing.
-Travis
On Dec 4, 2011, at 7:16 AM, Alan G Isaac wrote:
On 12/4/2011 1:43 AM, Charles R Harris wrote:
I don't think there are 5 active developers, let alone 11. With hard work you might scrape together two or three. Having 5 or 11 people making decisions for the two or three actually doing the work isn't going to go over well.
Very true! But you might consider including on any board a developer or two from important projects that are very NumPy dependent. (E.g., Matplotlib.)
One other thing: how about starting with a "board" of 3 and a rule that says any active developer can request to join, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority? (Fixing the board size in advance for a project we all hope will grow substantially seems odd.)
fwiw, Alan Isaac
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
--- Travis Oliphant Enthought, Inc. oliphant@enthought.com 1-512-536-1057 http://www.enthought.com
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 12/05/2011 06:22 AM, Perry Greenfield wrote:
I'm not sure I'm crazy about leaving final decision making for a board. A board may be a good way of carefully considering the issues, and it could make it's own recommendation (with a sufficient majority). But in the end I think one person needs to decide (and that decision may go against the board consensus, presumably only rarely).
Why shouldn't that person be you?
Perry
I have similar thoughts because I just do not see how a board would work especially given that anyone can be a 'core developer' because the distributed aspect removes that 'entry barrier'. I also think that there needs to be something formal like Linux Kernel Summit (see the excellent coverage by LWN.net; http://lwn.net/Articles/KernelSummit2011/). I know that people get together to talk at meetings or via invitation (http://blog.fperez.org/2011/05/austin-trip-ipython-at-tacc-and.html). This would provide a good opportunity to hash out concerns, introduce new features and identify community needs that cannot be adequately addressed via electronic communication. The datarray is a 'good' example of how this could work except that it has not been pushed upstream yet! (It would be a excellent example if it had been pushed upstream :-) hint, hint.) I also must disagree with statement of Travis that "discussions happen as they do now on the mailing list". This is simply not true because the mailing lists, tickets and pull requests are not connected so these have their own discussion threads. Sure there are some nice examples, Mark did tell us about this NA branch but the actual merge was still a surprise. So I think better communication of these such as emailing the list with a set 'public comment period' before requests are merged (longer periods for major changes). Bruce
On Dec 4, 2011, at 11:32 PM, Travis Oliphant wrote:
Great points. My initial suggestion of 5-11 was more about current board size rather than trying to fix it.
I agree that having someone represent from major downstream projects would be a great thing.
-Travis
On Dec 4, 2011, at 7:16 AM, Alan G Isaac wrote:
On 12/4/2011 1:43 AM, Charles R Harris wrote:
I don't think there are 5 active developers, let alone 11. With hard work you might scrape together two or three. Having 5 or 11 people making decisions for the two or three actually doing the work isn't going to go over well. Very true! But you might consider including on any board a developer or two from important projects that are very NumPy dependent. (E.g., Matplotlib.)
One other thing: how about starting with a "board" of 3 and a rule that says any active developer can request to join, that additions are determined by majority vote of the existing board, and that having the board both small and odd numbered is a priority? (Fixing the board size in advance for a project we all hope will grow substantially seems odd.)
fwiw, Alan Isaac
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Travis Oliphant Enthought, Inc. oliphant@enthought.com 1-512-536-1057 http://www.enthought.com
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Mon, Dec 5, 2011 at 4:22 AM, Perry Greenfield <perry@stsci.edu> wrote:
I'm not sure I'm crazy about leaving final decision making for a board. A board may be a good way of carefully considering the issues, and it could make it's own recommendation (with a sufficient majority). But in the end I think one person needs to decide (and that decision may go against the board consensus, presumably only rarely).
Why shouldn't that person be you?
I haven't contributed to NumPy directly. But I can offer my experience with SymPy. I agree with Perry. Having one person being in charge as the last call (project leader) works excellent in my experience. For SymPy, that person has been me, up until a year ago (when I realized that I am too busy to do a good job as a project leader), when I passed it to Aaron Meurer. We always try to reach consensus, and the project leader's main job is to encourage such discussion. When consensus cannot be reached, he needs to make the decision (that happened maybe once or twice in the last 5 years and it is very rare). There seems to be quite strong "community ownership" in SymPy (that was Stefan's objection). I think the reason being that in fact we probably have something like a "board of members", except that it is informal and it simply consists of people whose opinions the project leader highly values. And I think that it is very easy for anybody who gets involved with SymPy development to become trusted and thus his or her opinion will count. As such, for NumPy I think by default the project leader is Travis, who created it. He became busy in the last few years and so he could appoint a person, who will be the project leader. The list of possible people seems quite simple, I would choose somebody who is involved a lot with NumPy in the last 1 year (let's say): $ git shortlog -ns --since="1 year ago" | head 651 Mark Wiebe 137 Charles Harris 72 David Cournapeau 61 Ralf Gommers 52 rgommers 29 Pearu Peterson 17 Pauli Virtanen 11 Chris Jordan-Squire 11 Matthew Brett 10 Christopher L. Farrow So anybody from the top 5 or 10 people seems ok. This has to be a personal decision, and I don't know what the actual contribution and involvement (and personal ability to be a project leader) is of the above people, so that's why it should be done by Travis (possibly consulting with somebody who he trusts and who is involved). For SymPy, here is the list from the "1 year ago" when I passed the project leadership: $ git shortlog -ns --since="January 2010" --until "January 2011" | head 317 Øyvind Jensen 150 Mateusz Paprocki 93 Aaron Meurer 81 Addison Cugini 79 Brian E. Granger 64 Ronan Lamy 61 Matt Curry 58 Ondřej Čertík 36 Chris Smith 34 Christian Muise It's not exactly accurate, as some of the branches from 2010 were merged in 2011, but it gives you a picture. The above list doesn't tell you who the best person should be. I knew that Aaron would be the best choice, and I consulted it with several "core developers" to see what the "community" thinks, and everybody told me, that if I need to pass it on, Aaron would be the choice. Since this was the first time for me doing this, I simply stated, that Aaron is the project leader from now on. And in couple months we clarified it a little bit, that I am the "owner", in a sense that I own the domain and some servers and other things and I am ultimately responsible for the project (and I still have a say in non-coding related issues, like Google Summer of Code and such). For anything code related, Aaron has the last word, and I will not override it. The precise email is here: https://groups.google.com/d/topic/sympy/i-XD15syvqs/discussion You can compare it to today's list: $ git shortlog -ns --since="1 year ago" | head 805 Chris Smith 583 Mateusz Paprocki 508 Aaron Meurer 183 Ronan Lamy 150 Saptarshi Mandal 112 Tom Bachmann 101 Vladimir Perić 93 Gilbert Gede 91 Ondřej Čertík 89 Brian E. Granger So the activity has gone up after I stopped being the bottleneck, and after there was again a clear person, who is in charge and has time for it. Anyway, I just wanted to offer some experience that I gained with SymPy with this regard. As I said, I am not a NumPy developer and as such, this decision should be made by NumPy developers and Travis as the original project leader. I could see a familiar pattern here --- Travis has spent enormous time to develop NumPy and to build a community, and later became busy. This is exactly what happened to me with SymPy (when I was back in Prague, I spent months, every evening, many hours with sympy....). In fact, Travis once said at some lecture, that opensource is addictive. And not only that, also, if you develop (start) something, it really feels like it's yours. And then when I didn't have time and I knew I am not doing good job with SymPy, it was probably the hardest decision I had to make to pass the leadership on. Now, from retrospect, I should have done it much earlier and it is now obvious, that it was the right thing to do. But at that time, it was not obvious and I was very unsure what is going to happen. So anyway, good luck with any decision that you make. :) Ondrej
That was an extremely helpful and useful post. Thank you Ondrej for sharing it and taking the time to provide that insight. Travis -- Travis Oliphant (on a mobile) 512-826-7480 On Dec 29, 2011, at 12:51 AM, Ondrej Certik <ondrej@certik.cz> wrote:
On Mon, Dec 5, 2011 at 4:22 AM, Perry Greenfield <perry@stsci.edu> wrote:
I'm not sure I'm crazy about leaving final decision making for a board. A board may be a good way of carefully considering the issues, and it could make it's own recommendation (with a sufficient majority). But in the end I think one person needs to decide (and that decision may go against the board consensus, presumably only rarely).
Why shouldn't that person be you?
I haven't contributed to NumPy directly. But I can offer my experience with SymPy.
I agree with Perry. Having one person being in charge as the last call (project leader) works excellent in my experience. For SymPy, that person has been me, up until a year ago (when I realized that I am too busy to do a good job as a project leader), when I passed it to Aaron Meurer. We always try to reach consensus, and the project leader's main job is to encourage such discussion. When consensus cannot be reached, he needs to make the decision (that happened maybe once or twice in the last 5 years and it is very rare).
There seems to be quite strong "community ownership" in SymPy (that was Stefan's objection). I think the reason being that in fact we probably have something like a "board of members", except that it is informal and it simply consists of people whose opinions the project leader highly values. And I think that it is very easy for anybody who gets involved with SymPy development to become trusted and thus his or her opinion will count.
As such, for NumPy I think by default the project leader is Travis, who created it. He became busy in the last few years and so he could appoint a person, who will be the project leader.
The list of possible people seems quite simple, I would choose somebody who is involved a lot with NumPy in the last 1 year (let's say):
$ git shortlog -ns --since="1 year ago" | head 651 Mark Wiebe 137 Charles Harris 72 David Cournapeau 61 Ralf Gommers 52 rgommers 29 Pearu Peterson 17 Pauli Virtanen 11 Chris Jordan-Squire 11 Matthew Brett 10 Christopher L. Farrow
So anybody from the top 5 or 10 people seems ok. This has to be a personal decision, and I don't know what the actual contribution and involvement (and personal ability to be a project leader) is of the above people, so that's why it should be done by Travis (possibly consulting with somebody who he trusts and who is involved).
For SymPy, here is the list from the "1 year ago" when I passed the project leadership:
$ git shortlog -ns --since="January 2010" --until "January 2011" | head 317 Øyvind Jensen 150 Mateusz Paprocki 93 Aaron Meurer 81 Addison Cugini 79 Brian E. Granger 64 Ronan Lamy 61 Matt Curry 58 Ondřej Čertík 36 Chris Smith 34 Christian Muise
It's not exactly accurate, as some of the branches from 2010 were merged in 2011, but it gives you a picture. The above list doesn't tell you who the best person should be. I knew that Aaron would be the best choice, and I consulted it with several "core developers" to see what the "community" thinks, and everybody told me, that if I need to pass it on, Aaron would be the choice.
Since this was the first time for me doing this, I simply stated, that Aaron is the project leader from now on. And in couple months we clarified it a little bit, that I am the "owner", in a sense that I own the domain and some servers and other things and I am ultimately responsible for the project (and I still have a say in non-coding related issues, like Google Summer of Code and such). For anything code related, Aaron has the last word, and I will not override it. The precise email is here:
https://groups.google.com/d/topic/sympy/i-XD15syvqs/discussion
You can compare it to today's list:
$ git shortlog -ns --since="1 year ago" | head 805 Chris Smith 583 Mateusz Paprocki 508 Aaron Meurer 183 Ronan Lamy 150 Saptarshi Mandal 112 Tom Bachmann 101 Vladimir Perić 93 Gilbert Gede 91 Ondřej Čertík 89 Brian E. Granger
So the activity has gone up after I stopped being the bottleneck, and after there was again a clear person, who is in charge and has time for it.
Anyway, I just wanted to offer some experience that I gained with SymPy with this regard. As I said, I am not a NumPy developer and as such, this decision should be made by NumPy developers and Travis as the original project leader.
I could see a familiar pattern here --- Travis has spent enormous time to develop NumPy and to build a community, and later became busy. This is exactly what happened to me with SymPy (when I was back in Prague, I spent months, every evening, many hours with sympy....). In fact, Travis once said at some lecture, that opensource is addictive. And not only that, also, if you develop (start) something, it really feels like it's yours. And then when I didn't have time and I knew I am not doing good job with SymPy, it was probably the hardest decision I had to make to pass the leadership on. Now, from retrospect, I should have done it much earlier and it is now obvious, that it was the right thing to do. But at that time, it was not obvious and I was very unsure what is going to happen.
So anyway, good luck with any decision that you make. :)
Ondrej _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Sat, Dec 3, 2011 at 6:18 PM, Travis Oliphant <teoliphant@gmail.com>wrote:
Hi everyone,
There have been some wonderfully vigorous discussions over the past few months that have made it clear that we need some clarity about how decisions will be made in the NumPy community.
When we were a smaller bunch of people it seemed easier to come to an agreement and things pretty much evolved based on (mostly) consensus and who was available to actually do the work.
There is a need for a more clear structure so that we know how decisions will get made and so that code can move forward while paying attention to the current user-base. There has been a "steering committee" structure for SciPy in the past, and I have certainly been prone to lump both NumPy and SciPy together given that I have a strong interest in and have spent a great amount of time working on both projects. Others have also spent time on both projects.
However, I think it is critical at this stage to clearly separate the projects and define a governing structure that is fair and agreeable for NumPy. SciPy has multiple modules and will probably need structure around each module independently. For now, I wanted to open up a discussion to see what people thought about NumPy's governance.
My initial thoughts:
* discussions happen as they do now on the mailing list * a small group of developers (5-11) constitute the "board" and major decisions are made by vote of that group (not just simple majority --- needs at least 2/3 +1 votes). * votes are +1/+0/-0/-1 * if a topic is difficult to resolve it is moved off the main list and discussed on a separate "board" mailing list --- these should be rare, but parts of the NA discussion would probably qualify * This board mailing list is "publically" viewable but only board members may post. * The board is renewed and adjusted each year --- based on nomination and 2/3 vote of the current board until board is at 11. * The chairman of the board is voted by a majority of the board and has veto power unless over-ridden by 3/4 of the board. * Petitions to remove people off the board can be made by 50+ independent reverse nominations (hopefully people will just withdraw if they are no longer active).
All of these points are open for discussion. I just thought I would start the conversation. I will be much more active this next year with NumPy and will be very interested in the direction NumPy is taking. I'm hoping to discern by this conversation, who else is very interested in the direction of NumPy so that the first board can be formally constituted.
I'm definitely in support of something along these lines. My experience entering NumPy development was that the development process, coding standards, and other aspects of the process are not very well specified, and people have many differing ideas about what has already been agreed upon. I would recommend that fixing this state of affairs be placed high on the agenda of the board, with the goal of making it easier to attract new developers. A few people have proposed the BDFL approach, as in CPython development. In practice, I believe Guido has done very well in the role because he only uses the power as a last resort. Even if NumPy adopts a similar approach, having a board along the lines Travis proposes would still be a good thing, and having a BDFL would just mean that there's someone who could override the will of the board and make an entirely different choice. It may be worth considering how the governance structure is related to the different levels of the NumPy codebase. There is a (very) small group of people who have contributed significant amounts of C code, a larger group of people who have contributed significant amounts of Python code, many people who have contributed small C and/or Python patches, and a large number of people who contribute bug reports, email list comments, etc. It may be worth designing the board taking into account these different groups of developers and users. -Mark
Best regards,
-Travis
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Mon, Dec 5, 2011 at 12:06 PM, Mark Wiebe <mwwiebe@gmail.com> wrote:
On Sat, Dec 3, 2011 at 6:18 PM, Travis Oliphant <teoliphant@gmail.com>wrote:
Hi everyone,
There have been some wonderfully vigorous discussions over the past few months that have made it clear that we need some clarity about how decisions will be made in the NumPy community.
When we were a smaller bunch of people it seemed easier to come to an agreement and things pretty much evolved based on (mostly) consensus and who was available to actually do the work.
There is a need for a more clear structure so that we know how decisions will get made and so that code can move forward while paying attention to the current user-base. There has been a "steering committee" structure for SciPy in the past, and I have certainly been prone to lump both NumPy and SciPy together given that I have a strong interest in and have spent a great amount of time working on both projects. Others have also spent time on both projects.
However, I think it is critical at this stage to clearly separate the projects and define a governing structure that is fair and agreeable for NumPy. SciPy has multiple modules and will probably need structure around each module independently. For now, I wanted to open up a discussion to see what people thought about NumPy's governance.
My initial thoughts:
* discussions happen as they do now on the mailing list * a small group of developers (5-11) constitute the "board" and major decisions are made by vote of that group (not just simple majority --- needs at least 2/3 +1 votes). * votes are +1/+0/-0/-1 * if a topic is difficult to resolve it is moved off the main list and discussed on a separate "board" mailing list --- these should be rare, but parts of the NA discussion would probably qualify * This board mailing list is "publically" viewable but only board members may post. * The board is renewed and adjusted each year --- based on nomination and 2/3 vote of the current board until board is at 11. * The chairman of the board is voted by a majority of the board and has veto power unless over-ridden by 3/4 of the board. * Petitions to remove people off the board can be made by 50+ independent reverse nominations (hopefully people will just withdraw if they are no longer active).
All of these points are open for discussion. I just thought I would start the conversation. I will be much more active this next year with NumPy and will be very interested in the direction NumPy is taking. I'm hoping to discern by this conversation, who else is very interested in the direction of NumPy so that the first board can be formally constituted.
I'm definitely in support of something along these lines. My experience entering NumPy development was that the development process, coding standards, and other aspects of the process are not very well specified, and people have many differing ideas about what has already been agreed upon. I would recommend that fixing this state of affairs be placed high on the agenda of the board, with the goal of making it easier to attract new developers.
A few people have proposed the BDFL approach, as in CPython development. In practice, I believe Guido has done very well in the role because he only uses the power as a last resort. Even if NumPy adopts a similar approach, having a board along the lines Travis proposes would still be a good thing, and having a BDFL would just mean that there's someone who could override the will of the board and make an entirely different choice.
It may be worth considering how the governance structure is related to the different levels of the NumPy codebase. There is a (very) small group of people who have contributed significant amounts of C code, a larger group of people who have contributed significant amounts of Python code, many people who have contributed small C and/or Python patches, and a large number of people who contribute bug reports, email list comments, etc. It may be worth designing the board taking into account these different groups of developers and users.
-Mark
Best regards,
-Travis
Just some thoughts I have from this discussion.
1. I think that we need to encourage and entice more NumPy developers/contributors. Having a board of only a few core developers puts us right back in the same boat we were in during the whole NA discussion, only more codified. Increasing the size of the board with more core developers would diversify thought and counter-act "group-think". I think that this problem needs to be solved before anything else. Travis, I am currently in the process of getting hired by a company that has really embraced the Python/SciPy software stack, but unfortunately has not fully embraced OSS community development. I wonder if Enthought could produce (or maybe already has) teaching material showing other companies how to incorporate community development into their business models (in particular development of the SciPy stack)? Maybe we can get some more commitments of resources from other companies? 2. Nothing against Travis, but I am somewhat wary of declaring Travis as a BDFL while he is the head of Enthought. I think Travis has done an excellent job of respecting the hierarchy of development (where EPD is downstream of the OSS SciPy stack). Having Travis as BDFL might, IMHO, create possible conflicts of interest in the future. Again, I am not saying that Travis would act against the interest of the SciPy stack, but rather, I would like to avoid putting Travis in a position where such decisions could become tempting. 3. I definitely +1 the idea of including representatives of other projects on the board. Each project could have their own process for selecting a member to represent them. 4. I agree with Charles that a separate list would probably be detrimental. I also agree with Bruce that since the move to github, we have had things become fragmented, and I think that needs to be re-unified. matplotlib is experiencing the same problem, and I have to wonder if some other groups have already come up with a solution to this. Maybe make a psuedo github account with the mailing list address as its email address? Just my 2 cents for now. Ben Root
On Mon, Dec 5, 2011 at 12:43 PM, Benjamin Root <ben.root@ou.edu> wrote:
On Mon, Dec 5, 2011 at 12:06 PM, Mark Wiebe <mwwiebe@gmail.com> wrote:
On Sat, Dec 3, 2011 at 6:18 PM, Travis Oliphant <teoliphant@gmail.com>wrote:
Hi everyone,
There have been some wonderfully vigorous discussions over the past few months that have made it clear that we need some clarity about how decisions will be made in the NumPy community.
When we were a smaller bunch of people it seemed easier to come to an agreement and things pretty much evolved based on (mostly) consensus and who was available to actually do the work.
There is a need for a more clear structure so that we know how decisions will get made and so that code can move forward while paying attention to the current user-base. There has been a "steering committee" structure for SciPy in the past, and I have certainly been prone to lump both NumPy and SciPy together given that I have a strong interest in and have spent a great amount of time working on both projects. Others have also spent time on both projects.
However, I think it is critical at this stage to clearly separate the projects and define a governing structure that is fair and agreeable for NumPy. SciPy has multiple modules and will probably need structure around each module independently. For now, I wanted to open up a discussion to see what people thought about NumPy's governance.
My initial thoughts:
* discussions happen as they do now on the mailing list * a small group of developers (5-11) constitute the "board" and major decisions are made by vote of that group (not just simple majority --- needs at least 2/3 +1 votes). * votes are +1/+0/-0/-1 * if a topic is difficult to resolve it is moved off the main list and discussed on a separate "board" mailing list --- these should be rare, but parts of the NA discussion would probably qualify * This board mailing list is "publically" viewable but only board members may post. * The board is renewed and adjusted each year --- based on nomination and 2/3 vote of the current board until board is at 11. * The chairman of the board is voted by a majority of the board and has veto power unless over-ridden by 3/4 of the board. * Petitions to remove people off the board can be made by 50+ independent reverse nominations (hopefully people will just withdraw if they are no longer active).
All of these points are open for discussion. I just thought I would start the conversation. I will be much more active this next year with NumPy and will be very interested in the direction NumPy is taking. I'm hoping to discern by this conversation, who else is very interested in the direction of NumPy so that the first board can be formally constituted.
I'm definitely in support of something along these lines. My experience entering NumPy development was that the development process, coding standards, and other aspects of the process are not very well specified, and people have many differing ideas about what has already been agreed upon. I would recommend that fixing this state of affairs be placed high on the agenda of the board, with the goal of making it easier to attract new developers.
A few people have proposed the BDFL approach, as in CPython development. In practice, I believe Guido has done very well in the role because he only uses the power as a last resort. Even if NumPy adopts a similar approach, having a board along the lines Travis proposes would still be a good thing, and having a BDFL would just mean that there's someone who could override the will of the board and make an entirely different choice.
It may be worth considering how the governance structure is related to the different levels of the NumPy codebase. There is a (very) small group of people who have contributed significant amounts of C code, a larger group of people who have contributed significant amounts of Python code, many people who have contributed small C and/or Python patches, and a large number of people who contribute bug reports, email list comments, etc. It may be worth designing the board taking into account these different groups of developers and users.
-Mark
Best regards,
-Travis
Just some thoughts I have from this discussion.
1. I think that we need to encourage and entice more NumPy developers/contributors. Having a board of only a few core developers puts us right back in the same boat we were in during the whole NA discussion, only more codified. Increasing the size of the board with more core developers would diversify thought and counter-act "group-think". I think that this problem needs to be solved before anything else.
Well, that's a tough one. Numpy development tends to attract folks with spare time, i.e., students*, and those with an itch to scratch. Itched scratched, degree obtained, they go back to their primary interest or on to jobs and the rest of life. So developers come and go and a board needs to be pretty flexible to reflect that. We can't wave around wads of cash or stock options to attract new people. Mark does a good job of pointing out some of the barriers to entry, and we could try to lower those. Indeed, things are much better than they were but more can be done. <snip> Chuck * I expect students think they are overworked, and sometimes they are, but they also tend to be young and energetic with flexible schedules and few outside commitments.
On Mon, Dec 5, 2011 at 12:10 PM, Charles R Harris <charlesr.harris@gmail.com> wrote:
Well, that's a tough one. Numpy development tends to attract folks with spare time, i.e., students*, and those with an itch to scratch. Itched scratched, degree obtained, they go back to their primary interest or on to jobs and the rest of life.
NumPy does seem to be different in this regard, in that many of the developers stick around (even if they're not active on the code any longer), think about potential issues and new directions, take part in discussions, teach at conferences, organise workshops, write, etc. I agree with Matthew that using a board should be a last resort, and mildly disagree with Perry that it would be better to have a single person make the final call. The advantage of a benevolent dictator is that you have a coherent driving vision, but at the cost of sacrificing community ownership. As for barriers to entry, improving the the nature of discourse on the mailing list (when it comes to thorny issues) would be good. Technical barriers are not that hard to breach for our community; setting the right social atmosphere is crucial. Regards Stéfan
Hi, 2011/12/5 Stéfan van der Walt <stefan@sun.ac.za>:
As for barriers to entry, improving the the nature of discourse on the mailing list (when it comes to thorny issues) would be good. Technical barriers are not that hard to breach for our community; setting the right social atmosphere is crucial.
I'm just about to get on a plane and am going to be out of internet range for a while, so, in the spirit of constructive discussion: In the spirit of use-cases: Would it be fair to say that the two contentious recent discussions have been: The numpy ABI breakage, 2.0 vs 1.5.1 discussion The masked array discussion(s) ? What did we do wrong or right in each of these two discussions? What could we have done better? What process would help us to do better? Travis - for your board-only-post mailing list - my feeling is that this is going in the wrong direction. The effect of the board-only mailing list is to explicitly remove non-qualified people from the discussion. This will make it more explicit that the substantial decisions will be make by a few important people. Do you (Travis - or Mark?) think that, if this had happened earlier in the masked array discussion, it would have been less contentious, or had more substantial content? My instinct would be the reverse, and the best solution would have been to pause and commit to beating out the issues and getting agreement. See you, Matthew
On Mon, Dec 5, 2011 at 11:32 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
2011/12/5 Stéfan van der Walt <stefan@sun.ac.za>:
As for barriers to entry, improving the the nature of discourse on the mailing list (when it comes to thorny issues) would be good. Technical barriers are not that hard to breach for our community; setting the right social atmosphere is crucial.
I'm just about to get on a plane and am going to be out of internet range for a while, so, in the spirit of constructive discussion:
In the spirit of use-cases:
Would it be fair to say that the two contentious recent discussions have been:
The numpy ABI breakage, 2.0 vs 1.5.1 discussion The masked array discussion(s) ?
What did we do wrong or right in each of these two discussions? What could we have done better? What process would help us to do better?
Travis - for your board-only-post mailing list - my feeling is that this is going in the wrong direction. The effect of the board-only mailing list is to explicitly remove non-qualified people from the discussion. This will make it more explicit that the substantial decisions will be make by a few important people. Do you (Travis - or Mark?) think that, if this had happened earlier in the masked array discussion, it would have been less contentious, or had more substantial content? My instinct would be the reverse, and the best solution would have been to pause and commit to beating out the issues and getting agreement.
See you,
Matthew
I would also like to know the long term model for development since some of issues a direct results of that model. At the moment we seem stuck in the old svn model as we have a release that essentially splits from the current development branch where key developers just merge into without any discussion. This 'old svn' view did create some discussion regarding the NA object including the pull request. But we lacked the step about moving it into the current developmental branch. So at times that seemed to add 'insult to injury' (http://idioms.thefreedictionary.com/add+insult+to+injury) which tended to decrease some of the interesting ideas expressed. So perhaps we could find a model that allows real bug fixes (ie have a valid ticket or maximum of 5 lines of changed code) to go the current developmental branch and enhancements come through as some other process that involves community discussions? Bruce
On Sat, Dec 10, 2011 at 8:19 AM, Bruce Southey <bsouthey@gmail.com> wrote:
On Mon, Dec 5, 2011 at 11:32 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
2011/12/5 Stéfan van der Walt <stefan@sun.ac.za>:
As for barriers to entry, improving the the nature of discourse on the mailing list (when it comes to thorny issues) would be good. Technical barriers are not that hard to breach for our community; setting the right social atmosphere is crucial.
I'm just about to get on a plane and am going to be out of internet range for a while, so, in the spirit of constructive discussion:
In the spirit of use-cases:
Would it be fair to say that the two contentious recent discussions have been:
The numpy ABI breakage, 2.0 vs 1.5.1 discussion The masked array discussion(s) ?
What did we do wrong or right in each of these two discussions? What could we have done better? What process would help us to do better?
Travis - for your board-only-post mailing list - my feeling is that this is going in the wrong direction. The effect of the board-only mailing list is to explicitly remove non-qualified people from the discussion. This will make it more explicit that the substantial decisions will be make by a few important people. Do you (Travis - or Mark?) think that, if this had happened earlier in the masked array discussion, it would have been less contentious, or had more substantial content? My instinct would be the reverse, and the best solution would have been to pause and commit to beating out the issues and getting agreement.
See you,
Matthew
I would also like to know the long term model for development since some of issues a direct results of that model. At the moment we seem stuck in the old svn model as we have a release that essentially splits from the current development branch where key developers just merge into without any discussion. This 'old svn' view did create some discussion regarding the NA object including the pull request. But we lacked the step about moving it into the current developmental branch. So at times that seemed to add 'insult to injury' (http://idioms.thefreedictionary.com/add+insult+to+injury) which tended to decrease some of the interesting ideas expressed.
So perhaps we could find a model that allows real bug fixes (ie have a valid ticket or maximum of 5 lines of changed code) to go the current developmental branch and enhancements come through as some other process that involves community discussions?
I think the rule should be that *anyone* seriously interested in what is happening in numpy development should be watching the pull requests. The complementary rule is that all commits go in through a pull request. There isn't much traffic on the pull requests and code/functionality review at that point is both useful and desirable. Large topics do tend to turn up on the list, the NA work, for example. But github makes it easy to follow what is going on and folks should take advantage of it. Chuck
On Sat, Dec 10, 2011 at 11:07 AM, Charles R Harris <charlesr.harris@gmail.com> wrote:
I think the rule should be that *anyone* seriously interested in what is happening in numpy development should be watching the pull requests.
It's good to encourage that, but in the end big changes should always be discussed on the mailing list, to make sure everyone is on board. Stéfan
Travis Oliphant wrote:
My initial thoughts: I don't have a horse in this race, but I do suggest people read Karl Fogel's book before too much designing of governance structure: http://producingoss.com/ (alas, it's not short, but it's a fairly easy read and you can get convenient dead-tree or ebook editions.)
-- Jonathan Niehof ISR-3 Space Data Systems Los Alamos National Laboratory MS-D466 Los Alamos, NM 87545 Phone: 505-667-9595 email: jniehof@lanl.gov Correspondence / Technical data or Software Publicly Available
participants (12)
-
Alan G Isaac -
Benjamin Root -
Bruce Southey -
Charles R Harris -
Jonathan T. Niehof -
Mark Wiebe -
Matthew Brett -
Ondrej Certik -
Perry Greenfield -
Stéfan van der Walt -
Travis Oliphant -
Travis Oliphant