Future of numpy (was: DARPA funding for Blaze and passing the NumPy torch)
Hi, [Travis wrote...]
My strong suggestion is that development discussions of the project continue on this list with consensus among the active participants being the goal for development. I don't think 100% consensus is a rigid requirement --- but certainly a super-majority should be the goal, and serious changes should not be made with out a clear consensus. I would pay special attention to under-represented people (users with intense usage of NumPy but small voices on this list). There are many of them. If you push me for specifics then at this point in NumPy's history, I would say that if Chuck, Nathaniel, and Ralf agree on a course of action, it will likely be a good thing for the project. I suspect that even if only 2 of the 3 agree at one time it might still be a good thing (but I would expect more detail and discussion). There are others whose opinion should be sought as well: Ondrej Certik, Perry Greenfield, Robert Kern, David Cournapeau, Francesc Alted, and Mark Wiebe to name a few. For some questions, I might even seek input from people like Konrad Hinsen and Paul Dubois --- if they have time to give it. I will still be willing to offer my view from time to time and if I am asked.
Thank you for starting this discussion. I am more or less offline at the moment in Cuba and flying, but I hope very much this will be an opportunity for a good discussion on the best way forward for numpy. Travis - I think you are suggesting that there should be no one person in charge of numpy, and I think this is very unlikely to work well. Perhaps there are good examples of well-led projects where there is not a clear leader, but I can't think of any myself at the moment. My worry would be that, without a clear leader, it will be unclear how decisions are made, and that will make it very hard to take strategic decisions. I would like to humbly suggest the following in the hope that it spurs discussion. As first pass, Ralf, Chuck and Nathaniel decide on a core group of people that will form the succession committee. Maybe this could be the list of people you listed above. Ralf, Chuck and Nathaniel then ask for people to wish to propose themselves as the leader of numpy. Anyone proposing themselves to lead numpy would remove themselves from the succession committee. The proposed leaders of numpy write a short manifesto saying why they are the right choice for the job, and what they intend to do if elected. The manifestos and issues arising are discussed in public on the mailing list - the equivalent of an online presidential debate. In due course - say after 2 weeks or after the discussion seems to be dying out - the succession committee votes on the leader. I propose that these votes should be public, but I can see the opposite argument. How does that sound? Best, Matthew
On Thu, Dec 20, 2012 at 11:46 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Travis - I think you are suggesting that there should be no one person in charge of numpy, and I think this is very unlikely to work well. Perhaps there are good examples of well-led projects where there is not a clear leader, but I can't think of any myself at the moment. My worry would be that, without a clear leader, it will be unclear how decisions are made, and that will make it very hard to take strategic decisions.
Curious; my feeling is the opposite, that among mature and successful FOSS projects, having a clear leader is the uncommon case. GCC doesn't, Glibc not only has no leader but they recently decided to get rid of their formal steering committee, I'm pretty sure git doesn't, Apache certainly doesn't, Samba doesn't really, etc. As usual Karl Fogel has sensible comments on this: http://producingoss.com/en/consensus-democracy.html In practice the main job of a successful FOSS leader is to refuse to make decisions, nudge people to work things out, and then if they refuse to work things out tell them to go away until they do: https://lwn.net/Articles/105375/ and what actually gives people influence in a project is the respect of the other members. The former stuff is stuff anyone can do, and the latter isn't something you can confer or take away with a vote. Nor do we necessarily have a great track record for executive decisions actually working things out. -n
On Dec 20, 2012, at 7:39 PM, Nathaniel Smith wrote:
On Thu, Dec 20, 2012 at 11:46 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Travis - I think you are suggesting that there should be no one person in charge of numpy, and I think this is very unlikely to work well. Perhaps there are good examples of well-led projects where there is not a clear leader, but I can't think of any myself at the moment. My worry would be that, without a clear leader, it will be unclear how decisions are made, and that will make it very hard to take strategic decisions.
Curious; my feeling is the opposite, that among mature and successful FOSS projects, having a clear leader is the uncommon case. GCC doesn't, Glibc not only has no leader but they recently decided to get rid of their formal steering committee, I'm pretty sure git doesn't, Apache certainly doesn't, Samba doesn't really, etc. As usual Karl Fogel has sensible comments on this: http://producingoss.com/en/consensus-democracy.html
In practice the main job of a successful FOSS leader is to refuse to make decisions, nudge people to work things out, and then if they refuse to work things out tell them to go away until they do: https://lwn.net/Articles/105375/ and what actually gives people influence in a project is the respect of the other members. The former stuff is stuff anyone can do, and the latter isn't something you can confer or take away with a vote.
I will strongly voice my opinion that NumPy does not need an official single "leader". What it needs are committed, experienced, service-oriented developers and users who are willing to express their concerns and requests because they are used to being treated well. It also needs new developers who are willing to dive into code, contribute to discussions, tackle issues, make pull requests, and review pull requests. As people do this regularly, the leaders of the project will emerge as they have done in the past. Even though I called out three people explicitly --- there are many more contributors to NumPy whose voices deserve attention. But, you don't need me to point out the obvious to what the Github record shows about who is shepherding NumPy these days. But, the Github record is not the only one that matters. I would love to see NumPy developers continue to pay attention to and deeply respect the users (especially of downstream projects that depend on NumPy). I plan to continue using NumPy myself and plan to continue to encourage others around me to contribute patches, fixes and features. Obviously, there are people who have rights to merge pull-requests to the repository. But, this group seems always open to new, willing help. From a practical matter, this group is the head development group of the official NumPy fork. I believe this group will continue to be open enough to new, motivated contributors which will allow it to grow to the degree that such developers are available.
Nor do we necessarily have a great track record for executive decisions actually working things out.
-n _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi, On Fri, Dec 21, 2012 at 12:14 AM, Travis Oliphant <travis@continuum.io> wrote:
On Dec 20, 2012, at 7:39 PM, Nathaniel Smith wrote:
On Thu, Dec 20, 2012 at 11:46 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Travis - I think you are suggesting that there should be no one person in charge of numpy, and I think this is very unlikely to work well. Perhaps there are good examples of well-led projects where there is not a clear leader, but I can't think of any myself at the moment. My worry would be that, without a clear leader, it will be unclear how decisions are made, and that will make it very hard to take strategic decisions.
Curious; my feeling is the opposite, that among mature and successful FOSS projects, having a clear leader is the uncommon case. GCC doesn't, Glibc not only has no leader but they recently decided to get rid of their formal steering committee, I'm pretty sure git doesn't, Apache certainly doesn't, Samba doesn't really, etc. As usual Karl Fogel has sensible comments on this: http://producingoss.com/en/consensus-democracy.html
In practice the main job of a successful FOSS leader is to refuse to make decisions, nudge people to work things out, and then if they refuse to work things out tell them to go away until they do: https://lwn.net/Articles/105375/ and what actually gives people influence in a project is the respect of the other members. The former stuff is stuff anyone can do, and the latter isn't something you can confer or take away with a vote.
I will strongly voice my opinion that NumPy does not need an official single "leader".
I am sorry, I have a feeling this question might be unwelcome - but I think it's reasonable to say that having three people in joint charge is an unusual choice. I suppose it has various risks and advantages. Would you mind saying a little bit about why you chose this option instead of the more common one of having a single lead? What problems do you think might arise? How can they be detected and avoided? Thanks a lot, Matthew
Hi, On Thu, Dec 20, 2012 at 5:39 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Thu, Dec 20, 2012 at 11:46 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Travis - I think you are suggesting that there should be no one person in charge of numpy, and I think this is very unlikely to work well. Perhaps there are good examples of well-led projects where there is not a clear leader, but I can't think of any myself at the moment. My worry would be that, without a clear leader, it will be unclear how decisions are made, and that will make it very hard to take strategic decisions.
Curious; my feeling is the opposite, that among mature and successful FOSS projects, having a clear leader is the uncommon case. GCC doesn't, Glibc not only has no leader but they recently decided to get rid of their formal steering committee, I'm pretty sure git doesn't, Apache certainly doesn't, Samba doesn't really, etc. As usual Karl Fogel has sensible comments on this: http://producingoss.com/en/consensus-democracy.html
Ah yes - that is curious. My - er - speculation was based on: Numpy - Travis golden age in which we still bask Sympy - Ondrej, then Aaron - evolving into group decision making AFAICT IPython - Fernando, evolving into group decision making, AFAICT Cython - Robert Bradshaw - evolving into ... - you get the idea. and then reading about businesses particularly Good to Great, Built to Last, the disaster at HP when they didn't take care about succession. In general, that reading gave me the impression that successful organizations take enormous care about succession. I can't think of any case in the business literature I've read where a successful leader handed over to a group of three.
In practice the main job of a successful FOSS leader is to refuse to make decisions, nudge people to work things out, and then if they refuse to work things out tell them to go away until they do: https://lwn.net/Articles/105375/ and what actually gives people influence in a project is the respect of the other members. The former stuff is stuff anyone can do, and the latter isn't something you can confer or take away with a vote.
Right. My impression is - I'm happy to be corrected with better information - that the leader of a to-be-successful organization is very good at encouraging a spirit of free and vigorous debate, strong opinion, and reasoned decisions - and that may be the main gift they give to the organization. At that point, usually under that leader's supervision, the decision making starts diffusing over the group, as they learn to discuss and make decisions together. As I was teaching my niece and nephew to say to their parents in the car - Daddy - are we there yet? If we are not already there, how are we going to get there?
Nor do we necessarily have a great track record for executive decisions actually working things out.
No, I agree, the right leader will help form the group well for making good group decisions. I think. In the mean-time - now that there is a change - could I ask - where do you three see Numpy going in the next five years? What do you see as the challenges to solve? What are the big risks? What are the big possibilities? Cheers, Matthew
On 12/22/2012 06:36 PM, Matthew Brett wrote:
Hi,
On Thu, Dec 20, 2012 at 5:39 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Thu, Dec 20, 2012 at 11:46 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Travis - I think you are suggesting that there should be no one person in charge of numpy, and I think this is very unlikely to work well. Perhaps there are good examples of well-led projects where there is not a clear leader, but I can't think of any myself at the moment. My worry would be that, without a clear leader, it will be unclear how decisions are made, and that will make it very hard to take strategic decisions.
Curious; my feeling is the opposite, that among mature and successful FOSS projects, having a clear leader is the uncommon case. GCC doesn't, Glibc not only has no leader but they recently decided to get rid of their formal steering committee, I'm pretty sure git doesn't, Apache certainly doesn't, Samba doesn't really, etc. As usual Karl Fogel has sensible comments on this: http://producingoss.com/en/consensus-democracy.html
Ah yes - that is curious. My - er - speculation was based on:
Numpy - Travis golden age in which we still bask Sympy - Ondrej, then Aaron - evolving into group decision making AFAICT IPython - Fernando, evolving into group decision making, AFAICT Cython - Robert Bradshaw - evolving into ... - you get the idea.
I don't really want to prolong this thread, but I feel like I should correct a factual error. Cython started with Robert Bradshaw (and other Sage members) and Stefan Behnel exchanging patches on top of Pyrex; there was definitely no leader at that point. Then I came along; there was no leader at that point either (but I was aware that the two others had a longer track record of course). Robert Bradshaw was declared leader in order to break the tie when I and Stefan Behnel had argued for a 100-post long thread and could not reach a conclusion. And at least in this case, we were able to settle on a leadership structure then, when we needed it, and didn't regret not doing it earlier. Dag Sverre
and then reading about businesses particularly Good to Great, Built to Last, the disaster at HP when they didn't take care about succession. In general, that reading gave me the impression that successful organizations take enormous care about succession. I can't think of any case in the business literature I've read where a successful leader handed over to a group of three.
In practice the main job of a successful FOSS leader is to refuse to make decisions, nudge people to work things out, and then if they refuse to work things out tell them to go away until they do: https://lwn.net/Articles/105375/ and what actually gives people influence in a project is the respect of the other members. The former stuff is stuff anyone can do, and the latter isn't something you can confer or take away with a vote.
Right. My impression is - I'm happy to be corrected with better information - that the leader of a to-be-successful organization is very good at encouraging a spirit of free and vigorous debate, strong opinion, and reasoned decisions - and that may be the main gift they give to the organization. At that point, usually under that leader's supervision, the decision making starts diffusing over the group, as they learn to discuss and make decisions together.
As I was teaching my niece and nephew to say to their parents in the car - Daddy - are we there yet?
If we are not already there, how are we going to get there?
Nor do we necessarily have a great track record for executive decisions actually working things out.
No, I agree, the right leader will help form the group well for making good group decisions. I think.
In the mean-time - now that there is a change - could I ask - where do you three see Numpy going in the next five years? What do you see as the challenges to solve? What are the big risks? What are the big possibilities?
Cheers,
Matthew _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi, On Sun, Dec 23, 2012 at 8:56 AM, Dag Sverre Seljebotn <d.s.seljebotn@astro.uio.no> wrote:
On 12/22/2012 06:36 PM, Matthew Brett wrote:
Hi,
On Thu, Dec 20, 2012 at 5:39 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Thu, Dec 20, 2012 at 11:46 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Travis - I think you are suggesting that there should be no one person in charge of numpy, and I think this is very unlikely to work well. Perhaps there are good examples of well-led projects where there is not a clear leader, but I can't think of any myself at the moment. My worry would be that, without a clear leader, it will be unclear how decisions are made, and that will make it very hard to take strategic decisions.
Curious; my feeling is the opposite, that among mature and successful FOSS projects, having a clear leader is the uncommon case. GCC doesn't, Glibc not only has no leader but they recently decided to get rid of their formal steering committee, I'm pretty sure git doesn't, Apache certainly doesn't, Samba doesn't really, etc. As usual Karl Fogel has sensible comments on this: http://producingoss.com/en/consensus-democracy.html
Ah yes - that is curious. My - er - speculation was based on:
Numpy - Travis golden age in which we still bask Sympy - Ondrej, then Aaron - evolving into group decision making AFAICT IPython - Fernando, evolving into group decision making, AFAICT Cython - Robert Bradshaw - evolving into ... - you get the idea.
I don't really want to prolong this thread, but I feel like I should correct a factual error. Cython started with Robert Bradshaw (and other Sage members) and Stefan Behnel exchanging patches on top of Pyrex; there was definitely no leader at that point. Then I came along; there was no leader at that point either (but I was aware that the two others had a longer track record of course).
Robert Bradshaw was declared leader in order to break the tie when I and Stefan Behnel had argued for a 100-post long thread and could not reach a conclusion. And at least in this case, we were able to settle on a leadership structure then, when we needed it, and didn't regret not doing it earlier.
Thanks for correcting the error - sorry for passing on my half-understood knowledge, better history is useful. I am sure not discussing stuff works for some groups, but I very much doubt that it will work for numpy. The masked array discussion was a particularly good example where the attempt to shut down the discussion led to a great deal of wasted time and effort and lots of bad feeling, when some good time spent to hammer out the issues would (probably) have been a much more efficient use of energy. I hope that this time, instead of trying to shut down the conversation as fast as possible, we can have a productive and reasoned discussion about what to do next, in order to make the best possible decision. See you, Matthew
On Sat, Dec 22, 2012 at 5:36 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Thu, Dec 20, 2012 at 5:39 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Thu, Dec 20, 2012 at 11:46 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Travis - I think you are suggesting that there should be no one person in charge of numpy, and I think this is very unlikely to work well. Perhaps there are good examples of well-led projects where there is not a clear leader, but I can't think of any myself at the moment. My worry would be that, without a clear leader, it will be unclear how decisions are made, and that will make it very hard to take strategic decisions.
Curious; my feeling is the opposite, that among mature and successful FOSS projects, having a clear leader is the uncommon case. GCC doesn't, Glibc not only has no leader but they recently decided to get rid of their formal steering committee, I'm pretty sure git doesn't, Apache certainly doesn't, Samba doesn't really, etc. As usual Karl Fogel has sensible comments on this: http://producingoss.com/en/consensus-democracy.html
Ah yes - that is curious. My - er - speculation was based on:
Numpy - Travis golden age in which we still bask Sympy - Ondrej, then Aaron - evolving into group decision making AFAICT IPython - Fernando, evolving into group decision making, AFAICT Cython - Robert Bradshaw - evolving into ... - you get the idea.
and then reading about businesses particularly Good to Great, Built to Last, the disaster at HP when they didn't take care about succession. In general, that reading gave me the impression that successful organizations take enormous care about succession. I can't think of any case in the business literature I've read where a successful leader handed over to a group of three.
I think this is just a case of different organizational styles working differently. If organizations were optimisation algorithms, good businesses would be Newton's method, and good FOSS projects would be simulated annealing, or maybe GAs. Slower and less focused, but more robust against noise and local minima, and less susceptible to perturbations. They depend much less on the focused attention of visionary leaders. (Also I wouldn't consider numpy to have a formal "group of three leaders" now just because Travis mentioned three names in his email. Leadership is something people do, not something people are, so it's a fuzzy category in the first place.)
In practice the main job of a successful FOSS leader is to refuse to make decisions, nudge people to work things out, and then if they refuse to work things out tell them to go away until they do: https://lwn.net/Articles/105375/ and what actually gives people influence in a project is the respect of the other members. The former stuff is stuff anyone can do, and the latter isn't something you can confer or take away with a vote.
Right. My impression is - I'm happy to be corrected with better information - that the leader of a to-be-successful organization is very good at encouraging a spirit of free and vigorous debate, strong opinion, and reasoned decisions - and that may be the main gift they give to the organization. At that point, usually under that leader's supervision, the decision making starts diffusing over the group, as they learn to discuss and make decisions together.
As I was teaching my niece and nephew to say to their parents in the car - Daddy - are we there yet?
If we are not already there, how are we going to get there?
Nor do we necessarily have a great track record for executive decisions actually working things out.
No, I agree, the right leader will help form the group well for making good group decisions. I think.
In the mean-time - now that there is a change - could I ask - where do you three see Numpy going in the next five years? What do you see as the challenges to solve? What are the big risks? What are the big possibilities?
Personally I'd like to see NA support and sparse ndarrays in numpy proper, but I'm not going to have the time to write them myself in the forseeable future... In the long run of course everyone wants a version of numpy+python that can do automatic loop fusion (since that's the core feature for achieving throughput on modern CPUs) without giving up the ability to interface with C code and CPython compatibility. In my dreams the PyPy people will get their act together WRT interfacing with C code, the Cython people will take advantage of this to write a Cython-to-RPython compiler that lets the PyPy optimizer see the internals of Cython-written code, and then we port numpy to Cython and get a single compatible code-base that can run fast on both CPython and PyPy. But who knows what will actually make sense, if anything; as they say, it's very hard to make predictions, especially about the future. And of course the actual long-term strategic plan is "review PRs, merge the good ones". -n
Hi, On Sun, Dec 23, 2012 at 7:00 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Sat, Dec 22, 2012 at 5:36 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Thu, Dec 20, 2012 at 5:39 PM, Nathaniel Smith <njs@pobox.com> wrote:
On Thu, Dec 20, 2012 at 11:46 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Travis - I think you are suggesting that there should be no one person in charge of numpy, and I think this is very unlikely to work well. Perhaps there are good examples of well-led projects where there is not a clear leader, but I can't think of any myself at the moment. My worry would be that, without a clear leader, it will be unclear how decisions are made, and that will make it very hard to take strategic decisions.
Curious; my feeling is the opposite, that among mature and successful FOSS projects, having a clear leader is the uncommon case. GCC doesn't, Glibc not only has no leader but they recently decided to get rid of their formal steering committee, I'm pretty sure git doesn't, Apache certainly doesn't, Samba doesn't really, etc. As usual Karl Fogel has sensible comments on this: http://producingoss.com/en/consensus-democracy.html
Ah yes - that is curious. My - er - speculation was based on:
Numpy - Travis golden age in which we still bask Sympy - Ondrej, then Aaron - evolving into group decision making AFAICT IPython - Fernando, evolving into group decision making, AFAICT Cython - Robert Bradshaw - evolving into ... - you get the idea.
and then reading about businesses particularly Good to Great, Built to Last, the disaster at HP when they didn't take care about succession. In general, that reading gave me the impression that successful organizations take enormous care about succession. I can't think of any case in the business literature I've read where a successful leader handed over to a group of three.
I think this is just a case of different organizational styles working differently. If organizations were optimisation algorithms, good businesses would be Newton's method, and good FOSS projects would be simulated annealing, or maybe GAs. Slower and less focused, but more robust against noise and local minima, and less susceptible to perturbations. They depend much less on the focused attention of visionary leaders.
You seem to be implying that any management organization for numpy would have the same effect as any other as far as we know, and if that were true, it would certainly not be worth discussing in any detail. But I doubt very much that is true, and, following the optimization strategy logic, there may be a reason that having 3 people lead an organization has not been a common and visible option, and that is that it doesn't work very well.
(Also I wouldn't consider numpy to have a formal "group of three leaders" now just because Travis mentioned three names in his email. Leadership is something people do, not something people are, so it's a fuzzy category in the first place.)
In Travis' email I saw the three of you and a 2 to 1 voting suggestion. There doesn't seem to be much appetite for discussing alternatives and neither Chuck nor Ralf have joined this discussion, so that seems to be the only option on the table. Is there another one? Or are you thinking that something will gradually evolve from - essentially - no prescription of how things work. Again, I doubt very much that "no prescription" has been a successful option in the past, even for FOSS. It just never happens in businesses or successful governments - does it? The problem is obviously going to be what to do when we get problems, as we have in the past. When there is no or little structure to use, then decisions don't get made, or get made by default, and the debate deteriorates because there is no-one to moderate it. That seems to me the situation numpy is in, and doing nothing or having poorly defined management can only prolong that or even make it worse.
In practice the main job of a successful FOSS leader is to refuse to make decisions, nudge people to work things out, and then if they refuse to work things out tell them to go away until they do: https://lwn.net/Articles/105375/ and what actually gives people influence in a project is the respect of the other members. The former stuff is stuff anyone can do, and the latter isn't something you can confer or take away with a vote.
Right. My impression is - I'm happy to be corrected with better information - that the leader of a to-be-successful organization is very good at encouraging a spirit of free and vigorous debate, strong opinion, and reasoned decisions - and that may be the main gift they give to the organization. At that point, usually under that leader's supervision, the decision making starts diffusing over the group, as they learn to discuss and make decisions together.
As I was teaching my niece and nephew to say to their parents in the car - Daddy - are we there yet?
If we are not already there, how are we going to get there?
For example - you didn't answer this one. What are your thoughts?
Nor do we necessarily have a great track record for executive decisions actually working things out.
No, I agree, the right leader will help form the group well for making good group decisions. I think.
In the mean-time - now that there is a change - could I ask - where do you three see Numpy going in the next five years? What do you see as the challenges to solve? What are the big risks? What are the big possibilities?
Personally I'd like to see NA support and sparse ndarrays in numpy proper, but I'm not going to have the time to write them myself in the forseeable future...
In the long run of course everyone wants a version of numpy+python that can do automatic loop fusion (since that's the core feature for achieving throughput on modern CPUs) without giving up the ability to interface with C code and CPython compatibility. In my dreams the PyPy people will get their act together WRT interfacing with C code, the Cython people will take advantage of this to write a Cython-to-RPython compiler that lets the PyPy optimizer see the internals of Cython-written code, and then we port numpy to Cython and get a single compatible code-base that can run fast on both CPython and PyPy. But who knows what will actually make sense, if anything; as they say, it's very hard to make predictions, especially about the future.
And of course the actual long-term strategic plan is "review PRs, merge the good ones".
The contrast between these last two paragraphs is strong. The first is a vision for how numpy might be - to coin a phrase - "the next generation of numpy". It seems exciting and interesting, but you don't hold out much hope of getting there, and having not-much-management makes it less likely we will have any real new direction to the project. That's your second paragraph - keep on keeping on. In effect it condemns numpy to be a slow-moving development effort waiting for something more interesting to overtake it. Is that really necessary? Can we not hope for better? Is there any real chance of attracting a force of new developers in that situation? Best, Matthew
Hi Matthew, On Thu, Dec 20, 2012 at 3:46 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
[Travis wrote...]
My strong suggestion is that development discussions of the project continue on this list with consensus among the active participants being the goal for development. I don't think 100% consensus is a rigid requirement --- but certainly a super-majority should be the goal, and serious changes should not be made with out a clear consensus. I would pay special attention to under-represented people (users with intense usage of NumPy but small voices on this list). There are many of them. If you push me for specifics then at this point in NumPy's history, I would say that if Chuck, Nathaniel, and Ralf agree on a course of action, it will likely be a good thing for the project. I suspect that even if only 2 of the 3 agree at one time it might still be a good thing (but I would expect more detail and discussion). There are others whose opinion should be sought as well: Ondrej Certik, Perry Greenfield, Robert Kern, David Cournapeau, Francesc Alted, and Mark Wiebe to name a few. For some questions, I might even seek input from people like Konrad Hinsen and Paul Dubois --- if they have time to give it. I will still be willing to offer my view from time to time and if I am asked.
Thank you for starting this discussion.
I am more or less offline at the moment in Cuba and flying, but I hope very much this will be an opportunity for a good discussion on the best way forward for numpy.
Travis - I think you are suggesting that there should be no one person in charge of numpy, and I think this is very unlikely to work well. Perhaps there are good examples of well-led projects where there is not a clear leader, but I can't think of any myself at the moment. My worry would be that, without a clear leader, it will be unclear how decisions are made, and that will make it very hard to take strategic decisions.
I would like to humbly suggest the following in the hope that it spurs discussion.
As first pass, Ralf, Chuck and Nathaniel decide on a core group of people that will form the succession committee. Maybe this could be the list of people you listed above.
Ralf, Chuck and Nathaniel then ask for people to wish to propose themselves as the leader of numpy. Anyone proposing themselves to lead numpy would remove themselves from the succession committee.
The proposed leaders of numpy write a short manifesto saying why they are the right choice for the job, and what they intend to do if elected. The manifestos and issues arising are discussed in public on the mailing list - the equivalent of an online presidential debate.
In due course - say after 2 weeks or after the discussion seems to be dying out - the succession committee votes on the leader. I propose that these votes should be public, but I can see the opposite argument.
Travis has very clearly made "Chuck, Nathaniel, and Ralf" as the leaders of the project. But as always ---- he didn't pick them because he needed to create leaders. They were already the de-facto leaders of the project due to their actions, involvements and respect in the community, so he just made it official.
How does that sound?
To me that sounds like a bad idea. That being said, if Chuck, Nathaniel, and Ralf agree that it would be a good idea, that's fine with me too. Ondrej
On Thu, Dec 20, 2012 at 5:48 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
Hi Matthew,
On Thu, Dec 20, 2012 at 3:46 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
[Travis wrote...]
My strong suggestion is that development discussions of the project continue on this list with consensus among the active participants being the goal for development. I don't think 100% consensus is a rigid requirement --- but certainly a super-majority should be the goal, and serious changes should not be made with out a clear consensus. I would pay special attention to under-represented people (users with intense usage of NumPy but small voices on this list). There are many of them. If you push me for specifics then at this point in NumPy's history, I would say that if Chuck, Nathaniel, and Ralf agree on a course of action, it will likely be a good thing for the project. I suspect that even if only 2 of the 3 agree at one time it might still be a good thing (but I would expect more detail and discussion). There are others whose opinion should be sought as well: Ondrej Certik, Perry Greenfield, Robert Kern, David Cournapeau, Francesc Alted, and Mark Wiebe to name a few. For some questions, I might even seek input from people like Konrad Hinsen and Paul Dubois --- if they have time to give it. I will still be willing to offer my view from time to time and if I am asked.
Thank you for starting this discussion.
I am more or less offline at the moment in Cuba and flying, but I hope very much this will be an opportunity for a good discussion on the best way forward for numpy.
Travis - I think you are suggesting that there should be no one person in charge of numpy, and I think this is very unlikely to work well. Perhaps there are good examples of well-led projects where there is not a clear leader, but I can't think of any myself at the moment. My worry would be that, without a clear leader, it will be unclear how decisions are made, and that will make it very hard to take strategic decisions.
I would like to humbly suggest the following in the hope that it spurs discussion.
As first pass, Ralf, Chuck and Nathaniel decide on a core group of people that will form the succession committee. Maybe this could be the list of people you listed above.
Ralf, Chuck and Nathaniel then ask for people to wish to propose themselves as the leader of numpy. Anyone proposing themselves to lead numpy would remove themselves from the succession committee.
The proposed leaders of numpy write a short manifesto saying why they are the right choice for the job, and what they intend to do if elected. The manifestos and issues arising are discussed in public on the mailing list - the equivalent of an online presidential debate.
In due course - say after 2 weeks or after the discussion seems to be dying out - the succession committee votes on the leader. I propose that these votes should be public, but I can see the opposite argument.
Travis has very clearly made "Chuck, Nathaniel, and Ralf" as the leaders of the project. But as always ---- he didn't pick them because he needed to create leaders. They were already the de-facto leaders of the project due to their actions, involvements and respect in the community, so he just made it official.
How does that sound?
To me that sounds like a bad idea. That being said, if Chuck, Nathaniel, and Ralf agree that it would be a good idea, that's fine with me too.
I forgot to add --- Matthew, I'll be happy to discuss this over phone once you get back to the US. I know we had discussions over this in the past, but I couldn't offer much advice, as I didn't understand the inner working of the numpy development. I now understand it much better. Ondrej
participants (5)
-
Dag Sverre Seljebotn -
Matthew Brett -
Nathaniel Smith -
Ondřej Čertík -
Travis Oliphant