[Numpy-discussion] Numpy governance update
Bruce Southey
bsouthey at gmail.com
Wed Feb 15 22:31:48 EST 2012
On Wed, Feb 15, 2012 at 6:57 PM, <josef.pktd at gmail.com> wrote:
> On Wed, Feb 15, 2012 at 7:27 PM, Dag Sverre Seljebotn
> <d.s.seljebotn at astro.uio.no> wrote:
>> On 02/15/2012 02:24 PM, Mark Wiebe wrote:
>>> On Wed, Feb 15, 2012 at 1:36 PM, Matthew Brett <matthew.brett at gmail.com
>>> <mailto:matthew.brett at gmail.com>> wrote:
>>>
>>> Hi,
>>>
>>> On Wed, Feb 15, 2012 at 12:55 PM, Mark Wiebe <mwwiebe at gmail.com
>>> <mailto:mwwiebe at gmail.com>> wrote:
>>> > On Wed, Feb 15, 2012 at 12:09 PM, Matthew Brett
>>> <matthew.brett at gmail.com <mailto:matthew.brett at gmail.com>>
>>> > wrote:
>>> >>
>>> >> Hi,
>>> >>
>>> >> On Wed, Feb 15, 2012 at 11:46 AM, Benjamin Root <ben.root at ou.edu
>>> <mailto:ben.root at ou.edu>> wrote:
>>> >> >
>>> >> >
>>> >> > On Wed, Feb 15, 2012 at 1:32 PM, Alan G Isaac
>>> <alan.isaac at gmail.com <mailto:alan.isaac at gmail.com>>
>>> >> > wrote:
>>> >> >> Can you provide an example where a more formal
>>> >> >> governance structure for NumPy would have meant
>>> >> >> more or better code development? (Please do not
>>> >> >> suggest the NA discussion!)
>>> >> >>
>>> >> >
>>> >> > Why not the NA discussion? Would we really want to have that
>>> happen
>>> >> > again?
>>> >> > Note that it still isn't fully resolved and progress still
>>> needs to be
>>> >> > made
>>> >> > (I think the last thread did an excellent job of fleshing out
>>> the ideas,
>>> >> > but
>>> >> > it became too much to digest. We may need to have someone go
>>> through
>>> >> > the
>>> >> > information, reduce it down and make one last push to bring it
>>> to a
>>> >> > conclusion). The NA discussion is the perfect example where a
>>> >> > governance
>>> >> > structure would help resolve disputes.
>>> >>
>>> >> Yes, that was the most obvious example. I don't know about you,
>>> but I
>>> >> can't see any sign of that one being resolved.
>>> >>
>>> >> The other obvious example was the dispute about ABI breakage for
>>> numpy
>>> >> 1.5.0 where I believe Travis did invoke some sort of committee to
>>> >> vote, but (Travis can correct me if I'm wrong), the committee was
>>> >> named ad-hoc and contacted off-list.
>>> >>
>>> >> >
>>> >> >>
>>> >> >> Can you provide an example of what you might
>>> >> >> envision as a "more formal governance structure"?
>>> >> >> (I assume that any such structure will not put people
>>> >> >> who are not core contributors to NumPy in a position
>>> >> >> to tell core contributors what to spend their time on.)
>>> >> >>
>>> >> >> Early last December, Chuck Harris estimated that three
>>> >> >> people were active NumPy developers. I liked the idea of
>>> >> >> creating a "board" of these 3 and a rule that says any
>>> >> >> active developer can request to join the board, that
>>> >> >> additions are determined by majority vote of the existing
>>> >> >> board, and that having the board both small and odd
>>> >> >> numbered is a priority. I also suggested inviting to this
>>> >> >> board a developer or two from important projects that are
>>> >> >> very NumPy dependent (e.g., Matplotlib).
>>> >> >>
>>> >> >> I still like this idea. Would it fully satisfy you?
>>> >> >>
>>> >> >
>>> >> > I actually like that idea. Matthew, is this along the lines
>>> of what you
>>> >> > were thinking?
>>> >>
>>> >> Honestly it would make me very happy if the discussion moved to what
>>> >> form the governance should take. I would have thought that 3
>>> was too
>>> >> small a number.
>>> >
>>> >
>>> > One thing to note about this point is that during the NA
>>> discussion, the
>>> > only people doing active C-level development were Charles and me.
>>> I suspect
>>> > a discussion about how to recruit more people into that group
>>> might be more
>>> > important than governance at this point in time.
>>>
>>> Mark - a) thanks for replying, it's good to hear your voice and b) I
>>> don't think there's any competition between the discussion about
>>> governance and the need to recruit more people into the group who
>>> understand the C code.
>>>
>>>
>>> There hasn't really been any discussion about recruiting developers to
>>> compete with the governance topic, now we can let the topics compete. :)
>>>
>>> Some of the mechanisms which will help are already being set in motion
>>> through the discussion about better infrastructure support like bug
>>> trackers and continuous integration. The forthcoming roadmap discussion
>>> Travis alluded to, where we will propose a roadmap for review by the
>>> numpy user community, will include many more such points.
>>>
>>> Remember we are deciding here between governance - of a form to be
>>> decided - and no governance - which I think is the current situation.
>>> I know your desire is to see more people contributing to the C code.
>>> It would help a lot if you could say what you think the barriers are,
>>> how they could be lowered, and the risks that you see as a result of
>>> the numpy C expertise moving essentially into one company. Then we
>>> can formulate some governance that would help lower those barriers and
>>> reduce those risks.
>>>
>>>
>>> There certainly is governance now, it's just informal. It's a
>>> combination of how the design discussions are carried out, how pull
>>> requests occur, and who has commit rights.
>>
>> +1
>>
>> If non-contributing users came along on the Cython list demanding that
>> we set up a system to select non-developers along on a board that would
>> have discussions in order to veto pull requests, I don't know whether
>> we'd ignore it or ridicule it or try to show some patience, but we
>> certainly wouldn't take it seriously.
>>
>> It's obvious that one should try for consensus as long as possible,
>> including listening to users. But in the very end, when agreement can't
>> be reached by other means, the developers are the one making the calls.
>> (This is simply a consequence that they are the only ones who can
>> credibly threaten to fork the project.)
>>
>> Sure, structures that includes users in the process could be useful...
>> but, if the devs are fine with the current situation (and I don't see
>> Mark or Charles complaining), then I honestly think it is quite rude to
>> not let the matter drop after the first ten posts or so.
>>
>> Making things the way one wants it and scratching *ones own* itch is THE
>> engine of open source development (whether one is putting in spare time
>> or monetary funding). Trying to work against that with artificial
>> structures doesn't sound wise for a project with as few devs as NumPy...
>
> I don't think you can restrict the Numpy developer or contributor
> group just to the developers that work on the C core like Charles and
> Mark, and others over the years I have been following it ( Pauli and
> David, ...).
> There is a large part of non C numpy, Pierre for example, and Joe
> Harrington put money and a lot of effort into bringing the
> documentation into the current state, the documentation was mostly a
> community effort.
>
> Of course I only ever contributed to scipy, except of two or three
> bugfixes in numpy.random, but I still care about the direction numpy
> is going, as do developers of the "SciPy" community which crucially
> rely on numpy.
>
> Josef
>
>
>>
>> Dag
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
+1
As Josef points out that there is *tremendous* effort to document
numpy and scipy which was not done by just the core developers!
Not to forget that numpy is built on a very long stable history of
Numeric and numarray. So it should be no surprise that there a great
amount of numpy usage (for example, Google trends -
http://www.google.com/trends/?q=numpy ) and that many projects do
indeed to depend on it. Not just the obvious ones - indeed matlibplot
apparently has more nice functions tools that perhaps should be
included. Also there are forked projects like tabulate that probably
could have been included in numpy instead- maybe with the new commit
rights, carray will be added. Yet to me the most interesting is that
the pypy project where 'the survey said' "25% of respondants said
somthing about NumPy"! Not meaning to push this, but I guess I am,
(http://pypy.org/numpydonate), but their donation level is almost 70%
of the target.
We do need a structure that represents the needs of the community not
just core developers. It is also where I very much disagree with Dag,
(not a CS major to know about the 'gang of four' -
http://en.wikipedia.org/wiki/Design_Patterns) but 'non-contributing
users' are essential for not only providing direction but, more
importantly, getting the code right! Thus, any governance has to
ensure quality code that meets the community standards and
expectations plus that the code is maintainable in the long term.
(Yes, I am very frustrated when numpy and scipy release candidates
fail under supported Python versions.)
Bruce
More information about the NumPy-Discussion
mailing list