[Numpy-discussion] What is consensus anyway

[Matthew Brett]

Hi,

On Tue, Apr 24, 2012 at 6:12 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
> On Tue, Apr 24, 2012 at 6:56 PM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> On Tue, Apr 24, 2012 at 2:14 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> >
>> > On Mon, Apr 23, 2012 at 11:35 PM, Fernando Perez <fperez.net at gmail.com>
>> > wrote:
>> >>
>> >> On Mon, Apr 23, 2012 at 8:49 PM, Stéfan van der Walt <stefan at sun.ac.za>
>> >> wrote:
>> >> > If you are referring to the traditional concept of a fork, and not to
>> >> > the type we frequently make on GitHub, then I'm surprised that no one
>> >> > has objected already.  What would a fork solve? To paraphrase the
>> >> > regexp saying: after forking, we'll simply have two problems.
>> >>
>> >> I concur with you here: github 'forks', yes, as many as possible!
>> >> Hopefully every one of those will produce one or more PRs :)  But a
>> >> fork in the sense of a divergent parallel project?  I think that would
>> >> only be indicative of a complete failure to find a way to make
>> >> progress here, and I doubt we're anywhere near that state.
>> >>
>> >> That forks are *possible* is indeed a valuable and important option in
>> >> open source software, because it means that a truly dysfunctional
>> >> original project team/direction can't hold a community hostage
>> >> forever.  But that doesn't mean that full-blown forks should be
>> >> considered lightly, as they also carry enormous costs.
>> >>
>> >> I see absolutely nothing in the current scenario to even remotely
>> >> consider that a full-blown fork would be a good idea, and I hope I'm
>> >> right.  It seems to me we're making progress on problems that led to
>> >> real difficulties last year, but from multiple parties I see signs
>> >> that give me reason to be optimistic that the project is getting
>> >> better, not worse.
>> >>
>> >
>> > We certainly aren't there at the moment, but I can see us heading that
>> > way.
>> > But let's back up a bit. Numpy 1.6.0 came out just about 1 year ago.
>> > Since
>> > then datetime, NA, polynomial work, and various other enhancements have
>> > gone
>> > in along with some 280 bug fixes. The major technical problem blocking a
>> > 1.7
>> > release is getting datetime working reliably on windows. So I think that
>> > is
>> > where the short term effort needs to be. Meanwhile, we are spending
>> > effort
>> > to get out a 1.6.2 just so people can work with a stable version with
>> > some
>> > of the bug fixes, and potentially we will spend more time and effort to
>> > pull
>> > out the NA code. In the future there may be a transition to C++ and
>> > eventually a break with the current ABI. Or not.
>> >
>> > There are at least two motivations that get folks to write code for open
>> > source projects, scratching an itch and money. Money hasn't been a big
>> > part
>> > of the Numpy picture so far, so that leaves scratching an itch. One of
>> > the
>> > attractions of Numpy is that it is a small project, BSD licensed, and
>> > not
>> > overburdened with governance and process. This makes scratching an itch
>> > not
>> > as difficult as it would be in a large project. If Numpy remains a small
>> > project but acquires the encumbrances of a big project much of that
>> > attraction will be lost. Momentum and direction also attracts people,
>> > but
>> > numpy is stalled at the moment as the whole NA thing circles around once
>> > again.
>>
>> I don't think we need a fork, or to start maintaining separate stable
>> and unstable trees, or any of the other complicated process changes
>> that have been suggested. There are tons of projects that routinely
>> make much bigger changes than we're talking about, and they do it
>> without needing that kind of overhead. I know that these suggestions
>> are all made in good faith, but they remind me of a line from that
>> Apache page I linked earlier: "People tend to avoid conflict and
>> thrash around looking for something to substitute - somebody in
>> charge, a rule, a process, stagnation. None of these tend to be very
>> good substitutes for doing the hard work of resolving the conflict."
>>
>> I also think if you talk to potential contributors, you'll find that
>> clear, simple processes and a history of respecting everyone's input
>> are much more attractive than a no-rules free-for-all. Good
>> engineering practices are not an "encumbrance". Resolving conflicts
>> before merging is a good engineering practice.
>>
>> What happened with the NA discussion is this:
>>  - There was substantial disagreement about whether NEP-style masks,
>> or indeed, focusing on a mask-based implementation *at all*, was the
>> best way forward.
>>  - There was also a perceived time constraint, that we had to either
>> implement something immediately while Mark was there, or have nothing.
>>
>> So in the end, the latter concern outweighed the former, the
>> discussion was cut off, and Mark's best guess at an API was merged
>> into master. I totally understand how this decision made sense at the
>> time, but the result is what we see now: it's left numpy stalled,
>> rifts on the mailing list, boring discussions about process, and still
>> no agreement about whether NEP-style masks will actually solve our
>> users' problems.
>>
>> Getting past this isn't *complicated* -- it's just "hard work".
>
>
> I admit to a certain curiosity about your own involvement in FOSS projects,
> and I know I'm not alone in this. Google shows several years of discussion
> on Monotone, but I have no idea what your contributions were outside of
> documentation. Monotone has sort of died, but that is the luck of the draw.
> Would you care to comment on the success of the process in that project and
> what went right or wrong? The other thing I see your name attached to is
> xpra, which gets good reviews, but that was a personal project and forked
> after you found more interesting things to work on.

I'm sorry, but I think this is really not OK. This is an ad-hominem attack [1].

I'm not going to join in by justifying Nathaniel's expertise, because
I do not think he should have to do that.  I just think it is wrong to
ask people to justify their expertise when we should be discussing the
ideas.

If I was a new subscriber to this list, and saw that my right to speak
was likely to be audited in public if I disagree, I would be very
reluctant to differ from the majority.  I do not think that is what we
want.

I think Nathaniel's summary of how the masked array dispute arose and
continued is entirely reasonable.

I think his analysis of the issues in the masked array API are also
reasonable, well-thought out, and well-argued.  I think we should
address his arguments.  I think we're getting stuck precisely because
you will not do that.

Best,

Matthew

[1] http://en.wikipedia.org/wiki/Ad_hominem