On Thu, Aug 27, 2015 at 8:57 AM, Matthew Brett <matthew.brett@gmail.com> wrote:

Hi,

On Thu, Aug 27, 2015 at 12:11 PM, Sebastian Berg
<sebastian@sipsolutions.net> wrote:
> On Do, 2015-08-27 at 10:45 +0100, Matthew Brett wrote:
>> Hi,
>>
>> On Thu, Aug 27, 2015 at 10:35 AM, Bryan Van de Ven <bryanv@continuum.io> wrote:
>> >
>> >> On Aug 27, 2015, at 10:22 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
>> >>
>> >> In the case of the 'core' model, we have some compelling testimony
>> >> from someone with a great deal of experience:
>> >>
>> >> """
>> >> Much of this early structure (CVS, web site, cabal ["core" group],
>> >> etc.) was copied verbatim by other open source (this term not being in
>> >> wide use yet) projects -- even the form of the project name and the
>> >> term "core". This later became a kind of standard template for
>> >> starting up an open source project. [...] I'm sorry to say that I
>> >> helped create this problem, and that most of the projects which
>> >> modeled themselves after NetBSD (probably due to its high popularity
>> >> in 1993 and 1994) have suffered similar problems. FreeBSD and XFree86,
>> >> for example, have both forked successor projects (Dragonfly and X.org)
>> >> for very similar reasons.
>> >> """
>> >
>> > Who goes on to propose:
>> >
>> > 7) The "core" group must be replaced with people who are actually
>> > competent and dedicated enough to review proposals, accept feedback,
>> > and make good decisions. More to the point, though, the "core" group
>> > must only act when *needed* -- most technical decisions should be
>> > left to the community to hash out; it must not preempt the community
>> > from developing better solutions. (This is how the "core" group
>> > worked during most of the project's growth period.)
>>
>> Sure. I think it's reasonable to give high weight to Hannum's
>> assessment of the failure of the core group, but less weight to his
>> proposal for a replacement, because at the time, I don't believe he
>> was in a good position to assess whether his (apparent) alternative
>> would run into the same trouble.
>>
>> It's always tempting to blame the people rather than the system, but
>> in this case, I strongly suspect that it was the system that was
>> fundamentally flawed, therefore either promoting the wrong people or
>> putting otherwise competent people into situations where they are no
>> longer getting useful feedback.
>
> Maybe so. I do not know much at all about these models, but I am not
> sure how much applies here to numpy. Isn't at least FreeBSD a magnitude
> larger then numpy?

It seems to me that numpy suffers from the same risks of poor
accountability, stagnation and conservatism that larger projects do.
Is there a reason that would not be the case?

> We do need to have some formality about how to give out commit rights,
> and do final decision when all else fails.

Yes, sure, something formal is probably but not certainly better than
nothing, depending on what the 'something formal' is.

> One thing I do not know is how a "community vote" could work at all,
> considering I do not even know how to count its members. Votes and
> presidents make sense to me for large projects with hundrets of
> developers on different corners (think of the gnome foundation, debian
> probably) [1].

The 'president' idea is to get at the problem of lack of
accountability, along with selection for leadership skill rather than
coding ability. It's trying to get at the advantages of the BDFL
model in our situation where there is no obvious BDFL. For the me
the problem is that, at the moment, if the formal or informal
governing body makes a bad decision, then no member will feel
responsible for that decision or its consequences. That tends to lead
to an atmosphere of - "oh well, what could we do, X wouldn't agree to
A and Y wouldn't agree to B so we're stuck". It seems to me we need
a system such that whoever is in charge feels so strongly that it is
their job to make numpy as good as possible, that they will take
whatever difficult or sensitive decisions are necessary to make that
happen. On the other hand the 'core' system seems to function on a
model of mutual deference and personal loyalty that I believe is
destructive of good management.

I don't really see a problem with "codifying" the status quo.

It might become necessary to have something like an administrative director if numpy becomes a more formal organization with funding, but for the development of the project I don't see any need for a president.

If there is no obvious BDFL, then I guess there is also no obvious president. (I would vote for Ralf as president of everything, but I don't think he's available.)

As the current debate shows it's possible to have a public discussion about the direction of the project without having to delegate providing a vision to a president.

Given the current pattern all critical issues end up in a public debate on the mailing list. What numpy (and scipy) need is to have someone as a tie breaker to make any final decisions if there is no clear consensus, if there is no BDFL for it, then having the "core" group making those decisions looks appropriate to me.

> "On the other hand the 'core' system seems to function on a
model of mutual deference and personal loyalty that I believe is
destructive of good management."

That sounds actually like a good basis for team work to me. Also that has "mutual" in it instead of just deferring and being loyal to a president.

Since I know scipy development much better:

scipy has made a huge progress in the last 5 or 6 years since I've been following it, both in terms of code, in terms of development workflow, and in the number of developers. (When I started, I was essentially alone in scipy.stats, now there are 3 to 5 "core" developers that at least partially work on it, everything goes through PRs with public discussion and with critical issues additionally raised on the mailing list.)

Ralf and Pauli are the defacto BDFLs for scipy overall, but all decisions in recent years have been without a fight, but not without lots of arguments, and, given the size and breadth of scipy, there are field experts (although not enough of those) to help in the discussion.

There are still stalled PRs, blocked proposals, and decisions that I didn't like, but I think that's unavoidable.

One feature of the current "core" system is that it is relatively open, almost all discussions are public, and it is relatively (?) easy to get into it for new developers. I currently don't worry that a closed clique is taking over the numpy or scipy "core" group:.

Josef

Cheers,

Matthew
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion