On Sat, Sep 5, 2015 at 7:14 PM, Nathaniel Smith email@example.com wrote:
On Sat, Sep 5, 2015 at 4:20 PM, Matthew Brett firstname.lastname@example.org wrote:
Thanks for replying, that is helpful.
I'm going to write carefully, because I suspect we've only got this one shot at getting it right.
Not sure why? "Change the governance document" is explicitly listed as one of the steering council's powers, so I don't see what we'd lose by adopting what we have now.
In practice, I doubt that there will be any proposal to change the governance in the medium term, especially if the governance does turn out to be risk averse and conservative.
I first wanted to come back to an earlier email of yours:
On Thu, Aug 27, 2015 at 3:05 AM, Nathaniel Smith email@example.com wrote:
At least personally, I have to admit that the idea of a governance model involving elections fills me with creeping horror. The reason is that whole point of having a governance model (IMHO) is to (a) minimize the rise of interpersonal drama, (b) when some amount of interpersonal drama does inevitably arise anyway, provide some regulated channel for it, hopefully one that leads to a drama sink.
I think this is the wrong way to think of the problem. There is no drama when:
- everyone agrees (problem is trivial);
- no one cares (problem is not important);
- everyone who does care has come to the conclusion that it is not
The opposite of drama is not efficiency, but boredom and frustration.
As evidence, vigorous, substantial and bad-tempered debates are characteristic of successful companies:
""" "They were chaos. We would stay there for hours, ironing out the issues, until we came to something ... At times the meetings would get so violent that people almost went across the table at each other.... People yelled. They waved their arms around and pounded on tables. Faces would get red and veins bulged out". """ 
No, by drama I mean situations where technical issues become personal, and where people end up feeling emotionally hurt and disrespected and so forth. I agree that vigorous debate is important, but strongly disagree that veins bulging and violence are necessary or helpful to success. (This is tangential to your main point, but I wanted to point it out.) Since I happened to be reading Kate Heddleston's blog yesterday, here's another relevant link... https://www.kateheddleston.com/blog/argument-cultures-and-unregulated-aggres...
The distinction is between a culture that welcomes expressions of disagreement with one that does not. In practice, if a group is easily alarmed by expressions of strong feeling, most people will be careful about expressing disagreement. This is of course a rather subtle point. It is easy to say that we do welcome disagreement, when in practice what happens is that the person disagreeing will in fact be labeled as being impolite.
then hear their description of how much the team disagree with each other, and often, how important that disagreement is, to finding the right answer.
Indeed, that's... why we have a document that is designed to provide a framework for exactly that sort of debate? I don't really understand the criticism here.
Er - I was arguing that, by attempting to formalize a system that doesn't have a leader, that will lead to a culture where debate is more difficult and less effective.
My objection to elections is that they create a whole *extra* source of drama that runs the risk of compromising technical debates. Suddenly people have to campaign and have fraught arguments over who gets to vote and how to run the election and it's just borrowing trouble IMO. (Are you by chance on the PSF members mailing list? If not then ask someone how things have been going there vis-a-vis elections -- I'm not myself, but I've heard stories...) Obviously it's a fallacy to think that we can entirely ignore social issues and focus only on technical problems, but still -- I'd rather we spend our difficult debates on technical issues as much as possible.
Even worse, elections mean that someone might think "Hmm, should I give in on this point, even though I know I'm right, so as to avoid annoying this person who might otherwise retaliate by telling their friends not to vote for me next year?" Or "hmm, did they give in on that point because they actually agree or because the person they work for wants the person they're agreeing with to win the next election?" Or etc. Hopefully no-one would actually start playing politics to that degree, but IMHO it's poisonous to even set up a situation where people are tempted to perform such calculations.
I suppose these things might happen. The benefit of elections is that it forces us to be explicit about what the issues are, and what is at stake. It seems to me that is something that numpy has to do, and soon, whatever the mechanism.
On the other hand, I don't think it is our culture in numpy to value that kind of debate. That brings us to my next comment:
On Fri, Sep 4, 2015 at 8:35 PM, Nathaniel Smith firstname.lastname@example.org wrote:
On Fri, Aug 28, 2015 at 1:46 AM, Matthew Brett email@example.com wrote:
On Fri, Aug 28, 2015 at 5:59 AM, Jaime Fernández del Río firstname.lastname@example.org wrote:
<joke> Are you trying to prove the point that consensus doesn't work by making it impossible to reach a consensus on this? ;-) </joke>
Forgive me if I use this joke to see if I can get us any further.
If this was code, I think this joke would not be funny, because we wouldn't expect to reach consensus without considering all the options, and discussing their pros and cons.
Why would that not be useful in the case of forms of governance?
One reason might be that the specific form of governance can have no influence on the long-term health of the project.
I am convinced that that is wrong - that the form of governance has a large influence on the long-term health of a project.
If there is some possibility that this is true, then it seems to me that we would be foolish not to try and come to some reasoned choice about the form of governance.
That seems fair, and I also think that even if we end up not changing anything then there's still some utility in having the discussion. Certainly one of my secret fears in writing all this up is that it would end up being just some words that only I cared about and everyone else ignored, so seeing other people engaging with it is heartening :-).
I'm not feeling inspired this Friday evening to write a full white paper reviewing the universe of possible governance, but I'll summarize a bit what the reasoning was that led us to what we ended up with (at least as I remember it and from my perspective).
Some of the sources that were mentioned for reference and positive or negative inspiration include Karl Fogel's chapter on governance, the Debian constitution, the gcc/egcs fork+reverse-fork, glibc/eglibc ditto, subversion, Jupyter/ipython, the Linux kernel, python, matplotlib, the Apache foundation's processes, the gnome foundation, nodejs, ... I'm certainly forgetting some.
The basic observation that we started from was that all the successful projects seemed to fit a similar template: (1) basically all actual decision making is done by informal and maximally inclusive consensus-based processes, (2) there are some sort of more formal "backstop" rules that are called into play if informal processes break down (which hopefully never even happens). These projects don't tend to look that similar on paper, because people tend to only write down the "backstop" rules, and those vary a lot.
So e.g., for some projects like Python or IPython, the formal rule is "the BDFL decides". But then if you look at what successful BDFLs do with this power, they mostly enforce rule #1 above. (Karl discusses this: http://producingoss.com/en/producingoss.html#benevolent-dictator). E.g., if you watch Guido, I don't think I've ever seen him say "I know people don't like X but we're doing it anyway". He mostly sits back to watch the debate, steps in occasionally to prune off unproductive lines of conversation (e.g. in the PEP 465 debate where he watched for a while, stepped in to tell people to knock it off with the pointless bikeshedding about which character to use, and then went back to watching) or encourage people to keep thinking about something, is always very careful not to actually make any strong statements, etc. Or, e.g., the bit in the draft governance about how the steering committee's secondary job is to make decisions, and their primary job is to prevent things reaching the point where they have to? That's a straight paraphrase of Fernando talking about his BDFL philosophy.
Or on the other end of things, you have e.g. Subversion, which had an elaborate defined governance system with different levels of "core-ness", a voting system, etc. -- and they were 6 years into the project before they had their first vote. (The vote was on the crucial technical decision of whether to write function calls like "f ()" or "f()".) Or e.g. the massive and famously fractious Debian has their CTTE, which theoretically is the final arbiter of technical decisions in Debian and has immense powers... but they issue like ~5 rulings a year on average, and have gone years without issuing a ruling at all.
So, our thought process was: for our purposes, since we don't have a BDFL, we can't depend on them to be the unwritten mechanism for enforcing rule #1, so first we should write down rule #1.
What you are describing in the previous paragraph seems to be a well-known feature of successful leaders. The CEOs of highly successful companies described in  differed from less-successful comparators in encouraging debate and acting as the final arbiter for the decisions of the group.
I don't think you are right that you can create that effect without having such a leader. The point about the leader is that they are a trusted referee and mentor to the group. They provide stability and space for the debate. For example, in my little church, there is an elected position called the 'moderator' with that purpose. Most groups recognize that someone has to do that job, and that the group cannot do that themselves.
Having a good moderator is certainly a wonderful thing if you can manage it, but it's just not true that "Most groups recognize [...] the group cannot do that themselves". FOSS projects with a formal leader are in the minority.
I believe you - I was referring to groups like businesses, universities, governments, churches and so on.
And there's no particular need for a specific person to be designated to always do the job: in a consensus-based framework, pretty much anyone has the power to step up to act as an arbitrator or to protect a minority opinion.
This is of course the ideal, but I think that just doesn't work in practice. When tempers fray and people feel strongly, they will tend not to accept arbitration from any of the other active participants, or a self-designated outsider.
On Sat, Sep 5, 2015 at 12:46 AM, Nathaniel Smith email@example.com wrote:
I guess I just don't see how you think we can legislate ourselves into having a vision. Like... we'll elect a volunteer to produce a roadmap, and then if they don't we'll fire them (accountability!), leaving us with no volunteer and no roadmap? How will that help?
I agree it is not sensible for the president or moderator to produce a roadmap, but a roadmap is different from a vision.
We often run into trouble on the list, discussing whether to make backwards-incompatible changes. The reason for the trouble is that different people have different ideas about what numpy is. Some people think numpy is a legacy library who's job it is to tread water while waiting for a better alternative. For them, backwards compatibility is of overriding importance. Others think that numpy should be gradually refactored into a better and more extensible version that will continue to be the backbone of scientific computing into the fairly distant future. These are two different visions that are not compatible one with another. Trying to keep both groups happy is likely to lead to dissatisfaction and frustration from both sides.
Yep. And like you said above re: IPython and so forth, the solution is to confront that problem head-on and have the debate. Having a dictator come in and dictate the outcome wouldn't help anything.
Yes, I see I am not being clear. I was claiming that it is difficult to have this debate without someone who will make the final call. That person doesn't 'dictate' but they have the final say when all the arguments are in. If I am right, then, with the no-leader consensus model you propose, it will be hard to have that debate because it is harder to get to the final decision.
On Sat, Sep 5, 2015 at 12:46 AM, Nathaniel Smith firstname.lastname@example.org wrote:
I'm not sure how useful this focus on NetBSD is, given the elephant in the room: the fundamental challenge for NetBSD is that NetBSD has no compelling reason to exist;
That is now true, but I would argue (I think Hannum would argue) that this was the direct effect of very poor management, with no vision for where the project was going. I imagine you'd agree that OpenBSD does have a purpose, and OpenBSD started as a one-person-in-charge fork of NetBSD, where the fork was the result of very bad management by the 'core'.
I think you might be trying too hard to make history line up with your conclusions here :-/. Theo de Raadt didn't quit; he got kicked out in 1994 because they felt his "long history of rudeness towards and abuse of users and developers" finally outweighed his "long history of positive contributions":
I know that history very well, sorry that I did not unpack what I meant. I believe that the 'core' group made a terrible social mess of trying to deal with TdR's interaction style, in a characteristic 'core' way, which was that they ended up throwing him out with an officious email where no one member of the core prepared to accept the responsibility for doing that, and with no way back once they had done that. I would also argue that the 'core' way tends to a rather conservative and joyless social model that made it particularly difficult for them to deal with TdR.
But anyway - that was not my point. My point was that, in 1994, as NetBSD was losing direction, it was perfectly possible to fork it and take it in a new and fruitful direction, with the right governance and leader. Despite his later prescription, Hannum is arguing that the core structure, specifically, was at fault for preventing that happening in NetBSD and in other projects:
""" Much of this early structure (CVS, web site, cabal ["core" group], etc.) was copied verbatim by other open source (this term not being in wide use yet) projects -- even the form of the project name and the term "core". This later became a kind of standard template for starting up an open source project. [...] I'm sorry to say that I helped create this problem, and that most of the projects which modeled themselves after NetBSD (probably due to its high popularity in 1993 and 1994) have suffered similar problems. FreeBSD and XFree86, for example, have both forked successor projects (Dragonfly and X.org) for very similar reasons. """ [1[