After long conversations at BIDS this weekend and after reading the entire governance document, I realized that the steering council is very large and I don't agree with the mechanism by which it is chosen. A one year time frame is pretty short on the context of a two decades old project and I believe the current council has too few people who have been around the community long enough to help unstuck difficult situations if that were necessary. I would recommend three possible adjustments to the steering council concept. 1 - define a BDFL for the council. I would nominate chuck Harris 2 - limit the council to 3 people. I would nominate chuck, nathaniel, and pauli. 3 - add me as a permanent member of the steering council. Writing NumPy was a significant amount of work. I have been working indirectly or directly in support of NumPy continously since I wrote it. While I don't actively participate all the time, I still have a lot of knowledge, context, and experience in how NumPy is used, why it is the way it is, and how things could be better. I also work with people directly who have and will contribute regularly. I am formally requesting that the steering council concept be adjusted in one of these three ways. Thanks, Travis
On So, 2015-09-20 at 11:20 -0700, Travis Oliphant wrote:
Hmmm, well I never had the impression that the steering council would be huge. But maybe you are right, and if it is, I could imagine something like option 2, but vote based (could possibly dual use those in charge of NumFOCUS relations, we had even discussed this possibility) which would have final say if necessary (could mean that the contributers definition could be broadened a bit). However, I am not sure this is what you suggested, because for me it should be a regular vote (if just because I am scared of having to make the right pick). And while I will not block this if others agree, I am currently not comfortable with either picking a BDFL (sorry guys :P) or very fond of an oligarchy for live. Anyway, I still don't claim to have a good grasp on these things, but without a vote, it seems a bit what Matthew warned about. One thing I could imagine is something like an "Advisory Board", without (much) formal power. If we had a voted Steering Council, it could be the former members + old time contributers which we would choose now. These could be invited to meetings at the very least. Just my current, probably not well thought out thoughts on the matter. But neither of your three options feel very obvious to me unfortunately. - Sebastian
I wrote my recommendations quickly before heading on a plane. I hope the spirit of them was caught correctly. I also want to re-emphasize that I completely understand that the Steering Council is not to be making decisions that often and almost all activity will be similar to it is now --- discussion, debate, proposals, and pull-requests --- that is a good thing. However, there is a need for leadership to help unstick things and move the project forward from time to time because quite often doing *something* can be better than trying to please everyone with a voice. My concerns about how to do this judgment have 2 major components: 1) The need for long-term consistency --- a one-year horizon on defining this group is too short in my mind for a decades-old project like NumPy. 2) The group that helps unstick things needs to be small (1, 3, or 5 at the most) We could call this group the "adjudication group" rather than the "Steering Council" as well. I could see that having a formal method of changing that "adjudication group" would be a good idea as well (and perhaps that formal vote could be made by a vote of a group of active contributors. In that case, I would define active as having a time-window of 5 years instead of just 1). Thanks, -Travis On Mon, Sep 21, 2015 at 2:39 AM, Sebastian Berg <sebastian@sipsolutions.net> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On Mon, Sep 21, 2015 at 9:20 AM, Travis Oliphant <travis@continuum.io> wrote:
For reference, the rules for steering council membership were taken directly from those used by the Jupyter project, and their steering council currently has 10 people, making it larger than the "seed council" proposed in the numpy document: https://github.com/jupyter/governance/blob/master/people.md
We could call this group the "adjudication group" rather than the "Steering Council" as well. I could see that having a formal method of changing that "adjudication group" would be a good idea as well (and perhaps that formal vote could be made by a vote of a group of active contributors. In that case, I would define active as having a time-window of 5 years instead of just 1).
I may be misreading things, but I'm getting the impression that the active "adjudication group" you envision is radically different from the "steering council" as envisioned by the current governance document. It also, I think, radically different from anything I've ever seen in a functioning community-run FOSS project and frankly it's something where if I saw a project using this model, it would make me extremely wary about contributing. The key point that I think differs is that you envision that this "adjudication group" will actually intervene into discussions and make formal decisions in situations other than true irreconcilable crises, which in my estimation happen approximately never. The only two kinds of F/OSS projects that I can think of that run like this are (a) projects that are not really community driven at all, but rather run as internal company projects that happen to have a public repository, (b) massive projects like Debian and Fedora that have to manage literally thousands of contributors, and thus have especially robust backstop procedures to handle the rare truly irreconcilable situation. E.g., the Debian CTTE acts as an "adjudication group" in the way it sounds like you envision it: on a regular basis, irreconcilable arguments in Debian get taken to them to decide, and they issue a ruling. By some back of the envelope calculations, it looks like they issue approximately ~0.002 rulings per debian-contributor-year [1][2]. If we assume crudely that irreconcilable differences scale linearly with the size of a project, this suggests that a ~20 person project like NumPy should require a ruling ~once every 20 years. Or quoting myself from the last thread about this [3]: ] Or on the other end of things, you have e.g. Subversion, which had an ] elaborate defined governance system with different levels of ] "core-ness", a voting system, etc. -- and they were 6 years into the ] project before they had their first vote. (The vote was on the crucial ] technical decision of whether to write function calls like "f ()" or ] "f()".) These are two real projects and how they really work. And even in projects that do have a BDFL, the successful ones almost never use this power to actually "unstick things" (i.e., use their formal power to resolve a discussion). Consider PEP 484, Guido's somewhat controversial type hints proposal: rather than use his power to move the debate along, he explicitly delegated his power to one of the idea's strongest critics [4]. Of course, things to get stuck. But the only time that getting them unstuck needs or even benefits from the existence of a formal "unsticking things" group is if the group is actually using some powers that they have. In 99.9% of cases, though, the correct way to get things unstuck is for a new idea to be introduced, or for someone respected to act as a mediator, or for someone to summarize the situation and isolate some core of agreement that allows for forward progress. But all of *these* things can and should be done by anyone and everyone who can contribute them -- restricting them to a small "adjudication group" makes no sense. In terms of real-world examples: From my point of view, the worst parts of the NA fiasco was caused by the decision to cut off debate with a "ruling". (Specifically, that instead of working on implementing bitpattern-NAs -- which did have consensus -- then Mark would go off and work on the masked-NA strategy, and eventually that it should be merged, despite it not having consensus. I note that dynd supports only bitpattern-NAs.) OTOH, probably the most difficult technical debate we've had recently is issue 5844, about the potential interactions between __binop__ methods and __numpy_ufunc__, and this is a debate that AFAICT certainly would not have benefited from the existence of an adjudication body. As it happens, all three of your proposed adjudication-body-members actually are in the debate anyway; if they hadn't been, then the very last thing that debate needs is someone wading in without any understanding of the complex technical issues and trying to force some resolution. I'm not saying that truly irreconcilable problems never arise. But given that they're so rare, and that realistically any system that could solve them would need to be backed by basically the same group of people that are in the current governance document's "steering council" (because in an OSS project it's the people doing the work that ultimately have the power), there's no point in building an elaborate voting system and method for defining the electorate just to handle these cases -- and quite a bit of harm. So the proposed system is basically the simplest one that could work. (I described this line of reasoning in more detail in [3].) I think it would really help if you could provide some examples of successful community-driven projects and/or specific debates that benefited from having an adjudication body like you envision? I also suspect this fundamental difference in how we view the role of a governance body explains why I am unbothered by the idea of steering council membership "timing out" after 2 years of inactivity. The steering council's job is to informally manage things day to day, and in extreme cases to make judgement calls. These two things both crucially require a broad and up-to-date understanding of the issues facing the project, ongoing debates, personalities, etc. But the council is never intended to make any judgement call on its own; the whole idea of the structure is to make sure that decisions are based on as broad a scope of expertise as possible. In particular we regularly get historical insight from Chuck, Ralf, Pauli, Robert Kern, David Cournapeau, Pearu Peterson, Anne Archibald, Matthew Brett, Stefan van der Walt... and it doesn't matter at all whether they're on the council or not. (If it did matter, that would be terrible, right? Why would you want to *not* listen to any of those people?) If you plan to become more active and give your perspective on things more then that's awesome and welcome, but AFAICT this particular point is pretty orthogonal to the composition of the steering council. See also this comment from Fernando about the Jupyter steering council, which happened by chance to cross my email this morning: https://github.com/jupyter/governance/pull/6#issuecomment-142036050 -n [1] https://www.debian.org/devel/tech-ctte#status [2] https://contributors.debian.org/ [3] https://mail.scipy.org/pipermail/numpy-discussion/2015-September/073537.html [4] See e.g. the bottom of this message from Nick Coghlan, where he talks about how PEP "pronouncements" are done -- emphasizing in particular (my paraphrase) that the point of a BDFL/BDFL-delegate is not to resolve any substantive issues, and that a common strategy is to choose the person who's most skeptical of the idea to be the BDFL-delegate: http://thread.gmane.org/gmane.comp.python.distutils.devel/23867 -- Nathaniel J. Smith -- http://vorpus.org
I actually do agree with your view of the steering council as being usually not really being needed. You are creating a straw-man by indicating otherwise. I don't believe a small council should do anything *except* resolve disputes that cannot be resolved without one. Like you, I would expect that would almost never happen --- but I would argue that extrapolating from Debian's experience is not actually relevant here. So, if the steering council is not really needed then why have it at all? Let's just eliminate the concept entirely. But there are real questions that have to have an answer or an approach to making a decision. The answer to these questions cannot really be a vague notion of "lack of vigorous opposition by people who read the mailing list" which then gets parried about as "the community decided this." The NumPy user base is far, far larger than the number of people that read this list. For better or for worse, we will always be subject to the "tyranny of who has time to contribute lately". Fundamentally, I would argue that this kind of "tyranny" should at least be tempered by additional considerations from long-time contributors who may also be acting more indirectly than is measured by a simple git log. So, what are the questions that have to have an answer that even calls for some kind of governance? I know Nathaniel's document listed some of them (and perhaps all of them). Here are mine: 1) who gets commit rights to the repo (and who has them removed)? 2) who tags the release of NumPy? 3) how is it decided where money is spent if it's donated to the project (and not just to people directly)? 4) how is it decided if someone needs to be removed from participation in the group because they are not adding to the conversation (we have been fortunate that this hasn't happened in NumPy before --- but it could)? 5) how is it decided what goes on the NumPy website --- i.e. will advertisers be able to put their logos or book-covers there? If I understand what you are proposing, then basically the "steering council" decides these things. Perhaps rather than a steering council, though, we just need clear answers to questions like the above --- which might be handled differently for different questions. I don't think these questions have very easy answers. Ultimately NumPy has relied on and continues to rely on the mutual respect of all the people that have worked on the code and tried to make it better. We all have opinions about how things have gone in the past, and what has gone well and what hasn't. But, nothing you have said persuades me that you have a full picture of past history with respect to a lot of the difficult kinds of conversations that have happened and the different modes of activity that have tried to help move NumPy along. In fact, I think you mis-understand and mis-interpret that history quite often. I'm convinced you are well-intentioned and doing the very best you can, and I'm very grateful that you are passionate and eager about moving NumPy forward. Ultimately I hope it will help things. Here is my attempt at a proposal for how to answer the above questions: 1) who gets commit rights to the repo (and who has them removed)? * people who contribute regularly are granted commit rights by another committer with at least two additional nominations and the lack of a veto within 1 week of the proposal. * nobody has commit rights removed except by unanimous consent of all the other committers (with consent being implied if not responded to within 2 weeks). 2) who tags the release of NumPy? * whomever volunteers to be release manager and if there is no veto from committers. 3) how is it decided where money is spent if it's donated to the project (and not just to people directly)? * three people who self-select represent NumPy to Numfocus following the rules of Numfocus (that there is only one representative from any organization). * If 3 committers oppose one of those people and nominate another in place, then that person is replaced. 4) how is it decided if someone needs to be removed from participation in the group because they are not adding to the conversation (we have been fortunate that this hasn't happened in NumPy before --- but it could)? * unanimous consent of all committers (with a 2 week period given for consent to be given --- and it is assumed given if they are not heard from). 5) how is it decided what goes on the NumPy website --- i.e. will advertisers be able to put their logos or book-covers there? * only Numfocus can advertise and put their logo on the website Now, I'm sure one can poke holes in the above --- and I would welcome better answers to the above questions. Perhaps we should just decide how specific decisions get made and make a document that lists that and only talks about committers instead of inventing another "bit" to differentiate people in the community. -Travis On Tue, Sep 22, 2015 at 2:11 AM, Nathaniel Smith <njs@pobox.com> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On Tue, Sep 22, 2015 at 1:24 AM, Travis Oliphant <travis@continuum.io> wrote:
To be clear, Debian was only one example -- what I'm extrapolating from is every community-driven F/OSS project that I'm aware of. It's entirely possible my data set is incomplete -- if you have some other examples that you think would be better to extrapolate from, then I'd be genuinely glad to hear them. You may have noticed that I'm a bit of an enthusiast on this topic :-).
So, if the steering council is not really needed then why have it at all?
Let's just eliminate the concept entirely.
In my view, the reasons for having such a council are: 1) The framework is useful even if you never use it, because it means people can run "what if" scenarios in their mind and make decisions on that basis. In the US legal system, only a vanishingly small fraction of cases go to the Supreme Court -- but the rules governing the Supreme Court have a huge effect on all cases, because people can reason about what would happen *if* they tried to appeal to the Supreme Court. 2) It provides a formal structure for interfacing with the outside world. E.g., one can't do anything with money or corporate contributions without having some kind of written-down and enforceable rules for making decisions (even if in practice you always stick to the "everyone is equal and we govern by consensus" part of the rules). 3) There are rare but important cases where discussions have to be had in private. The main one is "personnel decisions" like inviting people to join the council; another example Fernando has mentioned to me is that when they need to coordinate a press release between the project and a funding body, the steering council reviews the press release before it goes public. That's pretty much it, IMO. The framework we all worked out at the dev meeting in Austin seems to handle these cases well AFAICT.
According to the dev meeting rules, no particularly "vigorous opposition" is required -- anyone who notices that something bad is happening can write a single email and stop an idea dead in its tracks, with only the steering council able to overrule. We expect this will rarely if ever happen, because the threat will be enough to keep everyone honest and listening, but about the only way we could possibly be *more* democratic is if we started phoning up random users at home to ask their opinion. This is actually explicitly designed to prevent the situation where whoever talks the loudest and longest wins, and to put those with more and less time available on an equal footing.
I guess I am missing something fundamental here. Who are these long-time contributors who will sit on your council of 1/3/5 but who don't even read the mailing list? How will they know when their special insight is necessary?
No, absolutely not. The proposal is that these issues are decided by open discussion on the mailing list. (With the possible exception of #4 -- I suspect that given an extreme situation like this, then once all other efforts to mitigate the situation had failed the steering council would probably feel compelled to talk things over to double-check they had consensus before acting, and likely some part of this would have to be done in private.) This is pretty explicit in the document (and again, this is text and ideas stolen directly from Jupyter/IPython): "During the everyday project activities, council members participate in all discussions, code review and other project activities as peers with all other Contributors and the Community. In these everyday activities, Council Members do not have any special power or privilege through their membership on the Council. [...] the Council may, if necessary [do pretty much anything, but] the Council's primary responsibility is to facilitate the ordinary community-based decision making procedure described above. If we ever have to step in and formally override the community for the health of the Project, then we will do so, but we will consider reaching this point to indicate a failure in our leadership." If this is unclear then suggestions for improvement would certainly be welcome. This is pretty much how we do things now, and it has worked pretty well so far -- people do get commit bits, releases get tagged, and everyone seems happy with this part. I'd actually like to see a more explicit set of rules around e.g. who gets commit bits, just to set expectations and provide a clearer onramp for new contributors, but that's something that we can easily discuss and make a decision on later. All of these questions are ones that are easy enough to solve given some framework for governance, but they do all require substantive discussion. We're not going to do them justice if we try to solve them off-the-cuff in this thread, and trying would just derail the underlying discussion about how we make decisions. So I think we should stay focused on that.
I'm honestly somewhat unclear on why having a "full picture of past history" is necessary to contribute effectively (surely if that were the case, then only Jim Hugunin could comment?), but if there are things I'm missing then I'd certainly like to know. You've made comments along these lines several times now, but unfortunately I can't do much to improve my understanding without more concrete examples. I'm convinced you are well-intentioned and doing the very best you can, and
I'm very grateful that you are passionate and eager about moving NumPy forward. Ultimately I hope it will help things.
Thank you. I am also convinced that you're well-intentioned and doing the very best you can, and grateful that you are also passionate and eager about moving NumPy forward. -n -- Nathaniel J. Smith -- http://vorpus.org
On Tue, Sep 22, 2015 at 4:33 AM, Nathaniel Smith <njs@pobox.com> wrote:
Yes, you are much better at that than I am. I'm not even sure where I would look for this kind of data.
O.K. That is a good point. I can see the value in that.
O.K.
O.K.
How did we "all" work it out when not everyone was there? This is where I get lost. You talk about community decision making and yet any actual decision is always a subset of the community. I suppose you just rely on the "if nobody complains than it's o.k." rule? That really only works if the project is moving slowly.
O.K. so how long is the time allowed for this kind of opposition to be noted?
The long-time contributors wouldn't necessarily sit on that council. But, I would support the idea of an advisory council that could act if it saw things going the wrong direction. This is where those people would sit. In the case of a 1 / 3 / 5 member council -- I would not argue to be on it at all (but I would argue that some care should be taken to be sure it has people with some years of experience if they are available). I'm only asking to be on a steering council that is larger than 5 people, and I don't actually prefer that the steering council be larger than 5 people.
O.K. Then, I am misunderstanding.
Granting commit rights to the repo does not seem to me to be an "everyday project activity" --- so I suppose I was confused. I suppose that could happen with no serious objections on the list. It seems that the people who actually have the commit bit presently should be consulted more than the general public, though.
O.K. I think that the commit bit has not been that clear --- and was closer when I was more active to the commiters rule I gave before.
The reason is that you in particular make very long arguments that try to persuade on the basis of your understanding of how things were done in the past (both here and in other projects). If you don't understand the history and the arguments that have been had before, you are throwing away information. It's not a show-stopper, for sure, but it does potentially make it less likely that a better solution will be found. I will have to respond to your incorrect characterizations in another thread. -Travis
On Sun, Sep 20, 2015 at 11:20 AM, Travis Oliphant <travis@continuum.io> wrote:
After long conversations at BIDS this weekend and after reading the entire governance document, I realized that the steering council is very large
How large are we talking? I think there were 8 people named -- and I'm not sure all 8 want to do it. But 5 or seems like a fine number -- I wonder if defining the number is a good idea.
A one year time frame is pretty short on the context of a two decades old project
I see: """ who have produced contributions that are substantial in quality and quantity, and sustained over at least one year """ that could be longer, but it looks like it DOESN'T mean "have contributed within the last year" -- so we could certainly, at this point, have a lot of folks that have been around a long time that could qualify. A certain guy named Travis come to mind... and I believe the current council has too few people who have been around
the community long enough to help unstuck difficult situations if that were necessary.
That's actually a bad sign right there -- the suggested group are the folks that have actually been contributing lately -- too bad that apparently isn't anyone that has been around a long time :-( also see: """ This will include but is not limited to code, code review, infrastructure work, mailing list and chat participation, community help/building, education and outreach, design work, etc. """ so it's not only people doing the actual coding ( A really small number :-( ) But I note that the the suggestion to "seed the council" with "everyone who has reviewed/merged a pull request since Jan 1, 2014" - I'm not sure that gives us as diverse a council as we'd like -- but it is only the seed...
1 - define a BDFL for the council.
Well, that would make a number of things easier -- but I think there was a consensus that there simply was no obvious candidate -- and if the candidate is not obvious, then they can't really be a BDFL.
2 - limit the council to 3 people. I would nominate chuck, nathaniel, and pauli.
Good group -- but maybe 3 a bit too small.
3 - add me as a permanent member of the steering council.
I'd love to see you on the council, and there is no question of your qualifications -- but I don't think anyone should be anymore permanent than any other. According to the docs so far, the only thing anyone needs to do to remain on the council is stay active -- and not piss everyone off so much as to get a consensus from everyone else that you need to be kicked off. And the end of this -- I think the big question still at hand is how big the council should be. My experience with consensus suggests that it not be very big :-) -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Hi Travis, and all, You might have seen I was advocating for having someone who takes final responsibility for the project, partly to get discussions unstuck, as you said. I agree with Chris, that at this stage, there is no-one who could be Benevolent Dictator for the project. It seems to me that (nearly) everyone would agree that, if there is a leader, we would not want a person who dictates, but someone who is good at putting in the hard work and time it takes to hear all the arguments, and guide people to agreement, where that is possible. If there is a leader, I think we should select them for those skills. For the proposal that you join the steering committee, I see two problems. The first is, that the members of the steering committee have to be the people who are keeping in touch with the day to day work of the project. I am sure you would agree that, in the past, you have not had enough time for that. Your recent emails about the new work you want to do, also imply that you may be too busy to get involved in the detailed discussions needed for getting the code merged. In any case, I think you'd also agree that in the past you have hoped that you would have more time for numpy than you did. The second problem is that you have a potential conflict of interest, in that it is possible for the needs of Continuum to conflict with the needs of numpy. I believe, from previous emails on this list, that you don't think that is very important, but I continue to disagree about that. For example, see this interview with Linus Torvalds, where he talks about going out of his way to make sure that people can trust his decisions, despite the fact he is paid to work on Linux [1]. In practice, the most obvious step that I can think of, is to defer the decision as to whether you are on the steering committee for six months. I guess over that time you will be working with the other numpy developers more closely. I should say that Stefan and I are working on a governance proposal, in this case for scikit-image, where we split the governance into 1) the developers doing the work and making the day to day decisions and 2) the trustees, usually from other relevant projects, or no-longer-active developers, who make sure the project does not go off the rails. I think you'd be an excellent and obvious trustee, in that model. Cheers, Matthew [1] http://www.bbc.com/news/technology-18419231
Specific examples to support that claim would be appreciated. In particular, examples where an OSS project was corrupted (is that the word?) by a company specifically at the hand of the project's original creator would be especially relevant. Beyond that, what (even in a broad sense) is an example of a goal that "Continuum might need" that would conceivably do detriment to the NumPy community? That it be faster? Simpler to maintain? Easier to extend? Integrate better with more OS projects? Attract new active developers? Receive more financial support? Grow its user base even more? And then, even if there is some sliver of daylight to be uncovered between Travis and the community, what is the current organizational mechanism by which a problematic contribution is forced into NumPy against the will of all other core developers? Finally, is there any previous instance whatsoever that can be pointed to of Travis forcing some change into NumPy contrary to the interest of the community? If not, what actual basis is there to exclude him from a steering committee? I obviously have a point of view; but my opinion here is my own, which is: this sort of pre-emptive impugning of someone's integrity is speculation upon speculation upon speculation Bryan
On 2015-09-21 22:15:55, Bryan Van de Ven <bryanv@continuum.io> wrote:
I don't know how productive it is to dream up examples, but it's not very hard to do. Currently, e.g., the community is not ready to adopt numba as part of the ufunc core. But it's been stated by some that, with so many people running Conda, breaking the ABI is of little consequence. And then it wouldn't be much of a leap to think that numba is an acceptable dependency. There's a broad range of Continuum projects that intersect with what NumPy does: numba, DyND, dask and Odo to name a few. Integrating them into NumPy may make a lot more sense for someone from Continuum than for other members of the community. Stéfan
I don't know how productive it is to dream up examples, but it's not
Well, agreed, to be honest.
very hard to do. Currently, e.g., the community is not ready to adopt numba as part of the ufunc core. But it's been stated by some that,
Who are you speaking for? The entire community? Under what mandate?
The current somewhat concrete proposal I am aware of involves funding cleaning up dtypes. Is there another concrete, credible proposal to make Numba a dependency of NumPy that you can refer to? If not, why are we mired in hypotheticals?
May? Can you elaborate? More speculation. My own position is that these projects want to integrate with NumPy, not the converse. Regardless of my opinion, can you actually make any specific arguements, one way or the otehr? What if if some integrations actually make more sense for the community? Is this simply a dogmatic ideological position that anything whatsoever that benefits both NumPy and Continuum simultaneously is bad, on principle? That's fine, as such, but let's make that position explicit if that's all it is. Bryan
Hi Brian On 2015-09-21 23:28:48, Bryan Van de Ven <bryanv@continuum.io> wrote:
I am speaking on behalf of myself, under no mandate.
I'm sorry if I misunderstood, but I thought you wanted us to explore hypothetical confictual situations.
No, I don't have such a dogmatic ideological position. I think, however, that it is somewhat unimaginative to propose that there are no potential conflicts whatsoever. I am happy if we can find solutions that benefit both numpy and any company out there. But in the end, I'm sure you'd agree that we want the decisions that lead to such solutions to be taken in the best interest of the project, and not be weighed by alterior motivations of any sorts. In the end, even the *perception* that that is not the case can be very harmful. Stéfan
I will only comment on the last point. I completely agree that the *perception* that this is not the case can be harmful. But, what concerns me is where this perception comes from --- from actual evidence of anything that is not in the best interests of the project --- or just ideological differences of opinion about the way the world works and the perceptions around open source and markets. It is quite easy for someone to spread FUD about companies that contribute to open source --- and it has the effect of discouraging companies from continuing to contribute to community projects. This removes a huge amount of potential support from projects. In NumPy's case in particular, this kind of attitude basically guarantees that I won't be able to contribute effectively and potentially even people I fund to contribute might not be accepted --- not because we can't faithfully participate in the same spirit that we have always contributed to SciPy and NumPy and other open source projects --- but because people are basically going to question things just because. What exactly do you need me to say to get you to believe that I have nothing but the best interests of array computing in Python at heart? The only thing that is different between me today and me 18 years ago is that 1) I have more resources now, 2) I have more knowledge about computer science and software architecture and 3) I have more experience with how NumPy gets used. All I can do is continue to try and make things better the best way I know how. -Travis -- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
Hi Bryan, I understand where you're coming from, but I'd appreciate it if we could keep the discussion on a less visceral level? Nobody's personal integrity is being impugned, but it's the nature of this kind of governance discussion that we have to consider unlikely-and-unpleasant hypotheticals. It's like when you talk to a lawyer about a contract or a will or whatever: they'll make you think about all kinds of horrible possibilities, not because any of them are likely, but because sooner or later *something* will go wrong, and the point of having a contract/will/governance document is to provide some framework to handle whichever unlikely edge case does arise. And the other purpose of this kind of framework is to avoid the *perception* (whether justified or not) of these kinds of conflicts of interest -- if not handled well then this can easily scare away volunteers, contributions from other companies, etc. Obviously you know Travis and Continuum well as an employee there, but to most people without that personal connection, Continuum is just a large corporate entity with unknown motives. Imagine if instead of Continuum we were talking about it was Google or RandomAggressiveStartup -- some company that you didn't have any personal connection or insight into. For someone in this position, it's not unreasonable to want more of a reassurance that things will work out then just being asked to trust that the CEO is personally committed to not being evil and they can trust him. Also, in these messages you seem to be working from a framework where people working in good faith will always agree, and so any suggestion of a possible conflict of interest can only arise through bad faith. But this isn't true. Is it really so difficult to imagine that, say, Continuum and Enthought might at some point have conflicting needs for numpy, or for Continuum's vision of the future of numpy could be less-than-perfectly-representative with every bit of numpy's entire giant userbase? Continuum is a company that has in the past submitted rather obscure patches to numpy that AFAICT are used exclusively by a particular contracting customer (e.g. [1]), and that is currently investing a substantial multiple of numpy's development budget on building a direct competitor to numpy. To emphasize: I personally am not concerned by these facts -- we did merge that patch I linked, and there's no animosity between the numpy and dynd teams -- but reasonable people could reasonably be concerned that tricky situations might emerge in the future, and I've talked to plenty of people who are nervous about Continuum's influence in general. And with my numpy developer hat on, I don't even care which "side" is right, that's irrelevant to me, because my concern is with providing a space where both "sides" feel comfortable working together. This is why it's crucial that numpy-the-project be clearly distinguishable as an independent entity that is beholden only to its own community: it's *exactly because* we *value* the contribution of companies like Continuum, and want to be able to freely foster those relationships without creating suspicion and bad blood. Also to emphasize: none of this means that Travis can't be on the steering council -- I think that's a more complex issue that I'll address separately. All I'm saying is that pretending that you aren't going to reassure people by pretending this elephant isn't in the room, or by taking a reasonable set of concerns and aggressively turning them into a referendum on individual people's integrity. Finally, can I point out... anyone who has some wariness around the possible influence of financial interests on the community (whether justified or not!) is in particular not going to be reassured if you keep aggressively attempting to shut down any perceived criticism of *your own employer*. I know that your paycheck is not dictating your opinions, and probably the hypothetical people I'm talking about are being totally unfair to you for even considering such a thing, but... strictly as a matter of practical rhetoric, I don't think this is the most effective way to accomplish your goals. -n [1] https://github.com/numpy/numpy/pull/359 On Mon, Sep 21, 2015 at 11:28 PM, Bryan Van de Ven <bryanv@continuum.io> wrote:
-- Nathaniel J. Smith -- http://vorpus.org
On Tue, Sep 22, 2015 at 2:33 AM, Nathaniel Smith <njs@pobox.com> wrote:
Anybody who comes to NumPy should know that I wrote it and then gave it to the community -- creating an independent foundation at the same time as I created a Company to create a division between them so that my actions could be understood. I really don't know how to give more reassurance. I'm actually offended that so many at BIDS seem eager to crucify my intentions when I've done nothing but give away my time, my energy, my resources, and my sleep to NumPy for many, many years. I guess if your intent is to drive me away, then you are succeeding.
Of course not, but this is no different than anyone else and a company should not be singled out. All you are doing is forcing any contribution to be either only from a non-profit or have individuals hide their actual allegiances.
Good grief! These comments are so much bunk that I have to call you on it emphatically. You claim below that you are unconcerned but yet you insinuate some crazy, ulterior motivations for Continuum actually helping people upgrade their NumPy installation that broke their old code because of changes to NumPy. This kind of comment really upsets me. You dismiss real things and real problems that happen and brush it away because it's *commercial*. That patch you reference was actually an attempt to fix a problem that the community pushed on the world --- therefore breaking people's code (but good thing the ABI didn't change...). We were up front about it and worked with the community to get a change into NumPy to accommodate a user of the tool. It was hard work to figure out how to do that. To have you use that as some kind of argument against Continuum is not only unfair, it is exactly the kind of mis-characterization and mis-interpretation of events that I refer to in other posts. And to say that we are investing a substantial multiple of Numpy's development budget in a competitor is also incorrect. Continuum invests in competent people who want to do interesting things. We don't have a rule that says things are "off-limits" including NumPy. If competent people feel like an alternative to NumPy is a good idea, then a certain amount of exploration in that direction is allowed. DyND does not have to be a competitor to NumPy. It might compete with *your* vision of NumPy, but it doesn't have to compete with NumPy.
Who are these people? How about they come forward and express what it is they are actually nervous about. Really? nervous? What kind of "tricky" situations are we talking about. Can't you see that this sounds as odd to me as me telling you that I'm concerned about BIDS influence? What about Dato, Databricks, Enthought, or Cloudera influence? What does this even mean? Is this just Matthew and Stefan or are there others as well with these feelings? These are the only actual people who have expressed in public what might be considered concern that I am aware of. I think this kind of anti-commercial commentary has no place in a community that also includes people that work at companies. I can't see how we can agree to *any* governance document with this kind of FUD being spread around.
Of course we agree on this. I have no idea why anyone thinks we don't? That's the one thing we *do* agree on. That NumPy is an independent project which can be influenced by anyone in the community and should be developed based on technical discussions and not fear of hob-goblins and people that also work at companies that may benefit from the work that goes on here (there is a large list in this camp). I am deeply saddened by the insinuation and the implication of what these threads are telling me about how little my efforts have been valued by people I care about.
We should call out the elephant in the room. But, I think we should understand who and what the elephant is. Perhaps there are too many off-list and back-channel conversations happening at BIDS and elsewhere that are serving to bias people against facts. Facts that otherwise would show that I and Continuum have always just been trying to ensure the success of NumPy as an independent project that is fully supported, backward compatible, maintained, available to the world in easy to install ways, and documented.
I agree with this. I certainly did not encourage Bryan. He was acting out of his own sense of injustice. But, I would add, that your insinuation and mis-characterization of my activities at Continuum and potentially elsewhere are unfair and incorrect and also not effective at getting governance documents approved and ratified. I will get over feeling offended and work to get over my frustration at academics for thrusting this anti-company and potentially anti-Travis rhetoric on the community. But, if you indeed have such hard feelings, then please air all of them so we can hopefully just get past this.
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On Di, 2015-09-22 at 05:44 -0500, Travis Oliphant wrote:
Frankly, I am bit surprised at how this is developing. Why?! Nobody, except being silly, really would doubt your intentions. That said, I will be honest with you. I do not feel I know you well enough (or the mode you operate) to for example accept you as BDFL of numpy [1] and I frankly do not think that -- just like anyone else -- you should have any special place in the governance document (which obviously does not mean you should be blocked from being on the steering council) [2]. I just think there should probably not be any specific name in the NumPy governance document at all. The thing is, NOBODY really seems suggests that [3]. I have dislike giving any special "power" to ANYONE. Frankly, I think if some people are nervous (not so much those active in the discussion), it is probably because they perceive that you have some direct power over numpy. As you have said, this is just not true. But *because* of the lack of a governance document, I would not be surprised if it *appears* to many like you could wield a lot of power if you so wished. Simply because few people really know how things are decided currently. And I think this wrong perception is what makes some nervous. If you say "maybe we should stop worrying about ABI" it may sound like "two years from now we will definitely break ABI", the curse being that it does not matter that you and those who know numpy's well know that it is just you stressing strongly that we should seriously discuss it. I have to admit, you sometimes sound a bit too definite in your suggestions given your former position ;). I hope I have not been rude here, Sebastian [1] I am sure you were *exactly* the right person to start numpy and be the de-facto BDFL then, but today this is not on the table anyway. [2] Also, lets be honest, you probably do have quite a bit of soft influence, just by knowing the community, being at the conferences, having NumFOCUS close by, etc. [3] I guess you have in some sense, but I now understand that as a suggestion to approach a different problem. And we can find another solution for that problem!
Hi Travis On 2015-09-22 03:44:12, Travis Oliphant <travis@continuum.io> wrote:
I guess we've gone off the rails pretty far at this point, so let me at least take a step back, and make sure that you know that: - I have never doubted that your intensions for NumPy are anything but good (I know they are!), - I *want* the community to be a welcoming place for companies to contribute (otherwise, I guess I'd not be such a fervent supporter of the scientific eco-system using the BSD license), and - I love your enthusiasm for the project. After all, that is a big part of what inspired me to become involved in the first place. My goal is not to spread uncertainty, fear nor doubt—if that was the perception left, I apologize. I'll re-iterate that I wanted to highlight a concern about the interactions of a (somewhat weakly cohesive) community and strong, driven personalities such as yourself backed by a formidable amount of development power. No matter how good your intensions are, there are risks involved in this kind of interaction, and if we fail to even *admit* that, we are in trouble. Lest the above be read in a negative light again, let me state it up-front: *I don't think you will hijack the project, use it for your own gain, or attempt to do anything you don't believe to be in the best interest of NumPy.* What I'm saying is that we absolutely need to move forward in a way that brings everyone along, and makes everyone rest assured that their voice will be heard. Also, please know that I have not discussed these matters with Nathaniel behind the scenes, other than an informal hour-long discussion about his original governance proposal. There is no BIDS conspiracy or attempts at crucifixion. After all, you were an invited guest speaker at an event I organized this weekend, since I value your opinion and insights. Either way, let me again apologize if my suggested lack of insight hurt people's feelings. I can only hope that, in educating me, we all learn a few lessons. Stéfan
To expand on Ryan's point a bit about recusal... this is why we have a general policy against self-merging and why peer review is so valuable. A ban on self-merging is much like recusal, and I think it is a fantastic policy. As for a BDFL, I used to like that idea having seen it work well for Linux and Python, but I have found it at odds with the policy of recusal and no self-merging. That said, as I am sure Thomas Caswell can attest, a non-self-merging policy can result in a lot of ideas getting stalled, waiting for review that may or may not happen. I don't know what the solution is, but I am sympathetic to those who are apprehensive about a BDFL -- regardless of who is in that role. Ben Root On Tue, Sep 22, 2015 at 2:20 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
I completely agree. I don't think self-merging is a good idea. I had gotten used to self merging to SciPy and NumPy until roughly 2008 or 2009 when I was finally broken of that habit after stepping on a few people's toes and learning better habits. On Tue, Sep 22, 2015 at 1:48 PM, Benjamin Root <ben.v.root@gmail.com> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
Hi, On Tue, Sep 22, 2015 at 11:20 AM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
I'm also in favor of taking a step back. The point is, that a sensible organization and a sensible leader has to take the possibility of conflict of interest into account. They also have to consider the perception of a conflict of interest. It is the opposite of sensible, to respond to this with 'how dare you" or by asserting that this could never happen or by saying that we shouldn't talk about that in case people get frightened. I point you again to Linus' interview [1]. He is not upset that he has been insulted by the implication of conflict of interest, he soberly accepts that this will always be an issue, with companies in particular, and goes out of his way to address that in an explicit and reasonable way. Cheers, Matthew [1] http://www.bbc.com/news/technology-18419231
I am not upset nor was I ever upset about discussing the possibility of conflict of interest. Of course it can be discussed --- but it should be discussed directly about specific things --- and as others have said it is generally easily handled when it actually could arise. The key is to understand affiliations. We should not do things in the community that actually encourage people to hide their affiliations for fear of backlash or bias. I was annoyed at the insinuation that conflict of interest is a company-only problem that academics are somehow immune to. I was upset about accusations and mis-interpretations of my activities and those of my colleagues in behalf of the community. On Tue, Sep 22, 2015 at 1:48 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
Of course, and the policies to deal with conflicts have deal with the possibility that *anyone* at all might have conflict. But that was not your suggestion. Your suggestion was to make one individual be subject to additional scrutiny that no one else would be subject to. Please explain why should one person be singled out for a special "six-month waiting period" when the exact same possibility for conflict exists with anyone who is ever on the committee?
Red herring. The objection (as many people have now voiced) is the double standard you proposed.
Your selective quotation is impressive. Also in that interview, Linus points out that his employment contract is probably "unique in the entire world". Which means in practical terms that the details of what he has does are fairly well irrelevant to any other situation. So what is the point in bringing it up, at all, except to try and diminish someone else by comparison? (I'm done.) Bryan
On Tue, Sep 22, 2015 at 8:55 PM, Bryan Van de Ven <bryanv@continuum.io> wrote:
Andrew Morton would be a prominent counter example in the Linux community. He is employed by Google. Many other Linux developers are employed by Red Hat, Intel, and other companies. At this point Linus relies on those people for decisions on what goes into the kernel; it is much too big for a single person to deal with even in review. I also expect the Linux community would be overjoyed if every company provided drivers for their hardware. Subject to review, of course. Chuck
On Tue, Sep 22, 2015 at 10:55 PM, Bryan Van de Ven <bryanv@continuum.io> wrote:
I don't quite understand where the discussion went. The question was not whether Travis is singled out but whether he is "singled in".
Based on all the comments, Travis doesn't have time for this. And I think the final (last resort) decisions about code should be left to the active developers that know and participate in the actual work. If Travis is treated as developer but doesn't have a special status, then there is also no reason to "single out" him and Continuum for possibly too much influence. This is already the current status quo as it developed over the last several years, AFAICS. In my view, in a narrow sense, Travis is a "hit and run" contributor. good ideas and providing good contributions, but somebody has to integrate it, maintain it and fit it into the development plan. (Given my experience I would compare it more with GSOC contributions that need the "core developers" to provide the continuity in the development to absorb the good work.) Travis has too many other obligations and interests to provide this day to day continuity. Travis is still very important for providing ideas, pushing projects forward and as one of the community leaders, but I would leave the final decisions for the development of numpy to the developers in the trenches. I pretty much agree completely with Nathanial, and Sebastian, (except that I don't know much about any other FOSS communities) And to repeat my point from another thread on this: I'm very skeptical about any committee or board that is based on "outside members" and that is involved in the decision about code development. Josef
On Tue, Sep 22, 2015 at 1:20 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
Thank you for the clarification. I'm sorry that I started to question your intentions. I agree that everyone should rest assured that their voice will be heard. I have been and continue to be a staunch advocate for the voices that are not even on this mailing list.
Thank you. I'm sorry for implying otherwise. That was wrong of me. I know we are just trying to bring all the voices to the table. Best, -Travis
Hi all, I would like to pitch in here, I am sorry that I didn't have the time before... First, I want to disclose that recently Continuum made a research gift to the Jupyter project; we were just now writing up a blog post to acknowledge this, but in light of this discussion, I feel that I should say this up front so folks can gauge any potential bias accordingly. On Tue, Sep 22, 2015 at 3:44 AM, Travis Oliphant <travis@continuum.io> wrote:
Travis, first, I'd like to kindly ask you not to conflate BIDS, an institution where a large number of people work, with the personal opinions of some, who happen to work there but in this case are speaking only for themselves. You say "so many at BIDS", but as far as I know, your disagreements are with Stefan and Nathaniel (Matthew doesn't work at BIDS). You are painting with a very wide brush the work of many people, and in the process, unfairly impacting others who have nothing to do with this. If anything, the only things I'm aware BIDS has done in an official capacity regarding you or Continuum is to offer hosting for Continuum developers at the DS4DS workshop and beyond (after an explicit request by Matt Rocklin, and one we were delighted to honor), and our hosting of your lecture in our Friday Data Science Lecture Series last week. With that out of the way... 1. I hope the discussion can move past the suspicion and innuendo about Continuum and Travis. I haven't always agreed with how Travis communicates some of his ideas, and I've said it to him in such instances (e.g. this weekend, as I myself was surprised at how his last round of comments had landed on the list a few days back). But I also have worked closely with him for years because I know that he has proven, not in words, but in actions, that he has the best interests of our community at heart, and that he is willing to try and do everything in his power to help whenever he can. When we founded Numfocus back in 2012, it would have been impossible for it to really bootstrap without Travis' generosity, since he effectively footed the bill for resources that were critically needed at the start. And yet, he was always willing to take every step necessary to help Numfocus grow independent of Continuum, so that it could be a real community asset: today, there's not a single Continuum employee on the NF board (Travis and I both resigned from the board a while back to allow for some fresh blood). The creation and open-sourcing of conda has also been a critical contribution, that I know many of us have benefited from: we all carry the scars from the python packaging horror shows, and conda/anaconda has been a life-changer. The fact that conda itself is open, means we have a core tool that we can build upon. To put it bluntly, few people in the whole world have given more of their life, energy and resources to our community than Travis, and have done so as generously as he has. He may have made mistakes, and again, I often disagree with how he communicates. But accusations and innuendo like the ones in this thread are damaging, hurtful and useless. And one thing that I hope people will remember is that, famous and powerful as Travis may be, he's still our colleague, a member of our community, and *a human being*, so let's remember that as well... 2. Conflicts of interest are a fact of life, in fact, I would argue that every healthy and sufficiently interconnected community eventually *should* have conflicts of interest. They are a sign that there is activity across multiple centers of interest, and individuals with connections in multiple areas of the community. And we *want* folks who are engaged enough precisely to have such interests! For conflict of interest management, we don't need to reinvent the wheel, this is actually something where our beloved institutions, blessed be their bureaucratic souls, have tons of training materials that happen to be not completely useless. Most universities and the national labs have information on COIs that provides guidelines, and Numpy could include in its governance model more explicit language about COIs if desired. So, the issue is not to view COIs as something evil or undesirable, but rather as the very real consequence of operating in an interconnected set of institutions. And once you take that stance, you deal with that rationally and realistically. For example, you accept that companies aren't the only ones with potential COIs: *all* entities have them. As Ryan May aptly pointed out, the notion that academic institutions are somehow immune to hidden agendas or other interests is naive at best... And I say that as someone who has happily stayed in academia, resisting multiple overtures from industry over the years, but not out of some quaint notion that academia is a pristine haven of principled purity. Quite the opposite: in building large and complex projects, I've seen painfully close how the university/government research world has its own flavor of the same power, financial and political ugliness that we attribute to the commercial side. 3. Commercial actors. Following up on the last paragraph, we should accept that *all* institutions have agendas, not just companies. We live in a world with companies, and I think it's naive to take a knee-jerk anti-commercial stance: our community has had a productive and successful history of interaction with industry in the past, and hopefully that will continue in the future. What is true, however, is that community projects should maintain the "seat of power" in the community, and *not* in any single company. In fact, this is important even to ensure that many companies feel comfortable engaging the projects, precisely so they know that the technology is driven in an open and neutral way even if some of their competitors participate. That's why a governance model that is anchored in neutral ground is so important. We've worked hard to make Numfocus the legal entity that can play that role (that's why it's a 501(c)3), and that's why we've framed our governance model for Jupyter in a way that makes all the institutions (including Berkeley and Cal Poly) simply 'partners' that contribute by virtue of supporting employees. But the owners of the decisions are the *individuals* who do the work and form the community, not the companies/institutions. If we accept these premises, then hopefully we can have a rational conversation about how to build a community, where at any point in time, any of us should be judged on the merit of our actions, not the hypotheticals of our intentions or our affiliations (commercial, government, academic, etc). Sorry for the long wall of text, I rarely post on this list anymore. But I was saddened to see the turn of this thread, and I hope I can contribute some perspective (and not make things worse :) Cheers, -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
Hi Fernando, I am happy that you decided to chime in here. This thread derailed in a bad way and I hope that your wise words will help to redress the situation. In fact, I would like to propose you having part of a future steering committee of NumPy. I know that you may never have been implied in the hard core development of NumPy, but my perception is that your opinions are highly respected by almost everybody in the NumPy ecosystem. More than that, you have this rare ability of being able to get both a donation from Microsoft and at the same time (same year?) being awarded by the FSF, which frankly, is not an easy thing to do ;) Just to clear, I am not saying that you should act as the person for deciding the roadmap for NumPy at all, but a person that should be in charge of acting as an independent referee in the foreseeable Conflicts of Interest in the NumPy roadmap. Sorry if that means more work for you Fernando, because I know that you have become a very busy person, but I also know how much do you care about the NumPy ecosystem, and IMHO the NumPy community needs a person like you *now*. Francesc 2015-09-23 10:02 GMT+02:00 Fernando Perez <fperez.net@gmail.com>:
-- Francesc Alted
On Wed Sep 23 12:39:59 2015 GMT+0200, Francesc Alted wrote:
Is it ok if I get a bit angry soon ;)? We can find many great community leaders. But please tell me that you all feel that numpy is so central or whatever that these community leaders should be in some sense in charge of numpy. I am ready to accept that and maybe it can be helpful, and a huge gain for numpy, but I would like to see a clear statement and reasons. I like Fernandos mail, too, nor do I doubt Travis achievements. But discussing who is great community leader, etc. is frankly not obvious to me related to numpy governance. Now if you say: Guys, you should have some community leader guidance in numpy, then we can discuss who. But to me it is not obvious that community leaders who are *not* directly active in numpy should be explicitely in governance. I am aware that everyone wants to help, but right now I do not feel helped at all :). - Sebastian
On Wed, Sep 23, 2015 at 6:55 AM, Sebastian Berg <sebastian@sipsolutions.net> wrote:
Is it ok if I get a bit angry soon ;)?
Don't worry, Sebastian :) I appreciate Francesc's kind words, but I have no intention of imposing my presence anywhere, it's not like I'm looking for extra work at this point. The process of establishing governance has to come organically from within a community. Cheers -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
But discussing who is great community leader, etc. is frankly not obvious to me related to numpy governance.
Thank you Sebastian. Could we please try to get back to the governance issues, without naming names? There are some specific questions on the table that need to get hashed out. Numpy does not have a BDFL. BDFLs are not selected, they are naturally produced, and there is no one that fits that bill now. We *could* decide to have a single individual executive leader, selected by some sort of democratic process. But that is not the same as a BDFL. And I think the almost-consensus now is to not have that. So here is what I think is on the table: We have the steering council. Which leaves two questions: -How big should it be? -Who will be on the original, "seed" council? Note that as I understand the current draft of the governance doc, once established, the council itself decides who is on the council. So at this point we really are ONLY deciding how it's going to start. It has to be bootstrapped somehow. However, that had been contentious enough that it would probably be a good idea to hash out some guidelines about the council membership. Personally, I have no idea how big the council should be. Too big, and there is no point, consensus is harder to reach the larger the group, and the main (only?) role of the council is to resolve issues where consensus has not been reached in the larger community. But what is too big? As for make-up of the council, I think we need to expand beyond people who have recently contributed core code. Yes, the council does need to have expertise to make technical decisions, but if you think about the likely contentious issues like ABI breakage, a core-code focused view is incomplete. So there should be representation by: Someone(s) with a long history of working with the code -- that institutional memory of why decisions were made the way they were could be key. Someone(s) that may not have worked on the core code, but is a major player in the community, perhaps as the leader of a Numpy-dependent project. This would provide representation for the broad community. I do want to note that the governance document as I understand it is consistent with these suggestions. As for conflict of interest issues, etc: Chill out people! Numpy is an open source project, if it gets hijacked, it can be forked. And the council is also democratic -- no one person can drive the project. If a council member is not acting in the interests of the community, s/he can be removed. NOTE: while x.org, and egcs, Xemacs, and ... may be examples of failures of governance, they are also examples of successes of open source. -Chris
On Wed, Sep 23, 2015 at 3:02 AM, Fernando Perez <fperez.net@gmail.com> wrote:
I accept that criticism and apologize for doing that. My *human* side was coming out, and I was not being fair. In my head, though I was also trying to illustrate how some seemed to be doing the same thing for Continuum or other companies. This did not come out very artfully in the early morning hours. I'm sorry. BIDS is doing a lot for the community --- the recent DS4DS workshop, for example, was a spectacularly useful summit --- I hope that many different write-ups and reports of the event make their way out into the world.
I really hope it's just a perception problem (perhaps on my end). There are challenges with working in the commercial world (there are a lot of things to do that have nothing to do with the technology creation) and communicating on open-source mailing lists. As many have noticed, despite my intentions to contribute, I really can't do the same level of contribution personally that I could when I was a student and a professor and had more time. However, I think that it is also under-appreciated (or mis-understood) how much time I have spent with training and helping people who have contributed instead. It's important to me to build a company that can sponsor people to work on open-source (in a community setting). We are still working on that, but it has been my intent. So, far it's actually easier to sponsor new projects than it is to sponsor people on old projects. I am quite sure that if Continuum had put 3 people full time on NumPy in 2012, there would have been a lot of back-lash and mis-understanding. That's why we didn't do it. The collateral effect of that was the creation of other tools that could be somewhat competitive with NumPy long term -- or not. I'd like to learn how to work with the community in an optimal way so that everyone benefits --- and progress happens. That's also why we created Numfocus --- though it is ironic that NumPy has been one of the last projects to actually sign up and be a formally sponsored project. 2. Conflicts of interest are a fact of life, in fact, I would argue that
Thanks for re-emphasizing this. I agree it's O.K. and even necessary to talk about COIs. If I have not been upfront about any possible COIs it's not because of a desire to hide them -- I may not be aware of them myself as I'm still focused on what technically can be better, and how to create a company that can produce as much relevant open source software as possible. I appreciate your helping us get to a better conversational ground. Stefan's and Nathaniel's recent emails have done that as well. Thank you, -Travis
On Wed, Sep 23, 2015 at 12:18 PM, Travis Oliphant <travis@continuum.io> wrote:
Thank you.
Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
Hi, On Wed, Sep 23, 2015 at 1:02 AM, Fernando Perez <fperez.net@gmail.com> wrote: [snip]
1. I hope the discussion can move past the suspicion and innuendo about Continuum and Travis
I'm glad the discussion has become a little more calm now, but I find it difficult not to be annoyed by this statement above. I see a severe reaction to perceived 'suspicion and innuendo', but I see no 'suspicion and innuendo'. Unless you mean that any suggestion of potential conflict of interest is suspicion and innuendo. You imply rudeness and bad faith when you say this. I suppose this must be aimed at me or Stefan or both of us. In any case, I think the accusation is unhelpful and unfair. It makes this discussion more difficult to have in the future. See you, Matthew
On Wed, Sep 23, 2015 at 12:57 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
No, as I said, COIs are absolutely a fact of life, and *should* be talked about, openly and directly. I was referring generically about the tone of this thread, that Ryan described as "bizarre", others as "surpised", "disheartened", etc.
It was sincerely my intention to frame my critique in a specific way that made the discussion *easier* to have in the future, precisely by acknowledging that COIs were part of life, and not only for commercial entities. I was NOT implying that Stefan and you were somehow guilty of anything, I only mentioned your names when I asked Travis not to paint Berkeley folks with a broad brush, that's all. If only the two of you took offense at my wording, in the interest of keeping this already contentious and fraught thread contained, I offer: a) a public apology for my choice of words, since it was certainly not my intent to offend you, and I was not aiming at you personally. b) a suggestion that we discuss it further personally, taking advantage of the fact that we happen to be physically close. If anyone else would like me to answer in public because they feel a slight on my part, I will do so. Best, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
On 2015-09-23 13:36:35, Fernando Perez <fperez.net@gmail.com> wrote:
What I thought Matthew was saying is that you writing "I hope the discussion can move past the suspicion and innuendo about Continuum and Travis" gives validity to the view that those things are real whereas, in my view, they were constructed in responses by others who didn't accurately summarize the intent of what was being said (and I'm not laying blame to anyone, I could certainly have worded myself more clearly). But, yes, it was disheartening—especially because we all know one another and have hacked together late into the night at SciPy conferences. My experience has always been that we are reasonable people who listen; but perhaps we forget that about one another from time to time. It is important to me that we work towards making this mailing list a safe place for discussion again. Part of that may be to do some maintenance on bridges of trust that seem to have eroded a bit over the years. Perhaps some kind of group bonding activity, such as working on a shared project, would help? ;)
b) a suggestion that we discuss it further personally, taking advantage of the fact that we happen to be physically close.
Sure, I'm happy to discuss the personal side of this offline. Stéfan
On Wed, Sep 23, 2015 at 2:31 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
Hey! I want to have a beer with you folks too! Maybe time for a trip back to the old stomping grounds.... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Wed, Sep 23, 2015 at 2:31 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
Perhaps some kind of group bonding activity, such as working on a shared project, would help? ;)
Indeed, there's a bunch of fresh 2x4s, screws and bolts with your name on them ;) Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
Hi, On Wed, Sep 23, 2015 at 1:36 PM, Fernando Perez <fperez.net@gmail.com> wrote:
Sure, I said above, it is clearly true the some people reacted very strongly. Did you see any remark made by me or Stefan or anyone else that could reasonably be described as bizarre, surprising or disheartening? Cheers, Matthew
On Wed, Sep 23, 2015 at 2:32 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Did you see any remark made by me or Stefan or anyone else that could reasonably be described as bizarre, surprising or disheartening?
As I said already, I wasn't referring to you personally, but to the tone of the thread. It was the taste that the thread left in my mouth after reading tens of very long emails, and I guess I wasn't the only one, since multiple folks used such adjectives. But it was a poor choice of words on my part. And no, I didn't make a quote-by-quote analysis of the thread, I'm sorry. One last time, it was *not* a personal reference to you: the only reason I mentioned your names was because of the Berkeley clarification regarding BIDS that I asked of Travis, that's all. If that comment hadn't been made, I would not have made any mention whatsoever of anyone in particular. I apologize for not foreseeing that this would have made you feel singled out, in retrospect, I should have. In my mind, it was the opposite, as I felt that you had every right to express whatever opinions you have speaking for yourselves, independent of your affiliations, and I was simply asking Travis to separate individuals from institutions. But I should have realized that calling anyone out by name in a context like this is a bad idea regardless. Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
This was my fault for not being more careful in my words. I felt multiple things when I wrote my emails that led to incorrectly chosen words --- but mostly I was feeling unappreciated, attacked, and accused. I'm sure now that was not intended --- but there have been mis-understandings. I expect they will happen again. I know if we listen to each other and trust that while we may see the world differently and have different framings of solutions --- we can work to coordinate on an important technical activity together. In retrospect, my initial email requesting inclusion on the seed council could have been worded better (as there were multiple things conflated together). I am responding to the actual text of the governance document in the other thread so as to clarify what my proposal actually is in the context of that document. -Travis
On Wed, Sep 23, 2015 at 3:19 PM, Travis Oliphant <travis@continuum.io> wrote:
No worries, I wasn't digging it up further, really! Just clarifying for other reasons, not digging into you :) Glad to see the other thread proceeding further, let's let this one die as peacefully as it can... Best, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
On Wed, Sep 23, 2015 at 3:37 PM, Fernando Perez <fperez.net@gmail.com> wrote:
Glad to see the other thread proceeding further, let's let this one die as peacefully as it can...
This Monty Python sketch seems relevant: https://www.youtube.com/watch?v=grbSQ6O6kbs (hope everyone's in good health and regains better spirits, back to lurking for me) -- _ / \ A* \^ - ,./ _.`\\ / \ / ,--.S \/ \ / `"~,_ \ \ __o ? _ \<,_ /:\ --(_)/-(_)----.../ | \ --------------.......J Paul Ivanov http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7
On Tue, Sep 22, 2015 at 1:07 AM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
A couple of things to help clarify: 1) nobody believes that the community should be forced to adopt numba as part of ufunc core yet --- but this could happen someday just as Cython is now being adopted but was proposed 8 years ago that it "could be adopted" That's a red-hearing. 2) I have stated that breaking the ABI is of little consequence because of conda as well as other tools. I still believe that. This has nothing to do with any benefit Continuum might or might not receive because of conda. Everyone else who wants to make a conda-based distribution also benefits (Cloudera, Microsoft, Intel, ...) or use conda also benefits. I don't think the community realizes the damange that is done with FUD like this. There are real implications. It halts progress, creates confusion, and I think ultimately damages the community. Numba being an acceptable dependency means a lot more than conda --- it's dependent on LLVM compiled support which would have to be carefully tested --- first as only an optional dependency for many years.
I don't see how. None of these have been proposed for integrating into NumPy. I don't see how integrating numba into NumPy benefits Continuum at all. It's much easier for us to keep it separate. At this point Continuum doesn't have an opinion about integrating DyND into NumPy or not. These projects will all succeed or fail on their own based on users needs. Whether or not they every become a part of NumPy will depend on whether they are useful as such not because a person at Continuum is part of a steering committee (with other people on it). I know that you were responding to specific question by Brian as to how their could be a conflict of interest for Continuum and NumPy development. I don't think this is a useful conversation --- we could dream up all kinds of conflicts of interest for BIDS and NumPy too (e.g. perhaps BIDS really wants Spark to take over and for NumPy to have special connections to Spark). Are we to not allow anyone at BIDS to participate in the steering council because of their other interests? But remember, the original point is whether or not someone from Continuum (or I presume any company and not just singling out Continuum for special treatment) should be on the steering council. Are you really arguing that they shouldn't because there are other projects Continuum is working on that have some overlap with NumPy. I really hope you don't actually believe that. -Travis
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
Hi Travis On 2015-09-21 23:29:12, Travis Oliphant <travis@continuum.io> wrote:
Yes, I'd like to clarify: I was not against including any specific technology in NumPy. I was highlighting that there may be different motivations for members of the general community and those working for, say, Continuum, to get certain features adopted.
This is an old argument, and the reason why we have extensive measures in place to guard against ABI breakage. But, reading what you wrote above, I would like to understand better what FUD you are referring to, because I, rightly or wrongly, believe there is a real concern here that is being glossed over.
I think that touches, tangentially at least, on the problem. If an employee of Continuum were steering NumPy, and the company developed an opinion on those integrations, would such a person not feel compelled to toe the company line? (Whether the company is Continuum or another is besides the point—I am only trying to understand the dynamics of working for a company and leading an open source project that closely interacts with their producs.)
I guess that's an interesting example, but BIDS (which sits inside a university and is funded primarily by foundations) has no financial, and very few other, incentives to do so.
Here's what I'm trying to say (and I apologise for ruffling feathers in the process): There are concerns amongst members of the community that (will) arise when strong players from industry try / hint at exerting (some) executive control over NumPy. We can say that these concerns amount to spreading FUD, that they are uninformed, unrealistic, etc., but ultimately they are still out there, and until they are discussed and addressed, I find it hard to see how we can move forward with ease. Stéfan
On Tue, Sep 22, 2015 at 2:16 AM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
This is what I'm calling you out on. Why? I think that is an unfair statement and inaccurate. The general community includes Continuum, Enthought, Microsoft, Intel, various hedge funds, investment banks and companies large and small. Are you saying that people should not be upfront about their affiliations with a company? That if they are not academics, then they should not participate in the discussion? It is hard enough to be at a company and get time to contribute effort back to an open source project. We should not be questioning people's motives just *because* they are at a company. We should not assume people cannot think in terns of the success of the project, just because they are at a company. Their proposals and contributions can be evaluated on their merits and value --- so this whole discussion seems to be just revealing an anti-company paranoia rather than helping understand the actual concern.
I don't know which is the "old argument". Anyway, old arguments can still be right. The fact is that not breaking the ABI has caused real damage to the community. NumPy was never designed to not have it's ABI broken for over a decade. We have some attempts to guard against ABI breakage --- but they are not perfect. We have not moved the code-base forward for fear of breaking the ABI. When it was hard to update your Python installation that was a concern. There are very few cases where this is still the concern (conda is a big part of it but not the only part as other distros and approaches for easily updating the install exist) --- having this drive major architecture decisions is a serious mistake in my mind, and causes a lot more work than it should. The FUD I'm talking about is the anti-company FUD that has influenced discussions in the past. I really hope that we can move past this.
O.K. if you are honestly asking this question out of inexperience, then I can at least help you understand because perhaps that is the problem (creating a straw-man that doesn't exist). I have never seen a motivated open source developer at a company who "tows the company line" within a community project that is accepted long term. All that would do is drive the developer out of the company and be a sure-fire way to make sure their contributions are not accepted. I know that at Continuum, for example, if someone disagreed with me about an open source technology, didn't tell me and just "towed the company line" (whatever they thought that was) --- they would be fired. Most software companies that participate in open source are like that. The worst thing that happens is the company no longer participates in the discussion --- that is what happens if they can't get contributions. It really is no different than me being worried that an academic might push to get something into a library because it would "help their publication record" --- which I also think is a very real potential concern. Or perhaps you have a favor owed to another colleague who helped you along in your academic career and they "really want" some feature added. The actual conflict of interest that exists there has the same amount of weight in my mind as the ones you hypothesize about.
I think you don't understand how academic finances work. I could see a lot of ways such interest could develop --- and could influence funding agencies. I don't think they are very likely --- but they are possible --- and to me the same degree of possibility that Continuum would try to "force anything" on the NumPy community.
I'm sorry you have these concerns. I don't think they are as warranted as you believe. NumPy will get strong players from industry coming in. In fact, one of the reasons we all felt it was important to establish Numfocus was to be an organization that could shield these players from the development of NumPy and other projects. I am not one of those "strong industry players" you speak of. I am a friend. I am a participant. I am one of the community. Perhaps you and others feel that *I* am that strong industry player trying to exert "executive control" over NumPy. Is that true? Is that what you feel? I'm starting to wonder if that is actually the problem here. -Travis
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On Tue, Sep 22, 2015 at 2:33 AM, Travis Oliphant <travis@continuum.io> wrote:
The FUD I'm talking about is the anti-company FUD that has influenced discussions in the past. I really hope that we can move past this.
I have mostly stayed out of the governance discussion, in deference to how new I am in this community, but I do want to take a moment to speak up here to echo Travis's concern about anti-company FUD. Everyone invested in NumPy has their own projects, priorities and employers which shape their agenda. As far as I can tell, Travis and Continuum have only ever acted with the long term health of the scipy ecosystem in mind. Stephan
On Mon, Sep 21, 2015 at 10:15 PM, Bryan Van de Ven <bryanv@continuum.io> wrote:
I have no expectation that continuum will follow any of these paths, and in most cases am not even sure what that would mean, BUT just because I think it is useful to have a wide variety of concrete examples to draw on -- data is good! -- there actually are *lots* of examples of "community revolts" wresting projects from their original founders, in a variety of corporate and non-corporate contexts. Some examples include the nodejs->iojs fork and merge (which was about wresting control of the project from the founding company), the gcc->egcs fork and merge (which removed RMS's control over day-to-day running of the project), the openoffice->libreoffice fork, the xfree86->x.org fork (where the original core team decided to change the license and all the developers left), the mambo->joomla fork, the xchat->hexchat fork (triggered partially by people's annoyance at the original developer for trying to monetize the project), ... Along somewhat similar lines, there's also the fraught history of Qt and Trolltech and the conflicts between the community and commercial interests there. -n -- Nathaniel J. Smith -- http://vorpus.org
On Mon, Sep 21, 2015 at 8:24 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
I don't really agree that someone couldn't be found. I think the key is that 1) the person would be able to make a decision about what needs to be done if the community is stuck and 2) their is not really a "for life" clause in that their would be a way to call for a replacement.
I don't think this is true. The steering committee would only be called upon to unstick things and make decisions. At those times, it would not take long to come up to speed on the issues and make a decision.
I think this is a red-herring and should not be an issue --- except for obvious situations where I would have to not participate in a "vote" (such as whether or not Continuum would be an institutional sponsor or something like that). Everyone has associations and affiliations and goals and plans that could lead to conflicts of interest. This kind of requirement un-necessarily limits the governance to only certain kinds of people who have only "volunteer" time. This is actually quite damaging as it limits the ability for people to get paid to work on NumPy. This feels like a world-view debate that is actually best left to a different mailing list.
For example, see this interview with Linus Torvalds,
where he talks about going out of his way to make sure that people can trust his decisions, despite the fact he is paid to work on Linux [1].
But this is in a BDFL role and not a steering committee role with 9 other members. I don't think the situation compares or applies at all. The unwarranted fear this kind of reasoning can create in the mind of otherwise reasonable people is unfortunate.
Fundamentally the "steering committee" is too big. I think the committee should be much smaller (no more than 5 and really three is the right size for what it is supposed to actually do). I've had experience with small committees and large committees. Large councils make it very difficult to move things forward. If it is to remain that big, I think it needs people on it like me, Robert Kern, or David Cournapeau who have a longer history with the project and understand why some of the things are the way they are. I still hear far too many conversations that start from a lack of understanding of the early conversations that led to why NumPy is the way it is. This lack of context is not helpful and potentially dangerous. The advanced indexing discussions happening right now and the renewed array-scalar discussions for example are both examples of features that have had previous discussions and have a long history of back and forth between various people.
I like the trustee model too and think such an addition to the NumPy concept would help alleviate my concerns about actually being on a "steering committee" but my preferred outcome is actually that the agreed upon steering council be smaller and that people who have a right to vote on things like the make-up of the steering committee be comprised of people who have been significantly involved in the past 3 years (not just the past one year). -Travis
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On 2015-09-20 11:20:28, Travis Oliphant <travis@continuum.io> wrote:
I would split the above into two parts: a suggestion on how to change the governance model (first half of 1 and 2) and then some thoughts on what to do once those changes have been made (latter half of 1 and 2, as well as 3). For now, since those changes are not in place yet, it's probably best to focus on the governance model. I would agree that one person (or a very small group) is best suited to "getting things unstuck". And, personally, I believe it best for that person/persons to be elected by the community (whatever we define "the community" to be)---which is what I presume you suggested when you mentioned nominating candidates. Since Matthew mentioned the governance proposal we're working on, here is a very early draft: https://github.com/stefanv/skimage-org/blob/governance_proposal/governance.m... As I said, this is still a work-in-progress--comments are welcome. E.g., the weighting element in the voting has to be fine tuned (but was put in place to prevent rapid take-overs). Essentially, we need: - a way for community members to express disagreement without being ousted, - protection against individuals who want to exert disproportional influence, - protection against those in leadership roles who cause the project long-term harm, - and a way for the community to change the direction of the project if they so wished. Stéfan
Thank you for posting that draft as it is a useful comparison to borrow from. I think Nathaniel's original document is a great start. Perhaps some tweaks along the lines of what you and Matt have suggested could also be useful. I agree that my proposal is mostly about altering the governance model, mixed with some concern about being "automatically disqualified" from a council that can decide the future of NumPy if things don't move forward. -Travis On Tue, Sep 22, 2015 at 12:57 AM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
I’ve also stayed out of this until now. I’m surprised and disheartened at the amount of suspicion and distrust directed towards Travis. I don’t think anyone has invested as much personal time and resources (e.g., money) towards supporting numpy, and not just in creating it but through efforts at Continuum and Numfocus. So much of this distrust appears based on what Continuum might do rather than what the actual record is. I agree with Travis that virtually everyone has their own interests. I don’t think non-profit institutions (academia or otherwise) are any more virtuous than for-profit companies when it comes to possible conflicts of interest. As long as everyone understands the interests involved, that shouldn’t bar anyone from participating in governance. If anyone deserves a special seat at the table it is Travis. I’m completely convinced he has the community’s greater interests at heart (not to say that he always understand all the interests and may need input to help in that). Perry
On 22/09/15 14:31, Perry Greenfield wrote:
I’ve also stayed out of this until now. I’m surprised and disheartened at the amount of suspicion and distrust directed towards Travis.
I have no idea where this distrust comes from. Nobody has invested so much of their time in NumPy. Without Travis there would not even be a NumPy.
So much of this distrust appears based on what Continuum might do rather than what the actual record is.
Being CEO of Continuum should not disqualify him in any way.
I agree with Travis that virtually everyone has their own interests.
Even Guido and Linus have employers. Should we distrust Guido as Python BDFL because some day Dropbox will be evil? It is just nonsense. Sturla
Hi All, I've been reading this thread with amazement and a bit of worry. It seems Nathaniel's proposal is clearly an improvement, even if it is not perfect. But it is in the end for a project where, at least as seen from the outside, the main challenge is not in governance, but rather in having only a small group of people who understand the code well enough that they are able to make and judge modifications. As such, this discussion doesn't seem worthy of the effort, and even less of needless heat and irritation, of the type that seems unlikely would have arisen if this conversation had been in person instead of per e-mail. Might it be an idea to accept the proposal provisionally, returning to it a year from now with practical experience? This certainly has the benefit of allowing to switch focus to the more pressing and fortunately also more interesting work to be done on interfacing numpy/ndarray nicely with other classes (i.e., __numpy_ufunc__ and/or similar, and the dtype generalisations, which have me quite intrigued -- either might be very interesting for the Quantity class in astropy, as well as for a work-in-progress Variable class [which propagates uncertainties including covariances]). All the best, Marten -- Prof. M. H. van Kerkwijk Dept. of Astronomy & Astroph., 50 St George St., Toronto, ON, M5S 3H4, Canada McLennan Labs, room 1203B, tel: +1(416)9467288, fax: +1(416)9467287 mhvk@astro.utoronto.ca, http://www.astro.utoronto.ca/~mhvk
I don't think I've contributed code to NumPy itself, but as someone involved in the scientific python ecosystem for a while, I can't see why people would consider Continuum less of a legitimate participant or community member than individual contributors, especially if the person behind it has had the opportunity to control things previously and instead passed NumPy onto the community. I'd be wary of commercial interests dominating the agenda, but that's different from them having a proportionate (in this case minor) say when they have something to offer. And that's all true *even if* Travis were heavily biased to his own commercial ends, which is not consistent with my understanding of his wider efforts and sacrifices over most of a decade that I've been paying attention. Remember this is free/open source software and if enough people don't like the committee at some point, the project can be forked as an option of last resort. Nothing is set in stone, nor code lost. Just saying (I probably won't reply to any criticism or corrections, to avoid adding peripheral noise/heat to the thread). Cheers, James (from, but not on behalf of, a non-profit research facility).
This has to be one of the most bizarre threads I've ever read in my life. Somehow companies are lurking around like the boogeyman and academics are completely free of ulterior motives and conflicts of interest? This is just asinine--we're all people and have various motivations. (Having just gotten out of my university after 15 years, the idea that academics are somehow immune to ulterior motives and conflicts of interest makes me laugh hysterically.) The sad part is that this worry completely unnecessary. This is an open source project, not international politics, and the end goal is to produce software. Therefore, 99.9% of the time motives (ulterior, profit, or otherwise) are completely orthogonal to the question of: IS IT A GOOD TECHNICAL IDEA? It's really that simple: does the proposed change move the project in a direction that we want to go? Now, for the 0.01% of the time, where nobody can agree on that answer, or the question is non-technical, and there is concern about the motives of members of the "council" (or whatnot), it's again simple: RECUSAL. It's a simple concept that I learned in the godawful ethics class NSF forced grad students to take: if you have a conflict of interest, you don't vote. It's how the grownups from the Supreme Court to the college football playoff deal with the fact that people WILL have conflicts; potential conflicts don't disbar qualified individuals from being included in the group, just from weighing in when their decisions can be clouded. So how about we stop making up reasons to discourage participation by (over-)qualified individuals, and actually take advantage of the fact that people actually want to move numpy forward? Ryan
On So, 2015-09-20 at 11:20 -0700, Travis Oliphant wrote:
Hmmm, well I never had the impression that the steering council would be huge. But maybe you are right, and if it is, I could imagine something like option 2, but vote based (could possibly dual use those in charge of NumFOCUS relations, we had even discussed this possibility) which would have final say if necessary (could mean that the contributers definition could be broadened a bit). However, I am not sure this is what you suggested, because for me it should be a regular vote (if just because I am scared of having to make the right pick). And while I will not block this if others agree, I am currently not comfortable with either picking a BDFL (sorry guys :P) or very fond of an oligarchy for live. Anyway, I still don't claim to have a good grasp on these things, but without a vote, it seems a bit what Matthew warned about. One thing I could imagine is something like an "Advisory Board", without (much) formal power. If we had a voted Steering Council, it could be the former members + old time contributers which we would choose now. These could be invited to meetings at the very least. Just my current, probably not well thought out thoughts on the matter. But neither of your three options feel very obvious to me unfortunately. - Sebastian
I wrote my recommendations quickly before heading on a plane. I hope the spirit of them was caught correctly. I also want to re-emphasize that I completely understand that the Steering Council is not to be making decisions that often and almost all activity will be similar to it is now --- discussion, debate, proposals, and pull-requests --- that is a good thing. However, there is a need for leadership to help unstick things and move the project forward from time to time because quite often doing *something* can be better than trying to please everyone with a voice. My concerns about how to do this judgment have 2 major components: 1) The need for long-term consistency --- a one-year horizon on defining this group is too short in my mind for a decades-old project like NumPy. 2) The group that helps unstick things needs to be small (1, 3, or 5 at the most) We could call this group the "adjudication group" rather than the "Steering Council" as well. I could see that having a formal method of changing that "adjudication group" would be a good idea as well (and perhaps that formal vote could be made by a vote of a group of active contributors. In that case, I would define active as having a time-window of 5 years instead of just 1). Thanks, -Travis On Mon, Sep 21, 2015 at 2:39 AM, Sebastian Berg <sebastian@sipsolutions.net> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On Mon, Sep 21, 2015 at 9:20 AM, Travis Oliphant <travis@continuum.io> wrote:
For reference, the rules for steering council membership were taken directly from those used by the Jupyter project, and their steering council currently has 10 people, making it larger than the "seed council" proposed in the numpy document: https://github.com/jupyter/governance/blob/master/people.md
We could call this group the "adjudication group" rather than the "Steering Council" as well. I could see that having a formal method of changing that "adjudication group" would be a good idea as well (and perhaps that formal vote could be made by a vote of a group of active contributors. In that case, I would define active as having a time-window of 5 years instead of just 1).
I may be misreading things, but I'm getting the impression that the active "adjudication group" you envision is radically different from the "steering council" as envisioned by the current governance document. It also, I think, radically different from anything I've ever seen in a functioning community-run FOSS project and frankly it's something where if I saw a project using this model, it would make me extremely wary about contributing. The key point that I think differs is that you envision that this "adjudication group" will actually intervene into discussions and make formal decisions in situations other than true irreconcilable crises, which in my estimation happen approximately never. The only two kinds of F/OSS projects that I can think of that run like this are (a) projects that are not really community driven at all, but rather run as internal company projects that happen to have a public repository, (b) massive projects like Debian and Fedora that have to manage literally thousands of contributors, and thus have especially robust backstop procedures to handle the rare truly irreconcilable situation. E.g., the Debian CTTE acts as an "adjudication group" in the way it sounds like you envision it: on a regular basis, irreconcilable arguments in Debian get taken to them to decide, and they issue a ruling. By some back of the envelope calculations, it looks like they issue approximately ~0.002 rulings per debian-contributor-year [1][2]. If we assume crudely that irreconcilable differences scale linearly with the size of a project, this suggests that a ~20 person project like NumPy should require a ruling ~once every 20 years. Or quoting myself from the last thread about this [3]: ] Or on the other end of things, you have e.g. Subversion, which had an ] elaborate defined governance system with different levels of ] "core-ness", a voting system, etc. -- and they were 6 years into the ] project before they had their first vote. (The vote was on the crucial ] technical decision of whether to write function calls like "f ()" or ] "f()".) These are two real projects and how they really work. And even in projects that do have a BDFL, the successful ones almost never use this power to actually "unstick things" (i.e., use their formal power to resolve a discussion). Consider PEP 484, Guido's somewhat controversial type hints proposal: rather than use his power to move the debate along, he explicitly delegated his power to one of the idea's strongest critics [4]. Of course, things to get stuck. But the only time that getting them unstuck needs or even benefits from the existence of a formal "unsticking things" group is if the group is actually using some powers that they have. In 99.9% of cases, though, the correct way to get things unstuck is for a new idea to be introduced, or for someone respected to act as a mediator, or for someone to summarize the situation and isolate some core of agreement that allows for forward progress. But all of *these* things can and should be done by anyone and everyone who can contribute them -- restricting them to a small "adjudication group" makes no sense. In terms of real-world examples: From my point of view, the worst parts of the NA fiasco was caused by the decision to cut off debate with a "ruling". (Specifically, that instead of working on implementing bitpattern-NAs -- which did have consensus -- then Mark would go off and work on the masked-NA strategy, and eventually that it should be merged, despite it not having consensus. I note that dynd supports only bitpattern-NAs.) OTOH, probably the most difficult technical debate we've had recently is issue 5844, about the potential interactions between __binop__ methods and __numpy_ufunc__, and this is a debate that AFAICT certainly would not have benefited from the existence of an adjudication body. As it happens, all three of your proposed adjudication-body-members actually are in the debate anyway; if they hadn't been, then the very last thing that debate needs is someone wading in without any understanding of the complex technical issues and trying to force some resolution. I'm not saying that truly irreconcilable problems never arise. But given that they're so rare, and that realistically any system that could solve them would need to be backed by basically the same group of people that are in the current governance document's "steering council" (because in an OSS project it's the people doing the work that ultimately have the power), there's no point in building an elaborate voting system and method for defining the electorate just to handle these cases -- and quite a bit of harm. So the proposed system is basically the simplest one that could work. (I described this line of reasoning in more detail in [3].) I think it would really help if you could provide some examples of successful community-driven projects and/or specific debates that benefited from having an adjudication body like you envision? I also suspect this fundamental difference in how we view the role of a governance body explains why I am unbothered by the idea of steering council membership "timing out" after 2 years of inactivity. The steering council's job is to informally manage things day to day, and in extreme cases to make judgement calls. These two things both crucially require a broad and up-to-date understanding of the issues facing the project, ongoing debates, personalities, etc. But the council is never intended to make any judgement call on its own; the whole idea of the structure is to make sure that decisions are based on as broad a scope of expertise as possible. In particular we regularly get historical insight from Chuck, Ralf, Pauli, Robert Kern, David Cournapeau, Pearu Peterson, Anne Archibald, Matthew Brett, Stefan van der Walt... and it doesn't matter at all whether they're on the council or not. (If it did matter, that would be terrible, right? Why would you want to *not* listen to any of those people?) If you plan to become more active and give your perspective on things more then that's awesome and welcome, but AFAICT this particular point is pretty orthogonal to the composition of the steering council. See also this comment from Fernando about the Jupyter steering council, which happened by chance to cross my email this morning: https://github.com/jupyter/governance/pull/6#issuecomment-142036050 -n [1] https://www.debian.org/devel/tech-ctte#status [2] https://contributors.debian.org/ [3] https://mail.scipy.org/pipermail/numpy-discussion/2015-September/073537.html [4] See e.g. the bottom of this message from Nick Coghlan, where he talks about how PEP "pronouncements" are done -- emphasizing in particular (my paraphrase) that the point of a BDFL/BDFL-delegate is not to resolve any substantive issues, and that a common strategy is to choose the person who's most skeptical of the idea to be the BDFL-delegate: http://thread.gmane.org/gmane.comp.python.distutils.devel/23867 -- Nathaniel J. Smith -- http://vorpus.org
I actually do agree with your view of the steering council as being usually not really being needed. You are creating a straw-man by indicating otherwise. I don't believe a small council should do anything *except* resolve disputes that cannot be resolved without one. Like you, I would expect that would almost never happen --- but I would argue that extrapolating from Debian's experience is not actually relevant here. So, if the steering council is not really needed then why have it at all? Let's just eliminate the concept entirely. But there are real questions that have to have an answer or an approach to making a decision. The answer to these questions cannot really be a vague notion of "lack of vigorous opposition by people who read the mailing list" which then gets parried about as "the community decided this." The NumPy user base is far, far larger than the number of people that read this list. For better or for worse, we will always be subject to the "tyranny of who has time to contribute lately". Fundamentally, I would argue that this kind of "tyranny" should at least be tempered by additional considerations from long-time contributors who may also be acting more indirectly than is measured by a simple git log. So, what are the questions that have to have an answer that even calls for some kind of governance? I know Nathaniel's document listed some of them (and perhaps all of them). Here are mine: 1) who gets commit rights to the repo (and who has them removed)? 2) who tags the release of NumPy? 3) how is it decided where money is spent if it's donated to the project (and not just to people directly)? 4) how is it decided if someone needs to be removed from participation in the group because they are not adding to the conversation (we have been fortunate that this hasn't happened in NumPy before --- but it could)? 5) how is it decided what goes on the NumPy website --- i.e. will advertisers be able to put their logos or book-covers there? If I understand what you are proposing, then basically the "steering council" decides these things. Perhaps rather than a steering council, though, we just need clear answers to questions like the above --- which might be handled differently for different questions. I don't think these questions have very easy answers. Ultimately NumPy has relied on and continues to rely on the mutual respect of all the people that have worked on the code and tried to make it better. We all have opinions about how things have gone in the past, and what has gone well and what hasn't. But, nothing you have said persuades me that you have a full picture of past history with respect to a lot of the difficult kinds of conversations that have happened and the different modes of activity that have tried to help move NumPy along. In fact, I think you mis-understand and mis-interpret that history quite often. I'm convinced you are well-intentioned and doing the very best you can, and I'm very grateful that you are passionate and eager about moving NumPy forward. Ultimately I hope it will help things. Here is my attempt at a proposal for how to answer the above questions: 1) who gets commit rights to the repo (and who has them removed)? * people who contribute regularly are granted commit rights by another committer with at least two additional nominations and the lack of a veto within 1 week of the proposal. * nobody has commit rights removed except by unanimous consent of all the other committers (with consent being implied if not responded to within 2 weeks). 2) who tags the release of NumPy? * whomever volunteers to be release manager and if there is no veto from committers. 3) how is it decided where money is spent if it's donated to the project (and not just to people directly)? * three people who self-select represent NumPy to Numfocus following the rules of Numfocus (that there is only one representative from any organization). * If 3 committers oppose one of those people and nominate another in place, then that person is replaced. 4) how is it decided if someone needs to be removed from participation in the group because they are not adding to the conversation (we have been fortunate that this hasn't happened in NumPy before --- but it could)? * unanimous consent of all committers (with a 2 week period given for consent to be given --- and it is assumed given if they are not heard from). 5) how is it decided what goes on the NumPy website --- i.e. will advertisers be able to put their logos or book-covers there? * only Numfocus can advertise and put their logo on the website Now, I'm sure one can poke holes in the above --- and I would welcome better answers to the above questions. Perhaps we should just decide how specific decisions get made and make a document that lists that and only talks about committers instead of inventing another "bit" to differentiate people in the community. -Travis On Tue, Sep 22, 2015 at 2:11 AM, Nathaniel Smith <njs@pobox.com> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On Tue, Sep 22, 2015 at 1:24 AM, Travis Oliphant <travis@continuum.io> wrote:
To be clear, Debian was only one example -- what I'm extrapolating from is every community-driven F/OSS project that I'm aware of. It's entirely possible my data set is incomplete -- if you have some other examples that you think would be better to extrapolate from, then I'd be genuinely glad to hear them. You may have noticed that I'm a bit of an enthusiast on this topic :-).
So, if the steering council is not really needed then why have it at all?
Let's just eliminate the concept entirely.
In my view, the reasons for having such a council are: 1) The framework is useful even if you never use it, because it means people can run "what if" scenarios in their mind and make decisions on that basis. In the US legal system, only a vanishingly small fraction of cases go to the Supreme Court -- but the rules governing the Supreme Court have a huge effect on all cases, because people can reason about what would happen *if* they tried to appeal to the Supreme Court. 2) It provides a formal structure for interfacing with the outside world. E.g., one can't do anything with money or corporate contributions without having some kind of written-down and enforceable rules for making decisions (even if in practice you always stick to the "everyone is equal and we govern by consensus" part of the rules). 3) There are rare but important cases where discussions have to be had in private. The main one is "personnel decisions" like inviting people to join the council; another example Fernando has mentioned to me is that when they need to coordinate a press release between the project and a funding body, the steering council reviews the press release before it goes public. That's pretty much it, IMO. The framework we all worked out at the dev meeting in Austin seems to handle these cases well AFAICT.
According to the dev meeting rules, no particularly "vigorous opposition" is required -- anyone who notices that something bad is happening can write a single email and stop an idea dead in its tracks, with only the steering council able to overrule. We expect this will rarely if ever happen, because the threat will be enough to keep everyone honest and listening, but about the only way we could possibly be *more* democratic is if we started phoning up random users at home to ask their opinion. This is actually explicitly designed to prevent the situation where whoever talks the loudest and longest wins, and to put those with more and less time available on an equal footing.
I guess I am missing something fundamental here. Who are these long-time contributors who will sit on your council of 1/3/5 but who don't even read the mailing list? How will they know when their special insight is necessary?
No, absolutely not. The proposal is that these issues are decided by open discussion on the mailing list. (With the possible exception of #4 -- I suspect that given an extreme situation like this, then once all other efforts to mitigate the situation had failed the steering council would probably feel compelled to talk things over to double-check they had consensus before acting, and likely some part of this would have to be done in private.) This is pretty explicit in the document (and again, this is text and ideas stolen directly from Jupyter/IPython): "During the everyday project activities, council members participate in all discussions, code review and other project activities as peers with all other Contributors and the Community. In these everyday activities, Council Members do not have any special power or privilege through their membership on the Council. [...] the Council may, if necessary [do pretty much anything, but] the Council's primary responsibility is to facilitate the ordinary community-based decision making procedure described above. If we ever have to step in and formally override the community for the health of the Project, then we will do so, but we will consider reaching this point to indicate a failure in our leadership." If this is unclear then suggestions for improvement would certainly be welcome. This is pretty much how we do things now, and it has worked pretty well so far -- people do get commit bits, releases get tagged, and everyone seems happy with this part. I'd actually like to see a more explicit set of rules around e.g. who gets commit bits, just to set expectations and provide a clearer onramp for new contributors, but that's something that we can easily discuss and make a decision on later. All of these questions are ones that are easy enough to solve given some framework for governance, but they do all require substantive discussion. We're not going to do them justice if we try to solve them off-the-cuff in this thread, and trying would just derail the underlying discussion about how we make decisions. So I think we should stay focused on that.
I'm honestly somewhat unclear on why having a "full picture of past history" is necessary to contribute effectively (surely if that were the case, then only Jim Hugunin could comment?), but if there are things I'm missing then I'd certainly like to know. You've made comments along these lines several times now, but unfortunately I can't do much to improve my understanding without more concrete examples. I'm convinced you are well-intentioned and doing the very best you can, and
I'm very grateful that you are passionate and eager about moving NumPy forward. Ultimately I hope it will help things.
Thank you. I am also convinced that you're well-intentioned and doing the very best you can, and grateful that you are also passionate and eager about moving NumPy forward. -n -- Nathaniel J. Smith -- http://vorpus.org
On Tue, Sep 22, 2015 at 4:33 AM, Nathaniel Smith <njs@pobox.com> wrote:
Yes, you are much better at that than I am. I'm not even sure where I would look for this kind of data.
O.K. That is a good point. I can see the value in that.
O.K.
O.K.
How did we "all" work it out when not everyone was there? This is where I get lost. You talk about community decision making and yet any actual decision is always a subset of the community. I suppose you just rely on the "if nobody complains than it's o.k." rule? That really only works if the project is moving slowly.
O.K. so how long is the time allowed for this kind of opposition to be noted?
The long-time contributors wouldn't necessarily sit on that council. But, I would support the idea of an advisory council that could act if it saw things going the wrong direction. This is where those people would sit. In the case of a 1 / 3 / 5 member council -- I would not argue to be on it at all (but I would argue that some care should be taken to be sure it has people with some years of experience if they are available). I'm only asking to be on a steering council that is larger than 5 people, and I don't actually prefer that the steering council be larger than 5 people.
O.K. Then, I am misunderstanding.
Granting commit rights to the repo does not seem to me to be an "everyday project activity" --- so I suppose I was confused. I suppose that could happen with no serious objections on the list. It seems that the people who actually have the commit bit presently should be consulted more than the general public, though.
O.K. I think that the commit bit has not been that clear --- and was closer when I was more active to the commiters rule I gave before.
The reason is that you in particular make very long arguments that try to persuade on the basis of your understanding of how things were done in the past (both here and in other projects). If you don't understand the history and the arguments that have been had before, you are throwing away information. It's not a show-stopper, for sure, but it does potentially make it less likely that a better solution will be found. I will have to respond to your incorrect characterizations in another thread. -Travis
On Sun, Sep 20, 2015 at 11:20 AM, Travis Oliphant <travis@continuum.io> wrote:
After long conversations at BIDS this weekend and after reading the entire governance document, I realized that the steering council is very large
How large are we talking? I think there were 8 people named -- and I'm not sure all 8 want to do it. But 5 or seems like a fine number -- I wonder if defining the number is a good idea.
A one year time frame is pretty short on the context of a two decades old project
I see: """ who have produced contributions that are substantial in quality and quantity, and sustained over at least one year """ that could be longer, but it looks like it DOESN'T mean "have contributed within the last year" -- so we could certainly, at this point, have a lot of folks that have been around a long time that could qualify. A certain guy named Travis come to mind... and I believe the current council has too few people who have been around
the community long enough to help unstuck difficult situations if that were necessary.
That's actually a bad sign right there -- the suggested group are the folks that have actually been contributing lately -- too bad that apparently isn't anyone that has been around a long time :-( also see: """ This will include but is not limited to code, code review, infrastructure work, mailing list and chat participation, community help/building, education and outreach, design work, etc. """ so it's not only people doing the actual coding ( A really small number :-( ) But I note that the the suggestion to "seed the council" with "everyone who has reviewed/merged a pull request since Jan 1, 2014" - I'm not sure that gives us as diverse a council as we'd like -- but it is only the seed...
1 - define a BDFL for the council.
Well, that would make a number of things easier -- but I think there was a consensus that there simply was no obvious candidate -- and if the candidate is not obvious, then they can't really be a BDFL.
2 - limit the council to 3 people. I would nominate chuck, nathaniel, and pauli.
Good group -- but maybe 3 a bit too small.
3 - add me as a permanent member of the steering council.
I'd love to see you on the council, and there is no question of your qualifications -- but I don't think anyone should be anymore permanent than any other. According to the docs so far, the only thing anyone needs to do to remain on the council is stay active -- and not piss everyone off so much as to get a consensus from everyone else that you need to be kicked off. And the end of this -- I think the big question still at hand is how big the council should be. My experience with consensus suggests that it not be very big :-) -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Hi Travis, and all, You might have seen I was advocating for having someone who takes final responsibility for the project, partly to get discussions unstuck, as you said. I agree with Chris, that at this stage, there is no-one who could be Benevolent Dictator for the project. It seems to me that (nearly) everyone would agree that, if there is a leader, we would not want a person who dictates, but someone who is good at putting in the hard work and time it takes to hear all the arguments, and guide people to agreement, where that is possible. If there is a leader, I think we should select them for those skills. For the proposal that you join the steering committee, I see two problems. The first is, that the members of the steering committee have to be the people who are keeping in touch with the day to day work of the project. I am sure you would agree that, in the past, you have not had enough time for that. Your recent emails about the new work you want to do, also imply that you may be too busy to get involved in the detailed discussions needed for getting the code merged. In any case, I think you'd also agree that in the past you have hoped that you would have more time for numpy than you did. The second problem is that you have a potential conflict of interest, in that it is possible for the needs of Continuum to conflict with the needs of numpy. I believe, from previous emails on this list, that you don't think that is very important, but I continue to disagree about that. For example, see this interview with Linus Torvalds, where he talks about going out of his way to make sure that people can trust his decisions, despite the fact he is paid to work on Linux [1]. In practice, the most obvious step that I can think of, is to defer the decision as to whether you are on the steering committee for six months. I guess over that time you will be working with the other numpy developers more closely. I should say that Stefan and I are working on a governance proposal, in this case for scikit-image, where we split the governance into 1) the developers doing the work and making the day to day decisions and 2) the trustees, usually from other relevant projects, or no-longer-active developers, who make sure the project does not go off the rails. I think you'd be an excellent and obvious trustee, in that model. Cheers, Matthew [1] http://www.bbc.com/news/technology-18419231
Specific examples to support that claim would be appreciated. In particular, examples where an OSS project was corrupted (is that the word?) by a company specifically at the hand of the project's original creator would be especially relevant. Beyond that, what (even in a broad sense) is an example of a goal that "Continuum might need" that would conceivably do detriment to the NumPy community? That it be faster? Simpler to maintain? Easier to extend? Integrate better with more OS projects? Attract new active developers? Receive more financial support? Grow its user base even more? And then, even if there is some sliver of daylight to be uncovered between Travis and the community, what is the current organizational mechanism by which a problematic contribution is forced into NumPy against the will of all other core developers? Finally, is there any previous instance whatsoever that can be pointed to of Travis forcing some change into NumPy contrary to the interest of the community? If not, what actual basis is there to exclude him from a steering committee? I obviously have a point of view; but my opinion here is my own, which is: this sort of pre-emptive impugning of someone's integrity is speculation upon speculation upon speculation Bryan
On 2015-09-21 22:15:55, Bryan Van de Ven <bryanv@continuum.io> wrote:
I don't know how productive it is to dream up examples, but it's not very hard to do. Currently, e.g., the community is not ready to adopt numba as part of the ufunc core. But it's been stated by some that, with so many people running Conda, breaking the ABI is of little consequence. And then it wouldn't be much of a leap to think that numba is an acceptable dependency. There's a broad range of Continuum projects that intersect with what NumPy does: numba, DyND, dask and Odo to name a few. Integrating them into NumPy may make a lot more sense for someone from Continuum than for other members of the community. Stéfan
I don't know how productive it is to dream up examples, but it's not
Well, agreed, to be honest.
very hard to do. Currently, e.g., the community is not ready to adopt numba as part of the ufunc core. But it's been stated by some that,
Who are you speaking for? The entire community? Under what mandate?
The current somewhat concrete proposal I am aware of involves funding cleaning up dtypes. Is there another concrete, credible proposal to make Numba a dependency of NumPy that you can refer to? If not, why are we mired in hypotheticals?
May? Can you elaborate? More speculation. My own position is that these projects want to integrate with NumPy, not the converse. Regardless of my opinion, can you actually make any specific arguements, one way or the otehr? What if if some integrations actually make more sense for the community? Is this simply a dogmatic ideological position that anything whatsoever that benefits both NumPy and Continuum simultaneously is bad, on principle? That's fine, as such, but let's make that position explicit if that's all it is. Bryan
Hi Brian On 2015-09-21 23:28:48, Bryan Van de Ven <bryanv@continuum.io> wrote:
I am speaking on behalf of myself, under no mandate.
I'm sorry if I misunderstood, but I thought you wanted us to explore hypothetical confictual situations.
No, I don't have such a dogmatic ideological position. I think, however, that it is somewhat unimaginative to propose that there are no potential conflicts whatsoever. I am happy if we can find solutions that benefit both numpy and any company out there. But in the end, I'm sure you'd agree that we want the decisions that lead to such solutions to be taken in the best interest of the project, and not be weighed by alterior motivations of any sorts. In the end, even the *perception* that that is not the case can be very harmful. Stéfan
I will only comment on the last point. I completely agree that the *perception* that this is not the case can be harmful. But, what concerns me is where this perception comes from --- from actual evidence of anything that is not in the best interests of the project --- or just ideological differences of opinion about the way the world works and the perceptions around open source and markets. It is quite easy for someone to spread FUD about companies that contribute to open source --- and it has the effect of discouraging companies from continuing to contribute to community projects. This removes a huge amount of potential support from projects. In NumPy's case in particular, this kind of attitude basically guarantees that I won't be able to contribute effectively and potentially even people I fund to contribute might not be accepted --- not because we can't faithfully participate in the same spirit that we have always contributed to SciPy and NumPy and other open source projects --- but because people are basically going to question things just because. What exactly do you need me to say to get you to believe that I have nothing but the best interests of array computing in Python at heart? The only thing that is different between me today and me 18 years ago is that 1) I have more resources now, 2) I have more knowledge about computer science and software architecture and 3) I have more experience with how NumPy gets used. All I can do is continue to try and make things better the best way I know how. -Travis -- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
Hi Bryan, I understand where you're coming from, but I'd appreciate it if we could keep the discussion on a less visceral level? Nobody's personal integrity is being impugned, but it's the nature of this kind of governance discussion that we have to consider unlikely-and-unpleasant hypotheticals. It's like when you talk to a lawyer about a contract or a will or whatever: they'll make you think about all kinds of horrible possibilities, not because any of them are likely, but because sooner or later *something* will go wrong, and the point of having a contract/will/governance document is to provide some framework to handle whichever unlikely edge case does arise. And the other purpose of this kind of framework is to avoid the *perception* (whether justified or not) of these kinds of conflicts of interest -- if not handled well then this can easily scare away volunteers, contributions from other companies, etc. Obviously you know Travis and Continuum well as an employee there, but to most people without that personal connection, Continuum is just a large corporate entity with unknown motives. Imagine if instead of Continuum we were talking about it was Google or RandomAggressiveStartup -- some company that you didn't have any personal connection or insight into. For someone in this position, it's not unreasonable to want more of a reassurance that things will work out then just being asked to trust that the CEO is personally committed to not being evil and they can trust him. Also, in these messages you seem to be working from a framework where people working in good faith will always agree, and so any suggestion of a possible conflict of interest can only arise through bad faith. But this isn't true. Is it really so difficult to imagine that, say, Continuum and Enthought might at some point have conflicting needs for numpy, or for Continuum's vision of the future of numpy could be less-than-perfectly-representative with every bit of numpy's entire giant userbase? Continuum is a company that has in the past submitted rather obscure patches to numpy that AFAICT are used exclusively by a particular contracting customer (e.g. [1]), and that is currently investing a substantial multiple of numpy's development budget on building a direct competitor to numpy. To emphasize: I personally am not concerned by these facts -- we did merge that patch I linked, and there's no animosity between the numpy and dynd teams -- but reasonable people could reasonably be concerned that tricky situations might emerge in the future, and I've talked to plenty of people who are nervous about Continuum's influence in general. And with my numpy developer hat on, I don't even care which "side" is right, that's irrelevant to me, because my concern is with providing a space where both "sides" feel comfortable working together. This is why it's crucial that numpy-the-project be clearly distinguishable as an independent entity that is beholden only to its own community: it's *exactly because* we *value* the contribution of companies like Continuum, and want to be able to freely foster those relationships without creating suspicion and bad blood. Also to emphasize: none of this means that Travis can't be on the steering council -- I think that's a more complex issue that I'll address separately. All I'm saying is that pretending that you aren't going to reassure people by pretending this elephant isn't in the room, or by taking a reasonable set of concerns and aggressively turning them into a referendum on individual people's integrity. Finally, can I point out... anyone who has some wariness around the possible influence of financial interests on the community (whether justified or not!) is in particular not going to be reassured if you keep aggressively attempting to shut down any perceived criticism of *your own employer*. I know that your paycheck is not dictating your opinions, and probably the hypothetical people I'm talking about are being totally unfair to you for even considering such a thing, but... strictly as a matter of practical rhetoric, I don't think this is the most effective way to accomplish your goals. -n [1] https://github.com/numpy/numpy/pull/359 On Mon, Sep 21, 2015 at 11:28 PM, Bryan Van de Ven <bryanv@continuum.io> wrote:
-- Nathaniel J. Smith -- http://vorpus.org
On Tue, Sep 22, 2015 at 2:33 AM, Nathaniel Smith <njs@pobox.com> wrote:
Anybody who comes to NumPy should know that I wrote it and then gave it to the community -- creating an independent foundation at the same time as I created a Company to create a division between them so that my actions could be understood. I really don't know how to give more reassurance. I'm actually offended that so many at BIDS seem eager to crucify my intentions when I've done nothing but give away my time, my energy, my resources, and my sleep to NumPy for many, many years. I guess if your intent is to drive me away, then you are succeeding.
Of course not, but this is no different than anyone else and a company should not be singled out. All you are doing is forcing any contribution to be either only from a non-profit or have individuals hide their actual allegiances.
Good grief! These comments are so much bunk that I have to call you on it emphatically. You claim below that you are unconcerned but yet you insinuate some crazy, ulterior motivations for Continuum actually helping people upgrade their NumPy installation that broke their old code because of changes to NumPy. This kind of comment really upsets me. You dismiss real things and real problems that happen and brush it away because it's *commercial*. That patch you reference was actually an attempt to fix a problem that the community pushed on the world --- therefore breaking people's code (but good thing the ABI didn't change...). We were up front about it and worked with the community to get a change into NumPy to accommodate a user of the tool. It was hard work to figure out how to do that. To have you use that as some kind of argument against Continuum is not only unfair, it is exactly the kind of mis-characterization and mis-interpretation of events that I refer to in other posts. And to say that we are investing a substantial multiple of Numpy's development budget in a competitor is also incorrect. Continuum invests in competent people who want to do interesting things. We don't have a rule that says things are "off-limits" including NumPy. If competent people feel like an alternative to NumPy is a good idea, then a certain amount of exploration in that direction is allowed. DyND does not have to be a competitor to NumPy. It might compete with *your* vision of NumPy, but it doesn't have to compete with NumPy.
Who are these people? How about they come forward and express what it is they are actually nervous about. Really? nervous? What kind of "tricky" situations are we talking about. Can't you see that this sounds as odd to me as me telling you that I'm concerned about BIDS influence? What about Dato, Databricks, Enthought, or Cloudera influence? What does this even mean? Is this just Matthew and Stefan or are there others as well with these feelings? These are the only actual people who have expressed in public what might be considered concern that I am aware of. I think this kind of anti-commercial commentary has no place in a community that also includes people that work at companies. I can't see how we can agree to *any* governance document with this kind of FUD being spread around.
Of course we agree on this. I have no idea why anyone thinks we don't? That's the one thing we *do* agree on. That NumPy is an independent project which can be influenced by anyone in the community and should be developed based on technical discussions and not fear of hob-goblins and people that also work at companies that may benefit from the work that goes on here (there is a large list in this camp). I am deeply saddened by the insinuation and the implication of what these threads are telling me about how little my efforts have been valued by people I care about.
We should call out the elephant in the room. But, I think we should understand who and what the elephant is. Perhaps there are too many off-list and back-channel conversations happening at BIDS and elsewhere that are serving to bias people against facts. Facts that otherwise would show that I and Continuum have always just been trying to ensure the success of NumPy as an independent project that is fully supported, backward compatible, maintained, available to the world in easy to install ways, and documented.
I agree with this. I certainly did not encourage Bryan. He was acting out of his own sense of injustice. But, I would add, that your insinuation and mis-characterization of my activities at Continuum and potentially elsewhere are unfair and incorrect and also not effective at getting governance documents approved and ratified. I will get over feeling offended and work to get over my frustration at academics for thrusting this anti-company and potentially anti-Travis rhetoric on the community. But, if you indeed have such hard feelings, then please air all of them so we can hopefully just get past this.
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On Di, 2015-09-22 at 05:44 -0500, Travis Oliphant wrote:
Frankly, I am bit surprised at how this is developing. Why?! Nobody, except being silly, really would doubt your intentions. That said, I will be honest with you. I do not feel I know you well enough (or the mode you operate) to for example accept you as BDFL of numpy [1] and I frankly do not think that -- just like anyone else -- you should have any special place in the governance document (which obviously does not mean you should be blocked from being on the steering council) [2]. I just think there should probably not be any specific name in the NumPy governance document at all. The thing is, NOBODY really seems suggests that [3]. I have dislike giving any special "power" to ANYONE. Frankly, I think if some people are nervous (not so much those active in the discussion), it is probably because they perceive that you have some direct power over numpy. As you have said, this is just not true. But *because* of the lack of a governance document, I would not be surprised if it *appears* to many like you could wield a lot of power if you so wished. Simply because few people really know how things are decided currently. And I think this wrong perception is what makes some nervous. If you say "maybe we should stop worrying about ABI" it may sound like "two years from now we will definitely break ABI", the curse being that it does not matter that you and those who know numpy's well know that it is just you stressing strongly that we should seriously discuss it. I have to admit, you sometimes sound a bit too definite in your suggestions given your former position ;). I hope I have not been rude here, Sebastian [1] I am sure you were *exactly* the right person to start numpy and be the de-facto BDFL then, but today this is not on the table anyway. [2] Also, lets be honest, you probably do have quite a bit of soft influence, just by knowing the community, being at the conferences, having NumFOCUS close by, etc. [3] I guess you have in some sense, but I now understand that as a suggestion to approach a different problem. And we can find another solution for that problem!
Hi Travis On 2015-09-22 03:44:12, Travis Oliphant <travis@continuum.io> wrote:
I guess we've gone off the rails pretty far at this point, so let me at least take a step back, and make sure that you know that: - I have never doubted that your intensions for NumPy are anything but good (I know they are!), - I *want* the community to be a welcoming place for companies to contribute (otherwise, I guess I'd not be such a fervent supporter of the scientific eco-system using the BSD license), and - I love your enthusiasm for the project. After all, that is a big part of what inspired me to become involved in the first place. My goal is not to spread uncertainty, fear nor doubt—if that was the perception left, I apologize. I'll re-iterate that I wanted to highlight a concern about the interactions of a (somewhat weakly cohesive) community and strong, driven personalities such as yourself backed by a formidable amount of development power. No matter how good your intensions are, there are risks involved in this kind of interaction, and if we fail to even *admit* that, we are in trouble. Lest the above be read in a negative light again, let me state it up-front: *I don't think you will hijack the project, use it for your own gain, or attempt to do anything you don't believe to be in the best interest of NumPy.* What I'm saying is that we absolutely need to move forward in a way that brings everyone along, and makes everyone rest assured that their voice will be heard. Also, please know that I have not discussed these matters with Nathaniel behind the scenes, other than an informal hour-long discussion about his original governance proposal. There is no BIDS conspiracy or attempts at crucifixion. After all, you were an invited guest speaker at an event I organized this weekend, since I value your opinion and insights. Either way, let me again apologize if my suggested lack of insight hurt people's feelings. I can only hope that, in educating me, we all learn a few lessons. Stéfan
To expand on Ryan's point a bit about recusal... this is why we have a general policy against self-merging and why peer review is so valuable. A ban on self-merging is much like recusal, and I think it is a fantastic policy. As for a BDFL, I used to like that idea having seen it work well for Linux and Python, but I have found it at odds with the policy of recusal and no self-merging. That said, as I am sure Thomas Caswell can attest, a non-self-merging policy can result in a lot of ideas getting stalled, waiting for review that may or may not happen. I don't know what the solution is, but I am sympathetic to those who are apprehensive about a BDFL -- regardless of who is in that role. Ben Root On Tue, Sep 22, 2015 at 2:20 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
I completely agree. I don't think self-merging is a good idea. I had gotten used to self merging to SciPy and NumPy until roughly 2008 or 2009 when I was finally broken of that habit after stepping on a few people's toes and learning better habits. On Tue, Sep 22, 2015 at 1:48 PM, Benjamin Root <ben.v.root@gmail.com> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
Hi, On Tue, Sep 22, 2015 at 11:20 AM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
I'm also in favor of taking a step back. The point is, that a sensible organization and a sensible leader has to take the possibility of conflict of interest into account. They also have to consider the perception of a conflict of interest. It is the opposite of sensible, to respond to this with 'how dare you" or by asserting that this could never happen or by saying that we shouldn't talk about that in case people get frightened. I point you again to Linus' interview [1]. He is not upset that he has been insulted by the implication of conflict of interest, he soberly accepts that this will always be an issue, with companies in particular, and goes out of his way to address that in an explicit and reasonable way. Cheers, Matthew [1] http://www.bbc.com/news/technology-18419231
I am not upset nor was I ever upset about discussing the possibility of conflict of interest. Of course it can be discussed --- but it should be discussed directly about specific things --- and as others have said it is generally easily handled when it actually could arise. The key is to understand affiliations. We should not do things in the community that actually encourage people to hide their affiliations for fear of backlash or bias. I was annoyed at the insinuation that conflict of interest is a company-only problem that academics are somehow immune to. I was upset about accusations and mis-interpretations of my activities and those of my colleagues in behalf of the community. On Tue, Sep 22, 2015 at 1:48 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
Of course, and the policies to deal with conflicts have deal with the possibility that *anyone* at all might have conflict. But that was not your suggestion. Your suggestion was to make one individual be subject to additional scrutiny that no one else would be subject to. Please explain why should one person be singled out for a special "six-month waiting period" when the exact same possibility for conflict exists with anyone who is ever on the committee?
Red herring. The objection (as many people have now voiced) is the double standard you proposed.
Your selective quotation is impressive. Also in that interview, Linus points out that his employment contract is probably "unique in the entire world". Which means in practical terms that the details of what he has does are fairly well irrelevant to any other situation. So what is the point in bringing it up, at all, except to try and diminish someone else by comparison? (I'm done.) Bryan
On Tue, Sep 22, 2015 at 8:55 PM, Bryan Van de Ven <bryanv@continuum.io> wrote:
Andrew Morton would be a prominent counter example in the Linux community. He is employed by Google. Many other Linux developers are employed by Red Hat, Intel, and other companies. At this point Linus relies on those people for decisions on what goes into the kernel; it is much too big for a single person to deal with even in review. I also expect the Linux community would be overjoyed if every company provided drivers for their hardware. Subject to review, of course. Chuck
On Tue, Sep 22, 2015 at 10:55 PM, Bryan Van de Ven <bryanv@continuum.io> wrote:
I don't quite understand where the discussion went. The question was not whether Travis is singled out but whether he is "singled in".
Based on all the comments, Travis doesn't have time for this. And I think the final (last resort) decisions about code should be left to the active developers that know and participate in the actual work. If Travis is treated as developer but doesn't have a special status, then there is also no reason to "single out" him and Continuum for possibly too much influence. This is already the current status quo as it developed over the last several years, AFAICS. In my view, in a narrow sense, Travis is a "hit and run" contributor. good ideas and providing good contributions, but somebody has to integrate it, maintain it and fit it into the development plan. (Given my experience I would compare it more with GSOC contributions that need the "core developers" to provide the continuity in the development to absorb the good work.) Travis has too many other obligations and interests to provide this day to day continuity. Travis is still very important for providing ideas, pushing projects forward and as one of the community leaders, but I would leave the final decisions for the development of numpy to the developers in the trenches. I pretty much agree completely with Nathanial, and Sebastian, (except that I don't know much about any other FOSS communities) And to repeat my point from another thread on this: I'm very skeptical about any committee or board that is based on "outside members" and that is involved in the decision about code development. Josef
On Tue, Sep 22, 2015 at 1:20 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
Thank you for the clarification. I'm sorry that I started to question your intentions. I agree that everyone should rest assured that their voice will be heard. I have been and continue to be a staunch advocate for the voices that are not even on this mailing list.
Thank you. I'm sorry for implying otherwise. That was wrong of me. I know we are just trying to bring all the voices to the table. Best, -Travis
Hi all, I would like to pitch in here, I am sorry that I didn't have the time before... First, I want to disclose that recently Continuum made a research gift to the Jupyter project; we were just now writing up a blog post to acknowledge this, but in light of this discussion, I feel that I should say this up front so folks can gauge any potential bias accordingly. On Tue, Sep 22, 2015 at 3:44 AM, Travis Oliphant <travis@continuum.io> wrote:
Travis, first, I'd like to kindly ask you not to conflate BIDS, an institution where a large number of people work, with the personal opinions of some, who happen to work there but in this case are speaking only for themselves. You say "so many at BIDS", but as far as I know, your disagreements are with Stefan and Nathaniel (Matthew doesn't work at BIDS). You are painting with a very wide brush the work of many people, and in the process, unfairly impacting others who have nothing to do with this. If anything, the only things I'm aware BIDS has done in an official capacity regarding you or Continuum is to offer hosting for Continuum developers at the DS4DS workshop and beyond (after an explicit request by Matt Rocklin, and one we were delighted to honor), and our hosting of your lecture in our Friday Data Science Lecture Series last week. With that out of the way... 1. I hope the discussion can move past the suspicion and innuendo about Continuum and Travis. I haven't always agreed with how Travis communicates some of his ideas, and I've said it to him in such instances (e.g. this weekend, as I myself was surprised at how his last round of comments had landed on the list a few days back). But I also have worked closely with him for years because I know that he has proven, not in words, but in actions, that he has the best interests of our community at heart, and that he is willing to try and do everything in his power to help whenever he can. When we founded Numfocus back in 2012, it would have been impossible for it to really bootstrap without Travis' generosity, since he effectively footed the bill for resources that were critically needed at the start. And yet, he was always willing to take every step necessary to help Numfocus grow independent of Continuum, so that it could be a real community asset: today, there's not a single Continuum employee on the NF board (Travis and I both resigned from the board a while back to allow for some fresh blood). The creation and open-sourcing of conda has also been a critical contribution, that I know many of us have benefited from: we all carry the scars from the python packaging horror shows, and conda/anaconda has been a life-changer. The fact that conda itself is open, means we have a core tool that we can build upon. To put it bluntly, few people in the whole world have given more of their life, energy and resources to our community than Travis, and have done so as generously as he has. He may have made mistakes, and again, I often disagree with how he communicates. But accusations and innuendo like the ones in this thread are damaging, hurtful and useless. And one thing that I hope people will remember is that, famous and powerful as Travis may be, he's still our colleague, a member of our community, and *a human being*, so let's remember that as well... 2. Conflicts of interest are a fact of life, in fact, I would argue that every healthy and sufficiently interconnected community eventually *should* have conflicts of interest. They are a sign that there is activity across multiple centers of interest, and individuals with connections in multiple areas of the community. And we *want* folks who are engaged enough precisely to have such interests! For conflict of interest management, we don't need to reinvent the wheel, this is actually something where our beloved institutions, blessed be their bureaucratic souls, have tons of training materials that happen to be not completely useless. Most universities and the national labs have information on COIs that provides guidelines, and Numpy could include in its governance model more explicit language about COIs if desired. So, the issue is not to view COIs as something evil or undesirable, but rather as the very real consequence of operating in an interconnected set of institutions. And once you take that stance, you deal with that rationally and realistically. For example, you accept that companies aren't the only ones with potential COIs: *all* entities have them. As Ryan May aptly pointed out, the notion that academic institutions are somehow immune to hidden agendas or other interests is naive at best... And I say that as someone who has happily stayed in academia, resisting multiple overtures from industry over the years, but not out of some quaint notion that academia is a pristine haven of principled purity. Quite the opposite: in building large and complex projects, I've seen painfully close how the university/government research world has its own flavor of the same power, financial and political ugliness that we attribute to the commercial side. 3. Commercial actors. Following up on the last paragraph, we should accept that *all* institutions have agendas, not just companies. We live in a world with companies, and I think it's naive to take a knee-jerk anti-commercial stance: our community has had a productive and successful history of interaction with industry in the past, and hopefully that will continue in the future. What is true, however, is that community projects should maintain the "seat of power" in the community, and *not* in any single company. In fact, this is important even to ensure that many companies feel comfortable engaging the projects, precisely so they know that the technology is driven in an open and neutral way even if some of their competitors participate. That's why a governance model that is anchored in neutral ground is so important. We've worked hard to make Numfocus the legal entity that can play that role (that's why it's a 501(c)3), and that's why we've framed our governance model for Jupyter in a way that makes all the institutions (including Berkeley and Cal Poly) simply 'partners' that contribute by virtue of supporting employees. But the owners of the decisions are the *individuals* who do the work and form the community, not the companies/institutions. If we accept these premises, then hopefully we can have a rational conversation about how to build a community, where at any point in time, any of us should be judged on the merit of our actions, not the hypotheticals of our intentions or our affiliations (commercial, government, academic, etc). Sorry for the long wall of text, I rarely post on this list anymore. But I was saddened to see the turn of this thread, and I hope I can contribute some perspective (and not make things worse :) Cheers, -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
Hi Fernando, I am happy that you decided to chime in here. This thread derailed in a bad way and I hope that your wise words will help to redress the situation. In fact, I would like to propose you having part of a future steering committee of NumPy. I know that you may never have been implied in the hard core development of NumPy, but my perception is that your opinions are highly respected by almost everybody in the NumPy ecosystem. More than that, you have this rare ability of being able to get both a donation from Microsoft and at the same time (same year?) being awarded by the FSF, which frankly, is not an easy thing to do ;) Just to clear, I am not saying that you should act as the person for deciding the roadmap for NumPy at all, but a person that should be in charge of acting as an independent referee in the foreseeable Conflicts of Interest in the NumPy roadmap. Sorry if that means more work for you Fernando, because I know that you have become a very busy person, but I also know how much do you care about the NumPy ecosystem, and IMHO the NumPy community needs a person like you *now*. Francesc 2015-09-23 10:02 GMT+02:00 Fernando Perez <fperez.net@gmail.com>:
-- Francesc Alted
On Wed Sep 23 12:39:59 2015 GMT+0200, Francesc Alted wrote:
Is it ok if I get a bit angry soon ;)? We can find many great community leaders. But please tell me that you all feel that numpy is so central or whatever that these community leaders should be in some sense in charge of numpy. I am ready to accept that and maybe it can be helpful, and a huge gain for numpy, but I would like to see a clear statement and reasons. I like Fernandos mail, too, nor do I doubt Travis achievements. But discussing who is great community leader, etc. is frankly not obvious to me related to numpy governance. Now if you say: Guys, you should have some community leader guidance in numpy, then we can discuss who. But to me it is not obvious that community leaders who are *not* directly active in numpy should be explicitely in governance. I am aware that everyone wants to help, but right now I do not feel helped at all :). - Sebastian
On Wed, Sep 23, 2015 at 6:55 AM, Sebastian Berg <sebastian@sipsolutions.net> wrote:
Is it ok if I get a bit angry soon ;)?
Don't worry, Sebastian :) I appreciate Francesc's kind words, but I have no intention of imposing my presence anywhere, it's not like I'm looking for extra work at this point. The process of establishing governance has to come organically from within a community. Cheers -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
But discussing who is great community leader, etc. is frankly not obvious to me related to numpy governance.
Thank you Sebastian. Could we please try to get back to the governance issues, without naming names? There are some specific questions on the table that need to get hashed out. Numpy does not have a BDFL. BDFLs are not selected, they are naturally produced, and there is no one that fits that bill now. We *could* decide to have a single individual executive leader, selected by some sort of democratic process. But that is not the same as a BDFL. And I think the almost-consensus now is to not have that. So here is what I think is on the table: We have the steering council. Which leaves two questions: -How big should it be? -Who will be on the original, "seed" council? Note that as I understand the current draft of the governance doc, once established, the council itself decides who is on the council. So at this point we really are ONLY deciding how it's going to start. It has to be bootstrapped somehow. However, that had been contentious enough that it would probably be a good idea to hash out some guidelines about the council membership. Personally, I have no idea how big the council should be. Too big, and there is no point, consensus is harder to reach the larger the group, and the main (only?) role of the council is to resolve issues where consensus has not been reached in the larger community. But what is too big? As for make-up of the council, I think we need to expand beyond people who have recently contributed core code. Yes, the council does need to have expertise to make technical decisions, but if you think about the likely contentious issues like ABI breakage, a core-code focused view is incomplete. So there should be representation by: Someone(s) with a long history of working with the code -- that institutional memory of why decisions were made the way they were could be key. Someone(s) that may not have worked on the core code, but is a major player in the community, perhaps as the leader of a Numpy-dependent project. This would provide representation for the broad community. I do want to note that the governance document as I understand it is consistent with these suggestions. As for conflict of interest issues, etc: Chill out people! Numpy is an open source project, if it gets hijacked, it can be forked. And the council is also democratic -- no one person can drive the project. If a council member is not acting in the interests of the community, s/he can be removed. NOTE: while x.org, and egcs, Xemacs, and ... may be examples of failures of governance, they are also examples of successes of open source. -Chris
On Wed, Sep 23, 2015 at 3:02 AM, Fernando Perez <fperez.net@gmail.com> wrote:
I accept that criticism and apologize for doing that. My *human* side was coming out, and I was not being fair. In my head, though I was also trying to illustrate how some seemed to be doing the same thing for Continuum or other companies. This did not come out very artfully in the early morning hours. I'm sorry. BIDS is doing a lot for the community --- the recent DS4DS workshop, for example, was a spectacularly useful summit --- I hope that many different write-ups and reports of the event make their way out into the world.
I really hope it's just a perception problem (perhaps on my end). There are challenges with working in the commercial world (there are a lot of things to do that have nothing to do with the technology creation) and communicating on open-source mailing lists. As many have noticed, despite my intentions to contribute, I really can't do the same level of contribution personally that I could when I was a student and a professor and had more time. However, I think that it is also under-appreciated (or mis-understood) how much time I have spent with training and helping people who have contributed instead. It's important to me to build a company that can sponsor people to work on open-source (in a community setting). We are still working on that, but it has been my intent. So, far it's actually easier to sponsor new projects than it is to sponsor people on old projects. I am quite sure that if Continuum had put 3 people full time on NumPy in 2012, there would have been a lot of back-lash and mis-understanding. That's why we didn't do it. The collateral effect of that was the creation of other tools that could be somewhat competitive with NumPy long term -- or not. I'd like to learn how to work with the community in an optimal way so that everyone benefits --- and progress happens. That's also why we created Numfocus --- though it is ironic that NumPy has been one of the last projects to actually sign up and be a formally sponsored project. 2. Conflicts of interest are a fact of life, in fact, I would argue that
Thanks for re-emphasizing this. I agree it's O.K. and even necessary to talk about COIs. If I have not been upfront about any possible COIs it's not because of a desire to hide them -- I may not be aware of them myself as I'm still focused on what technically can be better, and how to create a company that can produce as much relevant open source software as possible. I appreciate your helping us get to a better conversational ground. Stefan's and Nathaniel's recent emails have done that as well. Thank you, -Travis
On Wed, Sep 23, 2015 at 12:18 PM, Travis Oliphant <travis@continuum.io> wrote:
Thank you.
Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
Hi, On Wed, Sep 23, 2015 at 1:02 AM, Fernando Perez <fperez.net@gmail.com> wrote: [snip]
1. I hope the discussion can move past the suspicion and innuendo about Continuum and Travis
I'm glad the discussion has become a little more calm now, but I find it difficult not to be annoyed by this statement above. I see a severe reaction to perceived 'suspicion and innuendo', but I see no 'suspicion and innuendo'. Unless you mean that any suggestion of potential conflict of interest is suspicion and innuendo. You imply rudeness and bad faith when you say this. I suppose this must be aimed at me or Stefan or both of us. In any case, I think the accusation is unhelpful and unfair. It makes this discussion more difficult to have in the future. See you, Matthew
On Wed, Sep 23, 2015 at 12:57 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
No, as I said, COIs are absolutely a fact of life, and *should* be talked about, openly and directly. I was referring generically about the tone of this thread, that Ryan described as "bizarre", others as "surpised", "disheartened", etc.
It was sincerely my intention to frame my critique in a specific way that made the discussion *easier* to have in the future, precisely by acknowledging that COIs were part of life, and not only for commercial entities. I was NOT implying that Stefan and you were somehow guilty of anything, I only mentioned your names when I asked Travis not to paint Berkeley folks with a broad brush, that's all. If only the two of you took offense at my wording, in the interest of keeping this already contentious and fraught thread contained, I offer: a) a public apology for my choice of words, since it was certainly not my intent to offend you, and I was not aiming at you personally. b) a suggestion that we discuss it further personally, taking advantage of the fact that we happen to be physically close. If anyone else would like me to answer in public because they feel a slight on my part, I will do so. Best, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
On 2015-09-23 13:36:35, Fernando Perez <fperez.net@gmail.com> wrote:
What I thought Matthew was saying is that you writing "I hope the discussion can move past the suspicion and innuendo about Continuum and Travis" gives validity to the view that those things are real whereas, in my view, they were constructed in responses by others who didn't accurately summarize the intent of what was being said (and I'm not laying blame to anyone, I could certainly have worded myself more clearly). But, yes, it was disheartening—especially because we all know one another and have hacked together late into the night at SciPy conferences. My experience has always been that we are reasonable people who listen; but perhaps we forget that about one another from time to time. It is important to me that we work towards making this mailing list a safe place for discussion again. Part of that may be to do some maintenance on bridges of trust that seem to have eroded a bit over the years. Perhaps some kind of group bonding activity, such as working on a shared project, would help? ;)
b) a suggestion that we discuss it further personally, taking advantage of the fact that we happen to be physically close.
Sure, I'm happy to discuss the personal side of this offline. Stéfan
On Wed, Sep 23, 2015 at 2:31 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
Hey! I want to have a beer with you folks too! Maybe time for a trip back to the old stomping grounds.... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Wed, Sep 23, 2015 at 2:31 PM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
Perhaps some kind of group bonding activity, such as working on a shared project, would help? ;)
Indeed, there's a bunch of fresh 2x4s, screws and bolts with your name on them ;) Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
Hi, On Wed, Sep 23, 2015 at 1:36 PM, Fernando Perez <fperez.net@gmail.com> wrote:
Sure, I said above, it is clearly true the some people reacted very strongly. Did you see any remark made by me or Stefan or anyone else that could reasonably be described as bizarre, surprising or disheartening? Cheers, Matthew
On Wed, Sep 23, 2015 at 2:32 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Did you see any remark made by me or Stefan or anyone else that could reasonably be described as bizarre, surprising or disheartening?
As I said already, I wasn't referring to you personally, but to the tone of the thread. It was the taste that the thread left in my mouth after reading tens of very long emails, and I guess I wasn't the only one, since multiple folks used such adjectives. But it was a poor choice of words on my part. And no, I didn't make a quote-by-quote analysis of the thread, I'm sorry. One last time, it was *not* a personal reference to you: the only reason I mentioned your names was because of the Berkeley clarification regarding BIDS that I asked of Travis, that's all. If that comment hadn't been made, I would not have made any mention whatsoever of anyone in particular. I apologize for not foreseeing that this would have made you feel singled out, in retrospect, I should have. In my mind, it was the opposite, as I felt that you had every right to express whatever opinions you have speaking for yourselves, independent of your affiliations, and I was simply asking Travis to separate individuals from institutions. But I should have realized that calling anyone out by name in a context like this is a bad idea regardless. Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
This was my fault for not being more careful in my words. I felt multiple things when I wrote my emails that led to incorrectly chosen words --- but mostly I was feeling unappreciated, attacked, and accused. I'm sure now that was not intended --- but there have been mis-understandings. I expect they will happen again. I know if we listen to each other and trust that while we may see the world differently and have different framings of solutions --- we can work to coordinate on an important technical activity together. In retrospect, my initial email requesting inclusion on the seed council could have been worded better (as there were multiple things conflated together). I am responding to the actual text of the governance document in the other thread so as to clarify what my proposal actually is in the context of that document. -Travis
On Wed, Sep 23, 2015 at 3:19 PM, Travis Oliphant <travis@continuum.io> wrote:
No worries, I wasn't digging it up further, really! Just clarifying for other reasons, not digging into you :) Glad to see the other thread proceeding further, let's let this one die as peacefully as it can... Best, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail
On Wed, Sep 23, 2015 at 3:37 PM, Fernando Perez <fperez.net@gmail.com> wrote:
Glad to see the other thread proceeding further, let's let this one die as peacefully as it can...
This Monty Python sketch seems relevant: https://www.youtube.com/watch?v=grbSQ6O6kbs (hope everyone's in good health and regains better spirits, back to lurking for me) -- _ / \ A* \^ - ,./ _.`\\ / \ / ,--.S \/ \ / `"~,_ \ \ __o ? _ \<,_ /:\ --(_)/-(_)----.../ | \ --------------.......J Paul Ivanov http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7
On Tue, Sep 22, 2015 at 1:07 AM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
A couple of things to help clarify: 1) nobody believes that the community should be forced to adopt numba as part of ufunc core yet --- but this could happen someday just as Cython is now being adopted but was proposed 8 years ago that it "could be adopted" That's a red-hearing. 2) I have stated that breaking the ABI is of little consequence because of conda as well as other tools. I still believe that. This has nothing to do with any benefit Continuum might or might not receive because of conda. Everyone else who wants to make a conda-based distribution also benefits (Cloudera, Microsoft, Intel, ...) or use conda also benefits. I don't think the community realizes the damange that is done with FUD like this. There are real implications. It halts progress, creates confusion, and I think ultimately damages the community. Numba being an acceptable dependency means a lot more than conda --- it's dependent on LLVM compiled support which would have to be carefully tested --- first as only an optional dependency for many years.
I don't see how. None of these have been proposed for integrating into NumPy. I don't see how integrating numba into NumPy benefits Continuum at all. It's much easier for us to keep it separate. At this point Continuum doesn't have an opinion about integrating DyND into NumPy or not. These projects will all succeed or fail on their own based on users needs. Whether or not they every become a part of NumPy will depend on whether they are useful as such not because a person at Continuum is part of a steering committee (with other people on it). I know that you were responding to specific question by Brian as to how their could be a conflict of interest for Continuum and NumPy development. I don't think this is a useful conversation --- we could dream up all kinds of conflicts of interest for BIDS and NumPy too (e.g. perhaps BIDS really wants Spark to take over and for NumPy to have special connections to Spark). Are we to not allow anyone at BIDS to participate in the steering council because of their other interests? But remember, the original point is whether or not someone from Continuum (or I presume any company and not just singling out Continuum for special treatment) should be on the steering council. Are you really arguing that they shouldn't because there are other projects Continuum is working on that have some overlap with NumPy. I really hope you don't actually believe that. -Travis
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
Hi Travis On 2015-09-21 23:29:12, Travis Oliphant <travis@continuum.io> wrote:
Yes, I'd like to clarify: I was not against including any specific technology in NumPy. I was highlighting that there may be different motivations for members of the general community and those working for, say, Continuum, to get certain features adopted.
This is an old argument, and the reason why we have extensive measures in place to guard against ABI breakage. But, reading what you wrote above, I would like to understand better what FUD you are referring to, because I, rightly or wrongly, believe there is a real concern here that is being glossed over.
I think that touches, tangentially at least, on the problem. If an employee of Continuum were steering NumPy, and the company developed an opinion on those integrations, would such a person not feel compelled to toe the company line? (Whether the company is Continuum or another is besides the point—I am only trying to understand the dynamics of working for a company and leading an open source project that closely interacts with their producs.)
I guess that's an interesting example, but BIDS (which sits inside a university and is funded primarily by foundations) has no financial, and very few other, incentives to do so.
Here's what I'm trying to say (and I apologise for ruffling feathers in the process): There are concerns amongst members of the community that (will) arise when strong players from industry try / hint at exerting (some) executive control over NumPy. We can say that these concerns amount to spreading FUD, that they are uninformed, unrealistic, etc., but ultimately they are still out there, and until they are discussed and addressed, I find it hard to see how we can move forward with ease. Stéfan
On Tue, Sep 22, 2015 at 2:16 AM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
This is what I'm calling you out on. Why? I think that is an unfair statement and inaccurate. The general community includes Continuum, Enthought, Microsoft, Intel, various hedge funds, investment banks and companies large and small. Are you saying that people should not be upfront about their affiliations with a company? That if they are not academics, then they should not participate in the discussion? It is hard enough to be at a company and get time to contribute effort back to an open source project. We should not be questioning people's motives just *because* they are at a company. We should not assume people cannot think in terns of the success of the project, just because they are at a company. Their proposals and contributions can be evaluated on their merits and value --- so this whole discussion seems to be just revealing an anti-company paranoia rather than helping understand the actual concern.
I don't know which is the "old argument". Anyway, old arguments can still be right. The fact is that not breaking the ABI has caused real damage to the community. NumPy was never designed to not have it's ABI broken for over a decade. We have some attempts to guard against ABI breakage --- but they are not perfect. We have not moved the code-base forward for fear of breaking the ABI. When it was hard to update your Python installation that was a concern. There are very few cases where this is still the concern (conda is a big part of it but not the only part as other distros and approaches for easily updating the install exist) --- having this drive major architecture decisions is a serious mistake in my mind, and causes a lot more work than it should. The FUD I'm talking about is the anti-company FUD that has influenced discussions in the past. I really hope that we can move past this.
O.K. if you are honestly asking this question out of inexperience, then I can at least help you understand because perhaps that is the problem (creating a straw-man that doesn't exist). I have never seen a motivated open source developer at a company who "tows the company line" within a community project that is accepted long term. All that would do is drive the developer out of the company and be a sure-fire way to make sure their contributions are not accepted. I know that at Continuum, for example, if someone disagreed with me about an open source technology, didn't tell me and just "towed the company line" (whatever they thought that was) --- they would be fired. Most software companies that participate in open source are like that. The worst thing that happens is the company no longer participates in the discussion --- that is what happens if they can't get contributions. It really is no different than me being worried that an academic might push to get something into a library because it would "help their publication record" --- which I also think is a very real potential concern. Or perhaps you have a favor owed to another colleague who helped you along in your academic career and they "really want" some feature added. The actual conflict of interest that exists there has the same amount of weight in my mind as the ones you hypothesize about.
I think you don't understand how academic finances work. I could see a lot of ways such interest could develop --- and could influence funding agencies. I don't think they are very likely --- but they are possible --- and to me the same degree of possibility that Continuum would try to "force anything" on the NumPy community.
I'm sorry you have these concerns. I don't think they are as warranted as you believe. NumPy will get strong players from industry coming in. In fact, one of the reasons we all felt it was important to establish Numfocus was to be an organization that could shield these players from the development of NumPy and other projects. I am not one of those "strong industry players" you speak of. I am a friend. I am a participant. I am one of the community. Perhaps you and others feel that *I* am that strong industry player trying to exert "executive control" over NumPy. Is that true? Is that what you feel? I'm starting to wonder if that is actually the problem here. -Travis
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On Tue, Sep 22, 2015 at 2:33 AM, Travis Oliphant <travis@continuum.io> wrote:
The FUD I'm talking about is the anti-company FUD that has influenced discussions in the past. I really hope that we can move past this.
I have mostly stayed out of the governance discussion, in deference to how new I am in this community, but I do want to take a moment to speak up here to echo Travis's concern about anti-company FUD. Everyone invested in NumPy has their own projects, priorities and employers which shape their agenda. As far as I can tell, Travis and Continuum have only ever acted with the long term health of the scipy ecosystem in mind. Stephan
On Mon, Sep 21, 2015 at 10:15 PM, Bryan Van de Ven <bryanv@continuum.io> wrote:
I have no expectation that continuum will follow any of these paths, and in most cases am not even sure what that would mean, BUT just because I think it is useful to have a wide variety of concrete examples to draw on -- data is good! -- there actually are *lots* of examples of "community revolts" wresting projects from their original founders, in a variety of corporate and non-corporate contexts. Some examples include the nodejs->iojs fork and merge (which was about wresting control of the project from the founding company), the gcc->egcs fork and merge (which removed RMS's control over day-to-day running of the project), the openoffice->libreoffice fork, the xfree86->x.org fork (where the original core team decided to change the license and all the developers left), the mambo->joomla fork, the xchat->hexchat fork (triggered partially by people's annoyance at the original developer for trying to monetize the project), ... Along somewhat similar lines, there's also the fraught history of Qt and Trolltech and the conflicts between the community and commercial interests there. -n -- Nathaniel J. Smith -- http://vorpus.org
On Mon, Sep 21, 2015 at 8:24 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
I don't really agree that someone couldn't be found. I think the key is that 1) the person would be able to make a decision about what needs to be done if the community is stuck and 2) their is not really a "for life" clause in that their would be a way to call for a replacement.
I don't think this is true. The steering committee would only be called upon to unstick things and make decisions. At those times, it would not take long to come up to speed on the issues and make a decision.
I think this is a red-herring and should not be an issue --- except for obvious situations where I would have to not participate in a "vote" (such as whether or not Continuum would be an institutional sponsor or something like that). Everyone has associations and affiliations and goals and plans that could lead to conflicts of interest. This kind of requirement un-necessarily limits the governance to only certain kinds of people who have only "volunteer" time. This is actually quite damaging as it limits the ability for people to get paid to work on NumPy. This feels like a world-view debate that is actually best left to a different mailing list.
For example, see this interview with Linus Torvalds,
where he talks about going out of his way to make sure that people can trust his decisions, despite the fact he is paid to work on Linux [1].
But this is in a BDFL role and not a steering committee role with 9 other members. I don't think the situation compares or applies at all. The unwarranted fear this kind of reasoning can create in the mind of otherwise reasonable people is unfortunate.
Fundamentally the "steering committee" is too big. I think the committee should be much smaller (no more than 5 and really three is the right size for what it is supposed to actually do). I've had experience with small committees and large committees. Large councils make it very difficult to move things forward. If it is to remain that big, I think it needs people on it like me, Robert Kern, or David Cournapeau who have a longer history with the project and understand why some of the things are the way they are. I still hear far too many conversations that start from a lack of understanding of the early conversations that led to why NumPy is the way it is. This lack of context is not helpful and potentially dangerous. The advanced indexing discussions happening right now and the renewed array-scalar discussions for example are both examples of features that have had previous discussions and have a long history of back and forth between various people.
I like the trustee model too and think such an addition to the NumPy concept would help alleviate my concerns about actually being on a "steering committee" but my preferred outcome is actually that the agreed upon steering council be smaller and that people who have a right to vote on things like the make-up of the steering committee be comprised of people who have been significantly involved in the past 3 years (not just the past one year). -Travis
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
On 2015-09-20 11:20:28, Travis Oliphant <travis@continuum.io> wrote:
I would split the above into two parts: a suggestion on how to change the governance model (first half of 1 and 2) and then some thoughts on what to do once those changes have been made (latter half of 1 and 2, as well as 3). For now, since those changes are not in place yet, it's probably best to focus on the governance model. I would agree that one person (or a very small group) is best suited to "getting things unstuck". And, personally, I believe it best for that person/persons to be elected by the community (whatever we define "the community" to be)---which is what I presume you suggested when you mentioned nominating candidates. Since Matthew mentioned the governance proposal we're working on, here is a very early draft: https://github.com/stefanv/skimage-org/blob/governance_proposal/governance.m... As I said, this is still a work-in-progress--comments are welcome. E.g., the weighting element in the voting has to be fine tuned (but was put in place to prevent rapid take-overs). Essentially, we need: - a way for community members to express disagreement without being ousted, - protection against individuals who want to exert disproportional influence, - protection against those in leadership roles who cause the project long-term harm, - and a way for the community to change the direction of the project if they so wished. Stéfan
Thank you for posting that draft as it is a useful comparison to borrow from. I think Nathaniel's original document is a great start. Perhaps some tweaks along the lines of what you and Matt have suggested could also be useful. I agree that my proposal is mostly about altering the governance model, mixed with some concern about being "automatically disqualified" from a council that can decide the future of NumPy if things don't move forward. -Travis On Tue, Sep 22, 2015 at 12:57 AM, Stefan van der Walt <stefanv@berkeley.edu> wrote:
-- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io
I’ve also stayed out of this until now. I’m surprised and disheartened at the amount of suspicion and distrust directed towards Travis. I don’t think anyone has invested as much personal time and resources (e.g., money) towards supporting numpy, and not just in creating it but through efforts at Continuum and Numfocus. So much of this distrust appears based on what Continuum might do rather than what the actual record is. I agree with Travis that virtually everyone has their own interests. I don’t think non-profit institutions (academia or otherwise) are any more virtuous than for-profit companies when it comes to possible conflicts of interest. As long as everyone understands the interests involved, that shouldn’t bar anyone from participating in governance. If anyone deserves a special seat at the table it is Travis. I’m completely convinced he has the community’s greater interests at heart (not to say that he always understand all the interests and may need input to help in that). Perry
On 22/09/15 14:31, Perry Greenfield wrote:
I’ve also stayed out of this until now. I’m surprised and disheartened at the amount of suspicion and distrust directed towards Travis.
I have no idea where this distrust comes from. Nobody has invested so much of their time in NumPy. Without Travis there would not even be a NumPy.
So much of this distrust appears based on what Continuum might do rather than what the actual record is.
Being CEO of Continuum should not disqualify him in any way.
I agree with Travis that virtually everyone has their own interests.
Even Guido and Linus have employers. Should we distrust Guido as Python BDFL because some day Dropbox will be evil? It is just nonsense. Sturla
Hi All, I've been reading this thread with amazement and a bit of worry. It seems Nathaniel's proposal is clearly an improvement, even if it is not perfect. But it is in the end for a project where, at least as seen from the outside, the main challenge is not in governance, but rather in having only a small group of people who understand the code well enough that they are able to make and judge modifications. As such, this discussion doesn't seem worthy of the effort, and even less of needless heat and irritation, of the type that seems unlikely would have arisen if this conversation had been in person instead of per e-mail. Might it be an idea to accept the proposal provisionally, returning to it a year from now with practical experience? This certainly has the benefit of allowing to switch focus to the more pressing and fortunately also more interesting work to be done on interfacing numpy/ndarray nicely with other classes (i.e., __numpy_ufunc__ and/or similar, and the dtype generalisations, which have me quite intrigued -- either might be very interesting for the Quantity class in astropy, as well as for a work-in-progress Variable class [which propagates uncertainties including covariances]). All the best, Marten -- Prof. M. H. van Kerkwijk Dept. of Astronomy & Astroph., 50 St George St., Toronto, ON, M5S 3H4, Canada McLennan Labs, room 1203B, tel: +1(416)9467288, fax: +1(416)9467287 mhvk@astro.utoronto.ca, http://www.astro.utoronto.ca/~mhvk
I don't think I've contributed code to NumPy itself, but as someone involved in the scientific python ecosystem for a while, I can't see why people would consider Continuum less of a legitimate participant or community member than individual contributors, especially if the person behind it has had the opportunity to control things previously and instead passed NumPy onto the community. I'd be wary of commercial interests dominating the agenda, but that's different from them having a proportionate (in this case minor) say when they have something to offer. And that's all true *even if* Travis were heavily biased to his own commercial ends, which is not consistent with my understanding of his wider efforts and sacrifices over most of a decade that I've been paying attention. Remember this is free/open source software and if enough people don't like the committee at some point, the project can be forked as an option of last resort. Nothing is set in stone, nor code lost. Just saying (I probably won't reply to any criticism or corrections, to avoid adding peripheral noise/heat to the thread). Cheers, James (from, but not on behalf of, a non-profit research facility).
This has to be one of the most bizarre threads I've ever read in my life. Somehow companies are lurking around like the boogeyman and academics are completely free of ulterior motives and conflicts of interest? This is just asinine--we're all people and have various motivations. (Having just gotten out of my university after 15 years, the idea that academics are somehow immune to ulterior motives and conflicts of interest makes me laugh hysterically.) The sad part is that this worry completely unnecessary. This is an open source project, not international politics, and the end goal is to produce software. Therefore, 99.9% of the time motives (ulterior, profit, or otherwise) are completely orthogonal to the question of: IS IT A GOOD TECHNICAL IDEA? It's really that simple: does the proposed change move the project in a direction that we want to go? Now, for the 0.01% of the time, where nobody can agree on that answer, or the question is non-technical, and there is concern about the motives of members of the "council" (or whatnot), it's again simple: RECUSAL. It's a simple concept that I learned in the godawful ethics class NSF forced grad students to take: if you have a conflict of interest, you don't vote. It's how the grownups from the Supreme Court to the college football playoff deal with the fact that people WILL have conflicts; potential conflicts don't disbar qualified individuals from being included in the group, just from weighing in when their decisions can be clouded. So how about we stop making up reasons to discourage participation by (over-)qualified individuals, and actually take advantage of the fact that people actually want to move numpy forward? Ryan
participants (21)
-
Benjamin Root -
Bryan Van de Ven -
Charles R Harris -
Chris Barker -
Chris Barker - NOAA Federal
-
David Cournapeau -
Fernando Perez -
Francesc Alted -
James E.H. Turner -
josef.pktd＠gmail.com -
Marten van Kerkwijk -
Matthew Brett -
Nathaniel Smith -
Paul Ivanov -
Perry Greenfield -
Ryan May -
Sebastian Berg -
Stefan van der Walt -
Stephan Hoyer -
Sturla Molden -
Travis Oliphant