[Numpy-discussion] Governance model request

Tue Sep 22 05:33:24 EDT 2015

On Tue, Sep 22, 2015 at 2:16 AM, Stefan van der Walt <stefanv at berkeley.edu>
wrote:

> Hi Travis
>
> On 2015-09-21 23:29:12, Travis Oliphant <travis at continuum.io> wrote:
> >   1) nobody believes that the community should be forced to adopt numba
> as
> > part of ufunc core yet --- but this could happen someday just as Cython
> is
> > now being adopted but was proposed 8 years ago that it "could be adopted"
> > That's a red-hearing.
>
> Yes, I'd like to clarify: I was not against including any specific
> technology in NumPy.  I was highlighting that there may be different
> motivations for members of the general community and those working for,
> say, Continuum, to get certain features adopted.
>

This is what I'm calling you out on.  Why?   I think that is an unfair
statement and inaccurate.   The general community includes Continuum,
Enthought, Microsoft, Intel, various hedge funds, investment banks and
companies large and small.   Are you saying that people should not be
upfront about their affiliations with a company?  That if they are not
academics, then they should not participate in the discussion?   It is hard
enough to be at a company and get time to contribute effort back to an open
source project.    We should not be questioning people's motives just
*because* they are at a company.   We should not assume people cannot think
in terns of the success of the project, just because they are at a company.

Their proposals and contributions can be evaluated on their merits and
value --- so this whole discussion seems to be just revealing an
anti-company paranoia rather than helping understand the actual concern.

> >   2) I have stated that breaking the ABI is of little consequence because
> > of conda as well as other tools.    I still believe that.  This has
> nothing
> > to do with any benefit Continuum might or might not receive because of
> > conda.   Everyone else who wants to make a conda-based distribution also
> > benefits (Cloudera, Microsoft, Intel, ...) or use conda also benefits.
> > I don't think the community realizes the damange that is done with FUD
> like
> > this.  There are real implications.  It halts progress, creates
> confusion,
> > and I think ultimately damages the community.
>
> This is an old argument, and the reason why we have extensive measures
> in place to guard against ABI breakage.  But, reading what you wrote
> above, I would like to understand better what FUD you are referring to,
> because I, rightly or wrongly, believe there is a real concern here that
> is being glossed over.
>

I don't know which is the "old argument".   Anyway, old arguments can still
be right.  The fact is that not breaking the ABI has caused real damage to
the community.  NumPy was never designed to not have it's ABI broken for
over a decade.     We have some attempts to guard against ABI breakage ---
but they are not perfect.

We have not moved the code-base forward for fear of breaking the ABI.
When it was hard to update your Python installation that was a concern.
There are very few cases where this is still the concern (conda is a big
part of it but not the only part as other distros and approaches for easily
updating the install exist) --- having this drive major architecture
decisions is a serious mistake in my mind, and causes a lot more work than
it should.

The FUD I'm talking about is the anti-company FUD that has influenced
discussions in the past.    I really hope that we can move past this.

>
> > I don't see how.    None of these have been proposed for integrating into
> > NumPy.    I don't see how integrating numba into NumPy benefits Continuum
> > at all.  It's much easier for us to keep it separate.   At this point
> > Continuum doesn't have an opinion about integrating DyND into NumPy or
> > not.
>
> I think that touches, tangentially at least, on the problem.  If an
> employee of Continuum were steering NumPy, and the company developed an
> opinion on those integrations, would such a person not feel compelled to
> toe the company line?  (Whether the company is Continuum or another is
> besides the point—I am only trying to understand the dynamics of working
> for a company and leading an open source project that closely interacts
> with their producs.)
>

O.K.  if you are honestly asking this question out of inexperience, then I
can at least help you understand because perhaps that is the problem
(creating a straw-man that doesn't exist).    I have never seen a motivated
open source developer at a company who "tows the company line" within a
community project that is accepted long term.    All that would do is drive
the developer out of the company and be a sure-fire way to make sure their
contributions are not accepted.   I know that at Continuum, for example, if
someone disagreed with me about an open source technology, didn't tell me
and just "towed the company line" (whatever they thought that was) --- they
would be fired.   Most software companies that participate in open source
are like that.   The worst thing that happens is the company no longer
participates in the discussion --- that is what happens if they can't get
contributions.

It really is no different than me being worried that an academic might push
to get something into a library because it would "help their publication
record" --- which I also think is a very real potential concern.    Or
perhaps you have a favor owed to another colleague who helped you along in
your academic career and they "really want" some feature added.     The
actual conflict of interest that exists there has the same amount of weight
in my mind as the ones you hypothesize about.

>
> > I know that you were responding to specific question by Brian as to how
> > their could be a conflict of interest for Continuum and NumPy
> development.
> >     I don't think this is a useful conversation --- we could dream up all
> > kinds of conflicts of interest for BIDS and NumPy too (e.g. perhaps BIDS
> > really wants Spark to take over and for NumPy to have special connections
> > to Spark).   Are we to not allow anyone at BIDS to participate in the
> > steering council because of their other interests?
>
> I guess that's an interesting example, but BIDS (which sits inside a
> university and is funded primarily by foundations) has no financial, and
> very few other, incentives to do so.
>

I think you don't understand how academic finances work.   I could see a
lot of ways such interest could develop --- and could influence funding
agencies.   I don't think they are very likely --- but they are possible
--- and to me the same degree of possibility that Continuum would try to
"force anything" on the NumPy community.

>
> > But remember, the original point is whether or not someone from Continuum
> > (or I presume any company and not just singling out Continuum for special
> > treatment) should be on the steering council.    Are you really arguing
> > that they shouldn't because there are other projects Continuum is working
> > on that have some overlap with NumPy.    I really hope you don't actually
> > believe that.
>
> Here's what I'm trying to say (and I apologise for ruffling feathers in
> the process):
>
> There are concerns amongst members of the community that (will) arise
> when strong players from industry try / hint at exerting (some)
> executive control over NumPy.  We can say that these concerns amount to
> spreading FUD, that they are uninformed, unrealistic, etc., but
> ultimately they are still out there, and until they are discussed and
> addressed, I find it hard to see how we can move forward with ease.
>

I'm sorry you have these concerns.    I don't think they are as warranted
as you believe.   NumPy will get strong players from industry coming in.
In fact, one of the reasons we all felt it was important to establish
Numfocus was to be an organization that could shield these players from the
development of NumPy and other projects.

I am not one of those "strong industry players" you speak of.   I am a
friend.  I am a participant.  I am one of the community.  Perhaps you and
others feel that *I* am that strong industry player trying to exert
"executive control" over NumPy.    Is that true?    Is that what you feel?
  I'm starting to wonder if that is actually the problem here.

-Travis

> Stéfan
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>

-- 

*Travis Oliphant*
*Co-founder and CEO*

@teoliphant
512-222-5440
http://www.continuum.io
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20150922/e598a0e5/attachment.html>