[Numpy-discussion] What is consensus anyway

[josef.pktd at gmail.com]

On Wed, Apr 25, 2012 at 7:08 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Hi,
>
> On Wed, Apr 25, 2012 at 3:24 PM,  <josef.pktd at gmail.com> wrote:
>> On Wed, Apr 25, 2012 at 5:54 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
>>> Hi,
>>>
>>> On Wed, Apr 25, 2012 at 2:35 PM, Travis Oliphant <travis at continuum.io> wrote:
>>>>>
>>>>> Do you agree that Numpy has not been very successful in recruiting and
>>>>> maintaining new developers compared to its large user-base?
>>>>>
>>>>> Compared to - say - Sympy?
>>>>>
>>>>> Why do you think this is?
>>>>
>>>> I think it's mostly because it's infrastructure that is a means to an end.   I certainly wasn't excited to have to work on NumPy originally, when my main interest was SciPy.    I've come to love the interesting plateau that NumPy lives on.    But, I think it mostly does the job it is supposed to do.     The fact that it is in C is also not very sexy.   It is also rather complicated with a lot of inter-related parts.
>>>>
>>>> I think NumPy could do much, much more --- but getting there is going to be a challenge of execution and education.
>>>>
>>>> You can get to know the code base.  It just takes some time and patience.   You also have to be comfortable with compilers and building software just to tweak the code.
>>>>
>>>>
>>>>>
>>>>> Would you consider asking that question directly on list and asking
>>>>> for the most honest possible answers?
>>>>
>>>> I'm always interested in honest answers and welcome any sincere perspective.
>>>
>>> Of course, there are potential explanations:
>>>
>>> 1) Numpy is too low-level for most people
>>> 2) The C code is too complicated
>>> 3) It's fine already, more or less
>>>
>>> are some obvious ones. I would say there are the easy answers. But of
>>> course, the easy answer may not be the right answer. It may not be
>>> easy to get right answer [1].   As you can see from Alan Isaac's reply
>>> on this thread, even asking the question can be taken as being in bad
>>> faith.  In that situation, I think you'll find it hard to get sincere
>>> replies.
>>
>> I don't see why this shouldn't be the sincere replies, I think these
>> easy answers are also the right answer for most people.
>
> I wasn't saying these replies are not sincere, of course they are factors.
>
> I have heard other people give reasons why they didn't enjoy numpy
> development much, but I can't speak for them, only for me.
>
> I have done some numpy development, but very little.
>
> I've done a moderate amount of scipy development.
>
> I have considered doing more numpy development, in particular, I did
> want to do some work on the longdouble parts of numpy.
>
> Part of the reason I didn't do this was because, when I raised the
> question on the list, it did not seem there was much interest in a
> change, or even a real discussion.
>
> Partly from the masked array discussions, but not only, it seemed that
> the process of making decisions was not clear, and there seemed to be
> as many views about how this was done as there were developers.
>
> I suppose I'd summarize the atmosphere, as I have have felt it, as
> being that numpy was owned by someone else, and I wasn't quite sure
> who that was, but I was fairly sure it wasn't me.   On the other hand,
> in some projects at least - of which Sympy is the most obvious
> example, I think it's easy to feel that all of us own Sympy (and I've
> only made one commit to Sympy, and that of someone else's idea).
>
> Adding to that, it does seem to me that the atmosphere on this list
> get ugly sometimes.  In particular it seems to me that there's a sort
> of conformity that starts to emerge in which people feel it is
> necessary to praise or criticize people, but not the arguments.   I
> suppose that is because there was a long time during which Travis was
> not on the list to model what kind of discussion he wanted.  I'm glad
> that has changed now.
>
> The reason I keep returning to process, even though it is
> 'non-technical' - is because it seems to me that the atmosphere that
> I'm describing will have the strong effect of discouraging
> enthusiastic developers.  It certainly discourages me.  I don't think
> open-source software is just developers scratching an itch, I think
> it's about community, and the pleasure of working with people you like
> and trust, to do something you think is important.

Except for the big changes like NA and datetime, I think the debate is
pretty boring.
The main problem that I see for discussing technical issues is whether
there are many
developers really interested in commenting on code and coding.
I think it mostly comes down to the discussion on tickets or pull requests.

First my own experience with scipy.stats. Most of the time when I was
cleaning up scipy.stats,
I was alone, except for some helpful comments by Robert. My "itch" was that the
bugs in scipy.stats were bugging me, and I just kept working and
committing without
code review until the bugs that I thought urgent were gone.
Now, with Warren and Ralf also working on scipy.stats it is a lot more
fun, since there
is actually a "regular" community of 3 developers.

My impression (since I only pay partially attention to this) is that
Pierre's work on np.ma masked arrays and David's first cleanup of the
c source and build issues were pretty lonely jobs. Just get the work
done with whatever motivation.

Chuck is writing np.polynomial, but since Anne left there is very
little discussion on the details.

My impression is also that the numpy community is much more a user
than a developer community. The big successful community project (with
some financial support) was the documentation improvement which made
it possible for many users to contribute.
(The user community is moving to stackoverflow.)

So, I don't think it will be easy or possible to get the same
enthusiasm for building this great new package such as sympy or
scikit-learn with numpy or scipy. But still scipy gets one module
after another cleaned up and new enhancements, but often it's a
contribution to make a function available to other users, not because
we are part of the scipy community but because we are part of the
SciPy community.

slightly skewed view from a numpy user

Josef


>
> If I've made that harder, then I am sorry, and I'm very happy to hear
> why that is, and how I can help.
>
> Best,
>
> Matthew
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion