[IPython-dev] Should I still contribute to IPython ?

[Brian Granger]

Matthias,

Thanks so much for bringing this up to the list...you bring up issues
that I have been thinking a lot about recently.  I am sure Fernando
has thoughts too, but he is enjoying some snow up at Mammoth right
now.  I should mention that I will also be visiting Fernando later
this week at Berkeley.  I had planned on talking with him about these
issues so it is a good time to start the discussion...

> This morning I received an personal email from one of our user with the
> following inside:
>
>> I was wondering whether my involvement in the development […] make too
>> much
>> sense now, that IPython has received a grant for the next two years, […]
>> it
>> seems that there are going to be a number of people on the team, who are
>> very
>> proficient, […] and I wouldn't be able to contribute too much, anyway.

Here is my short answer: absolutely, we *really* want and need the
contributions of the larger community.

And my long answer:

A few years back, IPython development was moving along very slowly.
There were only a few core developers and we didn't get many
contributions from the community.  Then we moved the github and
developed the qtconsole, notebook and zmq based parallel capabilities
- development has *exploded*.  Our core team of developers has grown
dramatically and the number of contributions from the larger community
has grown by leaps and bounds.  What stands out to me about this
growth is how it happens.  As Matthias describes so well, it starts by
new people contributing a single small fix, feature, etc.  They they
make another contribution, and then another and another until they are
making massive contributions to the project and helping to steer its
direction.  Why do people do this?  I am convinced that it is *deeply
rewarding and enjoyable* for them.  Why else would they put in so much
effort on something that pulls them away from their day job, friends
and family.  So what criteria should someone use to determine if they
will contribute to the project?

Do you find contributing enjoyable and rewarding?  If you do, then you
should contribute no matter how many of us are paid to work on the
project.

But I think there is another question lurking here: will we continue
to welcome or value the contributions of the larger community, or will
people be excluded if they are not part of the Sloan funded team.
This is something that Fernando and I have talked about extensively
and the answer is that the community is and will always be the
foundation of the project and we will do everything we can to enable
people to participate in the project.  This is extremely important and
something that the Sloan foundation is paying very close attention to.
 In fact, I would say that the Sloan foundation would never have even
talked to us if it were not for the life, excitement and health of our
community.

> I realized, that even we got a lot a congratulation mails after the announce
> of the grant,
> we didn't get any expression of concern or questions on what will happened,
> this was
> a little less the case on hacker-news.
>
> What's follow is my personal opinion, how I feel about the development of
> IPython over the past Year.  Because I feel it is important to tell that
> being founded will not change how we work, will not change the people.
>
> So as you may know, IPython did receive a Sloan Foundation Grant for the
> next
> two years, that will help the development of IPython, But this does not mean
> we
> will ever cut ourselves from our user bases.

Absolutely!

> First of all, being in the core development team does not necessarily mean
> we
> will get any money from the grant.  Some of the current core team dev will
> be
> "hired" to work part or full time on IPython. Personally I won't, even if I
> were
> paid for that it wouldn't change what I would be doing (just maybe my
> schedule).

Yes, this is a very important point: not all of the core devs will be
funded by the Sloan foundation.  That means that our Sloan funded work
*has to* be integrated with the larger project and community.

> This will not prevent me from fixing bug and bring improvement to IPython.
> Nor
> will I assume that Devs that are paid from this grant have more obligation
> to
> fix bugs or answer to question than I am.

Absolutely, the devotion of volunteer developers is in no way inferior
to that of people who are paid to work on the project.

> Being a core dev does not mean that we are more capable than any other
> user of the list. There is no official way to become a core dev, but
> empirically, I would say that when you submit (or review) too many good pull
> request we give you the responsibility  to merge others'.
>
> One of the things that make IPython so great is that we develop things that
> we
> use, I do make patches for IPython because I feel the use for it in my work,
> the diversity of people that contribute and use it make IPython what it is
> now :
> If we look a the opened PR, we see that more than the half are opened by
> people
> that are not in the core team, and *big* PR are in this big half.

Yep, at this point, we receive a significant number of PRs from
non-core developers.

> You might be concern to not being able to keep up with something some
> could do in minutes, and I guess this is the main concern most of you have.
> I
> reassure you we will alway take time to guide you to make a PR.
>
> I first contributed 1 Year ago, from an ~10 lines change that took me a week
> to
> write, I had kind words from fperez, epatters, and minrk to explain me how
> to
> do things they could have done in 10 minutes tops.
>
> Now I do the same for every pull request from users. Even if fixing a bug
> would
> take me less time than writing the explanation for an user, I still write
> the
> explanation and help do the right thing even for one line. I do strongly
> believe that this is worth doing in the long run. And I know that I am not
> the
> only one that will keep up doing so as long as he can. Often the most
> difficult
> thing is to know the code base, not actually writing the patch, and nothing
> but
> time helps that.
>
> Heck, the Sloan-Foundation almost exclusively give to project linked to
> education, if we can't be good teacher, how could build a tool to help other
> to
> teach ?

Absolutely...

> So yes I will continue to contribute, on my free time, I hope I'll find some
> way to get more time to be involve. Yes the development will probably go
> faster, and there are things I won't be able to follow, but even now, I
> already
> discover that some stuff had been added to IPython during the week that I
> wasn't aware of, and I'm convinced there are still room for all kind of
> contribution, from QTconsole UI to big AST hook to do amazing stuff I don't
> even dream of.
>
> Hopping to have convince you that we will not change, and that you will
> continue to
> bring us big surprises with awesome idea and PRs.
>
> Thanks to all of you, also and especially to Fernando without whom IPython
> would not exist today.

Absolutely, thank you to *everyone* who participates in the project
and community - and to Fernando for blowing off his thesis..

So I think there are some important issues that the Sloan foundation
grant does bring up, that we will have to work though.  Right now we
are already flooded by the number of pull requests and contributions
that we are getting.  With 5 of us working full time, the level of
activity is going to go through the roof.  This is exciting, but will
definitely change some aspects of the project.  Some of the questions
that we are thinking about right now:

* How do we coordinate all of this activity?  Already, our github
issues are becoming unusable because of the sheer number of them.  We
have to figure out a better way of using github issues, wiki pages,
etc. to coordinate the increased activity the grant will bring.
* Currently, we don't really have any long/medium term planning for
the project.  Our current model works great for attracting the types
of contributions we are getting right now, but it makes it very
difficult to tackle larger projects that require a coordinated and
sustained effort by multiple people.  I don't know what it will look
like, but we are going to need to do this type of planning for the
project.
* How do we manage communication?  Verbal communication is much more
efficient than emails or even IRC.  The 4 people at Berkeley will have
an incredible advantage in being able to talk daily.  We don't want to
cripple or remove that advantage, but we need to figure out how to
include other core devs and people from the community.  This is
particularly relevant to myself as I am the only person involved in
the Sloan work that is not at Berkeley.

Cheers,

Brian


> Sorry for the length,
> --
> Matthias
>
>
>
> [Grant announce on HN] http://news.ycombinator.com/item?id=4909070
> And a small link, also from hacker news to conclude :
> [Dear Open Source Project Leader: Quit Being A Jerk]
> http://news.ycombinator.com/item?id=4921152
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



--
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com