[IPython-dev] Feature and scope creep in IPython

Fernando Perez fperez.net at gmail.com
Thu Jan 10 20:38:16 EST 2013

Hi folks,

On Wed, Jan 9, 2013 at 11:12 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> I think this issue is really important for the project to consider if
> we are going to successfully scale our efforts.  Please read this,
> think about it, and post your thoughts here, as it related to IPython.
>  At a practical level, we need to figure out better work flow for our
> GitHub issues, especially ones for new features.  I propose one option
> in my blog post, but I am sure there are others.

I just wanted to mention that Brian has encapsulated here very much my
thoughts as well (and not by accident, as we've been talking a lot
about this).  It's important to note that this is NOT a 'hostile'
attitude towards new contributions, since we are keenly aware that the
value of IPython is in its utility to end users, who tend to always
want new features for their problems.  And who are kind and energetic
enough to often go out and contribute them, for which we're incredibly

But this is a careful consideration of the painful reality that it's
possible for a project to die under the weight of too much code, too
many features.  So the challenge we are facing is how to keep a
project that is very, very useful to people and allows them to do
novel things, and yet stays tight and solid enough to be
comprehensible and manageable by a relatively small team.

I want to illustrate the kind of thing I see as where we want to go,
with some examples from our own development:

- the cell magics work: it enabled a ton of new features, but the
underlying implementation was actually a major *reduction* of internal
complexity, as we were able to factor a bunch of scattered
functionality into a single model.

- the merge of the parallel and interactive kernel into a single
object: again, less code to be maintained but the automatic gain of
all 'real ipython' functionality (magics, special syntax, etc) in all
parallel contexts.

- the ongoing effort to rationalize our input transformation machinery
(IPEP 2 and PR #2447): while not done yet, we're trying precisely to
get all the functionality we want with an ultimately simpler system.

So we hope that it's clear that the vision carefully detailed by Brian
is not a user-hostile position, quite the opposite.  It's simply
driven by a very strong desire to ensure that, two years after we have
put a solid team working full time on this, we are standing in front
of a project that is tight, comprehensible and a very robust
foundation for the work of others, and not a sprawling mess of many
features that nobody understands internally.

It is very easy to die under the weight of complexity, and we really,
really want to avoid that.

A big thanks to Brian for taking the time to articulate these ideas so
well; one of the things we'll be doing more as we move forward (and
Aaron pointed that out in the comments) is trying to lay out these
ideas and vision in an explicit way, so that everyone has a common
reference point they can go to, rather than working off implicit
assumptions and finding out after a lot of work they had gone in the
wrong direction.



More information about the IPython-dev mailing list