[IPython-dev] Proposal: soft moratorium on re-architecting for 5.0

[Brian Granger]

Fernando,

Could you start the documentation stuff as a separate thread? This one
is getting a bit difficult to follow (remembering why email is so
painful :(

Cheers,

Brian

On Sat, Jun 27, 2015 at 3:53 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Here's an email that Andrew sent me privately, but with his authorization,
> I'm replying to it on the public list as I think his concerns have a lot of
> merit and I'm sure others would be interested in the topic as well...
>
>
> On Fri, Jun 26, 2015 at 9:56 PM, Andrew Gibiansky
> <andrew.gibiansky at gmail.com> wrote:
>>
>> Fernando,
>>
>> However, I would like to urge you and the IPython team to consider
>> focusing a little bit of effort towards something that has gone unmentioned:
>> documentation.
>>
>> The IPython documentation has suffered from a few main problems:
>>
>> It is often out of date. The messaging spec that was accessible, as of a
>> few weeks ago, was out of date and simply wrong about some things in IPython
>> 3.1.
>> Significant portions are undocumented, or documented only through their
>> original IPEPs. For example, the backbone widgets! I am currently working
>> with a student for GSOC to implement widget support for IHaskell, and he
>> regularly has to resort to spelunking through the IPython source to
>> implement the same functionality, rather than looking at a spec.
>> Finally, and most importantly, the frontend is completely undocumented. As
>> far as I can tell, the suggested way towards developing anything for the
>> frontend is to grep through the source code, for example for which events
>> are available to be bound to. The result of this is that developing frontend
>> custom.js or extensions is nigh impossible; even simple ones become a
>> gargantuan task requiring you to delve deep into IPython JS source code, and
>> more extensive extensions can only be done by core IPython contributors (or
>> people with similarly deep knowledge). As someone who is very familiar with
>> IPython (having written IHaskell and a few frontend extensions and
>> customizations), it is very difficult even for me to get anything done on
>> the frontend, much less someone with less experience. I believe this
>> significantly hinders development of new extensions, widgets, and frontends.
>>
>> Seeing Thomas' suggestion, I think it would be really great if the team
>> could tackle the occasionally poor documentation right now. (I say
>> 'occasionally' because the documentation is still much much better than many
>> other projects!) From an external standpoint, it seems like the amount of
>> technical debt in IPython is growing somewhat unmanageable to deal with for
>> an outsider, in part because of the lack of documentation. Even if now is
>> not the right time for a focus on documentation, it would be wonderful if
>> this were brought up and if perhaps it were scoped out as part of the
>> schedule for the upcoming months.
>>
>> Again, please take this in the best way possible – this is not meant as
>> criticism of the way IPython is being developed but rather as an expression
>> of what would be important to me, a very frequent user of IPython and
>> developer of a fairly popular backend. With all of this in mind, I want to
>> say that I'm incredibly impressed with the quality and pace of development,
>> and it's exciting to see plans for 4.0 and 5.0 being made!
>
>
> These concerns are absolutely spot-on, and we're painfully aware of the
> impact that the rather sad state of our docs has on users and developers of
> the system alike.  Furthermore, we know that it creates a significant
> barrier to entry to new contributions, and to growing a healthier community
> of participation around the project.
>
> We have, sadly, dug a pretty deep hole here, and it will take some real
> effort to climb back out of it, so this isn't going to happen overnight.
>
> We've been, over the last few months, working very hard on trying to secure
> some resources to ensure that the core team can remain employed, which is an
> even more critical concern for some of us, as I hope you'll understand :)
>
> We hope to have some good news to report on that front before too long, and
> once we can get some proper resources in place, we actually will put
> documentation efforts as a first-class priority.  Even before that, we have
> a few small steps in the right direction:
>
> - Min went this spring to the WriteTheDocs conference, to start meeting
> technical writers and get a bit more acquainted with that community,
>
> - Brian has already hired some new summer students who are starting to
> improve the layout of the websites.  That's not the same as writing a bunch
> of new docs, but having better websites will help.  The PRs are here:
> https://github.com/jupyter/design/issues/15,
> https://github.com/jupyter/design/issues/16, help always welcome.
>
> - We've actually already started work on restructuring all the repos for
> better docs, and I know that Thomas, Brian, Jess, Jon, Matthias and others
> (sorry folks for whoever I'm not crediting directly) have put work into it.
> At least now we have an overall skeleton so that the navigation of the docs
> for the overall project is more comprehensible:
>
> http://jupyter.readthedocs.org/en/latest/
>
> and we're also working on tooling to smooth the integration of notebooks
> into the sphinx workflow.
>
>
> These efforts are all partial, and cooking at slow simmer because people are
> more or less at max capacity trying to get 4.0 out the door and with the
> Scipy conference breathing down our necks.
>
> I would say though that, once 4.0 is out the door, with this new skeleton in
> place, contributing docs will be a place where the community can
> *definitely* help the project.  I know until now it was hard to even help
> with documentation, because the layout of our docs was so messy and
> monolithic.  We're trying to precisely help with making it easier to
> *contribute*, please let us know if this new structure looks like a step in
> the right direction or not...
>
>
> But we are not, not at all, ignoring your concerns. Hopefully in a few
> days/weeks we'll have a bit more we can share on that front that will help
> too...
>
> Cheers,
>
> f
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger at calpoly.edu and ellisonbg at gmail.com