[IPython-dev] Proposal: soft moratorium on re-architecting for 5.0

Nicholas Bollweg nick.bollweg at gmail.com
Mon Jun 29 08:44:08 EDT 2015


Wow, go away for a weekend and all hell breaks loose! Really good stuff
here.

+1 to docs living in notebooks, and using as their primary presentation,
the notebook ui.

When authoring a notebook, what if a user didn't have to leave the ui to go
search five websites to find out a) what a particular class does and b)
what other classes do that?

For why, see the Mathematica documentation, which is really a better
yardstick than other open source projects.

We already maintain and ship to every user a sophisticated document
management platform, the benefits of which are mostly destroyed when
serialized to static html. Mix in indexed search, and wiki-style "what
links here", and we've got something really powerful. If sphinx plays a
role in exposing source files as notebooks, so much the better, but
Mathematica, for example, has the burden of documenting everything for
users in prose/signatures. I'm sure their developers write beautiful source
for each other, as the platform is now something like 30 years old....
Can't be much debt left, right?

Bringing in another thread, typescript will give us the descriptive power
to provide deep, linked data links to browser code, and improve
browser-side documentation generally. Phosphor then also has the ui
abstracting power to make an in browser documentation viewer unsurprising.

Additionally, kernel authors, kernel-agnostic nbextenders and such would
then have a compelling reason to author, ship and maintain notebooks... A
hook here and there, and you are indexed along with everything else.

Something like ipymd could close the vcs maintainability gap (once it has
metadata), while nosebook (once it does coverage, and in addition to other
tests) could close the loop, making the shipping of uncovered/undocumented
code a much more intentional act.

Excited for more discussion!

On 06:55, Mon, Jun 29, 2015 Fernando Perez <fperez.net at gmail.com> wrote:

> Here's an email that Andrew sent me privately, but with his authorization,
> I'm replying to it on the public list as I think his concerns have a lot of
> merit and I'm sure others would be interested in the topic as well...
>>
> On Fri, Jun 26, 2015 at 9:56 PM, Andrew Gibiansky <
> andrew.gibiansky at gmail.com> wrote:
>
>> Fernando,
>>
>> However, I would like to urge you and the IPython team to consider
>> focusing a little bit of effort towards something that has gone
>> unmentioned: *documentation*.
>>
>> The IPython documentation has suffered from a few main problems:
>>
>>    - It is often out of date. The messaging spec that was accessible, as
>>    of a few weeks ago, was out of date and simply wrong about some things in
>>    IPython 3.1.
>>    - Significant portions are undocumented, or documented only through
>>    their original IPEPs. For example, the backbone widgets! I am currently
>>    working with a student for GSOC to implement widget support for IHaskell,
>>    and he regularly has to resort to spelunking through the IPython source to
>>    implement the same functionality, rather than looking at a spec.
>>    - Finally, and *most* importantly, the frontend is completely
>>    undocumented. As far as I can tell, the suggested way towards developing
>>    anything for the frontend is to grep through the source code, for example
>>    for which events are available to be bound to. The result of this is that
>>    developing frontend custom.js or extensions is nigh impossible; even simple
>>    ones become a gargantuan task requiring you to delve deep into IPython JS
>>    source code, and more extensive extensions can only be done by core IPython
>>    contributors (or people with similarly deep knowledge). As someone who is
>>    very familiar with IPython (having written IHaskell and a few frontend
>>    extensions and customizations), it is very difficult even for me to get
>>    anything done on the frontend, much less someone with less experience. I
>>    believe this *significantly* hinders development of new extensions,
>>    widgets, and frontends.
>>
>> Seeing Thomas' suggestion, I think it would be *really* great if the
>> team could tackle the occasionally poor documentation right now. (I say
>> 'occasionally' because the documentation is still much much better than
>> many other projects!) From an external standpoint, it seems like the amount
>> of technical debt in IPython is growing somewhat unmanageable to deal with
>> for an outsider, in part because of the lack of documentation. Even if now
>> is not the right time for a focus on documentation, it would be wonderful
>> if this were brought up and if perhaps it were scoped out as part of the
>> schedule for the upcoming months.
>>
>> Again, please take this in the best way possible – this is not meant as
>> criticism of the way IPython is being developed but rather as an expression
>> of what would be important to me, a very frequent user of IPython and
>> developer of a fairly popular backend. With all of this in mind, I want to
>> say that I'm incredibly impressed with the quality and pace of development,
>> and it's exciting to see plans for 4.0 and 5.0 being made!
>>
>
> These concerns are absolutely spot-on, and we're painfully aware of the
> impact that the rather sad state of our docs has on users and developers of
> the system alike.  Furthermore, we know that it creates a significant
> barrier to entry to new contributions, and to growing a healthier community
> of participation around the project.
>
> We have, sadly, dug a pretty deep hole here, and it will take some real
> effort to climb back out of it, so this isn't going to happen overnight.
>
> We've been, over the last few months, working very hard on trying to
> secure some resources to ensure that the core team can remain employed,
> which is an even more critical concern for some of us, as I hope you'll
> understand :)
>
> We hope to have some good news to report on that front before too long,
> and once we can get some proper resources in place, we actually will put
> documentation efforts as a first-class priority.  Even before that, we have
> a few small steps in the right direction:
>
> - Min went this spring to the WriteTheDocs conference, to start meeting
> technical writers and get a bit more acquainted with that community,
>
> - Brian has already hired some new summer students who are starting to
> improve the layout of the websites.  That's not the same as writing a bunch
> of new docs, but having better websites will help.  The PRs are here:
> https://github.com/jupyter/design/issues/15,
> https://github.com/jupyter/design/issues/16, help always welcome.
>
> - We've actually already started work on restructuring all the repos for
> better docs, and I know that Thomas, Brian, Jess, Jon, Matthias and others
> (sorry folks for whoever I'm not crediting directly) have put work into it.
>   At least now we have an overall skeleton so that the navigation of the
> docs for the overall project is more comprehensible:
>
> http://jupyter.readthedocs.org/en/latest/
>
> and we're also working on tooling to smooth the integration of notebooks
> into the sphinx workflow.
>
>
> These efforts are all partial, and cooking at slow simmer because people
> are more or less at max capacity trying to get 4.0 out the door and with
> the Scipy conference breathing down our necks.
>
> I would say though that, once 4.0 is out the door, with this new skeleton
> in place, contributing docs will be a place where the community can
> *definitely* help the project.  I know until now it was hard to even help
> with documentation, because the layout of our docs was so messy and
> monolithic.  We're trying to precisely help with making it easier to
> *contribute*, please let us know if this new structure looks like a step in
> the right direction or not...
>
>
> But we are not, not at all, ignoring your concerns. Hopefully in a few
> days/weeks we'll have a bit more we can share on that front that will help
> too...
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150629/b7555455/attachment.html>


More information about the IPython-dev mailing list