[IPython-dev] Re: Notebook roadmap

[Robert Kern]

Toni Alatalo wrote:

> That and the discussions here have been something to digest indeed, but 
> the picture is pretty clear for me now (as in understanding the targets 
> and the Robert's implementation).
> 
> Where next? Tzanko, can you interface with the 'notabene' API, is there 
> something to discuss about it, or things missing I could start to add?
> 
> Some random remarks from reading the code:
> 
> - i also like the target of ReST output, and now we perhaps don't need 
> to write that ourselves, 'cause a xhtml2rest 'renderer' was recenlty 
> posted to docutils-dev. added a comment about that to the wiki page with 
> a link to the code and discussions at 
> http://permalink.gmane.org/gmane.text.docutils.devel/3076

I didn't really have in mind a reST target. Embedding the
<ipython-block> tags in text was just the simplest thing I could think
of at the time. There are better targets now.

> - don't know how important writing notebooks as ReST will be (as 
> sketched in sketch.nbk), but there some docutils directive would be nice 
> for adding the code snippets (cells). .. python:  ..even?

http://bob.pythonmac.org/archives/2003/11/10/making-docutils-do-python-syntax-highlighting-for-html-output/

It still needs a LaTeX handler.

> as XML is not 
> for humans to write.. (and hardly even to read). and of course the 
> ipython log would be also nicer to read in a non-XML format, e.g. like 
> it originally shows in the ipython prompt or something (this is a bit 
> strangely inverse, 'cause rst was originally planned to be written in 
> .py files, but here it's vice versa..?)

Well, outputing just a log to any number of formats has shown itself to
be pretty trivial.

> - the 'ipython monkeypatch' seems to work (except that unlike the 
> example shows a name attribute for the Notebook is required - would it 
> be better to add some default name for notebooks, or keep requiring that 
> and fix the docstring?). am happy that didn't have to write that :o

Tell me about it. Actually, that bit needs to be refactored out to an
intermediary object with some other stuff to leave the notebook object
pure. Its docstring is the least of the things wrong with it.

> Hm, i feel like am missing all the important issues that do lurk 
> somewhere in the back of the mind, but those are at least some concrete 
> points -- i guess when we get actually going with experimenting with a 
> prorotype etc. they surface.

I've been fiddling around trying to settle on an XML markup to use. We
could try a subset of XHTML, but that is a little less semantic and more
presentational than I would prefer. For example, it uses

  <h1>Title</h1>
  <p>Something</p>
  <h2>Lower title</h2>
  <p>Something else in a sub-section</p>

where Docutils/DocBook uses

   <section>
     <title>Title</title>
     <para>Something</para>
     <section>
       <title>Lower title</title>
       <para>Something else in a sub-section</para>
     </section>
   </section>

On the plus side, a subset of XHTML is what twisted.lore uses, and
twisted.lore seems to be well-designed for plugins for custom tags. It
outputs both XHTML and LaTeX.

I haven't been so thrilled with prospect of using Docutils as a document
generation library. It seems to be very much focused on the input
reST-format. Ignoring that bit and generating documents programmatically
is apparently not so straightforward. Plugging in custom content is even 
less so.

By far the easiest approach so far has been to use DocBook's XML,
transforming the <ipython-block> into a <programlisting> (with syntax
highlighting tags inside), and getting HTML and LaTeX out via the XSL
stylesheets. Using lxml instead of ElementTree (same basic API but adds
full XPath and XSLT support), this all happens in Python. The output is
very configurable and looks quite nice.

Incidentally, Docutils' model seems to be mostly borrowed from DocBook,
so if we were to choose an XML subset from Docutils, we'd end up with a
subset of DocBook, too. I have a feeling that we'll settle on a
DocBook subset even if we handle the document generation ourselves or
massage it slightly to pass to some other library and don't
bother with the XSL stylesheets.

> So ideas about what to tackle next would be very welcome. Still this 
> morning I was not totally convinced if Robert's approach was the way to 
> go, but now it seems like that again. Thanks a lot for putting it together!

For the document side, one thing we're going to need no matter what is
an object(s) to hold style information. I made a stupid stab at it in my
SVN sandbox.

On the GUI side, we should try to get to the point of rendering 
something as simple as this:

<sheet>
   <para>I am a paragraph.</para>
   <ipython-block>
      ...
   </ipython-block>
   <para>I am another paragraph.</para>
   <ipython-block>
      ...
   </ipython-block>
   <para>Oh yes, it's me again.</para>
</sheet>

On the ipython side, we need clean ways to do all kinds of things. I'll 
get around to writing them down later.

-- 
Robert Kern
rkern at ucsd.edu

"In the fields of hell where the grass grows high
  Are the graves of dreams allowed to die."
   -- Richard Harter