[IPython-dev] using reST for representing the notebook cells+text

[Brian Granger]

Ondrej,

I have have thought about possible notebook formats many times.  IPython has
partially implemented at least two different notebook formats.  Both of them
have not been very successful.  The first was XML based and the second was
based on an sqlalchemy db.  Both were insanely complex.

Fernando and I have talked many times about a notebook format that is simply
restructured text.  There are many advantages of this approach:

* All of us use reST for lots of things any ways.
* There are existing parsers for reST.
* It naturally knows about code + text.
* Latex support, graphics.
* And more.

In many respects it seems almost perfect.  But, one day recently I was
thinking about how successful Mathematica's notebook continues to be.  So I
began to look more at the Mathematica notebook format.  Amazingly, the
Mathematica notebook format is a plain text file that itself is *valid
Mathematica code*.  This id documented here:

http://reference.wolfram.com/mathematica/guide/LowLevelNotebookProgramming.html

For examples a simple notebook with one text cell is just:

Notebook[{Cell['Here is my text', 'Text']}]

Everything - input cells, output cells, static images and all are
represented in this way and embedded in the plain text notebook file.  The
Python generalization of this would be the following:

* A Python notebook is plain text, importable Python code.
* That code is simply a tree of objects that declare the relevant parts of
the notebook.

This has a number of advantages:

* A notebook can be imported, manipulated and run by anyone who has the
support code (the notebook module that defines the relevant classes).
* A notebook doesn't need to be parsed.  It is valid Python and can be
imported or exec'd.  Once that is done, you have the full notebook in
memory.  You can immediately do anything you want with it.
* The various Notebook, Cell, Image, etc. classes can know about how to
output to various formats, latex, html, reST, XML, etc:

import mynotebook
mynotebook.notebook.export('rest')

* Each individual format (HTML, reST, latex) has weaknesses.  If you pick
any one to be *the* notebook format, you are building those weaknesses into
your design.  A pure python based notebook format won't suffer from that
syndrome.
* It is a clean separation of the model (Notebook, Cell, Image, etc.) and
the view (HTML, reST, etc.).  Picking HTML or reST for the notebook format
confuses (at some level) the model and view...
* Third party code can define new Notebook elements that specify how they
can be rendered in different contexts.  For example, matplotlib could ship a
Figure element that knows how to render itself as a native PyQt GUI, a
static image, a web page, etc.
* A notebook remains a single plain text file that anyone can edit - even if
it has embedded images.  Neither HTML nor reST have the ability to inline
graphics in plain text files.  While I love reST, it is a pain that I need
an entire directory of files to render a single Sphinx doc.

Anyways, that is where my brain is wandering these days.

Cheers,

Brian




On Tue, Feb 16, 2010 at 6:21 PM, Ondrej Certik <ondrej at certik.cz> wrote:

> Hi,
>
> has anyone given a deeper thought how to best implement this functionality?
>
> Currently the Sage notebook seems to remember the worksheet using:
>
> --------------
> sdsd
> system:sage
>
>
> <p>Well, some text</p>
>
> {{{id=5|
> awe
> ///
> }}}
>
> {{{id=1|
> 4
> ///
> 4
> }}}
>
> {{{id=2|
> 5
> ///
> 5
> }}}
>
> {{{id=3|
>
> ///
> }}}
> ---------------------
>
> E.g. each cell is either a text cell, or a cell that can be evaluated.
> Is that right?
>
> I know that Fernando suggested here:
>
>
> http://groups.google.com/group/sage-devel/browse_thread/thread/3f1f5a0c26975519/
>
> to use reST, e.g. something like:
>
> ----------
> Well, some text
>
> ::
>     >>> print 3
>     3
>
> ------------
>
>
> I am implementing a notebook prototype using pure Python
> (automatically translated to javascript using pyjamas) and I would
> like to run it on the google app engine (the javascript in your
> browser would then communicate either with the app engine, or your own
> hardware that would host Sage/FEMhub/your own code):
>
> http://gamma.sympy.org/nb/
>
> and here is my development version with logins and settings implemented:
>
> http://2.latest.sympy-gamma.appspot.com/nb/
>
> now I need to implement the worksheets, so I am thinking how to best
> represent it.
>
>
> Did the syntax for input/output blocks in Sphinx settled already? I
> guess internally I will implement the worksheet as a set of cells,
> either text cells, or "evaluatable" cells (that contain both
> input/output blocks) and then support conversion to and from
> reST/Sphinx.
>
> I would be interested in any feedback.
>
> Ondrej
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100223/c89fe6da/attachment.html>