[IPython-dev] notebook composition UI thoughts

[Michael Tobis]

John Hunter has mentioned the IPython notebook effort to me a couple
of times. I've been thinking about the Mathematica user interface for
years, and I think it is flawed. I hope that eventually IPython can do
better, and I think the steps being plotted out for the short term
move in the right direction.

A Mathematica notebook serves two primary purposes, which are at odds
with each other. The first is as an interactive environment for
exploration of mathematical and algorithmic ideas, and the second is
as an environment for the development and presentation of formal
exposition of the ideas so developed.

Consider this Mathematica session, obtained by simply moving the
insertion point between the second and third inputs:

==> 
In[1]:= x = 5

Out[1]= 5

--

In[3] = x * x

Out[3]= 36

--

In[2]:= x = 6

Out[2]= 6 
<==

I save it and get the notebook

==> 
x = 5

5

--

x * x

36

--

x = 6

6 
<==

Of course, when I recalculate the cells, the order of execution
changes and the second result is the expected 25. There are far more
confusing results that are similar in concept but much more confusing
in practice. There is also the fact that the notebook inherits 
context from the entire Mathematica session, including all open 
notebooks, that leads to similar flavors of confusion.

One way to look at the problem with the Mathematica model of notebooks
is that it conflates the purposes of authoring and presentation. As a
presentation mechanism, the organizing principle is sequential in the
file space, but as an interactive execution model, the organizing
principle is sequential in time.

The quandary is that explorations are developed sequentially in time
and read and executed sequentially in text.

In practice what I will want if I am developing an interactive
educational, um, thing (I'm not sure "notebook" is the right name) is
not one but two Python sessions: first an interactive environment not
dramatically different from our current iPython, for developing the
concepts, and second a live editable interface into which I can paste
those bits of text, code and other widgets as I determine that they
enhance the presentation.

The order in which I edit a document is not always sequential.
I need to be able to insert executable statements into the middle of
the document. For repeatability, I also want each document to be able
to provide all of its own context.

For this to make sense I need a new python session (or at least a
fresh namespace)  invoked every time I make a change in the
middle of a document. I need anything after the insertion point that
changes to be highlighted and flagged, and I need multiple undos.

In the case of a very long document this will cause quite a lot of
thrashing around as everything gets recalculated. Perhaps the user
interface needs to permit "molecular" rather than atomic changes
before enforcing a recalculation. That is, I would insert multiple
input lines from my live session into various places in the notebook
document before forcing a recalculation of the document.

It seems clear to me that some sort of computer mediated presentation
mechanism will eventually take over from textbooks in mathematics,
engineering and physical science. I think Python deserves to be a big
player in this. 

Mathematica notebooks show us that a great deal of care needs to be
put into the authoring interface.

The postprocessing model described as the summer project seems to me
to support this idea rather than thwart it, so there is fortunately no
great urgency in considering these ideas.

Basically, my proposed notebook development UI is not one Python
session but two, one a conventional iPython session, and the other a
rather more ethereal beast that ensures that everything is executed in
the order I as author intend and that all context is provided within
the notebook and not accidentally in the session.

The overlap between these problems and those addressed by Knuth in his
original Literate Programming work is, to me, striking. I'm agnostic
on the question as to whether the extant Leo code base is the right
approach, but the ideas of literate programming are very germane to
the ideas of developing scientific "notebooks". 

Knuth observes that the order in which the reader wants to read the
code does not necessarily match the order in which the compiler wants
to process it. The flaws in the Mathematica noteboook model show us
(among other things) that the order in which an author writes code and
documentation is not necessarily the same as either of them.

--
mt
--
Michael Tobis
Geophysical Sciences
University of Chicago