[IPython-dev] ipython with sync

William Stein wstein at gmail.com
Tue Sep 10 12:11:50 EDT 2013


Somewhat by accident I spent the last two weeks implementing hosted
IPython notebooks with sync for https://cloud.sagemath.com.
Initially I had just plan to simplify the port forwarding setup, since
what Ondrej Certik was doing with multiple forward and reverse port
forwards seemed complicated.  But then I became concerned about
multiple users (or users with multiple browsers) overwriting each
other's notebooks, because cloud.sagemath projects are frequently
shared between multiple people, and everything else does realtime
sync.    I had planned just to add some very minimal merge-on-save
functionality to avoid major issues, but somehow got sucked into
realtime sync (even with the other person's cursor showing).

It would be enormously helpful to me if a couple of expert IPython
users were to try out what I implemented and just ask a bunch of

1. Go to https://cloud.sagemath.com and make an account; this is
completely free, and is hosted on computers at University of

2. Create a new project.

3. Click +New, then click "IPython" (or paste in a link to an ipython
notebook, or upload a file).

4. An IPython notebook server will start, the given .ipynb file should
load in a same-domain iframe, and then some of the ipython notebook
code is and iframe contents are monkey patched, in order to support
sync and better integration with https://cloud.sagemath.com.

5. Open the ipynb file in multiple browsers, and see that changes in
one appear in the other, including moving cells around, creating new
cells, editing markdown (the rendered version appears elsewhere), etc.
  Anything that sets the notebook.dirty flag in IPython causes a sync
(evaluating a cell that creates no output doesn't set this flag, at
least in 1.0.0, which is a bug in IPython, I guess).

Since this is all very new and the first (I guess) realtime sync
implementation on top of IPython, there are probably a lot of issues.
 Note that if you click the "i" info button to the right, you'll get a
link to the standard IPython

The other thing of interest is a little Python script called
"ipython-notebook", which I wrote.  It basically makes it easy to run
an IPython notebook server as a daemon, get the port it is running on,
etc.  It's pretty simple but satisfies my use case, and has
JSON-output, to make it web friendly.    As I've written it, my script
passes several base_url options through by default, which are needed
for cloud.sagemath.  Anyway, I've attached it to this email (with a
BSD license) in case there is any interest.

Regarding the monkey patching in 4 above, the right thing to do would
be to explain exactly what hooks/changes in the IPython html client I
need in order to do sync, etc., make sure these makes sense to the
IPython devs, and send a pull request (or have a coding sprint in
Seattle or Berkeley?).  As an example, in order to do sync
*efficiently*, I have to be able to set a given cell from JSON -- it's
critical to do this in place when possible, since the overhead of
creating a new cell is huge (due probably to the overhead of creating
CodeMirror editors); however, the fromJSON method in IPython assumes
that the cell is brand new -- it would be nice to add an option to
make a cell fromJSON without assuming it is empty.

The ultimate outcome of this could be a clean well-defined way of
doing sync for IPython notebooks using any third-party sync
implementation.  IPython might provide their own sync service and
there are starting to be others available these days -- e.g., Google
has one: https://developers.google.com/drive/realtime/, and maybe
Guido van Rosum helped write one for Dropbox recently?

Subdirectories:   I noticed, incidentally, that the wakari version of
the IPython notebook server allows one to load ipynb files that are in
any subdirectory, whereas the standard IPython notebook server
doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
server for each directory that a user accesses files in right now.
This seems cludgy, so I'm interested in the situation regarding adding
support for subdirectories.

-- William

William Stein
Professor of Mathematics
University of Washington
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipython-notebook
Type: application/octet-stream
Size: 6553 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130910/20225a2f/attachment.obj>

More information about the IPython-dev mailing list