[IPython-dev] ipython with sync

Fernando Perez fperez.net at gmail.com
Tue Sep 10 15:54:04 EDT 2013

Hi William,

On Tue, Sep 10, 2013 at 9:11 AM, William Stein <wstein at gmail.com> wrote:

> 5. Open the ipynb file in multiple browsers, and see that changes in
> one appear in the other, including moving cells around, creating new
> cells, editing markdown (the rendered version appears elsewhere), etc.
>   Anything that sets the notebook.dirty flag in IPython causes a sync
> (evaluating a cell that creates no output doesn't set this flag, at
> least in 1.0.0, which is a bug in IPython, I guess).

Wow, very impressive!  At the very first cell I saw some glitches: I
simply typed '1' in the first empty cell and ran it; it produced '1'
as output but the input got deleted and copied to the next (new) cell.
Weird.  It didn't happen again.

Overall it seems to work, though for now I've only tested it very
lightly.  The startup problem I saw yesterday seems gone.

> Since this is all very new and the first (I guess) realtime sync
> implementation on top of IPython, there are probably a lot of issues.
>  Note that if you click the "i" info button to the right, you'll get a
> link to the standard IPython

My first question is: to what extent does this rely on your
surrounding node.js machinery and to what extent is it something
self-contained that could be used by the current IPython design?  This
is obviously something of high interest to us (and oft-requested by
users), so it would be good to understand what parts of the design
would make sense to upstream and what parts are deeply intertwined
with the entire sagemath architecture.

> The other thing of interest is a little Python script called
> "ipython-notebook", which I wrote.  It basically makes it easy to run
> an IPython notebook server as a daemon, get the port it is running on,
> etc.  It's pretty simple but satisfies my use case, and has
> JSON-output, to make it web friendly.    As I've written it, my script
> passes several base_url options through by default, which are needed
> for cloud.sagemath.  Anyway, I've attached it to this email (with a
> BSD license) in case there is any interest.

On that front, there are two related projects worth mentioning:

https://github.com/UnataInc/ipydra: along the lines of yours but more elaborate.

https://github.com/ptone/jiffylab: docker-based full containerization
of a notebook+shell.

> Regarding the monkey patching in 4 above, the right thing to do would
> be to explain exactly what hooks/changes in the IPython html client I
> need in order to do sync, etc., make sure these makes sense to the
> IPython devs, and send a pull request (or have a coding sprint in
> Seattle or Berkeley?).  As an example, in order to do sync
> *efficiently*, I have to be able to set a given cell from JSON -- it's
> critical to do this in place when possible, since the overhead of
> creating a new cell is huge (due probably to the overhead of creating
> CodeMirror editors); however, the fromJSON method in IPython assumes
> that the cell is brand new -- it would be nice to add an option to
> make a cell fromJSON without assuming it is empty.

Yup, this seems like a sensible idea and I can imagine us using it in
many other places, actually.

> The ultimate outcome of this could be a clean well-defined way of
> doing sync for IPython notebooks using any third-party sync
> implementation.  IPython might provide their own sync service and
> there are starting to be others available these days -- e.g., Google
> has one: https://developers.google.com/drive/realtime/, and maybe
> Guido van Rosum helped write one for Dropbox recently?

Thanks for that Google link, I hadn't seen this one.  I remember
they'd opened up an older implementation of their Wave stuff, but when
I looked at it, it was a huge Java monster so it didn't look like we
could reuse any of it.  It would be interesting to consider relying on
an external service for syncing as an option, but honestly it seems to
me that in the end we should put that into the IPython server itself.
If nothing else, for latency reasons in locally hosted servers I can
imagine it making a big difference.

> Subdirectories:   I noticed, incidentally, that the wakari version of
> the IPython notebook server allows one to load ipynb files that are in
> any subdirectory, whereas the standard IPython notebook server
> doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
> server for each directory that a user accesses files in right now.
> This seems cludgy, so I'm interested in the situation regarding adding
> support for subdirectories.

See Thomas' response.  We hope to have this merged before too long,
and it will be a huge improvement for everyone.



More information about the IPython-dev mailing list