[IPython-dev] ipython with sync

[William Stein]

On Thu, Sep 12, 2013 at 2:53 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> William,
>
> Thanks for notes.  This is really exciting and we are very interested
> in getting live sync to work with IPython itself.  I am traveling this
> week, but will try to have a look soon.  Is the code available
> somewhere to look at?

Unfortunately, this implementation is very entangled as part of
something I'm building as part of a startup company with UW's Center
for commercialization, and I can't just open source everything.  But
the algorithm is here:

  https://neil.fraser.name/writing/sync/

and the author of that paper has an open source Python implementation,
I think...


>
> Cheers,
>
> Brian
>
> On Tue, Sep 10, 2013 at 9:11 AM, William Stein <wstein at gmail.com> wrote:
>> Hi,
>>
>> Somewhat by accident I spent the last two weeks implementing hosted
>> IPython notebooks with sync for https://cloud.sagemath.com.
>> Initially I had just plan to simplify the port forwarding setup, since
>> what Ondrej Certik was doing with multiple forward and reverse port
>> forwards seemed complicated.  But then I became concerned about
>> multiple users (or users with multiple browsers) overwriting each
>> other's notebooks, because cloud.sagemath projects are frequently
>> shared between multiple people, and everything else does realtime
>> sync.    I had planned just to add some very minimal merge-on-save
>> functionality to avoid major issues, but somehow got sucked into
>> realtime sync (even with the other person's cursor showing).
>>
>> It would be enormously helpful to me if a couple of expert IPython
>> users were to try out what I implemented and just ask a bunch of
>> questions.
>>
>> 1. Go to https://cloud.sagemath.com and make an account; this is
>> completely free, and is hosted on computers at University of
>> Washington.
>>
>> 2. Create a new project.
>>
>> 3. Click +New, then click "IPython" (or paste in a link to an ipython
>> notebook, or upload a file).
>>
>> 4. An IPython notebook server will start, the given .ipynb file should
>> load in a same-domain iframe, and then some of the ipython notebook
>> code is and iframe contents are monkey patched, in order to support
>> sync and better integration with https://cloud.sagemath.com.
>>
>> 5. Open the ipynb file in multiple browsers, and see that changes in
>> one appear in the other, including moving cells around, creating new
>> cells, editing markdown (the rendered version appears elsewhere), etc.
>>   Anything that sets the notebook.dirty flag in IPython causes a sync
>> (evaluating a cell that creates no output doesn't set this flag, at
>> least in 1.0.0, which is a bug in IPython, I guess).
>>
>> Since this is all very new and the first (I guess) realtime sync
>> implementation on top of IPython, there are probably a lot of issues.
>>  Note that if you click the "i" info button to the right, you'll get a
>> link to the standard IPython
>>
>> The other thing of interest is a little Python script called
>> "ipython-notebook", which I wrote.  It basically makes it easy to run
>> an IPython notebook server as a daemon, get the port it is running on,
>> etc.  It's pretty simple but satisfies my use case, and has
>> JSON-output, to make it web friendly.    As I've written it, my script
>> passes several base_url options through by default, which are needed
>> for cloud.sagemath.  Anyway, I've attached it to this email (with a
>> BSD license) in case there is any interest.
>>
>> Regarding the monkey patching in 4 above, the right thing to do would
>> be to explain exactly what hooks/changes in the IPython html client I
>> need in order to do sync, etc., make sure these makes sense to the
>> IPython devs, and send a pull request (or have a coding sprint in
>> Seattle or Berkeley?).  As an example, in order to do sync
>> *efficiently*, I have to be able to set a given cell from JSON -- it's
>> critical to do this in place when possible, since the overhead of
>> creating a new cell is huge (due probably to the overhead of creating
>> CodeMirror editors); however, the fromJSON method in IPython assumes
>> that the cell is brand new -- it would be nice to add an option to
>> make a cell fromJSON without assuming it is empty.
>>
>> The ultimate outcome of this could be a clean well-defined way of
>> doing sync for IPython notebooks using any third-party sync
>> implementation.  IPython might provide their own sync service and
>> there are starting to be others available these days -- e.g., Google
>> has one: https://developers.google.com/drive/realtime/, and maybe
>> Guido van Rosum helped write one for Dropbox recently?
>>
>> Subdirectories:   I noticed, incidentally, that the wakari version of
>> the IPython notebook server allows one to load ipynb files that are in
>> any subdirectory, whereas the standard IPython notebook server
>> doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
>> server for each directory that a user accesses files in right now.
>> This seems cludgy, so I'm interested in the situation regarding adding
>> support for subdirectories.
>>
>> -- William
>>
>>
>>
>> --
>> William Stein
>> Professor of Mathematics
>> University of Washington
>> http://wstein.org
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org