[IPython-dev] ipython with sync

Thu Nov 14 02:47:47 EST 2013

On Fri, Sep 13, 2013 at 10:44 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> Thanks for this link, that is helpful.

I have now also BSD licensed the CoffeeScript library I wrote that
implements Neil Fraser's library.  It's here

   https://github.com/sagemath/cloud/blob/master/diffsync.coffee

I'm pretty busy right now, or I would make that a nice standalone
project, etc.   In any case, if you want to implement sync, the above
code is what I'm actually using in the client/server/etc.

I hope it's useful,

   William

> Cheers,
>
> Brian
>
> On Thu, Sep 12, 2013 at 4:04 PM, William Stein <wstein at gmail.com> wrote:
>> On Thu, Sep 12, 2013 at 2:53 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>> William,
>>>
>>> Thanks for notes.  This is really exciting and we are very interested
>>> in getting live sync to work with IPython itself.  I am traveling this
>>> week, but will try to have a look soon.  Is the code available
>>> somewhere to look at?
>>
>> Unfortunately, this implementation is very entangled as part of
>> something I'm building as part of a startup company with UW's Center
>> for commercialization, and I can't just open source everything.  But
>> the algorithm is here:
>>
>>   https://neil.fraser.name/writing/sync/
>>
>> and the author of that paper has an open source Python implementation,
>> I think...
>>
>>
>>>
>>> Cheers,
>>>
>>> Brian
>>>
>>> On Tue, Sep 10, 2013 at 9:11 AM, William Stein <wstein at gmail.com> wrote:
>>>> Hi,
>>>>
>>>> Somewhat by accident I spent the last two weeks implementing hosted
>>>> IPython notebooks with sync for https://cloud.sagemath.com.
>>>> Initially I had just plan to simplify the port forwarding setup, since
>>>> what Ondrej Certik was doing with multiple forward and reverse port
>>>> forwards seemed complicated.  But then I became concerned about
>>>> multiple users (or users with multiple browsers) overwriting each
>>>> other's notebooks, because cloud.sagemath projects are frequently
>>>> shared between multiple people, and everything else does realtime
>>>> sync.    I had planned just to add some very minimal merge-on-save
>>>> functionality to avoid major issues, but somehow got sucked into
>>>> realtime sync (even with the other person's cursor showing).
>>>>
>>>> It would be enormously helpful to me if a couple of expert IPython
>>>> users were to try out what I implemented and just ask a bunch of
>>>> questions.
>>>>
>>>> 1. Go to https://cloud.sagemath.com and make an account; this is
>>>> completely free, and is hosted on computers at University of
>>>> Washington.
>>>>
>>>> 2. Create a new project.
>>>>
>>>> 3. Click +New, then click "IPython" (or paste in a link to an ipython
>>>> notebook, or upload a file).
>>>>
>>>> 4. An IPython notebook server will start, the given .ipynb file should
>>>> load in a same-domain iframe, and then some of the ipython notebook
>>>> code is and iframe contents are monkey patched, in order to support
>>>> sync and better integration with https://cloud.sagemath.com.
>>>>
>>>> 5. Open the ipynb file in multiple browsers, and see that changes in
>>>> one appear in the other, including moving cells around, creating new
>>>> cells, editing markdown (the rendered version appears elsewhere), etc.
>>>>   Anything that sets the notebook.dirty flag in IPython causes a sync
>>>> (evaluating a cell that creates no output doesn't set this flag, at
>>>> least in 1.0.0, which is a bug in IPython, I guess).
>>>>
>>>> Since this is all very new and the first (I guess) realtime sync
>>>> implementation on top of IPython, there are probably a lot of issues.
>>>>  Note that if you click the "i" info button to the right, you'll get a
>>>> link to the standard IPython
>>>>
>>>> The other thing of interest is a little Python script called
>>>> "ipython-notebook", which I wrote.  It basically makes it easy to run
>>>> an IPython notebook server as a daemon, get the port it is running on,
>>>> etc.  It's pretty simple but satisfies my use case, and has
>>>> JSON-output, to make it web friendly.    As I've written it, my script
>>>> passes several base_url options through by default, which are needed
>>>> for cloud.sagemath.  Anyway, I've attached it to this email (with a
>>>> BSD license) in case there is any interest.
>>>>
>>>> Regarding the monkey patching in 4 above, the right thing to do would
>>>> be to explain exactly what hooks/changes in the IPython html client I
>>>> need in order to do sync, etc., make sure these makes sense to the
>>>> IPython devs, and send a pull request (or have a coding sprint in
>>>> Seattle or Berkeley?).  As an example, in order to do sync
>>>> *efficiently*, I have to be able to set a given cell from JSON -- it's
>>>> critical to do this in place when possible, since the overhead of
>>>> creating a new cell is huge (due probably to the overhead of creating
>>>> CodeMirror editors); however, the fromJSON method in IPython assumes
>>>> that the cell is brand new -- it would be nice to add an option to
>>>> make a cell fromJSON without assuming it is empty.
>>>>
>>>> The ultimate outcome of this could be a clean well-defined way of
>>>> doing sync for IPython notebooks using any third-party sync
>>>> implementation.  IPython might provide their own sync service and
>>>> there are starting to be others available these days -- e.g., Google
>>>> has one: https://developers.google.com/drive/realtime/, and maybe
>>>> Guido van Rosum helped write one for Dropbox recently?
>>>>
>>>> Subdirectories:   I noticed, incidentally, that the wakari version of
>>>> the IPython notebook server allows one to load ipynb files that are in
>>>> any subdirectory, whereas the standard IPython notebook server
>>>> doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
>>>> server for each directory that a user accesses files in right now.
>>>> This seems cludgy, so I'm interested in the situation regarding adding
>>>> support for subdirectories.
>>>>
>>>> -- William
>>>>
>>>>
>>>>
>>>> --
>>>> William Stein
>>>> Professor of Mathematics
>>>> University of Washington
>>>> http://wstein.org
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>
>>>
>>>
>>> --
>>> Brian E. Granger
>>> Cal Poly State University, San Luis Obispo
>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> William Stein
>> Professor of Mathematics
>> University of Washington
>> http://wstein.org
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org