[IPython-dev] ipython with sync

Brian Granger ellisonbg at gmail.com
Thu Nov 14 15:27:26 EST 2013


Thanks William, will have a look at it!

Cheers,

Brian

On Thu, Nov 14, 2013 at 2:47 AM, William Stein <wstein at gmail.com> wrote:
> On Fri, Sep 13, 2013 at 10:44 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>> Thanks for this link, that is helpful.
>
> I have now also BSD licensed the CoffeeScript library I wrote that
> implements Neil Fraser's library.  It's here
>
>    https://github.com/sagemath/cloud/blob/master/diffsync.coffee
>
> I'm pretty busy right now, or I would make that a nice standalone
> project, etc.   In any case, if you want to implement sync, the above
> code is what I'm actually using in the client/server/etc.
>
> I hope it's useful,
>
>    William
>
>
>> Cheers,
>>
>> Brian
>>
>> On Thu, Sep 12, 2013 at 4:04 PM, William Stein <wstein at gmail.com> wrote:
>>> On Thu, Sep 12, 2013 at 2:53 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>>> William,
>>>>
>>>> Thanks for notes.  This is really exciting and we are very interested
>>>> in getting live sync to work with IPython itself.  I am traveling this
>>>> week, but will try to have a look soon.  Is the code available
>>>> somewhere to look at?
>>>
>>> Unfortunately, this implementation is very entangled as part of
>>> something I'm building as part of a startup company with UW's Center
>>> for commercialization, and I can't just open source everything.  But
>>> the algorithm is here:
>>>
>>>   https://neil.fraser.name/writing/sync/
>>>
>>> and the author of that paper has an open source Python implementation,
>>> I think...
>>>
>>>
>>>>
>>>> Cheers,
>>>>
>>>> Brian
>>>>
>>>> On Tue, Sep 10, 2013 at 9:11 AM, William Stein <wstein at gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> Somewhat by accident I spent the last two weeks implementing hosted
>>>>> IPython notebooks with sync for https://cloud.sagemath.com.
>>>>> Initially I had just plan to simplify the port forwarding setup, since
>>>>> what Ondrej Certik was doing with multiple forward and reverse port
>>>>> forwards seemed complicated.  But then I became concerned about
>>>>> multiple users (or users with multiple browsers) overwriting each
>>>>> other's notebooks, because cloud.sagemath projects are frequently
>>>>> shared between multiple people, and everything else does realtime
>>>>> sync.    I had planned just to add some very minimal merge-on-save
>>>>> functionality to avoid major issues, but somehow got sucked into
>>>>> realtime sync (even with the other person's cursor showing).
>>>>>
>>>>> It would be enormously helpful to me if a couple of expert IPython
>>>>> users were to try out what I implemented and just ask a bunch of
>>>>> questions.
>>>>>
>>>>> 1. Go to https://cloud.sagemath.com and make an account; this is
>>>>> completely free, and is hosted on computers at University of
>>>>> Washington.
>>>>>
>>>>> 2. Create a new project.
>>>>>
>>>>> 3. Click +New, then click "IPython" (or paste in a link to an ipython
>>>>> notebook, or upload a file).
>>>>>
>>>>> 4. An IPython notebook server will start, the given .ipynb file should
>>>>> load in a same-domain iframe, and then some of the ipython notebook
>>>>> code is and iframe contents are monkey patched, in order to support
>>>>> sync and better integration with https://cloud.sagemath.com.
>>>>>
>>>>> 5. Open the ipynb file in multiple browsers, and see that changes in
>>>>> one appear in the other, including moving cells around, creating new
>>>>> cells, editing markdown (the rendered version appears elsewhere), etc.
>>>>>   Anything that sets the notebook.dirty flag in IPython causes a sync
>>>>> (evaluating a cell that creates no output doesn't set this flag, at
>>>>> least in 1.0.0, which is a bug in IPython, I guess).
>>>>>
>>>>> Since this is all very new and the first (I guess) realtime sync
>>>>> implementation on top of IPython, there are probably a lot of issues.
>>>>>  Note that if you click the "i" info button to the right, you'll get a
>>>>> link to the standard IPython
>>>>>
>>>>> The other thing of interest is a little Python script called
>>>>> "ipython-notebook", which I wrote.  It basically makes it easy to run
>>>>> an IPython notebook server as a daemon, get the port it is running on,
>>>>> etc.  It's pretty simple but satisfies my use case, and has
>>>>> JSON-output, to make it web friendly.    As I've written it, my script
>>>>> passes several base_url options through by default, which are needed
>>>>> for cloud.sagemath.  Anyway, I've attached it to this email (with a
>>>>> BSD license) in case there is any interest.
>>>>>
>>>>> Regarding the monkey patching in 4 above, the right thing to do would
>>>>> be to explain exactly what hooks/changes in the IPython html client I
>>>>> need in order to do sync, etc., make sure these makes sense to the
>>>>> IPython devs, and send a pull request (or have a coding sprint in
>>>>> Seattle or Berkeley?).  As an example, in order to do sync
>>>>> *efficiently*, I have to be able to set a given cell from JSON -- it's
>>>>> critical to do this in place when possible, since the overhead of
>>>>> creating a new cell is huge (due probably to the overhead of creating
>>>>> CodeMirror editors); however, the fromJSON method in IPython assumes
>>>>> that the cell is brand new -- it would be nice to add an option to
>>>>> make a cell fromJSON without assuming it is empty.
>>>>>
>>>>> The ultimate outcome of this could be a clean well-defined way of
>>>>> doing sync for IPython notebooks using any third-party sync
>>>>> implementation.  IPython might provide their own sync service and
>>>>> there are starting to be others available these days -- e.g., Google
>>>>> has one: https://developers.google.com/drive/realtime/, and maybe
>>>>> Guido van Rosum helped write one for Dropbox recently?
>>>>>
>>>>> Subdirectories:   I noticed, incidentally, that the wakari version of
>>>>> the IPython notebook server allows one to load ipynb files that are in
>>>>> any subdirectory, whereas the standard IPython notebook server
>>>>> doesn't.  For cloud.sagemath, I just spawn a new IPython notebook
>>>>> server for each directory that a user accesses files in right now.
>>>>> This seems cludgy, so I'm interested in the situation regarding adding
>>>>> support for subdirectories.
>>>>>
>>>>> -- William
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> William Stein
>>>>> Professor of Mathematics
>>>>> University of Washington
>>>>> http://wstein.org
>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Brian E. Granger
>>>> Cal Poly State University, San Luis Obispo
>>>> bgranger at calpoly.edu and ellisonbg at gmail.com
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>>
>>> --
>>> William Stein
>>> Professor of Mathematics
>>> University of Washington
>>> http://wstein.org
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
> --
> William Stein
> Professor of Mathematics
> University of Washington
> http://wstein.org
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com



More information about the IPython-dev mailing list