[IPython-dev] interactive widgets with d3 and three.js

[Brian Granger]

On Fri, Oct 19, 2012 at 11:24 AM, Chris Kees <cekees at gmail.com> wrote:
> On Fri, Oct 19, 2012 at 12:06 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>> Chris,
>>
>>> I'm trying to build some interactive representations of geometric
>>> objects (domains of pde's) in the notebook. For now I'm using d3.js
>>> and three.js for the interactive representation and haven't written
>>> any callbacks to actually change the python representations so this
>>> equations is just about display for now. I'm starting with rectangles
>>> and rectangular cuboids. I can't seem to get the objects to display
>>> and interact properly without separating the html from the javascript
>>> like this:
>>>
>>> display(d.html) #add a canvas or div element
>>> d.javascript #get the canvas or div element and render to it with
>>> d3.js or three.js
>>>
>>> Shouldn't I be able to just stick the javascript in a <script> element
>>> and get the right behavior from d.html? Or should I actually be
>>> grabbing some element id in the javascript and just having the
>>> javascript modify it? It's not really that big of a deal--I'll
>>> probably add and edit() or show() function anyway. I'm more concerned
>>> about straightening out my understanding of how ipython is interacting
>>> with the generated html for a given notebook.
>>
>> Yes, if you want to do everything with JS, you will need to learn
>> about jquery and use that to modify the DOM.  When the JS code is run,
>> there are two jquery objects you have access to:  element and
>> container:
>>
>> element = the div that all output should go into.  You will usually do
>> element.append(new_stuff)
>> container = the div that is outside the element that starts out
>> hidden.  In your JS code you will want to call container.show()
>>
>
> Hey Brian,
>
> I have it sort of working with this:
>
> this.element.append("<div id='mydiv'></div>")
> var rectDemo = d3.select("#mydiv")
> .append("svg")
> .attr("width", 400)
> .attr("height", 400);
> rectDemo.append("rect")
> .style("stroke", "red")
> .attr("width",350)
> .attr("height", 350)
> $(".container").show();

Yes, this type of thing looks right.

> Seems like I should be able to do it with just d3 selections, but I
> don't seem to have mastered selections yet.  I can get by with a hack
> for now.
>
> To make sure I understand the path forward: In the future the process
> of creating and using a javascript widget will be to write Python code
> that packs up the object and some instructions as a json message in
> the python code which will be sent to the client (with
> display(my_json_msg) say).  Then there will be a case statement in the
> notebook javascript that matches some tag in my_json_msg with a set of
>  pre-defined javascript functions that  have to be configured server
> side. That way the only way to do damage is to exploit weaknesses in
> the catalog of javascript functions installed server side.
>
> I've spent a little time looking at the "Z Callbacks" notebook example
> and your branch/example. I'm wondering if there's a simple way for me
> to mimic this in my existing notebook that would allow me to put all
> the javascript code in the first markdown cell and do everything else
> with the standard display api. For example my domain class could
> implement
>
> def show():
>       display(my_display_only_json_msg)
> def edit():
>       display(my_editor_json_msg)

I don't think there is a way to get this working with what we have
now.  The problem is that there isn't any append_json method in master
that knows how to dispatch to the handlers.  That is something we need
to add first.

> @Matthias, thanks for the feedback on d3. I'm struggling with the docs
> a bit, but it seems to work ok, and there are some really nice
> examples at https://github.com/mbostock/d3/wiki/Gallery.
>
> Thanks,
> Chris
>
>> BUT, we should warn you that we are going to eliminate the ability to
>> run Python generated JS in the notebook.  There are two reasons for
>> this:
>>
>> * Security.  The current approach opens the door for some really nasty
>> (and trivial) attacks that we can't allow.
>> * Ease of development.  As you will find, writing Python code that
>> writes JS code that is run in the browser using eval is nearly
>> impossible to actually get anything done.  I wrote all of this code
>> and I still can't do it for anything other than something trivial.
>> eval makes debugging impossible and getting data from Python to JS in
>> this manner is horribly painful.
>>
>> Moving forward here is what we are going to do instead:
>>
>> * Rely on publishing JSON messages to get data back to the browser.
>> This will use the _repr_json_ method or publish_json function we
>> already have.
>> * Create JSON "handlers" in the notebook that know how to handle
>> different types of JSON messages.
>> * These handlers will be loaded when the main notebook page is loaded
>> and will be part of the notebook "server".
>> * We will ship basic handlers that "everyone" wants to use with
>> ipython proper - along with their JS dependencies (such as d3).
>> * We will allow users to install new handlers for plugins they want to
>> develop/use.
>>
>> We are a ways off from implementing all of this (I have just barely
>> starting to play with it) and we will need to discuss the details with
>> the rest of the dev team, but I wanted to let you know that we are
>> moving in this direction as it obviously affects your plans.
>>
>> Here is a branch that I have that starts to try this approach out:
>>
>> https://github.com/ellisonbg/ipython/tree/opt
>>
>> Here is the specific handler for some JSON I published:
>>
>> https://github.com/ellisonbg/ipython/blob/opt/IPython/frontend/html/notebook/static/js/outputarea.js#L455
>>
>> If you want to play, you could start to pick through this code, but
>> warning - it is a TOTAL mess and a half.  Probably best to keep
>> playing like you are, and later convert to the new approach.
>>
>> Cheers,
>>
>> Brian
>>
>>> Here is the notebook:
>>> https://github.com/erdc-cm/proteus-notebooks/blob/master/Domain%20Display.ipynb.
>>>  You'd have to either clone my repo + submodules or clone three.js
>>> into your notebook directory in order to run this.
>>>
>>> Thanks,
>>> Chris
>>>
>>> p.s. I'd be interested in any advice on whether d3.js and three.js are
>>> the way to go.  I would consider just working directly in svg and
>>> webgl. I've done a little of both, and right I'm not quite sure how
>>> much value the libraries add to the core html5 functionality.
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> --
>> Brian E. Granger
>> Cal Poly State University, San Luis Obispo
>> bgranger at calpoly.edu and ellisonbg at gmail.com
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com