[IPython-dev] Architecting interactive matplotlib figures in the IPython notebook

[Fernando Perez]

Phil, just to add to what Min already said: I think for IPython, the
right solution will be for mpl to expose the necessary calls that
handle the events in the backend and create the necessary image
responses as a standalone, pure-python api. You guys can obviously
ship your own custom web server that uses those to provide a
'matplotlib service' over http, but in the IPython notebook, what
would happen is that we would wrap the return of a matplotlib figure
in a JS container that calls those events in the backend, using our
yet-to-be-written API.

I am sure that one could devise a gross hack today that spins up the
mpl webagg server next to the IPython kernel (or in a thread, or some
other similarly nasty concoction) and feeds things over to the
notebook as it exists now.

But rather than dumping effort into that line of thought, I'd rather
ask for patience on our work this fall that will provide an official
API for this kind of situation.

What we *can* do is, at SciPy'13, spend as much time as possible
between anyone from MPL and IPython who will be in attendance,
thinking about this.  It will likely be an excellent user story to
guide our design in the fall, and a test case for our implementation.

How does that sound? Will you make it to SciPy'13?

Best

f

On Thu, Apr 18, 2013 at 10:03 AM, MinRK <benjaminrk at gmail.com> wrote:
> Interactive rich display will be our main focus for the Summer / Fall after
> shipping 1.0 in July.  Since the plotting lives in the kernel (which is not
> a webserver), and there shouldn't be any plotting logic in the notebook
> server itself, my guess is that the http server approach is not going to be
> workable inside IPython.  IPython's own mechanism for getting messages from
> the frontend to the kernel is with our message spec, and I expect that will
> be better suited to communicating between the frontend and the kernel within
> the context of the notebook.  However, this may be less useful as a
> general-purpose library within matplotlib, as it is quite IPython specific.
> Assuming the code is appropriately decoupled, there should not be any great
> difference between using websocket/JSON messages for the rpc calls, as
> opposed to http requests.
>
> We haven't yet worked out exactly what the APIs will look like (on both
> Javascript and Python sides) for exposing methods for this kind of rich
> widget, but this should be our primary task starting in late July.
>
> -MinRK
>
>
> On Thu, Apr 18, 2013 at 9:41 AM, Phil Elson <pelson.pub at gmail.com> wrote:
>>
>> You may be aware that as of matplotlib v1.3 there will be a "WebAgg"
>> backend which allows figures created on the server-side to be interacted
>> with on a client-side web browser.
>>
>> Specifically calling plt.show() with the "WebAgg" backend starts a tornado
>> http server which serves up the open figures in PNG form. When the client
>> connects to the server, the current state of the figure is sent to the
>> browser, and any client mouse or keyboard interactions are captured and sent
>> back to the server via web sockets. Anything which result in a change to the
>> draw state are re-rendered, and the resulting PNG is sent back to the
>> client. This process works well for everything that we've tested, including
>> Pickers and Animations (I'd definitely encourage you to have a go).
>>
>> So the obvious question is, can we use this technology to achieve
>> interactive figures in live IPython notebook environments?
>>
>> Unfortunately I don't have enough knowledge of IPython's internals to make
>> the informed decisions necessary to get out of the blocks, so I'm hoping the
>> mailing list can help me out.
>> The first step towards this goal involves deciding where a (or multiple?)
>> WebAgg server(s) would sit. A couple of my thoughts on this (some of which
>> are mutually exclusive):
>>
>> It'd be great to make use of the same twisted server as the Notebook
>> client, so that we can piggy-back on the IPython notebook configuration
>> (port, certificate etc.)
>>
>> This would mean that this is a Notebook only feature (without this, there
>> is no reason that other clients couldn't have implemented the necessary
>> interface to achieve inline interactive figures (e.g., the Qt console))
>> This would also give the WebAgg the necessary blocking IOLoop to handle
>> events and timers.
>>
>> As of matplotlib v1.2 there has been experimental pickle support, so:
>>
>> we are not bound to use the kernel that generated a figure in the first
>> instance, as we can pass figures (albeit slowly) across processes
>> we could store the figure in the notebook format to be restored when a
>> notebook is reconnected to a kernel (does this happen for other Display
>> elements?)
>>
>> As it stands, the WebAgg backend does not copy figure instances per
>> client, so any interactions that one user does, is shown in every other
>> client's browser. Is this a desirable feature for the Notebook? What about
>> figures which are shown more than once in the same notebook?
>>
>> I'm keen to work through these design decisions and am more than willing
>> to get my hands dirty - there is a matplotlib release scheduled for the end
>> of May, and myself and some of the other members of the matplotlib
>> development team are planning to attend SciPy with interactive figures for
>> the Notebook high on the wishlist.
>>
>> Any thoughts on where you think the Agg server should sit and its expected
>> behaviour would be really valuable.
>>
>> Thanks in advance,
>>
>> Phil
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>