[IPython-dev] Embedding the HTML notebook

Matthew Turk matthewturk at gmail.com
Tue Apr 10 08:41:20 EDT 2012

Hi Brian,

Thanks for your reply.  Over the last couple days I've spent some time
investigating, and I was able to make this work with minimal changes
-- in advance of going any further, seeing as this is on your radar, I
wanted to reply with what I've found to see if it fits in with your
vision, or if I should scrap it.

On Mon, Apr 9, 2012 at 6:21 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> Matthew,
> On Wed, Apr 4, 2012 at 3:46 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>> Hi there,
>> After something of a protracted discussion on Fernando's Google+
>> stream about embedding the IPython notebook, I started digging into
>> the notebook manager code.  As something of an introduction, I'm
>> relatively new to this aspect of IPython, but I work on a project (
>> yt-project.org ) that currently has a web GUI with a
>> (execution-blocking, much less fancy, somewhat clunky) web notebook.
>> On top of this we've built a number of display widgets, which really
>> are what we spend the most time on and what we're most proud of, but
>> more importantly they're what we want to focus our development energy
>> on moving forward.
>> Anyway, this is a long way around of saying we're looking really hard
>> at ditching our current cell-REPL system and instead building in the
>> IPython notebook, since it's awesome.  I'm trying to come up to speed
>> with how the web notebook works, and I have a couple questions about
>> the broad feasibility.
>> To embed as is, it looks like (as shown with NINJA-ide) one can embed
>> an iframe to link to a running notebook server.  But rather than doing
>> something precisely like that, since we're looking at providing
>> additional items such as widgets (although Fernando mentioned that
>> interoperation with callbacks is still underway) is it possible to
>> extend the templates from the API?
> Right now the answer is no.  We are currently using tornado as the
> templating library and it doesn't support loading templates from
> multiple directories.  However, we have plans to switch over the
> jinja2, which supports this fine.  As part of that change we will also
> be reorganizing the templates and javascript code to make it much
> easier for people to customize things and reuse things as part of
> their own web apps.  This is relatively high on my list of things
> todo, but still might take a while to get to.  Part of the problem is
> that this stuff is significant enough that it has to wait until after
> the 0.13.

The change to Jinja2 for the multiple template directories makes sense
(although the additional C dependency, with possible pure python
fallback, is a tradeoff), although after looking into how
RequestHandler is implemented it seems that subclassing handles this
very nicely.

>> Inside IPython/frontend/html/handlers.py, the render methods are code
>> with calls to specific template names, which then render out HTML
>> templates from a path that can be overridden with settings_override.
>> I *think* from reading the tornado docs that template_path has to be a
>> single string and not an iterable of strings,
> Yes, and this is exactly the weakness of tornado's template renderer
> and why we are moving away from it.
>>and the handlers
>> themselves are enumerated as classes in the NotebookWebApplication
>> handlers.  It was when I got this deep that I started to wonder if
>> perhaps there was an obvious, more straightforward way that I was
>> missing; if not, I'm definitely willing to do what I can to dive in
>> and contribute, if embedding is something of interest to the
>> community, too.
>> If we take as a given that an IPython engine is currently running,
>> what would the best route toward embedding a view -- that doesn't
>> fully duplicate the contents of static/ and templates/ -- into that
>> engine?
> Today there is not easy way of doing this.  You would basically have
> to duplicate everything.  After the refactoring, it should be much
> easier.

What I was able to implement, without the additional dependency of
Jinja2, was a single-notebook embedded instance with default and
sub-default handlers, allowing for the two basically independent web
apps to be served by the same tornado instance.  However, it required
a handful of changes to the IPython source which I've put into a
(readily-destroyable) fork on github under my username MatthewTurk.  I
attempted to comply with the coding practices in the file and in the
IPython developer guide.

It accomplished this with a couple main changes to the IPython
codebase.  The main weakness remains that the HTML fragment that
specifically corresponds to the notebook components is not
customizable, but it also allows the embedding user to access the
IPython components, so setting up their own full HTML fragment should
also work.  This is accomplished through:

1) Adding EmbeddedNotebookApp and EmbeddedNotebookWebApplication
classes.  These subclass the appropriate classes in the existing
implementation.  Both expose a subset of functionality (for instance,
no notebook manager), but most importantly the
EmbeddedNotebookWebApplication accepts an additional_handlers
argument.  These additional handlers will not be prepended with the
2) page.html has been modified to have {%%} blocks around every
component of the HTML.  This allows the initial and final HTML tags to
be stripped, along with body, script, etc.  embedded_notebook.html
subclasses notebook.html and overrides these tags, so that using
ExtJS's autoLoad functionality we can load it as a snippet (served by
Tornado) and insert it directly into an existing DOM.
3) To accommodate our existing widgets, I extended
IPython.Notebook.prototype with an external_handlers attribute.  Any
payloads that come through with the type "external_payload" will get
passed through to any callback that has been registered with the

Setting up an embedded application is then just a matter of creating a
set of handlers which are fed in as additional_handlers to an instance
of EmbeddedNotebookApp.  Passing through the new notebook ID was
somewhat tricky, but disallowing access to the notebook manager and
redirecting from the base URL /notebook (not
base_project_url/notebook) to the correct notebook worked around that.

While these changes worked for me to get something going to prototype
and experiment with layout inside our app, I understand that they may
not fit into the broader vision for the HTML notebook.  If they are of
some use I would be more than happy to work on them until they are up
to the quality necessary for contribution.  In my conversations with
Fernando the interest of the IPython development community in
collaborating with other sub-communities in scientific python has been
pretty clear, and I would like to do what I can to help with this

>>  Would prepending an additional path to the base_project_url
>> work, so that any calls to/from the engine would require that
>> base_project_url?  And then, post-instantiation, appending new
>> handlers to the NotebookWebApplication, for the "embedding" app's
>> templates, which perhaps have to be rendered from strings?
>> Any ideas would be greatly appreciated, and again, thanks for all your
>> hard work on IPython.
> Unfortunately, for now I think the best answer is to wait.  We will
> try our best to get to this ASAP though.

Ah, I guess I missed that there's a refactor coming.  I will dig
further through the IPython-dev archives to try to find out more
information about this before we modify any of our code to comply with
IPython's current API; most of our users upgrade dependencies
relatively slowly, so perhaps it is premature for us to start looking
at interoperation with or utilization of the new 0MQ engine system and
the HTML notebook.

Again, thanks for your reply, and all of your amazing hard work on IPython.


> Cheers,
> Brian
>> -Matt
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
> --
> Brian E. Granger
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu and ellisonbg at gmail.com
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

More information about the IPython-dev mailing list