[IPython-dev] Implementing inline images in a kernel

Brian Granger ellisonbg at gmail.com
Tue Jan 28 20:59:28 EST 2014


> In implementing a new kernel, I can appropriately turn returned values into
> their rich representation messages. But I'm wrestling with the idea that the
> kernel could somehow, in the middle of computing, send such rich
> representations to be displayed.

The best starting place for this is our message spec:

http://ipython.org/ipython-doc/dev/development/messaging.html

The basic idea is that you just need to send these messages.  But it
sounds like you are already doing this part?

> I've seen bits of code that appear to examine strings to see if they begin
> with certain signatures that could detect PNG or JPEG encodings. Is that how
> it would work... you just send a 64-bit encoded string to the front end, and
> it automatically detects it as a rich representation? Or is there a general
> standard for marking strings automatically as rich representations? Or must
> the kernel construct a message to get any rich reps? I could make the kernel
> monitor for special strings (sent from lower levels), intercept them, and
> then send the special rich message.

I don't think you want to try to guess the rich representation based
on pure string values. In the IPython kernel, we have very detailed
Python APIs for plugging into the messaging layer.  Thomas has
described some of that. I would have a look at the various parts of
IPython.core that have this if you want to see how we do it in
IPython:

https://github.com/ipython/ipython/tree/master/IPython/core

displayhook.py
display.py
displaypub.py
formatters.py

There is actually a lot of complexity there.


> If this question is confusing, perhaps it is because my kernel interfaces
> with many subsystems that can generate strings to the output, but hooking up
> the lower-level to the rich message system would be hard. I suspect that
> this is similar to making an external language, like R, generate images
> inline. How does it do it?

You will have to be able to call the subsystems and be able to track
the various rich representations that the subsystems can track.

A side point though:

Having one kernel that can run different languages is similar to the
main IPython kernel and our %% magic syntax. This works OK, but is
very limited because of a number of aspects:

* You have to abstract things like tab completion for all of your sub
systems through the main kernel.
* You have to figure out how to get rich reprs from the sub systems to
the main kernel which will send the messages.
* It will almost certainly be difficult to impossible to use our new
widget architecture in the subsystems.
* Things like syntax highlighting in the browser will be broken.

Because of this, our feeling is that for any real, serious notebook
work in another language, it should be done as a completely
independent native kernel.  What do I mean by this? The best example
is the Julia kernel, which is written in pure Julia and ties Julia
code to our message spec for rich reprs, tab completion, etc. It works
exactly like the real IPython kernel with all of the same
capabilities. Similarly, a Ruby kernel would be written in Ruby and
run only Ruby code.

As we move to IPython 3.0 our support for other kernels is going to
improve dramatically - it will be possible to change the kernels with
simple dropdown in the notebook.

Given the limitations of the nested kernel approach and all of the
strong support for multiple native kernels, I am wondering if the
nested kernel approach makes sense in the long run. The only case
where I think it makes sense to use the nested approach is for toy
languages like logo that are well suited to a full blown kernel
itself. But it would be quite simple to write a %%logo magic for the
existing IPython kernels.

Cheers,

Brian

> Thanks for any pointers,
>
> -Doug
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com



More information about the IPython-dev mailing list