[IPython-dev] javascript / Python communication for image viewer

[Matthew Brett]

Guys / gals,

I want to ask for advice about writing a brain image display widget for
IPython.

I would like to make an IPython widget that can take an in-memory numpy array
and do an interactive display of orthogonal slices from the array.  The
display will look something like Papaya:

http://rii.uthscsa.edu/mango/papaya

where clicking or moving the mouse causes matching slices to be displayed
through the three axes of the numpy array (here a brain image).

Papaya is pure javascript, so I am assuming that it loads the whole array
(brain image) into a javascript variable and takes slices from that.

What I would like to do, is to be able to keep the whole 3D array only in
Python, and pass the slices as needed to a javascript viewer.

In my ignorance, I am not sure which approach to go for first.

Should I use the comm / widget interface for this?  In that case I guess the
procedure would be:

* mouse movement generates a 'need slice' message from javascript
* python kernel accepts 'need slice' message, takes slice from array, base64
  encodes into JSON, sends 'here is your slice' message back to javascript
  with the data
* javascript accepts message, unpacks base64'ed JSONed slice into variable and
  displays slice variable

Is that right?  I guess that involves Is there any chance that this
will be fast enough for
satisfying interactive movement through the image, which would likely require
something like 20 slices per second, each of say 64 x 64 floating point?

If not - is there something else I should look at instead?

Another question for more thanks - should I use a Canvas or SVG element to
display the images?  The fabric.js README [1] seems to imply the
Canvas element is
faster for some interactive stuff, does anyone have relevant experience to
share?

Thanks a lot,

Matthew

[1] https://github.com/kangax/fabric.js/#history