[IPython-dev] javascript / Python communication for image viewer

[Cyrille Rossant]

We have pretty similar requirements for Vispy, as we target fast
visualization of big datasets using OpenGL in the IPython notebook. In
particular, we can't use Canvas or SVG because it's just too slow for
big data, so we have to use WebGL. This lets us leverage the GPU for
visualization.

I would be very interested in seeing the answers to your questions.
Notably, is it possible to use binary sockets for transferring large
amounts of binary data between Python and JavaScript?

Cyrille

2014-08-27 20:52 GMT+01:00 Matthew Brett <matthew.brett at gmail.com>:
> Guys / gals,
>
> I want to ask for advice about writing a brain image display widget for
> IPython.
>
> I would like to make an IPython widget that can take an in-memory numpy array
> and do an interactive display of orthogonal slices from the array.  The
> display will look something like Papaya:
>
> http://rii.uthscsa.edu/mango/papaya
>
> where clicking or moving the mouse causes matching slices to be displayed
> through the three axes of the numpy array (here a brain image).
>
> Papaya is pure javascript, so I am assuming that it loads the whole array
> (brain image) into a javascript variable and takes slices from that.
>
> What I would like to do, is to be able to keep the whole 3D array only in
> Python, and pass the slices as needed to a javascript viewer.
>
> In my ignorance, I am not sure which approach to go for first.
>
> Should I use the comm / widget interface for this?  In that case I guess the
> procedure would be:
>
> * mouse movement generates a 'need slice' message from javascript
> * python kernel accepts 'need slice' message, takes slice from array, base64
>   encodes into JSON, sends 'here is your slice' message back to javascript
>   with the data
> * javascript accepts message, unpacks base64'ed JSONed slice into variable and
>   displays slice variable
>
> Is that right?  I guess that involves Is there any chance that this
> will be fast enough for
> satisfying interactive movement through the image, which would likely require
> something like 20 slices per second, each of say 64 x 64 floating point?
>
> If not - is there something else I should look at instead?
>
> Another question for more thanks - should I use a Canvas or SVG element to
> display the images?  The fabric.js README [1] seems to imply the
> Canvas element is
> faster for some interactive stuff, does anyone have relevant experience to
> share?
>
> Thanks a lot,
>
> Matthew
>
> [1] https://github.com/kangax/fabric.js/#history
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev