[IPython-dev] Interactive Matplotlib in the browser

Michael Droettboom mdroe at stsci.edu
Fri Oct 12 15:21:28 EDT 2012

On 10/12/2012 01:06 PM, Brian Granger wrote:
> On Thu, Oct 11, 2012 at 11:39 PM, Jason Grout
> <jason-sage at creativetrax.com> wrote:
>> On 10/11/12 10:45 PM, Brian Granger wrote:
>>> It is not clear to me that the stream of PNGs will win in the end.
>> What do you see as alternatives?  Reimplementing matplotlib in the
>> browser?  SVG-based rendering, where we send over a complete svg image
>> at each step (or just the changed svd elements)?  I'm curious what other
>> ideas you have.
> There are two cases:
> * Small data sets and relatively simple plots types.  For these, I do
> think it makes sense to send the raw JSON data to the browser and
> implement a Javascript based plotting library that consumes the JSON
> and has an extremely high level of interactivity.  I don't think we
> need to do *all* possible plot types with the pixel perfection,
> embedded math, infinite configurability.  When people are working
> interactively they want: i) plots that look spectacular - pixelized
> images don't cut it and ii) high levels of interactivity.
> I am not suggesting that we reimplement all of Matplotlib.

I agree about the reduced interactivity -- but there is an 
interactivity/functionality tradeoff, as you point out.  I used to just 
think that it was a matter of implementing the missing functionality in 
Javascript, but I have since come to realize there are just things it 
can't do in a reasonable way (multiple projection types, multiple image 
interpolation modes) without another quantum leap in browser functionality.

As for image quality, the approach outlined here uses the native pixel 
resolution, so it shouldn't be any more pixelized than anything else 
displayed on screen.  I understand that Safari on Macs with a retina 
display does upsampling of images by default, but with the appropriate 
CSS we should be able to address that.  Is it the lack of subpixel 
sampling that is bothersome?  We've always eschewed that in matplotlib 
because it makes the PNGs non-portable (different displays have 
different RGB orderings), but we could flip it on in this case.

> Keep in mind that current situation with Matplotlib and the IPython
> Notebook is not very ideal.  The pngs that are the default don't look
> that great and svg is not yet good enough to use as the default.  We
> really do need in browser plotting that *looks* better.

What doesn't look great about the PNGs?  A lot of effort has gone in to 
antialiasing and hinting the fonts in the best possible way and rounding 
lines to pixel centers etc to make it look as good as possible.  If 
there's specific shortcomings, they can be addressed, since matplotlib 
controls the entire rendering stack.

> Another important aspect of the JSON data approach.  We want to enable
> Notebooks to be viewed and used in contexts outside the actual
> notebook interface (nbviewer for example) where there is *no* python
> code that is run.  Because we save the JSON data in the notebook, JSON
> based plotting can be done interactively in contexts like nbviewer.
> For sharing computational results with a wide audience that is
> critical.
> * Large data sets
> I don't really have any alternatives - this is simply a really hard
> problem.  In some contexts downsampling is possible.  In other
> contexts it is possible to stream subsets of JSON data at are required
> for the current view.  It is my impression that the folks at Continuum
> are working on this problem.  Maybe they have some insight into this
> problem.

I was actually speaking to Travis Oliphant at length about this last 
week, and the upshot of that conversation was this present approach.


>> If
>>> you make a single static plot of a large data set, that is way better
>>> than trying to send the data to the browser and rendering it there.
>>> But if you have to send hundreds or thousands of PNGs to get
>>> interactivity, that benefit may be washed out.  Especially if you have
>>> multiple users interacting with plots - the server could quickly grind
>>> to a halt.  I think we should do tests to see how bad it gets, taking
>>> into account the multiple user question.  The one performance benefit
>>> that I can think of is that you can tune the level of interactivity to
>>> limit the data that comes back.  For large data sets, users might be
>>> willing to settle for less interactivity.  That option doesn't exist
>>> when you send all the data back.
>> I implemented server-side rate-limiting (see my most recent
>> modifications at https://gist.github.com/3875846, for example).  Or are
>> you saying that the server might not be able to render pngs for hundreds
>> or thousands of users?  I wonder if we can get a better protocol than
>> just a "Motion PNG" [1].
>> A disappointing thing is that it seems to take about 100ms to render the
>> example png image; it would sure be nice if that was a lot faster.
>> [1] http://en.wikipedia.org/wiki/Motion_JPEG
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev

More information about the IPython-dev mailing list