[IPython-dev] matplotlib webagg benchmarks

Michael Droettboom mdroe at stsci.edu
Tue Jul 30 17:28:42 EDT 2013

As promised in last week's Google Hangout to the IPython developers 
meeting -- I have some concrete timings and numbers on the matplotlib 
WebAgg backend in a couple of different scenarios.

First, let me apologize -- the way I was timing binary websockets vs. 
text websockets previously was wrong.  The actual impact of it is much 
smaller than I had originally estimated -- so the discussion about 
whether to include binary websockets in IPython may have been all for 

For benchmarking, I used two different plots.  One is the classic 
"simple_plot.py" sine wave, which tests sort of the "easy case" where 
very little of the image is updated in each frame, and the other was 
"animation/dynamic_image.py" in which most of the plot is updated in 
each frame.

I tested both scenarios with client and server on my local machine, and 
through an ssh tunnel that goes over wifi, the public university 
network, to my home's 15/5 MBps cable connection 28 miles away and back.

For (A), the average frame weighs in at around 20kb.  For (B), it's 
around 90kb.  For base64, multiply by those numbers by 4 / 3.

On my local machine, I can push through about 18 fps, so a bandwidth of 
2.8MBps (were it sustained, which it rarely is).  On the tunnel, I 
fluctuate between 7 and 10 fps, which is quite usable, and quite near 
the practical upper limit on the bandwidth of that connection.

However, the problematic thing for the remote connection is the 
latency.  Locally, I average a fairly steady 250ms to roundtrip from a 
mouse event to an updated frame.  Remotely, it fluctuates randomly 
between 400ms (still usable) and 3000ms.  Some more careful dynamic 
scaling of events can probably make that easier to use, perhaps.  I know 
games often use UDP and handle robustness to packet loss in a different 
way as a way to remove some of the latency of TCP.  I have no idea if 
such a thing would be possible over a web socket, of course.

I could not measure any statistically significant change in framerate or 
latency between a binary websocket and a non-binary one.  However, there 
is a 10% increase in CPU time on both the python side and the browser.  
It so happens that I wasn't saturating my CPU, so it had no net impact.  
Likewise, I am not saturating my bandwidth, so the additional size 
doesn't matter in this case.  But I suspect if either one of those 
resources is starved, the additional 10% cpu time and 25% bandwidth 
increase may matter.


More information about the IPython-dev mailing list