[IPython-dev] matplotlib webagg benchmarks
Michael Droettboom
mdroe at stsci.edu
Tue Jul 30 17:28:42 EDT 2013
As promised in last week's Google Hangout to the IPython developers
meeting -- I have some concrete timings and numbers on the matplotlib
WebAgg backend in a couple of different scenarios.
First, let me apologize -- the way I was timing binary websockets vs.
text websockets previously was wrong. The actual impact of it is much
smaller than I had originally estimated -- so the discussion about
whether to include binary websockets in IPython may have been all for
naught.
For benchmarking, I used two different plots. One is the classic
"simple_plot.py" sine wave, which tests sort of the "easy case" where
very little of the image is updated in each frame, and the other was
"animation/dynamic_image.py" in which most of the plot is updated in
each frame.
I tested both scenarios with client and server on my local machine, and
through an ssh tunnel that goes over wifi, the public university
network, to my home's 15/5 MBps cable connection 28 miles away and back.
For (A), the average frame weighs in at around 20kb. For (B), it's
around 90kb. For base64, multiply by those numbers by 4 / 3.
On my local machine, I can push through about 18 fps, so a bandwidth of
2.8MBps (were it sustained, which it rarely is). On the tunnel, I
fluctuate between 7 and 10 fps, which is quite usable, and quite near
the practical upper limit on the bandwidth of that connection.
However, the problematic thing for the remote connection is the
latency. Locally, I average a fairly steady 250ms to roundtrip from a
mouse event to an updated frame. Remotely, it fluctuates randomly
between 400ms (still usable) and 3000ms. Some more careful dynamic
scaling of events can probably make that easier to use, perhaps. I know
games often use UDP and handle robustness to packet loss in a different
way as a way to remove some of the latency of TCP. I have no idea if
such a thing would be possible over a web socket, of course.
I could not measure any statistically significant change in framerate or
latency between a binary websocket and a non-binary one. However, there
is a 10% increase in CPU time on both the python side and the browser.
It so happens that I wasn't saturating my CPU, so it had no net impact.
Likewise, I am not saturating my bandwidth, so the additional size
doesn't matter in this case. But I suspect if either one of those
resources is starved, the additional 10% cpu time and 25% bandwidth
increase may matter.
Mike
More information about the IPython-dev
mailing list