[IPython-dev] First Performance Result

[MinRK]

I have the basic queue built into the controller, and a kernel embedded into
the Engine, enough to make a simple performance test.

I submitted 32k simple execute requests in a row (round robin to engines,
explicit multiplexing), then timed the receipt of the results (tic each 1k).
I did it once with 2 engines, once with 32. (still on a 2-core machine, all
over tcp on loopback).

Messages went out at an average of 5400 msgs/s, and the results came back at
~900 msgs/s.
So that's 32k jobs submitted in 5.85s, and the last job completed and
returned its result 43.24s  after the submission of the first one (37.30s
for 32 engines). On average, a message is sent and received every 1.25 ms.
When sending very small number of requests (1-10) in this way to just one
engine, it gets closer to 1.75 ms round trip.

In all, it seems to be a good order of magnitude quicker than the Twisted
implementation for these small messages.

Identifying the cost of json for small messages:

Outgoing messages go at 9500/s if I use cPickle for serialization instead of
json. Round trip to 1 engine for 32k messages: 35s. Round trip to 1 engine
for 32k messages with json: 53s.

It would appear that json is contributing 50% to the overall run time.

With %timeit x.loads(x.dumps(msg))
on a basic message, I find that json is ~15x slower than cPickle.
And by these crude estimates, with json, we spend about 35% of our time
serializing, as opposed to just 2.5% with pickle.

I attached a bar plot of the average replies per second over each 1000 msg
block, overlaying numbers for 2 engines and for 32. I did the same comparing
pickle and json for 1 and 2 engines.

The messages are small, but a tiny amount of work is done in the kernel.
The jobs were submitted like this:
        for i in xrange(32e3/len(engines)):
          for eid,key in engines.iteritems():
            thesession.send(queue, "execute_request",
dict(code='id=%i'%(int(eid)+i)),ident=str(key))
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100722/ebaf3b9f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bar.png
Type: image/png
Size: 29718 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100722/ebaf3b9f/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pickle.png
Type: image/png
Size: 51352 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100722/ebaf3b9f/attachment-0001.png>