[IPython-dev] load on controller node

[Glen W. Mabey]

Hello,

Using 'saw', I'm trying to understand the CPU usage levels that I'm
observing on the 40 engines (10x2x2 xeon) relative to the load on the
controller CPU (the remote controller is instantiated on yet another
host).

In my outermost for-loop, I use executeAll() followed by a pullAll().  I
inserted a time.sleep(5) between them, and observed the CPU load shown
on the attached graph.  On this graph, each host's % utilization for
each CPU is summed together, for a max of 4.  The mp* hosts run 4
engines each, and the tp host runs the controller, and the most recent
stats appear on the right (axis mis-labeled).  Thus, the hump between 10
and 17 corresponds to the executeAll(), between 17 and 21 the sleep(),
and 22 to 25 the pullAll().

What is surprising to me is that the load on the controller lasts so
much longer (in time) than the load on the engines caused by the
pullAll(), although they probably have the same area ...  

So, I guess there is nothing shocking in the graph, although it would be
interesting to see how things would change if the controller were able
to use more than 1 CPU.

Best Regards,
Glen Mabey

BTW, the low time between 25 and 27 is a disk access, and I plan to
reduce the effect of this by running a tandem set of engines that simply
pull the data into the NFS cache ...  we'll see how that goes.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: gkrellShoot_08-29-07_082048.png
Type: image/png
Size: 70286 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20070829/751a4957/attachment.png>