I'll get on the new tests, I already have a bandwidth one written, so I'm running it now. As for Twisted's throughput performance, it's at least partly our fault. Since the receiving is in Python, every time we try to send there are incoming results getting in the way. If we wrote it such that sending prevented the receipt of results, I'm sure the Twisted code would be faster for large numbers of messages. With ZMQ, though, we don't have to be receiving in Python to get the results to the client process, so they arrive in ZMQ and await simple memcpy/deserialization.<div>
<br></div><div>-MinRK<br><br><div class="gmail_quote">On Fri, Oct 22, 2010 at 09:27, Brian Granger <span dir="ltr"><<a href="mailto:ellisonbg@gmail.com">ellisonbg@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Min,<div><br></div><div>Also, can you get memory consumption numbers for the controller and queues. I want to see how much worse Twisted is in that respect.</div><div><br></div><div>Cheers,</div><div><br></div><div><font color="#888888">Brian<br>
<br></font><div class="gmail_quote"><div class="im">On Thu, Oct 21, 2010 at 11:53 PM, MinRK <span dir="ltr"><<a href="mailto:benjaminrk@gmail.com" target="_blank">benjaminrk@gmail.com</a>></span> wrote:<br></div><div>
<div></div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>I have my first performance numbers for throughput with the new parallel code riding on ZeroMQ, and results are fairly promising. Roundtrip time for ~512 tiny tasks submitted as fast as they can is ~100x faster than with Twisted.</div>
<div><br></div><div>As a throughput test, I submitted a flood of many very small tasks that should take ~no time:</div><div><div>new-style:</div><div>def wait(t=0):</div>
<div> import time</div><div> time.sleep(t)</div></div><div>submit:</div><div>client.apply(wait, args=(t,))</div><div><br></div><div>Twisted:</div><div>
task = StringTask("import time; time.sleep(%f)"%t)</div><div>submit:</div><div>client.run(task)</div><div><br></div><div>Flooding the queue with these tasks with t=0, and then waiting for the results, I tracked two times:</div>
<div>Sent: the time from the first submit until the last submit returns</div><div>Roundtrip: the time from the first submit to getting the last result</div><div><br></div><div>Plotting these times vs number of messages, we see some decent numbers:</div>
<div>* The pure ZMQ scheduler is fastest, 10-100 times faster than Twisted roundtrip</div><div>* The Python scheduler is ~3x slower roundtrip than pure ZMQ, but no penalty to the submission rate</div><div>* Twisted performance falls off very quickly as the number of tasks grows</div>
<div>* ZMQ performance is quite flat</div><div><br></div><div>Legend:</div><div>zmq: the pure ZMQ Device is used for routing tasks</div><div>lru/weighted: the simplest/most complicated routing schemes respectively in the Python ZMQ Scheduler (which supports dependencies)</div>
<div>twisted: the old IPython.kernel</div><div><br></div><div><img src="cid:ii_12bd2af6afc0eab2" alt="roundtrip.png" title="roundtrip.png"></div><div><img src="cid:ii_12bd2af38d3422df" alt="sent.png" title="sent.png"><br>
</div><div>Test system:</div><div>Core-i7 930, 4x2 cores (ht), 4-engine cluster all over tcp/loopback, Ubuntu 10.04, Python 2.6.5</div><div><br></div><font color="#888888"><div>-MinRK</div><div><a href="http://github.com/minrk" target="_blank">http://github.com/minrk</a></div>
</font></blockquote></div></div></div><br><br clear="all"><div class="im"><br>-- <br>Brian E. Granger, Ph.D.<br>Assistant Professor of Physics<br>Cal Poly State University, San Luis Obispo<br><a href="mailto:bgranger@calpoly.edu" target="_blank">bgranger@calpoly.edu</a><br>
<a href="mailto:ellisonbg@gmail.com" target="_blank">ellisonbg@gmail.com</a><br>
</div></div>
</blockquote></div><br></div>