On 27/03/07, Josiah Carlson email@example.com wrote:
Those numbers look pretty good. Would I be correct in assuming that there is a speedup sending blocks directly between processes? (though perhaps not the 6x that integer sending gains)
Yes, sending blocks directly between processes is over 3 times faster for 1k blocks, and twice as fast for 4k blocks, but after that it makes little difference. (This is using the 'processing.connection' sub-package which is partly written in C.)
Of course since these blocks are string data you can avoid the pickle translation which makes things get faster still: the peak bandwidth I get is 40,000 x 16k blocks / sec = 630 Mb/s.
PS. It would be nice if the standard library had support for sending message oriented data over a connection so that you could just do 'recv()' and 'send()' without worrying about whether the whole message was successfully read/written. You can use 'socket.makefile()' for line oriented text messages but not for binary data.