[Python-ideas] Python and Concurrency
r.m.oudkerk at googlemail.com
Wed Mar 28 02:36:32 CEST 2007
On 27/03/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> Those numbers look pretty good. Would I be correct in assuming that
> there is a speedup sending blocks directly between processes? (though
> perhaps not the 6x that integer sending gains)
Yes, sending blocks directly between processes is over 3 times faster
for 1k blocks, and twice as fast for 4k blocks, but after that it makes
little difference. (This is using the 'processing.connection'
sub-package which is partly written in C.)
Of course since these blocks are string data you can avoid the pickle
translation which makes things get faster still: the peak bandwidth I
get is 40,000 x 16k blocks / sec = 630 Mb/s.
PS. It would be nice if the standard library had support for sending
message oriented data over a connection so that you could just do
'recv()' and 'send()' without worrying about whether the whole message
was successfully read/written. You can use 'socket.makefile()' for
line oriented text messages but not for binary data.
More information about the Python-ideas