thread, multiprocessing: communication overhead

[Duncan Booth]

mk <mrkafk at gmail.com> wrote:

> This time I decided to test communication overhead in multithreaded / 
> multiprocess communication. The results are rather disappointing, that 
> is, communication overhead seems to be very high. In each of the 
> following functions, I send 10,000 numbers to the function / 10 threads 
> / 10 processes, which simply returns it in its respective way.
> 
> 
> Function: notfun            Best: 0.00622 sec   Average: 0.00633 sec
> (simple function)
> 
> Function: threadsemfun      Best: 0.64428 sec   Average: 0.64791 sec
> (10 threads synchronizing using semaphore)
> 
> Function: threadlockfun     Best: 0.66288 sec   Average: 0.66453 sec
> (10 threads synchronizing using locks)
> 
> Function: procqueuefun      Best: 1.16291 sec   Average: 1.17217 sec
> (10 processes communicating with main process using queues)
> 
> Function: procpoolfun       Best: 1.18648 sec   Average: 1.19577 sec
> (a pool of 10 processes)
> 
> If I'm doing smth wrong in the code below (smth that would result in 
> performance suffering), please point it out.

You aren't just timing the communication overhead: each of your functions 
creates a pool of 10 threads or 10 processes on every run, so your times 
include the startup and shutdown times.