multiprocessing vs thread performance
castironpi at gmail.com
Mon Dec 29 16:35:46 CET 2008
On Dec 29, 8:52 am, mk <mrk... at gmail.com> wrote:
> Hello everyone,
> After readinghttp://www.python.org/dev/peps/pep-0371/I was under
> impression that performance of multiprocessing package is similar to
> that of thread / threading. However, to familiarize myself with both
> packages I wrote my own test of spawning and returning 100,000 empty
> threads or processes (while maintaining at most 100 processes / threads
> active at any one time), respectively.
> The results I got are very different from the benchmark quoted in PEP
> 371. On twin Xeon machine the threaded version executed in 5.54 secs,
> while multiprocessing version took over 222 secs to complete!
> Am I doing smth wrong in code below? Or do I have to use
> multiprocessing.Pool to get any decent results?
I'm running a 1.6 GHz. I only ran 10000 empty threads and 10000 empty
processes. The threads were the ones you wrote. The processes were
empty executables written in a lower language, also run 100 at a time,
started with 'subprocess', not 'multiprocessing'. The threads took
1.2 seconds. The processes took 24 seconds.
The processes you wrote had only finished 3000 after several minutes.
More information about the Python-list