multiprocessing vs thread performance

James Mills prologic at
Tue Dec 30 01:05:06 CET 2008

On Tue, Dec 30, 2008 at 12:52 AM, mk <mrkafk at> wrote:
> Hello everyone,
> After reading I was under
> impression that performance of multiprocessing package is similar to that of
> thread / threading. However, to familiarize myself with both packages I
> wrote my own test of spawning and returning 100,000 empty threads or
> processes (while maintaining at most 100 processes / threads active at any
> one time), respectively.
> The results I got are very different from the benchmark quoted in PEP 371.
> On twin Xeon machine the threaded version executed in 5.54 secs, while
> multiprocessing version took over 222 secs to complete!
> Am I doing smth wrong in code below? Or do I have to use
> multiprocessing.Pool to get any decent results?

The overhead in starting OS level processes
is quite high. This is why event-driven, single
process servers can perform far better than
ones that fork (spawn multiple processes)
per request.

As others have mentioned, it's not suprising
that spawning even 100 processes took some

Bottom line: multiprocessing should not be used this way.
(nor should threading).


More information about the Python-list mailing list