Multiprocessing: don't push the pedal to the metal?

[Chris Angelico]

On Mon, May 23, 2011 at 7:06 AM, John Ladasky <ladasky at my-deja.com> wrote:
> If I spawn N worker sub-processes, my application in fact has N+1
> processes in all, because there's also the master process itself.

This would definitely be correct. How much impact the master process
has depends on how much it's doing.

> I'd still appreciate hearing from anyone else who has more experience
> with multiprocessing.  If there are general rules about how to do this
> best, I haven't seen them posted anywhere.  This may not be a Python-
> specific issue, of course.

I don't have much experience with Python's multiprocessing model, but
I've done concurrent programming on a variety of platforms, and there
are some common issues.

Each CPU (or core) has its own execution cache. If you can keep one
thread running on the same core all the time, it will benefit more
from that cache than if it has to keep flitting from one to another.

You undoubtedly will have other processes in the system, too. As well
as your master, there'll be processes over which you have no control
(unless you're on a bare-bones system). Some of them may preempt your
processes.

Leaving one CPU/core available for "everything else" may allow the OS
to keep each thread on its own core. Having as many workers as cores
means that every time there's something else to do, one of your
workers has to be kicked off its nice warm CPU and sent out into the
cold for a while. If all your workers are at the same priority, it
will then grab a timeslice off one of the other cores, kicking its
incumbent off... rinse and repeat.

This is a tradeoff, though. If the rest of your system is going to use
0.01 of a core, then 1% thrashing is worth having one more core
available 99% of the time. If the numbers are reversed, it's equally
obvious that you should leave one core available. In your case, it's
probably turning out that the contention causes more overhead than the
extra worker is worth.

That's just some general concepts, without an in-depth analysis of
your code and your entire system. It's probably easier to analyse by
results rather than inspection.

Chris Angelico