Overcoming python performance penalty for multicore CPU
Martin v. Loewis
martin at v.loewis.de
Sun Feb 21 22:22:28 CET 2010
John Nagle wrote:
> I know there's a performance penalty for running Python on a
> multicore CPU, but how bad is it? I've read the key paper
> ("www.dabeaz.com/python/GIL.pdf"), of course. It would be adequate
> if the GIL just limited Python to running on one CPU at a time,
> but it's worse than that; there's excessive overhead due to
> a lame locking implementation. Running CPU-bound multithreaded
> code on a dual-core CPU runs HALF AS FAST as on a single-core
> CPU, according to Beasley.
I couldn't reproduce these results on Linux. Not sure what "HALF AS
FAST" is; I suppose it means "it runs TWICE AS LONG" - this is what I
If I run Beazley's program on Linux 2.6.26, on a 4 processor Xeon (3GHz)
machine, I get 30s for the sequential execution, 40s for the
multi-threaded case, and 32s for the multi-threaded case when pinning
the Python process to a single CPU (using taskset(1)).
So it's 6% overhead for threading, and 25% penalty for multicore CPUs -
far from the 100% you seem to expect.
More information about the Python-list