Overcoming python performance penalty for multicore CPU

[exarkun at twistedmatrix.com]

On 11:02 pm, nagle at animats.com wrote:
>    I know there's a performance penalty for running Python on a
>multicore CPU, but how bad is it?  I've read the key paper
>("www.dabeaz.com/python/GIL.pdf"), of course.  It would be adequate
>if the GIL just limited Python to running on one CPU at a time,
>but it's worse than that; there's excessive overhead due to
>a lame locking implementation.  Running CPU-bound multithreaded
>code on a dual-core CPU runs HALF AS FAST as on a single-core
>CPU, according to Beasley.

It's not clear that Beasley's performance numbers apply to any platform 
except OS X, which has a particularly poor implementation of the 
threading primitives CPython uses to implement the GIL.

You should check to see if it actually applies to your deployment 
environment.

The GIL has been re-implemented recently.  Python 3.2, I think, will 
include the new implementation, which should bring OS X performance up 
to the level of other platforms.  It may also improve certain other 
aspects of thread switching.

Jean-Paul