python performance on Solaris

[Dieter Maurer]

Antoine Pitrou <solipsis at pitrou.net> writes on Thu, 15 Oct 2009 16:25:43 +0000 (UTC):
> Le Wed, 14 Oct 2009 22:39:14 -0700, John Nagle a écrit :
> > 
> >     Note that multithreaded compute-bound Python programs really suck
> > on multiprocessors.  Adding a second CPU makes the program go slower,
> > due to a lame mechanism for resolving conflicts over the global
> > interpreter lock.
> 
> I'm not sure what you're talking about. Python has no "mechanism for 
> resolving conflicts over the global interpreter lock" (let alone a lame 
> one :-)), it just trusts the OS to schedule a thread only when it is not 
> waiting on an unavailable resource (a lock). The GIL is just an OS-level 
> synchronization primitive and its behaviour (fairness, performance) will 
> depend on the behaviour of the underlying OS.

But, independent from the OS and the fairness/performance of the GIL
management itself: the GIL is there to prevent concurrent execution
of Python code. Thus, at any time, at most one thread (in a process)
is executing Python code -- other threads may run as well, as long
as they are inside non Python code but cannot be executing Python bytecode,
independent of available CPU resources. This implies that Python cannot
fully exploit the power of multiprocessors.

It is also true that adding CPUs may in fact reduce performance for
compute bound multithreaded Python programs. While the additional
computational resources cannot be use by Python, the additional overhead
(switching between CPUs) may reduce overall performance.
I agree with you that it is difficult to understand when this overhead
were really significant.

Dieter