basic thread question

[John Nagle]

Jan Kaliszewski wrote:
> 18-08-2009 o 22:10:15 Derek Martin <code at pizzashack.org> wrote:
> 
>> I have some simple threaded code...  If I run this
>> with an arg of 1 (start one thread), it pegs one cpu, as I would
>> expect.  If I run it with an arg of 2 (start 2 threads), it uses both
>> CPUs, but utilization of both is less than 50%.  Can anyone explain
>> why?
>>
>> I do not pretend it's impeccable code, and I'm not looking for a
>> critiqe of the code per se, excepting the case where what I've written
>> is actually *wrong*. I hacked this together in a couple of minutes,
>> with the intent of pegging my CPUs.  Performance with two threads is
>> actually *worse* than with one, which is highly unintuitive.  I can
>> accomplish my goal very easily with bash, but I still want to
>> understand what's going on here...
>>
>> The OS is Linux 2.6.24, on a Ubuntu base.  Here's the code:
> 
> Python threads can't benefit from multiple processors (because of GIL,
> see: http://docs.python.org/glossary.html#term-global-interpreter-lock).

     This is a CPython implementation restriction.  It's not inherent in
the language.

     Multiple threads make overall performance worse because Python's
approach to thread locking produces a large number of context switches.
The interpreter unlocks the "Global Interpreter Lock" every N interpreter
cycles and on any system call that can block, which, if there is a
thread waiting, causes a context switch.

     Multiple Python processes can run concurrently, but each process
has a copy of the entire Python system, so the memory and cache footprints are
far larger than for multiple threads.

				John Nagle