[Python-Dev] GIL, Python 3, and MP vs. UP

[Donovan Baarda]

On Tue, 2005-09-20 at 22:43, Guido van Rossum wrote:
> On 9/20/05, John J Lee <jjl at pobox.com> wrote:
[...]
> I don't know that any chips are designed with threading in mind. Fast
> threading benefits from fast context switches which benefits from
> small register sets. I believe the trend is towards ever large
> register sets. Also, multiple processors with shared memory don't
> scall all that well; multiple processors with explicit IPC channels
> scale much better. All arguments for multi-processing and against
> multi-threading.

Exactly! 

I believe the latest MP opteron chipsets use hypertransport busses to
directly access the other processor's memory and possibly CPU cache. In
theory this means shared memory will not hurt too badly, helping
threading. However, memory contention bottlenecks and cache coherency
will always mean shared memory hurts more, and will never scale better,
than IPC.

The reality is threads were invented as a low overhead way of easily
implementing concurrent applications... ON A SINGLE PROCESSOR. Taking
into account threading's limitations and objectives, Python's GIL is the
best way to support threads. When hardware (seriously) moves to multiple
processors, other concurrency models will start to shine. 

In the short term there will be various hacks to try and make the
existing plethora of threading applications run better on multiple
processors, but ultimately the overheads of shared memory will force
serious multi-processing to use IPC channels. If you want serious MP,
use processes, not threads.

I see anti-GIL threads again and again. Get over it... the GIL rocks for
threads :-)

-- 
Donovan Baarda <abo at minkirri.apana.org.au>