[Python-3000] Kill GIL?

[Josiah Carlson]

"Ross Jekel" <ross at sourcelabs.com> wrote:
> I know it is a bit old, but would Python Object Sharing (POSH) 
> http://poshmodule.sourceforge.net help you? Also, if you think you like the 
> current state-of-the-art threading model, you might not after reading this: 

The RPC-like mechanism I described earlier could be implemented on top
of POSH if desired, though I believe that some of the potential issues
that POSH has yet to fix (see its TODO file) aren't as much of a concern
(if at all) when only using shared memory as IPC and not as an object
store.

> "The Problem with Threads."

Getting *concurrency* right is hard.  One way of making it no so hard is
to use simple abstractions; like producer/consumer, deferred results,
etc.  But not everything fits into these abstractions, and sometimes
there are data structure manipulations that require locking. In that
sense, it's not so much that *concurrency* is hard to get right, as much
as locking is hard to get right.

But with Python 2.5, we get the 'with' statement and context managers. 
Add context managers to locks, always use RLocks (so that you can
.acquire() a lock multiple times), and while it hasn't gotten easy (one
needs to be careful with lock acquisition order to prevent deadlocks,
especially when mixing locks with queues), more concurrency tasks have
gotten *easier*.


Essentially the article points out that using abstractions like
producer/consumer, deferreds, etc., can make concurrent programming not
so hard, and you have to be basically insane to use threads in your
concurrent programming (I've been doing it for about 7 years, and am
thoroughly insane), but unless I'm missing something (I only skimmed the
article when it first came out, so this is quite possible), it's not
really saying anything new to the concurrent programmer (of nontrivial
systems).


With the API and RPC mechanism I sketched out earlier, threads are a
possible underlying implementation detail.  Essentially, it tries to
force everything into a producer/consumer abstraction; the function I
call is a consumer of the arguments I pass, and it produces a result
that I (or someone else) later consume.  This somewhat limits what kinds
of things can be done 'natively', but you can't get everything.


 - Josiah