[Python-Dev] GIL, Python 3, and MP vs. UP (was Re: Variant of removing GIL.)
Guido van Rossum
guido at python.org
Mon Sep 19 02:50:13 CEST 2005
On 9/17/05, John J Lee <jjl at pobox.com> wrote:
> Given the points you make, and the facts that both Python 3 and real
> problems with continuing to scale down semiconductor chip feature sizes
> are on the horizon, it seems that now would be an excellent time to start
> work on this, with the goal of introducing it at the same time as Python
> a. Python 3.0 will break lots of code anyway, so the extension module
> issue becomes far less significant.
> b. In x years time (x < 10?) it seems likely multiprocessor (MP) users
> will be in the majority. (As a result, the uniprocessor (UP) slowdown
> becomes less important in practice, and also Python has the opportunity
> of avoiding the risk of being sidelined by a real or perceived lack of
> MP performance.)
That assumes a very specific model for how all that MP power is going
to be used. I personally don't think the threaded programming model as
found in Java works all that well; without locks you end up with
concurrent modification errors, with locks you get deadlocks and
livelocks. A typical programmer has a hard enough time keeping track
of a bunch of variables being modified by a single thread; add
multiple threads acting simultaneously on the same variables to the
mix, and it's a nightmare.
If my hunch is right, I expect that instead of writing massively
parallel applications, we will continue to write single-threaded
applications that are tied together at the process level rather than
at the thread level.
> c. Since time is needed to iron out bugs (and perhaps also to reimplememt
> some pieces of code "from scratch"), very early in the life of Python 3
> seems like the least-worst time to begin work on such a change.
> I realize that not all algorithms (nor all computational problems) scale
> well to MP hardware. Is it feasible to usefully compile both MP and a UP
> binaries from one Python source code base?
That's an understatement. I expect that *most* problems (even most
problems that we will be programming 10-20 years from now) get little
benefit out of MP.
> (I'm also quite aware that the GIL does not prevent all means of achieving
> efficient use of multiprocessors. I'm just concious that different
> parellisation problems are presumably best expressed using different
> tools, and that Python 3 and increased prevalance of MP systems might tip
> the balance in favour of removing the GIL.)
> Of course, it still takes a (anti-)hero to step forward and do the work...
Be my guest. Prove me wrong. Talk is cheap; instead of arguing my
points (all of which can be argued ad infinitum), come back when
you've got a working GIL-free Python. Doesn't have to be CPython-based
-- C# would be fine too.
--Guido van Rossum (home page: http://www.python.org/~guido/)
More information about the Python-Dev