GIL in the new glossary

[Skip Montanaro]

    >> The lock used by Python threads to assure that only one thread can be
    >> run at a time. This simplifies Python by assuring that no two
    >> processes can access the same memory at the same time. Locking the
    >> entire interpreter makes it easier for the interpreter to be
    >> multi-threaded, at the expense of some parallelism on multi-processor
    >> machines.

    Luis> Some parallelism???  Wouldn't it be more accurate to say "at the
    Luis> expense of parallelism?"  The GIL doesn't eliminate "some"
    Luis> paralellism, it completely eliminates any chance of parallelism
    Luis> within the same interpreter.

The global interpreter lock can explicitly be released from C code.  Lots of
I/O code does this already.  Only one thread can be executing Python byte
code at a time, but multiple threads can execute non-interpreter C code.
(I'm sure someone more knowledgeable about threads will correct me if I'm
off-base here.)

    >> Efforts have been made in the past to create a "free-threaded"
    >> interpreter (one which locks shared data at a much finer
    >> granularity), but performance suffered in the common single-processor
    >> case.

    Luis> This is true for kernels too, which is why you see at least
    Luis> two versions of the linux kernel in red hat distributions,
    Luis> one with SMP enabled and one without it.

    Luis> Why not have a python_smp interpreter that allows parallelism
    Luis> (and scalability) on SMP machines and another for uniprocessor
    Luis> machines where paralellism is not possible?

    Luis> Yeah, yeah, I know what you're going to say ... 

    Luis> "Please submit a patch."

    Luis> Sigh, ... if only I had the time ... :-(

I think you'd need a lot more time than you think (hint: don't think in
terms of "a patch"; think more in terms of "code fork"). ;-)

Greg Stein (an extremely bright guy who doesn't seem to hang out in the
Python niches much anymore - our loss) put in a fair amount of effort on
this when 1.4 was the current Python version, so you know this is something
people have been asking about for quite some time.  Here's a relevant
message from Greg regarding the performance he saw:

    http://mail.python.org/pipermail/python-dev/2001-August/017099.html

Based upon his numbers, I'm not sure it would be worth the effort, even if
you could get it right.  Going from 1x to 1.2x performance when adding a
second processor hardly seems worth it.

To make matters worse, Python has only gotten more complex in the time since
Greg did his work.  In addition, a number of things added to the language or
the library since then have quietly assumed the presence of the global
interpreter lock - "quietly", as in there's no big old "XXX" comment in the
code alerting would-be free threaders.  Taken together, this means there are
many more bits of code which would need attention.  It's probably a lot
easier to wait for the next fastest uniprocessor machine or overclock the
one you have.

Skip