Status of Python threading support (GIL removal)?

[Piet van Oostrum]

>>>>> Jure Erznožnik <jure.erznoznik at gmail.com> (JE) wrote:

>JE> Sorry, just a few more thoughts:
>JE> Does anybody know why GIL can't be made more atomic? I mean, use
>JE> different locks for different parts of code?
>JE> This way there would be way less blocking and the plugin interface
>JE> could remain the same (the interpreter would know what lock it used
>JE> for the plugin, so the actual function for releasing / reacquiring the
>JE> lock could remain the same)
>JE> On second thought, forget this. This is probably exactly the cause of
>JE> free-threading reduced performance. Fine-graining the locks increased
>JE> the lock count and their implementation is rather slow per se. 

The major obstacles are the refcounts. It would mean that each
refcounted object would need a separate lock including constants like 0,
1, True, None. You would have much more locking and unlocking then with
the GIL. So to get rid of it first the refcounting has to go. But that
is not sufficient.

When the GIL will be removed I suspect that many concurrent programs
will start to fail subtly because they make assumptions about the
atomicity of the operations. In CPython each bytecode is atomic, but
when there is no GIL this is no longer true.

Java has defined this quite rigorously in its memory model (although
there first memory model had subtle bugs): Read and write operations on
32-bit quantities are atomic, others not. This means that on a 64-bit
system, where pointers are 64-bit even assignments on variables of
object types are not atomic and have to be protected with
synchronized. (My info is a few years old, so in the meantime it may
have changed.) In a GIL-less python the same would be true. Of course the
hardware that your program is running on could have atomic read/writes
on 64-bit quantities but if you rely upon that your program may no
longer be portable.
-- 
Piet van Oostrum <piet at cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: piet at vanoostrum.org