[Python-Dev] Reworking the GIL

[Jeffrey Yasskin]

On Mon, Nov 2, 2009 at 4:15 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Martin v. Löwis <martin <at> v.loewis.de> writes:
>>
> [gil_drop_request]
>> Even if it is read from memory, I still wonder what might happen on
>> systems that require explicit memory barriers to synchronize across
>> CPUs. What if CPU 1 keeps reading a 0 value out of its cache, even
>> though CPU 1 has written an 1 value a long time ago?
>>
>> IIUC, any (most?) pthread calls would cause synchronization in that
>> case, which is why applications that also use locks for reading:
>>
>> http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_04_10
>>
>> Of course, on x86, you won't see any issues, because it's cache-coherent
>> anyway.
>
> I think there are two things here:
> - all machines Python runs on should AFAIK be cache-coherent: CPUs synchronize
> their views of memory in a rather timely fashion.
> - memory ordering: writes made by a CPU can be seen in a different order by
> another CPU (i.e. CPU 1 writes A before B, but CPU 2 sees B written before A). I
> don't see how this can apply to gil_drop_request, since it's a single variable
> (and, moreover, only a single bit of it is significant).
>
> (there's an explanation of memory ordering issues here:
> http://www.linuxjournal.com/article/8211)
>
> As a side note, I remember Jeffrey Yasskin trying to specify an ordering model
> for Python code
> (see http://code.google.com/p/unladen-swallow/wiki/MemoryModel).

Note that that memory model was only for Python code; the C code
implementing it is subject to the C memory model, which is weaker (and
not even fully defined until the next C standard comes out).

To be really safe, we ought to have a couple primitives implementing
"atomic" rather than just "volatile" instructions, but until then a
signal that's just saying "do something" rather than "here's some data
you should look at" should be ok as a volatile int.

I'd like to look at the patch in detail, but I can't guarantee that
I'll get to it in a timely manner. I'd say check it in and let more
threading experts look at it in the tree. We've got some time before a
release for people to fix problems and make further improvements. +1
to Martin's request for detailed documentation though. :)