[issue8299] Improve GIL in 2.7

[Kristján Valur Jónsson]

Kristján Valur Jónsson <kristjan at ccpgames.com> added the comment:

Maybe the state of this discussion is my fault for not being clear enough.
Let's abandon terms such as "broken" and "roundrobin."  CS theory has the perfectly useful terms "fair" and "unfair."  The fact of the matter is this:
the pthread GIL (implemented as LEGACY gil) is an "unfair" syncronization primitve.  The GIL on windows, (and the poorly named ROUNDROBIN_GIL) is a  "fair" synchronization primitive.

Unfair mutexes have their place, and such is the behaviour of the windows condition variable (and the pthreads mutex, I suspect).  But they are not useful if you want to provide fair access to a resource that is held all the time.

Until after that GIL business at PyCon I wasn't aware of this fundamental difference between the GIL on windows and pthreads platforms in 2.x.  It is astonishing to me that no one appears to have noticed the difference, or made much of it.  CPU threads are scheduled fairly on windows, and incredibly unfairly on pthreads.

with the ROUNDROBIN_GIL I'm not proposing anything radical, I'm just suggesting that we adopt the superior behaviour that has been on windows all along.  Yes, people actually do use windows.

Antoine, I understand that your point about do_yield, yet the results for 3 seconds without it are telling on their own, and worthy of being studied, which is why I suggested disabling it.

Also, I think you will find that he imbalance in the throughput of the threads won't go away even after 30 seconds.  Unfortunately, the unfairness is such that it may actually diverge.

I've improved my patch some more.  I'll upload it soon.  In particular, I've addea a PyThread_gil_yield() method to enable whatever underlying gil there is to possible deal with this particular locking case differently, if possible, perhaps suggesting to the OS not to switch cores.


I´ve also created a simple program in visual studio to examine a GIL outside the context of python.  I'll put it here tomorrow too, for those interested.  It allows for simpler experimentation, although because the loop is small, you won't see the effect of the instruction cache problems.


David, I actually think that the checkinterval is a perfectly good mechanism, especially if augmented with an interrupt mechanism  What does it matter if some opcodes are slower than others?  when we are checking every 100 or 1000 (or 10000 as I am proposing) that hardly matters.  We just need to have an order of magnitude thing there.  But there are other ways to do it.  You can use a timer on windows, and on pthreads too, I think.

But the whole point of this patch is to take a step back, and to see if there is a way to fix the "gil problem" in a simpler way by first trying to understand it fully, and then apply minimal changes to solve it.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue8299>
_______________________________________