[Python-ideas] Add faster locks to the threading module?
sturla at molden.no
Wed Jul 21 02:51:53 CEST 2010
Thread synchronization with threading.Lock can be expensive. But
consider this: Why should the active thread need to aquire a mutex, when
it already holds one? That would be the GIL.
Instead of acqiring a lock (and possibly inducing thread swiching etc.),
it could just deny the other threads access to the GIL for a while. The
cost of that synchronization method would be completely amortized, as
check intervals happen anyway.
Here is what a very naïve implementation would look like in ctypes (real
code could use C instead, or perhaps should not attempt this at all...):
from contextlib import contextmanager
_Py_Ticker = ctypes.c_int.in_dll(ctypes.pythonapi,"_Py_Ticker")
tmp = _Py_Ticker.value
_Py_Ticker.value = 0x7fffffff
_Py_Ticker.value = tmp
Now we can do this:
# the GIL is mine,
# for as long as I want
The usecase for this "gillock" is about the same as for a spinlock C. We
want synchronization for a breif period of time, but don't want the
overhead of aquiring a mutex.
In Python this gillock has one big advantage over a spinlock: We don't
have to wait, so we don't risk a tread switch on __enter__/aquire. But
there can be only one instance of this lock, as there is only one GIL.
That is the drawback compared to a spinlock.
Therefore I think both a spinlock and a gillock should be added to the
threading module. These are synchronization methods that should be
P.S. A gillock working like the ctypes code above is of course very
dangerous. If a C extension releases the GIL while _Py_Ticker is
astronomic, we have a very bad situation... But real code could try to
safeguard against this. E.g. Py_BEGIN_ALLOW_THREADS and
Py_END_ALLOW_THREADS could be defined to do nothing if _Py_Ticker is
above some ridiculous threshold.
P.P.S. Yes I know about the newgil. But I have not thought about how to
achieve similar effect with that.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-ideas