global interpreter lock not working as it should

[Armin Steinhoff]

Tim Peters <tim.one at comcast.net> wrote in message news:<mailman.1028062913.30234.python-list at python.org>...
> [anton wilson]
> > ...
> > What's the purpose of releasing an reaquiring the lock if no
> > other threads can run?
> 
> They can, as other responses have amply demonstrated.  Whether they *will*
> is up to the platform thread package, which was presumably designed to pass
> out timeslices in a way maximally efficient for it.  It's not Python's job
> to force thread switches against the better judgment of the thread facility
> authors; you can do that yourself if you think it's required, e.g. via
> time.sleep(0.001).
> 
> A peculiar thing here is that "the Python lock" is an abstraction.  You said
> you were running on Linux, and "the Python lock" isn't a lock there.  In
> Pythons released to date, a Python lock under pthreads is a combination of a
> phtreads mutex never held across more than a few C instructions, and a
> pthreads condition variable.  When "the lock" is released, a condvar signal
> is done to notify some other thread that it can grab the lock

The other threads can only grab the lock if they get the CPU ... but
the releasing thread is still running at the same priority level!

>  This happens
 before the relinquishing thread tries to reacquire the lock.

Sounds like scheduling magic ... this can only happen if the time
slice of the releasing thread get exhausted before the thread can
re-aquire the lock, which may be happen 10 times a year :)

Armin






>  So the
> implementation of condition variables on your box apparently decides not to
> take action on a signal at once, dallying until the original thread has
> already reacquired "the lock".  That's fine -- it's allowed to do that, and
> it surely has its own performance reasons for doing so.
> 
> If you try the current CVS version of Python, the implementation of "the
> lock" on Linux has changed to use POSIX semaphores instead.  They may or may
> not act more like you hope -- I don't know (haven't tried it).  It seems
> possible that they might, since semaphores came out of the POSIX realtime
> extensions, where people feel much freer to make unreasonable <wink>
> demands.
> 
> > ...
> > I have a feeling that there is a hidden race condition between when the
> > thread wakes up the other thread and tries to reaquire the lock.
> 
> "The race" would be in the platform implementation of pthread_cond_signal(),
> but I expect that's working the way its author intended.