Python threading (was: Re: global interpreter lock not working as it should)

[Armin Steinhoff]

Jonathan Hogg <jonathan at onegoodidea.com> wrote in message news:<B9740AA3.F1B9%jonathan at onegoodidea.com>...
> On 5/8/2002 10:12, in article
> ddc19db7.0208050112.56a26827 at posting.google.com, "Armin Steinhoff"
> <a-steinhoff at web.de> wrote:
> 
> > martin at v.loewis.de (Martin v. Loewis) wrote in message
> > news:<m3u1maavgh.fsf at mira.informatik.hu-berlin.de>...
>  [...]
> >> CHECK_STATUS("pthread_mutex_lock[1]");
> >> success = thelock->locked == 0;
> >> if (success) thelock->locked = 1;
> > 
> > 'success' will always be 0 ... so why 'if(success)' ??????
> > We are in a critical section !!!
> 
 'Aha. You've slightly misunderstood' what I wrote. I mentioned that a
code sequence like

       success = thelock->locked == 0;
       if (success) thelock->locked = 1;

done in critical section is plain nonsense.

 the way the pthreads GIL works. The mutex
> is used only to protect access to 'thelock->locked'. This variable
> represents the actual "Global Interpreter Lock".
> 
> Grabbing 'thelock->mut' doesn't necessarily mean you have the GIL, it just
> means you can safely test and set - which is what the above part does.

Thank's for telling me this basics ...

> >> status = pthread_mutex_unlock( &thelock->mut );
> >> CHECK_STATUS("pthread_mutex_unlock[1]");
> >> 
> >> if ( !success && waitflag ) {
> > 
> > The variable 'success' is meaningless ... who sets the waitflag ??
> 
> 'waitflag' is an argument to the function, but it doesn't matter
> particularly. 'success' will be true if the lock was free and we took it.
> 
> >> /* continue trying until we get the lock */
> >                  ????????
> 
> This will become clear further down.
> 
> >> 
> >> /* mut must be locked by me -- part of the condition
> >> * protocol */
> >> status = pthread_mutex_lock( &thelock->mut );
> >> CHECK_STATUS("pthread_mutex_lock[2]");
> > 
> > There is nothing to check ... you own now simply the mutex here.
> > 
> > The while loop is plain nonsens and dangerous because 'thelock->mut'
> > isn't acquired again!!
> 
> The mutex has been released before we enter this loop (the second quoted
> section above). In order to wait on a condition we have to hold the mutex
> again, so we grab it.
> 
> >> while ( thelock->locked ) {
> >> status = pthread_cond_wait(&thelock->lock_released,
> >>   &thelock->mut);
> >> CHECK_STATUS("pthread_cond_wait");
> > 
> > After calling pthread_cond_wait() the mutex 'thelock->mut' will be
> > released!
> 
> No.

Yes ... IT WILL be released and after leaving pthread_cond_wait()
re-acquired.
That's the reason why the unlock of the mutex after leaving
pthread_cond_wait() is neccessary.

> That's not the way pthread_cond_wait works.

No ... please read again the semantic of pthread_cond_wait.

> We must hold the mutex
> before we call this. The implementation atomically releases the mutex and
> puts us to sleep on the condition variable. When the thread is later woken
> because of the condition being signalled, the implementation atomically
> reacquires the mutex and returns.
> 
> A condition wait always occurs inside a critical section. The reason for
> this is that we are usually waiting on a condition being signalled to notify
> us that something is available or has changed (in this case, the GIL). That
> something must be protected by a mutex otherwise we could end up with a race
> condition where the thread is awoken to say something is ready, but the
> something has been changed by someone else before we get to it.
> 
> > The mutex 'thelock->mut' is only used to build a critical section
> > around the condition variable 'thelock->lock_released'
> > 
> > Who does pthread_cond_signal() and when ???
> 
> The pthread_cond_signal is done by the code that releases the GIL. This code
> (which isn't quoted) grabs 'thelock->mutex', unsets 'thelock->locked',
> releases the mutex and then signals 'thelock->lock_released' allowing
> another thread to be unblocked (and possibly switched to depending on the
> scheduler).
> 
> >> }
> > 
> > 'thelock->mut is now locked again ...
> > 
> >> thelock->locked = 1;
> > 
> > That's really corious ...
> 
> We own the mutex, and the 'while ( thelock->locked )' loop has exited so we
> know the lock is now free, so we can safely grab it.
> 
> >> status = pthread_mutex_unlock( &thelock->mut );
> >> CHECK_STATUS("pthread_mutex_unlock[2]");
> > 
> > 
> >> success = 1;
> >> }
> >> 
> >> Nobody is blocking on the mutex;
> > 
> > Sorry ... that's not true! Please read the semantic of
> > pthread_mutex_lock.
> 
> OK, this is slightly misleading. It is possible for a thread to block on the
> mutex, but highly unlikely.

There is no propbability here ... a thread WILL be suspended if it
don't get the mutex!

Jonathan ... your interpretation of this code sequence is build on
wrong assumptions.

No further comments to the rest of your posting ...

Armin

> The mutex is used only to protect the few
> instructions that access 'thelock->locked'. It is not used by threads to
> wait for the GIL. Threads waiting for the GIL block on the condition
> variable 'thelock->lock_released' using pthread_cond_wait, as above.
> 
> > This code looks like a MEDIUM chaos for me ...
> > 
> > It should  be possible to implement the GIL by a single condition
> > variable!
> 
> No, one must have a mutex in order to use conditions and returning from the
> condition does not guarantee that the GIL is free as the thread may have
> been unblocked, but not scheduled yet. In the time before it is scheduled,
> the original thread may have reacquired the GIL.
> 
> This is the problem I described before with SCHED_RR threads. The thread
> switch will not occur until the timer interrupt, and that is very unlikely
> to occur while the lock is released.
> 
> The pthreads-GIL actually makes a lot of sense, but you have to think hard
> to understand it. I can probably explain it more clearly if you like.
> 
> Jonathan