Python threading (was: Re: global interpreter lock not working as it should)

[Armin Steinhoff]

martin at v.loewis.de (Martin v. Loewis) wrote in message news:<m3u1maavgh.fsf at mira.informatik.hu-berlin.de>...
> bokr at oz.net (Bengt Richter) writes:
> 
> > >> A mutex should result in a context switch every time there is a
> > >> release with a waiter present, since the releaser would reliably
> > >> fail to re-acquire.  Perhaps that is the way it works on BSD?
> > >> That might at least partly explain Jonathan's results.
>  
> > >I doubt that. 
> > Why?
> 
> Because the other threads waiting for the GIL do not block on a
> mutex. So mutex wait lists should not be relevant for this behaviour.
> 
> > IOW, I don't think the new owner has to execute before it becomes owner
> > of the mutex, I think the OS hands it over as part of the release operation
> > if there is a waiter. Why would it be done otherwise?
> 
> Because of this code in thread_pthread.h
> 
> 	status = pthread_mutex_lock( &thelock->mut );

this starts a critical section for the handling of 'thelock->locked'
It will (must) block other threads which are trying to update
'thelock->locked'

> 	CHECK_STATUS("pthread_mutex_lock[1]");
> 	success = thelock->locked == 0;
> 	if (success) thelock->locked = 1;

'success' will always be 0 ... so why 'if(success)' ??????
We are in a critical section !!!

> 	status = pthread_mutex_unlock( &thelock->mut );
> 	CHECK_STATUS("pthread_mutex_unlock[1]");
> 
> 	if ( !success && waitflag ) {

The variable 'success' is meaningless ... who sets the waitflag ??

> 		/* continue trying until we get the lock */
                   ????????
> 
> 		/* mut must be locked by me -- part of the condition
> 		 * protocol */
> 		status = pthread_mutex_lock( &thelock->mut );
> 		CHECK_STATUS("pthread_mutex_lock[2]");

There is nothing to check ... you own now simply the mutex here.

The while loop is plain nonsens and dangerous because 'thelock->mut'
isn't acquired again!!

> 		while ( thelock->locked ) {
> 			status = pthread_cond_wait(&thelock->lock_released,
> 						   &thelock->mut);
> 			CHECK_STATUS("pthread_cond_wait");

After calling pthread_cond_wait() the mutex 'thelock->mut' will be
released!

The mutex 'thelock->mut' is only used to build a critical section
around the condition variable 'thelock->lock_released'

Who does pthread_cond_signal() and when ???

> 		}

'thelock->mut is now locked again ...

> 		thelock->locked = 1;

That's really corious ... 

> 		status = pthread_mutex_unlock( &thelock->mut );
> 		CHECK_STATUS("pthread_mutex_unlock[2]");


> 		success = 1;
> 	}
> 
> Nobody is blocking on the mutex;

Sorry ... that's not true! Please read the semantic of
pthread_mutex_lock.

This code looks like a MEDIUM chaos for me ... 

It should  be possible to implement the GIL by a single condition
variable!


Armin


> the other thread blocks on the
> condition variable instead. The first thread signals the condition
> variable, then tries to lock the mutex. If that succeeds, it will
> reacquire the lock. The other thread will come out of the cond_wait,
> and find that the lock is still locked. It then will wait on the
> condition variable again (probably adding itself to the end of the
> condition's wait list).
> 
> > OTOH, if there is a variable associated with the mutex that is
> > supposed to represent some state of the interpreter, and other
> > threads are reading this without synchronizing, then I can see a
> > possible (different) race.
> 
> The ->locked field of the lock is protected by a mutex, so there is no
> danger of it getting inconsistent - it is just not clear who will lock
> it.
> 
> > If a race condition is possible, I think the OS mutex implementation
> > is not good or more likely there's a bug in its use and/or simulation.
> 
> The OS mutex implementation is not (directly) used. Only the cond_wait
> implementation (indirectly) tries to acquire the mutex.
> 
> > Because mutex is not used after 2.2? But I thought 2.2 was the
> > 'business' edition.  If there's a race condition there, should it
> > not be looked into and fixed?
> 
> There is no bug in the sense that inconsistency could occur. There is
> just no guarantee that locks are fair in Python.
> 
> Regards,
> Martin