Python threading (was: Re: global interpreter lock not working as it should)

[Armin Steinhoff]

Jonathan Hogg <jonathan at onegoodidea.com> wrote in message news:<B972B2D4.F12E%jonathan at onegoodidea.com>...
> On 3/8/2002 21:00, in article
> ddc19db7.0208031200.5823a2e at posting.google.com, "Armin Steinhoff"
> <a-steinhoff at web.de> wrote:
> 
> > Jonathan Hogg <jonathan at onegoodidea.com> wrote in message
> > news:<B97159DB.F0A9%jonathan at onegoodidea.com>...
>  [...]
> >

[ clip ...]
> > In all other cases the thread will be supended after exhausting its
> > timeslice (RR sched) and will own the GIL further .. other tasks will
> > be started and immediatly supended when they are trying to get the
> > GIL.
> > Have in mind that _ALL_ threads are running at the same
> > (ficed)priority level.
> > 
> >> There is nothing wrong with the code in ceval.c in this regard.
> > 
> > Releasing the GIL can't create a context switch if all threads running
> > at the same priority ... that's the problem.
> 
> I'm sorry, but that's just plain rubbish.

I'm not convinced ... 

> If all the threads have the same
> priority then the thread scheduler will schedule them according to
> timeslice.

Yes ... but this is only the worst case. 

> I'm really not sure where you get the idea that a thread can only
> be pre-empted by a higher priority thread. That's just not true.

Sorry ... where did I say that a thread can only
 be pre-empted by a higher priority thread ???  I'm sorry, but that's
just plain rubbish. (cat and paste :)

I said: it is absolute unlikely that a thread might exhaust its
timelslice between the 'release and re-acquire GIL' call.

That means in 99.99% of all cases ... there is no context switch after
releasing the GIL (-> if all threads asking for the GIL are running at
the same priority)

That's simply my statement.
 
> >> Priority inversion is actually extremely unlikely in Python 
> >> because the main shared resources is the GIL.

Sorry ... in that case I can follow your logic ..

> There is only one of them so all threads that
> >> are not waiting on I/O require it. Therefore a medium priority thread will
> >> be unable to pre-empt the lower priority thread (or more accurately it will
> >> pre-empt the lower priority thread, immediately attempt to obtain the GIL,
> >> and block allowing the lower-priority thread which holds the GIL to
> >> continue) until the GIL is released, at which point the highest priority
> >> thread will be scheduled.
> > 
> > Priority inversion happens if a lower priority thread is blocking a
> > higher priority thread ... IMHO.
> 
> The priority inversion you're thinking of is "bounded" priority inversion.
> The high priority thread

Or a medium one :) ... see above.

> is blocked waiting on the GIL, but it is known that
> the GIL will be released at some point in the near future. If you're
> attempting soft realtime in Python you have to accept this. There is no
> alternative.

I have no problem with it ... because QNX supports priority
inheritance.

[ clipp ..]
> > 
> > Ok, I will do some tests with the System Analyse Tool (SAT) of QNX6 PE
> > ... just to see what happens in detail when I use the different
> > scheduling strategies.
> 
> Yes, please do. I've already posted examples showing four different
> Operating Systems successfully pre-empting and scheduling CPU-bound Python
> threads - in most cases thousands of times a second.
> 
> I'm still waiting to see anyone demonstrate a real problem with Python
> threading.

I'm not digging for problems :)  IMHO, the proposal to implement a
sched_yield() isn't completely stupid.


Armin

> 
> Jonathan