Python threading (was: Re: global interpreter lock not working as it should)

[Martin v. Loewis]

Jonathan Hogg <jonathan at onegoodidea.com> writes:

> I'm sorry, but that's just plain rubbish. If all the threads have the same
> priority then the thread scheduler will schedule them according to
> timeslice. I'm really not sure where you get the idea that a thread can only
> be pre-empted by a higher priority thread. That's just not true.

I think it is true. The chance that the time slice expires within the
few instructions when the GIL is released is so low that you'ld
practically never see a thread switch - yet you do, on many systems.

The reason for that is that the operating system, in the SCHED_OTHER
scheduling class, dynamically adjusts priorities of threads - the
longer a thread runs, the lower its priority gets (down to a lower
boundary). The other Python threads block on the GIL, and keep their
priority. Then, when the thread releases the GIL, other threads become
ready, and have a higher priority - thus immediately causing a thread
switch (even though the time slice is not expired).

That, of course, assumes certain details about the OS thread
scheduler. The OP might be wrong that all operating systems follow
this strategy, and he is certainly wrong if he assumes that Python
relies on realtime-threading. But I do believe that such a strategy is
common across many systems.

> Yes, please do. I've already posted examples showing four different
> Operating Systems successfully pre-empting and scheduling CPU-bound Python
> threads - in most cases thousands of times a second.

But none of those were using realtime thread facilities, right? I do
believe that, in the SCHED_FIFO class, you would never observe a
thread switch in CPU-bound threads. That, of course, would also count
as "working properly": in SCHED_FIFO, you'd expect applications switch
threads only when the highest-priority thread blocks.

Regards,
Martin