Python threading (was: Re: global interpreter lock not working as it should)
anton.wilson at camotion.com
Tue Aug 6 01:51:23 CEST 2002
On Saturday 03 August 2002 04:56 am, Jonathan Hogg wrote:
> On 3/8/2002 2:43, in article aifcfh$jha$0 at 18.104.22.168, "Bengt Richter"
> <bokr at oz.net> wrote:
> >> This is true. Therefore, the only time another thread WILL grab the GIL
> >> under Linux is if
> >> 1) the GIL is released by the currently running thread
> >> 2) the thread that just released the GIL depletes its timeslice before
> >> it can grab the lock again
> >> 3) the OS notices the process has depleted it's timeslice and the yanks
> >> it from the CPU (this happens every 100 times per second by default on
> >> an i386)
> >> 4) the waiting thread that recieved the GIL release signal is chosen to
> >> run
> >> Therefore, we now have a large set of coincidences for CPU-bound python
> >> threads. The only reason it works at all is because it happens 100 times
> >> per second and the GIL is released frequently by default. So there is a
> >> sufficient probability that these 4 cases will happen simultaneously.
> > Hm. If that's an accurate description (and I am skeptical ;-), I don't
> > think I like it.
> > Seems like if another thread is waiting for the GIL, then after a full
> > time slice the
> > current GIL-holder ought to be told to give it up ASAP and be given a new
> > time slice
> > only to do the minimum it needs to do that.
> If this is true, and I'm also skeptical, then it's a bug in Linux not
> Python and Python should do nothing about it.
> I ran a little experiment (code enclosed below) to crudely check the thread
> switching on my machine (Mac OS X - well known for it's threading). Running
> 10 CPU-bound threads together for 10 seconds came up with them all getting
> fairly even CPU time (within +/- 10% of the mean).
> My crude count for the actual number of thread switches that occurred
> during those 10 seconds showed never less than 1200 thread switches per
> second and up to 3000 thread switches per second.
> I also tried the same code on a FreeBSD, Solaris, and Linux box. The
> summary results were:
> OS Architecture Python Thread Switches (avg 2 runs)
> Mac OS X 10.1 PPC 2.3-CVS 24971
> FreeBSD 4.3 i386 2.2.1 158499
> Linux 2.2.14 i386 2.2 18281
> Solaris 8 SPARC 2.2.1 243
> So an interesting spread with Solaris being way down there in terms of how
> often it switched threads (though the CPU distribution seemed to be pretty
> fair anyway) - I'm not sure what threading library Python was compiled
> against on the Solaris box (which has two - one OS and one user), I might
> look into that out of curiosity. Anyone know FreeBSD well enough to comment
> on the results for that platform?
> Note that Linux is still doing over 1800 switches per second, which makes
> me even more skeptical that it is only capable of thread switching due to a
> huge coincidence.
Well, when I said coincidence, I didn't necessarily mean huge, it works, but
when compared to the number of times it tries to give up the interpreter . .
The timer tick does happen 100x per second and since python gives up the lock
every 10 us or so, it's really not a huge coincidence for an overlap. I just
don't really like the reliance on this coincidence.
> Any comments?
> ----- threads.py -----
> import threading
> import time
> import operator
> def sum( xs ):
> return reduce( operator.add, xs )
> QUITTING = 0
> RUNNING = None
> class Counter( threading.Thread ):
> def run( self ):
> global QUITTING, RUNNING
> count = 0
> switches = 0
> while not QUITTING:
> count += 1
> if self is not RUNNING:
> RUNNING = self
> switches += 1
> self.count = count
> self.switches = switches
> counters = [ Counter() for i in range(10) ]
> for counter in counters:
> time.sleep( 10.0 )
> QUITTING = 1
> for counter in counters:
> counts = [ counter.count for counter in counters ]
> switches = [ counter.switches for counter in counters ]
> print "Counts:"
> print counts
> print "Total =", sum( counts )
> print "\nSwitches:"
> print switches
> print "Total =", sum( switches )
More information about the Python-list