[Python-Dev] Improved thread switching

[Stefan Ring]

Adam Olsen <rhamph <at> gmail.com> writes:

> 
> On Wed, Mar 19, 2008 at 10:09 AM, Stefan Ring <s.r <at> visotech.at> wrote:
> > Adam Olsen <rhamph <at> gmail.com> writes:
> >
> >  > Can you try with a call to sched_yield(), rather than nanosleep()?  It
> >  > should have the same benefit but without as much performance hit.
> >  >
> >  > If it works, but is still too much hit, try tuning the checkinterval
> >  > to see if you can find an acceptable throughput/responsiveness
> >  > balance.
> >  >
> >
> >  I tried that, and it had no effect whatsoever. I suppose it would make an
effect
> >  on a single CPU or an otherwise heavily loaded SMP system but that's not the
> >  secnario we care about.
> 
> So you've got a lightly loaded SMP system?  Multiple threads all
> blocked on the GIL, multiple CPUs to run them, but only one CPU is
> active?  I that case I can imagine how sched_yield() might finish
> before the other CPUs wake up a thread.
> 
> A FIFO scheduler would be the right thing here, but it's only a short
> term solution.  Care for a long term solution? ;)
> 
> http://code.google.com/p/python-safethread/
> 


I've already seen that but it would not help us in our current
situation. The performance penalty really is too heavy. Our system is
slow enough already ;). And it would be very difficult bordering on
impossible to parallelize Plus, I can imagine that all extension modules
(and our own code) would have to be adapted.

The FIFO scheduler is perfect for us because the load is typically quite
low. It's mostly at those times when someone runs a lengthy calculation
that all other users suffer greatly increased response times.