[Python-Dev] Improved thread switching
rhamph at gmail.com
Wed Mar 19 18:03:02 CET 2008
On Wed, Mar 19, 2008 at 10:42 AM, Stefan Ring <s.r at visotech.at> wrote:
> On Mar 19, 2008 05:24 PM, Adam Olsen <rhamph at gmail.com> wrote:
> > On Wed, Mar 19, 2008 at 10:09 AM, Stefan Ring <s.r at visotech.at> wrote:
> > > Adam Olsen <rhamph <at> gmail.com> writes:
> > >
> > > > Can you try with a call to sched_yield(), rather than nanosleep()?
> > > > It
> > > > should have the same benefit but without as much performance hit.
> > > >
> > > > If it works, but is still too much hit, try tuning the
> > > > checkinterval
> > > > to see if you can find an acceptable throughput/responsiveness
> > > > balance.
> > > >
> > >
> > > I tried that, and it had no effect whatsoever. I suppose it would
> > > make an effect
> > > on a single CPU or an otherwise heavily loaded SMP system but that's
> > > not the
> > > secnario we care about.
> > So you've got a lightly loaded SMP system? Multiple threads all
> > blocked on the GIL, multiple CPUs to run them, but only one CPU is
> > active? I that case I can imagine how sched_yield() might finish
> > before the other CPUs wake up a thread.
> > A FIFO scheduler would be the right thing here, but it's only a short
> > term solution. Care for a long term solution? ;)
> > http://code.google.com/p/python-safethread/
> I've already seen that but it would not help us in our current
> situation. The performance penalty really is too heavy. Our system is
> slow enough already ;). And it would be very difficult bordering on
> impossible to parallelize Plus, I can imagine that all extension modules
> (and our own code) would have to be adapted.
> The FIFO scheduler is perfect for us because the load is typically quite
> low. It's mostly at those times when someone runs a lengthy calculation
> that all other users suffer greatly increased response times.
So you want responsiveness when idle but throughput when busy?
Are those calculations primarily python code, or does a C library do
the grunt work? If it's a C library you shouldn't be affected by
safethread's increased overhead.
Adam Olsen, aka Rhamphoryncus
More information about the Python-Dev