On Wed, Mar 19, 2008 at 10:42 AM, Stefan Ring email@example.com wrote:
On Mar 19, 2008 05:24 PM, Adam Olsen firstname.lastname@example.org wrote:
On Wed, Mar 19, 2008 at 10:09 AM, Stefan Ring email@example.com wrote:
Adam Olsen <rhamph <at> gmail.com> writes:
Can you try with a call to sched_yield(), rather than nanosleep()? It should have the same benefit but without as much performance hit.
If it works, but is still too much hit, try tuning the checkinterval to see if you can find an acceptable throughput/responsiveness balance.
I tried that, and it had no effect whatsoever. I suppose it would make an effect on a single CPU or an otherwise heavily loaded SMP system but that's not the secnario we care about.
So you've got a lightly loaded SMP system? Multiple threads all blocked on the GIL, multiple CPUs to run them, but only one CPU is active? I that case I can imagine how sched_yield() might finish before the other CPUs wake up a thread.
A FIFO scheduler would be the right thing here, but it's only a short term solution. Care for a long term solution? ;)
I've already seen that but it would not help us in our current situation. The performance penalty really is too heavy. Our system is slow enough already ;). And it would be very difficult bordering on impossible to parallelize Plus, I can imagine that all extension modules (and our own code) would have to be adapted.
The FIFO scheduler is perfect for us because the load is typically quite low. It's mostly at those times when someone runs a lengthy calculation that all other users suffer greatly increased response times.
So you want responsiveness when idle but throughput when busy?
Are those calculations primarily python code, or does a C library do the grunt work? If it's a C library you shouldn't be affected by safethread's increased overhead.