On Wednesday 27 June 2007, Markus Schiltknecht wrote:
Now, I'm wondering if there's a profiling tool, which can quickly tell me what events are blocking my server. I have even deferred the file I/O to separate threads, as I've suspected them to make the server behave poorly when under load. But there still seems to be something blocking... What do you use to find the callbacks which block (for too long)?
Does your server process use a lot of memory? I had problems with a random simple request taking a long time every now and then and it turned out to be the Python garbage collector. The collection itself can take some time if you have a lot of objects. If some of the Python process ends up in the swap instead of main memory, garbage collection will block until it is swapped back in, which can slow it down dramatically. Using multiple threads does not give you any performance advantage in this case: during garbage collection the entire Python VM is frozen. One thing you could try is: import gc gc.disable() in the startup code of your server. Note that this does not turn off all memory recycling; it only turns off the mark-and-sweep part of the garbage collector. Objects which have a zero reference count are still collected. Bye, Maarten