Simple thread pools

Steve Holden steve at
Mon Nov 8 13:52:30 CET 2004

Josiah Carlson wrote:

> Jacob Friis <lists at> wrote:
>>I have built a script inspired by a post on Speno's Pythonic Avocado:
>>I'm setting NUM_FEEDERS to 1000.
>>Is that crazy?
> Not crazy, but foolish.  Thread scheduling in Python reduces performance
> beyond a few dozen threads.  If you are doing system calls (socket.recv,
>, etc.), your performance will be poor.
Is this speculative, or do you have some hard evidence to support it? I 
recently rewrote a billing program that delivers statements by email. 
The number of threads it uses is a parameter to the program, and we are 
currently running at 200 with every evidence of satisfaction - this 
month's live run sent something over 10,000 emails an hour.
>>Are there a better solution?
> Fewer threads.  Try running at 10-30.  If you are finding that you
> aren't able to handle the load with those threads, then your
> processor/disk/etc isn't fast enough to handle the load.
I'm tempted to say "rubbish", but that would be rude, so instead I'll 
just ask for some evidence :-). Don't forget that in network-based tasks 
the time spent waiting for connection turnarounds can dominate the 
elapsed time for execution - did you perhaps overlook that?

Holden Web LLC +1 800 494 3119

More information about the Python-list mailing list