Producer/consumer Queue "trick"
evan at tokenexchange.com
Fri Jan 14 17:26:02 EST 2005
WEBoggle needs a new game board every three minutes. Boards take an
unpredictable (much less than 3min, but non-trivial) amount of time to
generate. The system is driven by web requests, and I don't want the
request that happens to trigger the need for the new board to have to
pay the time cost of generating it.
I set up a producer thread that does nothing but generate boards and put
them into a length-two Queue (blocking). At the rate that boards are
pulled from the Queue, it's almost always full, but my poor consumer
thread was still being blocked for "a long time" each time it fetched a
At this point I realized that q.get() on a full Queue immediately wakes
up the producer, which has been blocked waiting to add a board to the
Queue. It sets about generating the next board, and the consumer
doesn't get to run again until the producer blocks again or is preempted.
The solution was simple: have the producer time.sleep(0.001) when
Evan @ 4-am
More information about the Python-list