Producer/consumer Queue "trick"

Jon Perez jbperez808 at wahoo.com
Sat Jan 15 04:12:35 EST 2005


I don't get it.

If the consumer and the producer are separate threads,
why does the consumer thread block when the producer
thread is generating a new board?  Or why does it
take forever for the producer thread to be pre-empted?

Also, I don't understand why the solution works.
How does sleeping for .001 seconds right after putting
a new board on the queue solve the problem?

Evan Simpson wrote:
> WEBoggle needs a new game board every three minutes.  Boards take an 
> unpredictable (much less than 3min, but non-trivial) amount of time to 
> generate. The system is driven by web requests, and I don't want the 
> request that happens to trigger the need for the new board to have to 
> pay the time cost of generating it.
> 
> I set up a producer thread that does nothing but generate boards and put 
> them into a length-two Queue (blocking).  At the rate that boards are 
> pulled from the Queue, it's almost always full, but my poor consumer 
> thread was still being blocked for "a long time" each time it fetched a 
> board.
> 
> At this point I realized that q.get() on a full Queue immediately wakes 
> up the producer, which has been blocked waiting to add a board to the 
> Queue.  It sets about generating the next board, and the consumer 
> doesn't get to run again until the producer blocks again or is preempted.
> 
> The solution was simple: have the producer time.sleep(0.001) when 
> q.put(board) returns.



More information about the Python-list mailing list