Producer/consumer Queue "trick"
Jon Perez
jbperez808 at wahoo.com
Sat Jan 15 04:12:35 EST 2005
I don't get it.
If the consumer and the producer are separate threads,
why does the consumer thread block when the producer
thread is generating a new board? Or why does it
take forever for the producer thread to be pre-empted?
Also, I don't understand why the solution works.
How does sleeping for .001 seconds right after putting
a new board on the queue solve the problem?
Evan Simpson wrote:
> WEBoggle needs a new game board every three minutes. Boards take an
> unpredictable (much less than 3min, but non-trivial) amount of time to
> generate. The system is driven by web requests, and I don't want the
> request that happens to trigger the need for the new board to have to
> pay the time cost of generating it.
>
> I set up a producer thread that does nothing but generate boards and put
> them into a length-two Queue (blocking). At the rate that boards are
> pulled from the Queue, it's almost always full, but my poor consumer
> thread was still being blocked for "a long time" each time it fetched a
> board.
>
> At this point I realized that q.get() on a full Queue immediately wakes
> up the producer, which has been blocked waiting to add a board to the
> Queue. It sets about generating the next board, and the consumer
> doesn't get to run again until the producer blocks again or is preempted.
>
> The solution was simple: have the producer time.sleep(0.001) when
> q.put(board) returns.
More information about the Python-list
mailing list