[BangPypers] Thread and Queues problem.

[Amish Anand]

Hi
I am writing an application which basically solves a simple producer
consumer problem. Both producer and consumer run on separate threads. The
producer function writes to a Queue and continues producing data, whereas
the consumer thread function waits for items to be enqueued and once
present, it pops them out for processing. In the current implementation  I
am waiting on an infinite while loop because if I don't then the thread
function will exit after processing one item in the queue and that is not
desired. Due to this infinite loop, thread function is hogging lot of CPU
(as a workaround I have added a sleep of 1sec to reduce the CPU usage),
however I feel there should be a proper mechanism to gracefully wait for
the items to be enqueued and then continuously wait for the items to be
added to the queue without exiting. I will try to experiment with threading
module event (set and wait) but that will be a tricky implementation I
believe. Appreciate your thoughts on this issue.

Stub code of the thread handler on the consumer side:

    while True:
        print '%s: Looking for the next enclosure' % i
        url = q.get()
        print '%s: Downloading:' % i, url
        q.task_done()
        time.sleep(1)

Thanks,
Amish