multiprocessing timing issue

[Philip Semanchuk]

On Aug 9, 2011, at 1:07 PM, Tim Arnold wrote:

> Hi, I'm having problems with an empty Queue using multiprocessing.
> 
> The task:
> I have a bunch of chapters that I want to gather data on individually and then update a report database with the results.
> I'm using multiprocessing to do the data-gathering simultaneously.
> 
> Each chapter report gets put on a Queue in their separate processes. Then each report gets picked off the queue and the report database is updated with the results.
> 
> My problem is that sometimes the Queue is empty and I guess it's
> because the get_data() method takes a lot of time.
> 
> I've used multiprocessing before, but never with a Queue like this.
> Any notes or suggestions are very welcome.


Hi Tim,
THis might be a dumb question, but...why is it a problem if the queue is empty? It sounds like you figured out already that get_data() sometimes takes longer than your timeout. So either increase your timeout or learn to live with the fact that the queue is sometimes empty. I don't mean to be rude, I just don't understand the problem. 

Cheers
Philip