Simple question about Queue.Queue and threads

[Gabriel Genellina]

En Fri, 05 Feb 2010 09:45:30 -0300, Frank Millman <frank at chagford.com>  
escribió:

> Assume you have a server process running, a pool of worker threads to  
> perform tasks, and a Queue.Queue() to pass the tasks to the workers.
>
> In order to shut down the server cleanly, you want to ensure that the  
> workers have all finished their tasks. I like the technique of putting a  
> None onto the queue, and have each worker check for None, put None back  
> onto the queue, and terminate itself.
>
> The main program would look something like this -
>
>     q.put(None)
>     for worker in worker_threads:
>         worker.join()
>
> At this point you can be sure that each thread has completed its tasks  
> and terminated itself.
>
> However, the queue is not empty - it still has the final None in it.

Yes - but who cares? :)

> Is it advisable to finalise the cleanup like this? -
>
>     while not q.empty():
>         q.get()
>         q.task_done()
>     q.join()
>
> Or is this completely redundant?

I don't see what you gain by doing that.

On the other hand, you might check whether the queue contains exactly one  
item and it is None, just to be sure that all queued work has been done.

(BTW, I'd use a different sentinel instead of None; perhaps this could not  
happen in your current code, but it's easy to inadvertedly put a None in  
the queue, stopping all workers prematurely.)

-- 
Gabriel Genellina