use of Queue

[Alexandru Mosoi]

On Aug 27, 1:06 pm, Gerhard Häring <g... at ghaering.de> wrote:
> Alexandru Mosoi wrote:
> > how is Queue intended to be used? I found the following code in python
> > manual, but I don't understand how to stop consumers after all items
> > have been produced. I tried different approaches but all of them
> > seemed incorrect (race, deadlock or duplicating queue functionality)
>
> >     def worker():
> >         while True:
> >             item = q.get()
>
>                if item is None:
>                    break
>
> >             do_work(item)
> >             q.task_done()
>
> >     q = Queue()
> >     for i in range(num_worker_threads):
> >          t = Thread(target=worker)
> >          t.setDaemon(True)
> >          t.start()
>
> >     for item in source():
> >         q.put(item)
>
> # stop all consumers
> for i in range(num_worker_threads):
>      q.put(None)
>
>
>
> >     q.join()       # block until all tasks are done
>
> This is how I do it.
>
> -- Gerhard


Your solution works assuming that you know how many consumer threads
you have :). I don't :). More than that, it's not correct if you have
more than one producer :). Having a sentinel was my very first idea,
but as you see... it's a race condition (there are cases in which not
all items are processed).