[issue12155] queue example doesn't stop worker threads
New submission from STINNER Victor <victor.stinner@haypocalc.com>: The queue doc contains the following example: ------------------ def worker(): while True: item = q.get() do_work(item) q.task_done() q = Queue() for i in range(num_worker_threads): t = Thread(target=worker) t.daemon = True t.start() for item in source(): q.put(item) q.join() # block until all tasks are done ------------------ http://docs.python.org/library/queue.html It doesn't define do_work(), num_worker_threads or do_work(), but my concern is that it doesn't stop worker threads. I consider "t.daemon = True" as an hack to not care about stopping threads. The example should pass a special value to each worker to stop it. For example: <worker> while True: job = queue.get() if job is None: break audio.play(*job) queue.task_done() Main thread: ... threads = [] for i in range(num_worker_threads): t = Thread(target=worker) threads.append(t) t.start() ... for i in range(num_worker_threads): queue.put(None) queue.join() for thread in threads: thread.join() ---------- assignee: docs@python components: Documentation messages: 136601 nosy: docs@python, haypo, pitrou priority: normal severity: normal status: open title: queue example doesn't stop worker threads versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12155> _______________________________________
Changes by Antoine Pitrou <pitrou@free.fr>: ---------- nosy: +rhettinger _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12155> _______________________________________
Changes by Raymond Hettinger <raymond.hettinger@gmail.com>: ---------- assignee: docs@python -> rhettinger _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12155> _______________________________________
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:
It doesn't define do_work(), num_worker_threads or do_work()
Is it unclear to you what those mean? They are placeholders for the user's actual task at hand.
my concern is that it doesn't stop worker threads.
Stopping the threads wasn't the intent of the example.
I consider "t.daemon = True" as an hack to not care about stopping threads.
If you post a high-quality self-contained example somewhere on the net, I would be happy to link to it. ---------- priority: normal -> low _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12155> _______________________________________
STINNER Victor <victor.stinner@haypocalc.com> added the comment:
Is it unclear to you what those mean?
Well, it's clear, but I like when I can simply copy/paste the example and it does just work:
If you post a high-quality self-contained example somewhere on the net, I would be happy to link to it.
I just propose to change the example to stop the threads: ------------------ def worker(): while True: item = q.get() if item is None: break do_work(item) q.task_done() q = Queue() threads = [] for i in range(num_worker_threads): t = Thread(target=worker) threads.append(t) t.start() for item in source(): q.put(item) q.join() # block until all tasks are done for i in range(num_worker_threads): q.put(None) for t in threads: t.join() ------------------ ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12155> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: The proposed change adds about 7 lines to show the 'trick' of putting num-worker Nones on the queue. I think that is worth it. ---------- nosy: +terry.reedy _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12155> _______________________________________
Roundup Robot added the comment: New changeset b44ec269abda by Victor Stinner in branch 'default': Issue #12155: Fix queue doc example to join threads https://hg.python.org/cpython/rev/b44ec269abda ---------- nosy: +python-dev _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12155> _______________________________________
Changes by STINNER Victor <victor.stinner@gmail.com>: ---------- resolution: -> fixed status: open -> closed versions: +Python 3.5 -Python 2.7, Python 3.1, Python 3.2, Python 3.3 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12155> _______________________________________
participants (5)
-
Antoine Pitrou -
Raymond Hettinger
-
Roundup Robot
-
STINNER Victor
-
Terry J. Reedy