[Python-checkins] bpo-36813: Fix QueueListener to call task_done() upon termination. (GH-13113)

[Miss Islington (bot)]

https://github.com/python/cpython/commit/6b282e18877ec84e927b381b4ce187eaf4ba3dd7
commit: 6b282e18877ec84e927b381b4ce187eaf4ba3dd7
branch: master
author: Bar Harel <bzvi7919 at gmail.com>
committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com>
date: 2019-06-01T02:19:09-07:00
summary:

bpo-36813: Fix QueueListener to call task_done() upon termination. (GH-13113)



Fixed QueueListener in order to avoid random deadlocks.
Unable to add regression tests atm due to time constraints, will add it in a bit.
Regarding implementation, although it's nested, it does not cause performance issues whatsoever, and does not call task_done() in case of an exception (which is the right thing to do IMHO).


https://bugs.python.org/issue36813

files:
A Misc/NEWS.d/next/Library/2019-05-06-18-28-38.bpo-36813.NXD0KZ.rst
M Lib/logging/handlers.py
M Lib/test/test_logging.py

diff --git a/Lib/logging/handlers.py b/Lib/logging/handlers.py
index 3727bf0677cd..a913d27389ab 100644
--- a/Lib/logging/handlers.py
+++ b/Lib/logging/handlers.py
@@ -1477,6 +1477,8 @@ def _monitor(self):
             try:
                 record = self.dequeue(True)
                 if record is self._sentinel:
+                    if has_task_done:
+                        q.task_done()
                     break
                 self.handle(record)
                 if has_task_done:
diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py
index b884753ad397..50148dc2f252 100644
--- a/Lib/test/test_logging.py
+++ b/Lib/test/test_logging.py
@@ -3633,6 +3633,16 @@ def test_no_messages_in_queue_after_stop(self):
                                     [m.msg if isinstance(m, logging.LogRecord)
                                      else m for m in items]))
 
+        def test_calls_task_done_after_stop(self):
+            # Issue 36813: Make sure queue.join does not deadlock.
+            log_queue = queue.Queue()
+            listener = logging.handlers.QueueListener(log_queue)
+            listener.start()
+            listener.stop()
+            with self.assertRaises(ValueError):
+                # Make sure all tasks are done and .join won't block.
+                log_queue.task_done()
+
 
 ZERO = datetime.timedelta(0)
 
diff --git a/Misc/NEWS.d/next/Library/2019-05-06-18-28-38.bpo-36813.NXD0KZ.rst b/Misc/NEWS.d/next/Library/2019-05-06-18-28-38.bpo-36813.NXD0KZ.rst
new file mode 100644
index 000000000000..e89358aa4051
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2019-05-06-18-28-38.bpo-36813.NXD0KZ.rst
@@ -0,0 +1,2 @@
+Fix :class:`~logging.handlers.QueueListener` to call ``queue.task_done()``
+upon stopping. Patch by Bar Harel.