[Python-checkins] bpo-42183: Fix a stack overflow error for asyncio Task or Future repr() (GH-23020)

miss-islington webhook-mailer at python.org
Tue Nov 10 09:20:56 EST 2020


https://github.com/python/cpython/commit/90115a2cf7033c990a54d1ecb90ebd850b5f13cf
commit: 90115a2cf7033c990a54d1ecb90ebd850b5f13cf
branch: 3.9
author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com>
committer: miss-islington <31488909+miss-islington at users.noreply.github.com>
date: 2020-11-10T06:20:52-08:00
summary:

bpo-42183: Fix a stack overflow error for asyncio Task or Future repr() (GH-23020)


The overflow occurs under some circumstances when a task or future
recursively returns itself.

Co-authored-by: Kyle Stanley <aeros167 at gmail.com>
(cherry picked from commit 42d873c63aa9d160c132be4a34599531574db12c)

Co-authored-by: Andrew Svetlov <andrew.svetlov at gmail.com>

files:
A Lib/test/test_asyncio/test_futures2.py
A Misc/NEWS.d/next/Library/2020-10-29-11-17-35.bpo-42183.50ZcIi.rst
M Lib/asyncio/base_futures.py

diff --git a/Lib/asyncio/base_futures.py b/Lib/asyncio/base_futures.py
index 22f298069c505..2c01ac98e10fc 100644
--- a/Lib/asyncio/base_futures.py
+++ b/Lib/asyncio/base_futures.py
@@ -1,6 +1,7 @@
 __all__ = ()
 
 import reprlib
+from _thread import get_ident
 
 from . import format_helpers
 
@@ -41,6 +42,16 @@ def format_cb(callback):
     return f'cb=[{cb}]'
 
 
+# bpo-42183: _repr_running is needed for repr protection
+# when a Future or Task result contains itself directly or indirectly.
+# The logic is borrowed from @reprlib.recursive_repr decorator.
+# Unfortunately, the direct decorator usage is impossible because of
+# AttributeError: '_asyncio.Task' object has no attribute '__module__' error.
+#
+# After fixing this thing we can return to the decorator based approach.
+_repr_running = set()
+
+
 def _future_repr_info(future):
     # (Future) -> str
     """helper function for Future.__repr__"""
@@ -49,9 +60,17 @@ def _future_repr_info(future):
         if future._exception is not None:
             info.append(f'exception={future._exception!r}')
         else:
-            # use reprlib to limit the length of the output, especially
-            # for very long strings
-            result = reprlib.repr(future._result)
+            key = id(future), get_ident()
+            if key in _repr_running:
+                result = '...'
+            else:
+                _repr_running.add(key)
+                try:
+                    # use reprlib to limit the length of the output, especially
+                    # for very long strings
+                    result = reprlib.repr(future._result)
+                finally:
+                    _repr_running.discard(key)
             info.append(f'result={result}')
     if future._callbacks:
         info.append(_format_callbacks(future._callbacks))
diff --git a/Lib/test/test_asyncio/test_futures2.py b/Lib/test/test_asyncio/test_futures2.py
new file mode 100644
index 0000000000000..13dbc703277c8
--- /dev/null
+++ b/Lib/test/test_asyncio/test_futures2.py
@@ -0,0 +1,18 @@
+# IsolatedAsyncioTestCase based tests
+import asyncio
+import unittest
+
+
+class FutureTests(unittest.IsolatedAsyncioTestCase):
+    async def test_recursive_repr_for_pending_tasks(self):
+        # The call crashes if the guard for recursive call
+        # in base_futures:_future_repr_info is absent
+        # See Also: https://bugs.python.org/issue42183
+
+        async def func():
+            return asyncio.all_tasks()
+
+        # The repr() call should not raise RecursiveError at first.
+        # The check for returned string is not very reliable but
+        # exact comparison for the whole string is even weaker.
+        self.assertIn('...', repr(await asyncio.wait_for(func(), timeout=10)))
diff --git a/Misc/NEWS.d/next/Library/2020-10-29-11-17-35.bpo-42183.50ZcIi.rst b/Misc/NEWS.d/next/Library/2020-10-29-11-17-35.bpo-42183.50ZcIi.rst
new file mode 100644
index 0000000000000..f6d7653f2cf09
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2020-10-29-11-17-35.bpo-42183.50ZcIi.rst
@@ -0,0 +1,4 @@
+Fix a stack overflow error for asyncio Task or Future repr().
+
+The overflow occurs under some circumstances when a Task or Future
+recursively returns itself.



More information about the Python-checkins mailing list