Python-checkins
Threads by month
- ----- 2024 -----
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2008 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2007 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2006 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2005 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2004 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2003 -----
- December
- November
- October
- September
- August
April 2021
- 1 participants
- 580 discussions
bpo-43534: Make dialogs in turtle.textinput() and turtle.numinput() transitient again (GH-24923)
by miss-islington 25 Apr '21
by miss-islington 25 Apr '21
25 Apr '21
https://github.com/python/cpython/commit/7248ce30bb84f2d5af855a867cc4e3cc40…
commit: 7248ce30bb84f2d5af855a867cc4e3cc40a07fb5
branch: 3.9
author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com>
committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com>
date: 2021-04-25T03:44:30-07:00
summary:
bpo-43534: Make dialogs in turtle.textinput() and turtle.numinput() transitient again (GH-24923)
(cherry picked from commit b5adc8a7e5c13d175b4d3e53b37bc61de35b1457)
Co-authored-by: Serhiy Storchaka <storchaka(a)gmail.com>
files:
A Misc/NEWS.d/next/Library/2021-03-18-15-46-08.bpo-43534.vPE9Us.rst
M Lib/turtle.py
diff --git a/Lib/turtle.py b/Lib/turtle.py
index ba8288daf83f2..13f662a8c2108 100644
--- a/Lib/turtle.py
+++ b/Lib/turtle.py
@@ -826,7 +826,7 @@ def textinput(self, title, prompt):
>>> screen.textinput("NIM", "Name of first player:")
"""
- return simpledialog.askstring(title, prompt)
+ return simpledialog.askstring(title, prompt, parent=self.cv)
def numinput(self, title, prompt, default=None, minval=None, maxval=None):
"""Pop up a dialog window for input of a number.
@@ -847,7 +847,8 @@ def numinput(self, title, prompt, default=None, minval=None, maxval=None):
"""
return simpledialog.askfloat(title, prompt, initialvalue=default,
- minvalue=minval, maxvalue=maxval)
+ minvalue=minval, maxvalue=maxval,
+ parent=self.cv)
##############################################################################
diff --git a/Misc/NEWS.d/next/Library/2021-03-18-15-46-08.bpo-43534.vPE9Us.rst b/Misc/NEWS.d/next/Library/2021-03-18-15-46-08.bpo-43534.vPE9Us.rst
new file mode 100644
index 0000000000000..7f2e5a46add03
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2021-03-18-15-46-08.bpo-43534.vPE9Us.rst
@@ -0,0 +1,2 @@
+:func:`turtle.textinput` and :func:`turtle.numinput` create now a transient
+window working on behalf of the canvas window.
1
0
bpo-39529: Deprecate creating new event loop in asyncio.get_event_loop() (GH-23554)
by serhiy-storchaka 25 Apr '21
by serhiy-storchaka 25 Apr '21
25 Apr '21
https://github.com/python/cpython/commit/172c0f2752d8708b6dda7b42e6c5a35194…
commit: 172c0f2752d8708b6dda7b42e6c5a3519420a4e8
branch: master
author: Serhiy Storchaka <storchaka(a)gmail.com>
committer: serhiy-storchaka <storchaka(a)gmail.com>
date: 2021-04-25T13:40:44+03:00
summary:
bpo-39529: Deprecate creating new event loop in asyncio.get_event_loop() (GH-23554)
asyncio.get_event_loop() emits now a deprecation warning when it creates a new event loop.
In future releases it will became an alias of asyncio.get_running_loop().
files:
A Misc/NEWS.d/next/Library/2020-12-06-20-21-16.bpo-39529.9Zrg43.rst
M Doc/library/asyncio-eventloop.rst
M Doc/library/asyncio-future.rst
M Doc/library/asyncio-task.rst
M Doc/whatsnew/3.10.rst
M Lib/asyncio/events.py
M Lib/asyncio/futures.py
M Lib/asyncio/streams.py
M Lib/asyncio/tasks.py
M Lib/test/test_asyncio/test_events.py
M Lib/test/test_asyncio/test_futures.py
M Lib/test/test_asyncio/test_queues.py
M Lib/test/test_asyncio/test_streams.py
M Lib/test/test_asyncio/test_tasks.py
M Modules/_asynciomodule.c
M Modules/clinic/_asynciomodule.c.h
diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst
index 7de5a0ab2595c..ca91efec260db 100644
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -53,6 +53,11 @@ an event loop:
Consider also using the :func:`asyncio.run` function instead of using
lower level functions to manually create and close an event loop.
+ .. deprecated:: 3.10
+ Deprecation warning is emitted if there is no running event loop.
+ If future Python releases this function will be an alias of
+ :func:`get_running_loop`.
+
.. function:: set_event_loop(loop)
Set *loop* as a current event loop for the current OS thread.
diff --git a/Doc/library/asyncio-future.rst b/Doc/library/asyncio-future.rst
index 939d4c1a84523..ef496a23f5cd4 100644
--- a/Doc/library/asyncio-future.rst
+++ b/Doc/library/asyncio-future.rst
@@ -57,12 +57,20 @@ Future Functions
.. versionchanged:: 3.5.1
The function accepts any :term:`awaitable` object.
+ .. deprecated:: 3.10
+ Deprecation warning is emitted if *obj* is not a Future-like object
+ and *loop* is not specified and there is no running event loop.
+
.. function:: wrap_future(future, *, loop=None)
Wrap a :class:`concurrent.futures.Future` object in a
:class:`asyncio.Future` object.
+ .. deprecated:: 3.10
+ Deprecation warning is emitted if *future* is not a Future-like object
+ and *loop* is not specified and there is no running event loop.
+
Future Object
=============
@@ -90,6 +98,10 @@ Future Object
.. versionchanged:: 3.7
Added support for the :mod:`contextvars` module.
+ .. deprecated:: 3.10
+ Deprecation warning is emitted if *loop* is not specified
+ and there is no running event loop.
+
.. method:: result()
Return the result of the Future.
diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst
index 1ca1b4a06d7e8..3f54ecb08efc1 100644
--- a/Doc/library/asyncio-task.rst
+++ b/Doc/library/asyncio-task.rst
@@ -397,6 +397,11 @@ Running Tasks Concurrently
If the *gather* itself is cancelled, the cancellation is
propagated regardless of *return_exceptions*.
+ .. deprecated:: 3.10
+ Deprecation warning is emitted if no positional arguments are provided
+ or not all positional arguments are Future-like objects
+ and there is no running event loop.
+
Shielding From Cancellation
===========================
@@ -434,6 +439,10 @@ Shielding From Cancellation
except CancelledError:
res = None
+ .. deprecated:: 3.10
+ Deprecation warning is emitted if *aw* is not Future-like object
+ and there is no running event loop.
+
Timeouts
========
@@ -593,6 +602,10 @@ Waiting Primitives
earliest_result = await coro
# ...
+ .. deprecated:: 3.10
+ Deprecation warning is emitted if not all awaitable objects in the *aws*
+ iterable are Future-like objects and there is no running event loop.
+
Running in Threads
==================
@@ -775,6 +788,10 @@ Task Object
.. deprecated-removed:: 3.8 3.10
The *loop* parameter.
+ .. deprecated:: 3.10
+ Deprecation warning is emitted if *loop* is not specified
+ and there is no running event loop.
+
.. method:: cancel(msg=None)
Request the Task to be cancelled.
diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst
index dac44cf03fa32..ab0b46f9f2da4 100644
--- a/Doc/whatsnew/3.10.rst
+++ b/Doc/whatsnew/3.10.rst
@@ -1349,6 +1349,19 @@ Deprecated
scheduled for removal in Python 3.12.
(Contributed by Erlend E. Aasland in :issue:`42264`.)
+* :func:`asyncio.get_event_loop` emits now a deprecation warning if there is
+ no running event loop. In future it will be an alias of
+ :func:`~asyncio.get_running_loop`.
+ :mod:`asyncio` functions which implicitly create a :class:`~asyncio.Future`
+ or :class:`~asyncio.Task` objects emit now
+ a deprecation warning if there is no running event loop and no explicit
+ *loop* argument is passed: :func:`~asyncio.ensure_future`,
+ :func:`~asyncio.wrap_future`, :func:`~asyncio.gather`,
+ :func:`~asyncio.shield`, :func:`~asyncio.as_completed` and constructors of
+ :class:`~asyncio.Future`, :class:`~asyncio.Task`,
+ :class:`~asyncio.StreamReader`, :class:`~asyncio.StreamReaderProtocol`.
+ (Contributed by Serhiy Storchaka in :issue:`39529`.)
+
* The undocumented built-in function ``sqlite3.enable_shared_cache`` is now
deprecated, scheduled for removal in Python 3.12. Its use is strongly
discouraged by the SQLite3 documentation. See `the SQLite3 docs
diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py
index 1a20f362ec386..b966ad26bf467 100644
--- a/Lib/asyncio/events.py
+++ b/Lib/asyncio/events.py
@@ -759,9 +759,16 @@ def get_event_loop():
the result of `get_event_loop_policy().get_event_loop()` call.
"""
# NOTE: this function is implemented in C (see _asynciomodule.c)
+ return _py__get_event_loop()
+
+
+def _get_event_loop(stacklevel=3):
current_loop = _get_running_loop()
if current_loop is not None:
return current_loop
+ import warnings
+ warnings.warn('There is no current event loop',
+ DeprecationWarning, stacklevel=stacklevel)
return get_event_loop_policy().get_event_loop()
@@ -791,6 +798,7 @@ def set_child_watcher(watcher):
_py__set_running_loop = _set_running_loop
_py_get_running_loop = get_running_loop
_py_get_event_loop = get_event_loop
+_py__get_event_loop = _get_event_loop
try:
@@ -798,7 +806,7 @@ def set_child_watcher(watcher):
# functions in asyncio. Pure Python implementation is
# about 4 times slower than C-accelerated.
from _asyncio import (_get_running_loop, _set_running_loop,
- get_running_loop, get_event_loop)
+ get_running_loop, get_event_loop, _get_event_loop)
except ImportError:
pass
else:
@@ -807,3 +815,4 @@ def set_child_watcher(watcher):
_c__set_running_loop = _set_running_loop
_c_get_running_loop = get_running_loop
_c_get_event_loop = get_event_loop
+ _c__get_event_loop = _get_event_loop
diff --git a/Lib/asyncio/futures.py b/Lib/asyncio/futures.py
index 2d22ef66c9c42..10f8f0554e4b6 100644
--- a/Lib/asyncio/futures.py
+++ b/Lib/asyncio/futures.py
@@ -76,7 +76,7 @@ def __init__(self, *, loop=None):
the default event loop.
"""
if loop is None:
- self._loop = events.get_event_loop()
+ self._loop = events._get_event_loop()
else:
self._loop = loop
self._callbacks = []
@@ -408,7 +408,7 @@ def wrap_future(future, *, loop=None):
assert isinstance(future, concurrent.futures.Future), \
f'concurrent.futures.Future is expected, got {future!r}'
if loop is None:
- loop = events.get_event_loop()
+ loop = events._get_event_loop()
new_future = loop.create_future()
_chain_future(future, new_future)
return new_future
diff --git a/Lib/asyncio/streams.py b/Lib/asyncio/streams.py
index 96a9f97200d0d..080d8a62cde1e 100644
--- a/Lib/asyncio/streams.py
+++ b/Lib/asyncio/streams.py
@@ -125,7 +125,7 @@ class FlowControlMixin(protocols.Protocol):
def __init__(self, loop=None):
if loop is None:
- self._loop = events.get_event_loop()
+ self._loop = events._get_event_loop(stacklevel=4)
else:
self._loop = loop
self._paused = False
@@ -283,9 +283,13 @@ def _get_close_waiter(self, stream):
def __del__(self):
# Prevent reports about unhandled exceptions.
# Better than self._closed._log_traceback = False hack
- closed = self._closed
- if closed.done() and not closed.cancelled():
- closed.exception()
+ try:
+ closed = self._closed
+ except AttributeError:
+ pass # failed constructor
+ else:
+ if closed.done() and not closed.cancelled():
+ closed.exception()
class StreamWriter:
@@ -381,7 +385,7 @@ def __init__(self, limit=_DEFAULT_LIMIT, loop=None):
self._limit = limit
if loop is None:
- self._loop = events.get_event_loop()
+ self._loop = events._get_event_loop()
else:
self._loop = loop
self._buffer = bytearray()
diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py
index 52f1e6629e2fc..9a9d0d6e3cc26 100644
--- a/Lib/asyncio/tasks.py
+++ b/Lib/asyncio/tasks.py
@@ -549,7 +549,7 @@ def as_completed(fs, *, timeout=None):
from .queues import Queue # Import here to avoid circular import problem.
done = Queue()
- loop = events.get_event_loop()
+ loop = events._get_event_loop()
todo = {ensure_future(f, loop=loop) for f in set(fs)}
timeout_handle = None
@@ -616,23 +616,26 @@ def ensure_future(coro_or_future, *, loop=None):
If the argument is a Future, it is returned directly.
"""
- if coroutines.iscoroutine(coro_or_future):
- if loop is None:
- loop = events.get_event_loop()
- task = loop.create_task(coro_or_future)
- if task._source_traceback:
- del task._source_traceback[-1]
- return task
- elif futures.isfuture(coro_or_future):
+ return _ensure_future(coro_or_future, loop=loop)
+
+
+def _ensure_future(coro_or_future, *, loop=None):
+ if futures.isfuture(coro_or_future):
if loop is not None and loop is not futures._get_loop(coro_or_future):
raise ValueError('The future belongs to a different loop than '
- 'the one specified as the loop argument')
+ 'the one specified as the loop argument')
return coro_or_future
- elif inspect.isawaitable(coro_or_future):
- return ensure_future(_wrap_awaitable(coro_or_future), loop=loop)
- else:
- raise TypeError('An asyncio.Future, a coroutine or an awaitable is '
- 'required')
+
+ if not coroutines.iscoroutine(coro_or_future):
+ if inspect.isawaitable(coro_or_future):
+ coro_or_future = _wrap_awaitable(coro_or_future)
+ else:
+ raise TypeError('An asyncio.Future, a coroutine or an awaitable '
+ 'is required')
+
+ if loop is None:
+ loop = events._get_event_loop(stacklevel=4)
+ return loop.create_task(coro_or_future)
@types.coroutine
@@ -655,7 +658,8 @@ class _GatheringFuture(futures.Future):
cancelled.
"""
- def __init__(self, children, *, loop=None):
+ def __init__(self, children, *, loop):
+ assert loop is not None
super().__init__(loop=loop)
self._children = children
self._cancel_requested = False
@@ -706,7 +710,7 @@ def gather(*coros_or_futures, return_exceptions=False):
gather won't cancel any other awaitables.
"""
if not coros_or_futures:
- loop = events.get_event_loop()
+ loop = events._get_event_loop()
outer = loop.create_future()
outer.set_result([])
return outer
@@ -773,7 +777,7 @@ def _done_callback(fut):
loop = None
for arg in coros_or_futures:
if arg not in arg_to_fut:
- fut = ensure_future(arg, loop=loop)
+ fut = _ensure_future(arg, loop=loop)
if loop is None:
loop = futures._get_loop(fut)
if fut is not arg:
@@ -823,7 +827,7 @@ def shield(arg):
except CancelledError:
res = None
"""
- inner = ensure_future(arg)
+ inner = _ensure_future(arg)
if inner.done():
# Shortcut.
return inner
diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py
index 5511407ba5395..55fc266cb714b 100644
--- a/Lib/test/test_asyncio/test_events.py
+++ b/Lib/test/test_asyncio/test_events.py
@@ -2702,14 +2702,18 @@ def get_event_loop(self):
asyncio.set_event_loop_policy(Policy())
loop = asyncio.new_event_loop()
- with self.assertRaises(TestError):
- asyncio.get_event_loop()
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaises(TestError):
+ asyncio.get_event_loop()
+ self.assertEqual(cm.warnings[0].filename, __file__)
asyncio.set_event_loop(None)
- with self.assertRaises(TestError):
- asyncio.get_event_loop()
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaises(TestError):
+ asyncio.get_event_loop()
+ self.assertEqual(cm.warnings[0].filename, __file__)
with self.assertRaisesRegex(RuntimeError, 'no running'):
- self.assertIs(asyncio.get_running_loop(), None)
+ asyncio.get_running_loop()
self.assertIs(asyncio._get_running_loop(), None)
async def func():
@@ -2720,12 +2724,16 @@ async def func():
loop.run_until_complete(func())
asyncio.set_event_loop(loop)
- with self.assertRaises(TestError):
- asyncio.get_event_loop()
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaises(TestError):
+ asyncio.get_event_loop()
+ self.assertEqual(cm.warnings[0].filename, __file__)
asyncio.set_event_loop(None)
- with self.assertRaises(TestError):
- asyncio.get_event_loop()
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaises(TestError):
+ asyncio.get_event_loop()
+ self.assertEqual(cm.warnings[0].filename, __file__)
finally:
asyncio.set_event_loop_policy(old_policy)
@@ -2733,7 +2741,56 @@ async def func():
loop.close()
with self.assertRaisesRegex(RuntimeError, 'no running'):
- self.assertIs(asyncio.get_running_loop(), None)
+ asyncio.get_running_loop()
+
+ self.assertIs(asyncio._get_running_loop(), None)
+
+ def test_get_event_loop_returns_running_loop2(self):
+ old_policy = asyncio.get_event_loop_policy()
+ try:
+ asyncio.set_event_loop_policy(asyncio.DefaultEventLoopPolicy())
+ loop = asyncio.new_event_loop()
+ self.addCleanup(loop.close)
+
+ with self.assertWarns(DeprecationWarning) as cm:
+ loop2 = asyncio.get_event_loop()
+ self.addCleanup(loop2.close)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+ asyncio.set_event_loop(None)
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaisesRegex(RuntimeError, 'no current'):
+ asyncio.get_event_loop()
+ self.assertEqual(cm.warnings[0].filename, __file__)
+
+ with self.assertRaisesRegex(RuntimeError, 'no running'):
+ asyncio.get_running_loop()
+ self.assertIs(asyncio._get_running_loop(), None)
+
+ async def func():
+ self.assertIs(asyncio.get_event_loop(), loop)
+ self.assertIs(asyncio.get_running_loop(), loop)
+ self.assertIs(asyncio._get_running_loop(), loop)
+
+ loop.run_until_complete(func())
+
+ asyncio.set_event_loop(loop)
+ with self.assertWarns(DeprecationWarning) as cm:
+ self.assertIs(asyncio.get_event_loop(), loop)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+
+ asyncio.set_event_loop(None)
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaisesRegex(RuntimeError, 'no current'):
+ asyncio.get_event_loop()
+ self.assertEqual(cm.warnings[0].filename, __file__)
+
+ finally:
+ asyncio.set_event_loop_policy(old_policy)
+ if loop is not None:
+ loop.close()
+
+ with self.assertRaisesRegex(RuntimeError, 'no running'):
+ asyncio.get_running_loop()
self.assertIs(asyncio._get_running_loop(), None)
diff --git a/Lib/test/test_asyncio/test_futures.py b/Lib/test/test_asyncio/test_futures.py
index ec00896cc620b..fe3d44227c83c 100644
--- a/Lib/test/test_asyncio/test_futures.py
+++ b/Lib/test/test_asyncio/test_futures.py
@@ -139,9 +139,26 @@ def test_initial_state(self):
f.cancel()
self.assertTrue(f.cancelled())
- def test_init_constructor_default_loop(self):
+ def test_constructor_without_loop(self):
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaisesRegex(RuntimeError, 'There is no current event loop'):
+ self._new_future()
+ self.assertEqual(cm.warnings[0].filename, __file__)
+
+ def test_constructor_use_running_loop(self):
+ async def test():
+ return self._new_future()
+ f = self.loop.run_until_complete(test())
+ self.assertIs(f._loop, self.loop)
+ self.assertIs(f.get_loop(), self.loop)
+
+ def test_constructor_use_global_loop(self):
+ # Deprecated in 3.10
asyncio.set_event_loop(self.loop)
- f = self._new_future()
+ self.addCleanup(asyncio.set_event_loop, None)
+ with self.assertWarns(DeprecationWarning) as cm:
+ f = self._new_future()
+ self.assertEqual(cm.warnings[0].filename, __file__)
self.assertIs(f._loop, self.loop)
self.assertIs(f.get_loop(), self.loop)
@@ -472,16 +489,41 @@ def test_wrap_future_future(self):
f2 = asyncio.wrap_future(f1)
self.assertIs(f1, f2)
+ def test_wrap_future_without_loop(self):
+ def run(arg):
+ return (arg, threading.get_ident())
+ ex = concurrent.futures.ThreadPoolExecutor(1)
+ f1 = ex.submit(run, 'oi')
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaises(RuntimeError):
+ asyncio.wrap_future(f1)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+ ex.shutdown(wait=True)
+
+ def test_wrap_future_use_running_loop(self):
+ def run(arg):
+ return (arg, threading.get_ident())
+ ex = concurrent.futures.ThreadPoolExecutor(1)
+ f1 = ex.submit(run, 'oi')
+ async def test():
+ return asyncio.wrap_future(f1)
+ f2 = self.loop.run_until_complete(test())
+ self.assertIs(self.loop, f2._loop)
+ ex.shutdown(wait=True)
+
def test_wrap_future_use_global_loop(self):
- with mock.patch('asyncio.futures.events') as events:
- events.get_event_loop = lambda: self.loop
- def run(arg):
- return (arg, threading.get_ident())
- ex = concurrent.futures.ThreadPoolExecutor(1)
- f1 = ex.submit(run, 'oi')
+ # Deprecated in 3.10
+ asyncio.set_event_loop(self.loop)
+ self.addCleanup(asyncio.set_event_loop, None)
+ def run(arg):
+ return (arg, threading.get_ident())
+ ex = concurrent.futures.ThreadPoolExecutor(1)
+ f1 = ex.submit(run, 'oi')
+ with self.assertWarns(DeprecationWarning) as cm:
f2 = asyncio.wrap_future(f1)
- self.assertIs(self.loop, f2._loop)
- ex.shutdown(wait=True)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+ self.assertIs(self.loop, f2._loop)
+ ex.shutdown(wait=True)
def test_wrap_future_cancel(self):
f1 = concurrent.futures.Future()
diff --git a/Lib/test/test_asyncio/test_queues.py b/Lib/test/test_asyncio/test_queues.py
index 0a0b529f621b2..63a9a5f270cc9 100644
--- a/Lib/test/test_asyncio/test_queues.py
+++ b/Lib/test/test_asyncio/test_queues.py
@@ -273,12 +273,12 @@ async def create_queue():
queue._get_loop()
return queue
- q = self.loop.run_until_complete(create_queue())
+ async def test():
+ q = await create_queue()
+ await asyncio.gather(producer(q, producer_num_items),
+ consumer(q, producer_num_items))
- self.loop.run_until_complete(
- asyncio.gather(producer(q, producer_num_items),
- consumer(q, producer_num_items)),
- )
+ self.loop.run_until_complete(test())
def test_cancelled_getters_not_being_held_in_self_getters(self):
def a_generator():
@@ -516,11 +516,14 @@ async def getter():
for _ in range(num):
item = queue.get_nowait()
- t0 = putter(0)
- t1 = putter(1)
- t2 = putter(2)
- t3 = putter(3)
- self.loop.run_until_complete(asyncio.gather(getter(), t0, t1, t2, t3))
+ async def test():
+ t0 = putter(0)
+ t1 = putter(1)
+ t2 = putter(2)
+ t3 = putter(3)
+ await asyncio.gather(getter(), t0, t1, t2, t3)
+
+ self.loop.run_until_complete(test())
def test_cancelled_puts_not_being_held_in_self_putters(self):
def a_generator():
diff --git a/Lib/test/test_asyncio/test_streams.py b/Lib/test/test_asyncio/test_streams.py
index a075358903920..6eaa289944218 100644
--- a/Lib/test/test_asyncio/test_streams.py
+++ b/Lib/test/test_asyncio/test_streams.py
@@ -40,11 +40,6 @@ def tearDown(self):
gc.collect()
super().tearDown()
- @mock.patch('asyncio.streams.events')
- def test_ctor_global_loop(self, m_events):
- stream = asyncio.StreamReader()
- self.assertIs(stream._loop, m_events.get_event_loop.return_value)
-
def _basetest_open_connection(self, open_connection_fut):
messages = []
self.loop.set_exception_handler(lambda loop, ctx: messages.append(ctx))
@@ -751,23 +746,59 @@ def test_read_all_from_pipe_reader(self):
data = self.loop.run_until_complete(reader.read(-1))
self.assertEqual(data, b'data')
- def test_streamreader_constructor(self):
- self.addCleanup(asyncio.set_event_loop, None)
- asyncio.set_event_loop(self.loop)
+ def test_streamreader_constructor_without_loop(self):
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaisesRegex(RuntimeError, 'There is no current event loop'):
+ asyncio.StreamReader()
+ self.assertEqual(cm.warnings[0].filename, __file__)
+ def test_streamreader_constructor_use_running_loop(self):
# asyncio issue #184: Ensure that StreamReaderProtocol constructor
# retrieves the current loop if the loop parameter is not set
- reader = asyncio.StreamReader()
+ async def test():
+ return asyncio.StreamReader()
+
+ reader = self.loop.run_until_complete(test())
self.assertIs(reader._loop, self.loop)
- def test_streamreaderprotocol_constructor(self):
+ def test_streamreader_constructor_use_global_loop(self):
+ # asyncio issue #184: Ensure that StreamReaderProtocol constructor
+ # retrieves the current loop if the loop parameter is not set
+ # Deprecated in 3.10
self.addCleanup(asyncio.set_event_loop, None)
asyncio.set_event_loop(self.loop)
+ with self.assertWarns(DeprecationWarning) as cm:
+ reader = asyncio.StreamReader()
+ self.assertEqual(cm.warnings[0].filename, __file__)
+ self.assertIs(reader._loop, self.loop)
+
+ def test_streamreaderprotocol_constructor_without_loop(self):
+ reader = mock.Mock()
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaisesRegex(RuntimeError, 'There is no current event loop'):
+ asyncio.StreamReaderProtocol(reader)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+
+ def test_streamreaderprotocol_constructor_use_running_loop(self):
+ # asyncio issue #184: Ensure that StreamReaderProtocol constructor
+ # retrieves the current loop if the loop parameter is not set
+ reader = mock.Mock()
+ async def test():
+ return asyncio.StreamReaderProtocol(reader)
+ protocol = self.loop.run_until_complete(test())
+ self.assertIs(protocol._loop, self.loop)
+
+ def test_streamreaderprotocol_constructor_use_global_loop(self):
# asyncio issue #184: Ensure that StreamReaderProtocol constructor
# retrieves the current loop if the loop parameter is not set
+ # Deprecated in 3.10
+ self.addCleanup(asyncio.set_event_loop, None)
+ asyncio.set_event_loop(self.loop)
reader = mock.Mock()
- protocol = asyncio.StreamReaderProtocol(reader)
+ with self.assertWarns(DeprecationWarning) as cm:
+ protocol = asyncio.StreamReaderProtocol(reader)
+ self.assertEqual(cm.warnings[0].filename, __file__)
self.assertIs(protocol._loop, self.loop)
def test_drain_raises(self):
diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py
index 7c2e85ceefde1..a9e4cf53566ca 100644
--- a/Lib/test/test_asyncio/test_tasks.py
+++ b/Lib/test/test_asyncio/test_tasks.py
@@ -200,22 +200,76 @@ async def notmuch():
loop.close()
def test_ensure_future_coroutine(self):
+ async def notmuch():
+ return 'ok'
+ t = asyncio.ensure_future(notmuch(), loop=self.loop)
+ self.assertIs(t._loop, self.loop)
+ self.loop.run_until_complete(t)
+ self.assertTrue(t.done())
+ self.assertEqual(t.result(), 'ok')
+
+ a = notmuch()
+ self.addCleanup(a.close)
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaisesRegex(RuntimeError, 'There is no current event loop'):
+ asyncio.ensure_future(a)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+
+ async def test():
+ return asyncio.ensure_future(notmuch())
+ t = self.loop.run_until_complete(test())
+ self.assertIs(t._loop, self.loop)
+ self.loop.run_until_complete(t)
+ self.assertTrue(t.done())
+ self.assertEqual(t.result(), 'ok')
+
+ # Deprecated in 3.10
+ asyncio.set_event_loop(self.loop)
+ self.addCleanup(asyncio.set_event_loop, None)
+ with self.assertWarns(DeprecationWarning) as cm:
+ t = asyncio.ensure_future(notmuch())
+ self.assertEqual(cm.warnings[0].filename, __file__)
+ self.assertIs(t._loop, self.loop)
+ self.loop.run_until_complete(t)
+ self.assertTrue(t.done())
+ self.assertEqual(t.result(), 'ok')
+
+ def test_ensure_future_coroutine_2(self):
with self.assertWarns(DeprecationWarning):
@asyncio.coroutine
def notmuch():
return 'ok'
t = asyncio.ensure_future(notmuch(), loop=self.loop)
+ self.assertIs(t._loop, self.loop)
self.loop.run_until_complete(t)
self.assertTrue(t.done())
self.assertEqual(t.result(), 'ok')
+
+ a = notmuch()
+ self.addCleanup(a.close)
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaisesRegex(RuntimeError, 'There is no current event loop'):
+ asyncio.ensure_future(a)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+
+ async def test():
+ return asyncio.ensure_future(notmuch())
+ t = self.loop.run_until_complete(test())
self.assertIs(t._loop, self.loop)
+ self.loop.run_until_complete(t)
+ self.assertTrue(t.done())
+ self.assertEqual(t.result(), 'ok')
- loop = asyncio.new_event_loop()
- self.set_event_loop(loop)
- t = asyncio.ensure_future(notmuch(), loop=loop)
- self.assertIs(t._loop, loop)
- loop.run_until_complete(t)
- loop.close()
+ # Deprecated in 3.10
+ asyncio.set_event_loop(self.loop)
+ self.addCleanup(asyncio.set_event_loop, None)
+ with self.assertWarns(DeprecationWarning) as cm:
+ t = asyncio.ensure_future(notmuch())
+ self.assertEqual(cm.warnings[0].filename, __file__)
+ self.assertIs(t._loop, self.loop)
+ self.loop.run_until_complete(t)
+ self.assertTrue(t.done())
+ self.assertEqual(t.result(), 'ok')
def test_ensure_future_future(self):
f_orig = self.new_future(self.loop)
@@ -1078,33 +1132,6 @@ async def coro():
res = loop.run_until_complete(asyncio.wait_for(coro(), timeout=None))
self.assertEqual(res, 'done')
- def test_wait_for_with_global_loop(self):
-
- def gen():
- when = yield
- self.assertAlmostEqual(0.2, when)
- when = yield 0
- self.assertAlmostEqual(0.01, when)
- yield 0.01
-
- loop = self.new_test_loop(gen)
-
- async def foo():
- await asyncio.sleep(0.2)
- return 'done'
-
- asyncio.set_event_loop(loop)
- try:
- fut = self.new_task(loop, foo())
- with self.assertRaises(asyncio.TimeoutError):
- loop.run_until_complete(asyncio.wait_for(fut, 0.01))
- finally:
- asyncio.set_event_loop(None)
-
- self.assertAlmostEqual(0.01, loop.time())
- self.assertTrue(fut.done())
- self.assertTrue(fut.cancelled())
-
def test_wait_for_race_condition(self):
def gen():
@@ -1293,32 +1320,6 @@ async def foo():
self.assertAlmostEqual(0.15, loop.time())
self.assertEqual(res, 42)
- def test_wait_with_global_loop(self):
-
- def gen():
- when = yield
- self.assertAlmostEqual(0.01, when)
- when = yield 0
- self.assertAlmostEqual(0.015, when)
- yield 0.015
-
- loop = self.new_test_loop(gen)
-
- a = self.new_task(loop, asyncio.sleep(0.01))
- b = self.new_task(loop, asyncio.sleep(0.015))
-
- async def foo():
- done, pending = await asyncio.wait([b, a])
- self.assertEqual(done, set([a, b]))
- self.assertEqual(pending, set())
- return 42
-
- asyncio.set_event_loop(loop)
- res = loop.run_until_complete(
- self.new_task(loop, foo()))
-
- self.assertEqual(res, 42)
-
def test_wait_duplicate_coroutines(self):
with self.assertWarns(DeprecationWarning):
@@ -1679,22 +1680,24 @@ def gen():
yield 0
loop = self.new_test_loop(gen)
- asyncio.set_event_loop(loop)
a = asyncio.sleep(0.05, 'a')
b = asyncio.sleep(0.10, 'b')
fs = {a, b}
- futs = list(asyncio.as_completed(fs))
- self.assertEqual(len(futs), 2)
+ async def test():
+ futs = list(asyncio.as_completed(fs))
+ self.assertEqual(len(futs), 2)
+
+ x = await futs[1]
+ self.assertEqual(x, 'a')
+ self.assertAlmostEqual(0.05, loop.time())
+ loop.advance_time(0.05)
+ y = await futs[0]
+ self.assertEqual(y, 'b')
+ self.assertAlmostEqual(0.10, loop.time())
- x = loop.run_until_complete(futs[1])
- self.assertEqual(x, 'a')
- self.assertAlmostEqual(0.05, loop.time())
- loop.advance_time(0.05)
- y = loop.run_until_complete(futs[0])
- self.assertEqual(y, 'b')
- self.assertAlmostEqual(0.10, loop.time())
+ loop.run_until_complete(test())
def test_as_completed_concurrent(self):
@@ -1705,20 +1708,22 @@ def gen():
self.assertAlmostEqual(0.05, when)
yield 0.05
- loop = self.new_test_loop(gen)
- asyncio.set_event_loop(loop)
-
a = asyncio.sleep(0.05, 'a')
b = asyncio.sleep(0.05, 'b')
fs = {a, b}
- futs = list(asyncio.as_completed(fs))
- self.assertEqual(len(futs), 2)
- waiter = asyncio.wait(futs)
- # Deprecation from passing coros in futs to asyncio.wait()
- with self.assertWarns(DeprecationWarning):
- done, pending = loop.run_until_complete(waiter)
- self.assertEqual(set(f.result() for f in done), {'a', 'b'})
+ async def test():
+ futs = list(asyncio.as_completed(fs))
+ self.assertEqual(len(futs), 2)
+ waiter = asyncio.wait(futs)
+ # Deprecation from passing coros in futs to asyncio.wait()
+ with self.assertWarns(DeprecationWarning) as cm:
+ done, pending = await waiter
+ self.assertEqual(cm.warnings[0].filename, __file__)
+ self.assertEqual(set(f.result() for f in done), {'a', 'b'})
+
+ loop = self.new_test_loop(gen)
+ loop.run_until_complete(test())
def test_as_completed_duplicate_coroutines(self):
@@ -1742,6 +1747,47 @@ def runner():
self.assertEqual(set(result), {'ham', 'spam'})
self.assertEqual(len(result), 2)
+ def test_as_completed_coroutine_without_loop(self):
+ async def coro():
+ return 42
+
+ a = coro()
+ self.addCleanup(a.close)
+
+ futs = asyncio.as_completed([a])
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaisesRegex(RuntimeError, 'There is no current event loop'):
+ list(futs)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+
+ def test_as_completed_coroutine_use_running_loop(self):
+ loop = self.new_test_loop()
+
+ async def coro():
+ return 42
+
+ async def test():
+ futs = list(asyncio.as_completed([coro()]))
+ self.assertEqual(len(futs), 1)
+ self.assertEqual(await futs[0], 42)
+
+ loop.run_until_complete(test())
+
+ def test_as_completed_coroutine_use_global_loop(self):
+ # Deprecated in 3.10
+ async def coro():
+ return 42
+
+ loop = self.new_test_loop()
+ asyncio.set_event_loop(loop)
+ self.addCleanup(asyncio.set_event_loop, None)
+ futs = asyncio.as_completed([coro()])
+ with self.assertWarns(DeprecationWarning) as cm:
+ futs = list(futs)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+ self.assertEqual(len(futs), 1)
+ self.assertEqual(loop.run_until_complete(futs[0]), 42)
+
def test_sleep(self):
def gen():
@@ -2201,6 +2247,42 @@ def test_gather_shield(self):
child2.set_result(2)
test_utils.run_briefly(self.loop)
+ def test_shield_coroutine_without_loop(self):
+ async def coro():
+ return 42
+
+ inner = coro()
+ self.addCleanup(inner.close)
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaisesRegex(RuntimeError, 'There is no current event loop'):
+ asyncio.shield(inner)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+
+ def test_shield_coroutine_use_running_loop(self):
+ async def coro():
+ return 42
+
+ async def test():
+ return asyncio.shield(coro())
+ outer = self.loop.run_until_complete(test())
+ self.assertEqual(outer._loop, self.loop)
+ res = self.loop.run_until_complete(outer)
+ self.assertEqual(res, 42)
+
+ def test_shield_coroutine_use_global_loop(self):
+ # Deprecated in 3.10
+ async def coro():
+ return 42
+
+ asyncio.set_event_loop(self.loop)
+ self.addCleanup(asyncio.set_event_loop, None)
+ with self.assertWarns(DeprecationWarning) as cm:
+ outer = asyncio.shield(coro())
+ self.assertEqual(cm.warnings[0].filename, __file__)
+ self.assertEqual(outer._loop, self.loop)
+ res = self.loop.run_until_complete(outer)
+ self.assertEqual(res, 42)
+
def test_as_completed_invalid_args(self):
fut = self.new_future(self.loop)
@@ -2507,16 +2589,17 @@ def test_cancel_gather_1(self):
"""Ensure that a gathering future refuses to be cancelled once all
children are done"""
loop = asyncio.new_event_loop()
- asyncio.set_event_loop(loop)
self.addCleanup(loop.close)
fut = self.new_future(loop)
- # The indirection fut->child_coro is needed since otherwise the
- # gathering task is done at the same time as the child future
- def child_coro():
- return (yield from fut)
- gather_future = asyncio.gather(child_coro())
- gather_task = asyncio.ensure_future(gather_future, loop=loop)
+ async def create():
+ # The indirection fut->child_coro is needed since otherwise the
+ # gathering task is done at the same time as the child future
+ def child_coro():
+ return (yield from fut)
+ gather_future = asyncio.gather(child_coro())
+ return asyncio.ensure_future(gather_future)
+ gather_task = loop.run_until_complete(create())
cancel_result = None
def cancelling_callback(_):
@@ -3222,7 +3305,7 @@ def _run_loop(self, loop):
def _check_success(self, **kwargs):
a, b, c = [self.one_loop.create_future() for i in range(3)]
- fut = asyncio.gather(*self.wrap_futures(a, b, c), **kwargs)
+ fut = self._gather(*self.wrap_futures(a, b, c), **kwargs)
cb = test_utils.MockCallback()
fut.add_done_callback(cb)
b.set_result(1)
@@ -3244,7 +3327,7 @@ def test_result_exception_success(self):
def test_one_exception(self):
a, b, c, d, e = [self.one_loop.create_future() for i in range(5)]
- fut = asyncio.gather(*self.wrap_futures(a, b, c, d, e))
+ fut = self._gather(*self.wrap_futures(a, b, c, d, e))
cb = test_utils.MockCallback()
fut.add_done_callback(cb)
exc = ZeroDivisionError()
@@ -3262,8 +3345,8 @@ def test_one_exception(self):
def test_return_exceptions(self):
a, b, c, d = [self.one_loop.create_future() for i in range(4)]
- fut = asyncio.gather(*self.wrap_futures(a, b, c, d),
- return_exceptions=True)
+ fut = self._gather(*self.wrap_futures(a, b, c, d),
+ return_exceptions=True)
cb = test_utils.MockCallback()
fut.add_done_callback(cb)
exc = ZeroDivisionError()
@@ -3315,22 +3398,38 @@ class FutureGatherTests(GatherTestsBase, test_utils.TestCase):
def wrap_futures(self, *futures):
return futures
- def _check_empty_sequence(self, seq_or_iter):
+ def _gather(self, *args, **kwargs):
+ return asyncio.gather(*args, **kwargs)
+
+ def test_constructor_empty_sequence_without_loop(self):
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaises(RuntimeError):
+ asyncio.gather()
+ self.assertEqual(cm.warnings[0].filename, __file__)
+
+ def test_constructor_empty_sequence_use_running_loop(self):
+ async def gather():
+ return asyncio.gather()
+ fut = self.one_loop.run_until_complete(gather())
+ self.assertIsInstance(fut, asyncio.Future)
+ self.assertIs(fut._loop, self.one_loop)
+ self._run_loop(self.one_loop)
+ self.assertTrue(fut.done())
+ self.assertEqual(fut.result(), [])
+
+ def test_constructor_empty_sequence_use_global_loop(self):
+ # Deprecated in 3.10
asyncio.set_event_loop(self.one_loop)
self.addCleanup(asyncio.set_event_loop, None)
- fut = asyncio.gather(*seq_or_iter)
+ with self.assertWarns(DeprecationWarning) as cm:
+ fut = asyncio.gather()
+ self.assertEqual(cm.warnings[0].filename, __file__)
self.assertIsInstance(fut, asyncio.Future)
self.assertIs(fut._loop, self.one_loop)
self._run_loop(self.one_loop)
self.assertTrue(fut.done())
self.assertEqual(fut.result(), [])
- def test_constructor_empty_sequence(self):
- self._check_empty_sequence([])
- self._check_empty_sequence(())
- self._check_empty_sequence(set())
- self._check_empty_sequence(iter(""))
-
def test_constructor_heterogenous_futures(self):
fut1 = self.one_loop.create_future()
fut2 = self.other_loop.create_future()
@@ -3392,10 +3491,6 @@ def test_result_exception_one_cancellation(self):
class CoroutineGatherTests(GatherTestsBase, test_utils.TestCase):
- def setUp(self):
- super().setUp()
- asyncio.set_event_loop(self.one_loop)
-
def wrap_futures(self, *futures):
coros = []
for fut in futures:
@@ -3404,22 +3499,47 @@ async def coro(fut=fut):
coros.append(coro())
return coros
- def test_constructor_loop_selection(self):
+ def _gather(self, *args, **kwargs):
+ async def coro():
+ return asyncio.gather(*args, **kwargs)
+ return self.one_loop.run_until_complete(coro())
+
+ def test_constructor_without_loop(self):
async def coro():
return 'abc'
gen1 = coro()
+ self.addCleanup(gen1.close)
gen2 = coro()
- fut = asyncio.gather(gen1, gen2)
+ self.addCleanup(gen2.close)
+ with self.assertWarns(DeprecationWarning) as cm:
+ with self.assertRaises(RuntimeError):
+ asyncio.gather(gen1, gen2)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+
+ def test_constructor_use_running_loop(self):
+ async def coro():
+ return 'abc'
+ gen1 = coro()
+ gen2 = coro()
+ async def gather():
+ return asyncio.gather(gen1, gen2)
+ fut = self.one_loop.run_until_complete(gather())
self.assertIs(fut._loop, self.one_loop)
self.one_loop.run_until_complete(fut)
- self.set_event_loop(self.other_loop, cleanup=False)
+ def test_constructor_use_global_loop(self):
+ # Deprecated in 3.10
+ async def coro():
+ return 'abc'
asyncio.set_event_loop(self.other_loop)
- gen3 = coro()
- gen4 = coro()
- fut2 = asyncio.gather(gen3, gen4)
- self.assertIs(fut2._loop, self.other_loop)
- self.other_loop.run_until_complete(fut2)
+ self.addCleanup(asyncio.set_event_loop, None)
+ gen1 = coro()
+ gen2 = coro()
+ with self.assertWarns(DeprecationWarning) as cm:
+ fut = asyncio.gather(gen1, gen2)
+ self.assertEqual(cm.warnings[0].filename, __file__)
+ self.assertIs(fut._loop, self.other_loop)
+ self.other_loop.run_until_complete(fut)
def test_duplicate_coroutines(self):
with self.assertWarns(DeprecationWarning):
@@ -3427,7 +3547,7 @@ def test_duplicate_coroutines(self):
def coro(s):
return s
c = coro('abc')
- fut = asyncio.gather(c, c, coro('def'), c)
+ fut = self._gather(c, c, coro('def'), c)
self._run_loop(self.one_loop)
self.assertEqual(fut.result(), ['abc', 'abc', 'def', 'abc'])
diff --git a/Misc/NEWS.d/next/Library/2020-12-06-20-21-16.bpo-39529.9Zrg43.rst b/Misc/NEWS.d/next/Library/2020-12-06-20-21-16.bpo-39529.9Zrg43.rst
new file mode 100644
index 0000000000000..bb1fd82c99e36
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2020-12-06-20-21-16.bpo-39529.9Zrg43.rst
@@ -0,0 +1,9 @@
+Deprecated use of :func:`asyncio.get_event_loop` without running event loop.
+Emit deprecation warning for :mod:`asyncio` functions which implicitly
+create a :class:`~asyncio.Future` or :class:`~asyncio.Task` objects if there
+is no running event loop and no explicit *loop* argument is passed:
+:func:`~asyncio.ensure_future`, :func:`~asyncio.wrap_future`,
+:func:`~asyncio.gather`, :func:`~asyncio.shield`,
+:func:`~asyncio.as_completed` and constructors of :class:`~asyncio.Future`,
+:class:`~asyncio.Task`, :class:`~asyncio.StreamReader`,
+:class:`~asyncio.StreamReaderProtocol`.
diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c
index 01e36c656da8f..a4d5d4551e9b0 100644
--- a/Modules/_asynciomodule.c
+++ b/Modules/_asynciomodule.c
@@ -319,7 +319,7 @@ set_running_loop(PyObject *loop)
static PyObject *
-get_event_loop(void)
+get_event_loop(int stacklevel)
{
PyObject *loop;
PyObject *policy;
@@ -331,6 +331,13 @@ get_event_loop(void)
return loop;
}
+ if (PyErr_WarnEx(PyExc_DeprecationWarning,
+ "There is no current event loop",
+ stacklevel))
+ {
+ return NULL;
+ }
+
policy = PyObject_CallNoArgs(asyncio_get_event_loop_policy);
if (policy == NULL) {
return NULL;
@@ -489,7 +496,7 @@ future_init(FutureObj *fut, PyObject *loop)
fut->fut_blocking = 0;
if (loop == Py_None) {
- loop = get_event_loop();
+ loop = get_event_loop(1);
if (loop == NULL) {
return -1;
}
@@ -3078,7 +3085,19 @@ static PyObject *
_asyncio_get_event_loop_impl(PyObject *module)
/*[clinic end generated code: output=2a2d8b2f824c648b input=9364bf2916c8655d]*/
{
- return get_event_loop();
+ return get_event_loop(1);
+}
+
+/*[clinic input]
+_asyncio._get_event_loop
+ stacklevel: int = 3
+[clinic start generated code]*/
+
+static PyObject *
+_asyncio__get_event_loop_impl(PyObject *module, int stacklevel)
+/*[clinic end generated code: output=9c1d6d3c802e67c9 input=d17aebbd686f711d]*/
+{
+ return get_event_loop(stacklevel-1);
}
/*[clinic input]
@@ -3375,6 +3394,7 @@ PyDoc_STRVAR(module_doc, "Accelerator module for asyncio");
static PyMethodDef asyncio_methods[] = {
_ASYNCIO_GET_EVENT_LOOP_METHODDEF
+ _ASYNCIO__GET_EVENT_LOOP_METHODDEF
_ASYNCIO_GET_RUNNING_LOOP_METHODDEF
_ASYNCIO__GET_RUNNING_LOOP_METHODDEF
_ASYNCIO__SET_RUNNING_LOOP_METHODDEF
diff --git a/Modules/clinic/_asynciomodule.c.h b/Modules/clinic/_asynciomodule.c.h
index a071efc1e2be3..c472e652fb7c5 100644
--- a/Modules/clinic/_asynciomodule.c.h
+++ b/Modules/clinic/_asynciomodule.c.h
@@ -669,6 +669,45 @@ _asyncio_get_event_loop(PyObject *module, PyObject *Py_UNUSED(ignored))
return _asyncio_get_event_loop_impl(module);
}
+PyDoc_STRVAR(_asyncio__get_event_loop__doc__,
+"_get_event_loop($module, /, stacklevel=3)\n"
+"--\n"
+"\n");
+
+#define _ASYNCIO__GET_EVENT_LOOP_METHODDEF \
+ {"_get_event_loop", (PyCFunction)(void(*)(void))_asyncio__get_event_loop, METH_FASTCALL|METH_KEYWORDS, _asyncio__get_event_loop__doc__},
+
+static PyObject *
+_asyncio__get_event_loop_impl(PyObject *module, int stacklevel);
+
+static PyObject *
+_asyncio__get_event_loop(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
+{
+ PyObject *return_value = NULL;
+ static const char * const _keywords[] = {"stacklevel", NULL};
+ static _PyArg_Parser _parser = {NULL, _keywords, "_get_event_loop", 0};
+ PyObject *argsbuf[1];
+ Py_ssize_t noptargs = nargs + (kwnames ? PyTuple_GET_SIZE(kwnames) : 0) - 0;
+ int stacklevel = 3;
+
+ args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 0, 1, 0, argsbuf);
+ if (!args) {
+ goto exit;
+ }
+ if (!noptargs) {
+ goto skip_optional_pos;
+ }
+ stacklevel = _PyLong_AsInt(args[0]);
+ if (stacklevel == -1 && PyErr_Occurred()) {
+ goto exit;
+ }
+skip_optional_pos:
+ return_value = _asyncio__get_event_loop_impl(module, stacklevel);
+
+exit:
+ return return_value;
+}
+
PyDoc_STRVAR(_asyncio_get_running_loop__doc__,
"get_running_loop($module, /)\n"
"--\n"
@@ -832,4 +871,4 @@ _asyncio__leave_task(PyObject *module, PyObject *const *args, Py_ssize_t nargs,
exit:
return return_value;
}
-/*[clinic end generated code: output=d0fc522bcbff9d61 input=a9049054013a1b77]*/
+/*[clinic end generated code: output=0d127162ac92e0c0 input=a9049054013a1b77]*/
1
0
bpo-42609: Check recursion depth in the AST validator and optimizer (GH-23744)
by serhiy-storchaka 25 Apr '21
by serhiy-storchaka 25 Apr '21
25 Apr '21
https://github.com/python/cpython/commit/face87c94e67ad9c72b9a3724f112fd76c…
commit: face87c94e67ad9c72b9a3724f112fd76c1002b9
branch: master
author: Serhiy Storchaka <storchaka(a)gmail.com>
committer: serhiy-storchaka <storchaka(a)gmail.com>
date: 2021-04-25T13:38:00+03:00
summary:
bpo-42609: Check recursion depth in the AST validator and optimizer (GH-23744)
files:
A Misc/NEWS.d/next/Core and Builtins/2020-12-12-14-28-31.bpo-42609.Qcd54b.rst
M Include/internal/pycore_compile.h
M Lib/test/test_compile.py
M Python/ast.c
M Python/ast_opt.c
diff --git a/Include/internal/pycore_compile.h b/Include/internal/pycore_compile.h
index e8859bbec6daa..06a6082cddae6 100644
--- a/Include/internal/pycore_compile.h
+++ b/Include/internal/pycore_compile.h
@@ -28,6 +28,9 @@ extern PyObject* _Py_Mangle(PyObject *p, PyObject *name);
typedef struct {
int optimize;
int ff_features;
+
+ int recursion_depth; /* current recursion depth */
+ int recursion_limit; /* recursion limit */
} _PyASTOptimizeState;
extern int _PyAST_Optimize(
diff --git a/Lib/test/test_compile.py b/Lib/test/test_compile.py
index fd1ef612fd613..d40347c3c6bb3 100644
--- a/Lib/test/test_compile.py
+++ b/Lib/test/test_compile.py
@@ -543,21 +543,26 @@ def test_compiler_recursion_limit(self):
# XXX (ncoghlan): duplicating the scaling factor here is a little
# ugly. Perhaps it should be exposed somewhere...
fail_depth = sys.getrecursionlimit() * 3
+ crash_depth = sys.getrecursionlimit() * 300
success_depth = int(fail_depth * 0.75)
- def check_limit(prefix, repeated):
+ def check_limit(prefix, repeated, mode="single"):
expect_ok = prefix + repeated * success_depth
- self.compile_single(expect_ok)
- broken = prefix + repeated * fail_depth
- details = "Compiling ({!r} + {!r} * {})".format(
- prefix, repeated, fail_depth)
- with self.assertRaises(RecursionError, msg=details):
- self.compile_single(broken)
+ compile(expect_ok, '<test>', mode)
+ for depth in (fail_depth, crash_depth):
+ broken = prefix + repeated * depth
+ details = "Compiling ({!r} + {!r} * {})".format(
+ prefix, repeated, depth)
+ with self.assertRaises(RecursionError, msg=details):
+ compile(broken, '<test>', mode)
check_limit("a", "()")
check_limit("a", ".b")
check_limit("a", "[0]")
check_limit("a", "*a")
+ # XXX Crashes in the parser.
+ # check_limit("a", " if a else a")
+ # check_limit("if a: pass", "\nelif a: pass", mode="exec")
def test_null_terminated(self):
# The source code is null-terminated internally, but bytes-like
diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-12-12-14-28-31.bpo-42609.Qcd54b.rst b/Misc/NEWS.d/next/Core and Builtins/2020-12-12-14-28-31.bpo-42609.Qcd54b.rst
new file mode 100644
index 0000000000000..d2b0c641b2ecc
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2020-12-12-14-28-31.bpo-42609.Qcd54b.rst
@@ -0,0 +1,3 @@
+Prevented crashes in the AST validator and optimizer when compiling some
+absurdly long expressions like ``"+0"*1000000``. :exc:`RecursionError` is
+now raised instead.
diff --git a/Python/ast.c b/Python/ast.c
index c87795305e507..2b965434ef461 100644
--- a/Python/ast.c
+++ b/Python/ast.c
@@ -4,14 +4,20 @@
*/
#include "Python.h"
#include "pycore_ast.h" // asdl_stmt_seq
+#include "pycore_pystate.h" // _PyThreadState_GET()
#include <assert.h>
-static int validate_stmts(asdl_stmt_seq *);
-static int validate_exprs(asdl_expr_seq*, expr_context_ty, int);
+struct validator {
+ int recursion_depth; /* current recursion depth */
+ int recursion_limit; /* recursion limit */
+};
+
+static int validate_stmts(struct validator *, asdl_stmt_seq *);
+static int validate_exprs(struct validator *, asdl_expr_seq*, expr_context_ty, int);
static int _validate_nonempty_seq(asdl_seq *, const char *, const char *);
-static int validate_stmt(stmt_ty);
-static int validate_expr(expr_ty, expr_context_ty);
+static int validate_stmt(struct validator *, stmt_ty);
+static int validate_expr(struct validator *, expr_ty, expr_context_ty);
static int
validate_name(PyObject *name)
@@ -33,7 +39,7 @@ validate_name(PyObject *name)
}
static int
-validate_comprehension(asdl_comprehension_seq *gens)
+validate_comprehension(struct validator *state, asdl_comprehension_seq *gens)
{
Py_ssize_t i;
if (!asdl_seq_LEN(gens)) {
@@ -42,31 +48,31 @@ validate_comprehension(asdl_comprehension_seq *gens)
}
for (i = 0; i < asdl_seq_LEN(gens); i++) {
comprehension_ty comp = asdl_seq_GET(gens, i);
- if (!validate_expr(comp->target, Store) ||
- !validate_expr(comp->iter, Load) ||
- !validate_exprs(comp->ifs, Load, 0))
+ if (!validate_expr(state, comp->target, Store) ||
+ !validate_expr(state, comp->iter, Load) ||
+ !validate_exprs(state, comp->ifs, Load, 0))
return 0;
}
return 1;
}
static int
-validate_keywords(asdl_keyword_seq *keywords)
+validate_keywords(struct validator *state, asdl_keyword_seq *keywords)
{
Py_ssize_t i;
for (i = 0; i < asdl_seq_LEN(keywords); i++)
- if (!validate_expr((asdl_seq_GET(keywords, i))->value, Load))
+ if (!validate_expr(state, (asdl_seq_GET(keywords, i))->value, Load))
return 0;
return 1;
}
static int
-validate_args(asdl_arg_seq *args)
+validate_args(struct validator *state, asdl_arg_seq *args)
{
Py_ssize_t i;
for (i = 0; i < asdl_seq_LEN(args); i++) {
arg_ty arg = asdl_seq_GET(args, i);
- if (arg->annotation && !validate_expr(arg->annotation, Load))
+ if (arg->annotation && !validate_expr(state, arg->annotation, Load))
return 0;
}
return 1;
@@ -88,19 +94,19 @@ expr_context_name(expr_context_ty ctx)
}
static int
-validate_arguments(arguments_ty args)
+validate_arguments(struct validator *state, arguments_ty args)
{
- if (!validate_args(args->posonlyargs) || !validate_args(args->args)) {
+ if (!validate_args(state, args->posonlyargs) || !validate_args(state, args->args)) {
return 0;
}
if (args->vararg && args->vararg->annotation
- && !validate_expr(args->vararg->annotation, Load)) {
+ && !validate_expr(state, args->vararg->annotation, Load)) {
return 0;
}
- if (!validate_args(args->kwonlyargs))
+ if (!validate_args(state, args->kwonlyargs))
return 0;
if (args->kwarg && args->kwarg->annotation
- && !validate_expr(args->kwarg->annotation, Load)) {
+ && !validate_expr(state, args->kwarg->annotation, Load)) {
return 0;
}
if (asdl_seq_LEN(args->defaults) > asdl_seq_LEN(args->posonlyargs) + asdl_seq_LEN(args->args)) {
@@ -112,11 +118,11 @@ validate_arguments(arguments_ty args)
"kw_defaults on arguments");
return 0;
}
- return validate_exprs(args->defaults, Load, 0) && validate_exprs(args->kw_defaults, Load, 1);
+ return validate_exprs(state, args->defaults, Load, 0) && validate_exprs(state, args->kw_defaults, Load, 1);
}
static int
-validate_constant(PyObject *value)
+validate_constant(struct validator *state, PyObject *value)
{
if (value == Py_None || value == Py_Ellipsis)
return 1;
@@ -130,9 +136,13 @@ validate_constant(PyObject *value)
return 1;
if (PyTuple_CheckExact(value) || PyFrozenSet_CheckExact(value)) {
- PyObject *it;
+ if (++state->recursion_depth > state->recursion_limit) {
+ PyErr_SetString(PyExc_RecursionError,
+ "maximum recursion depth exceeded during compilation");
+ return 0;
+ }
- it = PyObject_GetIter(value);
+ PyObject *it = PyObject_GetIter(value);
if (it == NULL)
return 0;
@@ -146,7 +156,7 @@ validate_constant(PyObject *value)
break;
}
- if (!validate_constant(item)) {
+ if (!validate_constant(state, item)) {
Py_DECREF(it);
Py_DECREF(item);
return 0;
@@ -155,6 +165,7 @@ validate_constant(PyObject *value)
}
Py_DECREF(it);
+ --state->recursion_depth;
return 1;
}
@@ -167,8 +178,14 @@ validate_constant(PyObject *value)
}
static int
-validate_expr(expr_ty exp, expr_context_ty ctx)
+validate_expr(struct validator *state, expr_ty exp, expr_context_ty ctx)
{
+ int ret;
+ if (++state->recursion_depth > state->recursion_limit) {
+ PyErr_SetString(PyExc_RecursionError,
+ "maximum recursion depth exceeded during compilation");
+ return 0;
+ }
int check_ctx = 1;
expr_context_ty actual_ctx;
@@ -218,19 +235,24 @@ validate_expr(expr_ty exp, expr_context_ty ctx)
PyErr_SetString(PyExc_ValueError, "BoolOp with less than 2 values");
return 0;
}
- return validate_exprs(exp->v.BoolOp.values, Load, 0);
+ ret = validate_exprs(state, exp->v.BoolOp.values, Load, 0);
+ break;
case BinOp_kind:
- return validate_expr(exp->v.BinOp.left, Load) &&
- validate_expr(exp->v.BinOp.right, Load);
+ ret = validate_expr(state, exp->v.BinOp.left, Load) &&
+ validate_expr(state, exp->v.BinOp.right, Load);
+ break;
case UnaryOp_kind:
- return validate_expr(exp->v.UnaryOp.operand, Load);
+ ret = validate_expr(state, exp->v.UnaryOp.operand, Load);
+ break;
case Lambda_kind:
- return validate_arguments(exp->v.Lambda.args) &&
- validate_expr(exp->v.Lambda.body, Load);
+ ret = validate_arguments(state, exp->v.Lambda.args) &&
+ validate_expr(state, exp->v.Lambda.body, Load);
+ break;
case IfExp_kind:
- return validate_expr(exp->v.IfExp.test, Load) &&
- validate_expr(exp->v.IfExp.body, Load) &&
- validate_expr(exp->v.IfExp.orelse, Load);
+ ret = validate_expr(state, exp->v.IfExp.test, Load) &&
+ validate_expr(state, exp->v.IfExp.body, Load) &&
+ validate_expr(state, exp->v.IfExp.orelse, Load);
+ break;
case Dict_kind:
if (asdl_seq_LEN(exp->v.Dict.keys) != asdl_seq_LEN(exp->v.Dict.values)) {
PyErr_SetString(PyExc_ValueError,
@@ -239,28 +261,35 @@ validate_expr(expr_ty exp, expr_context_ty ctx)
}
/* null_ok=1 for keys expressions to allow dict unpacking to work in
dict literals, i.e. ``{**{a:b}}`` */
- return validate_exprs(exp->v.Dict.keys, Load, /*null_ok=*/ 1) &&
- validate_exprs(exp->v.Dict.values, Load, /*null_ok=*/ 0);
+ ret = validate_exprs(state, exp->v.Dict.keys, Load, /*null_ok=*/ 1) &&
+ validate_exprs(state, exp->v.Dict.values, Load, /*null_ok=*/ 0);
+ break;
case Set_kind:
- return validate_exprs(exp->v.Set.elts, Load, 0);
+ ret = validate_exprs(state, exp->v.Set.elts, Load, 0);
+ break;
#define COMP(NAME) \
case NAME ## _kind: \
- return validate_comprehension(exp->v.NAME.generators) && \
- validate_expr(exp->v.NAME.elt, Load);
+ ret = validate_comprehension(state, exp->v.NAME.generators) && \
+ validate_expr(state, exp->v.NAME.elt, Load); \
+ break;
COMP(ListComp)
COMP(SetComp)
COMP(GeneratorExp)
#undef COMP
case DictComp_kind:
- return validate_comprehension(exp->v.DictComp.generators) &&
- validate_expr(exp->v.DictComp.key, Load) &&
- validate_expr(exp->v.DictComp.value, Load);
+ ret = validate_comprehension(state, exp->v.DictComp.generators) &&
+ validate_expr(state, exp->v.DictComp.key, Load) &&
+ validate_expr(state, exp->v.DictComp.value, Load);
+ break;
case Yield_kind:
- return !exp->v.Yield.value || validate_expr(exp->v.Yield.value, Load);
+ ret = !exp->v.Yield.value || validate_expr(state, exp->v.Yield.value, Load);
+ break;
case YieldFrom_kind:
- return validate_expr(exp->v.YieldFrom.value, Load);
+ ret = validate_expr(state, exp->v.YieldFrom.value, Load);
+ break;
case Await_kind:
- return validate_expr(exp->v.Await.value, Load);
+ ret = validate_expr(state, exp->v.Await.value, Load);
+ break;
case Compare_kind:
if (!asdl_seq_LEN(exp->v.Compare.comparators)) {
PyErr_SetString(PyExc_ValueError, "Compare with no comparators");
@@ -272,42 +301,56 @@ validate_expr(expr_ty exp, expr_context_ty ctx)
"of comparators and operands");
return 0;
}
- return validate_exprs(exp->v.Compare.comparators, Load, 0) &&
- validate_expr(exp->v.Compare.left, Load);
+ ret = validate_exprs(state, exp->v.Compare.comparators, Load, 0) &&
+ validate_expr(state, exp->v.Compare.left, Load);
+ break;
case Call_kind:
- return validate_expr(exp->v.Call.func, Load) &&
- validate_exprs(exp->v.Call.args, Load, 0) &&
- validate_keywords(exp->v.Call.keywords);
+ ret = validate_expr(state, exp->v.Call.func, Load) &&
+ validate_exprs(state, exp->v.Call.args, Load, 0) &&
+ validate_keywords(state, exp->v.Call.keywords);
+ break;
case Constant_kind:
- if (!validate_constant(exp->v.Constant.value)) {
+ if (!validate_constant(state, exp->v.Constant.value)) {
return 0;
}
- return 1;
+ ret = 1;
+ break;
case JoinedStr_kind:
- return validate_exprs(exp->v.JoinedStr.values, Load, 0);
+ ret = validate_exprs(state, exp->v.JoinedStr.values, Load, 0);
+ break;
case FormattedValue_kind:
- if (validate_expr(exp->v.FormattedValue.value, Load) == 0)
+ if (validate_expr(state, exp->v.FormattedValue.value, Load) == 0)
return 0;
- if (exp->v.FormattedValue.format_spec)
- return validate_expr(exp->v.FormattedValue.format_spec, Load);
- return 1;
+ if (exp->v.FormattedValue.format_spec) {
+ ret = validate_expr(state, exp->v.FormattedValue.format_spec, Load);
+ break;
+ }
+ ret = 1;
+ break;
case Attribute_kind:
- return validate_expr(exp->v.Attribute.value, Load);
+ ret = validate_expr(state, exp->v.Attribute.value, Load);
+ break;
case Subscript_kind:
- return validate_expr(exp->v.Subscript.slice, Load) &&
- validate_expr(exp->v.Subscript.value, Load);
+ ret = validate_expr(state, exp->v.Subscript.slice, Load) &&
+ validate_expr(state, exp->v.Subscript.value, Load);
+ break;
case Starred_kind:
- return validate_expr(exp->v.Starred.value, ctx);
+ ret = validate_expr(state, exp->v.Starred.value, ctx);
+ break;
case Slice_kind:
- return (!exp->v.Slice.lower || validate_expr(exp->v.Slice.lower, Load)) &&
- (!exp->v.Slice.upper || validate_expr(exp->v.Slice.upper, Load)) &&
- (!exp->v.Slice.step || validate_expr(exp->v.Slice.step, Load));
+ ret = (!exp->v.Slice.lower || validate_expr(state, exp->v.Slice.lower, Load)) &&
+ (!exp->v.Slice.upper || validate_expr(state, exp->v.Slice.upper, Load)) &&
+ (!exp->v.Slice.step || validate_expr(state, exp->v.Slice.step, Load));
+ break;
case List_kind:
- return validate_exprs(exp->v.List.elts, ctx, 0);
+ ret = validate_exprs(state, exp->v.List.elts, ctx, 0);
+ break;
case Tuple_kind:
- return validate_exprs(exp->v.Tuple.elts, ctx, 0);
+ ret = validate_exprs(state, exp->v.Tuple.elts, ctx, 0);
+ break;
case NamedExpr_kind:
- return validate_expr(exp->v.NamedExpr.value, Load);
+ ret = validate_expr(state, exp->v.NamedExpr.value, Load);
+ break;
case MatchAs_kind:
PyErr_SetString(PyExc_ValueError,
"MatchAs is only valid in match_case patterns");
@@ -318,10 +361,14 @@ validate_expr(expr_ty exp, expr_context_ty ctx)
return 0;
/* This last case doesn't have any checking. */
case Name_kind:
- return 1;
+ ret = 1;
+ break;
+ default:
+ PyErr_SetString(PyExc_SystemError, "unexpected expression");
+ return 0;
}
- PyErr_SetString(PyExc_SystemError, "unexpected expression");
- return 0;
+ state->recursion_depth--;
+ return ret;
}
static int
@@ -342,44 +389,56 @@ _validate_nonempty_seq(asdl_seq *seq, const char *what, const char *owner)
#define validate_nonempty_seq(seq, what, owner) _validate_nonempty_seq((asdl_seq*)seq, what, owner)
static int
-validate_assignlist(asdl_expr_seq *targets, expr_context_ty ctx)
+validate_assignlist(struct validator *state, asdl_expr_seq *targets, expr_context_ty ctx)
{
return validate_nonempty_seq(targets, "targets", ctx == Del ? "Delete" : "Assign") &&
- validate_exprs(targets, ctx, 0);
+ validate_exprs(state, targets, ctx, 0);
}
static int
-validate_body(asdl_stmt_seq *body, const char *owner)
+validate_body(struct validator *state, asdl_stmt_seq *body, const char *owner)
{
- return validate_nonempty_seq(body, "body", owner) && validate_stmts(body);
+ return validate_nonempty_seq(body, "body", owner) && validate_stmts(state, body);
}
static int
-validate_stmt(stmt_ty stmt)
+validate_stmt(struct validator *state, stmt_ty stmt)
{
+ int ret;
Py_ssize_t i;
+ if (++state->recursion_depth > state->recursion_limit) {
+ PyErr_SetString(PyExc_RecursionError,
+ "maximum recursion depth exceeded during compilation");
+ return 0;
+ }
switch (stmt->kind) {
case FunctionDef_kind:
- return validate_body(stmt->v.FunctionDef.body, "FunctionDef") &&
- validate_arguments(stmt->v.FunctionDef.args) &&
- validate_exprs(stmt->v.FunctionDef.decorator_list, Load, 0) &&
+ ret = validate_body(state, stmt->v.FunctionDef.body, "FunctionDef") &&
+ validate_arguments(state, stmt->v.FunctionDef.args) &&
+ validate_exprs(state, stmt->v.FunctionDef.decorator_list, Load, 0) &&
(!stmt->v.FunctionDef.returns ||
- validate_expr(stmt->v.FunctionDef.returns, Load));
+ validate_expr(state, stmt->v.FunctionDef.returns, Load));
+ break;
case ClassDef_kind:
- return validate_body(stmt->v.ClassDef.body, "ClassDef") &&
- validate_exprs(stmt->v.ClassDef.bases, Load, 0) &&
- validate_keywords(stmt->v.ClassDef.keywords) &&
- validate_exprs(stmt->v.ClassDef.decorator_list, Load, 0);
+ ret = validate_body(state, stmt->v.ClassDef.body, "ClassDef") &&
+ validate_exprs(state, stmt->v.ClassDef.bases, Load, 0) &&
+ validate_keywords(state, stmt->v.ClassDef.keywords) &&
+ validate_exprs(state, stmt->v.ClassDef.decorator_list, Load, 0);
+ break;
case Return_kind:
- return !stmt->v.Return.value || validate_expr(stmt->v.Return.value, Load);
+ ret = !stmt->v.Return.value || validate_expr(state, stmt->v.Return.value, Load);
+ break;
case Delete_kind:
- return validate_assignlist(stmt->v.Delete.targets, Del);
+ ret = validate_assignlist(state, stmt->v.Delete.targets, Del);
+ break;
case Assign_kind:
- return validate_assignlist(stmt->v.Assign.targets, Store) &&
- validate_expr(stmt->v.Assign.value, Load);
+ ret = validate_assignlist(state, stmt->v.Assign.targets, Store) &&
+ validate_expr(state, stmt->v.Assign.value, Load);
+ break;
case AugAssign_kind:
- return validate_expr(stmt->v.AugAssign.target, Store) &&
- validate_expr(stmt->v.AugAssign.value, Load);
+ ret = validate_expr(state, stmt->v.AugAssign.target, Store) &&
+ validate_expr(state, stmt->v.AugAssign.value, Load);
+ break;
case AnnAssign_kind:
if (stmt->v.AnnAssign.target->kind != Name_kind &&
stmt->v.AnnAssign.simple) {
@@ -387,74 +446,84 @@ validate_stmt(stmt_ty stmt)
"AnnAssign with simple non-Name target");
return 0;
}
- return validate_expr(stmt->v.AnnAssign.target, Store) &&
+ ret = validate_expr(state, stmt->v.AnnAssign.target, Store) &&
(!stmt->v.AnnAssign.value ||
- validate_expr(stmt->v.AnnAssign.value, Load)) &&
- validate_expr(stmt->v.AnnAssign.annotation, Load);
+ validate_expr(state, stmt->v.AnnAssign.value, Load)) &&
+ validate_expr(state, stmt->v.AnnAssign.annotation, Load);
+ break;
case For_kind:
- return validate_expr(stmt->v.For.target, Store) &&
- validate_expr(stmt->v.For.iter, Load) &&
- validate_body(stmt->v.For.body, "For") &&
- validate_stmts(stmt->v.For.orelse);
+ ret = validate_expr(state, stmt->v.For.target, Store) &&
+ validate_expr(state, stmt->v.For.iter, Load) &&
+ validate_body(state, stmt->v.For.body, "For") &&
+ validate_stmts(state, stmt->v.For.orelse);
+ break;
case AsyncFor_kind:
- return validate_expr(stmt->v.AsyncFor.target, Store) &&
- validate_expr(stmt->v.AsyncFor.iter, Load) &&
- validate_body(stmt->v.AsyncFor.body, "AsyncFor") &&
- validate_stmts(stmt->v.AsyncFor.orelse);
+ ret = validate_expr(state, stmt->v.AsyncFor.target, Store) &&
+ validate_expr(state, stmt->v.AsyncFor.iter, Load) &&
+ validate_body(state, stmt->v.AsyncFor.body, "AsyncFor") &&
+ validate_stmts(state, stmt->v.AsyncFor.orelse);
+ break;
case While_kind:
- return validate_expr(stmt->v.While.test, Load) &&
- validate_body(stmt->v.While.body, "While") &&
- validate_stmts(stmt->v.While.orelse);
+ ret = validate_expr(state, stmt->v.While.test, Load) &&
+ validate_body(state, stmt->v.While.body, "While") &&
+ validate_stmts(state, stmt->v.While.orelse);
+ break;
case If_kind:
- return validate_expr(stmt->v.If.test, Load) &&
- validate_body(stmt->v.If.body, "If") &&
- validate_stmts(stmt->v.If.orelse);
+ ret = validate_expr(state, stmt->v.If.test, Load) &&
+ validate_body(state, stmt->v.If.body, "If") &&
+ validate_stmts(state, stmt->v.If.orelse);
+ break;
case With_kind:
if (!validate_nonempty_seq(stmt->v.With.items, "items", "With"))
return 0;
for (i = 0; i < asdl_seq_LEN(stmt->v.With.items); i++) {
withitem_ty item = asdl_seq_GET(stmt->v.With.items, i);
- if (!validate_expr(item->context_expr, Load) ||
- (item->optional_vars && !validate_expr(item->optional_vars, Store)))
+ if (!validate_expr(state, item->context_expr, Load) ||
+ (item->optional_vars && !validate_expr(state, item->optional_vars, Store)))
return 0;
}
- return validate_body(stmt->v.With.body, "With");
+ ret = validate_body(state, stmt->v.With.body, "With");
+ break;
case AsyncWith_kind:
if (!validate_nonempty_seq(stmt->v.AsyncWith.items, "items", "AsyncWith"))
return 0;
for (i = 0; i < asdl_seq_LEN(stmt->v.AsyncWith.items); i++) {
withitem_ty item = asdl_seq_GET(stmt->v.AsyncWith.items, i);
- if (!validate_expr(item->context_expr, Load) ||
- (item->optional_vars && !validate_expr(item->optional_vars, Store)))
+ if (!validate_expr(state, item->context_expr, Load) ||
+ (item->optional_vars && !validate_expr(state, item->optional_vars, Store)))
return 0;
}
- return validate_body(stmt->v.AsyncWith.body, "AsyncWith");
+ ret = validate_body(state, stmt->v.AsyncWith.body, "AsyncWith");
+ break;
case Match_kind:
- if (!validate_expr(stmt->v.Match.subject, Load)
+ if (!validate_expr(state, stmt->v.Match.subject, Load)
|| !validate_nonempty_seq(stmt->v.Match.cases, "cases", "Match")) {
return 0;
}
for (i = 0; i < asdl_seq_LEN(stmt->v.Match.cases); i++) {
match_case_ty m = asdl_seq_GET(stmt->v.Match.cases, i);
if (!validate_pattern(m->pattern)
- || (m->guard && !validate_expr(m->guard, Load))
- || !validate_body(m->body, "match_case")) {
+ || (m->guard && !validate_expr(state, m->guard, Load))
+ || !validate_body(state, m->body, "match_case")) {
return 0;
}
}
- return 1;
+ ret = 1;
+ break;
case Raise_kind:
if (stmt->v.Raise.exc) {
- return validate_expr(stmt->v.Raise.exc, Load) &&
- (!stmt->v.Raise.cause || validate_expr(stmt->v.Raise.cause, Load));
+ ret = validate_expr(state, stmt->v.Raise.exc, Load) &&
+ (!stmt->v.Raise.cause || validate_expr(state, stmt->v.Raise.cause, Load));
+ break;
}
if (stmt->v.Raise.cause) {
PyErr_SetString(PyExc_ValueError, "Raise with cause but no exception");
return 0;
}
- return 1;
+ ret = 1;
+ break;
case Try_kind:
- if (!validate_body(stmt->v.Try.body, "Try"))
+ if (!validate_body(state, stmt->v.Try.body, "Try"))
return 0;
if (!asdl_seq_LEN(stmt->v.Try.handlers) &&
!asdl_seq_LEN(stmt->v.Try.finalbody)) {
@@ -469,55 +538,66 @@ validate_stmt(stmt_ty stmt)
for (i = 0; i < asdl_seq_LEN(stmt->v.Try.handlers); i++) {
excepthandler_ty handler = asdl_seq_GET(stmt->v.Try.handlers, i);
if ((handler->v.ExceptHandler.type &&
- !validate_expr(handler->v.ExceptHandler.type, Load)) ||
- !validate_body(handler->v.ExceptHandler.body, "ExceptHandler"))
+ !validate_expr(state, handler->v.ExceptHandler.type, Load)) ||
+ !validate_body(state, handler->v.ExceptHandler.body, "ExceptHandler"))
return 0;
}
- return (!asdl_seq_LEN(stmt->v.Try.finalbody) ||
- validate_stmts(stmt->v.Try.finalbody)) &&
+ ret = (!asdl_seq_LEN(stmt->v.Try.finalbody) ||
+ validate_stmts(state, stmt->v.Try.finalbody)) &&
(!asdl_seq_LEN(stmt->v.Try.orelse) ||
- validate_stmts(stmt->v.Try.orelse));
+ validate_stmts(state, stmt->v.Try.orelse));
+ break;
case Assert_kind:
- return validate_expr(stmt->v.Assert.test, Load) &&
- (!stmt->v.Assert.msg || validate_expr(stmt->v.Assert.msg, Load));
+ ret = validate_expr(state, stmt->v.Assert.test, Load) &&
+ (!stmt->v.Assert.msg || validate_expr(state, stmt->v.Assert.msg, Load));
+ break;
case Import_kind:
- return validate_nonempty_seq(stmt->v.Import.names, "names", "Import");
+ ret = validate_nonempty_seq(stmt->v.Import.names, "names", "Import");
+ break;
case ImportFrom_kind:
if (stmt->v.ImportFrom.level < 0) {
PyErr_SetString(PyExc_ValueError, "Negative ImportFrom level");
return 0;
}
- return validate_nonempty_seq(stmt->v.ImportFrom.names, "names", "ImportFrom");
+ ret = validate_nonempty_seq(stmt->v.ImportFrom.names, "names", "ImportFrom");
+ break;
case Global_kind:
- return validate_nonempty_seq(stmt->v.Global.names, "names", "Global");
+ ret = validate_nonempty_seq(stmt->v.Global.names, "names", "Global");
+ break;
case Nonlocal_kind:
- return validate_nonempty_seq(stmt->v.Nonlocal.names, "names", "Nonlocal");
+ ret = validate_nonempty_seq(stmt->v.Nonlocal.names, "names", "Nonlocal");
+ break;
case Expr_kind:
- return validate_expr(stmt->v.Expr.value, Load);
+ ret = validate_expr(state, stmt->v.Expr.value, Load);
+ break;
case AsyncFunctionDef_kind:
- return validate_body(stmt->v.AsyncFunctionDef.body, "AsyncFunctionDef") &&
- validate_arguments(stmt->v.AsyncFunctionDef.args) &&
- validate_exprs(stmt->v.AsyncFunctionDef.decorator_list, Load, 0) &&
+ ret = validate_body(state, stmt->v.AsyncFunctionDef.body, "AsyncFunctionDef") &&
+ validate_arguments(state, stmt->v.AsyncFunctionDef.args) &&
+ validate_exprs(state, stmt->v.AsyncFunctionDef.decorator_list, Load, 0) &&
(!stmt->v.AsyncFunctionDef.returns ||
- validate_expr(stmt->v.AsyncFunctionDef.returns, Load));
+ validate_expr(state, stmt->v.AsyncFunctionDef.returns, Load));
+ break;
case Pass_kind:
case Break_kind:
case Continue_kind:
- return 1;
+ ret = 1;
+ break;
default:
PyErr_SetString(PyExc_SystemError, "unexpected statement");
return 0;
}
+ state->recursion_depth--;
+ return ret;
}
static int
-validate_stmts(asdl_stmt_seq *seq)
+validate_stmts(struct validator *state, asdl_stmt_seq *seq)
{
Py_ssize_t i;
for (i = 0; i < asdl_seq_LEN(seq); i++) {
stmt_ty stmt = asdl_seq_GET(seq, i);
if (stmt) {
- if (!validate_stmt(stmt))
+ if (!validate_stmt(state, stmt))
return 0;
}
else {
@@ -530,13 +610,13 @@ validate_stmts(asdl_stmt_seq *seq)
}
static int
-validate_exprs(asdl_expr_seq *exprs, expr_context_ty ctx, int null_ok)
+validate_exprs(struct validator *state, asdl_expr_seq *exprs, expr_context_ty ctx, int null_ok)
{
Py_ssize_t i;
for (i = 0; i < asdl_seq_LEN(exprs); i++) {
expr_ty expr = asdl_seq_GET(exprs, i);
if (expr) {
- if (!validate_expr(expr, ctx))
+ if (!validate_expr(state, expr, ctx))
return 0;
}
else if (!null_ok) {
@@ -549,26 +629,53 @@ validate_exprs(asdl_expr_seq *exprs, expr_context_ty ctx, int null_ok)
return 1;
}
+/* See comments in symtable.c. */
+#define COMPILER_STACK_FRAME_SCALE 3
+
int
_PyAST_Validate(mod_ty mod)
{
int res = 0;
+ struct validator state;
+ PyThreadState *tstate;
+ int recursion_limit = Py_GetRecursionLimit();
+ int starting_recursion_depth;
+
+ /* Setup recursion depth check counters */
+ tstate = _PyThreadState_GET();
+ if (!tstate) {
+ return 0;
+ }
+ /* Be careful here to prevent overflow. */
+ starting_recursion_depth = (tstate->recursion_depth < INT_MAX / COMPILER_STACK_FRAME_SCALE) ?
+ tstate->recursion_depth * COMPILER_STACK_FRAME_SCALE : tstate->recursion_depth;
+ state.recursion_depth = starting_recursion_depth;
+ state.recursion_limit = (recursion_limit < INT_MAX / COMPILER_STACK_FRAME_SCALE) ?
+ recursion_limit * COMPILER_STACK_FRAME_SCALE : recursion_limit;
switch (mod->kind) {
case Module_kind:
- res = validate_stmts(mod->v.Module.body);
+ res = validate_stmts(&state, mod->v.Module.body);
break;
case Interactive_kind:
- res = validate_stmts(mod->v.Interactive.body);
+ res = validate_stmts(&state, mod->v.Interactive.body);
break;
case Expression_kind:
- res = validate_expr(mod->v.Expression.body, Load);
+ res = validate_expr(&state, mod->v.Expression.body, Load);
break;
default:
PyErr_SetString(PyExc_SystemError, "impossible module node");
res = 0;
break;
}
+
+ /* Check that the recursion depth counting balanced correctly */
+ if (res && state.recursion_depth != starting_recursion_depth) {
+ PyErr_Format(PyExc_SystemError,
+ "AST validator recursion depth mismatch (before=%d, after=%d)",
+ starting_recursion_depth, state.recursion_depth);
+ return 0;
+ }
return res;
}
diff --git a/Python/ast_opt.c b/Python/ast_opt.c
index dea20da07e69d..6eb514e24f9d7 100644
--- a/Python/ast_opt.c
+++ b/Python/ast_opt.c
@@ -2,6 +2,7 @@
#include "Python.h"
#include "pycore_ast.h" // _PyAST_GetDocString()
#include "pycore_compile.h" // _PyASTOptimizeState
+#include "pycore_pystate.h" // _PyThreadState_GET()
static int
@@ -488,6 +489,11 @@ astfold_mod(mod_ty node_, PyArena *ctx_, _PyASTOptimizeState *state)
static int
astfold_expr(expr_ty node_, PyArena *ctx_, _PyASTOptimizeState *state)
{
+ if (++state->recursion_depth > state->recursion_limit) {
+ PyErr_SetString(PyExc_RecursionError,
+ "maximum recursion depth exceeded during compilation");
+ return 0;
+ }
switch (node_->kind) {
case BoolOp_kind:
CALL_SEQ(astfold_expr, expr, node_->v.BoolOp.values);
@@ -586,6 +592,7 @@ astfold_expr(expr_ty node_, PyArena *ctx_, _PyASTOptimizeState *state)
case Name_kind:
if (node_->v.Name.ctx == Load &&
_PyUnicode_EqualToASCIIString(node_->v.Name.id, "__debug__")) {
+ state->recursion_depth--;
return make_const(node_, PyBool_FromLong(!state->optimize), ctx_);
}
break;
@@ -602,6 +609,7 @@ astfold_expr(expr_ty node_, PyArena *ctx_, _PyASTOptimizeState *state)
// No default case, so the compiler will emit a warning if new expression
// kinds are added without being handled here
}
+ state->recursion_depth--;
return 1;
}
@@ -648,6 +656,11 @@ astfold_arg(arg_ty node_, PyArena *ctx_, _PyASTOptimizeState *state)
static int
astfold_stmt(stmt_ty node_, PyArena *ctx_, _PyASTOptimizeState *state)
{
+ if (++state->recursion_depth > state->recursion_limit) {
+ PyErr_SetString(PyExc_RecursionError,
+ "maximum recursion depth exceeded during compilation");
+ return 0;
+ }
switch (node_->kind) {
case FunctionDef_kind:
CALL(astfold_arguments, arguments_ty, node_->v.FunctionDef.args);
@@ -757,6 +770,7 @@ astfold_stmt(stmt_ty node_, PyArena *ctx_, _PyASTOptimizeState *state)
// No default case, so the compiler will emit a warning if new statement
// kinds are added without being handled here
}
+ state->recursion_depth--;
return 1;
}
@@ -906,10 +920,38 @@ astfold_match_case(match_case_ty node_, PyArena *ctx_, _PyASTOptimizeState *stat
#undef CALL_SEQ
#undef CALL_INT_SEQ
+/* See comments in symtable.c. */
+#define COMPILER_STACK_FRAME_SCALE 3
+
int
_PyAST_Optimize(mod_ty mod, PyArena *arena, _PyASTOptimizeState *state)
{
+ PyThreadState *tstate;
+ int recursion_limit = Py_GetRecursionLimit();
+ int starting_recursion_depth;
+
+ /* Setup recursion depth check counters */
+ tstate = _PyThreadState_GET();
+ if (!tstate) {
+ return 0;
+ }
+ /* Be careful here to prevent overflow. */
+ starting_recursion_depth = (tstate->recursion_depth < INT_MAX / COMPILER_STACK_FRAME_SCALE) ?
+ tstate->recursion_depth * COMPILER_STACK_FRAME_SCALE : tstate->recursion_depth;
+ state->recursion_depth = starting_recursion_depth;
+ state->recursion_limit = (recursion_limit < INT_MAX / COMPILER_STACK_FRAME_SCALE) ?
+ recursion_limit * COMPILER_STACK_FRAME_SCALE : recursion_limit;
+
int ret = astfold_mod(mod, arena, state);
assert(ret || PyErr_Occurred());
+
+ /* Check that the recursion depth counting balanced correctly */
+ if (ret && state->recursion_depth != starting_recursion_depth) {
+ PyErr_Format(PyExc_SystemError,
+ "AST optimizer recursion depth mismatch (before=%d, after=%d)",
+ starting_recursion_depth, state->recursion_depth);
+ return 0;
+ }
+
return ret;
}
1
0
bpo-43534: Make dialogs in turtle.textinput() and turtle.numinput() transitient again (GH-24923)
by serhiy-storchaka 25 Apr '21
by serhiy-storchaka 25 Apr '21
25 Apr '21
https://github.com/python/cpython/commit/b5adc8a7e5c13d175b4d3e53b37bc61de3…
commit: b5adc8a7e5c13d175b4d3e53b37bc61de35b1457
branch: master
author: Serhiy Storchaka <storchaka(a)gmail.com>
committer: serhiy-storchaka <storchaka(a)gmail.com>
date: 2021-04-25T13:16:49+03:00
summary:
bpo-43534: Make dialogs in turtle.textinput() and turtle.numinput() transitient again (GH-24923)
files:
A Misc/NEWS.d/next/Library/2021-03-18-15-46-08.bpo-43534.vPE9Us.rst
M Lib/turtle.py
diff --git a/Lib/turtle.py b/Lib/turtle.py
index 81cfcfe8a7014..11429a62d9477 100644
--- a/Lib/turtle.py
+++ b/Lib/turtle.py
@@ -826,7 +826,7 @@ def textinput(self, title, prompt):
>>> screen.textinput("NIM", "Name of first player:")
"""
- return simpledialog.askstring(title, prompt)
+ return simpledialog.askstring(title, prompt, parent=self.cv)
def numinput(self, title, prompt, default=None, minval=None, maxval=None):
"""Pop up a dialog window for input of a number.
@@ -847,7 +847,8 @@ def numinput(self, title, prompt, default=None, minval=None, maxval=None):
"""
return simpledialog.askfloat(title, prompt, initialvalue=default,
- minvalue=minval, maxvalue=maxval)
+ minvalue=minval, maxvalue=maxval,
+ parent=self.cv)
##############################################################################
diff --git a/Misc/NEWS.d/next/Library/2021-03-18-15-46-08.bpo-43534.vPE9Us.rst b/Misc/NEWS.d/next/Library/2021-03-18-15-46-08.bpo-43534.vPE9Us.rst
new file mode 100644
index 0000000000000..7f2e5a46add03
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2021-03-18-15-46-08.bpo-43534.vPE9Us.rst
@@ -0,0 +1,2 @@
+:func:`turtle.textinput` and :func:`turtle.numinput` create now a transient
+window working on behalf of the canvas window.
1
0
bpo-43655: Tkinter and IDLE dialog windows are now recognized as dialogs by window managers on macOS and X Window (#25187)
by serhiy-storchaka 25 Apr '21
by serhiy-storchaka 25 Apr '21
25 Apr '21
https://github.com/python/cpython/commit/3bb3fb3be09d472a43cdc3d9d9578bd49f…
commit: 3bb3fb3be09d472a43cdc3d9d9578bd49f3dfb8c
branch: master
author: Serhiy Storchaka <storchaka(a)gmail.com>
committer: serhiy-storchaka <storchaka(a)gmail.com>
date: 2021-04-25T13:07:58+03:00
summary:
bpo-43655: Tkinter and IDLE dialog windows are now recognized as dialogs by window managers on macOS and X Window (#25187)
files:
A Misc/NEWS.d/next/IDLE/2021-04-04-20-52-07.bpo-43655.HSyaKH.rst
A Misc/NEWS.d/next/Library/2021-04-04-20-51-19.bpo-43655.LwGy8R.rst
M Lib/idlelib/config_key.py
M Lib/idlelib/query.py
M Lib/idlelib/searchbase.py
M Lib/tkinter/filedialog.py
M Lib/tkinter/simpledialog.py
diff --git a/Lib/idlelib/config_key.py b/Lib/idlelib/config_key.py
index 7510aa9f3d878..9ca3a156f4b97 100644
--- a/Lib/idlelib/config_key.py
+++ b/Lib/idlelib/config_key.py
@@ -4,6 +4,7 @@
from tkinter import Toplevel, Listbox, StringVar, TclError
from tkinter.ttk import Frame, Button, Checkbutton, Entry, Label, Scrollbar
from tkinter import messagebox
+from tkinter.simpledialog import _setup_dialog
import string
import sys
@@ -63,6 +64,7 @@ def __init__(self, parent, title, action, current_key_sequences,
self.resizable(height=False, width=False)
self.title(title)
self.transient(parent)
+ _setup_dialog(self)
self.grab_set()
self.protocol("WM_DELETE_WINDOW", self.cancel)
self.parent = parent
diff --git a/Lib/idlelib/query.py b/Lib/idlelib/query.py
index 015fc7ade459d..fefa5aac1b7f5 100644
--- a/Lib/idlelib/query.py
+++ b/Lib/idlelib/query.py
@@ -28,6 +28,7 @@
from tkinter.ttk import Frame, Button, Entry, Label, Checkbutton
from tkinter import filedialog
from tkinter.font import Font
+from tkinter.simpledialog import _setup_dialog
class Query(Toplevel):
"""Base class for getting verified answer from a user.
@@ -60,13 +61,8 @@ def __init__(self, parent, title, message, *, text0='', used_names={},
if not _utest: # Otherwise fail when directly run unittest.
self.grab_set()
- windowingsystem = self.tk.call('tk', 'windowingsystem')
- if windowingsystem == 'aqua':
- try:
- self.tk.call('::tk::unsupported::MacWindowStyle', 'style',
- self._w, 'moveableModal', '')
- except:
- pass
+ _setup_dialog(self)
+ if self._windowingsystem == 'aqua':
self.bind("<Command-.>", self.cancel)
self.bind('<Key-Escape>', self.cancel)
self.protocol("WM_DELETE_WINDOW", self.cancel)
diff --git a/Lib/idlelib/searchbase.py b/Lib/idlelib/searchbase.py
index fbef87aa2d3d0..64ed50c7364be 100644
--- a/Lib/idlelib/searchbase.py
+++ b/Lib/idlelib/searchbase.py
@@ -2,6 +2,7 @@
from tkinter import Toplevel
from tkinter.ttk import Frame, Entry, Label, Button, Checkbutton, Radiobutton
+from tkinter.simpledialog import _setup_dialog
class SearchDialogBase:
@@ -83,6 +84,7 @@ def create_widgets(self):
top.protocol("WM_DELETE_WINDOW", self.close)
top.wm_title(self.title)
top.wm_iconname(self.icon)
+ _setup_dialog(top)
self.top = top
self.frame = Frame(top, padding="5px")
self.frame.grid(sticky="nwes")
diff --git a/Lib/tkinter/filedialog.py b/Lib/tkinter/filedialog.py
index 3ed93eb8c163a..600d0bd49fe2c 100644
--- a/Lib/tkinter/filedialog.py
+++ b/Lib/tkinter/filedialog.py
@@ -24,6 +24,7 @@
)
from tkinter.dialog import Dialog
from tkinter import commondialog
+from tkinter.simpledialog import _setup_dialog
dialogstates = {}
@@ -62,6 +63,7 @@ def __init__(self, master, title=None):
self.top = Toplevel(master)
self.top.title(title)
self.top.iconname(title)
+ _setup_dialog(self.top)
self.botframe = Frame(self.top)
self.botframe.pack(side=BOTTOM, fill=X)
diff --git a/Lib/tkinter/simpledialog.py b/Lib/tkinter/simpledialog.py
index a66fbd6cb9885..538bbfc318d70 100644
--- a/Lib/tkinter/simpledialog.py
+++ b/Lib/tkinter/simpledialog.py
@@ -40,6 +40,9 @@ def __init__(self, master,
if title:
self.root.title(title)
self.root.iconname(title)
+
+ _setup_dialog(self.root)
+
self.message = Message(self.root, text=text, aspect=400)
self.message.pack(expand=1, fill=BOTH)
self.frame = Frame(self.root)
@@ -115,6 +118,8 @@ def __init__(self, parent, title = None):
if title:
self.title(title)
+ _setup_dialog(self)
+
self.parent = parent
self.result = None
@@ -252,6 +257,13 @@ def _place_window(w, parent=None):
w.wm_deiconify() # Become visible at the desired location
+def _setup_dialog(w):
+ if w._windowingsystem == "aqua":
+ w.tk.call("::tk::unsupported::MacWindowStyle", "style",
+ w, "moveableModal", "")
+ elif w._windowingsystem == "x11":
+ w.wm_attributes("-type", "dialog")
+
# --------------------------------------------------------------------
# convenience dialogues
diff --git a/Misc/NEWS.d/next/IDLE/2021-04-04-20-52-07.bpo-43655.HSyaKH.rst b/Misc/NEWS.d/next/IDLE/2021-04-04-20-52-07.bpo-43655.HSyaKH.rst
new file mode 100644
index 0000000000000..105ec9281f005
--- /dev/null
+++ b/Misc/NEWS.d/next/IDLE/2021-04-04-20-52-07.bpo-43655.HSyaKH.rst
@@ -0,0 +1,2 @@
+IDLE dialog windows are now recognized as dialogs by window managers on
+macOS and X Window.
diff --git a/Misc/NEWS.d/next/Library/2021-04-04-20-51-19.bpo-43655.LwGy8R.rst b/Misc/NEWS.d/next/Library/2021-04-04-20-51-19.bpo-43655.LwGy8R.rst
new file mode 100644
index 0000000000000..7916d2248b231
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2021-04-04-20-51-19.bpo-43655.LwGy8R.rst
@@ -0,0 +1,2 @@
+:mod:`tkinter` dialog windows are now recognized as dialogs by window
+managers on macOS and X Window.
1
0
bpo-42737: annotations with complex targets no longer causes any runtime effects (GH-23952)
by isidentical 24 Apr '21
by isidentical 24 Apr '21
24 Apr '21
https://github.com/python/cpython/commit/8cc3cfa8afab1651c4f6e9ba43a7ab7f10…
commit: 8cc3cfa8afab1651c4f6e9ba43a7ab7f10f64c32
branch: master
author: Batuhan Taskaya <isidentical(a)gmail.com>
committer: isidentical <isidentical(a)gmail.com>
date: 2021-04-25T05:31:20+03:00
summary:
bpo-42737: annotations with complex targets no longer causes any runtime effects (GH-23952)
files:
A Misc/NEWS.d/next/Core and Builtins/2021-04-22-22-48-30.bpo-42737.lsJ7pD.rst
M Doc/whatsnew/3.10.rst
M Lib/test/test_future.py
M Python/compile.c
diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst
index 78f3c2d36b845..dac44cf03fa32 100644
--- a/Doc/whatsnew/3.10.rst
+++ b/Doc/whatsnew/3.10.rst
@@ -791,6 +791,10 @@ Other Language Changes
Moreover, static methods are now callable as regular functions.
(Contributed by Victor Stinner in :issue:`43682`.)
+* Annotations for complex targets (everything beside ``simple name`` targets
+ defined by :pep:`526`) no longer cause any runtime effects with ``from __future__ import annotations``.
+ (Contributed by Batuhan Taskaya in :issue:`42737`.)
+
New Modules
===========
diff --git a/Lib/test/test_future.py b/Lib/test/test_future.py
index e4715587d21cf..8a09853d1f78f 100644
--- a/Lib/test/test_future.py
+++ b/Lib/test/test_future.py
@@ -134,8 +134,12 @@ async def f2() -> {ann}:
...
async def g2(arg: {ann}) -> None:
...
+ class H:
+ var: {ann}
+ object.attr: {ann}
var: {ann}
var2: {ann} = None
+ object.attr: {ann}
"""
)
@@ -343,6 +347,13 @@ def test_infinity_numbers(self):
self.assertAnnotationEqual("('inf', 1e1000, 'infxxx', 1e1000j)", expected=f"('inf', {inf}, 'infxxx', {infj})")
self.assertAnnotationEqual("(1e1000, (1e1000j,))", expected=f"({inf}, ({infj},))")
+ def test_annotation_with_complex_target(self):
+ with self.assertRaises(SyntaxError):
+ exec(
+ "from __future__ import annotations\n"
+ "object.__debug__: int"
+ )
+
if __name__ == "__main__":
unittest.main()
diff --git a/Misc/NEWS.d/next/Core and Builtins/2021-04-22-22-48-30.bpo-42737.lsJ7pD.rst b/Misc/NEWS.d/next/Core and Builtins/2021-04-22-22-48-30.bpo-42737.lsJ7pD.rst
new file mode 100644
index 0000000000000..e55db436896af
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2021-04-22-22-48-30.bpo-42737.lsJ7pD.rst
@@ -0,0 +1,2 @@
+Annotations for complex targets (everything beside simple names) no longer
+cause any runtime effects with ``from __future__ import annotations``.
diff --git a/Python/compile.c b/Python/compile.c
index 1b7a2e83b16ba..2cf2f4a382457 100644
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -5356,6 +5356,12 @@ check_ann_expr(struct compiler *c, expr_ty e)
static int
check_annotation(struct compiler *c, stmt_ty s)
{
+ /* Annotations of complex targets does not produce anything
+ under annotations future */
+ if (c->c_future->ff_features & CO_FUTURE_ANNOTATIONS) {
+ return 1;
+ }
+
/* Annotations are only evaluated in a module or class. */
if (c->u->u_scope_type == COMPILER_SCOPE_MODULE ||
c->u->u_scope_type == COMPILER_SCOPE_CLASS) {
1
0
[3.8] bpo-43930: Update bundled pip to 21.1 and setuptools to 56.0.0 (GH-25576) (GH-25579)
by pfmoore 24 Apr '21
by pfmoore 24 Apr '21
24 Apr '21
https://github.com/python/cpython/commit/fc82f3f8fb36f88a4e7238a463812c2916…
commit: fc82f3f8fb36f88a4e7238a463812c2916bd4db0
branch: 3.8
author: Stéphane Bidoul <stephane.bidoul(a)acsone.eu>
committer: pfmoore <p.f.moore(a)gmail.com>
date: 2021-04-24T23:28:55+01:00
summary:
[3.8] bpo-43930: Update bundled pip to 21.1 and setuptools to 56.0.0 (GH-25576) (GH-25579)
Update bundled pip to 21.1 and setuptools to 56.0.0.
(cherry picked from commit 196983563d05e32d2dcf217e955a919f9e0c25e1)
Co-authored-by: Stéphane Bidoul <stephane.bidoul(a)acsone.eu>
files:
A Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl
A Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl
A Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst
D Lib/ensurepip/_bundled/pip-20.2.3-py2.py3-none-any.whl
D Lib/ensurepip/_bundled/setuptools-49.2.1-py3-none-any.whl
M Lib/ensurepip/__init__.py
diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
index 38bb42104b550f..d77987011d3a8b 100644
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -9,14 +9,13 @@
__all__ = ["version", "bootstrap"]
+_SETUPTOOLS_VERSION = "56.0.0"
-_SETUPTOOLS_VERSION = "49.2.1"
-
-_PIP_VERSION = "20.2.3"
+_PIP_VERSION = "21.1"
_PROJECTS = [
("setuptools", _SETUPTOOLS_VERSION, "py3"),
- ("pip", _PIP_VERSION, "py2.py3"),
+ ("pip", _PIP_VERSION, "py3"),
]
diff --git a/Lib/ensurepip/_bundled/pip-20.2.3-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl
similarity index 51%
rename from Lib/ensurepip/_bundled/pip-20.2.3-py2.py3-none-any.whl
rename to Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl
index 7ebdc0f31d4e3e..59395be72f135f 100644
Binary files a/Lib/ensurepip/_bundled/pip-20.2.3-py2.py3-none-any.whl and b/Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl differ
diff --git a/Lib/ensurepip/_bundled/setuptools-49.2.1-py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl
similarity index 66%
rename from Lib/ensurepip/_bundled/setuptools-49.2.1-py3-none-any.whl
rename to Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl
index 308e2f2ed5ed9b..264ef10e826679 100644
Binary files a/Lib/ensurepip/_bundled/setuptools-49.2.1-py3-none-any.whl and b/Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl differ
diff --git a/Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst b/Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst
new file mode 100644
index 00000000000000..7dac21f3d9b197
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst
@@ -0,0 +1 @@
+Update bundled pip to 21.1 and setuptools to 56.0.0
\ No newline at end of file
1
0
[3.9] bpo-43930: Update bundled pip to 21.1 and setuptools to 56.0.0 (GH-25578)
by pfmoore 24 Apr '21
by pfmoore 24 Apr '21
24 Apr '21
https://github.com/python/cpython/commit/d962b00fcffa9070acdca850753f254828…
commit: d962b00fcffa9070acdca850753f254828caa1d7
branch: 3.9
author: Stéphane Bidoul <stephane.bidoul(a)acsone.eu>
committer: pfmoore <p.f.moore(a)gmail.com>
date: 2021-04-24T23:27:44+01:00
summary:
[3.9] bpo-43930: Update bundled pip to 21.1 and setuptools to 56.0.0 (GH-25578)
(cherry picked from commit 196983563d05e32d2dcf217e955a919f9e0c25e1)
files:
A Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl
A Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl
A Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst
D Lib/ensurepip/_bundled/pip-20.2.3-py2.py3-none-any.whl
D Lib/ensurepip/_bundled/setuptools-49.2.1-py3-none-any.whl
M Lib/ensurepip/__init__.py
diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
index 97dfa7ea71f8ff..783b3232c80fc4 100644
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -13,13 +13,13 @@
__all__ = ["version", "bootstrap"]
-_SETUPTOOLS_VERSION = "49.2.1"
+_SETUPTOOLS_VERSION = "56.0.0"
-_PIP_VERSION = "20.2.3"
+_PIP_VERSION = "21.1"
_PROJECTS = [
("setuptools", _SETUPTOOLS_VERSION, "py3"),
- ("pip", _PIP_VERSION, "py2.py3"),
+ ("pip", _PIP_VERSION, "py3"),
]
diff --git a/Lib/ensurepip/_bundled/pip-20.2.3-py2.py3-none-any.whl b/Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl
similarity index 51%
rename from Lib/ensurepip/_bundled/pip-20.2.3-py2.py3-none-any.whl
rename to Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl
index 7ebdc0f31d4e3e..59395be72f135f 100644
Binary files a/Lib/ensurepip/_bundled/pip-20.2.3-py2.py3-none-any.whl and b/Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl differ
diff --git a/Lib/ensurepip/_bundled/setuptools-49.2.1-py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl
similarity index 66%
rename from Lib/ensurepip/_bundled/setuptools-49.2.1-py3-none-any.whl
rename to Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl
index 308e2f2ed5ed9b..264ef10e826679 100644
Binary files a/Lib/ensurepip/_bundled/setuptools-49.2.1-py3-none-any.whl and b/Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl differ
diff --git a/Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst b/Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst
new file mode 100644
index 00000000000000..7dac21f3d9b197
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst
@@ -0,0 +1 @@
+Update bundled pip to 21.1 and setuptools to 56.0.0
\ No newline at end of file
1
0
24 Apr '21
https://github.com/python/cpython/commit/196983563d05e32d2dcf217e955a919f9e…
commit: 196983563d05e32d2dcf217e955a919f9e0c25e1
branch: master
author: Stéphane Bidoul <stephane.bidoul(a)acsone.eu>
committer: pfmoore <p.f.moore(a)gmail.com>
date: 2021-04-24T17:21:50+01:00
summary:
bpo-43930: Update bundled pip to 21.1 and setuptools to 56.0.0 (GH-25576)
Update bundled pip to 21.1 and setuptools to 56.0.0
files:
A Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl
A Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl
A Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst
D Lib/ensurepip/_bundled/pip-21.0.1-py3-none-any.whl
D Lib/ensurepip/_bundled/setuptools-52.0.0-py3-none-any.whl
M Lib/ensurepip/__init__.py
diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
index 0f1df6e8a839d1..93d8ed5bdc8581 100644
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -11,8 +11,8 @@
__all__ = ["version", "bootstrap"]
_PACKAGE_NAMES = ('setuptools', 'pip')
-_SETUPTOOLS_VERSION = "52.0.0"
-_PIP_VERSION = "21.0.1"
+_SETUPTOOLS_VERSION = "56.0.0"
+_PIP_VERSION = "21.1"
_PROJECTS = [
("setuptools", _SETUPTOOLS_VERSION, "py3"),
("pip", _PIP_VERSION, "py3"),
diff --git a/Lib/ensurepip/_bundled/pip-21.0.1-py3-none-any.whl b/Lib/ensurepip/_bundled/pip-21.0.1-py3-none-any.whl
deleted file mode 100644
index 6d55137b941255..00000000000000
Binary files a/Lib/ensurepip/_bundled/pip-21.0.1-py3-none-any.whl and /dev/null differ
diff --git a/Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl b/Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl
new file mode 100644
index 00000000000000..59395be72f135f
Binary files /dev/null and b/Lib/ensurepip/_bundled/pip-21.1-py3-none-any.whl differ
diff --git a/Lib/ensurepip/_bundled/setuptools-52.0.0-py3-none-any.whl b/Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl
similarity index 91%
rename from Lib/ensurepip/_bundled/setuptools-52.0.0-py3-none-any.whl
rename to Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl
index 9eb776ecfba275..264ef10e826679 100644
Binary files a/Lib/ensurepip/_bundled/setuptools-52.0.0-py3-none-any.whl and b/Lib/ensurepip/_bundled/setuptools-56.0.0-py3-none-any.whl differ
diff --git a/Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst b/Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst
new file mode 100644
index 00000000000000..7dac21f3d9b197
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2021-04-24-14-23-07.bpo-43930.R7ah0m.rst
@@ -0,0 +1 @@
+Update bundled pip to 21.1 and setuptools to 56.0.0
\ No newline at end of file
1
0
https://github.com/python/cpython/commit/c6ca368867bd68d44f333df840aa85d425…
commit: c6ca368867bd68d44f333df840aa85d425a51410
branch: master
author: Jason R. Coombs <jaraco(a)jaraco.com>
committer: jaraco <jaraco(a)jaraco.com>
date: 2021-04-24T10:13:51-04:00
summary:
bpo-43780: Sync with importlib_metadata 3.10 (GH-25297)
* bpo-43780: Sync with importlib_metadata 3.10.
* Add blurb
* Apply changes from importlib_metadata 3.10.1.
files:
A Lib/importlib/_collections.py
A Lib/importlib/_functools.py
A Misc/NEWS.d/next/Library/2021-04-08-20-04-46.bpo-43780.hUOgCh.rst
M Lib/importlib/metadata.py
M Lib/test/test_importlib/fixtures.py
M Lib/test/test_importlib/test_metadata_api.py
diff --git a/Lib/importlib/_collections.py b/Lib/importlib/_collections.py
new file mode 100644
index 0000000000000..cf0954e1a3054
--- /dev/null
+++ b/Lib/importlib/_collections.py
@@ -0,0 +1,30 @@
+import collections
+
+
+# from jaraco.collections 3.3
+class FreezableDefaultDict(collections.defaultdict):
+ """
+ Often it is desirable to prevent the mutation of
+ a default dict after its initial construction, such
+ as to prevent mutation during iteration.
+
+ >>> dd = FreezableDefaultDict(list)
+ >>> dd[0].append('1')
+ >>> dd.freeze()
+ >>> dd[1]
+ []
+ >>> len(dd)
+ 1
+ """
+
+ def __missing__(self, key):
+ return getattr(self, '_frozen', super().__missing__)(key)
+
+ def freeze(self):
+ self._frozen = lambda key: self.default_factory()
+
+
+class Pair(collections.namedtuple('Pair', 'name value')):
+ @classmethod
+ def parse(cls, text):
+ return cls(*map(str.strip, text.split("=", 1)))
diff --git a/Lib/importlib/_functools.py b/Lib/importlib/_functools.py
new file mode 100644
index 0000000000000..73f50d00bc04c
--- /dev/null
+++ b/Lib/importlib/_functools.py
@@ -0,0 +1,85 @@
+import types
+import functools
+
+
+# from jaraco.functools 3.3
+def method_cache(method, cache_wrapper=None):
+ """
+ Wrap lru_cache to support storing the cache data in the object instances.
+
+ Abstracts the common paradigm where the method explicitly saves an
+ underscore-prefixed protected property on first call and returns that
+ subsequently.
+
+ >>> class MyClass:
+ ... calls = 0
+ ...
+ ... @method_cache
+ ... def method(self, value):
+ ... self.calls += 1
+ ... return value
+
+ >>> a = MyClass()
+ >>> a.method(3)
+ 3
+ >>> for x in range(75):
+ ... res = a.method(x)
+ >>> a.calls
+ 75
+
+ Note that the apparent behavior will be exactly like that of lru_cache
+ except that the cache is stored on each instance, so values in one
+ instance will not flush values from another, and when an instance is
+ deleted, so are the cached values for that instance.
+
+ >>> b = MyClass()
+ >>> for x in range(35):
+ ... res = b.method(x)
+ >>> b.calls
+ 35
+ >>> a.method(0)
+ 0
+ >>> a.calls
+ 75
+
+ Note that if method had been decorated with ``functools.lru_cache()``,
+ a.calls would have been 76 (due to the cached value of 0 having been
+ flushed by the 'b' instance).
+
+ Clear the cache with ``.cache_clear()``
+
+ >>> a.method.cache_clear()
+
+ Same for a method that hasn't yet been called.
+
+ >>> c = MyClass()
+ >>> c.method.cache_clear()
+
+ Another cache wrapper may be supplied:
+
+ >>> cache = functools.lru_cache(maxsize=2)
+ >>> MyClass.method2 = method_cache(lambda self: 3, cache_wrapper=cache)
+ >>> a = MyClass()
+ >>> a.method2()
+ 3
+
+ Caution - do not subsequently wrap the method with another decorator, such
+ as ``@property``, which changes the semantics of the function.
+
+ See also
+ http://code.activestate.com/recipes/577452-a-memoize-decorator-for-instance…
+ for another implementation and additional justification.
+ """
+ cache_wrapper = cache_wrapper or functools.lru_cache()
+
+ def wrapper(self, *args, **kwargs):
+ # it's the first call, replace the method with a cached, bound method
+ bound_method = types.MethodType(method, self)
+ cached_method = cache_wrapper(bound_method)
+ setattr(self, method.__name__, cached_method)
+ return cached_method(*args, **kwargs)
+
+ # Support cache clear even before cache has been created.
+ wrapper.cache_clear = lambda: None
+
+ return wrapper
diff --git a/Lib/importlib/metadata.py b/Lib/importlib/metadata.py
index 53c1a145f5c43..7a427eb3b2870 100644
--- a/Lib/importlib/metadata.py
+++ b/Lib/importlib/metadata.py
@@ -7,15 +7,17 @@
import pathlib
import zipfile
import operator
+import textwrap
import warnings
import functools
import itertools
import posixpath
import collections
+from ._collections import FreezableDefaultDict, Pair
+from ._functools import method_cache
from ._itertools import unique_everseen
-from configparser import ConfigParser
from contextlib import suppress
from importlib import import_module
from importlib.abc import MetaPathFinder
@@ -51,6 +53,71 @@ def name(self):
return name
+class Sectioned:
+ """
+ A simple entry point config parser for performance
+
+ >>> for item in Sectioned.read(Sectioned._sample):
+ ... print(item)
+ Pair(name='sec1', value='# comments ignored')
+ Pair(name='sec1', value='a = 1')
+ Pair(name='sec1', value='b = 2')
+ Pair(name='sec2', value='a = 2')
+
+ >>> res = Sectioned.section_pairs(Sectioned._sample)
+ >>> item = next(res)
+ >>> item.name
+ 'sec1'
+ >>> item.value
+ Pair(name='a', value='1')
+ >>> item = next(res)
+ >>> item.value
+ Pair(name='b', value='2')
+ >>> item = next(res)
+ >>> item.name
+ 'sec2'
+ >>> item.value
+ Pair(name='a', value='2')
+ >>> list(res)
+ []
+ """
+
+ _sample = textwrap.dedent(
+ """
+ [sec1]
+ # comments ignored
+ a = 1
+ b = 2
+
+ [sec2]
+ a = 2
+ """
+ ).lstrip()
+
+ @classmethod
+ def section_pairs(cls, text):
+ return (
+ section._replace(value=Pair.parse(section.value))
+ for section in cls.read(text, filter_=cls.valid)
+ if section.name is not None
+ )
+
+ @staticmethod
+ def read(text, filter_=None):
+ lines = filter(filter_, map(str.strip, text.splitlines()))
+ name = None
+ for value in lines:
+ section_match = value.startswith('[') and value.endswith(']')
+ if section_match:
+ name = value.strip('[]')
+ continue
+ yield Pair(name, value)
+
+ @staticmethod
+ def valid(line):
+ return line and not line.startswith('#')
+
+
class EntryPoint(
collections.namedtuple('EntryPointBase', 'name value group')):
"""An entry point as defined by Python packaging conventions.
@@ -108,22 +175,6 @@ def extras(self):
match = self.pattern.match(self.value)
return list(re.finditer(r'\w+', match.group('extras') or ''))
- @classmethod
- def _from_config(cls, config):
- return (
- cls(name, value, group)
- for group in config.sections()
- for name, value in config.items(group)
- )
-
- @classmethod
- def _from_text(cls, text):
- config = ConfigParser(delimiters='=')
- # case sensitive: https://stackoverflow.com/q/1611799/812183
- config.optionxform = str
- config.read_string(text)
- return cls._from_config(config)
-
def _for(self, dist):
self.dist = dist
return self
@@ -193,7 +244,18 @@ def groups(self):
@classmethod
def _from_text_for(cls, text, dist):
- return cls(ep._for(dist) for ep in EntryPoint._from_text(text))
+ return cls(ep._for(dist) for ep in cls._from_text(text))
+
+ @classmethod
+ def _from_text(cls, text):
+ return itertools.starmap(EntryPoint, cls._parse_groups(text or ''))
+
+ @staticmethod
+ def _parse_groups(text):
+ return (
+ (item.value.name, item.value.value, item.name)
+ for item in Sectioned.section_pairs(text)
+ )
def flake8_bypass(func):
@@ -259,7 +321,7 @@ def values(self):
return super().values()
-class SelectableGroups(dict):
+class SelectableGroups(Deprecated, dict):
"""
A backward- and forward-compatible result from
entry_points that fully implements the dict interface.
@@ -277,7 +339,8 @@ def _all(self):
"""
Reconstruct a list of all entrypoints from the groups.
"""
- return EntryPoints(itertools.chain.from_iterable(self.values()))
+ groups = super(Deprecated, self).values()
+ return EntryPoints(itertools.chain.from_iterable(groups))
@property
def groups(self):
@@ -507,24 +570,7 @@ def _read_egg_info_reqs(self):
@classmethod
def _deps_from_requires_text(cls, source):
- section_pairs = cls._read_sections(source.splitlines())
- sections = {
- section: list(map(operator.itemgetter('line'), results))
- for section, results in itertools.groupby(
- section_pairs, operator.itemgetter('section')
- )
- }
- return cls._convert_egg_info_reqs_to_simple_reqs(sections)
-
- @staticmethod
- def _read_sections(lines):
- section = None
- for line in filter(None, lines):
- section_match = re.match(r'\[(.*)\]$', line)
- if section_match:
- section = section_match.group(1)
- continue
- yield locals()
+ return cls._convert_egg_info_reqs_to_simple_reqs(Sectioned.read(source))
@staticmethod
def _convert_egg_info_reqs_to_simple_reqs(sections):
@@ -549,9 +595,8 @@ def parse_condition(section):
conditions = list(filter(None, [markers, make_condition(extra)]))
return '; ' + ' and '.join(conditions) if conditions else ''
- for section, deps in sections.items():
- for dep in deps:
- yield dep + parse_condition(section)
+ for section in sections:
+ yield section.value + parse_condition(section.name)
class DistributionFinder(MetaPathFinder):
@@ -607,6 +652,10 @@ class FastPath:
children.
"""
+ @functools.lru_cache() # type: ignore
+ def __new__(cls, root):
+ return super().__new__(cls)
+
def __init__(self, root):
self.root = root
self.base = os.path.basename(self.root).lower()
@@ -629,11 +678,53 @@ def zip_children(self):
return dict.fromkeys(child.split(posixpath.sep, 1)[0] for child in names)
def search(self, name):
- return (
- self.joinpath(child)
- for child in self.children()
- if name.matches(child, self.base)
+ return self.lookup(self.mtime).search(name)
+
+ @property
+ def mtime(self):
+ with suppress(OSError):
+ return os.stat(self.root).st_mtime
+ self.lookup.cache_clear()
+
+ @method_cache
+ def lookup(self, mtime):
+ return Lookup(self)
+
+
+class Lookup:
+ def __init__(self, path: FastPath):
+ base = os.path.basename(path.root).lower()
+ base_is_egg = base.endswith(".egg")
+ self.infos = FreezableDefaultDict(list)
+ self.eggs = FreezableDefaultDict(list)
+
+ for child in path.children():
+ low = child.lower()
+ if low.endswith((".dist-info", ".egg-info")):
+ # rpartition is faster than splitext and suitable for this purpose.
+ name = low.rpartition(".")[0].partition("-")[0]
+ normalized = Prepared.normalize(name)
+ self.infos[normalized].append(path.joinpath(child))
+ elif base_is_egg and low == "egg-info":
+ name = base.rpartition(".")[0].partition("-")[0]
+ legacy_normalized = Prepared.legacy_normalize(name)
+ self.eggs[legacy_normalized].append(path.joinpath(child))
+
+ self.infos.freeze()
+ self.eggs.freeze()
+
+ def search(self, prepared):
+ infos = (
+ self.infos[prepared.normalized]
+ if prepared
+ else itertools.chain.from_iterable(self.infos.values())
+ )
+ eggs = (
+ self.eggs[prepared.legacy_normalized]
+ if prepared
+ else itertools.chain.from_iterable(self.eggs.values())
)
+ return itertools.chain(infos, eggs)
class Prepared:
@@ -642,22 +733,14 @@ class Prepared:
"""
normalized = None
- suffixes = 'dist-info', 'egg-info'
- exact_matches = [''][:0]
- egg_prefix = ''
- versionless_egg_name = ''
+ legacy_normalized = None
def __init__(self, name):
self.name = name
if name is None:
return
self.normalized = self.normalize(name)
- self.exact_matches = [
- self.normalized + '.' + suffix for suffix in self.suffixes
- ]
- legacy_normalized = self.legacy_normalize(self.name)
- self.egg_prefix = legacy_normalized + '-'
- self.versionless_egg_name = legacy_normalized + '.egg'
+ self.legacy_normalized = self.legacy_normalize(name)
@staticmethod
def normalize(name):
@@ -674,26 +757,8 @@ def legacy_normalize(name):
"""
return name.lower().replace('-', '_')
- def matches(self, cand, base):
- low = cand.lower()
- # rpartition is faster than splitext and suitable for this purpose.
- pre, _, ext = low.rpartition('.')
- name, _, rest = pre.partition('-')
- return (
- low in self.exact_matches
- or ext in self.suffixes
- and (not self.normalized or name.replace('.', '_') == self.normalized)
- # legacy case:
- or self.is_egg(base)
- and low == 'egg-info'
- )
-
- def is_egg(self, base):
- return (
- base == self.versionless_egg_name
- or base.startswith(self.egg_prefix)
- and base.endswith('.egg')
- )
+ def __bool__(self):
+ return bool(self.name)
class MetadataPathFinder(DistributionFinder):
@@ -718,6 +783,9 @@ def _search_paths(cls, name, paths):
path.search(prepared) for path in map(FastPath, paths)
)
+ def invalidate_caches(cls):
+ FastPath.__new__.cache_clear()
+
class PathDistribution(Distribution):
def __init__(self, path):
diff --git a/Lib/test/test_importlib/fixtures.py b/Lib/test/test_importlib/fixtures.py
index b50afda0f8f8f..1ae70c70f10a5 100644
--- a/Lib/test/test_importlib/fixtures.py
+++ b/Lib/test/test_importlib/fixtures.py
@@ -86,6 +86,10 @@ class DistInfoPkg(OnSysPath, SiteDir):
Version: 1.0.0
Requires-Dist: wheel >= 1.0
Requires-Dist: pytest; extra == 'test'
+ Keywords: sample package
+
+ Once upon a time
+ There was a distinfo pkg
""",
"RECORD": "mod.py,sha256=abc,20\n",
"entry_points.txt": """
@@ -157,6 +161,9 @@ class EggInfoPkg(OnSysPath, SiteDir):
Version: 1.0.0
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries
+ Keywords: sample package
+ Description: Once upon a time
+ There was an egginfo package
""",
"SOURCES.txt": """
mod.py
diff --git a/Lib/test/test_importlib/test_metadata_api.py b/Lib/test/test_importlib/test_metadata_api.py
index b54c3bd098d43..657c16603f668 100644
--- a/Lib/test/test_importlib/test_metadata_api.py
+++ b/Lib/test/test_importlib/test_metadata_api.py
@@ -2,6 +2,7 @@
import textwrap
import unittest
import warnings
+import importlib
from . import fixtures
from importlib.metadata import (
@@ -260,3 +261,9 @@ def test_distribution_at_str(self):
dist_info_path = self.site_dir / 'distinfo_pkg-1.0.0.dist-info'
dist = Distribution.at(str(dist_info_path))
assert dist.version == '1.0.0'
+
+
+class InvalidateCache(unittest.TestCase):
+ def test_invalidate_cache(self):
+ # No externally observable behavior, but ensures test coverage...
+ importlib.invalidate_caches()
diff --git a/Misc/NEWS.d/next/Library/2021-04-08-20-04-46.bpo-43780.hUOgCh.rst b/Misc/NEWS.d/next/Library/2021-04-08-20-04-46.bpo-43780.hUOgCh.rst
new file mode 100644
index 0000000000000..3adbe50512bed
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2021-04-08-20-04-46.bpo-43780.hUOgCh.rst
@@ -0,0 +1,3 @@
+In ``importlib.metadata``, incorporate changes from importlib_metadata 3.10:
+Add mtime-based caching during distribution discovery. Flagged use of dict
+result from ``entry_points()`` as deprecated.
1
0