Mailman 3 December 2021 - Python-checkins

Doc: Fix link in multiprocessing.starmap pointing to builtin map. (GH-26560)
by miss-islington Dec. 26, 2021

Dec. 26, 2021

https://github.com/python/cpython/commit/3c1100fc59aca4bb8d0b9cdfbd439d4d2d… commit: 3c1100fc59aca4bb8d0b9cdfbd439d4d2dc9651c branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2021-12-26T03:15:38-08:00 summary: Doc: Fix link in multiprocessing.starmap pointing to builtin map. (GH-26560) (cherry picked from commit 32096df0e00e692ee6dc688e62213bff0dffd573) Co-authored-by: Julien Palard <julien(a)palard.fr> files: M Doc/library/multiprocessing.rst diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 2c7d7d8c0e8b1..96bc6c153284a 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -2250,8 +2250,9 @@ with the :class:`Pool` class. .. method:: starmap(func, iterable[, chunksize]) - Like :meth:`map` except that the elements of the *iterable* are expected - to be iterables that are unpacked as arguments. + Like :meth:`~multiprocessing.pool.Pool.map` except that the + elements of the *iterable* are expected to be iterables that are + unpacked as arguments. Hence an *iterable* of ``[(1,2), (3, 4)]`` results in ``[func(1,2), func(3,4)]``.

1 0

Doc: Fix link in multiprocessing.starmap pointing to builtin map. (GH-26560)
by miss-islington Dec. 26, 2021

Dec. 26, 2021

https://github.com/python/cpython/commit/2cb9ed2a6a082972bfc496eccb320c3ae3… commit: 2cb9ed2a6a082972bfc496eccb320c3ae3df72df branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2021-12-26T03:15:14-08:00 summary: Doc: Fix link in multiprocessing.starmap pointing to builtin map. (GH-26560) (cherry picked from commit 32096df0e00e692ee6dc688e62213bff0dffd573) Co-authored-by: Julien Palard <julien(a)palard.fr> files: M Doc/library/multiprocessing.rst diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 4562128394f88..e81dd7e648f7f 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -2250,8 +2250,9 @@ with the :class:`Pool` class. .. method:: starmap(func, iterable[, chunksize]) - Like :meth:`map` except that the elements of the *iterable* are expected - to be iterables that are unpacked as arguments. + Like :meth:`~multiprocessing.pool.Pool.map` except that the + elements of the *iterable* are expected to be iterables that are + unpacked as arguments. Hence an *iterable* of ``[(1,2), (3, 4)]`` results in ``[func(1,2), func(3,4)]``.

1 0

bpo-23819: Fix asyncio tests on python optimized mode (GH-30195)
by serhiy-storchaka Dec. 26, 2021

Dec. 26, 2021

https://github.com/python/cpython/commit/a23ab7b6d8b3ae3a47747c0c4bceb2370c… commit: a23ab7b6d8b3ae3a47747c0c4bceb2370cc48dcc branch: main author: Kumar Aditya <59607654+kumaraditya303(a)users.noreply.github.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2021-12-26T13:13:14+02:00 summary: bpo-23819: Fix asyncio tests on python optimized mode (GH-30195) files: A Misc/NEWS.d/next/Tests/2021-12-19-08-44-32.bpo-23819.9ueiII.rst M Lib/asyncio/base_events.py M Lib/test/test_asyncio/test_base_events.py M Lib/test/test_asyncio/test_proactor_events.py M Lib/test/test_asyncio/test_selector_events.py diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py index cbf6d5db0a002..56ea7ba44e2ea 100644 --- a/Lib/asyncio/base_events.py +++ b/Lib/asyncio/base_events.py @@ -1285,8 +1285,8 @@ async def create_datagram_endpoint(self, protocol_factory, addr_infos = {} # Using order preserving dict for idx, addr in ((0, local_addr), (1, remote_addr)): if addr is not None: - assert isinstance(addr, tuple) and len(addr) == 2, ( - '2-tuple is expected') + if not (isinstance(addr, tuple) and len(addr) == 2): + raise TypeError('2-tuple is expected') infos = await self._ensure_resolved( addr, family=family, type=socket.SOCK_DGRAM, diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py index 4c0d09e6f702e..3f4c2d85e0380 100644 --- a/Lib/test/test_asyncio/test_base_events.py +++ b/Lib/test/test_asyncio/test_base_events.py @@ -1603,11 +1603,11 @@ def test_create_datagram_endpoint_addr_error(self): coro = self.loop.create_datagram_endpoint( MyDatagramProto, local_addr='localhost') self.assertRaises( - AssertionError, self.loop.run_until_complete, coro) + TypeError, self.loop.run_until_complete, coro) coro = self.loop.create_datagram_endpoint( MyDatagramProto, local_addr=('localhost', 1, 2, 3)) self.assertRaises( - AssertionError, self.loop.run_until_complete, coro) + TypeError, self.loop.run_until_complete, coro) def test_create_datagram_endpoint_connect_err(self): self.loop.sock_connect = mock.Mock() diff --git a/Lib/test/test_asyncio/test_proactor_events.py b/Lib/test/test_asyncio/test_proactor_events.py index 4c8906d531ce5..45af8101a0093 100644 --- a/Lib/test/test_asyncio/test_proactor_events.py +++ b/Lib/test/test_asyncio/test_proactor_events.py @@ -77,6 +77,7 @@ def test_loop_reading_data(self): self.loop._proactor.recv_into.assert_called_with(self.sock, called_buf) self.protocol.data_received.assert_called_with(bytearray(buf)) + @unittest.skipIf(sys.flags.optimize, "Assertions are disabled in optimized mode") def test_loop_reading_no_data(self): res = self.loop.create_future() res.set_result(0) @@ -869,6 +870,7 @@ def test_datagram_loop_reading_data(self): self.protocol.datagram_received.assert_called_with(b'data', ('127.0.0.1', 12068)) close_transport(tr) + @unittest.skipIf(sys.flags.optimize, "Assertions are disabled in optimized mode") def test_datagram_loop_reading_no_data(self): res = self.loop.create_future() res.set_result((b'', ('127.0.0.1', 12068))) diff --git a/Lib/test/test_asyncio/test_selector_events.py b/Lib/test/test_asyncio/test_selector_events.py index 1613c753c26ee..b684fab2771f2 100644 --- a/Lib/test/test_asyncio/test_selector_events.py +++ b/Lib/test/test_asyncio/test_selector_events.py @@ -1,5 +1,6 @@ """Tests for selector_events.py""" +import sys import selectors import socket import unittest @@ -804,6 +805,7 @@ def test_write_ready_closing(self): self.sock.close.assert_called_with() self.protocol.connection_lost.assert_called_with(None) + @unittest.skipIf(sys.flags.optimize, "Assertions are disabled in optimized mode") def test_write_ready_no_data(self): transport = self.socket_transport() # This is an internal error. diff --git a/Misc/NEWS.d/next/Tests/2021-12-19-08-44-32.bpo-23819.9ueiII.rst b/Misc/NEWS.d/next/Tests/2021-12-19-08-44-32.bpo-23819.9ueiII.rst new file mode 100644 index 0000000000000..4ef0fe6f6d5fa --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2021-12-19-08-44-32.bpo-23819.9ueiII.rst @@ -0,0 +1 @@ +Fixed :mod:`asyncio` tests in python optimized mode. Patch by Kumar Aditya. \ No newline at end of file

1 0

Fix typo in io.rst (GH-30218)
by serhiy-storchaka Dec. 26, 2021

Dec. 26, 2021

https://github.com/python/cpython/commit/f9a4352056175d96195d4cb54c97c1334e… commit: f9a4352056175d96195d4cb54c97c1334ea5150c branch: main author: David Gilbertson <gilbertson.david(a)gmail.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2021-12-26T13:11:08+02:00 summary: Fix typo in io.rst (GH-30218) files: M Doc/library/io.rst diff --git a/Doc/library/io.rst b/Doc/library/io.rst index 0881015c7fad6..de88c572f3662 100644 --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -131,7 +131,7 @@ explicitly when opening text files. If you want to use UTF-8, pass ``encoding="utf-8"``. To use the current locale encoding, ``encoding="locale"`` is supported in Python 3.10. -When you need to run existing code on Windows that attempts to opens +When you need to run existing code on Windows that attempts to open UTF-8 files using the default locale encoding, you can enable the UTF-8 mode. See :ref:`UTF-8 mode on Windows <win-utf8-mode>`.

1 0

doc: fix a typo in unittest.mock.rst (GH-30227)
by serhiy-storchaka Dec. 26, 2021

Dec. 26, 2021

https://github.com/python/cpython/commit/10bf0a9ac3c9bbde2fa8989d9435462f0e… commit: 10bf0a9ac3c9bbde2fa8989d9435462f0e7da545 branch: main author: Joe <nigelchiang(a)outlook.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2021-12-26T13:09:17+02:00 summary: doc: fix a typo in unittest.mock.rst (GH-30227) files: M Doc/library/unittest.mock.rst diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 0856c3fbded08..69f1035703ba5 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -1516,7 +1516,7 @@ attribute in a class) that does not exist will fail with :exc:`AttributeError`:: >>> test() Traceback (most recent call last): ... - AttributeError: <module 'sys' (built-in)> does not have the attribute 'non_existing' + AttributeError: <module 'sys' (built-in)> does not have the attribute 'non_existing_attribute' but adding ``create=True`` in the call to :func:`patch` will make the previous example work as expected::

1 0

docs: Fix typos and use anchor for internal link (GH-30236)
by serhiy-storchaka Dec. 26, 2021

Dec. 26, 2021

https://github.com/python/cpython/commit/c1d7a6bed9f2bb8cb7612d196fc0caba01… commit: c1d7a6bed9f2bb8cb7612d196fc0caba016fa98c branch: main author: Rafael Fontenelle <rffontenelle(a)users.noreply.github.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2021-12-26T12:43:06+02:00 summary: docs: Fix typos and use anchor for internal link (GH-30236) files: M Doc/whatsnew/3.10.rst diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index b56663f5eae8b..85bb1f8838e9b 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -1437,7 +1437,7 @@ and to match the behavior of static type checkers specified in the PEP. 1. ``Literal`` now de-duplicates parameters. 2. Equality comparisons between ``Literal`` objects are now order independent. -3. ``Literal`` comparisons now respects types. For example, +3. ``Literal`` comparisons now respect types. For example, ``Literal[0] == Literal[False]`` previously evaluated to ``True``. It is now ``False``. To support this change, the internally used type cache now supports differentiating types. @@ -1647,13 +1647,12 @@ Deprecated :meth:`importlib.machinery.FrozenImporter.find_module`, :meth:`importlib.machinery.WindowsRegistryFinder.find_module`, :meth:`importlib.machinery.PathFinder.find_module`, - :meth:`importlib.abc.MetaPathFinder.find_module`), + :meth:`importlib.abc.MetaPathFinder.find_module` ), :meth:`importlib.abc.PathEntryFinder.find_module` ( - :meth:`importlib.machinery.FileFinder.find_module`, - ), and + :meth:`importlib.machinery.FileFinder.find_module` ), and :meth:`importlib.abc.PathEntryFinder.find_loader` ( - :meth:`importlib.machinery.FileFinder.find_loader` - ) now raise :exc:`DeprecationWarning` and are slated for removal in + :meth:`importlib.machinery.FileFinder.find_loader` ) + now raise :exc:`DeprecationWarning` and are slated for removal in Python 3.12 (previously they were documented as deprecated in Python 3.4). (Contributed by Brett Cannon in :issue:`42135`.) @@ -1694,7 +1693,7 @@ Deprecated * :func:`asyncio.get_event_loop` now emits a deprecation warning if there is no running event loop. In the future it will be an alias of :func:`~asyncio.get_running_loop`. - :mod:`asyncio` functions which implicitly create a :class:`~asyncio.Future` + :mod:`asyncio` functions which implicitly create :class:`~asyncio.Future` or :class:`~asyncio.Task` objects now emit a deprecation warning if there is no running event loop and no explicit *loop* argument is passed: :func:`~asyncio.ensure_future`, @@ -1834,7 +1833,7 @@ Removed running in different threads. Note that the low-level API will still accept ``loop``. - See `Changes in the Python API`_ for examples of how to replace existing code. + See :ref:`changes-python-api` for examples of how to replace existing code. (Contributed by Yurii Karabas, Andrew Svetlov, Yury Selivanov and Kyle Stanley in :issue:`42392`.) @@ -1858,6 +1857,7 @@ Changes in the Python syntax following keyword. (Contributed by Serhiy Storchaka in :issue:`43833`). +.. _changes-python-api: Changes in the Python API ------------------------- @@ -1979,7 +1979,7 @@ Build Changes (Contributed by Victor Stinner in :issue:`36020`.) * :mod:`sqlite3` requires SQLite 3.7.15 or higher. (Contributed by Sergey Fedoseev - and Erlend E. Aasland :issue:`40744` and :issue:`40810`.) + and Erlend E. Aasland in :issue:`40744` and :issue:`40810`.) * The :mod:`atexit` module must now always be built as a built-in module. (Contributed by Victor Stinner in :issue:`42639`.)

1 0

Remove a NEWS entry for bpo-45878 (GH-30259)
by serhiy-storchaka Dec. 26, 2021

Dec. 26, 2021

https://github.com/python/cpython/commit/1fb7c61ca76c6fbff4d90b272e34e92bc2… commit: 1fb7c61ca76c6fbff4d90b272e34e92bc2c7d729 branch: main author: Serhiy Storchaka <storchaka(a)gmail.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2021-12-26T12:38:19+02:00 summary: Remove a NEWS entry for bpo-45878 (GH-30259) The docs linter complains about it, and in general news entries for such changes are not required. files: M Misc/NEWS.d/3.11.0a3.rst diff --git a/Misc/NEWS.d/3.11.0a3.rst b/Misc/NEWS.d/3.11.0a3.rst index b5bcd5ae5a70e..426531904993a 100644 --- a/Misc/NEWS.d/3.11.0a3.rst +++ b/Misc/NEWS.d/3.11.0a3.rst @@ -813,16 +813,6 @@ Add new Test for ``Lib/email/mime/nonmultipart.py::MIMENonMultipart``. .. -.. bpo: 45878 -.. date: 2021-11-23-12-36-21 -.. nonce: eOs_Mp -.. section: Tests - -Test ``Lib/ctypes/test/test_functions.py::test_mro`` now uses -``self.assertRaises`` instead of ``try/except``. - -.. - .. bpo: 45835 .. date: 2021-11-17-14-28-08 .. nonce: Mgyhjx

1 0

[3.10] Remove a NEWS entry for bpo-45878 (GH-30258)
by serhiy-storchaka Dec. 26, 2021

Dec. 26, 2021

https://github.com/python/cpython/commit/11909c12c75a7f377460561abc97707a40… commit: 11909c12c75a7f377460561abc97707a4006fc07 branch: 3.10 author: Serhiy Storchaka <storchaka(a)gmail.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2021-12-26T12:37:39+02:00 summary: [3.10] Remove a NEWS entry for bpo-45878 (GH-30258) The docs linter complains about it, and in general news entries for such changes are not required. files: D Misc/NEWS.d/next/Tests/2021-11-23-12-36-21.bpo-45878.eOs_Mp.rst diff --git a/Misc/NEWS.d/next/Tests/2021-11-23-12-36-21.bpo-45878.eOs_Mp.rst b/Misc/NEWS.d/next/Tests/2021-11-23-12-36-21.bpo-45878.eOs_Mp.rst deleted file mode 100644 index fca90ef46981b..0000000000000 --- a/Misc/NEWS.d/next/Tests/2021-11-23-12-36-21.bpo-45878.eOs_Mp.rst +++ /dev/null @@ -1,2 +0,0 @@ -Test ``Lib/ctypes/test/test_functions.py::test_mro`` now uses -``self.assertRaises`` instead of ``try/except``.

1 0

[3.10] bpo-46032: Check types in singledispatch's register() at declaration time (GH-30050) (GH-30254)
by serhiy-storchaka Dec. 25, 2021

Dec. 25, 2021

https://github.com/python/cpython/commit/03c7449fbc7c57f5e0365f234a0b65c1dd… commit: 03c7449fbc7c57f5e0365f234a0b65c1dde763f2 branch: 3.10 author: Serhiy Storchaka <storchaka(a)gmail.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2021-12-25T16:12:32+02:00 summary: [3.10] bpo-46032: Check types in singledispatch's register() at declaration time (GH-30050) (GH-30254) The registry() method of functools.singledispatch() functions checks now the first argument or the first parameter annotation and raises a TypeError if it is not supported. Previously unsupported "types" were ignored (e.g. typing.List[int]) or caused an error at calling time (e.g. list[int]). (cherry picked from commit 078abb676cf759b1e960f78390b6e80f256f0255) files: A Misc/NEWS.d/next/Library/2021-12-11-15-45-07.bpo-46032.HmciLT.rst M Lib/functools.py M Lib/test/test_functools.py diff --git a/Lib/functools.py b/Lib/functools.py index 77ec852805c10..305ceb450a71c 100644 --- a/Lib/functools.py +++ b/Lib/functools.py @@ -740,6 +740,7 @@ def _compose_mro(cls, types): # Remove entries which are already present in the __mro__ or unrelated. def is_related(typ): return (typ not in bases and hasattr(typ, '__mro__') + and not isinstance(typ, GenericAlias) and issubclass(cls, typ)) types = [n for n in types if is_related(n)] # Remove entries which are strict bases of other entries (they will end up @@ -837,6 +838,9 @@ def dispatch(cls): dispatch_cache[cls] = impl return impl + def _is_valid_dispatch_type(cls): + return isinstance(cls, type) and not isinstance(cls, GenericAlias) + def register(cls, func=None): """generic_func.register(cls, func) -> func @@ -844,9 +848,15 @@ def register(cls, func=None): """ nonlocal cache_token - if func is None: - if isinstance(cls, type): + if _is_valid_dispatch_type(cls): + if func is None: return lambda f: register(cls, f) + else: + if func is not None: + raise TypeError( + f"Invalid first argument to `register()`. " + f"{cls!r} is not a class." + ) ann = getattr(cls, '__annotations__', {}) if not ann: raise TypeError( @@ -859,11 +869,12 @@ def register(cls, func=None): # only import typing if annotation parsing is necessary from typing import get_type_hints argname, cls = next(iter(get_type_hints(func).items())) - if not isinstance(cls, type): + if not _is_valid_dispatch_type(cls): raise TypeError( f"Invalid annotation for {argname!r}. " f"{cls!r} is not a class." ) + registry[cls] = func if cache_token is None and hasattr(cls, '__abstractmethods__'): cache_token = get_cache_token() diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py index 7bc355ff7213e..9343d4e9fa316 100644 --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -2665,6 +2665,74 @@ def f(*args): with self.assertRaisesRegex(TypeError, msg): f() + def test_register_genericalias(self): + @functools.singledispatch + def f(arg): + return "default" + + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(list[int], lambda arg: "types.GenericAlias") + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.List[int], lambda arg: "typing.GenericAlias") + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(list[int] | str, lambda arg: "types.UnionTypes(types.GenericAlias)") + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.List[float] | bytes, lambda arg: "typing.Union[typing.GenericAlias]") + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.Any, lambda arg: "typing.Any") + + self.assertEqual(f([1]), "default") + self.assertEqual(f([1.0]), "default") + self.assertEqual(f(""), "default") + self.assertEqual(f(b""), "default") + + def test_register_genericalias_decorator(self): + @functools.singledispatch + def f(arg): + return "default" + + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(list[int]) + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.List[int]) + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(list[int] | str) + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.List[int] | str) + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.Any) + + def test_register_genericalias_annotation(self): + @functools.singledispatch + def f(arg): + return "default" + + with self.assertRaisesRegex(TypeError, "Invalid annotation for 'arg'"): + @f.register + def _(arg: list[int]): + return "types.GenericAlias" + with self.assertRaisesRegex(TypeError, "Invalid annotation for 'arg'"): + @f.register + def _(arg: typing.List[float]): + return "typing.GenericAlias" + with self.assertRaisesRegex(TypeError, "Invalid annotation for 'arg'"): + @f.register + def _(arg: list[int] | str): + return "types.UnionType(types.GenericAlias)" + with self.assertRaisesRegex(TypeError, "Invalid annotation for 'arg'"): + @f.register + def _(arg: typing.List[float] | bytes): + return "typing.Union[typing.GenericAlias]" + with self.assertRaisesRegex(TypeError, "Invalid annotation for 'arg'"): + @f.register + def _(arg: typing.Any): + return "typing.Any" + + self.assertEqual(f([1]), "default") + self.assertEqual(f([1.0]), "default") + self.assertEqual(f(""), "default") + self.assertEqual(f(b""), "default") + class CachedCostItem: _cost = 1 diff --git a/Misc/NEWS.d/next/Library/2021-12-11-15-45-07.bpo-46032.HmciLT.rst b/Misc/NEWS.d/next/Library/2021-12-11-15-45-07.bpo-46032.HmciLT.rst new file mode 100644 index 0000000000000..97a553d7ba29f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-12-11-15-45-07.bpo-46032.HmciLT.rst @@ -0,0 +1,5 @@ +The ``registry()`` method of :func:`functools.singledispatch` functions +checks now the first argument or the first parameter annotation and raises a +TypeError if it is not supported. Previously unsupported "types" were +ignored (e.g. ``typing.List[int]``) or caused an error at calling time (e.g. +``list[int]``).

1 0

bpo-46032: Check types in singledispatch's register() at declaration time (GH-30050)
by serhiy-storchaka Dec. 25, 2021

Dec. 25, 2021

https://github.com/python/cpython/commit/078abb676cf759b1e960f78390b6e80f25… commit: 078abb676cf759b1e960f78390b6e80f256f0255 branch: main author: Serhiy Storchaka <storchaka(a)gmail.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2021-12-25T14:16:14+02:00 summary: bpo-46032: Check types in singledispatch's register() at declaration time (GH-30050) The registry() method of functools.singledispatch() functions checks now the first argument or the first parameter annotation and raises a TypeError if it is not supported. Previously unsupported "types" were ignored (e.g. typing.List[int]) or caused an error at calling time (e.g. list[int]). files: A Misc/NEWS.d/next/Library/2021-12-11-15-45-07.bpo-46032.HmciLT.rst M Lib/functools.py M Lib/test/test_functools.py diff --git a/Lib/functools.py b/Lib/functools.py index ccac6f89996b6..91b678c226966 100644 --- a/Lib/functools.py +++ b/Lib/functools.py @@ -740,6 +740,7 @@ def _compose_mro(cls, types): # Remove entries which are already present in the __mro__ or unrelated. def is_related(typ): return (typ not in bases and hasattr(typ, '__mro__') + and not isinstance(typ, GenericAlias) and issubclass(cls, typ)) types = [n for n in types if is_related(n)] # Remove entries which are strict bases of other entries (they will end up @@ -841,9 +842,13 @@ def _is_union_type(cls): from typing import get_origin, Union return get_origin(cls) in {Union, types.UnionType} - def _is_valid_union_type(cls): + def _is_valid_dispatch_type(cls): + if isinstance(cls, type) and not isinstance(cls, GenericAlias): + return True from typing import get_args - return _is_union_type(cls) and all(isinstance(arg, type) for arg in get_args(cls)) + return (_is_union_type(cls) and + all(isinstance(arg, type) and not isinstance(arg, GenericAlias) + for arg in get_args(cls))) def register(cls, func=None): """generic_func.register(cls, func) -> func @@ -852,9 +857,15 @@ def register(cls, func=None): """ nonlocal cache_token - if func is None: - if isinstance(cls, type) or _is_valid_union_type(cls): + if _is_valid_dispatch_type(cls): + if func is None: return lambda f: register(cls, f) + else: + if func is not None: + raise TypeError( + f"Invalid first argument to `register()`. " + f"{cls!r} is not a class or union type." + ) ann = getattr(cls, '__annotations__', {}) if not ann: raise TypeError( @@ -867,7 +878,7 @@ def register(cls, func=None): # only import typing if annotation parsing is necessary from typing import get_type_hints argname, cls = next(iter(get_type_hints(func).items())) - if not isinstance(cls, type) and not _is_valid_union_type(cls): + if not _is_valid_dispatch_type(cls): if _is_union_type(cls): raise TypeError( f"Invalid annotation for {argname!r}. " diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py index 755ac038792b7..70ae8e06bb475 100644 --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -2722,6 +2722,74 @@ def _(arg: int | float): self.assertEqual(f(1), "types.UnionType") self.assertEqual(f(1.0), "types.UnionType") + def test_register_genericalias(self): + @functools.singledispatch + def f(arg): + return "default" + + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(list[int], lambda arg: "types.GenericAlias") + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.List[int], lambda arg: "typing.GenericAlias") + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(list[int] | str, lambda arg: "types.UnionTypes(types.GenericAlias)") + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.List[float] | bytes, lambda arg: "typing.Union[typing.GenericAlias]") + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.Any, lambda arg: "typing.Any") + + self.assertEqual(f([1]), "default") + self.assertEqual(f([1.0]), "default") + self.assertEqual(f(""), "default") + self.assertEqual(f(b""), "default") + + def test_register_genericalias_decorator(self): + @functools.singledispatch + def f(arg): + return "default" + + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(list[int]) + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.List[int]) + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(list[int] | str) + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.List[int] | str) + with self.assertRaisesRegex(TypeError, "Invalid first argument to "): + f.register(typing.Any) + + def test_register_genericalias_annotation(self): + @functools.singledispatch + def f(arg): + return "default" + + with self.assertRaisesRegex(TypeError, "Invalid annotation for 'arg'"): + @f.register + def _(arg: list[int]): + return "types.GenericAlias" + with self.assertRaisesRegex(TypeError, "Invalid annotation for 'arg'"): + @f.register + def _(arg: typing.List[float]): + return "typing.GenericAlias" + with self.assertRaisesRegex(TypeError, "Invalid annotation for 'arg'"): + @f.register + def _(arg: list[int] | str): + return "types.UnionType(types.GenericAlias)" + with self.assertRaisesRegex(TypeError, "Invalid annotation for 'arg'"): + @f.register + def _(arg: typing.List[float] | bytes): + return "typing.Union[typing.GenericAlias]" + with self.assertRaisesRegex(TypeError, "Invalid annotation for 'arg'"): + @f.register + def _(arg: typing.Any): + return "typing.Any" + + self.assertEqual(f([1]), "default") + self.assertEqual(f([1.0]), "default") + self.assertEqual(f(""), "default") + self.assertEqual(f(b""), "default") + class CachedCostItem: _cost = 1 diff --git a/Misc/NEWS.d/next/Library/2021-12-11-15-45-07.bpo-46032.HmciLT.rst b/Misc/NEWS.d/next/Library/2021-12-11-15-45-07.bpo-46032.HmciLT.rst new file mode 100644 index 0000000000000..97a553d7ba29f --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-12-11-15-45-07.bpo-46032.HmciLT.rst @@ -0,0 +1,5 @@ +The ``registry()`` method of :func:`functools.singledispatch` functions +checks now the first argument or the first parameter annotation and raises a +TypeError if it is not supported. Previously unsupported "types" were +ignored (e.g. ``typing.List[int]``) or caused an error at calling time (e.g. +``list[int]``).

1 0