Mailman 3 March 2022 - Python-checkins

bpo-21910: Clarify docs for codecs writelines method (GH-31245)
by miss-islington March 3, 2022

March 3, 2022

https://github.com/python/cpython/commit/cf8aff6319794807aa578215710e6caa44… commit: cf8aff6319794807aa578215710e6caa4479516f branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-03-02T17:45:36-08:00 summary: bpo-21910: Clarify docs for codecs writelines method (GH-31245) Co-authored-by: Jelle Zijlstra <jelle.zijlstra(a)gmail.com> (cherry picked from commit a8c87a239ee1414d6dd0b062fe9ec3e5b0c50cb8) Co-authored-by: slateny <46876382+slateny(a)users.noreply.github.com> files: M Doc/library/codecs.rst diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst index 2f7497c0c4934..3386208d5ef5f 100644 --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -691,8 +691,9 @@ compatible with the Python codec registry. .. method:: writelines(list) - Writes the concatenated list of strings to the stream (possibly by reusing - the :meth:`write` method). The standard bytes-to-bytes codecs + Writes the concatenated iterable of strings to the stream (possibly by reusing + the :meth:`write` method). Infinite or + very large iterables are not supported. The standard bytes-to-bytes codecs do not support this method.

1 0

bpo-21910: Clarify docs for codecs writelines method (GH-31245)
by miss-islington March 3, 2022

March 3, 2022

https://github.com/python/cpython/commit/60b561c246da2073672a016340457e4534… commit: 60b561c246da2073672a016340457e4534dfdf5b branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-03-02T17:43:00-08:00 summary: bpo-21910: Clarify docs for codecs writelines method (GH-31245) Co-authored-by: Jelle Zijlstra <jelle.zijlstra(a)gmail.com> (cherry picked from commit a8c87a239ee1414d6dd0b062fe9ec3e5b0c50cb8) Co-authored-by: slateny <46876382+slateny(a)users.noreply.github.com> files: M Doc/library/codecs.rst diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst index ef71832bcef1d..1a1ce9237b010 100644 --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -694,8 +694,9 @@ compatible with the Python codec registry. .. method:: writelines(list) - Writes the concatenated list of strings to the stream (possibly by reusing - the :meth:`write` method). The standard bytes-to-bytes codecs + Writes the concatenated iterable of strings to the stream (possibly by reusing + the :meth:`write` method). Infinite or + very large iterables are not supported. The standard bytes-to-bytes codecs do not support this method.

1 0

bpo-21910: Clarify docs for codecs writelines method (GH-31245)
by JelleZijlstra March 3, 2022

March 3, 2022

https://github.com/python/cpython/commit/a8c87a239ee1414d6dd0b062fe9ec3e5b0… commit: a8c87a239ee1414d6dd0b062fe9ec3e5b0c50cb8 branch: main author: slateny <46876382+slateny(a)users.noreply.github.com> committer: JelleZijlstra <jelle.zijlstra(a)gmail.com> date: 2022-03-02T17:21:41-08:00 summary: bpo-21910: Clarify docs for codecs writelines method (GH-31245) Co-authored-by: Jelle Zijlstra <jelle.zijlstra(a)gmail.com> files: M Doc/library/codecs.rst diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst index 949288b7c6d9b..76710974dd427 100644 --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -697,8 +697,9 @@ compatible with the Python codec registry. .. method:: writelines(list) - Writes the concatenated list of strings to the stream (possibly by reusing - the :meth:`write` method). The standard bytes-to-bytes codecs + Writes the concatenated iterable of strings to the stream (possibly by reusing + the :meth:`write` method). Infinite or + very large iterables are not supported. The standard bytes-to-bytes codecs do not support this method.

1 0

bpo-46643: Fix stringized P.args/P.kwargs with get_type_hints (GH-31238)
by JelleZijlstra March 3, 2022

March 3, 2022

https://github.com/python/cpython/commit/75d2d945b4e28ca34506b2d4902367b61a… commit: 75d2d945b4e28ca34506b2d4902367b61a8dff82 branch: main author: Gregory Beauregard <greg(a)greg.red> committer: JelleZijlstra <jelle.zijlstra(a)gmail.com> date: 2022-03-02T17:14:52-08:00 summary: bpo-46643: Fix stringized P.args/P.kwargs with get_type_hints (GH-31238) files: A Misc/NEWS.d/next/Library/2022-02-09-22-40-11.bpo-46643.aBlIx1.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 8fcc24c25eb95..bd9920436223c 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -5187,6 +5187,18 @@ def test_args_kwargs(self): self.assertEqual(repr(P.args), "P.args") self.assertEqual(repr(P.kwargs), "P.kwargs") + def test_stringized(self): + P = ParamSpec('P') + class C(Generic[P]): + func: Callable["P", int] + def foo(self, *args: "P.args", **kwargs: "P.kwargs"): + pass + + self.assertEqual(gth(C, globals(), locals()), {"func": Callable[P, int]}) + self.assertEqual( + gth(C.foo, globals(), locals()), {"args": P.args, "kwargs": P.kwargs} + ) + def test_user_generics(self): T = TypeVar("T") P = ParamSpec("P") diff --git a/Lib/typing.py b/Lib/typing.py index 9d668b3cf4a2a..6e0c68c842420 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -181,7 +181,8 @@ def _type_check(arg, msg, is_argument=True, module=None, *, allow_special_forms= return arg if isinstance(arg, _SpecialForm) or arg in (Generic, Protocol): raise TypeError(f"Plain {arg} is not valid as type argument") - if isinstance(arg, (type, TypeVar, ForwardRef, types.UnionType, ParamSpec)): + if isinstance(arg, (type, TypeVar, ForwardRef, types.UnionType, ParamSpec, + ParamSpecArgs, ParamSpecKwargs)): return arg if not callable(arg): raise TypeError(f"{msg} Got {arg!r:.100}.") diff --git a/Misc/NEWS.d/next/Library/2022-02-09-22-40-11.bpo-46643.aBlIx1.rst b/Misc/NEWS.d/next/Library/2022-02-09-22-40-11.bpo-46643.aBlIx1.rst new file mode 100644 index 0000000000000..e8b4d66e943f6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-09-22-40-11.bpo-46643.aBlIx1.rst @@ -0,0 +1 @@ +In :func:`typing.get_type_hints`, support evaluating stringified ``ParamSpecArgs`` and ``ParamSpecKwargs`` annotations. Patch by Gregory Beauregard. \ No newline at end of file

1 0

bpo-46860: Respect `--with-suffix` on case-insensitive file systems (GH-31593)
by brettcannon March 2, 2022

March 2, 2022

https://github.com/python/cpython/commit/50ec3453c50c490eca7e990103c79671bd… commit: 50ec3453c50c490eca7e990103c79671bd08ab2e branch: main author: Brett Cannon <brett(a)python.org> committer: brettcannon <brett(a)python.org> date: 2022-03-02T14:23:59-08:00 summary: bpo-46860: Respect `--with-suffix` on case-insensitive file systems (GH-31593) Previously, case-insensitive file systems were forced to use `.exe` as the file suffix no matter what `--with-suffix` was set to. files: A Misc/NEWS.d/next/Build/2022-02-25-16-19-40.bpo-46860.jfciLG.rst M configure M configure.ac diff --git a/Misc/NEWS.d/next/Build/2022-02-25-16-19-40.bpo-46860.jfciLG.rst b/Misc/NEWS.d/next/Build/2022-02-25-16-19-40.bpo-46860.jfciLG.rst new file mode 100644 index 0000000000000..7eab15db335e4 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2022-02-25-16-19-40.bpo-46860.jfciLG.rst @@ -0,0 +1 @@ +Respect `--with-suffix` when building on case-insensitive file systems. diff --git a/configure b/configure index 49d5abeac3024..07ecb804c1bd0 100755 --- a/configure +++ b/configure @@ -6296,7 +6296,7 @@ if test ! -d CaseSensitiveTestDir; then mkdir CaseSensitiveTestDir fi -if test -d casesensitivetestdir +if test -d casesensitivetestdir && test -z "$EXEEXT" then { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } diff --git a/configure.ac b/configure.ac index 83dd854248b42..fedec52f05a75 100644 --- a/configure.ac +++ b/configure.ac @@ -1112,7 +1112,7 @@ if test ! -d CaseSensitiveTestDir; then mkdir CaseSensitiveTestDir fi -if test -d casesensitivetestdir +if test -d casesensitivetestdir && test -z "$EXEEXT" then AC_MSG_RESULT(yes) BUILDEXEEXT=.exe

1 0

[3.10] bpo-45492: Corrected documentation for co_names in inspect library doc (GH-31456). (GH-31645)
by iritkatriel March 2, 2022

March 2, 2022

https://github.com/python/cpython/commit/eb65e46b9b28103767c115ccf71a97a9f4… commit: eb65e46b9b28103767c115ccf71a97a9f4237d4c branch: 3.10 author: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2022-03-02T19:37:17Z summary: [3.10] bpo-45492: Corrected documentation for co_names in inspect library doc (GH-31456). (GH-31645) (cherry picked from commit 3257d49d236e5f3453fe9d2fd8338bcdfe9756b7) Co-authored-by: slateny <46876382+slateny(a)users.noreply.github.com> Co-authored-by: slateny <46876382+slateny(a)users.noreply.github.com> files: M Doc/library/inspect.rst diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index ed33cbb8a61ab..d7ef2adf7281b 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -187,8 +187,9 @@ attributes: | | co_name | name with which this code | | | | object was defined | +-----------+-------------------+---------------------------+ -| | co_names | tuple of names of local | -| | | variables | +| | co_names | tuple of names other | +| | | than arguments and | +| | | function locals | +-----------+-------------------+---------------------------+ | | co_nlocals | number of local variables | +-----------+-------------------+---------------------------+

1 0

bpo-45492: Corrected documentation for co_names in inspect library doc (GH-31456)
by iritkatriel March 2, 2022

March 2, 2022

https://github.com/python/cpython/commit/3257d49d236e5f3453fe9d2fd8338bcdfe… commit: 3257d49d236e5f3453fe9d2fd8338bcdfe9756b7 branch: main author: slateny <46876382+slateny(a)users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2022-03-02T19:03:38Z summary: bpo-45492: Corrected documentation for co_names in inspect library doc (GH-31456) files: M Doc/library/inspect.rst diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index 7a6dc9c0337d8..8ee2c070cccf5 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -191,8 +191,9 @@ attributes: | | | which this code object | | | | was defined | +-----------+-------------------+---------------------------+ -| | co_names | tuple of names of local | -| | | variables | +| | co_names | tuple of names other | +| | | than arguments and | +| | | function locals | +-----------+-------------------+---------------------------+ | | co_nlocals | number of local variables | +-----------+-------------------+---------------------------+

1 0

[3.10] bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632) (GH-31634) (GH-31644)
by vstinner March 2, 2022

March 2, 2022

https://github.com/python/cpython/commit/359bc392ba2b8f2acca223426c8210bb74… commit: 359bc392ba2b8f2acca223426c8210bb74f724c6 branch: 3.9 author: Victor Stinner <vstinner(a)python.org> committer: vstinner <vstinner(a)python.org> date: 2022-03-02T18:12:26+01:00 summary: [3.10] bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632) (GH-31634) (GH-31644) * Refactor sanitiser skip tests into test.support (GH-30889) * Refactor sanitizer skip tests into test.support (cherry picked from commit b1cb8430504931f7854eac5d32cba74770078a4e) * Add skips to crashing tests under sanitizers instead of manually skipping them (GH-30897) (cherry picked from commit a27505345e34d462139f5f8b6b5e7c9a59955150) * bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632) Skip tests on ASAN and/or MSAN builds: * multiprocessing tests * test___all__ * test_concurrent_futures * test_decimal * test_peg_generator * test_tools (cherry picked from commit 9204bb72a2da5885facc747e63d2bd2d654606fe) Co-authored-by: Pablo Galindo Salgado <Pablogsal(a)gmail.com> (cherry picked from commit 93264452d952d9ba604bacf2947c2df5dd477931) files: M Lib/test/_test_multiprocessing.py M Lib/test/support/__init__.py M Lib/test/test___all__.py M Lib/test/test_concurrent_futures.py M Lib/test/test_crypt.py M Lib/test/test_decimal.py M Lib/test/test_faulthandler.py M Lib/test/test_idle.py M Lib/test/test_io.py M Lib/test/test_peg_generator/__init__.py M Lib/test/test_tix.py M Lib/test/test_tk.py M Lib/test/test_tools/__init__.py M Lib/test/test_ttk_guionly.py diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index 3ae0cb976863d..bcbf5676b2e3a 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -69,6 +69,12 @@ msvcrt = None +if support.check_sanitizer(address=True): + # bpo-45200: Skip multiprocessing tests if Python is built with ASAN to + # work around a libasan race condition: dead lock in pthread_create(). + raise unittest.SkipTest("libasan has a pthread_create() dead lock") + + def latin(s): return s.encode('latin') diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 53804f13fc8a2..c4115a850b2e4 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -54,7 +54,7 @@ "requires_IEEE_754", "skip_unless_xattr", "requires_zlib", "anticipate_failure", "load_package_tests", "detect_api_mismatch", "check__all__", "skip_if_buggy_ucrt_strfptime", - "ignore_warnings", + "ignore_warnings", "check_sanitizer", "skip_if_sanitizer", # sys "is_jython", "is_android", "check_impl_detail", "unix_shell", "setswitchinterval", @@ -644,6 +644,41 @@ def wrapper(*args, **kw): return decorator +def check_sanitizer(*, address=False, memory=False, ub=False): + """Returns True if Python is compiled with sanitizer support""" + if not (address or memory or ub): + raise ValueError('At least one of address, memory, or ub must be True') + + + _cflags = sysconfig.get_config_var('CFLAGS') or '' + _config_args = sysconfig.get_config_var('CONFIG_ARGS') or '' + memory_sanitizer = ( + '-fsanitize=memory' in _cflags or + '--with-memory-sanitizer' in _config_args + ) + address_sanitizer = ( + '-fsanitize=address' in _cflags or + '--with-memory-sanitizer' in _config_args + ) + ub_sanitizer = ( + '-fsanitize=undefined' in _cflags or + '--with-undefined-behavior-sanitizer' in _config_args + ) + return ( + (memory and memory_sanitizer) or + (address and address_sanitizer) or + (ub and ub_sanitizer) + ) + + +def skip_if_sanitizer(reason=None, *, address=False, memory=False, ub=False): + """Decorator raising SkipTest if running with a sanitizer active.""" + if not reason: + reason = 'not working with sanitizers active' + skip = check_sanitizer(address=address, memory=memory, ub=ub) + return unittest.skipIf(skip, reason) + + def system_must_validate_cert(f): """Skip the test on TLS certificate validation failures.""" @functools.wraps(f) diff --git a/Lib/test/test___all__.py b/Lib/test/test___all__.py index 0ba243ee4e74e..7e1f80f251256 100644 --- a/Lib/test/test___all__.py +++ b/Lib/test/test___all__.py @@ -4,6 +4,13 @@ import sys +if support.check_sanitizer(address=True, memory=True): + # bpo-46633: test___all__ is skipped because importing some modules + # directly can trigger known problems with ASAN (like tk or crypt). + raise unittest.SkipTest("workaround ASAN build issues on loading tests " + "like tk or crypt") + + class NoAll(RuntimeError): pass diff --git a/Lib/test/test_concurrent_futures.py b/Lib/test/test_concurrent_futures.py index d693fb4ee199b..d421b02754cf5 100644 --- a/Lib/test/test_concurrent_futures.py +++ b/Lib/test/test_concurrent_futures.py @@ -32,6 +32,12 @@ import multiprocessing.util +if support.check_sanitizer(address=True, memory=True): + # bpo-46633: Skip the test because it is too slow when Python is built + # with ASAN/MSAN: between 5 and 20 minutes on GitHub Actions. + raise unittest.SkipTest("test too slow on ASAN/MSAN build") + + def create_future(state=PENDING, exception=None, result=None): f = Future() f._state = state diff --git a/Lib/test/test_crypt.py b/Lib/test/test_crypt.py index 5dc83b4ecbfa0..877c575c5534a 100644 --- a/Lib/test/test_crypt.py +++ b/Lib/test/test_crypt.py @@ -1,8 +1,11 @@ import sys import unittest +from test.support import check_sanitizer try: + if check_sanitizer(address=True, memory=True): + raise unittest.SkipTest("The crypt module SEGFAULTs on ASAN/MSAN builds") import crypt IMPORT_ERROR = None except ImportError as ex: diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py index 3f30a935d583c..58f4df3060169 100644 --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -33,24 +33,14 @@ import numbers import locale from test.support import (run_unittest, run_doctest, is_resource_enabled, - requires_IEEE_754, requires_docstrings) -from test.support import (import_fresh_module, TestFailed, + requires_IEEE_754, requires_docstrings, + import_fresh_module, TestFailed, run_with_locale, cpython_only, - darwin_malloc_err_warning) + darwin_malloc_err_warning, + check_sanitizer) import random import inspect import threading -import sysconfig -_cflags = sysconfig.get_config_var('CFLAGS') or '' -_config_args = sysconfig.get_config_var('CONFIG_ARGS') or '' -MEMORY_SANITIZER = ( - '-fsanitize=memory' in _cflags or - '--with-memory-sanitizer' in _config_args -) - -ADDRESS_SANITIZER = ( - '-fsanitize=address' in _cflags -) if sys.platform == 'darwin': @@ -5497,7 +5487,8 @@ def __abs__(self): # Issue 41540: @unittest.skipIf(sys.platform.startswith("aix"), "AIX: default ulimit: test is flaky because of extreme over-allocation") - @unittest.skipIf(MEMORY_SANITIZER or ADDRESS_SANITIZER, "sanitizer defaults to crashing " + @unittest.skipIf(check_sanitizer(address=True, memory=True), + "ASAN/MSAN sanitizer defaults to crashing " "instead of returning NULL for malloc failure.") def test_maxcontext_exact_arith(self): diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py index c64afe88c25d6..2f8f5eec475f1 100644 --- a/Lib/test/test_faulthandler.py +++ b/Lib/test/test_faulthandler.py @@ -5,9 +5,9 @@ import signal import subprocess import sys -import sysconfig from test import support from test.support import script_helper, is_android +from test.support import skip_if_sanitizer import tempfile import unittest from textwrap import dedent @@ -19,16 +19,6 @@ TIMEOUT = 0.5 MS_WINDOWS = (os.name == 'nt') -_cflags = sysconfig.get_config_var('CFLAGS') or '' -_config_args = sysconfig.get_config_var('CONFIG_ARGS') or '' -UB_SANITIZER = ( - '-fsanitize=undefined' in _cflags or - '--with-undefined-behavior-sanitizer' in _config_args -) -MEMORY_SANITIZER = ( - '-fsanitize=memory' in _cflags or - '--with-memory-sanitizer' in _config_args -) def expected_traceback(lineno1, lineno2, header, min_count=1): @@ -271,8 +261,8 @@ def test_gil_released(self): 3, 'Segmentation fault') - @unittest.skipIf(UB_SANITIZER or MEMORY_SANITIZER, - "sanitizer builds change crashing process output.") + @skip_if_sanitizer(memory=True, ub=True, reason="sanitizer " + "builds change crashing process output.") @skip_segfault_on_android def test_enable_file(self): with temporary_filename() as filename: @@ -288,8 +278,8 @@ def test_enable_file(self): @unittest.skipIf(sys.platform == "win32", "subprocess doesn't support pass_fds on Windows") - @unittest.skipIf(UB_SANITIZER or MEMORY_SANITIZER, - "sanitizer builds change crashing process output.") + @skip_if_sanitizer(memory=True, ub=True, reason="sanitizer " + "builds change crashing process output.") @skip_segfault_on_android def test_enable_fd(self): with tempfile.TemporaryFile('wb+') as fp: diff --git a/Lib/test/test_idle.py b/Lib/test/test_idle.py index 310b72c1d72e7..c747b02f64306 100644 --- a/Lib/test/test_idle.py +++ b/Lib/test/test_idle.py @@ -1,5 +1,9 @@ import unittest from test.support import import_module +from test.support import check_sanitizer + +if check_sanitizer(address=True, memory=True): + raise unittest.SkipTest("Tests involvin libX11 can SEGFAULT on ASAN/MSAN builds") # Skip test_idle if _tkinter wasn't built, if tkinter is missing, # if tcl/tk is not the 8.5+ needed for ttk widgets, diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py index 49463e54e3c72..feee861830e40 100644 --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -28,7 +28,6 @@ import random import signal import sys -import sysconfig import textwrap import threading import time @@ -40,7 +39,7 @@ from test import support from test.support.script_helper import ( assert_python_ok, assert_python_failure, run_python_until_end) -from test.support import FakePath +from test.support import FakePath, skip_if_sanitizer import codecs import io # C implementation of io @@ -62,17 +61,6 @@ def byteslike(*pos, **kw): class EmptyStruct(ctypes.Structure): pass -_cflags = sysconfig.get_config_var('CFLAGS') or '' -_config_args = sysconfig.get_config_var('CONFIG_ARGS') or '' -MEMORY_SANITIZER = ( - '-fsanitize=memory' in _cflags or - '--with-memory-sanitizer' in _config_args -) - -ADDRESS_SANITIZER = ( - '-fsanitize=address' in _cflags -) - # Does io.IOBase finalizer log the exception if the close() method fails? # The exception is ignored silently by default in release build. IOBASE_EMITS_UNRAISABLE = (hasattr(sys, "gettotalrefcount") or sys.flags.dev_mode) @@ -1543,8 +1531,8 @@ def test_truncate_on_read_only(self): class CBufferedReaderTest(BufferedReaderTest, SizeofTest): tp = io.BufferedReader - @unittest.skipIf(MEMORY_SANITIZER or ADDRESS_SANITIZER, "sanitizer defaults to crashing " - "instead of returning NULL for malloc failure.") + @skip_if_sanitizer(memory=True, address=True, reason= "sanitizer defaults to crashing " + "instead of returning NULL for malloc failure.") def test_constructor(self): BufferedReaderTest.test_constructor(self) # The allocation can succeed on 32-bit builds, e.g. with more @@ -1892,8 +1880,8 @@ def test_slow_close_from_thread(self): class CBufferedWriterTest(BufferedWriterTest, SizeofTest): tp = io.BufferedWriter - @unittest.skipIf(MEMORY_SANITIZER or ADDRESS_SANITIZER, "sanitizer defaults to crashing " - "instead of returning NULL for malloc failure.") + @skip_if_sanitizer(memory=True, address=True, reason= "sanitizer defaults to crashing " + "instead of returning NULL for malloc failure.") def test_constructor(self): BufferedWriterTest.test_constructor(self) # The allocation can succeed on 32-bit builds, e.g. with more @@ -2391,8 +2379,8 @@ def test_interleaved_readline_write(self): class CBufferedRandomTest(BufferedRandomTest, SizeofTest): tp = io.BufferedRandom - @unittest.skipIf(MEMORY_SANITIZER or ADDRESS_SANITIZER, "sanitizer defaults to crashing " - "instead of returning NULL for malloc failure.") + @skip_if_sanitizer(memory=True, address=True, reason= "sanitizer defaults to crashing " + "instead of returning NULL for malloc failure.") def test_constructor(self): BufferedRandomTest.test_constructor(self) # The allocation can succeed on 32-bit builds, e.g. with more diff --git a/Lib/test/test_peg_generator/__init__.py b/Lib/test/test_peg_generator/__init__.py index fa855f2104c58..77f72fcc7c6e3 100644 --- a/Lib/test/test_peg_generator/__init__.py +++ b/Lib/test/test_peg_generator/__init__.py @@ -1,7 +1,15 @@ -import os - +import os.path +import unittest +from test import support from test.support import load_package_tests + +if support.check_sanitizer(address=True, memory=True): + # bpo-46633: Skip the test because it is too slow when Python is built + # with ASAN/MSAN: between 5 and 20 minutes on GitHub Actions. + raise unittest.SkipTest("test too slow on ASAN/MSAN build") + + # Load all tests in package def load_tests(*args): return load_package_tests(os.path.dirname(__file__), *args) diff --git a/Lib/test/test_tix.py b/Lib/test/test_tix.py index e6ea3d07444ce..9dbf9b240b4fc 100644 --- a/Lib/test/test_tix.py +++ b/Lib/test/test_tix.py @@ -1,7 +1,12 @@ import unittest from test import support +from test.support import check_sanitizer import sys +if check_sanitizer(address=True, memory=True): + raise unittest.SkipTest("Tests involvin libX11 can SEGFAULT on ASAN/MSAN builds") + + # Skip this test if the _tkinter module wasn't built. _tkinter = support.import_module('_tkinter') diff --git a/Lib/test/test_tk.py b/Lib/test/test_tk.py index d45acbfc6e1a8..577ed133ee907 100644 --- a/Lib/test/test_tk.py +++ b/Lib/test/test_tk.py @@ -1,4 +1,10 @@ +import unittest from test import support +from test.support import check_sanitizer + +if check_sanitizer(address=True, memory=True): + raise unittest.SkipTest("Tests involvin libX11 can SEGFAULT on ASAN/MSAN builds") + # Skip test if _tkinter wasn't built. support.import_module('_tkinter') diff --git a/Lib/test/test_tools/__init__.py b/Lib/test/test_tools/__init__.py index eb9acad677d58..3b5c5b2df5f1d 100644 --- a/Lib/test/test_tools/__init__.py +++ b/Lib/test/test_tools/__init__.py @@ -5,6 +5,13 @@ import unittest from test import support + +if support.check_sanitizer(address=True, memory=True): + # bpo-46633: Skip the test because it is too slow when Python is built + # with ASAN/MSAN: between 5 and 20 minutes on GitHub Actions. + raise unittest.SkipTest("test too slow on ASAN/MSAN build") + + basepath = os.path.normpath( os.path.dirname( # <src/install dir> os.path.dirname( # Lib diff --git a/Lib/test/test_ttk_guionly.py b/Lib/test/test_ttk_guionly.py index 0b57d5a0d6c25..e62636637c35f 100644 --- a/Lib/test/test_ttk_guionly.py +++ b/Lib/test/test_ttk_guionly.py @@ -1,5 +1,9 @@ import unittest from test import support +from test.support import check_sanitizer + +if check_sanitizer(address=True, memory=True): + raise unittest.SkipTest("Tests involvin libX11 can SEGFAULT on ASAN/MSAN builds") # Skip this test if _tkinter wasn't built. support.import_module('_tkinter')

1 0

[3.10] bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632) (GH-31634)
by vstinner March 2, 2022

March 2, 2022

https://github.com/python/cpython/commit/93264452d952d9ba604bacf2947c2df5dd… commit: 93264452d952d9ba604bacf2947c2df5dd477931 branch: 3.10 author: Victor Stinner <vstinner(a)python.org> committer: vstinner <vstinner(a)python.org> date: 2022-03-02T17:05:14+01:00 summary: [3.10] bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632) (GH-31634) * Refactor sanitiser skip tests into test.support (GH-30889) * Refactor sanitizer skip tests into test.support (cherry picked from commit b1cb8430504931f7854eac5d32cba74770078a4e) * Add skips to crashing tests under sanitizers instead of manually skipping them (GH-30897) (cherry picked from commit a27505345e34d462139f5f8b6b5e7c9a59955150) * bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632) Skip tests on ASAN and/or MSAN builds: * multiprocessing tests * test___all__ * test_concurrent_futures * test_decimal * test_peg_generator * test_tools (cherry picked from commit 9204bb72a2da5885facc747e63d2bd2d654606fe) Co-authored-by: Pablo Galindo Salgado <Pablogsal(a)gmail.com> files: M Lib/test/_test_multiprocessing.py M Lib/test/support/__init__.py M Lib/test/test___all__.py M Lib/test/test_concurrent_futures.py M Lib/test/test_crypt.py M Lib/test/test_decimal.py M Lib/test/test_faulthandler.py M Lib/test/test_idle.py M Lib/test/test_io.py M Lib/test/test_peg_generator/__init__.py M Lib/test/test_tix.py M Lib/test/test_tk.py M Lib/test/test_tools/__init__.py M Lib/test/test_ttk_guionly.py diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py index 3bc5b8f3d79b0..04657c08db92c 100644 --- a/Lib/test/_test_multiprocessing.py +++ b/Lib/test/_test_multiprocessing.py @@ -73,6 +73,12 @@ msvcrt = None +if support.check_sanitizer(address=True): + # bpo-45200: Skip multiprocessing tests if Python is built with ASAN to + # work around a libasan race condition: dead lock in pthread_create(). + raise unittest.SkipTest("libasan has a pthread_create() dead lock") + + def latin(s): return s.encode('latin') diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 44c5bd5820a75..ed6f1979732d3 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -41,7 +41,7 @@ "requires_IEEE_754", "requires_zlib", "anticipate_failure", "load_package_tests", "detect_api_mismatch", "check__all__", "skip_if_buggy_ucrt_strfptime", - "check_disallow_instantiation", + "check_disallow_instantiation", "check_sanitizer", "skip_if_sanitizer", # sys "is_jython", "is_android", "check_impl_detail", "unix_shell", "setswitchinterval", @@ -367,6 +367,41 @@ def wrapper(*args, **kw): return decorator +def check_sanitizer(*, address=False, memory=False, ub=False): + """Returns True if Python is compiled with sanitizer support""" + if not (address or memory or ub): + raise ValueError('At least one of address, memory, or ub must be True') + + + _cflags = sysconfig.get_config_var('CFLAGS') or '' + _config_args = sysconfig.get_config_var('CONFIG_ARGS') or '' + memory_sanitizer = ( + '-fsanitize=memory' in _cflags or + '--with-memory-sanitizer' in _config_args + ) + address_sanitizer = ( + '-fsanitize=address' in _cflags or + '--with-memory-sanitizer' in _config_args + ) + ub_sanitizer = ( + '-fsanitize=undefined' in _cflags or + '--with-undefined-behavior-sanitizer' in _config_args + ) + return ( + (memory and memory_sanitizer) or + (address and address_sanitizer) or + (ub and ub_sanitizer) + ) + + +def skip_if_sanitizer(reason=None, *, address=False, memory=False, ub=False): + """Decorator raising SkipTest if running with a sanitizer active.""" + if not reason: + reason = 'not working with sanitizers active' + skip = check_sanitizer(address=address, memory=memory, ub=ub) + return unittest.skipIf(skip, reason) + + def system_must_validate_cert(f): """Skip the test on TLS certificate validation failures.""" @functools.wraps(f) diff --git a/Lib/test/test___all__.py b/Lib/test/test___all__.py index 15f42d2d114a6..6368ef02253b6 100644 --- a/Lib/test/test___all__.py +++ b/Lib/test/test___all__.py @@ -5,6 +5,13 @@ import sys +if support.check_sanitizer(address=True, memory=True): + # bpo-46633: test___all__ is skipped because importing some modules + # directly can trigger known problems with ASAN (like tk or crypt). + raise unittest.SkipTest("workaround ASAN build issues on loading tests " + "like tk or crypt") + + class NoAll(RuntimeError): pass diff --git a/Lib/test/test_concurrent_futures.py b/Lib/test/test_concurrent_futures.py index 29e041deeca57..50fa1f189b174 100644 --- a/Lib/test/test_concurrent_futures.py +++ b/Lib/test/test_concurrent_futures.py @@ -32,6 +32,12 @@ import multiprocessing.util +if support.check_sanitizer(address=True, memory=True): + # bpo-46633: Skip the test because it is too slow when Python is built + # with ASAN/MSAN: between 5 and 20 minutes on GitHub Actions. + raise unittest.SkipTest("test too slow on ASAN/MSAN build") + + def create_future(state=PENDING, exception=None, result=None): f = Future() f._state = state diff --git a/Lib/test/test_crypt.py b/Lib/test/test_crypt.py index 5dc83b4ecbfa0..877c575c5534a 100644 --- a/Lib/test/test_crypt.py +++ b/Lib/test/test_crypt.py @@ -1,8 +1,11 @@ import sys import unittest +from test.support import check_sanitizer try: + if check_sanitizer(address=True, memory=True): + raise unittest.SkipTest("The crypt module SEGFAULTs on ASAN/MSAN builds") import crypt IMPORT_ERROR = None except ImportError as ex: diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py index b6173a5ffec96..310c105d9a0e8 100644 --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -34,7 +34,7 @@ import locale from test.support import (run_unittest, run_doctest, is_resource_enabled, requires_IEEE_754, requires_docstrings, - requires_legacy_unicode_capi) + requires_legacy_unicode_capi, check_sanitizer) from test.support import (TestFailed, run_with_locale, cpython_only, darwin_malloc_err_warning) @@ -43,17 +43,6 @@ import random import inspect import threading -import sysconfig -_cflags = sysconfig.get_config_var('CFLAGS') or '' -_config_args = sysconfig.get_config_var('CONFIG_ARGS') or '' -MEMORY_SANITIZER = ( - '-fsanitize=memory' in _cflags or - '--with-memory-sanitizer' in _config_args -) - -ADDRESS_SANITIZER = ( - '-fsanitize=address' in _cflags -) if sys.platform == 'darwin': @@ -5511,7 +5500,8 @@ def __abs__(self): # Issue 41540: @unittest.skipIf(sys.platform.startswith("aix"), "AIX: default ulimit: test is flaky because of extreme over-allocation") - @unittest.skipIf(MEMORY_SANITIZER or ADDRESS_SANITIZER, "sanitizer defaults to crashing " + @unittest.skipIf(check_sanitizer(address=True, memory=True), + "ASAN/MSAN sanitizer defaults to crashing " "instead of returning NULL for malloc failure.") def test_maxcontext_exact_arith(self): diff --git a/Lib/test/test_faulthandler.py b/Lib/test/test_faulthandler.py index ee3f41a108a14..e0f09e821da0c 100644 --- a/Lib/test/test_faulthandler.py +++ b/Lib/test/test_faulthandler.py @@ -6,10 +6,10 @@ import signal import subprocess import sys -import sysconfig from test import support from test.support import os_helper from test.support import script_helper, is_android +from test.support import skip_if_sanitizer import tempfile import unittest from textwrap import dedent @@ -21,16 +21,6 @@ TIMEOUT = 0.5 MS_WINDOWS = (os.name == 'nt') -_cflags = sysconfig.get_config_var('CFLAGS') or '' -_config_args = sysconfig.get_config_var('CONFIG_ARGS') or '' -UB_SANITIZER = ( - '-fsanitize=undefined' in _cflags or - '--with-undefined-behavior-sanitizer' in _config_args -) -MEMORY_SANITIZER = ( - '-fsanitize=memory' in _cflags or - '--with-memory-sanitizer' in _config_args -) def expected_traceback(lineno1, lineno2, header, min_count=1): @@ -310,8 +300,8 @@ def test_gil_released(self): 3, 'Segmentation fault') - @unittest.skipIf(UB_SANITIZER or MEMORY_SANITIZER, - "sanitizer builds change crashing process output.") + @skip_if_sanitizer(memory=True, ub=True, reason="sanitizer " + "builds change crashing process output.") @skip_segfault_on_android def test_enable_file(self): with temporary_filename() as filename: @@ -327,8 +317,8 @@ def test_enable_file(self): @unittest.skipIf(sys.platform == "win32", "subprocess doesn't support pass_fds on Windows") - @unittest.skipIf(UB_SANITIZER or MEMORY_SANITIZER, - "sanitizer builds change crashing process output.") + @skip_if_sanitizer(memory=True, ub=True, reason="sanitizer " + "builds change crashing process output.") @skip_segfault_on_android def test_enable_fd(self): with tempfile.TemporaryFile('wb+') as fp: diff --git a/Lib/test/test_idle.py b/Lib/test/test_idle.py index 8756b766334e8..b94b18a541a70 100644 --- a/Lib/test/test_idle.py +++ b/Lib/test/test_idle.py @@ -1,5 +1,9 @@ import unittest from test.support.import_helper import import_module +from test.support import check_sanitizer + +if check_sanitizer(address=True, memory=True): + raise unittest.SkipTest("Tests involvin libX11 can SEGFAULT on ASAN/MSAN builds") # Skip test_idle if _tkinter wasn't built, if tkinter is missing, # if tcl/tk is not the 8.5+ needed for ttk widgets, diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py index 35013b6a090cb..fb83762cb6af4 100644 --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -28,7 +28,6 @@ import random import signal import sys -import sysconfig import textwrap import threading import time @@ -44,6 +43,7 @@ from test.support import os_helper from test.support import threading_helper from test.support import warnings_helper +from test.support import skip_if_sanitizer from test.support.os_helper import FakePath import codecs @@ -66,17 +66,6 @@ def byteslike(*pos, **kw): class EmptyStruct(ctypes.Structure): pass -_cflags = sysconfig.get_config_var('CFLAGS') or '' -_config_args = sysconfig.get_config_var('CONFIG_ARGS') or '' -MEMORY_SANITIZER = ( - '-fsanitize=memory' in _cflags or - '--with-memory-sanitizer' in _config_args -) - -ADDRESS_SANITIZER = ( - '-fsanitize=address' in _cflags -) - # Does io.IOBase finalizer log the exception if the close() method fails? # The exception is ignored silently by default in release build. IOBASE_EMITS_UNRAISABLE = (hasattr(sys, "gettotalrefcount") or sys.flags.dev_mode) @@ -1550,8 +1539,8 @@ def test_truncate_on_read_only(self): class CBufferedReaderTest(BufferedReaderTest, SizeofTest): tp = io.BufferedReader - @unittest.skipIf(MEMORY_SANITIZER or ADDRESS_SANITIZER, "sanitizer defaults to crashing " - "instead of returning NULL for malloc failure.") + @skip_if_sanitizer(memory=True, address=True, reason= "sanitizer defaults to crashing " + "instead of returning NULL for malloc failure.") def test_constructor(self): BufferedReaderTest.test_constructor(self) # The allocation can succeed on 32-bit builds, e.g. with more @@ -1915,8 +1904,8 @@ def test_slow_close_from_thread(self): class CBufferedWriterTest(BufferedWriterTest, SizeofTest): tp = io.BufferedWriter - @unittest.skipIf(MEMORY_SANITIZER or ADDRESS_SANITIZER, "sanitizer defaults to crashing " - "instead of returning NULL for malloc failure.") + @skip_if_sanitizer(memory=True, address=True, reason= "sanitizer defaults to crashing " + "instead of returning NULL for malloc failure.") def test_constructor(self): BufferedWriterTest.test_constructor(self) # The allocation can succeed on 32-bit builds, e.g. with more @@ -2414,8 +2403,8 @@ def test_interleaved_readline_write(self): class CBufferedRandomTest(BufferedRandomTest, SizeofTest): tp = io.BufferedRandom - @unittest.skipIf(MEMORY_SANITIZER or ADDRESS_SANITIZER, "sanitizer defaults to crashing " - "instead of returning NULL for malloc failure.") + @skip_if_sanitizer(memory=True, address=True, reason= "sanitizer defaults to crashing " + "instead of returning NULL for malloc failure.") def test_constructor(self): BufferedRandomTest.test_constructor(self) # The allocation can succeed on 32-bit builds, e.g. with more diff --git a/Lib/test/test_peg_generator/__init__.py b/Lib/test/test_peg_generator/__init__.py index fa855f2104c58..77f72fcc7c6e3 100644 --- a/Lib/test/test_peg_generator/__init__.py +++ b/Lib/test/test_peg_generator/__init__.py @@ -1,7 +1,15 @@ -import os - +import os.path +import unittest +from test import support from test.support import load_package_tests + +if support.check_sanitizer(address=True, memory=True): + # bpo-46633: Skip the test because it is too slow when Python is built + # with ASAN/MSAN: between 5 and 20 minutes on GitHub Actions. + raise unittest.SkipTest("test too slow on ASAN/MSAN build") + + # Load all tests in package def load_tests(*args): return load_package_tests(os.path.dirname(__file__), *args) diff --git a/Lib/test/test_tix.py b/Lib/test/test_tix.py index 8a60c7c8e1fbd..454baeb38a934 100644 --- a/Lib/test/test_tix.py +++ b/Lib/test/test_tix.py @@ -2,6 +2,11 @@ import unittest from test import support from test.support import import_helper +from test.support import check_sanitizer + +if check_sanitizer(address=True, memory=True): + raise unittest.SkipTest("Tests involvin libX11 can SEGFAULT on ASAN/MSAN builds") + # Skip this test if the _tkinter module wasn't built. _tkinter = import_helper.import_module('_tkinter') diff --git a/Lib/test/test_tk.py b/Lib/test/test_tk.py index 69cc2322cd9aa..8f90cbaba9f7c 100644 --- a/Lib/test/test_tk.py +++ b/Lib/test/test_tk.py @@ -1,5 +1,11 @@ +import unittest from test import support from test.support import import_helper +from test.support import check_sanitizer + +if check_sanitizer(address=True, memory=True): + raise unittest.SkipTest("Tests involvin libX11 can SEGFAULT on ASAN/MSAN builds") + # Skip test if _tkinter wasn't built. import_helper.import_module('_tkinter') diff --git a/Lib/test/test_tools/__init__.py b/Lib/test/test_tools/__init__.py index 61af6578e0953..34b0d3b8fb3eb 100644 --- a/Lib/test/test_tools/__init__.py +++ b/Lib/test/test_tools/__init__.py @@ -6,6 +6,13 @@ from test import support from test.support import import_helper + +if support.check_sanitizer(address=True, memory=True): + # bpo-46633: Skip the test because it is too slow when Python is built + # with ASAN/MSAN: between 5 and 20 minutes on GitHub Actions. + raise unittest.SkipTest("test too slow on ASAN/MSAN build") + + basepath = os.path.normpath( os.path.dirname( # <src/install dir> os.path.dirname( # Lib diff --git a/Lib/test/test_ttk_guionly.py b/Lib/test/test_ttk_guionly.py index 8f59839d066e6..c4919045d75cb 100644 --- a/Lib/test/test_ttk_guionly.py +++ b/Lib/test/test_ttk_guionly.py @@ -1,6 +1,10 @@ import unittest from test import support from test.support import import_helper +from test.support import check_sanitizer + +if check_sanitizer(address=True, memory=True): + raise unittest.SkipTest("Tests involvin libX11 can SEGFAULT on ASAN/MSAN builds") # Skip this test if _tkinter wasn't built. import_helper.import_module('_tkinter')

1 0

bpo-46756: Fix authorization check in urllib.request (GH-31353) (GH-31572)
by pablogsal March 2, 2022

March 2, 2022

https://github.com/python/cpython/commit/1c9701a3de0566c085e03dddc14a8508aa… commit: 1c9701a3de0566c085e03dddc14a8508aaae349e branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: pablogsal <Pablogsal(a)gmail.com> date: 2022-03-02T14:50:32+01:00 summary: bpo-46756: Fix authorization check in urllib.request (GH-31353) (GH-31572) Fix a bug in urllib.request.HTTPPasswordMgr.find_user_password() and urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated() which allowed to bypass authorization. For example, access to URI "example.org/foobar" was allowed if the user was authorized for URI "example.org/foo". (cherry picked from commit e2e72567a1c94c548868f6ee5329363e6036057a) Co-authored-by: Serhiy Storchaka <storchaka(a)gmail.com> Co-authored-by: Serhiy Storchaka <storchaka(a)gmail.com> files: A Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst M Lib/test/test_urllib2.py M Lib/urllib/request.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index b77f90a6ccbf1..b61a40cfd3e8f 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -158,7 +158,6 @@ def test_password_manager(self): self.assertEqual(find_user_pass("Some Realm", "http://example.com/spam"), ('joe', 'password')) - self.assertEqual(find_user_pass("Some Realm", "http://example.com/spam/spam"), ('joe', 'password')) @@ -167,12 +166,29 @@ def test_password_manager(self): add("c", "http://example.com/foo", "foo", "ni") add("c", "http://example.com/bar", "bar", "nini") + add("c", "http://example.com/foo/bar", "foobar", "nibar") self.assertEqual(find_user_pass("c", "http://example.com/foo"), ('foo', 'ni')) - self.assertEqual(find_user_pass("c", "http://example.com/bar"), ('bar', 'nini')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/bar"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foo/baz"), + ('foo', 'ni')) + self.assertEqual(find_user_pass("c", "http://example.com/foobar"), + (None, None)) + + add("c", "http://example.com/baz/", "baz", "ninini") + + self.assertEqual(find_user_pass("c", "http://example.com/baz"), + (None, None)) + self.assertEqual(find_user_pass("c", "http://example.com/baz/"), + ('baz', 'ninini')) + self.assertEqual(find_user_pass("c", "http://example.com/baz/bar"), + ('baz', 'ninini')) # For the same path, newer password should be considered. @@ -1653,8 +1669,9 @@ def test_basic_prior_auth_auto_send(self): auth_prior_handler.add_password( None, request_url, user, password, is_authenticated=True) - is_auth = pwd_manager.is_authenticated(request_url) - self.assertTrue(is_auth) + self.assertTrue(pwd_manager.is_authenticated(request_url)) + self.assertTrue(pwd_manager.is_authenticated(request_url + '/nested')) + self.assertFalse(pwd_manager.is_authenticated(request_url + 'plain')) opener = OpenerDirector() opener.add_handler(auth_prior_handler) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py index 5f67077fb0a5c..8ff5e3e24a496 100644 --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -895,10 +895,10 @@ def is_suburi(self, base, test): return True if base[0] != test[0]: return False - common = posixpath.commonprefix((base[1], test[1])) - if len(common) == len(base[1]): - return True - return False + prefix = base[1] + if prefix[-1:] != '/': + prefix += '/' + return test[1].startswith(prefix) class HTTPPasswordMgrWithDefaultRealm(HTTPPasswordMgr): diff --git a/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst b/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst new file mode 100644 index 0000000000000..1660640c5d3fb --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-02-15-11-57-53.bpo-46756.AigSPi.rst @@ -0,0 +1,5 @@ +Fix a bug in :meth:`urllib.request.HTTPPasswordMgr.find_user_password` and +:meth:`urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated` which +allowed to bypass authorization. For example, access to URI +``example.org/foobar`` was allowed if the user was authorized for URI +``example.org/foo``.

1 0