Mailman 3 October 2024 - Python-checkins

[3.13] gh-126256: Update time.rst to use the same clock as instead of the same clock than (GH-126257) (#126258)
by sobolevn 31 Oct '24

31 Oct '24

https://github.com/python/cpython/commit/256fc5529af23f16f0dca2d0e24a936e1c… commit: 256fc5529af23f16f0dca2d0e24a936e1c4d67d3 branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: sobolevn <mail(a)sobolevn.me> date: 2024-10-31T21:50:20Z summary: [3.13] gh-126256: Update time.rst to use the same clock as instead of the same clock than (GH-126257) (#126258) gh-126256: Update time.rst to use the same clock as instead of the same clock than (GH-126257) Update time.rst to use `the same clock as` instead of `the same clock than` The time documentation uses the same clock than time.monotonic instead of the same clock as time.monotonic, which is grammatically false. This PR fixes changes two instances of `the same clock than` to `the same clock as`. (cherry picked from commit d0abd0b826cfa574d1515c6f8459c9901939388f) Co-authored-by: Alperen Keleş <alpkeles99(a)gmail.com> files: M Doc/library/time.rst diff --git a/Doc/library/time.rst b/Doc/library/time.rst index 8e29e57d00f9b2..9cd5db768e9853 100644 --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -327,7 +327,7 @@ Functions .. impl-detail:: - On CPython, use the same clock than :func:`time.monotonic` and is a + On CPython, use the same clock as :func:`time.monotonic` and is a monotonic clock, i.e. a clock that cannot go backwards. Use :func:`perf_counter_ns` to avoid the precision loss caused by the @@ -339,7 +339,7 @@ Functions On Windows, the function is now system-wide. .. versionchanged:: 3.13 - Use the same clock than :func:`time.monotonic`. + Use the same clock as :func:`time.monotonic`. .. function:: perf_counter_ns() -> int

1 0

gh-126256: Update time.rst to use the same clock as instead of the same clock than (#126257)
by sobolevn 31 Oct '24

31 Oct '24

https://github.com/python/cpython/commit/d0abd0b826cfa574d1515c6f8459c99019… commit: d0abd0b826cfa574d1515c6f8459c9901939388f branch: main author: Alperen Keleş <alpkeles99(a)gmail.com> committer: sobolevn <mail(a)sobolevn.me> date: 2024-10-31T21:44:34Z summary: gh-126256: Update time.rst to use the same clock as instead of the same clock than (#126257) Update time.rst to use `the same clock as` instead of `the same clock than` The time documentation uses the same clock than time.monotonic instead of the same clock as time.monotonic, which is grammatically false. This PR fixes changes two instances of `the same clock than` to `the same clock as`. files: M Doc/library/time.rst diff --git a/Doc/library/time.rst b/Doc/library/time.rst index 8e29e57d00f9b2..9cd5db768e9853 100644 --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -327,7 +327,7 @@ Functions .. impl-detail:: - On CPython, use the same clock than :func:`time.monotonic` and is a + On CPython, use the same clock as :func:`time.monotonic` and is a monotonic clock, i.e. a clock that cannot go backwards. Use :func:`perf_counter_ns` to avoid the precision loss caused by the @@ -339,7 +339,7 @@ Functions On Windows, the function is now system-wide. .. versionchanged:: 3.13 - Use the same clock than :func:`time.monotonic`. + Use the same clock as :func:`time.monotonic`. .. function:: perf_counter_ns() -> int

1 0

[3.12] gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (GH-126120) (#126251)
by willingc 31 Oct '24

31 Oct '24

https://github.com/python/cpython/commit/dc3c075d9eebc82c63ec54bb3f217d67b2… commit: dc3c075d9eebc82c63ec54bb3f217d67b2aea914 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: willingc <carolcode(a)willingconsulting.com> date: 2024-10-31T12:33:48-07:00 summary: [3.12] gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (GH-126120) (#126251) gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (GH-126120) (cherry picked from commit 0e8665554b2f1334e530fd6de5b3a4e908405419) Co-authored-by: Bénédikt Tran <10796600+picnixz(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst M Modules/_asynciomodule.c diff --git a/Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst b/Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst new file mode 100644 index 00000000000000..e54ac17b217c92 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst @@ -0,0 +1,3 @@ +Fix a use-after-free crash on :class:`asyncio.Task` objects for which the +underlying event loop implements an evil :meth:`~object.__getattribute__`. +Reported by Nico-Posada. Patch by Bénédikt Tran. diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 96821762c5ae68..9bb71623ba6c7e 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -2751,7 +2751,11 @@ task_call_step_soon(asyncio_state *state, TaskObj *task, PyObject *arg) return -1; } - int ret = call_soon(state, task->task_loop, cb, NULL, task->task_context); + // Beware: An evil call_soon could alter task_context. + // See: https://github.com/python/cpython/issues/126080. + PyObject *task_context = Py_NewRef(task->task_context); + int ret = call_soon(state, task->task_loop, cb, NULL, task_context); + Py_DECREF(task_context); Py_DECREF(cb); return ret; }

1 0

[3.13] gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (GH-126120) (#126250)
by willingc 31 Oct '24

31 Oct '24

https://github.com/python/cpython/commit/abe64a3780ecaf954cb9bafb5dc6eaee1b… commit: abe64a3780ecaf954cb9bafb5dc6eaee1b680bf4 branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: willingc <carolcode(a)willingconsulting.com> date: 2024-10-31T19:32:47Z summary: [3.13] gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (GH-126120) (#126250) gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (GH-126120) (cherry picked from commit 0e8665554b2f1334e530fd6de5b3a4e908405419) Co-authored-by: Bénédikt Tran <10796600+picnixz(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst M Modules/_asynciomodule.c diff --git a/Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst b/Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst new file mode 100644 index 00000000000000..e54ac17b217c92 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst @@ -0,0 +1,3 @@ +Fix a use-after-free crash on :class:`asyncio.Task` objects for which the +underlying event loop implements an evil :meth:`~object.__getattribute__`. +Reported by Nico-Posada. Patch by Bénédikt Tran. diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 44529f633fbe2e..40fe7d839552cc 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -2678,7 +2678,11 @@ task_call_step_soon(asyncio_state *state, TaskObj *task, PyObject *arg) return -1; } - int ret = call_soon(state, task->task_loop, cb, NULL, task->task_context); + // Beware: An evil call_soon could alter task_context. + // See: https://github.com/python/cpython/issues/126080. + PyObject *task_context = Py_NewRef(task->task_context); + int ret = call_soon(state, task->task_loop, cb, NULL, task_context); + Py_DECREF(task_context); Py_DECREF(cb); return ret; }

1 0

gh-126156: Improve performance of creating `Morsel` objects (#126157)
by hauntsaninja 31 Oct '24

31 Oct '24

https://github.com/python/cpython/commit/dd3c0fa3fd2795326dae0e0ed63c668f55… commit: dd3c0fa3fd2795326dae0e0ed63c668f5506cf32 branch: main author: J. Nick Koston <nick(a)koston.org> committer: hauntsaninja <12621235+hauntsaninja(a)users.noreply.github.com> date: 2024-10-31T12:05:40-07:00 summary: gh-126156: Improve performance of creating `Morsel` objects (#126157) Replaces the manually constructed loop with a call to `dict.update` files: A Misc/NEWS.d/next/Library/2024-10-30-00-12-22.gh-issue-126156.BOSqv0.rst M Lib/http/cookies.py diff --git a/Lib/http/cookies.py b/Lib/http/cookies.py index 6b9ed24ad8ec78..d7e8d08b2d92c1 100644 --- a/Lib/http/cookies.py +++ b/Lib/http/cookies.py @@ -266,6 +266,8 @@ class Morsel(dict): "samesite" : "SameSite", } + _reserved_defaults = dict.fromkeys(_reserved, "") + _flags = {'secure', 'httponly'} def __init__(self): @@ -273,8 +275,7 @@ def __init__(self): self._key = self._value = self._coded_value = None # Set default attributes - for key in self._reserved: - dict.__setitem__(self, key, "") + dict.update(self, self._reserved_defaults) @property def key(self): diff --git a/Misc/NEWS.d/next/Library/2024-10-30-00-12-22.gh-issue-126156.BOSqv0.rst b/Misc/NEWS.d/next/Library/2024-10-30-00-12-22.gh-issue-126156.BOSqv0.rst new file mode 100644 index 00000000000000..4fe18275ab9384 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-30-00-12-22.gh-issue-126156.BOSqv0.rst @@ -0,0 +1 @@ +Improved performances of creating :py:class:`~http.cookies.Morsel` objects by a factor of 3.8x.

1 0

[3.12] gh-124651: Quote template strings in `venv` activation scripts (GH-124712) (GH-126185)
by vsajip 31 Oct '24

31 Oct '24

https://github.com/python/cpython/commit/8450b2482586857d689b6658f08de9c817… commit: 8450b2482586857d689b6658f08de9c8179af7db branch: 3.12 author: Victor Stinner <vstinner(a)python.org> committer: vsajip <vinay_sajip(a)yahoo.co.uk> date: 2024-10-31T18:09:20Z summary: [3.12] gh-124651: Quote template strings in `venv` activation scripts (GH-124712) (GH-126185) (cherry picked from commit d48cc82ed25e26b02eb97c6263d95dcaa1e9111b) files: A Misc/NEWS.d/next/Library/2024-09-28-02-03-04.gh-issue-124651.bLBGtH.rst M Lib/test/test_venv.py M Lib/venv/__init__.py M Lib/venv/scripts/common/activate M Lib/venv/scripts/nt/activate.bat M Lib/venv/scripts/posix/activate.csh M Lib/venv/scripts/posix/activate.fish diff --git a/Lib/test/test_venv.py b/Lib/test/test_venv.py index 83d03aa3bf0d9c..8254a701e09a10 100644 --- a/Lib/test/test_venv.py +++ b/Lib/test/test_venv.py @@ -17,6 +17,7 @@ import sys import sysconfig import tempfile +import shlex from test.support import (captured_stdout, captured_stderr, skip_if_broken_multiprocessing_synchronize, verbose, requires_subprocess, is_emscripten, is_wasi, @@ -97,6 +98,10 @@ def get_text_file_contents(self, *args, encoding='utf-8'): result = f.read() return result + def assertEndsWith(self, string, tail): + if not string.endswith(tail): + self.fail(f"String {string!r} does not end with {tail!r}") + class BasicTest(BaseTest): """Test venv module functionality.""" @@ -446,6 +451,82 @@ def test_executable_symlinks(self): 'import sys; print(sys.executable)']) self.assertEqual(out.strip(), envpy.encode()) + # gh-124651: test quoted strings + @unittest.skipIf(os.name == 'nt', 'contains invalid characters on Windows') + def test_special_chars_bash(self): + """ + Test that the template strings are quoted properly (bash) + """ + rmtree(self.env_dir) + bash = shutil.which('bash') + if bash is None: + self.skipTest('bash required for this test') + env_name = '"\';&&$e|\'"' + env_dir = os.path.join(os.path.realpath(self.env_dir), env_name) + builder = venv.EnvBuilder(clear=True) + builder.create(env_dir) + activate = os.path.join(env_dir, self.bindir, 'activate') + test_script = os.path.join(self.env_dir, 'test_special_chars.sh') + with open(test_script, "w") as f: + f.write(f'source {shlex.quote(activate)}\n' + 'python -c \'import sys; print(sys.executable)\'\n' + 'python -c \'import os; print(os.environ["VIRTUAL_ENV"])\'\n' + 'deactivate\n') + out, err = check_output([bash, test_script]) + lines = out.splitlines() + self.assertTrue(env_name.encode() in lines[0]) + self.assertEndsWith(lines[1], env_name.encode()) + + # gh-124651: test quoted strings + @unittest.skipIf(os.name == 'nt', 'contains invalid characters on Windows') + def test_special_chars_csh(self): + """ + Test that the template strings are quoted properly (csh) + """ + rmtree(self.env_dir) + csh = shutil.which('tcsh') or shutil.which('csh') + if csh is None: + self.skipTest('csh required for this test') + env_name = '"\';&&$e|\'"' + env_dir = os.path.join(os.path.realpath(self.env_dir), env_name) + builder = venv.EnvBuilder(clear=True) + builder.create(env_dir) + activate = os.path.join(env_dir, self.bindir, 'activate.csh') + test_script = os.path.join(self.env_dir, 'test_special_chars.csh') + with open(test_script, "w") as f: + f.write(f'source {shlex.quote(activate)}\n' + 'python -c \'import sys; print(sys.executable)\'\n' + 'python -c \'import os; print(os.environ["VIRTUAL_ENV"])\'\n' + 'deactivate\n') + out, err = check_output([csh, test_script]) + lines = out.splitlines() + self.assertTrue(env_name.encode() in lines[0]) + self.assertEndsWith(lines[1], env_name.encode()) + + # gh-124651: test quoted strings on Windows + @unittest.skipUnless(os.name == 'nt', 'only relevant on Windows') + def test_special_chars_windows(self): + """ + Test that the template strings are quoted properly on Windows + """ + rmtree(self.env_dir) + env_name = "'&&^$e" + env_dir = os.path.join(os.path.realpath(self.env_dir), env_name) + builder = venv.EnvBuilder(clear=True) + builder.create(env_dir) + activate = os.path.join(env_dir, self.bindir, 'activate.bat') + test_batch = os.path.join(self.env_dir, 'test_special_chars.bat') + with open(test_batch, "w") as f: + f.write('@echo off\n' + f'"{activate}" & ' + f'{self.exe} -c "import sys; print(sys.executable)" & ' + f'{self.exe} -c "import os; print(os.environ[\'VIRTUAL_ENV\'])" & ' + 'deactivate') + out, err = check_output([test_batch]) + lines = out.splitlines() + self.assertTrue(env_name.encode() in lines[0]) + self.assertEndsWith(lines[1], env_name.encode()) + @unittest.skipUnless(os.name == 'nt', 'only relevant on Windows') def test_unicode_in_batch_file(self): """ diff --git a/Lib/venv/__init__.py b/Lib/venv/__init__.py index d5dec4ab44b8b5..aeb522c3199ca2 100644 --- a/Lib/venv/__init__.py +++ b/Lib/venv/__init__.py @@ -11,6 +11,7 @@ import sys import sysconfig import types +import shlex CORE_VENV_DEPS = ('pip',) @@ -422,11 +423,41 @@ def replace_variables(self, text, context): :param context: The information for the environment creation request being processed. """ - text = text.replace('__VENV_DIR__', context.env_dir) - text = text.replace('__VENV_NAME__', context.env_name) - text = text.replace('__VENV_PROMPT__', context.prompt) - text = text.replace('__VENV_BIN_NAME__', context.bin_name) - text = text.replace('__VENV_PYTHON__', context.env_exe) + replacements = { + '__VENV_DIR__': context.env_dir, + '__VENV_NAME__': context.env_name, + '__VENV_PROMPT__': context.prompt, + '__VENV_BIN_NAME__': context.bin_name, + '__VENV_PYTHON__': context.env_exe, + } + + def quote_ps1(s): + """ + This should satisfy PowerShell quoting rules [1], unless the quoted + string is passed directly to Windows native commands [2]. + [1]: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.co… + [2]: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.co… + """ + s = s.replace("'", "''") + return f"'{s}'" + + def quote_bat(s): + return s + + # gh-124651: need to quote the template strings properly + quote = shlex.quote + script_path = context.script_path + if script_path.endswith('.ps1'): + quote = quote_ps1 + elif script_path.endswith('.bat'): + quote = quote_bat + else: + # fallbacks to POSIX shell compliant quote + quote = shlex.quote + + replacements = {key: quote(s) for key, s in replacements.items()} + for key, quoted in replacements.items(): + text = text.replace(key, quoted) return text def install_scripts(self, context, path): @@ -466,6 +497,7 @@ def install_scripts(self, context, path): with open(srcfile, 'rb') as f: data = f.read() if not srcfile.endswith(('.exe', '.pdb')): + context.script_path = srcfile try: data = data.decode('utf-8') data = self.replace_variables(data, context) diff --git a/Lib/venv/scripts/common/activate b/Lib/venv/scripts/common/activate index 44df44a7404b43..74825877c38313 100644 --- a/Lib/venv/scripts/common/activate +++ b/Lib/venv/scripts/common/activate @@ -40,14 +40,14 @@ deactivate nondestructive if [ "${OSTYPE:-}" = "cygwin" ] || [ "${OSTYPE:-}" = "msys" ] ; then # transform D:\path\to\venv to /d/path/to/venv on MSYS # and to /cygdrive/d/path/to/venv on Cygwin - export VIRTUAL_ENV=$(cygpath "__VENV_DIR__") + export VIRTUAL_ENV=$(cygpath __VENV_DIR__) else # use the path as-is - export VIRTUAL_ENV="__VENV_DIR__" + export VIRTUAL_ENV=__VENV_DIR__ fi _OLD_VIRTUAL_PATH="$PATH" -PATH="$VIRTUAL_ENV/__VENV_BIN_NAME__:$PATH" +PATH="$VIRTUAL_ENV/"__VENV_BIN_NAME__":$PATH" export PATH # unset PYTHONHOME if set @@ -60,9 +60,9 @@ fi if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then _OLD_VIRTUAL_PS1="${PS1:-}" - PS1="__VENV_PROMPT__${PS1:-}" + PS1=__VENV_PROMPT__"${PS1:-}" export PS1 - VIRTUAL_ENV_PROMPT="__VENV_PROMPT__" + VIRTUAL_ENV_PROMPT=__VENV_PROMPT__ export VIRTUAL_ENV_PROMPT fi diff --git a/Lib/venv/scripts/nt/activate.bat b/Lib/venv/scripts/nt/activate.bat index c1c3c82ee37f10..715b21b13fbe35 100644 --- a/Lib/venv/scripts/nt/activate.bat +++ b/Lib/venv/scripts/nt/activate.bat @@ -8,7 +8,7 @@ if defined _OLD_CODEPAGE ( "%SystemRoot%\System32\chcp.com" 65001 > nul ) -set VIRTUAL_ENV=__VENV_DIR__ +set "VIRTUAL_ENV=__VENV_DIR__" if not defined PROMPT set PROMPT=$P$G @@ -24,8 +24,8 @@ set PYTHONHOME= if defined _OLD_VIRTUAL_PATH set PATH=%_OLD_VIRTUAL_PATH% if not defined _OLD_VIRTUAL_PATH set _OLD_VIRTUAL_PATH=%PATH% -set PATH=%VIRTUAL_ENV%\__VENV_BIN_NAME__;%PATH% -set VIRTUAL_ENV_PROMPT=__VENV_PROMPT__ +set "PATH=%VIRTUAL_ENV%\__VENV_BIN_NAME__;%PATH%" +set "VIRTUAL_ENV_PROMPT=__VENV_PROMPT__" :END if defined _OLD_CODEPAGE ( diff --git a/Lib/venv/scripts/posix/activate.csh b/Lib/venv/scripts/posix/activate.csh index 5e8d66fa9e5061..08f79296f59352 100644 --- a/Lib/venv/scripts/posix/activate.csh +++ b/Lib/venv/scripts/posix/activate.csh @@ -9,17 +9,17 @@ alias deactivate 'test $?_OLD_VIRTUAL_PATH != 0 && setenv PATH "$_OLD_VIRTUAL_PA # Unset irrelevant variables. deactivate nondestructive -setenv VIRTUAL_ENV "__VENV_DIR__" +setenv VIRTUAL_ENV __VENV_DIR__ set _OLD_VIRTUAL_PATH="$PATH" -setenv PATH "$VIRTUAL_ENV/__VENV_BIN_NAME__:$PATH" +setenv PATH "$VIRTUAL_ENV/"__VENV_BIN_NAME__":$PATH" set _OLD_VIRTUAL_PROMPT="$prompt" if (! "$?VIRTUAL_ENV_DISABLE_PROMPT") then - set prompt = "__VENV_PROMPT__$prompt" - setenv VIRTUAL_ENV_PROMPT "__VENV_PROMPT__" + set prompt = __VENV_PROMPT__"$prompt" + setenv VIRTUAL_ENV_PROMPT __VENV_PROMPT__ endif alias pydoc python -m pydoc diff --git a/Lib/venv/scripts/posix/activate.fish b/Lib/venv/scripts/posix/activate.fish index 91ad6442e05692..508cab0db468dd 100644 --- a/Lib/venv/scripts/posix/activate.fish +++ b/Lib/venv/scripts/posix/activate.fish @@ -33,10 +33,10 @@ end # Unset irrelevant variables. deactivate nondestructive -set -gx VIRTUAL_ENV "__VENV_DIR__" +set -gx VIRTUAL_ENV __VENV_DIR__ set -gx _OLD_VIRTUAL_PATH $PATH -set -gx PATH "$VIRTUAL_ENV/__VENV_BIN_NAME__" $PATH +set -gx PATH "$VIRTUAL_ENV/"__VENV_BIN_NAME__ $PATH # Unset PYTHONHOME if set. if set -q PYTHONHOME @@ -56,7 +56,7 @@ if test -z "$VIRTUAL_ENV_DISABLE_PROMPT" set -l old_status $status # Output the venv prompt; color taken from the blue of the Python logo. - printf "%s%s%s" (set_color 4B8BBE) "__VENV_PROMPT__" (set_color normal) + printf "%s%s%s" (set_color 4B8BBE) __VENV_PROMPT__ (set_color normal) # Restore the return status of the previous command. echo "exit $old_status" | . @@ -65,5 +65,5 @@ if test -z "$VIRTUAL_ENV_DISABLE_PROMPT" end set -gx _OLD_FISH_PROMPT_OVERRIDE "$VIRTUAL_ENV" - set -gx VIRTUAL_ENV_PROMPT "__VENV_PROMPT__" + set -gx VIRTUAL_ENV_PROMPT __VENV_PROMPT__ end diff --git a/Misc/NEWS.d/next/Library/2024-09-28-02-03-04.gh-issue-124651.bLBGtH.rst b/Misc/NEWS.d/next/Library/2024-09-28-02-03-04.gh-issue-124651.bLBGtH.rst new file mode 100644 index 00000000000000..17fc9171390dd9 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-09-28-02-03-04.gh-issue-124651.bLBGtH.rst @@ -0,0 +1 @@ +Properly quote template strings in :mod:`venv` activation scripts.

1 0

gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (#126120)
by 1st1 31 Oct '24

31 Oct '24

https://github.com/python/cpython/commit/0e8665554b2f1334e530fd6de5b3a4e908… commit: 0e8665554b2f1334e530fd6de5b3a4e908405419 branch: main author: Bénédikt Tran <10796600+picnixz(a)users.noreply.github.com> committer: 1st1 <yury(a)edgedb.com> date: 2024-10-31T10:14:47-07:00 summary: gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (#126120) files: A Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst M Modules/_asynciomodule.c diff --git a/Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst b/Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst new file mode 100644 index 00000000000000..e54ac17b217c92 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-29-10-38-28.gh-issue-126080.qKRBuo.rst @@ -0,0 +1,3 @@ +Fix a use-after-free crash on :class:`asyncio.Task` objects for which the +underlying event loop implements an evil :meth:`~object.__getattribute__`. +Reported by Nico-Posada. Patch by Bénédikt Tran. diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index c2500fbd692d4d..7483e9c0f43acf 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -2738,7 +2738,11 @@ task_call_step_soon(asyncio_state *state, TaskObj *task, PyObject *arg) return -1; } - int ret = call_soon(state, task->task_loop, cb, NULL, task->task_context); + // Beware: An evil call_soon could alter task_context. + // See: https://github.com/python/cpython/issues/126080. + PyObject *task_context = Py_NewRef(task->task_context); + int ret = call_soon(state, task->task_loop, cb, NULL, task_context); + Py_DECREF(task_context); Py_DECREF(cb); return ret; }

1 0

[3.12] GH-126205: Fix conversion of UNC paths to file URIs (GH-126208) (#126248)
by barneygale 31 Oct '24

31 Oct '24

https://github.com/python/cpython/commit/78f307f39e6d3f4ebb172140945ad2ec1e… commit: 78f307f39e6d3f4ebb172140945ad2ec1ee10811 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: barneygale <barney.gale(a)gmail.com> date: 2024-10-31T16:59:15Z summary: [3.12] GH-126205: Fix conversion of UNC paths to file URIs (GH-126208) (#126248) GH-126205: Fix conversion of UNC paths to file URIs (GH-126208) File URIs for Windows UNC paths should begin with two slashes, not four. (cherry picked from commit 951cb2c369e8b8fb9f93662658391a53fd727787) Co-authored-by: Barney Gale <barney.gale(a)gmail.com> files: A Misc/NEWS.d/next/Library/2024-10-30-20-45-17.gh-issue-126205.CHEmtx.rst M Lib/nturl2path.py M Lib/test/test_urllib.py diff --git a/Lib/nturl2path.py b/Lib/nturl2path.py index 61852aff58912d..6453f202c26d14 100644 --- a/Lib/nturl2path.py +++ b/Lib/nturl2path.py @@ -55,16 +55,11 @@ def pathname2url(p): if p[:4] == '\\\\?\\': p = p[4:] if p[:4].upper() == 'UNC\\': - p = '\\' + p[4:] + p = '\\\\' + p[4:] elif p[1:2] != ':': raise OSError('Bad path: ' + p) if not ':' in p: # No drive specifier, just convert slashes and quote the name - if p[:2] == '\\\\': - # path is something like \\host\path\on\remote\host - # convert this to ////host/path/on/remote/host - # (notice doubling of slashes at the start of the path) - p = '\\\\' + p components = p.split('\\') return urllib.parse.quote('/'.join(components)) comp = p.split(':', maxsplit=2) diff --git a/Lib/test/test_urllib.py b/Lib/test/test_urllib.py index 5a6d877f6a3094..0e1b13d31c9273 100644 --- a/Lib/test/test_urllib.py +++ b/Lib/test/test_urllib.py @@ -1533,7 +1533,7 @@ def test_pathname2url_win(self): # Test special prefixes are correctly handled in pathname2url() fn = urllib.request.pathname2url self.assertEqual(fn('\\\\?\\C:\\dir'), '///C:/dir') - self.assertEqual(fn('\\\\?\\unc\\server\\share\\dir'), '/server/share/dir') + self.assertEqual(fn('\\\\?\\unc\\server\\share\\dir'), '//server/share/dir') self.assertEqual(fn("C:"), '///C:') self.assertEqual(fn("C:\\"), '///C:') self.assertEqual(fn('C:\\a\\b.c'), '///C:/a/b.c') @@ -1544,14 +1544,14 @@ def test_pathname2url_win(self): self.assertRaises(IOError, fn, "XX:\\") # No drive letter self.assertEqual(fn("\\folder\\test\\"), '/folder/test/') - self.assertEqual(fn("\\\\folder\\test\\"), '////folder/test/') - self.assertEqual(fn("\\\\\\folder\\test\\"), '/////folder/test/') - self.assertEqual(fn('\\\\some\\share\\'), '////some/share/') - self.assertEqual(fn('\\\\some\\share\\a\\b.c'), '////some/share/a/b.c') - self.assertEqual(fn('\\\\some\\share\\a\\b%#c\xe9'), '////some/share/a/b%25%23c%C3%A9') + self.assertEqual(fn("\\\\folder\\test\\"), '//folder/test/') + self.assertEqual(fn("\\\\\\folder\\test\\"), '///folder/test/') + self.assertEqual(fn('\\\\some\\share\\'), '//some/share/') + self.assertEqual(fn('\\\\some\\share\\a\\b.c'), '//some/share/a/b.c') + self.assertEqual(fn('\\\\some\\share\\a\\b%#c\xe9'), '//some/share/a/b%25%23c%C3%A9') # Round-tripping urls = ['///C:', - '/////folder/test/', + '///folder/test/', '///C:/foo/bar/spam.foo'] for url in urls: self.assertEqual(fn(urllib.request.url2pathname(url)), url) diff --git a/Misc/NEWS.d/next/Library/2024-10-30-20-45-17.gh-issue-126205.CHEmtx.rst b/Misc/NEWS.d/next/Library/2024-10-30-20-45-17.gh-issue-126205.CHEmtx.rst new file mode 100644 index 00000000000000..c92ffb75056606 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-30-20-45-17.gh-issue-126205.CHEmtx.rst @@ -0,0 +1,2 @@ +Fix issue where :func:`urllib.request.pathname2url` generated URLs beginning +with four slashes (rather than two) when given a Windows UNC path.

1 0

[3.13] GH-126205: Fix conversion of UNC paths to file URIs (GH-126208) (#126249)
by barneygale 31 Oct '24

31 Oct '24

https://github.com/python/cpython/commit/588da2e26a986cb94d9e2aee57f6ecd965… commit: 588da2e26a986cb94d9e2aee57f6ecd965adf918 branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: barneygale <barney.gale(a)gmail.com> date: 2024-10-31T16:58:56Z summary: [3.13] GH-126205: Fix conversion of UNC paths to file URIs (GH-126208) (#126249) GH-126205: Fix conversion of UNC paths to file URIs (GH-126208) File URIs for Windows UNC paths should begin with two slashes, not four. (cherry picked from commit 951cb2c369e8b8fb9f93662658391a53fd727787) Co-authored-by: Barney Gale <barney.gale(a)gmail.com> files: A Misc/NEWS.d/next/Library/2024-10-30-20-45-17.gh-issue-126205.CHEmtx.rst M Lib/nturl2path.py M Lib/test/test_urllib.py diff --git a/Lib/nturl2path.py b/Lib/nturl2path.py index 61852aff58912d..6453f202c26d14 100644 --- a/Lib/nturl2path.py +++ b/Lib/nturl2path.py @@ -55,16 +55,11 @@ def pathname2url(p): if p[:4] == '\\\\?\\': p = p[4:] if p[:4].upper() == 'UNC\\': - p = '\\' + p[4:] + p = '\\\\' + p[4:] elif p[1:2] != ':': raise OSError('Bad path: ' + p) if not ':' in p: # No drive specifier, just convert slashes and quote the name - if p[:2] == '\\\\': - # path is something like \\host\path\on\remote\host - # convert this to ////host/path/on/remote/host - # (notice doubling of slashes at the start of the path) - p = '\\\\' + p components = p.split('\\') return urllib.parse.quote('/'.join(components)) comp = p.split(':', maxsplit=2) diff --git a/Lib/test/test_urllib.py b/Lib/test/test_urllib.py index dc852c8f02758c..3ee17f96b817e1 100644 --- a/Lib/test/test_urllib.py +++ b/Lib/test/test_urllib.py @@ -1524,7 +1524,7 @@ def test_pathname2url_win(self): # Test special prefixes are correctly handled in pathname2url() fn = urllib.request.pathname2url self.assertEqual(fn('\\\\?\\C:\\dir'), '///C:/dir') - self.assertEqual(fn('\\\\?\\unc\\server\\share\\dir'), '/server/share/dir') + self.assertEqual(fn('\\\\?\\unc\\server\\share\\dir'), '//server/share/dir') self.assertEqual(fn("C:"), '///C:') self.assertEqual(fn("C:\\"), '///C:') self.assertEqual(fn('C:\\a\\b.c'), '///C:/a/b.c') @@ -1535,14 +1535,14 @@ def test_pathname2url_win(self): self.assertRaises(IOError, fn, "XX:\\") # No drive letter self.assertEqual(fn("\\folder\\test\\"), '/folder/test/') - self.assertEqual(fn("\\\\folder\\test\\"), '////folder/test/') - self.assertEqual(fn("\\\\\\folder\\test\\"), '/////folder/test/') - self.assertEqual(fn('\\\\some\\share\\'), '////some/share/') - self.assertEqual(fn('\\\\some\\share\\a\\b.c'), '////some/share/a/b.c') - self.assertEqual(fn('\\\\some\\share\\a\\b%#c\xe9'), '////some/share/a/b%25%23c%C3%A9') + self.assertEqual(fn("\\\\folder\\test\\"), '//folder/test/') + self.assertEqual(fn("\\\\\\folder\\test\\"), '///folder/test/') + self.assertEqual(fn('\\\\some\\share\\'), '//some/share/') + self.assertEqual(fn('\\\\some\\share\\a\\b.c'), '//some/share/a/b.c') + self.assertEqual(fn('\\\\some\\share\\a\\b%#c\xe9'), '//some/share/a/b%25%23c%C3%A9') # Round-tripping urls = ['///C:', - '/////folder/test/', + '///folder/test/', '///C:/foo/bar/spam.foo'] for url in urls: self.assertEqual(fn(urllib.request.url2pathname(url)), url) diff --git a/Misc/NEWS.d/next/Library/2024-10-30-20-45-17.gh-issue-126205.CHEmtx.rst b/Misc/NEWS.d/next/Library/2024-10-30-20-45-17.gh-issue-126205.CHEmtx.rst new file mode 100644 index 00000000000000..c92ffb75056606 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-30-20-45-17.gh-issue-126205.CHEmtx.rst @@ -0,0 +1,2 @@ +Fix issue where :func:`urllib.request.pathname2url` generated URLs beginning +with four slashes (rather than two) when given a Windows UNC path.

1 0

[3.12] gh-126240: handle `NULL` returned by `_Py_asdl_expr_seq_new` (GH-126241) (#126245)
by sobolevn 31 Oct '24

31 Oct '24

https://github.com/python/cpython/commit/3416ffaa0844477e006eb1245992262d4c… commit: 3416ffaa0844477e006eb1245992262d4cf2e69a branch: 3.12 author: Bénédikt Tran <10796600+picnixz(a)users.noreply.github.com> committer: sobolevn <mail(a)sobolevn.me> date: 2024-10-31T15:04:17Z summary: [3.12] gh-126240: handle `NULL` returned by `_Py_asdl_expr_seq_new` (GH-126241) (#126245) gh-126240: handle `NULL` returned by `_Py_asdl_expr_seq_new` (#126241) check return value of `_Py_asdl_expr_seq_new` files: M Parser/action_helpers.c diff --git a/Parser/action_helpers.c b/Parser/action_helpers.c index 085622e6551203..be52d89495c7dc 100644 --- a/Parser/action_helpers.c +++ b/Parser/action_helpers.c @@ -1043,6 +1043,9 @@ expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_expr_seq *a, asdl_seq *b, } asdl_expr_seq *args = _Py_asdl_expr_seq_new(total_len, arena); + if (args == NULL) { + return NULL; + } Py_ssize_t i = 0; for (i = 0; i < args_len; i++) { @@ -1210,6 +1213,9 @@ unpack_top_level_joined_strs(Parser *p, asdl_expr_seq *raw_expressions) { } asdl_expr_seq *expressions = _Py_asdl_expr_seq_new(req_size, p->arena); + if (expressions == NULL) { + return NULL; + } Py_ssize_t raw_index, req_index = 0; for (raw_index = 0; raw_index < raw_size; raw_index++) { @@ -1400,6 +1406,9 @@ expr_ty _PyPegen_formatted_value(Parser *p, expr_ty expression, Token *debug, } asdl_expr_seq *values = _Py_asdl_expr_seq_new(2, arena); + if (values == NULL) { + return NULL; + } asdl_seq_SET(values, 0, debug_text); asdl_seq_SET(values, 1, formatted_value); return _PyAST_JoinedStr(values, lineno, col_offset, debug_end_line,

1 0