Mailman 3 November 2024 - Python-checkins

Link to correct class methods in asyncio primitives docs (#127270)
by kumaraditya303 Nov. 29, 2024

Nov. 29, 2024

https://github.com/python/cpython/commit/15d6506d175780bb29e5fcde654e386040… commit: 15d6506d175780bb29e5fcde654e3860408aa93e branch: main author: Ollanta Cuba Gyllensten <ollantaster(a)gmail.com> committer: kumaraditya303 <kumaraditya(a)python.org> date: 2024-11-29T16:20:40Z summary: Link to correct class methods in asyncio primitives docs (#127270) files: M Doc/library/asyncio-sync.rst diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst index 3cf8e2737e85dc..77c2e97da11990 100644 --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -259,8 +259,8 @@ Condition Note that a task *may* return from this call spuriously, which is why the caller should always re-check the state - and be prepared to :meth:`wait` again. For this reason, you may - prefer to use :meth:`wait_for` instead. + and be prepared to :meth:`~Condition.wait` again. For this reason, you may + prefer to use :meth:`~Condition.wait_for` instead. .. coroutinemethod:: wait_for(predicate) @@ -268,7 +268,7 @@ Condition The predicate must be a callable which result will be interpreted as a boolean value. The method will repeatedly - :meth:`wait` until the predicate evaluates to *true*. The final value is the + :meth:`~Condition.wait` until the predicate evaluates to *true*. The final value is the return value. @@ -434,7 +434,7 @@ Barrier .. coroutinemethod:: abort() Put the barrier into a broken state. This causes any active or future - calls to :meth:`wait` to fail with the :class:`BrokenBarrierError`. + calls to :meth:`~Barrier.wait` to fail with the :class:`BrokenBarrierError`. Use this for example if one of the tasks needs to abort, to avoid infinite waiting tasks.

1 0

gh-127316: fix incorrect assertion in setting `__class__` in free-threading (#127399)
by kumaraditya303 Nov. 29, 2024

Nov. 29, 2024

https://github.com/python/cpython/commit/45c5cba318a19dda3ee6f9fc84781cc7a2… commit: 45c5cba318a19dda3ee6f9fc84781cc7a2fbde80 branch: main author: Kumar Aditya <kumaraditya(a)python.org> committer: kumaraditya303 <kumaraditya(a)python.org> date: 2024-11-29T21:44:20+05:30 summary: gh-127316: fix incorrect assertion in setting `__class__` in free-threading (#127399) files: M Lib/test/test_free_threading/test_type.py M Objects/dictobject.c diff --git a/Lib/test/test_free_threading/test_type.py b/Lib/test/test_free_threading/test_type.py index 51463b6bb8c1b4..53f6d778bbecbc 100644 --- a/Lib/test/test_free_threading/test_type.py +++ b/Lib/test/test_free_threading/test_type.py @@ -124,6 +124,21 @@ def work(): for thread in threads: thread.join() + def test_object_class_change(self): + class Base: + def __init__(self): + self.attr = 123 + class ClassA(Base): + pass + class ClassB(Base): + pass + + obj = ClassA() + # keep reference to __dict__ + d = obj.__dict__ + obj.__class__ = ClassB + + def run_one(self, writer_func, reader_func): writer = Thread(target=writer_func) readers = [] diff --git a/Objects/dictobject.c b/Objects/dictobject.c index 49b213eaa817e2..a13d8084d14d66 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -7300,7 +7300,7 @@ _PyDict_DetachFromObject(PyDictObject *mp, PyObject *obj) // We could be called with an unlocked dict when the caller knows the // values are already detached, so we assert after inline values check. - _Py_CRITICAL_SECTION_ASSERT_OBJECT_LOCKED(mp); + ASSERT_WORLD_STOPPED_OR_OBJ_LOCKED(mp); assert(mp->ma_values->embedded == 1); assert(mp->ma_values->valid == 1); assert(Py_TYPE(obj)->tp_flags & Py_TPFLAGS_INLINE_VALUES);

1 0

[3.12] gh-127208: Reject null character in _imp.create_dynamic() (#127400) (#127419)
by vstinner Nov. 29, 2024

Nov. 29, 2024

https://github.com/python/cpython/commit/e7ab97862deb8ad69caccbc5c6e1f7b4e6… commit: e7ab97862deb8ad69caccbc5c6e1f7b4e670afbd branch: 3.12 author: Victor Stinner <vstinner(a)python.org> committer: vstinner <vstinner(a)python.org> date: 2024-11-29T17:03:24+01:00 summary: [3.12] gh-127208: Reject null character in _imp.create_dynamic() (#127400) (#127419) gh-127208: Reject null character in _imp.create_dynamic() (#127400) _imp.create_dynamic() now rejects embedded null characters in the path and in the module name. Backport also the _PyUnicode_AsUTF8NoNUL() function. (cherry picked from commit b14fdadc6c620875a20b7ccc3c9b069e85d8557a) files: M Include/internal/pycore_unicodeobject.h M Lib/test/test_import/__init__.py M Objects/unicodeobject.c M Python/import.c diff --git a/Include/internal/pycore_unicodeobject.h b/Include/internal/pycore_unicodeobject.h index 44eccdea55beb0..cecdabe4155063 100644 --- a/Include/internal/pycore_unicodeobject.h +++ b/Include/internal/pycore_unicodeobject.h @@ -76,6 +76,9 @@ struct _Py_unicode_state { extern void _PyUnicode_ClearInterned(PyInterpreterState *interp); +// Like PyUnicode_AsUTF8(), but check for embedded null characters. +extern const char* _PyUnicode_AsUTF8NoNUL(PyObject *); + #ifdef __cplusplus } diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py index c6accc4183a67f..81b2f33a840f1d 100644 --- a/Lib/test/test_import/__init__.py +++ b/Lib/test/test_import/__init__.py @@ -787,6 +787,19 @@ def test_issue105979(self): self.assertIn("Frozen object named 'x' is invalid", str(cm.exception)) + def test_create_dynamic_null(self): + with self.assertRaisesRegex(ValueError, 'embedded null character'): + class Spec: + name = "a\x00b" + origin = "abc" + _imp.create_dynamic(Spec()) + + with self.assertRaisesRegex(ValueError, 'embedded null character'): + class Spec2: + name = "abc" + origin = "a\x00b" + _imp.create_dynamic(Spec2()) + @skip_if_dont_write_bytecode class FilePermissionTests(unittest.TestCase): diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 7bd4b221c83cf7..5f3dd26ad7b762 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -3991,6 +3991,18 @@ PyUnicode_AsUTF8(PyObject *unicode) return PyUnicode_AsUTF8AndSize(unicode, NULL); } +const char * +_PyUnicode_AsUTF8NoNUL(PyObject *unicode) +{ + Py_ssize_t size; + const char *s = PyUnicode_AsUTF8AndSize(unicode, &size); + if (s && strlen(s) != (size_t)size) { + PyErr_SetString(PyExc_ValueError, "embedded null character"); + return NULL; + } + return s; +} + /* PyUnicode_GetSize() has been deprecated since Python 3.3 because it returned length of Py_UNICODE. diff --git a/Python/import.c b/Python/import.c index 4d0b7fd95569b3..66391c04a0baaa 100644 --- a/Python/import.c +++ b/Python/import.c @@ -917,12 +917,14 @@ extensions_lock_release(void) static void * hashtable_key_from_2_strings(PyObject *str1, PyObject *str2, const char sep) { - Py_ssize_t str1_len, str2_len; - const char *str1_data = PyUnicode_AsUTF8AndSize(str1, &str1_len); - const char *str2_data = PyUnicode_AsUTF8AndSize(str2, &str2_len); + const char *str1_data = _PyUnicode_AsUTF8NoNUL(str1); + const char *str2_data = _PyUnicode_AsUTF8NoNUL(str2); if (str1_data == NULL || str2_data == NULL) { return NULL; } + Py_ssize_t str1_len = strlen(str1_data); + Py_ssize_t str2_len = strlen(str2_data); + /* Make sure sep and the NULL byte won't cause an overflow. */ assert(SIZE_MAX - str1_len - str2_len > 2); size_t size = str1_len + 1 + str2_len + 1;

1 0

[3.13] gh-127208: Reject null character in _imp.create_dynamic() (GH-127400) (#127418)
by vstinner Nov. 29, 2024

Nov. 29, 2024

https://github.com/python/cpython/commit/c4a359dc426bcb1a5f16ef25b0e3110bb4… commit: c4a359dc426bcb1a5f16ef25b0e3110bb463bdad branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: vstinner <vstinner(a)python.org> date: 2024-11-29T15:45:21Z summary: [3.13] gh-127208: Reject null character in _imp.create_dynamic() (GH-127400) (#127418) gh-127208: Reject null character in _imp.create_dynamic() (GH-127400) _imp.create_dynamic() now rejects embedded null characters in the path and in the module name. (cherry picked from commit b14fdadc6c620875a20b7ccc3c9b069e85d8557a) Co-authored-by: Victor Stinner <vstinner(a)python.org> files: M Lib/test/test_import/__init__.py M Python/import.c diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py index 3f71a2bf3d9c39..9139d455f4a16d 100644 --- a/Lib/test/test_import/__init__.py +++ b/Lib/test/test_import/__init__.py @@ -1152,6 +1152,19 @@ def test_script_shadowing_stdlib_sys_path_modification(self): stdout, stderr = popen.communicate() self.assertRegex(stdout, expected_error) + def test_create_dynamic_null(self): + with self.assertRaisesRegex(ValueError, 'embedded null character'): + class Spec: + name = "a\x00b" + origin = "abc" + _imp.create_dynamic(Spec()) + + with self.assertRaisesRegex(ValueError, 'embedded null character'): + class Spec2: + name = "abc" + origin = "a\x00b" + _imp.create_dynamic(Spec2()) + @skip_if_dont_write_bytecode class FilePermissionTests(unittest.TestCase): diff --git a/Python/import.c b/Python/import.c index 0f08e6dfc247bd..ea5a3e4a7622d0 100644 --- a/Python/import.c +++ b/Python/import.c @@ -1155,12 +1155,14 @@ del_extensions_cache_value(struct extensions_cache_value *value) static void * hashtable_key_from_2_strings(PyObject *str1, PyObject *str2, const char sep) { - Py_ssize_t str1_len, str2_len; - const char *str1_data = PyUnicode_AsUTF8AndSize(str1, &str1_len); - const char *str2_data = PyUnicode_AsUTF8AndSize(str2, &str2_len); + const char *str1_data = _PyUnicode_AsUTF8NoNUL(str1); + const char *str2_data = _PyUnicode_AsUTF8NoNUL(str2); if (str1_data == NULL || str2_data == NULL) { return NULL; } + Py_ssize_t str1_len = strlen(str1_data); + Py_ssize_t str2_len = strlen(str2_data); + /* Make sure sep and the NULL byte won't cause an overflow. */ assert(SIZE_MAX - str1_len - str2_len > 2); size_t size = str1_len + 1 + str2_len + 1;

1 0

gh-127208: Reject null character in _imp.create_dynamic() (#127400)
by vstinner Nov. 29, 2024

Nov. 29, 2024

https://github.com/python/cpython/commit/b14fdadc6c620875a20b7ccc3c9b069e85… commit: b14fdadc6c620875a20b7ccc3c9b069e85d8557a branch: main author: Victor Stinner <vstinner(a)python.org> committer: vstinner <vstinner(a)python.org> date: 2024-11-29T16:20:38+01:00 summary: gh-127208: Reject null character in _imp.create_dynamic() (#127400) _imp.create_dynamic() now rejects embedded null characters in the path and in the module name. files: M Lib/test/test_import/__init__.py M Python/import.c diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py index 0e39998ebb3783..c52b7f3e09bea1 100644 --- a/Lib/test/test_import/__init__.py +++ b/Lib/test/test_import/__init__.py @@ -1133,6 +1133,19 @@ def test_script_shadowing_stdlib_sys_path_modification(self): stdout, stderr = popen.communicate() self.assertRegex(stdout, expected_error) + def test_create_dynamic_null(self): + with self.assertRaisesRegex(ValueError, 'embedded null character'): + class Spec: + name = "a\x00b" + origin = "abc" + _imp.create_dynamic(Spec()) + + with self.assertRaisesRegex(ValueError, 'embedded null character'): + class Spec2: + name = "abc" + origin = "a\x00b" + _imp.create_dynamic(Spec2()) + @skip_if_dont_write_bytecode class FilePermissionTests(unittest.TestCase): diff --git a/Python/import.c b/Python/import.c index 09fe95fa1fb647..b3c384c27718ce 100644 --- a/Python/import.c +++ b/Python/import.c @@ -1157,12 +1157,14 @@ del_extensions_cache_value(struct extensions_cache_value *value) static void * hashtable_key_from_2_strings(PyObject *str1, PyObject *str2, const char sep) { - Py_ssize_t str1_len, str2_len; - const char *str1_data = PyUnicode_AsUTF8AndSize(str1, &str1_len); - const char *str2_data = PyUnicode_AsUTF8AndSize(str2, &str2_len); + const char *str1_data = _PyUnicode_AsUTF8NoNUL(str1); + const char *str2_data = _PyUnicode_AsUTF8NoNUL(str2); if (str1_data == NULL || str2_data == NULL) { return NULL; } + Py_ssize_t str1_len = strlen(str1_data); + Py_ssize_t str2_len = strlen(str2_data); + /* Make sure sep and the NULL byte won't cause an overflow. */ assert(SIZE_MAX - str1_len - str2_len > 2); size_t size = str1_len + 1 + str2_len + 1;

1 0

[3.13] gh-124873: Skip timerfd tests on Android (GH-127279) (#127290)
by vstinner Nov. 29, 2024

Nov. 29, 2024

https://github.com/python/cpython/commit/6a0663404b6c76a6d137776b1912f58ad4… commit: 6a0663404b6c76a6d137776b1912f58ad44e8c86 branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: vstinner <vstinner(a)python.org> date: 2024-11-29T11:46:47Z summary: [3.13] gh-124873: Skip timerfd tests on Android (GH-127279) (#127290) gh-124873: Skip timerfd tests on Android (GH-127279) * Revert "[3.13] gh-124873: Tolerate 100 ms in TimerfdTests on Android (GH-127101) (GH-127105)" This reverts commit c09366b1fed2289530581505834b2b262120a7c7. * Skip timerfd tests on Android. (cherry picked from commit 4ca2c82862e3810a7d68799df2bb5198a9afb219) Co-authored-by: Malcolm Smith <smith(a)chaquo.com> Co-authored-by: Victor Stinner <vstinner(a)python.org> files: M Lib/test/test_os.py diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index 10b489fb3e3538..87e7328acab8f8 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -4124,12 +4124,13 @@ def test_eventfd_select(self): os.eventfd_read(fd) @unittest.skipUnless(hasattr(os, 'timerfd_create'), 'requires os.timerfd_create') +(a)unittest.skipIf(sys.platform == "android", "gh-124873: Test is flaky on Android") @support.requires_linux_version(2, 6, 30) class TimerfdTests(unittest.TestCase): # 1 ms accuracy is reliably achievable on every platform except Android - # emulators, where we allow 100 ms (gh-124873). + # emulators, where we allow 10 ms (gh-108277). if sys.platform == "android" and platform.android_ver().is_emulator: - CLOCK_RES_PLACES = 1 + CLOCK_RES_PLACES = 2 else: CLOCK_RES_PLACES = 3

1 0

[3.12] gh-127359: Pin Tcl/Tk to 8 (8.6) for testing macOS (GH-127365) (#127394)
by hugovk Nov. 29, 2024

Nov. 29, 2024

https://github.com/python/cpython/commit/fc0564b3658f4bda9e808042cb9752fb56… commit: fc0564b3658f4bda9e808042cb9752fb56853bc1 branch: 3.12 author: Hugo van Kemenade <1324225+hugovk(a)users.noreply.github.com> committer: hugovk <1324225+hugovk(a)users.noreply.github.com> date: 2024-11-29T13:02:41+02:00 summary: [3.12] gh-127359: Pin Tcl/Tk to 8 (8.6) for testing macOS (GH-127365) (#127394) files: M .github/workflows/reusable-macos.yml diff --git a/.github/workflows/reusable-macos.yml b/.github/workflows/reusable-macos.yml index 6df272c8839f16..acaab45e5b5e12 100644 --- a/.github/workflows/reusable-macos.yml +++ b/.github/workflows/reusable-macos.yml @@ -36,7 +36,10 @@ jobs: path: config.cache key: ${{ github.job }}-${{ inputs.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }} - name: Install Homebrew dependencies - run: brew install pkg-config openssl(a)3.0 xz gdbm tcl-tk + run: | + brew install pkg-config openssl(a)3.0 xz gdbm tcl-tk@8 + # Because alternate versions are not symlinked into place by default: + brew link tcl-tk@8 - name: Configure CPython run: | GDBM_CFLAGS="-I$(brew --prefix gdbm)/include" \

1 0

[3.13] gh-127258: Fix asyncio test_staggered_race_with_eager_tasks() (GH-127358) (#127401)
by vstinner Nov. 29, 2024

Nov. 29, 2024

https://github.com/python/cpython/commit/1056f38b3a94c5024d31d7acf4d6c5fd28… commit: 1056f38b3a94c5024d31d7acf4d6c5fd28f0059b branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: vstinner <vstinner(a)python.org> date: 2024-11-29T11:01:59Z summary: [3.13] gh-127258: Fix asyncio test_staggered_race_with_eager_tasks() (GH-127358) (#127401) gh-127258: Fix asyncio test_staggered_race_with_eager_tasks() (GH-127358) Replace the sleep(2) with a task which is blocked forever. (cherry picked from commit bfabf96b50b7d6a9c15b298a86ba3633b05a1fd7) Co-authored-by: Victor Stinner <vstinner(a)python.org> files: M Lib/test/test_asyncio/test_eager_task_factory.py diff --git a/Lib/test/test_asyncio/test_eager_task_factory.py b/Lib/test/test_asyncio/test_eager_task_factory.py index 31d2a00dbb8c9c..0e2b189f761521 100644 --- a/Lib/test/test_asyncio/test_eager_task_factory.py +++ b/Lib/test/test_asyncio/test_eager_task_factory.py @@ -220,10 +220,14 @@ async def fail(): await asyncio.sleep(0) raise ValueError("no good") + async def blocked(): + fut = asyncio.Future() + await fut + async def run(): winner, index, excs = await asyncio.staggered.staggered_race( [ - lambda: asyncio.sleep(2, result="sleep2"), + lambda: blocked(), lambda: asyncio.sleep(1, result="sleep1"), lambda: fail() ],

1 0

[3.12] gh-127258: Fix asyncio test_staggered_race_with_eager_tasks() (GH-127358) (#127402)
by vstinner Nov. 29, 2024

Nov. 29, 2024

https://github.com/python/cpython/commit/fc6c2de6de787b99ed41610dda09eb227d… commit: fc6c2de6de787b99ed41610dda09eb227d5e2726 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: vstinner <vstinner(a)python.org> date: 2024-11-29T10:59:22Z summary: [3.12] gh-127258: Fix asyncio test_staggered_race_with_eager_tasks() (GH-127358) (#127402) gh-127258: Fix asyncio test_staggered_race_with_eager_tasks() (GH-127358) Replace the sleep(2) with a task which is blocked forever. (cherry picked from commit bfabf96b50b7d6a9c15b298a86ba3633b05a1fd7) Co-authored-by: Victor Stinner <vstinner(a)python.org> files: M Lib/test/test_asyncio/test_eager_task_factory.py diff --git a/Lib/test/test_asyncio/test_eager_task_factory.py b/Lib/test/test_asyncio/test_eager_task_factory.py index b06832e02f00d6..e2a54dd8d768c8 100644 --- a/Lib/test/test_asyncio/test_eager_task_factory.py +++ b/Lib/test/test_asyncio/test_eager_task_factory.py @@ -225,10 +225,14 @@ async def fail(): await asyncio.sleep(0) raise ValueError("no good") + async def blocked(): + fut = asyncio.Future() + await fut + async def run(): winner, index, excs = await asyncio.staggered.staggered_race( [ - lambda: asyncio.sleep(2, result="sleep2"), + lambda: blocked(), lambda: asyncio.sleep(1, result="sleep1"), lambda: fail() ],

1 0

[3.13] fix param type in PyObject_HasAttrWithError (docs) (GH-127403) (#127404)
by kumaraditya303 Nov. 29, 2024

Nov. 29, 2024

https://github.com/python/cpython/commit/92d14b7fab38e1fdd592bc7c22b6898ac6… commit: 92d14b7fab38e1fdd592bc7c22b6898ac6c7231c branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: kumaraditya303 <kumaraditya(a)python.org> date: 2024-11-29T10:59:13Z summary: [3.13] fix param type in PyObject_HasAttrWithError (docs) (GH-127403) (#127404) fix param type in PyObject_HasAttrWithError (docs) (GH-127403) (cherry picked from commit 3afb639f39e89888194d8e74cc498c8da3a58d8e) Co-authored-by: biggus-developerus <74741815+biggus-developerus(a)users.noreply.github.com> files: M Doc/c-api/object.rst diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 630114a4339110..1e822fad13aa5d 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -85,7 +85,7 @@ Object Protocol instead of the :func:`repr`. -.. c:function:: int PyObject_HasAttrWithError(PyObject *o, const char *attr_name) +.. c:function:: int PyObject_HasAttrWithError(PyObject *o, PyObject *attr_name) Returns ``1`` if *o* has the attribute *attr_name*, and ``0`` otherwise. This is equivalent to the Python expression ``hasattr(o, attr_name)``.

1 0