Mailman 3 March 2022 - Python-checkins

bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186)
by miss-islington 31 Mar '22

31 Mar '22

https://github.com/python/cpython/commit/9ed179b07df6ce7432f972f5d069a7c8de… commit: 9ed179b07df6ce7432f972f5d069a7c8dee56e79 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-03-31T14:09:50-07:00 summary: bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186) bpo-47151: Fallback to fork when vfork fails in subprocess. An OS kernel can specifically decide to disallow vfork() in a process. No need for that to prevent us from launching subprocesses. (cherry picked from commit 4a08c4c469d36f99d3a5e0f17ad82ab35dcf2835) Co-authored-by: Gregory P. Smith <greg(a)krypto.org> files: A Misc/NEWS.d/next/Library/2022-03-30-01-17-43.bpo-47151.z-nQkR.rst M Modules/_posixsubprocess.c diff --git a/Misc/NEWS.d/next/Library/2022-03-30-01-17-43.bpo-47151.z-nQkR.rst b/Misc/NEWS.d/next/Library/2022-03-30-01-17-43.bpo-47151.z-nQkR.rst new file mode 100644 index 0000000000000..d4d02459d35de --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-03-30-01-17-43.bpo-47151.z-nQkR.rst @@ -0,0 +1,3 @@ +When subprocess tries to use vfork, it now falls back to fork if vfork +returns an error. This allows use in situations where vfork isn't allowed +by the OS kernel. diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index a58159a277bea..18a81c6ed8b35 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -681,6 +681,12 @@ do_fork_exec(char *const exec_array[], assert(preexec_fn == Py_None); pid = vfork(); + if (pid == -1) { + /* If vfork() fails, fall back to using fork(). When it isn't + * allowed in a process by the kernel, vfork can return -1 + * with errno EINVAL. https://bugs.python.org/issue47151. */ + pid = fork(); + } } else #endif {

1 0

bpo-45099: Document asyncio internal API (GH-32166)
by asvetlov 31 Mar '22

31 Mar '22

https://github.com/python/cpython/commit/ab89ccff3ca6efc2a8e6f5f45c30d568fb… commit: ab89ccff3ca6efc2a8e6f5f45c30d568fb3d212f branch: main author: Andrew Svetlov <andrew.svetlov(a)gmail.com> committer: asvetlov <andrew.svetlov(a)gmail.com> date: 2022-04-01T00:06:07+03:00 summary: bpo-45099: Document asyncio internal API (GH-32166) files: A Doc/library/asyncio-extending.rst A Misc/NEWS.d/next/Documentation/2022-03-29-13-25-49.bpo-45099.dagdhx.rst M Doc/library/asyncio.rst diff --git a/Doc/library/asyncio-extending.rst b/Doc/library/asyncio-extending.rst new file mode 100644 index 0000000000000..619723e61b5f9 --- /dev/null +++ b/Doc/library/asyncio-extending.rst @@ -0,0 +1,93 @@ +.. currentmodule:: asyncio + + +========= +Extending +========= + +The main direction for :mod:`asyncio` extending is writing custom *event loop* +classes. Asyncio has helpers that could be used to simplify this task. + +.. note:: + + Third-parties should reuse existing asyncio code with caution, + a new Python version is free to break backward compatibility + in *internal* part of API. + + +Writing a Custom Event Loop +=========================== + +:class:`asyncio.AbstractEventLoop` declares very many methods. Implementing all them +from scratch is a tedious job. + +A loop can get many common methods implementation for free by inheriting from +:class:`asyncio.BaseEventLoop`. + +In turn, the successor should implement a bunch of *private* methods declared but not +implemented in :class:`asyncio.BaseEventLoop`. + +For example, ``loop.create_connection()`` checks arguments, resolves DNS addresses, and +calls ``loop._make_socket_transport()`` that should be implemented by inherited class. +The ``_make_socket_transport()`` method is not documented and is considered as an +*internal* API. + + + +Future and Task private constructors +==================================== + +:class:`asyncio.Future` and :class:`asyncio.Task` should be never created directly, +please use corresponding :meth:`loop.create_future` and :meth:`loop.create_task`, +or :func:`asyncio.create_task` factories instead. + +However, third-party *event loops* may *reuse* built-in future and task implementations +for the sake of getting a complex and highly optimized code for free. + +For this purpose the following, *private* constructors are listed: + +.. method:: Future.__init__(*, loop=None) + +Create a built-in future instance. + +*loop* is an optional event loop instance. + +.. method:: Task.__init__(coro, *, loop=None, name=None, context=None) + +Create a built-in task instance. + +*loop* is an optional event loop instance. The rest of arguments are described in +:meth:`loop.create_task` description. + + +Task lifetime support +===================== + +A third party task implementation should call the following functions to keep a task +visible by :func:`asyncio.get_tasks` and :func:`asyncio.current_task`: + +.. function:: _register_task(task) + + Register a new *task* as managed by *asyncio*. + + Call the function from a task constructor. + +.. function:: _unregister_task(task) + + Unregister a *task* from *asyncio* internal structures. + + The function should be called when a task is about to finish. + +.. function:: _enter_task(loop, task) + + Switch the current task to the *task* argument. + + Call the function just before executing a portion of embedded *coroutine* + (:meth:`coroutine.send` or :meth:`coroutine.throw`). + +.. function:: _leave_task(loop, task) + + Switch the current task back from *task* to ``None``. + + Call the function just after :meth:`coroutine.send` or :meth:`coroutine.throw` + execution. diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index 8b3a060ffad52..8f4d55a9de059 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -84,6 +84,7 @@ Additionally, there are **low-level** APIs for asyncio-protocol.rst asyncio-policy.rst asyncio-platforms.rst + asyncio-extending.rst .. toctree:: :caption: Guides and Tutorials diff --git a/Misc/NEWS.d/next/Documentation/2022-03-29-13-25-49.bpo-45099.dagdhx.rst b/Misc/NEWS.d/next/Documentation/2022-03-29-13-25-49.bpo-45099.dagdhx.rst new file mode 100644 index 0000000000000..7b38682b452d2 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2022-03-29-13-25-49.bpo-45099.dagdhx.rst @@ -0,0 +1 @@ +Document internal :mod:`asyncio` API.

1 0

bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186)
by gpshead 31 Mar '22

31 Mar '22

https://github.com/python/cpython/commit/4a08c4c469d36f99d3a5e0f17ad82ab35d… commit: 4a08c4c469d36f99d3a5e0f17ad82ab35dcf2835 branch: main author: Gregory P. Smith <greg(a)krypto.org> committer: gpshead <greg(a)krypto.org> date: 2022-03-31T13:42:28-07:00 summary: bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186) bpo-47151: Fallback to fork when vfork fails in subprocess. An OS kernel can specifically decide to disallow vfork() in a process. No need for that to prevent us from launching subprocesses. files: A Misc/NEWS.d/next/Library/2022-03-30-01-17-43.bpo-47151.z-nQkR.rst M Modules/_posixsubprocess.c diff --git a/Misc/NEWS.d/next/Library/2022-03-30-01-17-43.bpo-47151.z-nQkR.rst b/Misc/NEWS.d/next/Library/2022-03-30-01-17-43.bpo-47151.z-nQkR.rst new file mode 100644 index 0000000000000..d4d02459d35de --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-03-30-01-17-43.bpo-47151.z-nQkR.rst @@ -0,0 +1,3 @@ +When subprocess tries to use vfork, it now falls back to fork if vfork +returns an error. This allows use in situations where vfork isn't allowed +by the OS kernel. diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index de599f8c970e3..440c7c5b33599 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -685,6 +685,12 @@ do_fork_exec(char *const exec_array[], assert(preexec_fn == Py_None); pid = vfork(); + if (pid == -1) { + /* If vfork() fails, fall back to using fork(). When it isn't + * allowed in a process by the kernel, vfork can return -1 + * with errno EINVAL. https://bugs.python.org/issue47151. */ + pid = fork(); + } } else #endif {

1 0

bpo-47146: Stop Depending On regen-deepfreeze For regen-global-objects (gh-32218)
by ericsnowcurrently 31 Mar '22

31 Mar '22

https://github.com/python/cpython/commit/e7bb7c2f047b4f97e4426c42ae209c9698… commit: e7bb7c2f047b4f97e4426c42ae209c969808069d branch: main author: Eric Snow <ericsnowcurrently(a)gmail.com> committer: ericsnowcurrently <ericsnowcurrently(a)gmail.com> date: 2022-03-31T14:29:52-06:00 summary: bpo-47146: Stop Depending On regen-deepfreeze For regen-global-objects (gh-32218) This effectively reverts the Makefile change in gh-31637. I've added some notes so it is more clear what is going on. We also update the "Check if generated files are up to date" job to run "make regen-deepfreeze" to ensure "make regen-global-objects" catches deepfreeze.c. https://bugs.python.org/issue47146 files: M .github/workflows/build.yml M Makefile.pre.in M Tools/scripts/freeze_modules.py diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index bda6dde37d185..0ca8d3910814a 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -82,6 +82,9 @@ jobs: run: make regen-configure - name: Build CPython run: | + # Deepfreeze will usually cause global objects to be added or removed, + # so we run it before regen-global-objects gets rum (in regen-all). + make regen-deepfreeze make -j4 regen-all make regen-stdlib-module-names - name: Check for changes diff --git a/Makefile.pre.in b/Makefile.pre.in index e784b43d0362d..6dda71bc49cff 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1161,8 +1161,9 @@ Python/deepfreeze/deepfreeze.c: $(DEEPFREEZE_DEPS) Python/frozen_modules/__phello__.spam.h:__phello__.spam \ Python/frozen_modules/frozen_only.h:frozen_only \ -o Python/deepfreeze/deepfreeze.c - # END: deepfreeze modules + @echo "Note: Deepfreeze may have added some global objects," + @echo " so run 'make regen-global-objects' if necessary." # We keep this renamed target around for folks with muscle memory. .PHONY: regen-importlib @@ -1171,24 +1172,11 @@ regen-importlib: regen-frozen ############################################################################ # Global objects -GLOBAL_OBJECTS_TARGETS = \ - $(srcdir)/Include/internal/pycore_global_objects.h \ - $(srcdir)/Include/internal/pycore_global_strings.h - -# The global objects will get regenerated as soon these files -# are required, including as a prerequisite for regen-deepfreeze. -$(GLOBAL_OBJECTS_TARGETS): generate-global-objects - -.PHONY: generate-global-objects -generate-global-objects: $(srcdir)/Tools/scripts/generate_global_objects.py - $(PYTHON_FOR_REGEN) $(srcdir)/Tools/scripts/generate_global_objects.py - -.PHONY: generate-global-objects-after-deepfreeze -generate-global-objects-after-deepfreeze: regen-deepfreeze $(srcdir)/Tools/scripts/generate_global_objects.py - $(PYTHON_FOR_REGEN) $(srcdir)/Tools/scripts/generate_global_objects.py - .PHONY: regen-global-objects -regen-global-objects: regen-deepfreeze generate-global-objects-after-deepfreeze +regen-global-objects: $(srcdir)/Tools/scripts/generate_global_objects.py + $(PYTHON_FOR_REGEN) $(srcdir)/Tools/scripts/generate_global_objects.py + @echo "Note: Global objects can be added or removed by other tools (e.g. deepfreeze), " + @echo " so be sure to re-run regen-global-objects after those tools." ############################################################################ # ABI diff --git a/Tools/scripts/freeze_modules.py b/Tools/scripts/freeze_modules.py index c8d8a7dd1fbab..dd208c7847194 100644 --- a/Tools/scripts/freeze_modules.py +++ b/Tools/scripts/freeze_modules.py @@ -606,7 +606,6 @@ def regen_makefile(modules): ]) deepfreezerules.append(f"\t{frozen_header}:{src.frozenid} \\") deepfreezerules.append('\t-o Python/deepfreeze/deepfreeze.c') - deepfreezerules.append('') pyfiles[-1] = pyfiles[-1].rstrip(" \\") frozenfiles[-1] = frozenfiles[-1].rstrip(" \\")

1 0

bpo-40421: Add missing getters for frame object attributes to C-API. (GH-32114)
by markshannon 31 Mar '22

31 Mar '22

https://github.com/python/cpython/commit/74b95d86e0f14603f878c4df3133bc8a93… commit: 74b95d86e0f14603f878c4df3133bc8a93f8f80a branch: main author: Mark Shannon <mark(a)hotpy.org> committer: markshannon <mark(a)hotpy.org> date: 2022-03-31T17:13:25+01:00 summary: bpo-40421: Add missing getters for frame object attributes to C-API. (GH-32114) files: A Misc/NEWS.d/next/C API/2022-03-25-13-40-46.bpo-40421.wJREl2.rst M Doc/c-api/frame.rst M Doc/whatsnew/3.11.rst M Include/cpython/frameobject.h M Lib/test/test_capi.py M Modules/_testcapimodule.c M Objects/frameobject.c diff --git a/Doc/c-api/frame.rst b/Doc/c-api/frame.rst index 0c11bc163b417..6d265e45659f9 100644 --- a/Doc/c-api/frame.rst +++ b/Doc/c-api/frame.rst @@ -30,6 +30,17 @@ See also :ref:`Reflection <reflection>`. .. versionadded:: 3.9 +.. c:function:: PyObject* PyFrame_GetBuiltins(PyFrameObject *frame) + + Get the *frame*'s ``f_builtins`` attribute. + + Return a :term:`strong reference`. The result cannot be ``NULL``. + + *frame* must not be ``NULL``. + + .. versionadded:: 3.11 + + .. c:function:: PyCodeObject* PyFrame_GetCode(PyFrameObject *frame) Get the *frame* code. @@ -41,6 +52,30 @@ See also :ref:`Reflection <reflection>`. .. versionadded:: 3.9 +.. c:function:: PyObject* PyFrame_GetGenerator(PyFrameObject *frame) + + Get the generator, coroutine, or async generator that owns this frame, + or ``NULL`` if this frame is not owned by a generator. + Does not raise an exception, even if the return value is ``NULL``. + + Return a :term:`strong reference`, or ``NULL``. + + *frame* must not be ``NULL``. + + .. versionadded:: 3.11 + + +.. c:function:: PyObject* PyFrame_GetGlobals(PyFrameObject *frame) + + Get the *frame*'s ``f_globals`` attribute. + + Return a :term:`strong reference`. The result cannot be ``NULL``. + + *frame* must not be ``NULL``. + + .. versionadded:: 3.11 + + .. c:function:: PyObject* PyFrame_GetLocals(PyFrameObject *frame) Get the *frame*'s ``f_locals`` attribute (:class:`dict`). diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 1bd958724f3be..16715c32502e4 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -868,6 +868,9 @@ New Features :c:func:`PyFloat_Unpack8`. (Contributed by Victor Stinner in :issue:`46906`.) +* Add new functions to get frame object attributes: + :c:func:`PyFrame_GetBuiltins`, :c:func:`PyFrame_GetGenerator`, + :c:func:`PyFrame_GetGlobals`. Porting to Python 3.11 ---------------------- @@ -985,13 +988,13 @@ Porting to Python 3.11 * ``f_back``: use :c:func:`PyFrame_GetBack`. * ``f_blockstack``: removed. - * ``f_builtins``: use ``PyObject_GetAttrString((PyObject*)frame, "f_builtins")``. + * ``f_builtins``: use :c:func:`PyFrame_GetBuiltins`. * ``f_code``: use :c:func:`PyFrame_GetCode`. - * ``f_gen``: removed. - * ``f_globals``: use ``PyObject_GetAttrString((PyObject*)frame, "f_globals")``. + * ``f_gen``: use :c:func:`PyFrame_GetGenerator`. + * ``f_globals``: use :c:func:`PyFrame_GetGlobals`. * ``f_iblock``: removed. * ``f_lasti``: use ``PyObject_GetAttrString((PyObject*)frame, "f_lasti")``. - Code using ``f_lasti`` with ``PyCode_Addr2Line()`` must use + Code using ``f_lasti`` with ``PyCode_Addr2Line()`` should use :c:func:`PyFrame_GetLineNumber` instead. * ``f_lineno``: use :c:func:`PyFrame_GetLineNumber` * ``f_locals``: use :c:func:`PyFrame_GetLocals`. diff --git a/Include/cpython/frameobject.h b/Include/cpython/frameobject.h index d54d3652a0dbc..ffeb8bd04a46f 100644 --- a/Include/cpython/frameobject.h +++ b/Include/cpython/frameobject.h @@ -24,3 +24,8 @@ PyAPI_FUNC(void) PyFrame_FastToLocals(PyFrameObject *); PyAPI_FUNC(PyFrameObject *) PyFrame_GetBack(PyFrameObject *frame); PyAPI_FUNC(PyObject *) PyFrame_GetLocals(PyFrameObject *frame); + +PyAPI_FUNC(PyObject *) PyFrame_GetGlobals(PyFrameObject *frame); +PyAPI_FUNC(PyObject *) PyFrame_GetBuiltins(PyFrameObject *frame); + +PyAPI_FUNC(PyObject *) PyFrame_GetGenerator(PyFrameObject *frame); diff --git a/Lib/test/test_capi.py b/Lib/test/test_capi.py index d9615430327a4..238acf94526a3 100644 --- a/Lib/test/test_capi.py +++ b/Lib/test/test_capi.py @@ -1087,5 +1087,25 @@ class Subclass(BaseException, self.module.StateAccessType): self.assertIs(Subclass().get_defining_module(), self.module) +class Test_FrameAPI(unittest.TestCase): + + def getframe(self): + return sys._getframe() + + def getgenframe(self): + yield sys._getframe() + + def test_frame_getters(self): + frame = self.getframe() + self.assertEquals(frame.f_locals, _testcapi.frame_getlocals(frame)) + self.assertIs(frame.f_globals, _testcapi.frame_getglobals(frame)) + self.assertIs(frame.f_builtins, _testcapi.frame_getbuiltins(frame)) + + def test_frame_get_generator(self): + gen = self.getgenframe() + frame = next(gen) + self.assertIs(gen, _testcapi.frame_getgenerator(frame)) + + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/C API/2022-03-25-13-40-46.bpo-40421.wJREl2.rst b/Misc/NEWS.d/next/C API/2022-03-25-13-40-46.bpo-40421.wJREl2.rst new file mode 100644 index 0000000000000..95b7b69347ce4 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-03-25-13-40-46.bpo-40421.wJREl2.rst @@ -0,0 +1,3 @@ +Add ``PyFrame_GetBuiltins``, ``PyFrame_GetGenerator`` and +``PyFrame_GetGlobals`` C-API functions to access frame object attributes +safely from C code. diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 019c2b85b6156..759656ae1f8a1 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -5853,6 +5853,46 @@ test_float_unpack(PyObject *self, PyObject *args) return PyFloat_FromDouble(d); } +static PyObject * +frame_getlocals(PyObject *self, PyObject *frame) +{ + if (!PyFrame_Check(frame)) { + PyErr_SetString(PyExc_TypeError, "argument must be a frame"); + return NULL; + } + return PyFrame_GetLocals((PyFrameObject *)frame); +} + +static PyObject * +frame_getglobals(PyObject *self, PyObject *frame) +{ + if (!PyFrame_Check(frame)) { + PyErr_SetString(PyExc_TypeError, "argument must be a frame"); + return NULL; + } + return PyFrame_GetGlobals((PyFrameObject *)frame); +} + +static PyObject * +frame_getgenerator(PyObject *self, PyObject *frame) +{ + if (!PyFrame_Check(frame)) { + PyErr_SetString(PyExc_TypeError, "argument must be a frame"); + return NULL; + } + return PyFrame_GetGenerator((PyFrameObject *)frame); +} + +static PyObject * +frame_getbuiltins(PyObject *self, PyObject *frame) +{ + if (!PyFrame_Check(frame)) { + PyErr_SetString(PyExc_TypeError, "argument must be a frame"); + return NULL; + } + return PyFrame_GetBuiltins((PyFrameObject *)frame); +} + static PyObject *negative_dictoffset(PyObject *, PyObject *); static PyObject *test_buildvalue_issue38913(PyObject *, PyObject *); @@ -6142,6 +6182,10 @@ static PyMethodDef TestMethods[] = { {"test_tstate_capi", test_tstate_capi, METH_NOARGS, NULL}, {"float_pack", test_float_pack, METH_VARARGS, NULL}, {"float_unpack", test_float_unpack, METH_VARARGS, NULL}, + {"frame_getlocals", frame_getlocals, METH_O, NULL}, + {"frame_getglobals", frame_getglobals, METH_O, NULL}, + {"frame_getgenerator", frame_getgenerator, METH_O, NULL}, + {"frame_getbuiltins", frame_getbuiltins, METH_O, NULL}, {NULL, NULL} /* sentinel */ }; diff --git a/Objects/frameobject.c b/Objects/frameobject.c index d49931048a625..581de22587219 100644 --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -1134,6 +1134,28 @@ PyFrame_GetLocals(PyFrameObject *frame) return frame_getlocals(frame, NULL); } +PyObject* +PyFrame_GetGlobals(PyFrameObject *frame) +{ + return frame_getglobals(frame, NULL); +} + +PyObject* +PyFrame_GetBuiltins(PyFrameObject *frame) +{ + return frame_getbuiltins(frame, NULL); +} + +PyObject * +PyFrame_GetGenerator(PyFrameObject *frame) +{ + if (frame->f_frame->owner != FRAME_OWNED_BY_GENERATOR) { + return NULL; + } + PyGenObject *gen = _PyFrame_GetGenerator(frame->f_frame); + return Py_NewRef(gen); +} + PyObject* _PyEval_BuiltinsFromGlobals(PyThreadState *tstate, PyObject *globals) {

1 0

bpo-47182: Fix crash by named unicode characters after interpreter reinitialization (GH-32212)
by miss-islington 31 Mar '22

31 Mar '22

https://github.com/python/cpython/commit/44e915028d75f7cef141aa1aada962465a… commit: 44e915028d75f7cef141aa1aada962465a5907d6 branch: main author: Christian Heimes <christian(a)python.org> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-03-31T08:14:50-07:00 summary: bpo-47182: Fix crash by named unicode characters after interpreter reinitialization (GH-32212) Automerge-Triggered-By: GH:tiran files: A Misc/NEWS.d/next/Core and Builtins/2022-03-31-15-37-02.bpo-47182.e_4SsC.rst M Lib/test/test_embed.py M Objects/unicodeobject.c diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index f0c88de68e89e..7e5e4c144851e 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -343,6 +343,11 @@ def test_finalize_structseq(self): out, err = self.run_embedded_interpreter("test_repeated_init_exec", code) self.assertEqual(out, 'Tests passed\n' * INIT_LOOPS) + def test_ucnhash_capi_reset(self): + # bpo-47182: unicodeobject.c:ucnhash_capi was not reset on shutdown. + code = "print('\\N{digit nine}')" + out, err = self.run_embedded_interpreter("test_repeated_init_exec", code) + self.assertEqual(out, '9\n' * INIT_LOOPS) class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase): maxDiff = 4096 diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-03-31-15-37-02.bpo-47182.e_4SsC.rst b/Misc/NEWS.d/next/Core and Builtins/2022-03-31-15-37-02.bpo-47182.e_4SsC.rst new file mode 100644 index 0000000000000..08036bc680933 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-03-31-15-37-02.bpo-47182.e_4SsC.rst @@ -0,0 +1,2 @@ +Fix a crash when using a named unicode character like ``"\N{digit nine}"`` +after the main interpreter has been initialized a second time. diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 5a1d2c0630167..2d4096397784f 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -16085,6 +16085,9 @@ _PyUnicode_Fini(PyInterpreterState *interp) if (_Py_IsMainInterpreter(interp)) { // _PyUnicode_ClearInterned() must be called before _PyUnicode_Fini() assert(interned == NULL); + // bpo-47182: force a unicodedata CAPI capsule re-import on + // subsequent initialization of main interpreter. + ucnhash_capi = NULL; } _PyUnicode_FiniEncodings(&state->fs_codec);

1 0

bpo-14911: Corrected generator.throw() documentation (GH-32207)
by miss-islington 31 Mar '22

31 Mar '22

https://github.com/python/cpython/commit/98d57737de73342d33d1b90dc0285f5864… commit: 98d57737de73342d33d1b90dc0285f586465d22b branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-03-31T07:24:39-07:00 summary: bpo-14911: Corrected generator.throw() documentation (GH-32207) Co-authored-by: Andrew Svetlov <andrew.svetlov(a)gmail.com> (cherry picked from commit 8be7c2bc5ad5e295f0f855bb31db412eef2c7c92) Co-authored-by: Dave Goncalves <davegoncalves(a)gmail.com> files: M Doc/howto/functional.rst M Doc/reference/datamodel.rst M Doc/reference/expressions.rst M Objects/genobject.c diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst index c7f8bc8f17f43..695b9b31a762b 100644 --- a/Doc/howto/functional.rst +++ b/Doc/howto/functional.rst @@ -589,7 +589,7 @@ generator function. In addition to :meth:`~generator.send`, there are two other methods on generators: -* :meth:`throw(type, value=None, traceback=None) <generator.throw>` is used to +* :meth:`throw(value) <generator.throw>` is used to raise an exception inside the generator; the exception is raised by the ``yield`` expression where the generator's execution is paused. diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 84320f41f6cae..033d65fd7c90f 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2854,7 +2854,8 @@ generators, coroutines do not directly support iteration. :exc:`StopIteration`, or other exception) is the same as when iterating over the :meth:`__await__` return value, described above. -.. method:: coroutine.throw(type[, value[, traceback]]) +.. method:: coroutine.throw(value) + coroutine.throw(type[, value[, traceback]]) Raises the specified exception in the coroutine. This method delegates to the :meth:`~generator.throw` method of the iterator that caused diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 4ffb6512107f2..502c022501eff 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -557,14 +557,27 @@ is already executing raises a :exc:`ValueError` exception. could receive the value. -.. method:: generator.throw(type[, value[, traceback]]) +.. method:: generator.throw(value) + generator.throw(type[, value[, traceback]]) - Raises an exception of type ``type`` at the point where the generator was paused, + Raises an exception at the point where the generator was paused, and returns the next value yielded by the generator function. If the generator exits without yielding another value, a :exc:`StopIteration` exception is raised. If the generator function does not catch the passed-in exception, or raises a different exception, then that exception propagates to the caller. + In typical use, this is called with a single exception instance similar to the + way the :keyword:`raise` keyword is used. + + For backwards compatability, however, the second signature is + supported, following a convention from older versions of Python. + The *type* argument should be an exception class, and *value* + should be an exception instance. If the *value* is not provided, the + *type* constructor is called to get an instance. If *traceback* + is provided, it is set on the exception, otherwise any existing + :attr:`~BaseException.__traceback__` attribute stored in *value* may + be cleared. + .. index:: exception: GeneratorExit diff --git a/Objects/genobject.c b/Objects/genobject.c index 5ba4de82ea70e..95614729c8fab 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -384,8 +384,11 @@ gen_close(PyGenObject *gen, PyObject *args) PyDoc_STRVAR(throw_doc, -"throw(typ[,val[,tb]]) -> raise exception in generator,\n\ -return next yielded value or raise StopIteration."); +"throw(value)\n\ +throw(type[,value[,tb]])\n\ +\n\ +Raise exception in generator, return next yielded value or raise\n\ +StopIteration."); static PyObject * _gen_throw(PyGenObject *gen, int close_on_genexit, @@ -943,8 +946,11 @@ PyDoc_STRVAR(coro_send_doc, return next iterated value or raise StopIteration."); PyDoc_STRVAR(coro_throw_doc, -"throw(typ[,val[,tb]]) -> raise exception in coroutine,\n\ -return next iterated value or raise StopIteration."); +"throw(value)\n\ +throw(type[,value[,traceback]])\n\ +\n\ +Raise exception in coroutine, return next iterated value or raise\n\ +StopIteration."); PyDoc_STRVAR(coro_close_doc, "close() -> raise GeneratorExit inside coroutine.");

1 0

bpo-14911: Corrected generator.throw() documentation (GH-32207)
by miss-islington 31 Mar '22

31 Mar '22

https://github.com/python/cpython/commit/625f6704c0d783360574bbab2f78b0b9bb… commit: 625f6704c0d783360574bbab2f78b0b9bbed5891 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-03-31T07:23:04-07:00 summary: bpo-14911: Corrected generator.throw() documentation (GH-32207) Co-authored-by: Andrew Svetlov <andrew.svetlov(a)gmail.com> (cherry picked from commit 8be7c2bc5ad5e295f0f855bb31db412eef2c7c92) Co-authored-by: Dave Goncalves <davegoncalves(a)gmail.com> files: M Doc/howto/functional.rst M Doc/reference/datamodel.rst M Doc/reference/expressions.rst M Objects/genobject.c diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst index c7f8bc8f17f43..695b9b31a762b 100644 --- a/Doc/howto/functional.rst +++ b/Doc/howto/functional.rst @@ -589,7 +589,7 @@ generator function. In addition to :meth:`~generator.send`, there are two other methods on generators: -* :meth:`throw(type, value=None, traceback=None) <generator.throw>` is used to +* :meth:`throw(value) <generator.throw>` is used to raise an exception inside the generator; the exception is raised by the ``yield`` expression where the generator's execution is paused. diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 3c32210a6ac7b..a5739e6e5e472 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2913,7 +2913,8 @@ generators, coroutines do not directly support iteration. :exc:`StopIteration`, or other exception) is the same as when iterating over the :meth:`__await__` return value, described above. -.. method:: coroutine.throw(type[, value[, traceback]]) +.. method:: coroutine.throw(value) + coroutine.throw(type[, value[, traceback]]) Raises the specified exception in the coroutine. This method delegates to the :meth:`~generator.throw` method of the iterator that caused diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 9f136c9a88ff3..f77927d86d9d6 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -556,14 +556,27 @@ is already executing raises a :exc:`ValueError` exception. could receive the value. -.. method:: generator.throw(type[, value[, traceback]]) +.. method:: generator.throw(value) + generator.throw(type[, value[, traceback]]) - Raises an exception of type ``type`` at the point where the generator was paused, + Raises an exception at the point where the generator was paused, and returns the next value yielded by the generator function. If the generator exits without yielding another value, a :exc:`StopIteration` exception is raised. If the generator function does not catch the passed-in exception, or raises a different exception, then that exception propagates to the caller. + In typical use, this is called with a single exception instance similar to the + way the :keyword:`raise` keyword is used. + + For backwards compatability, however, the second signature is + supported, following a convention from older versions of Python. + The *type* argument should be an exception class, and *value* + should be an exception instance. If the *value* is not provided, the + *type* constructor is called to get an instance. If *traceback* + is provided, it is set on the exception, otherwise any existing + :attr:`~BaseException.__traceback__` attribute stored in *value* may + be cleared. + .. index:: exception: GeneratorExit diff --git a/Objects/genobject.c b/Objects/genobject.c index 33fc4a592492a..123c17aae7e7c 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -403,8 +403,11 @@ gen_close(PyGenObject *gen, PyObject *args) PyDoc_STRVAR(throw_doc, -"throw(typ[,val[,tb]]) -> raise exception in generator,\n\ -return next yielded value or raise StopIteration."); +"throw(value)\n\ +throw(type[,value[,tb]])\n\ +\n\ +Raise exception in generator, return next yielded value or raise\n\ +StopIteration."); static PyObject * _gen_throw(PyGenObject *gen, int close_on_genexit, @@ -1001,8 +1004,11 @@ PyDoc_STRVAR(coro_send_doc, return next iterated value or raise StopIteration."); PyDoc_STRVAR(coro_throw_doc, -"throw(typ[,val[,tb]]) -> raise exception in coroutine,\n\ -return next iterated value or raise StopIteration."); +"throw(value)\n\ +throw(type[,value[,traceback]])\n\ +\n\ +Raise exception in coroutine, return next iterated value or raise\n\ +StopIteration."); PyDoc_STRVAR(coro_close_doc, "close() -> raise GeneratorExit inside coroutine.");

1 0

Remove unnecessary registration of weakref.WeakSet to _collections_abc.Set (GH-32211)
by rhettinger 31 Mar '22

31 Mar '22

https://github.com/python/cpython/commit/5458b7e39eb41b146c650b76e04ac67213… commit: 5458b7e39eb41b146c650b76e04ac67213138a82 branch: main author: Géry Ogam <gery.ogam(a)gmail.com> committer: rhettinger <rhettinger(a)users.noreply.github.com> date: 2022-03-31T09:11:35-05:00 summary: Remove unnecessary registration of weakref.WeakSet to _collections_abc.Set (GH-32211) files: M Lib/weakref.py diff --git a/Lib/weakref.py b/Lib/weakref.py index 42aba654de549..25b70927e29c3 100644 --- a/Lib/weakref.py +++ b/Lib/weakref.py @@ -33,7 +33,6 @@ "WeakSet", "WeakMethod", "finalize"] -_collections_abc.Set.register(WeakSet) _collections_abc.MutableSet.register(WeakSet) class WeakMethod(ref):

1 0

bpo-14911: Corrected generator.throw() documentation (GH-32207)
by asvetlov 31 Mar '22

31 Mar '22

https://github.com/python/cpython/commit/8be7c2bc5ad5e295f0f855bb31db412eef… commit: 8be7c2bc5ad5e295f0f855bb31db412eef2c7c92 branch: main author: Dave Goncalves <davegoncalves(a)gmail.com> committer: asvetlov <andrew.svetlov(a)gmail.com> date: 2022-03-31T16:56:48+03:00 summary: bpo-14911: Corrected generator.throw() documentation (GH-32207) Co-authored-by: Andrew Svetlov <andrew.svetlov(a)gmail.com> files: M Doc/howto/functional.rst M Doc/reference/datamodel.rst M Doc/reference/expressions.rst M Objects/genobject.c diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst index c7f8bc8f17f43..695b9b31a762b 100644 --- a/Doc/howto/functional.rst +++ b/Doc/howto/functional.rst @@ -589,7 +589,7 @@ generator function. In addition to :meth:`~generator.send`, there are two other methods on generators: -* :meth:`throw(type, value=None, traceback=None) <generator.throw>` is used to +* :meth:`throw(value) <generator.throw>` is used to raise an exception inside the generator; the exception is raised by the ``yield`` expression where the generator's execution is paused. diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 804332ffab6fd..8ac9a8c0566bf 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2984,7 +2984,8 @@ generators, coroutines do not directly support iteration. :exc:`StopIteration`, or other exception) is the same as when iterating over the :meth:`__await__` return value, described above. -.. method:: coroutine.throw(type[, value[, traceback]]) +.. method:: coroutine.throw(value) + coroutine.throw(type[, value[, traceback]]) Raises the specified exception in the coroutine. This method delegates to the :meth:`~generator.throw` method of the iterator that caused diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index bb6d1dc1cdd04..b914c48d3d4cd 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -561,14 +561,27 @@ is already executing raises a :exc:`ValueError` exception. could receive the value. -.. method:: generator.throw(type[, value[, traceback]]) +.. method:: generator.throw(value) + generator.throw(type[, value[, traceback]]) - Raises an exception of type ``type`` at the point where the generator was paused, + Raises an exception at the point where the generator was paused, and returns the next value yielded by the generator function. If the generator exits without yielding another value, a :exc:`StopIteration` exception is raised. If the generator function does not catch the passed-in exception, or raises a different exception, then that exception propagates to the caller. + In typical use, this is called with a single exception instance similar to the + way the :keyword:`raise` keyword is used. + + For backwards compatability, however, the second signature is + supported, following a convention from older versions of Python. + The *type* argument should be an exception class, and *value* + should be an exception instance. If the *value* is not provided, the + *type* constructor is called to get an instance. If *traceback* + is provided, it is set on the exception, otherwise any existing + :attr:`~BaseException.__traceback__` attribute stored in *value* may + be cleared. + .. index:: exception: GeneratorExit diff --git a/Objects/genobject.c b/Objects/genobject.c index f071390d6d32b..cdb2a0f76b085 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -410,8 +410,11 @@ gen_close(PyGenObject *gen, PyObject *args) PyDoc_STRVAR(throw_doc, -"throw(typ[,val[,tb]]) -> raise exception in generator,\n\ -return next yielded value or raise StopIteration."); +"throw(value)\n\ +throw(type[,value[,tb]])\n\ +\n\ +Raise exception in generator, return next yielded value or raise\n\ +StopIteration."); static PyObject * _gen_throw(PyGenObject *gen, int close_on_genexit, @@ -1157,8 +1160,11 @@ PyDoc_STRVAR(coro_send_doc, return next iterated value or raise StopIteration."); PyDoc_STRVAR(coro_throw_doc, -"throw(typ[,val[,tb]]) -> raise exception in coroutine,\n\ -return next iterated value or raise StopIteration."); +"throw(value)\n\ +throw(type[,value[,traceback]])\n\ +\n\ +Raise exception in coroutine, return next iterated value or raise\n\ +StopIteration."); PyDoc_STRVAR(coro_close_doc, "close() -> raise GeneratorExit inside coroutine.");

1 0