Mailman 3 August 2022 - Python-checkins

gh-96021: Explicitly close the IsolatedAsyncioTestCase runner in tests (GH-96135)
by miss-islington Aug. 25, 2022

Aug. 25, 2022

https://github.com/python/cpython/commit/f51216df073310c783f1890993fe9b67cc… commit: f51216df073310c783f1890993fe9b67ccb3c6b4 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-08-24T21:53:39-07:00 summary: gh-96021: Explicitly close the IsolatedAsyncioTestCase runner in tests (GH-96135) Tests for IsolatedAsyncioTestCase.debug() rely on the runner be closed in __del__. It makes tests depending on the GC an unreliable on other implementations. It is better to close the runner explicitly even if currently there is no a public API for this. (cherry picked from commit 4de06e3cc0a58d73934f9a2759ad9cd2f6b031b0) Co-authored-by: Serhiy Storchaka <storchaka(a)gmail.com> files: M Lib/unittest/test/test_async_case.py diff --git a/Lib/unittest/test/test_async_case.py b/Lib/unittest/test/test_async_case.py index f59fc760d38..d7d4dc91316 100644 --- a/Lib/unittest/test/test_async_case.py +++ b/Lib/unittest/test/test_async_case.py @@ -43,10 +43,10 @@ async def __aenter__(self): class TestAsyncCase(unittest.TestCase): maxDiff = None - def tearDown(self): + def setUp(self): # Ensure that IsolatedAsyncioTestCase instances are destroyed before # starting a new event loop - support.gc_collect() + self.addCleanup(support.gc_collect) def test_full_cycle(self): class Test(unittest.IsolatedAsyncioTestCase): @@ -151,6 +151,7 @@ async def on_cleanup(self): events = [] test = Test("test_func") + self.addCleanup(test._tearDownAsyncioRunner) try: test.debug() except MyException: @@ -186,6 +187,7 @@ async def on_cleanup(self): events = [] test = Test("test_func") + self.addCleanup(test._tearDownAsyncioRunner) try: test.debug() except MyException: @@ -221,6 +223,7 @@ async def on_cleanup(self): events = [] test = Test("test_func") + self.addCleanup(test._tearDownAsyncioRunner) try: test.debug() except MyException: @@ -262,6 +265,7 @@ async def on_cleanup2(self): events = [] test = Test("test_func") + self.addCleanup(test._tearDownAsyncioRunner) try: test.debug() except MyException: @@ -424,6 +428,7 @@ async def cleanup(self, fut): events = [] test = Test("test_func") + self.addCleanup(test._tearDownAsyncioRunner) try: test.debug() except MyException:

1 0

gh-95243: Mitigate the race condition in testSockName (GH-96173)
by miss-islington Aug. 25, 2022

Aug. 25, 2022

https://github.com/python/cpython/commit/915ee9ed43b199de6f9fb371c2224ef459… commit: 915ee9ed43b199de6f9fb371c2224ef4593522ab branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-08-24T19:03:29-07:00 summary: gh-95243: Mitigate the race condition in testSockName (GH-96173) find_unused_port() has an inherent race condition, but we can't use bind_port() as that uses .getsockname() which this test is exercising. Try binding to unused ports a few times before failing. Signed-off-by: Ross Burton <ross.burton(a)arm.com> (cherry picked from commit df110126971d0271a977ce10779083b3e335b4da) Co-authored-by: Ross Burton <ross.burton(a)arm.com> files: A Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index 72e3394df2d..b07954989fd 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -1389,10 +1389,21 @@ def testStringToIPv6(self): def testSockName(self): # Testing getsockname() - port = socket_helper.find_unused_port() sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.addCleanup(sock.close) - sock.bind(("0.0.0.0", port)) + + # Since find_unused_port() is inherently subject to race conditions, we + # call it a couple times if necessary. + for i in itertools.count(): + port = socket_helper.find_unused_port() + try: + sock.bind(("0.0.0.0", port)) + except OSError as e: + if e.errno != errno.EADDRINUSE or i == 5: + raise + else: + break + name = sock.getsockname() # XXX(nnorwitz): http://tinyurl.com/os5jz seems to indicate # it reasonable to get the host's addr in addition to 0.0.0.0. diff --git a/Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst b/Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst new file mode 100644 index 00000000000..a9ca1f8b867 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst @@ -0,0 +1,3 @@ +Mitigate the inherent race condition from using find_unused_port() in +testSockName() by trying to find an unused port a few times before failing. +Patch by Ross Burton.

1 0

gh-95243: Mitigate the race condition in testSockName (GH-96173)
by miss-islington Aug. 25, 2022

Aug. 25, 2022

https://github.com/python/cpython/commit/979a3b8cd3861c2a40b560f891e5828500… commit: 979a3b8cd3861c2a40b560f891e5828500f35416 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-08-24T19:01:50-07:00 summary: gh-95243: Mitigate the race condition in testSockName (GH-96173) find_unused_port() has an inherent race condition, but we can't use bind_port() as that uses .getsockname() which this test is exercising. Try binding to unused ports a few times before failing. Signed-off-by: Ross Burton <ross.burton(a)arm.com> (cherry picked from commit df110126971d0271a977ce10779083b3e335b4da) Co-authored-by: Ross Burton <ross.burton(a)arm.com> files: A Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index 9c5f6d3dc9a..e0e0f2437bb 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -1374,10 +1374,21 @@ def testStringToIPv6(self): def testSockName(self): # Testing getsockname() - port = socket_helper.find_unused_port() sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.addCleanup(sock.close) - sock.bind(("0.0.0.0", port)) + + # Since find_unused_port() is inherently subject to race conditions, we + # call it a couple times if necessary. + for i in itertools.count(): + port = socket_helper.find_unused_port() + try: + sock.bind(("0.0.0.0", port)) + except OSError as e: + if e.errno != errno.EADDRINUSE or i == 5: + raise + else: + break + name = sock.getsockname() # XXX(nnorwitz): http://tinyurl.com/os5jz seems to indicate # it reasonable to get the host's addr in addition to 0.0.0.0. diff --git a/Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst b/Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst new file mode 100644 index 00000000000..a9ca1f8b867 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst @@ -0,0 +1,3 @@ +Mitigate the inherent race condition from using find_unused_port() in +testSockName() by trying to find an unused port a few times before failing. +Patch by Ross Burton.

1 0

gh-95243: Mitigate the race condition in testSockName (#96173)
by gpshead Aug. 25, 2022

Aug. 25, 2022

https://github.com/python/cpython/commit/df110126971d0271a977ce10779083b3e3… commit: df110126971d0271a977ce10779083b3e335b4da branch: main author: Ross Burton <ross.burton(a)arm.com> committer: gpshead <greg(a)krypto.org> date: 2022-08-24T18:37:18-07:00 summary: gh-95243: Mitigate the race condition in testSockName (#96173) find_unused_port() has an inherent race condition, but we can't use bind_port() as that uses .getsockname() which this test is exercising. Try binding to unused ports a few times before failing. Signed-off-by: Ross Burton <ross.burton(a)arm.com> files: A Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst M Lib/test/test_socket.py diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index f1b57fc4071..d808f3f62b9 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -1398,10 +1398,21 @@ def testStringToIPv6(self): def testSockName(self): # Testing getsockname() - port = socket_helper.find_unused_port() sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.addCleanup(sock.close) - sock.bind(("0.0.0.0", port)) + + # Since find_unused_port() is inherently subject to race conditions, we + # call it a couple times if necessary. + for i in itertools.count(): + port = socket_helper.find_unused_port() + try: + sock.bind(("0.0.0.0", port)) + except OSError as e: + if e.errno != errno.EADDRINUSE or i == 5: + raise + else: + break + name = sock.getsockname() # XXX(nnorwitz): http://tinyurl.com/os5jz seems to indicate # it reasonable to get the host's addr in addition to 0.0.0.0. diff --git a/Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst b/Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst new file mode 100644 index 00000000000..a9ca1f8b867 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2022-08-22-14-59-42.gh-issue-95243.DeD66V.rst @@ -0,0 +1,3 @@ +Mitigate the inherent race condition from using find_unused_port() in +testSockName() by trying to find an unused port a few times before failing. +Patch by Ross Burton.

1 0

GH-93503: Add thread-specific APIs to set profiling and tracing functions in the C-API (#93504)
by pablogsal Aug. 24, 2022

Aug. 24, 2022

https://github.com/python/cpython/commit/e34c82abeb7ace09e6b5d116585c47cc37… commit: e34c82abeb7ace09e6b5d116585c47cc372996c1 branch: main author: Pablo Galindo Salgado <Pablogsal(a)gmail.com> committer: pablogsal <Pablogsal(a)gmail.com> date: 2022-08-24T23:21:39+01:00 summary: GH-93503: Add thread-specific APIs to set profiling and tracing functions in the C-API (#93504) * gh-93503: Add APIs to set profiling and tracing functions in all threads in the C-API * Use a separate API * Fix NEWS entry * Add locks around the loop * Document ignoring exceptions * Use the new APIs in the sys module * Update docs files: A Misc/NEWS.d/next/C API/2022-06-06-16-04-14.gh-issue-93503.MHJTu8.rst M Doc/c-api/init.rst M Doc/data/refcounts.dat M Doc/library/threading.rst M Include/cpython/ceval.h M Lib/test/test_threading.py M Lib/threading.py M Python/ceval.c M Python/clinic/sysmodule.c.h M Python/sysmodule.c diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 038498f325c..2a9cf0ea702 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1774,6 +1774,18 @@ Python-level trace functions in previous versions. The caller must hold the :term:`GIL`. +.. c:function:: void PyEval_SetProfileAllThreads(Py_tracefunc func, PyObject *obj) + + Like :c:func:`PyEval_SetProfile` but sets the profile function in all running threads + belonging to the current interpreter instead of the setting it only on the current thread. + + The caller must hold the :term:`GIL`. + + As :c:func:`PyEval_SetProfile`, this function ignores any exceptions raised while + setting the profile functions in all threads. + +.. versionadded:: 3.12 + .. c:function:: void PyEval_SetTrace(Py_tracefunc func, PyObject *obj) @@ -1788,6 +1800,18 @@ Python-level trace functions in previous versions. The caller must hold the :term:`GIL`. +.. c:function:: void PyEval_SetTraceAllThreads(Py_tracefunc func, PyObject *obj) + + Like :c:func:`PyEval_SetTrace` but sets the tracing function in all running threads + belonging to the current interpreter instead of the setting it only on the current thread. + + The caller must hold the :term:`GIL`. + + As :c:func:`PyEval_SetTrace`, this function ignores any exceptions raised while + setting the trace functions in all threads. + +.. versionadded:: 3.12 + .. _advanced-debugging: diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat index 1694cad6f43..51ccacf13f9 100644 --- a/Doc/data/refcounts.dat +++ b/Doc/data/refcounts.dat @@ -796,10 +796,18 @@ PyEval_SetProfile:void::: PyEval_SetProfile:Py_tracefunc:func:: PyEval_SetProfile:PyObject*:obj:+1: +PyEval_SetProfileAllThreads:void::: +PyEval_SetProfileAllThreads:Py_tracefunc:func:: +PyEval_SetProfileAllThreads:PyObject*:obj:+1: + PyEval_SetTrace:void::: PyEval_SetTrace:Py_tracefunc:func:: PyEval_SetTrace:PyObject*:obj:+1: +PyEval_SetTraceAllThreads:void::: +PyEval_SetTraceAllThreads:Py_tracefunc:func:: +PyEval_SetTraceAllThreads:PyObject*:obj:+1: + PyEval_EvalCode:PyObject*::+1: PyEval_EvalCode:PyObject*:co:0: PyEval_EvalCode:PyObject*:globals:0: diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index b22d4876622..b352125551f 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -158,6 +158,15 @@ This module defines the following functions: The *func* will be passed to :func:`sys.settrace` for each thread, before its :meth:`~Thread.run` method is called. +.. function:: settrace_all_threads(func) + + Set a trace function for all threads started from the :mod:`threading` module + and all Python threads that are currently executing. + + The *func* will be passed to :func:`sys.settrace` for each thread, before its + :meth:`~Thread.run` method is called. + + .. versionadded:: 3.12 .. function:: gettrace() @@ -178,6 +187,15 @@ This module defines the following functions: The *func* will be passed to :func:`sys.setprofile` for each thread, before its :meth:`~Thread.run` method is called. +.. function:: setprofile_all_threads(func) + + Set a profile function for all threads started from the :mod:`threading` module + and all Python threads that are currently executing. + + The *func* will be passed to :func:`sys.setprofile` for each thread, before its + :meth:`~Thread.run` method is called. + + .. versionadded:: 3.12 .. function:: getprofile() diff --git a/Include/cpython/ceval.h b/Include/cpython/ceval.h index 9d4eeafb427..74665c9fa10 100644 --- a/Include/cpython/ceval.h +++ b/Include/cpython/ceval.h @@ -3,8 +3,10 @@ #endif PyAPI_FUNC(void) PyEval_SetProfile(Py_tracefunc, PyObject *); +PyAPI_FUNC(void) PyEval_SetProfileAllThreads(Py_tracefunc, PyObject *); PyAPI_DATA(int) _PyEval_SetProfile(PyThreadState *tstate, Py_tracefunc func, PyObject *arg); PyAPI_FUNC(void) PyEval_SetTrace(Py_tracefunc, PyObject *); +PyAPI_FUNC(void) PyEval_SetTraceAllThreads(Py_tracefunc, PyObject *); PyAPI_FUNC(int) _PyEval_SetTrace(PyThreadState *tstate, Py_tracefunc func, PyObject *arg); /* Helper to look up a builtin object */ diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index dcd27697bb4..c6649962331 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -853,6 +853,7 @@ def callback(): callback() finally: sys.settrace(old_trace) + threading.settrace(old_trace) def test_gettrace(self): def noop_trace(frame, event, arg): @@ -866,6 +867,35 @@ def noop_trace(frame, event, arg): finally: threading.settrace(old_trace) + def test_gettrace_all_threads(self): + def fn(*args): pass + old_trace = threading.gettrace() + first_check = threading.Event() + second_check = threading.Event() + + trace_funcs = [] + def checker(): + trace_funcs.append(sys.gettrace()) + first_check.set() + second_check.wait() + trace_funcs.append(sys.gettrace()) + + try: + t = threading.Thread(target=checker) + t.start() + first_check.wait() + threading.settrace_all_threads(fn) + second_check.set() + t.join() + self.assertEqual(trace_funcs, [None, fn]) + self.assertEqual(threading.gettrace(), fn) + self.assertEqual(sys.gettrace(), fn) + finally: + threading.settrace_all_threads(old_trace) + + self.assertEqual(threading.gettrace(), old_trace) + self.assertEqual(sys.gettrace(), old_trace) + def test_getprofile(self): def fn(*args): pass old_profile = threading.getprofile() @@ -875,6 +905,35 @@ def fn(*args): pass finally: threading.setprofile(old_profile) + def test_getprofile_all_threads(self): + def fn(*args): pass + old_profile = threading.getprofile() + first_check = threading.Event() + second_check = threading.Event() + + profile_funcs = [] + def checker(): + profile_funcs.append(sys.getprofile()) + first_check.set() + second_check.wait() + profile_funcs.append(sys.getprofile()) + + try: + t = threading.Thread(target=checker) + t.start() + first_check.wait() + threading.setprofile_all_threads(fn) + second_check.set() + t.join() + self.assertEqual(profile_funcs, [None, fn]) + self.assertEqual(threading.getprofile(), fn) + self.assertEqual(sys.getprofile(), fn) + finally: + threading.setprofile_all_threads(old_profile) + + self.assertEqual(threading.getprofile(), old_profile) + self.assertEqual(sys.getprofile(), old_profile) + @cpython_only def test_shutdown_locks(self): for daemon in (False, True): diff --git a/Lib/threading.py b/Lib/threading.py index e32ad1418d9..f28597c9930 100644 --- a/Lib/threading.py +++ b/Lib/threading.py @@ -28,7 +28,8 @@ 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread', 'Barrier', 'BrokenBarrierError', 'Timer', 'ThreadError', 'setprofile', 'settrace', 'local', 'stack_size', - 'excepthook', 'ExceptHookArgs', 'gettrace', 'getprofile'] + 'excepthook', 'ExceptHookArgs', 'gettrace', 'getprofile', + 'setprofile_all_threads','settrace_all_threads'] # Rename some stuff so "from threading import *" is safe _start_new_thread = _thread.start_new_thread @@ -60,11 +61,20 @@ def setprofile(func): The func will be passed to sys.setprofile() for each thread, before its run() method is called. - """ global _profile_hook _profile_hook = func +def setprofile_all_threads(func): + """Set a profile function for all threads started from the threading module + and all Python threads that are currently executing. + + The func will be passed to sys.setprofile() for each thread, before its + run() method is called. + """ + setprofile(func) + _sys._setprofileallthreads(func) + def getprofile(): """Get the profiler function as set by threading.setprofile().""" return _profile_hook @@ -74,11 +84,20 @@ def settrace(func): The func will be passed to sys.settrace() for each thread, before its run() method is called. - """ global _trace_hook _trace_hook = func +def settrace_all_threads(func): + """Set a trace function for all threads started from the threading module + and all Python threads that are currently executing. + + The func will be passed to sys.settrace() for each thread, before its run() + method is called. + """ + settrace(func) + _sys._settraceallthreads(func) + def gettrace(): """Get the trace function as set by threading.settrace().""" return _trace_hook diff --git a/Misc/NEWS.d/next/C API/2022-06-06-16-04-14.gh-issue-93503.MHJTu8.rst b/Misc/NEWS.d/next/C API/2022-06-06-16-04-14.gh-issue-93503.MHJTu8.rst new file mode 100644 index 00000000000..6df9f95fc94 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2022-06-06-16-04-14.gh-issue-93503.MHJTu8.rst @@ -0,0 +1,7 @@ +Add two new public functions to the public C-API, +:c:func:`PyEval_SetProfileAllThreads` and +:c:func:`PyEval_SetTraceAllThreads`, that allow to set tracking and +profiling functions in all running threads in addition to the calling one. +Also, add a new *running_threads* parameter to :func:`threading.setprofile` +and :func:`threading.settrace` that allows to do the same from Python. Patch +by Pablo Galindo diff --git a/Python/ceval.c b/Python/ceval.c index 1ab104c18ed..ac77ab8e869 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -96,6 +96,10 @@ #define _Py_atomic_load_relaxed_int32(ATOMIC_VAL) _Py_atomic_load_relaxed(ATOMIC_VAL) #endif +#define HEAD_LOCK(runtime) \ + PyThread_acquire_lock((runtime)->interpreters.mutex, WAIT_LOCK) +#define HEAD_UNLOCK(runtime) \ + PyThread_release_lock((runtime)->interpreters.mutex) /* Forward declarations */ static PyObject *trace_call_function( @@ -6455,6 +6459,27 @@ PyEval_SetProfile(Py_tracefunc func, PyObject *arg) } } +void +PyEval_SetProfileAllThreads(Py_tracefunc func, PyObject *arg) +{ + PyThreadState *this_tstate = _PyThreadState_GET(); + PyInterpreterState* interp = this_tstate->interp; + + _PyRuntimeState *runtime = &_PyRuntime; + HEAD_LOCK(runtime); + PyThreadState* ts = PyInterpreterState_ThreadHead(interp); + HEAD_UNLOCK(runtime); + + while (ts) { + if (_PyEval_SetProfile(ts, func, arg) < 0) { + _PyErr_WriteUnraisableMsg("in PyEval_SetProfileAllThreads", NULL); + } + HEAD_LOCK(runtime); + ts = PyThreadState_Next(ts); + HEAD_UNLOCK(runtime); + } +} + int _PyEval_SetTrace(PyThreadState *tstate, Py_tracefunc func, PyObject *arg) { @@ -6508,6 +6533,26 @@ PyEval_SetTrace(Py_tracefunc func, PyObject *arg) } } +void +PyEval_SetTraceAllThreads(Py_tracefunc func, PyObject *arg) +{ + PyThreadState *this_tstate = _PyThreadState_GET(); + PyInterpreterState* interp = this_tstate->interp; + + _PyRuntimeState *runtime = &_PyRuntime; + HEAD_LOCK(runtime); + PyThreadState* ts = PyInterpreterState_ThreadHead(interp); + HEAD_UNLOCK(runtime); + + while (ts) { + if (_PyEval_SetTrace(ts, func, arg) < 0) { + _PyErr_WriteUnraisableMsg("in PyEval_SetTraceAllThreads", NULL); + } + HEAD_LOCK(runtime); + ts = PyThreadState_Next(ts); + HEAD_UNLOCK(runtime); + } +} int _PyEval_SetCoroutineOriginTrackingDepth(int depth) diff --git a/Python/clinic/sysmodule.c.h b/Python/clinic/sysmodule.c.h index beaf21c85bc..0f9636690a4 100644 --- a/Python/clinic/sysmodule.c.h +++ b/Python/clinic/sysmodule.c.h @@ -292,6 +292,18 @@ sys_intern(PyObject *module, PyObject *arg) return return_value; } +PyDoc_STRVAR(sys__settraceallthreads__doc__, +"_settraceallthreads($module, arg, /)\n" +"--\n" +"\n" +"Set the global debug tracing function in all running threads belonging to the current interpreter.\n" +"\n" +"It will be called on each function call. See the debugger chapter\n" +"in the library manual."); + +#define SYS__SETTRACEALLTHREADS_METHODDEF \ + {"_settraceallthreads", (PyCFunction)sys__settraceallthreads, METH_O, sys__settraceallthreads__doc__}, + PyDoc_STRVAR(sys_gettrace__doc__, "gettrace($module, /)\n" "--\n" @@ -312,6 +324,18 @@ sys_gettrace(PyObject *module, PyObject *Py_UNUSED(ignored)) return sys_gettrace_impl(module); } +PyDoc_STRVAR(sys__setprofileallthreads__doc__, +"_setprofileallthreads($module, arg, /)\n" +"--\n" +"\n" +"Set the profiling function in all running threads belonging to the current interpreter.\n" +"\n" +"It will be called on each function call and return. See the profiler chapter\n" +"in the library manual."); + +#define SYS__SETPROFILEALLTHREADS_METHODDEF \ + {"_setprofileallthreads", (PyCFunction)sys__setprofileallthreads, METH_O, sys__setprofileallthreads__doc__}, + PyDoc_STRVAR(sys_getprofile__doc__, "getprofile($module, /)\n" "--\n" @@ -1170,4 +1194,4 @@ sys_getandroidapilevel(PyObject *module, PyObject *Py_UNUSED(ignored)) #ifndef SYS_GETANDROIDAPILEVEL_METHODDEF #define SYS_GETANDROIDAPILEVEL_METHODDEF #endif /* !defined(SYS_GETANDROIDAPILEVEL_METHODDEF) */ -/*[clinic end generated code: output=38446a4c76e2f3b6 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=322fb0409e376ad4 input=a9049054013a1b77]*/ diff --git a/Python/sysmodule.c b/Python/sysmodule.c index b8009b2db45..c2864387941 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -1021,6 +1021,36 @@ Set the global debug tracing function. It will be called on each\n\ function call. See the debugger chapter in the library manual." ); +/*[clinic input] +sys._settraceallthreads + + arg: object + / + +Set the global debug tracing function in all running threads belonging to the current interpreter. + +It will be called on each function call. See the debugger chapter +in the library manual. +[clinic start generated code]*/ + +static PyObject * +sys__settraceallthreads(PyObject *module, PyObject *arg) +/*[clinic end generated code: output=161cca30207bf3ca input=5906aa1485a50289]*/ +{ + PyObject* argument = NULL; + Py_tracefunc func = NULL; + + if (arg != Py_None) { + func = trace_trampoline; + argument = arg; + } + + + PyEval_SetTraceAllThreads(func, argument); + + Py_RETURN_NONE; +} + /*[clinic input] sys.gettrace @@ -1066,6 +1096,35 @@ Set the profiling function. It will be called on each function call\n\ and return. See the profiler chapter in the library manual." ); +/*[clinic input] +sys._setprofileallthreads + + arg: object + / + +Set the profiling function in all running threads belonging to the current interpreter. + +It will be called on each function call and return. See the profiler chapter +in the library manual. +[clinic start generated code]*/ + +static PyObject * +sys__setprofileallthreads(PyObject *module, PyObject *arg) +/*[clinic end generated code: output=2d61319e27b309fe input=d1a356d3f4f9060a]*/ +{ + PyObject* argument = NULL; + Py_tracefunc func = NULL; + + if (arg != Py_None) { + func = profile_trampoline; + argument = arg; + } + + PyEval_SetProfileAllThreads(func, argument); + + Py_RETURN_NONE; +} + /*[clinic input] sys.getprofile @@ -2035,9 +2094,11 @@ static PyMethodDef sys_methods[] = { SYS_GETSWITCHINTERVAL_METHODDEF SYS_SETDLOPENFLAGS_METHODDEF {"setprofile", sys_setprofile, METH_O, setprofile_doc}, + SYS__SETPROFILEALLTHREADS_METHODDEF SYS_GETPROFILE_METHODDEF SYS_SETRECURSIONLIMIT_METHODDEF {"settrace", sys_settrace, METH_O, settrace_doc}, + SYS__SETTRACEALLTHREADS_METHODDEF SYS_GETTRACE_METHODDEF SYS_CALL_TRACING_METHODDEF SYS__DEBUGMALLOCSTATS_METHODDEF

1 0

Add gvanrossum to asyncio experts (#96248)
by gvanrossum Aug. 24, 2022

Aug. 24, 2022

https://github.com/python/cpython/commit/657976ad950e56b33b7dc15e64a0baecdd… commit: 657976ad950e56b33b7dc15e64a0baecdd184f5a branch: main author: Guido van Rossum <guido(a)python.org> committer: gvanrossum <gvanrossum(a)gmail.com> date: 2022-08-24T13:16:41-07:00 summary: Add gvanrossum to asyncio experts (#96248) files: M .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 0e38536ba99..2568560c074 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -5,7 +5,7 @@ # https://git-scm.com/docs/gitignore#_pattern_format # asyncio -**/*asyncio* @1st1 @asvetlov +**/*asyncio* @1st1 @asvetlov @gvanrossum # Core **/*context* @1st1

1 0

gh-87092: use basicblock_last_instr consistently in the compiler (GH-96243)
by iritkatriel Aug. 24, 2022

Aug. 24, 2022

https://github.com/python/cpython/commit/fba3b67af47723c201fe3425cb61593297… commit: fba3b67af47723c201fe3425cb615932970a8f28 branch: main author: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2022-08-24T16:58:42+01:00 summary: gh-87092: use basicblock_last_instr consistently in the compiler (GH-96243) files: M Python/compile.c diff --git a/Python/compile.c b/Python/compile.c index e5ac162ccc0..627f86a8ce9 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -291,7 +291,9 @@ typedef struct basicblock_ { static struct instr * basicblock_last_instr(const basicblock *b) { - if (b->b_iused) { + assert(b->b_iused >= 0); + if (b->b_iused > 0) { + assert(b->b_instr != NULL); return &b->b_instr[b->b_iused - 1]; } return NULL; @@ -7168,10 +7170,8 @@ assemble_free(struct assembler *a) static int blocksize(basicblock *b) { - int i; int size = 0; - - for (i = 0; i < b->b_iused; i++) { + for (int i = 0; i < b->b_iused; i++) { size += instr_size(&b->b_instr[i]); } return size; @@ -7746,10 +7746,10 @@ normalize_jumps(basicblock *entryblock) } for (basicblock *b = entryblock; b != NULL; b = b->b_next) { b->b_visited = 1; - if (b->b_iused == 0) { + struct instr *last = basicblock_last_instr(b); + if (last == NULL) { continue; } - struct instr *last = &b->b_instr[b->b_iused-1]; assert(!IS_ASSEMBLER_OPCODE(last->i_opcode)); if (is_jump(last)) { bool is_forward = last->i_target->b_visited == 0; @@ -7915,8 +7915,8 @@ scan_block_for_local(int target, basicblock *b, bool unsafe_to_start, if (b->b_next && BB_HAS_FALLTHROUGH(b)) { MAYBE_PUSH(b->b_next); } - if (b->b_iused > 0) { - struct instr *last = &b->b_instr[b->b_iused-1]; + struct instr *last = basicblock_last_instr(b); + if (last != NULL) { if (is_jump(last)) { assert(last->i_target != NULL); MAYBE_PUSH(last->i_target); @@ -8405,10 +8405,10 @@ guarantee_lineno_for_exits(basicblock *entryblock, int firstlineno) { int lineno = firstlineno; assert(lineno > 0); for (basicblock *b = entryblock; b != NULL; b = b->b_next) { - if (b->b_iused == 0) { + struct instr *last = basicblock_last_instr(b); + if (last == NULL) { continue; } - struct instr *last = &b->b_instr[b->b_iused-1]; if (last->i_loc.lineno < 0) { if (last->i_opcode == RETURN_VALUE) { for (int i = 0; i < b->b_iused; i++) { @@ -8486,18 +8486,18 @@ remove_redundant_jumps(cfg_builder *g) { */ int removed = 0; for (basicblock *b = g->g_entryblock; b != NULL; b = b->b_next) { - if (b->b_iused > 0) { - struct instr *b_last_instr = &b->b_instr[b->b_iused - 1]; - assert(!IS_ASSEMBLER_OPCODE(b_last_instr->i_opcode)); - if (b_last_instr->i_opcode == JUMP || - b_last_instr->i_opcode == JUMP_NO_INTERRUPT) { - if (b_last_instr->i_target == NULL) { + struct instr *last = basicblock_last_instr(b); + if (last != NULL) { + assert(!IS_ASSEMBLER_OPCODE(last->i_opcode)); + if (last->i_opcode == JUMP || + last->i_opcode == JUMP_NO_INTERRUPT) { + if (last->i_target == NULL) { PyErr_SetString(PyExc_SystemError, "jump with NULL target"); return -1; } - if (b_last_instr->i_target == b->b_next) { + if (last->i_target == b->b_next) { assert(b->b_next->b_iused); - b_last_instr->i_opcode = NOP; + last->i_opcode = NOP; removed++; } } @@ -9215,10 +9215,10 @@ basicblock_has_lineno(const basicblock *bb) { */ static int extend_block(basicblock *bb) { - if (bb->b_iused == 0) { + struct instr *last = basicblock_last_instr(bb); + if (last == NULL) { return 0; } - struct instr *last = &bb->b_instr[bb->b_iused-1]; if (last->i_opcode != JUMP && last->i_opcode != JUMP_FORWARD && last->i_opcode != JUMP_BACKWARD) { @@ -9399,7 +9399,8 @@ eliminate_empty_basic_blocks(cfg_builder *g) { static void propagate_line_numbers(basicblock *entryblock) { for (basicblock *b = entryblock; b != NULL; b = b->b_next) { - if (b->b_iused == 0) { + struct instr *last = basicblock_last_instr(b); + if (last == NULL) { continue; } @@ -9418,8 +9419,8 @@ propagate_line_numbers(basicblock *entryblock) { b->b_next->b_instr[0].i_loc = prev_location; } } - if (is_jump(&b->b_instr[b->b_iused-1])) { - basicblock *target = b->b_instr[b->b_iused-1].i_target; + if (is_jump(last)) { + basicblock *target = last->i_target; if (target->b_predecessors == 1) { if (target->b_instr[0].i_loc.lineno < 0) { target->b_instr[0].i_loc = prev_location; @@ -9576,15 +9577,16 @@ duplicate_exits_without_lineno(cfg_builder *g) */ basicblock *entryblock = g->g_entryblock; for (basicblock *b = entryblock; b != NULL; b = b->b_next) { - if (b->b_iused > 0 && is_jump(&b->b_instr[b->b_iused-1])) { - basicblock *target = b->b_instr[b->b_iused-1].i_target; + struct instr *last = basicblock_last_instr(b); + if (last != NULL && is_jump(last)) { + basicblock *target = last->i_target; if (is_exit_without_lineno(target) && target->b_predecessors > 1) { basicblock *new_target = copy_basicblock(g, target); if (new_target == NULL) { return -1; } - new_target->b_instr[0].i_loc = b->b_instr[b->b_iused-1].i_loc; - b->b_instr[b->b_iused-1].i_target = new_target; + new_target->b_instr[0].i_loc = last->i_loc; + last->i_target = new_target; target->b_predecessors--; new_target->b_predecessors = 1; new_target->b_next = target->b_next; @@ -9603,8 +9605,9 @@ duplicate_exits_without_lineno(cfg_builder *g) for (basicblock *b = entryblock; b != NULL; b = b->b_next) { if (BB_HAS_FALLTHROUGH(b) && b->b_next && b->b_iused > 0) { if (is_exit_without_lineno(b->b_next)) { - assert(b->b_next->b_iused > 0); - b->b_next->b_instr[0].i_loc = b->b_instr[b->b_iused-1].i_loc; + struct instr *last = basicblock_last_instr(b); + assert(last != NULL); + b->b_next->b_instr[0].i_loc = last->i_loc; } } }

1 0

GH-96179: Fix misleading example on the bisect documentation (GH-96228)
by miss-islington Aug. 24, 2022

Aug. 24, 2022

https://github.com/python/cpython/commit/41e8257449c2c44bbe8fc9577cdf5ba0c5… commit: 41e8257449c2c44bbe8fc9577cdf5ba0c54b867d branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-08-24T08:55:45-07:00 summary: GH-96179: Fix misleading example on the bisect documentation (GH-96228) The `movies[bisect(movies, 1960, key=by_year)]` will actually return only movies **after** 1960. (cherry picked from commit 4317b25a2323ae4be04574e45de0e335c571c463) Co-authored-by: prego <pedropregueiro(a)gmail.com> files: M Doc/library/bisect.rst M Misc/ACKS diff --git a/Doc/library/bisect.rst b/Doc/library/bisect.rst index 513675d3685..c2927c1ebd0 100644 --- a/Doc/library/bisect.rst +++ b/Doc/library/bisect.rst @@ -216,7 +216,7 @@ records in a table:: ... Movie('Aliens', 1986, 'Scott') ... ] - >>> # Find the first movie released on or after 1960 + >>> # Find the first movie released after 1960 >>> by_year = attrgetter('released') >>> movies.sort(key=by_year) >>> movies[bisect(movies, 1960, key=by_year)] diff --git a/Misc/ACKS b/Misc/ACKS index 8726c0f6e62..fc1dcafcc85 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1405,6 +1405,7 @@ John Popplewell Matheus Vieira Portela Davin Potts Guillaume Pratte +Pedro Pregueiro Florian Preinstorfer Alex Prengère Amrit Prem

1 0

GH-96179: Fix misleading example on the bisect documentation (GH-96228)
by miss-islington Aug. 24, 2022

Aug. 24, 2022

https://github.com/python/cpython/commit/d02e8fbc5a4a9f43ceaacb32dee84f8f66… commit: d02e8fbc5a4a9f43ceaacb32dee84f8f661a12d2 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-08-24T08:54:15-07:00 summary: GH-96179: Fix misleading example on the bisect documentation (GH-96228) The `movies[bisect(movies, 1960, key=by_year)]` will actually return only movies **after** 1960. (cherry picked from commit 4317b25a2323ae4be04574e45de0e335c571c463) Co-authored-by: prego <pedropregueiro(a)gmail.com> files: M Doc/library/bisect.rst M Misc/ACKS diff --git a/Doc/library/bisect.rst b/Doc/library/bisect.rst index 513675d3685..c2927c1ebd0 100644 --- a/Doc/library/bisect.rst +++ b/Doc/library/bisect.rst @@ -216,7 +216,7 @@ records in a table:: ... Movie('Aliens', 1986, 'Scott') ... ] - >>> # Find the first movie released on or after 1960 + >>> # Find the first movie released after 1960 >>> by_year = attrgetter('released') >>> movies.sort(key=by_year) >>> movies[bisect(movies, 1960, key=by_year)] diff --git a/Misc/ACKS b/Misc/ACKS index 4beb4fd6b3d..6e53d5a7cc4 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1415,6 +1415,7 @@ John Popplewell Matheus Vieira Portela Davin Potts Guillaume Pratte +Pedro Pregueiro Florian Preinstorfer Alex Prengère Amrit Prem

1 0

GH-96179: Fix misleading example on the bisect documentation (GH-96228)
by encukou Aug. 24, 2022

Aug. 24, 2022

https://github.com/python/cpython/commit/4317b25a2323ae4be04574e45de0e335c5… commit: 4317b25a2323ae4be04574e45de0e335c571c463 branch: main author: prego <pedropregueiro(a)gmail.com> committer: encukou <encukou(a)gmail.com> date: 2022-08-24T17:47:13+02:00 summary: GH-96179: Fix misleading example on the bisect documentation (GH-96228) The `movies[bisect(movies, 1960, key=by_year)]` will actually return only movies **after** 1960. files: M Doc/library/bisect.rst M Misc/ACKS diff --git a/Doc/library/bisect.rst b/Doc/library/bisect.rst index 76045ea511a..7856bf67a7b 100644 --- a/Doc/library/bisect.rst +++ b/Doc/library/bisect.rst @@ -216,7 +216,7 @@ records in a table:: ... Movie('Aliens', 1986, 'Scott') ... ] - >>> # Find the first movie released on or after 1960 + >>> # Find the first movie released after 1960 >>> by_year = attrgetter('released') >>> movies.sort(key=by_year) >>> movies[bisect(movies, 1960, key=by_year)] diff --git a/Misc/ACKS b/Misc/ACKS index 16a482e40a5..daa01c1a0f1 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1419,6 +1419,7 @@ John Popplewell Matheus Vieira Portela Davin Potts Guillaume Pratte +Pedro Pregueiro Florian Preinstorfer Alex Prengère Amrit Prem

1 0