Mailman 3 October 2023 - Python-checkins

[3.12] gh-111282: Fix NamedTemporaryFile example code (GH-111283) (GH-111579)
by zware Oct. 31, 2023

Oct. 31, 2023

https://github.com/python/cpython/commit/21c8fbf28dda1f338303295b094b9cc365… commit: 21c8fbf28dda1f338303295b094b9cc365d101b2 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: zware <zachary.ware(a)gmail.com> date: 2023-10-31T22:14:21Z summary: [3.12] gh-111282: Fix NamedTemporaryFile example code (GH-111283) (GH-111579) (cherry picked from commit 102685c4c8481ec5d9c132fcf06b46057e815969) Co-authored-by: Krzysiek Karbowiak <krzysztof.karbowiak(a)interia.pl> files: M Doc/library/tempfile.rst diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index b2baa54d9522d..b68a78e8267bc 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -404,13 +404,13 @@ Here are some examples of typical usage of the :mod:`tempfile` module:: # create a temporary file using a context manager # close the file, use the name to open the file again - >>> with tempfile.TemporaryFile(delete_on_close=False) as fp: - ... fp.write(b'Hello world!') - ... fp.close() - # the file is closed, but not removed - # open the file again by using its name - ... with open(fp.name) as f - ... f.read() + >>> with tempfile.NamedTemporaryFile(delete_on_close=False) as fp: + ... fp.write(b'Hello world!') + ... fp.close() + ... # the file is closed, but not removed + ... # open the file again by using its name + ... with open(fp.name, mode='rb') as f: + ... f.read() b'Hello world!' >>> # file is now removed

1 0

gh-111282: Fix NamedTemporaryFile example code (GH-111283)
by zware Oct. 31, 2023

Oct. 31, 2023

https://github.com/python/cpython/commit/102685c4c8481ec5d9c132fcf06b46057e… commit: 102685c4c8481ec5d9c132fcf06b46057e815969 branch: main author: Krzysiek Karbowiak <krzysztof.karbowiak(a)interia.pl> committer: zware <zachary.ware(a)gmail.com> date: 2023-10-31T17:06:02-05:00 summary: gh-111282: Fix NamedTemporaryFile example code (GH-111283) files: M Doc/library/tempfile.rst diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index b2baa54d9522d..b68a78e8267bc 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -404,13 +404,13 @@ Here are some examples of typical usage of the :mod:`tempfile` module:: # create a temporary file using a context manager # close the file, use the name to open the file again - >>> with tempfile.TemporaryFile(delete_on_close=False) as fp: - ... fp.write(b'Hello world!') - ... fp.close() - # the file is closed, but not removed - # open the file again by using its name - ... with open(fp.name) as f - ... f.read() + >>> with tempfile.NamedTemporaryFile(delete_on_close=False) as fp: + ... fp.write(b'Hello world!') + ... fp.close() + ... # the file is closed, but not removed + ... # open the file again by using its name + ... with open(fp.name, mode='rb') as f: + ... f.read() b'Hello world!' >>> # file is now removed

1 0

gh-110774: allow setting the Runner(loop_factory=...) from IsolatedAsyncioTestCase (#110776)
by gvanrossum Oct. 31, 2023

Oct. 31, 2023

https://github.com/python/cpython/commit/770530679e89b06f33655b34a8c466ed90… commit: 770530679e89b06f33655b34a8c466ed906842fe branch: main author: Thomas Grainger <tagrain(a)gmail.com> committer: gvanrossum <gvanrossum(a)gmail.com> date: 2023-10-31T15:03:54-07:00 summary: gh-110774: allow setting the Runner(loop_factory=...) from IsolatedAsyncioTestCase (#110776) Co-authored-by: Łukasz Langa <lukasz(a)langa.pl> Co-authored-by: Carol Willing <carolcode(a)willingconsulting.com> files: A Misc/NEWS.d/next/Library/2023-10-12-15-16-44.gh-issue-110774.AdCb5A.rst M Doc/library/unittest.rst M Lib/test/test_unittest/test_async_case.py M Lib/unittest/async_case.py diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index 21abc583f853a..c90c554591e74 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -1571,6 +1571,14 @@ Test cases .. versionadded:: 3.8 + .. attribute:: loop_factory + + The *loop_factory* passed to :class:`asyncio.Runner`. Override + in subclasses with :class:`asyncio.EventLoop` to avoid using the + asyncio policy system. + + .. versionadded:: 3.13 + .. coroutinemethod:: asyncSetUp() Method called to prepare the test fixture. This is called after :meth:`setUp`. diff --git a/Lib/test/test_unittest/test_async_case.py b/Lib/test/test_unittest/test_async_case.py index a465103b59b6e..ba1ab838cd4a2 100644 --- a/Lib/test/test_unittest/test_async_case.py +++ b/Lib/test/test_unittest/test_async_case.py @@ -484,5 +484,19 @@ async def test_demo1(self): result = test.run() self.assertTrue(result.wasSuccessful()) + def test_loop_factory(self): + asyncio.set_event_loop_policy(None) + + class TestCase1(unittest.IsolatedAsyncioTestCase): + loop_factory = asyncio.EventLoop + + async def test_demo1(self): + pass + + test = TestCase1('test_demo1') + result = test.run() + self.assertTrue(result.wasSuccessful()) + self.assertIsNone(support.maybe_get_event_loop_policy()) + if __name__ == "__main__": unittest.main() diff --git a/Lib/unittest/async_case.py b/Lib/unittest/async_case.py index bd2a471156065..63ff6a5d1f8b6 100644 --- a/Lib/unittest/async_case.py +++ b/Lib/unittest/async_case.py @@ -25,12 +25,15 @@ class IsolatedAsyncioTestCase(TestCase): # them inside the same task. # Note: the test case modifies event loop policy if the policy was not instantiated - # yet. + # yet, unless loop_factory=asyncio.EventLoop is set. # asyncio.get_event_loop_policy() creates a default policy on demand but never # returns None # I believe this is not an issue in user level tests but python itself for testing # should reset a policy in every test module # by calling asyncio.set_event_loop_policy(None) in tearDownModule() + # or set loop_factory=asyncio.EventLoop + + loop_factory = None def __init__(self, methodName='runTest'): super().__init__(methodName) @@ -118,7 +121,7 @@ def _callMaybeAsync(self, func, /, *args, **kwargs): def _setupAsyncioRunner(self): assert self._asyncioRunner is None, 'asyncio runner is already initialized' - runner = asyncio.Runner(debug=True) + runner = asyncio.Runner(debug=True, loop_factory=self.loop_factory) self._asyncioRunner = runner def _tearDownAsyncioRunner(self): diff --git a/Misc/NEWS.d/next/Library/2023-10-12-15-16-44.gh-issue-110774.AdCb5A.rst b/Misc/NEWS.d/next/Library/2023-10-12-15-16-44.gh-issue-110774.AdCb5A.rst new file mode 100644 index 0000000000000..a5a9fed11864d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2023-10-12-15-16-44.gh-issue-110774.AdCb5A.rst @@ -0,0 +1 @@ +Support setting the :class:`asyncio.Runner` loop_factory kwarg in :class:`unittest.IsolatedAsyncioTestCase`

1 0

[3.12] gh-111181: Fix enum doctests (GH-111180) (GH-111518)
by ethanfurman Oct. 31, 2023

Oct. 31, 2023

https://github.com/python/cpython/commit/ec00397912ea57a896cb7029e14fad8937… commit: ec00397912ea57a896cb7029e14fad89371b8b2f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: ethanfurman <ethan(a)stoneleaf.us> date: 2023-10-31T15:01:28-07:00 summary: [3.12] gh-111181: Fix enum doctests (GH-111180) (GH-111518) gh-111181: Fix enum doctests (GH-111180) (cherry picked from commit c4dc5a6ae8aa13abb743182df088f1a3526d1bcd) Co-authored-by: Nikita Sobolev <mail(a)sobolevn.me> Co-authored-by: Ethan Furman <ethan(a)stoneleaf.us> files: M Doc/howto/enum.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Doc/howto/enum.rst b/Doc/howto/enum.rst index ebaa1cfe4c8b5..0830fb630d701 100644 --- a/Doc/howto/enum.rst +++ b/Doc/howto/enum.rst @@ -483,6 +483,7 @@ Dataclass support When inheriting from a :class:`~dataclasses.dataclass`, the :meth:`~Enum.__repr__` omits the inherited class' name. For example:: + >>> from dataclasses import dataclass, field >>> @dataclass ... class CreatureDataMixin: ... size: str @@ -527,7 +528,8 @@ It is possible to modify how enum members are pickled/unpickled by defining :meth:`__reduce_ex__` in the enumeration class. The default method is by-value, but enums with complicated values may want to use by-name:: - >>> class MyEnum(Enum): + >>> import enum + >>> class MyEnum(enum.Enum): ... __reduce_ex__ = enum.pickle_by_enum_name .. note:: @@ -770,7 +772,7 @@ be combined with them (but may lose :class:`IntFlag` membership:: >>> Perm.X | 4 <Perm.R|X: 5> - >>> Perm.X | 8 + >>> Perm.X + 8 9 .. note:: @@ -1435,8 +1437,9 @@ alias:: ... GRENE = 2 ... Traceback (most recent call last): - ... + ... ValueError: aliases not allowed in DuplicateFreeEnum: 'GRENE' --> 'GREEN' + Error calling __set_name__ on '_proto_member' instance 'GRENE' in 'Color' .. note:: diff --git a/Lib/enum.py b/Lib/enum.py index c207dc234c0da..4bd3756ec5bc0 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1217,14 +1217,13 @@ def __str__(self): def __dir__(self): """ - Returns all members and all public methods + Returns public methods and other interesting attributes. """ - if self.__class__._member_type_ is object: - interesting = set(['__class__', '__doc__', '__eq__', '__hash__', '__module__', 'name', 'value']) - else: + interesting = set() + if self.__class__._member_type_ is not object: interesting = set(object.__dir__(self)) for name in getattr(self, '__dict__', []): - if name[0] != '_': + if name[0] != '_' and name not in self._member_map_: interesting.add(name) for cls in self.__class__.mro(): for name, obj in cls.__dict__.items(): @@ -1237,7 +1236,7 @@ def __dir__(self): else: # in case it was added by `dir(self)` interesting.discard(name) - else: + elif name not in self._member_map_: interesting.add(name) names = sorted( set(['__class__', '__doc__', '__eq__', '__hash__', '__module__']) diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 14f16f7f26efc..3bd918fb941c7 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -18,7 +18,7 @@ from io import StringIO from pickle import dumps, loads, PicklingError, HIGHEST_PROTOCOL from test import support -from test.support import ALWAYS_EQ +from test.support import ALWAYS_EQ, REPO_ROOT from test.support import threading_helper from datetime import timedelta @@ -26,14 +26,19 @@ def load_tests(loader, tests, ignore): tests.addTests(doctest.DocTestSuite(enum)) - if os.path.exists('Doc/library/enum.rst'): + + lib_tests = os.path.join(REPO_ROOT, 'Doc/library/enum.rst') + if os.path.exists(lib_tests): tests.addTests(doctest.DocFileSuite( - '../../Doc/library/enum.rst', + lib_tests, + module_relative=False, optionflags=doctest.ELLIPSIS|doctest.NORMALIZE_WHITESPACE, )) - if os.path.exists('Doc/howto/enum.rst'): + howto_tests = os.path.join(REPO_ROOT, 'Doc/howto/enum.rst') + if os.path.exists(howto_tests): tests.addTests(doctest.DocFileSuite( - '../../Doc/howto/enum.rst', + howto_tests, + module_relative=False, optionflags=doctest.ELLIPSIS|doctest.NORMALIZE_WHITESPACE, )) return tests @@ -5127,7 +5132,7 @@ def member_dir(member): allowed.add(name) else: allowed.discard(name) - else: + elif name not in member._member_map_: allowed.add(name) return sorted(allowed)

1 0

gh-108082: Add PyErr_FormatUnraisable() function (GH-111086)
by serhiy-storchaka Oct. 31, 2023

Oct. 31, 2023

https://github.com/python/cpython/commit/f6a02327b5fcdc10df855985ca9d2d9dc2… commit: f6a02327b5fcdc10df855985ca9d2d9dc2a0a46f branch: main author: Serhiy Storchaka <storchaka(a)gmail.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2023-10-31T23:42:44+02:00 summary: gh-108082: Add PyErr_FormatUnraisable() function (GH-111086) files: A Misc/NEWS.d/next/C API/2023-10-19-22-39-24.gh-issue-108082.uJytvc.rst M Doc/c-api/exceptions.rst M Doc/whatsnew/3.13.rst M Include/cpython/pyerrors.h M Lib/test/test_capi/test_exceptions.py M Modules/_testcapi/exceptions.c M Python/errors.c diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index f27e2bbfef05c..a3a63b38c432f 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -99,6 +99,18 @@ Printing and clearing Use :func:`sys.unraisablehook`. +.. c:function:: void PyErr_FormatUnraisable(const char *format, ...) + + Similar to :c:func:`PyErr_WriteUnraisable`, but the *format* and subsequent + parameters help format the warning message; they have the same meaning and + values as in :c:func:`PyUnicode_FromFormat`. + ``PyErr_WriteUnraisable(obj)`` is roughtly equivalent to + ``PyErr_FormatUnraisable("Exception ignored in: %R, obj)``. + If *format* is ``NULL``, only the traceback is printed. + + .. versionadded:: 3.13 + + .. c:function:: void PyErr_DisplayException(PyObject *exc) Print the standard traceback display of ``exc`` to ``sys.stderr``, including @@ -106,6 +118,7 @@ Printing and clearing .. versionadded:: 3.12 + Raising exceptions ================== diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 197790234a158..9181685736575 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -1107,6 +1107,10 @@ New Features limited C API. (Contributed by Victor Stinner in :gh:`85283`.) +* Add :c:func:`PyErr_FormatUnraisable` function: similar to + :c:func:`PyErr_WriteUnraisable`, but allow to customize the warning mesage. + (Contributed by Serhiy Storchaka in :gh:`108082`.) + * Add :c:func:`PyUnicode_AsUTF8` function to the limited C API. (Contributed by Victor Stinner in :gh:`111089`.) diff --git a/Include/cpython/pyerrors.h b/Include/cpython/pyerrors.h index da96eec4b35aa..479b908fb7058 100644 --- a/Include/cpython/pyerrors.h +++ b/Include/cpython/pyerrors.h @@ -120,4 +120,6 @@ PyAPI_FUNC(void) _Py_NO_RETURN _Py_FatalErrorFunc( const char *func, const char *message); +PyAPI_FUNC(void) PyErr_FormatUnraisable(const char *, ...); + #define Py_FatalError(message) _Py_FatalErrorFunc(__func__, (message)) diff --git a/Lib/test/test_capi/test_exceptions.py b/Lib/test/test_capi/test_exceptions.py index 1bff65b6559f0..1d158e3586e98 100644 --- a/Lib/test/test_capi/test_exceptions.py +++ b/Lib/test/test_capi/test_exceptions.py @@ -315,6 +315,63 @@ def test_err_writeunraisable(self): # CRASHES writeunraisable(NULL, hex) # CRASHES writeunraisable(NULL, NULL) + def test_err_formatunraisable(self): + # Test PyErr_FormatUnraisable() + formatunraisable = _testcapi.err_formatunraisable + firstline = self.test_err_formatunraisable.__code__.co_firstlineno + + with support.catch_unraisable_exception() as cm: + formatunraisable(CustomError('oops!'), b'Error in %R', []) + self.assertEqual(cm.unraisable.exc_type, CustomError) + self.assertEqual(str(cm.unraisable.exc_value), 'oops!') + self.assertEqual(cm.unraisable.exc_traceback.tb_lineno, + firstline + 6) + self.assertEqual(cm.unraisable.err_msg, 'Error in []') + self.assertIsNone(cm.unraisable.object) + + with support.catch_unraisable_exception() as cm: + formatunraisable(CustomError('oops!'), b'undecodable \xff') + self.assertEqual(cm.unraisable.exc_type, CustomError) + self.assertEqual(str(cm.unraisable.exc_value), 'oops!') + self.assertEqual(cm.unraisable.exc_traceback.tb_lineno, + firstline + 15) + self.assertIsNone(cm.unraisable.err_msg) + self.assertIsNone(cm.unraisable.object) + + with support.catch_unraisable_exception() as cm: + formatunraisable(CustomError('oops!'), NULL) + self.assertEqual(cm.unraisable.exc_type, CustomError) + self.assertEqual(str(cm.unraisable.exc_value), 'oops!') + self.assertEqual(cm.unraisable.exc_traceback.tb_lineno, + firstline + 24) + self.assertIsNone(cm.unraisable.err_msg) + self.assertIsNone(cm.unraisable.object) + + with (support.swap_attr(sys, 'unraisablehook', None), + support.captured_stderr() as stderr): + formatunraisable(CustomError('oops!'), b'Error in %R', []) + lines = stderr.getvalue().splitlines() + self.assertEqual(lines[0], f'Error in []:') + self.assertEqual(lines[1], 'Traceback (most recent call last):') + self.assertEqual(lines[-1], f'{__name__}.CustomError: oops!') + + with (support.swap_attr(sys, 'unraisablehook', None), + support.captured_stderr() as stderr): + formatunraisable(CustomError('oops!'), b'undecodable \xff') + lines = stderr.getvalue().splitlines() + self.assertEqual(lines[0], 'Traceback (most recent call last):') + self.assertEqual(lines[-1], f'{__name__}.CustomError: oops!') + + with (support.swap_attr(sys, 'unraisablehook', None), + support.captured_stderr() as stderr): + formatunraisable(CustomError('oops!'), NULL) + lines = stderr.getvalue().splitlines() + self.assertEqual(lines[0], 'Traceback (most recent call last):') + self.assertEqual(lines[-1], f'{__name__}.CustomError: oops!') + + # CRASHES formatunraisable(NULL, b'Error in %R', []) + # CRASHES formatunraisable(NULL, NULL) + class Test_PyUnstable_Exc_PrepReraiseStar(ExceptionIsLikeMixin, unittest.TestCase): diff --git a/Misc/NEWS.d/next/C API/2023-10-19-22-39-24.gh-issue-108082.uJytvc.rst b/Misc/NEWS.d/next/C API/2023-10-19-22-39-24.gh-issue-108082.uJytvc.rst new file mode 100644 index 0000000000000..b99a829e3f2a5 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-10-19-22-39-24.gh-issue-108082.uJytvc.rst @@ -0,0 +1 @@ +Add :c:func:`PyErr_FormatUnraisable` function. diff --git a/Modules/_testcapi/exceptions.c b/Modules/_testcapi/exceptions.c index aac672a3788ae..42a9915143e6f 100644 --- a/Modules/_testcapi/exceptions.c +++ b/Modules/_testcapi/exceptions.c @@ -319,6 +319,30 @@ err_writeunraisable(PyObject *Py_UNUSED(module), PyObject *args) Py_RETURN_NONE; } +static PyObject * +err_formatunraisable(PyObject *Py_UNUSED(module), PyObject *args) +{ + PyObject *exc; + const char *fmt; + Py_ssize_t fmtlen; + PyObject *objs[10] = {NULL}; + + if (!PyArg_ParseTuple(args, "Oz#|OOOOOOOOOO", &exc, &fmt, &fmtlen, + &objs[0], &objs[1], &objs[2], &objs[3], &objs[4], + &objs[5], &objs[6], &objs[7], &objs[8], &objs[9])) + { + return NULL; + } + NULLABLE(exc); + if (exc) { + PyErr_SetRaisedException(Py_NewRef(exc)); + } + PyErr_FormatUnraisable(fmt, + objs[0], objs[1], objs[2], objs[3], objs[4], + objs[5], objs[6], objs[7], objs[8], objs[9]); + Py_RETURN_NONE; +} + /*[clinic input] _testcapi.unstable_exc_prep_reraise_star orig: object @@ -364,6 +388,7 @@ static PyTypeObject PyRecursingInfinitelyError_Type = { static PyMethodDef test_methods[] = { {"err_restore", err_restore, METH_VARARGS}, {"err_writeunraisable", err_writeunraisable, METH_VARARGS}, + {"err_formatunraisable", err_formatunraisable, METH_VARARGS}, _TESTCAPI_ERR_SET_RAISED_METHODDEF _TESTCAPI_EXCEPTION_PRINT_METHODDEF _TESTCAPI_FATAL_ERROR_METHODDEF diff --git a/Python/errors.c b/Python/errors.c index 15af39b10dc07..f75c3e1fbd3f6 100644 --- a/Python/errors.c +++ b/Python/errors.c @@ -1569,14 +1569,16 @@ _PyErr_WriteUnraisableDefaultHook(PyObject *args) for Python to handle it. For example, when a destructor raises an exception or during garbage collection (gc.collect()). - If err_msg_str is non-NULL, the error message is formatted as: - "Exception ignored %s" % err_msg_str. Otherwise, use "Exception ignored in" - error message. + If format is non-NULL, the error message is formatted using format and + variable arguments as in PyUnicode_FromFormat(). + Otherwise, use "Exception ignored in" error message. An exception must be set when calling this function. */ -void -_PyErr_WriteUnraisableMsg(const char *err_msg_str, PyObject *obj) + +static void +format_unraisable_v(const char *format, va_list va, PyObject *obj) { + const char *err_msg_str; PyThreadState *tstate = _PyThreadState_GET(); _Py_EnsureTstateNotNULL(tstate); @@ -1610,8 +1612,8 @@ _PyErr_WriteUnraisableMsg(const char *err_msg_str, PyObject *obj) } } - if (err_msg_str != NULL) { - err_msg = PyUnicode_FromFormat("Exception ignored %s", err_msg_str); + if (format != NULL) { + err_msg = PyUnicode_FromFormatV(format, va); if (err_msg == NULL) { PyErr_Clear(); } @@ -1676,11 +1678,41 @@ _PyErr_WriteUnraisableMsg(const char *err_msg_str, PyObject *obj) _PyErr_Clear(tstate); /* Just in case */ } +void +PyErr_FormatUnraisable(const char *format, ...) +{ + va_list va; + + va_start(va, format); + format_unraisable_v(format, va, NULL); + va_end(va); +} + +static void +format_unraisable(PyObject *obj, const char *format, ...) +{ + va_list va; + + va_start(va, format); + format_unraisable_v(format, va, obj); + va_end(va); +} + +void +_PyErr_WriteUnraisableMsg(const char *err_msg_str, PyObject *obj) +{ + if (err_msg_str) { + format_unraisable(obj, "Exception ignored %s", err_msg_str); + } + else { + format_unraisable(obj, NULL); + } +} void PyErr_WriteUnraisable(PyObject *obj) { - _PyErr_WriteUnraisableMsg(NULL, obj); + format_unraisable(obj, NULL); }

1 0

gh-111420: Allow type comments in parenthesized `with` statements (#111468)
by pablogsal Oct. 31, 2023

Oct. 31, 2023

https://github.com/python/cpython/commit/453e96e3020d38cfcaebf82b24cb681c63… commit: 453e96e3020d38cfcaebf82b24cb681c6384fa82 branch: main author: Tomas R <tomas.roun8(a)gmail.com> committer: pablogsal <Pablogsal(a)gmail.com> date: 2023-10-31T21:02:42Z summary: gh-111420: Allow type comments in parenthesized `with` statements (#111468) files: A Misc/NEWS.d/next/Core and Builtins/2023-10-29-20-11-21.gh-issue-111420.IUT-GK.rst M Grammar/python.gram M Lib/test/test_ast.py M Lib/test/test_type_comments.py M Parser/parser.c diff --git a/Grammar/python.gram b/Grammar/python.gram index ec964d6783165..55930b0f357ca 100644 --- a/Grammar/python.gram +++ b/Grammar/python.gram @@ -391,8 +391,8 @@ for_stmt[stmt_ty]: with_stmt[stmt_ty]: | invalid_with_stmt_indent - | 'with' '(' a[asdl_withitem_seq*]=','.with_item+ ','? ')' ':' b=block { - CHECK_VERSION(stmt_ty, 9, "Parenthesized context managers are", _PyAST_With(a, b, NULL, EXTRA)) } + | 'with' '(' a[asdl_withitem_seq*]=','.with_item+ ','? ')' ':' tc=[TYPE_COMMENT] b=block { + CHECK_VERSION(stmt_ty, 9, "Parenthesized context managers are", _PyAST_With(a, b, NEW_TYPE_COMMENT(p, tc), EXTRA)) } | 'with' a[asdl_withitem_seq*]=','.with_item+ ':' tc=[TYPE_COMMENT] b=block { _PyAST_With(a, b, NEW_TYPE_COMMENT(p, tc), EXTRA) } | 'async' 'with' '(' a[asdl_withitem_seq*]=','.with_item+ ','? ')' ':' b=block { diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py index fa96e259b8c97..69c356ed88f57 100644 --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -100,6 +100,8 @@ def to_tuple(t): # With "with x as y: pass", "with x as y, z as q: pass", + "with (x as y): pass", + "with (x, y): pass", # Raise "raise Exception('string')", # TryExcept @@ -3015,6 +3017,8 @@ def main(): ('Module', [('If', (1, 0, 6, 6), ('Name', (1, 3, 1, 4), 'a', ('Load',)), [('Pass', (2, 2, 2, 6))], [('If', (3, 0, 6, 6), ('Name', (3, 5, 3, 6), 'b', ('Load',)), [('Pass', (4, 2, 4, 6))], [('Pass', (6, 2, 6, 6))])])], []), ('Module', [('With', (1, 0, 1, 17), [('withitem', ('Name', (1, 5, 1, 6), 'x', ('Load',)), ('Name', (1, 10, 1, 11), 'y', ('Store',)))], [('Pass', (1, 13, 1, 17))], None)], []), ('Module', [('With', (1, 0, 1, 25), [('withitem', ('Name', (1, 5, 1, 6), 'x', ('Load',)), ('Name', (1, 10, 1, 11), 'y', ('Store',))), ('withitem', ('Name', (1, 13, 1, 14), 'z', ('Load',)), ('Name', (1, 18, 1, 19), 'q', ('Store',)))], [('Pass', (1, 21, 1, 25))], None)], []), +('Module', [('With', (1, 0, 1, 19), [('withitem', ('Name', (1, 6, 1, 7), 'x', ('Load',)), ('Name', (1, 11, 1, 12), 'y', ('Store',)))], [('Pass', (1, 15, 1, 19))], None)], []), +('Module', [('With', (1, 0, 1, 17), [('withitem', ('Name', (1, 6, 1, 7), 'x', ('Load',)), None), ('withitem', ('Name', (1, 9, 1, 10), 'y', ('Load',)), None)], [('Pass', (1, 13, 1, 17))], None)], []), ('Module', [('Raise', (1, 0, 1, 25), ('Call', (1, 6, 1, 25), ('Name', (1, 6, 1, 15), 'Exception', ('Load',)), [('Constant', (1, 16, 1, 24), 'string', None)], []), None)], []), ('Module', [('Try', (1, 0, 4, 6), [('Pass', (2, 2, 2, 6))], [('ExceptHandler', (3, 0, 4, 6), ('Name', (3, 7, 3, 16), 'Exception', ('Load',)), None, [('Pass', (4, 2, 4, 6))])], [], [])], []), ('Module', [('Try', (1, 0, 4, 6), [('Pass', (2, 2, 2, 6))], [], [], [('Pass', (4, 2, 4, 6))])], []), diff --git a/Lib/test/test_type_comments.py b/Lib/test/test_type_comments.py index 9a11fab237235..5a911da56f8f8 100644 --- a/Lib/test/test_type_comments.py +++ b/Lib/test/test_type_comments.py @@ -66,6 +66,14 @@ def foo(): pass """ +parenthesized_withstmt = """\ +with (a as b): # type: int + pass + +with (a, b): # type: int + pass +""" + vardecl = """\ a = 0 # type: int """ @@ -300,6 +308,14 @@ def test_withstmt(self): tree = self.classic_parse(withstmt) self.assertEqual(tree.body[0].type_comment, None) + def test_parenthesized_withstmt(self): + for tree in self.parse_all(parenthesized_withstmt, minver=9): + self.assertEqual(tree.body[0].type_comment, "int") + self.assertEqual(tree.body[1].type_comment, "int") + tree = self.classic_parse(parenthesized_withstmt) + self.assertEqual(tree.body[0].type_comment, None) + self.assertEqual(tree.body[1].type_comment, None) + def test_vardecl(self): for tree in self.parse_all(vardecl): self.assertEqual(tree.body[0].type_comment, "int") diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-10-29-20-11-21.gh-issue-111420.IUT-GK.rst b/Misc/NEWS.d/next/Core and Builtins/2023-10-29-20-11-21.gh-issue-111420.IUT-GK.rst new file mode 100644 index 0000000000000..6646ecf920eeb --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-10-29-20-11-21.gh-issue-111420.IUT-GK.rst @@ -0,0 +1 @@ +Allow type comments in parenthesized ``with`` statements diff --git a/Parser/parser.c b/Parser/parser.c index f367cf9235f35..ca8e9d0e23661 100644 --- a/Parser/parser.c +++ b/Parser/parser.c @@ -6483,7 +6483,7 @@ for_stmt_rule(Parser *p) // with_stmt: // | invalid_with_stmt_indent -// | 'with' '(' ','.with_item+ ','? ')' ':' block +// | 'with' '(' ','.with_item+ ','? ')' ':' TYPE_COMMENT? block // | 'with' ','.with_item+ ':' TYPE_COMMENT? block // | 'async' 'with' '(' ','.with_item+ ','? ')' ':' block // | 'async' 'with' ','.with_item+ ':' TYPE_COMMENT? block @@ -6528,12 +6528,12 @@ with_stmt_rule(Parser *p) D(fprintf(stderr, "%*c%s with_stmt[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "invalid_with_stmt_indent")); } - { // 'with' '(' ','.with_item+ ','? ')' ':' block + { // 'with' '(' ','.with_item+ ','? ')' ':' TYPE_COMMENT? block if (p->error_indicator) { p->level--; return NULL; } - D(fprintf(stderr, "%*c> with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'with' '(' ','.with_item+ ','? ')' ':' block")); + D(fprintf(stderr, "%*c> with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'with' '(' ','.with_item+ ','? ')' ':' TYPE_COMMENT? block")); Token * _keyword; Token * _literal; Token * _literal_1; @@ -6542,6 +6542,7 @@ with_stmt_rule(Parser *p) UNUSED(_opt_var); // Silence compiler warnings asdl_withitem_seq* a; asdl_stmt_seq* b; + void *tc; if ( (_keyword = _PyPegen_expect_token(p, 629)) // token='with' && @@ -6555,10 +6556,12 @@ with_stmt_rule(Parser *p) && (_literal_2 = _PyPegen_expect_token(p, 11)) // token=':' && + (tc = _PyPegen_expect_token(p, TYPE_COMMENT), !p->error_indicator) // TYPE_COMMENT? + && (b = block_rule(p)) // block ) { - D(fprintf(stderr, "%*c+ with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'with' '(' ','.with_item+ ','? ')' ':' block")); + D(fprintf(stderr, "%*c+ with_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'with' '(' ','.with_item+ ','? ')' ':' TYPE_COMMENT? block")); Token *_token = _PyPegen_get_last_nonnwhitespace_token(p); if (_token == NULL) { p->level--; @@ -6568,7 +6571,7 @@ with_stmt_rule(Parser *p) UNUSED(_end_lineno); // Only used by EXTRA macro int _end_col_offset = _token->end_col_offset; UNUSED(_end_col_offset); // Only used by EXTRA macro - _res = CHECK_VERSION ( stmt_ty , 9 , "Parenthesized context managers are" , _PyAST_With ( a , b , NULL , EXTRA ) ); + _res = CHECK_VERSION ( stmt_ty , 9 , "Parenthesized context managers are" , _PyAST_With ( a , b , NEW_TYPE_COMMENT ( p , tc ) , EXTRA ) ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; p->level--; @@ -6578,7 +6581,7 @@ with_stmt_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s with_stmt[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'with' '(' ','.with_item+ ','? ')' ':' block")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'with' '(' ','.with_item+ ','? ')' ':' TYPE_COMMENT? block")); } { // 'with' ','.with_item+ ':' TYPE_COMMENT? block if (p->error_indicator) {

1 0

gh-108765: Python.h no longer includes <stddef.h> on Windows (#111563)
by vstinner Oct. 31, 2023

Oct. 31, 2023

https://github.com/python/cpython/commit/faa5f6053d7334a3ecc513c64947ac0264… commit: faa5f6053d7334a3ecc513c64947ac026439c03a branch: main author: Victor Stinner <vstinner(a)python.org> committer: vstinner <vstinner(a)python.org> date: 2023-10-31T21:53:57+01:00 summary: gh-108765: Python.h no longer includes <stddef.h> on Windows (#111563) In practice, only Windows is impacted, because the HAVE_STDDEF_H macro was only defined on Windows. files: A Misc/NEWS.d/next/C API/2023-10-31-18-22-03.gh-issue-108765._beYv8.rst M Doc/whatsnew/3.13.rst M Include/Python.h diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 3a2df4fc8a3de..197790234a158 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -1186,6 +1186,14 @@ Porting to Python 3.13 ``PyUnicode_AsUTF8AndSize(unicode, NULL)`` can be used instead. (Contributed by Victor Stinner in :gh:`111089`.) +* On Windows, ``Python.h`` no longer includes the ``<stddef.h>`` standard + header file. If needed, it should now be included explicitly. For example, it + provides ``offsetof()`` function, and ``size_t`` and ``ptrdiff_t`` types. + Including ``<stddef.h>`` explicitly was already needed by all other + platforms, the ``HAVE_STDDEF_H`` macro is only defined on Windows. + (Contributed by Victor Stinner in :gh:`108765`.) + + Deprecated ---------- diff --git a/Include/Python.h b/Include/Python.h index 7312cc87d5cc3..a1f26afbb1225 100644 --- a/Include/Python.h +++ b/Include/Python.h @@ -22,9 +22,6 @@ #include <math.h> // HUGE_VAL #include <stdarg.h> // va_list #include <wchar.h> // wchar_t -#ifdef HAVE_STDDEF_H -# include <stddef.h> // size_t -#endif #ifdef HAVE_SYS_TYPES_H # include <sys/types.h> // ssize_t #endif diff --git a/Misc/NEWS.d/next/C API/2023-10-31-18-22-03.gh-issue-108765._beYv8.rst b/Misc/NEWS.d/next/C API/2023-10-31-18-22-03.gh-issue-108765._beYv8.rst new file mode 100644 index 0000000000000..14af79eaea9a0 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2023-10-31-18-22-03.gh-issue-108765._beYv8.rst @@ -0,0 +1,3 @@ +On Windows, ``Python.h`` no longer includes the ``<stddef.h>`` standard +header file. If needed, it should now be included explicitly. Patch by +Victor Stinner.

1 0

gh-90815: Exclude mimalloc .c files from Windows build (#111532)
by DinoV Oct. 31, 2023

Oct. 31, 2023

https://github.com/python/cpython/commit/c42347d025b29e993a12925a3ddf58c0c9… commit: c42347d025b29e993a12925a3ddf58c0c9bb7f7b branch: main author: Dino Viehland <dinoviehland(a)fb.com> committer: DinoV <dinoviehland(a)gmail.com> date: 2023-10-31T11:54:35-07:00 summary: gh-90815: Exclude mimalloc .c files from Windows build (#111532) * Don't include mimalloc .c's in Windows build * Fix warnings on Windows related to mimalloc files: M Objects/obmalloc.c M PC/pyconfig.h M PCbuild/_freeze_module.vcxproj M PCbuild/_freeze_module.vcxproj.filters M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c index d230bbdad3c4c..2761c77420978 100644 --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -2730,22 +2730,23 @@ static bool _collect_alloc_stats( static void py_mimalloc_print_stats(FILE *out) { - fprintf(out, "Small block threshold = %ld, in %u size classes.\n", + fprintf(out, "Small block threshold = %zd, in %u size classes.\n", MI_SMALL_OBJ_SIZE_MAX, MI_BIN_HUGE); - fprintf(out, "Medium block threshold = %ld\n", + fprintf(out, "Medium block threshold = %zd\n", MI_MEDIUM_OBJ_SIZE_MAX); - fprintf(out, "Large object max size = %ld\n", + fprintf(out, "Large object max size = %zd\n", MI_LARGE_OBJ_SIZE_MAX); mi_heap_t *heap = mi_heap_get_default(); - struct _alloc_stats stats = {}; + struct _alloc_stats stats; + memset(&stats, 0, sizeof(stats)); mi_heap_visit_blocks(heap, false, &_collect_alloc_stats, &stats); - fprintf(out, " Allocated Blocks: %ld\n", stats.allocated_blocks); - fprintf(out, " Allocated Bytes: %ld\n", stats.allocated_bytes); - fprintf(out, " Allocated Bytes w/ Overhead: %ld\n", stats.allocated_with_overhead); - fprintf(out, " Bytes Reserved: %ld\n", stats.bytes_reserved); - fprintf(out, " Bytes Committed: %ld\n", stats.bytes_committed); + fprintf(out, " Allocated Blocks: %zd\n", stats.allocated_blocks); + fprintf(out, " Allocated Bytes: %zd\n", stats.allocated_bytes); + fprintf(out, " Allocated Bytes w/ Overhead: %zd\n", stats.allocated_with_overhead); + fprintf(out, " Bytes Reserved: %zd\n", stats.bytes_reserved); + fprintf(out, " Bytes Committed: %zd\n", stats.bytes_committed); } #endif diff --git a/PC/pyconfig.h b/PC/pyconfig.h index ac20129cd30fc..e6b368caffe28 100644 --- a/PC/pyconfig.h +++ b/PC/pyconfig.h @@ -511,6 +511,9 @@ Py_NO_ENABLE_SHARED to find out. Also support MS_NO_COREDLL for b/w compat */ /* Use Python's own small-block memory-allocator. */ #define WITH_PYMALLOC 1 +/* Define if you want to compile in mimalloc memory allocator. */ +#define WITH_MIMALLOC 1 + /* Define if you want to compile in object freelists optimization */ #define WITH_FREELISTS 1 diff --git a/PCbuild/_freeze_module.vcxproj b/PCbuild/_freeze_module.vcxproj index 05b8bfdc38a99..20d800a695955 100644 --- a/PCbuild/_freeze_module.vcxproj +++ b/PCbuild/_freeze_module.vcxproj @@ -147,20 +147,6 @@ <ClCompile Include="..\Objects\longobject.c" /> <ClCompile Include="..\Objects\memoryobject.c" /> <ClCompile Include="..\Objects\methodobject.c" /> - <ClCompile Include="..\Objects\mimalloc\alloc-aligned.c" /> - <ClCompile Include="..\Objects\mimalloc\alloc.c" /> - <ClCompile Include="..\Objects\mimalloc\arena.c" /> - <ClCompile Include="..\Objects\mimalloc\bitmap.c" /> - <ClCompile Include="..\Objects\mimalloc\heap.c" /> - <ClCompile Include="..\Objects\mimalloc\init.c" /> - <ClCompile Include="..\Objects\mimalloc\options.c" /> - <ClCompile Include="..\Objects\mimalloc\os.c" /> - <ClCompile Include="..\Objects\mimalloc\page.c" /> - <ClCompile Include="..\Objects\mimalloc\prim\prim.c" /> - <ClCompile Include="..\Objects\mimalloc\random.c" /> - <ClCompile Include="..\Objects\mimalloc\segment.c" /> - <ClCompile Include="..\Objects\mimalloc\segment-map.c" /> - <ClCompile Include="..\Objects\mimalloc\stats.c" /> <ClCompile Include="..\Objects\moduleobject.c" /> <ClCompile Include="..\Objects\namespaceobject.c" /> <ClCompile Include="..\Objects\object.c" /> diff --git a/PCbuild/_freeze_module.vcxproj.filters b/PCbuild/_freeze_module.vcxproj.filters index d6cbd2d3d4736..40256a22fb556 100644 --- a/PCbuild/_freeze_module.vcxproj.filters +++ b/PCbuild/_freeze_module.vcxproj.filters @@ -253,48 +253,6 @@ <ClCompile Include="..\Objects\methodobject.c"> <Filter>Source Files</Filter> </ClCompile> - <ClCompile Include="..\Objects\mimalloc\alloc-aligned.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\alloc.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\arena.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\bitmap.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\heap.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\init.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\options.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\os.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\page.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\prim\prim.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\random.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\segment.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\segment-map.c"> - <Filter>Source Files</Filter> - </CLCompile> - <ClCompile Include="..\Objects\mimalloc\stats.c"> - <Filter>Source Files</Filter> - </CLCompile> <ClCompile Include="..\Python\modsupport.c"> <Filter>Source Files</Filter> </ClCompile> diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 954a59a0bc701..4d3621a614680 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -498,20 +498,6 @@ <ClCompile Include="..\Objects\longobject.c" /> <ClCompile Include="..\Objects\memoryobject.c" /> <ClCompile Include="..\Objects\methodobject.c" /> - <ClCompile Include="..\Objects\mimalloc\alloc-aligned.c" /> - <ClCompile Include="..\Objects\mimalloc\alloc.c" /> - <ClCompile Include="..\Objects\mimalloc\arena.c" /> - <ClCompile Include="..\Objects\mimalloc\bitmap.c" /> - <ClCompile Include="..\Objects\mimalloc\heap.c" /> - <ClCompile Include="..\Objects\mimalloc\init.c" /> - <ClCompile Include="..\Objects\mimalloc\options.c" /> - <ClCompile Include="..\Objects\mimalloc\os.c" /> - <ClCompile Include="..\Objects\mimalloc\page.c" /> - <ClCompile Include="..\Objects\mimalloc\prim\prim.c" /> - <ClCompile Include="..\Objects\mimalloc\random.c" /> - <ClCompile Include="..\Objects\mimalloc\segment.c" /> - <ClCompile Include="..\Objects\mimalloc\segment-map.c" /> - <ClCompile Include="..\Objects\mimalloc\stats.c" /> <ClCompile Include="..\Objects\moduleobject.c" /> <ClCompile Include="..\Objects\namespaceobject.c" /> <ClCompile Include="..\Objects\object.c" /> diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 2f8b206f973f3..69d8e0312e016 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -1130,48 +1130,6 @@ <ClCompile Include="..\Objects\methodobject.c"> <Filter>Objects</Filter> </ClCompile> - <ClCompile Include="..\Objects\mimalloc\alloc.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\alloc-aligned.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\arena.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\bitmap.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\heap.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\init.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\options.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\os.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\page.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\prim\prim.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\random.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\segment.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\segment-map.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> - <ClCompile Include="..\Objects\mimalloc\stats.c"> - <Filter>Objects\mimalloc</Filter> - </ClCompile> <ClCompile Include="..\Objects\moduleobject.c"> <Filter>Objects</Filter> </ClCompile>

1 0

[3.12] gh-109181: Speed up Traceback object creation by lazily compute the line number (GH-111548) (#111551)
by pablogsal Oct. 31, 2023

Oct. 31, 2023

https://github.com/python/cpython/commit/4f619e83fe1416c9b8d618533a0aaec0b4… commit: 4f619e83fe1416c9b8d618533a0aaec0b4443b11 branch: 3.12 author: Pablo Galindo Salgado <Pablogsal(a)gmail.com> committer: pablogsal <Pablogsal(a)gmail.com> date: 2023-10-31T18:18:11Z summary: [3.12] gh-109181: Speed up Traceback object creation by lazily compute the line number (GH-111548) (#111551) . (cherry picked from commit abb15420c11d9dda9c89f74eac8417240b321109) files: A Misc/NEWS.d/next/Core and Builtins/2023-10-31-14-25-21.gh-issue-109181.11h6Mc.rst M Python/traceback.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2023-10-31-14-25-21.gh-issue-109181.11h6Mc.rst b/Misc/NEWS.d/next/Core and Builtins/2023-10-31-14-25-21.gh-issue-109181.11h6Mc.rst new file mode 100644 index 0000000000000..61a15b471cfb2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2023-10-31-14-25-21.gh-issue-109181.11h6Mc.rst @@ -0,0 +1,2 @@ +Speed up :obj:`Traceback` object creation by lazily compute the line number. +Patch by Pablo Galindo diff --git a/Python/traceback.c b/Python/traceback.c index 4fc4881603678..b627b06df7e6a 100644 --- a/Python/traceback.c +++ b/Python/traceback.c @@ -107,6 +107,26 @@ tb_next_get(PyTracebackObject *self, void *Py_UNUSED(_)) return Py_NewRef(ret); } +static int +tb_get_lineno(PyTracebackObject* tb) { + PyFrameObject* frame = tb->tb_frame; + assert(frame != NULL); + return PyCode_Addr2Line(PyFrame_GetCode(frame), tb->tb_lasti); +} + +static PyObject * +tb_lineno_get(PyTracebackObject *self, void *Py_UNUSED(_)) +{ + int lineno = self->tb_lineno; + if (lineno == -1) { + lineno = tb_get_lineno(self); + if (lineno < 0) { + Py_RETURN_NONE; + } + } + return PyLong_FromLong(lineno); +} + static int tb_next_set(PyTracebackObject *self, PyObject *new_next, void *Py_UNUSED(_)) { @@ -150,12 +170,12 @@ static PyMethodDef tb_methods[] = { static PyMemberDef tb_memberlist[] = { {"tb_frame", T_OBJECT, OFF(tb_frame), READONLY|PY_AUDIT_READ}, {"tb_lasti", T_INT, OFF(tb_lasti), READONLY}, - {"tb_lineno", T_INT, OFF(tb_lineno), READONLY}, {NULL} /* Sentinel */ }; static PyGetSetDef tb_getsetters[] = { {"tb_next", (getter)tb_next_get, (setter)tb_next_set, NULL, NULL}, + {"tb_lineno", (getter)tb_lineno_get, NULL, NULL, NULL}, {NULL} /* Sentinel */ }; @@ -234,8 +254,7 @@ _PyTraceBack_FromFrame(PyObject *tb_next, PyFrameObject *frame) assert(tb_next == NULL || PyTraceBack_Check(tb_next)); assert(frame != NULL); int addr = _PyInterpreterFrame_LASTI(frame->f_frame) * sizeof(_Py_CODEUNIT); - return tb_create_raw((PyTracebackObject *)tb_next, frame, addr, - PyFrame_GetLineNumber(frame)); + return tb_create_raw((PyTracebackObject *)tb_next, frame, addr, -1); } @@ -983,9 +1002,13 @@ tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit, } while (tb != NULL) { code = PyFrame_GetCode(tb->tb_frame); + int tb_lineno = tb->tb_lineno; + if (tb_lineno == -1) { + tb_lineno = tb_get_lineno(tb); + } if (last_file == NULL || code->co_filename != last_file || - last_line == -1 || tb->tb_lineno != last_line || + last_line == -1 || tb_lineno != last_line || last_name == NULL || code->co_name != last_name) { if (cnt > TB_RECURSIVE_CUTOFF) { if (tb_print_line_repeated(f, cnt) < 0) { @@ -993,13 +1016,13 @@ tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit, } } last_file = code->co_filename; - last_line = tb->tb_lineno; + last_line = tb_lineno; last_name = code->co_name; cnt = 0; } cnt++; if (cnt <= TB_RECURSIVE_CUTOFF) { - if (tb_displayline(tb, f, code->co_filename, tb->tb_lineno, + if (tb_displayline(tb, f, code->co_filename, tb_lineno, tb->tb_frame, code->co_name, indent, margin) < 0) { goto error; }

1 0

[3.11] gh-102249: Expand sys.call_tracing documentation (GH-102806) (#111558)
by hugovk Oct. 31, 2023

Oct. 31, 2023

https://github.com/python/cpython/commit/2bb10acfdd9782e36534a1cb95bec520d0… commit: 2bb10acfdd9782e36534a1cb95bec520d0338ab6 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: hugovk <hugovk(a)users.noreply.github.com> date: 2023-10-31T18:40:17+02:00 summary: [3.11] gh-102249: Expand sys.call_tracing documentation (GH-102806) (#111558) Co-authored-by: Quentin Peter <impact27(a)users.noreply.github.com> Co-authored-by: C.A.M. Gerlach <CAM.Gerlach(a)Gerlach.CAM> files: M Doc/library/sys.rst diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 064a9c47137b3..f5b94f567ff74 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -173,7 +173,11 @@ always available. Call ``func(*args)``, while tracing is enabled. The tracing state is saved, and restored afterwards. This is intended to be called from a debugger from - a checkpoint, to recursively debug some other code. + a checkpoint, to recursively debug or profile some other code. + + Tracing is suspended while calling a tracing function set by + :func:`settrace` or :func:`setprofile` to avoid infinite recursion. + :func:`!call_tracing` enables explicit recursion of the tracing function. .. data:: copyright @@ -1439,13 +1443,16 @@ always available. its return value is not used, so it can simply return ``None``. Error in the profile function will cause itself unset. + .. note:: + The same tracing mechanism is used for :func:`!setprofile` as :func:`settrace`. + To trace calls with :func:`!setprofile` inside a tracing function + (e.g. in a debugger breakpoint), see :func:`call_tracing`. + Profile functions should have three arguments: *frame*, *event*, and *arg*. *frame* is the current stack frame. *event* is a string: ``'call'``, ``'return'``, ``'c_call'``, ``'c_return'``, or ``'c_exception'``. *arg* depends on the event type. - .. audit-event:: sys.setprofile "" sys.setprofile - The events have the following meaning: ``'call'`` @@ -1467,6 +1474,9 @@ always available. ``'c_exception'`` A C function has raised an exception. *arg* is the C function object. + .. audit-event:: sys.setprofile "" sys.setprofile + + .. function:: setrecursionlimit(limit) Set the maximum depth of the Python interpreter stack to *limit*. This limit @@ -1526,6 +1536,10 @@ always available. If there is any error occurred in the trace function, it will be unset, just like ``settrace(None)`` is called. + .. note:: + Tracing is disabled while calling the trace function (e.g. a function set by + :func:`!settrace`). For recursive tracing see :func:`call_tracing`. + The events have the following meaning: ``'call'``

1 0