Mailman 3 January 2025 - Python-checkins

[3.12] gh-129020: Remove ambiguous sentence from `tokenize.untokenize` docs (GH-129021) (#129036)
by AA-Turner Jan. 20, 2025

Jan. 20, 2025

https://github.com/python/cpython/commit/22c1d895fc73c3142795d2b5e62579e8a5… commit: 22c1d895fc73c3142795d2b5e62579e8a5225d1e branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: AA-Turner <9087854+AA-Turner(a)users.noreply.github.com> date: 2025-01-20T00:12:39Z summary: [3.12] gh-129020: Remove ambiguous sentence from `tokenize.untokenize` docs (GH-129021) (#129036) gh-129020: Remove ambiguous sentence from `tokenize.untokenize` docs (GH-129021) (cherry picked from commit bca35f0e782848ae2acdcfbfb000cd4a2af49fbd) Co-authored-by: Tomas R <tomas.roun8(a)gmail.com> files: M Doc/library/tokenize.rst diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst index f719319a302a23..b80917eae66f8b 100644 --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -91,11 +91,10 @@ write back the modified script. sequences with at least two elements, the token type and the token string. Any additional sequence elements are ignored. - The reconstructed script is returned as a single string. The result is - guaranteed to tokenize back to match the input so that the conversion is - lossless and round-trips are assured. The guarantee applies only to the - token type and token string as the spacing between tokens (column - positions) may change. + The result is guaranteed to tokenize back to match the input so that the + conversion is lossless and round-trips are assured. The guarantee applies + only to the token type and token string as the spacing between tokens + (column positions) may change. It returns bytes, encoded using the :data:`~token.ENCODING` token, which is the first token sequence output by :func:`.tokenize`. If there is no

1 0

[3.13] gh-129020: Remove ambiguous sentence from `tokenize.untokenize` docs (GH-129021) (#129035)
by AA-Turner Jan. 20, 2025

Jan. 20, 2025

https://github.com/python/cpython/commit/080d17e9453d1a4fc77025eb14a5dcf49b… commit: 080d17e9453d1a4fc77025eb14a5dcf49b4f379c branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: AA-Turner <9087854+AA-Turner(a)users.noreply.github.com> date: 2025-01-20T00:11:27Z summary: [3.13] gh-129020: Remove ambiguous sentence from `tokenize.untokenize` docs (GH-129021) (#129035) gh-129020: Remove ambiguous sentence from `tokenize.untokenize` docs (GH-129021) (cherry picked from commit bca35f0e782848ae2acdcfbfb000cd4a2af49fbd) Co-authored-by: Tomas R <tomas.roun8(a)gmail.com> files: M Doc/library/tokenize.rst diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst index f719319a302a23..b80917eae66f8b 100644 --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -91,11 +91,10 @@ write back the modified script. sequences with at least two elements, the token type and the token string. Any additional sequence elements are ignored. - The reconstructed script is returned as a single string. The result is - guaranteed to tokenize back to match the input so that the conversion is - lossless and round-trips are assured. The guarantee applies only to the - token type and token string as the spacing between tokens (column - positions) may change. + The result is guaranteed to tokenize back to match the input so that the + conversion is lossless and round-trips are assured. The guarantee applies + only to the token type and token string as the spacing between tokens + (column positions) may change. It returns bytes, encoded using the :data:`~token.ENCODING` token, which is the first token sequence output by :func:`.tokenize`. If there is no

1 0

gh-129020: Remove ambiguous sentence from `tokenize.untokenize` docs (#129021)
by AA-Turner Jan. 20, 2025

Jan. 20, 2025

https://github.com/python/cpython/commit/bca35f0e782848ae2acdcfbfb000cd4a2a… commit: bca35f0e782848ae2acdcfbfb000cd4a2af49fbd branch: main author: Tomas R. <tomas.roun8(a)gmail.com> committer: AA-Turner <9087854+AA-Turner(a)users.noreply.github.com> date: 2025-01-20T00:05:22Z summary: gh-129020: Remove ambiguous sentence from `tokenize.untokenize` docs (#129021) files: M Doc/library/tokenize.rst diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst index f719319a302a23..b80917eae66f8b 100644 --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -91,11 +91,10 @@ write back the modified script. sequences with at least two elements, the token type and the token string. Any additional sequence elements are ignored. - The reconstructed script is returned as a single string. The result is - guaranteed to tokenize back to match the input so that the conversion is - lossless and round-trips are assured. The guarantee applies only to the - token type and token string as the spacing between tokens (column - positions) may change. + The result is guaranteed to tokenize back to match the input so that the + conversion is lossless and round-trips are assured. The guarantee applies + only to the token type and token string as the spacing between tokens + (column positions) may change. It returns bytes, encoded using the :data:`~token.ENCODING` token, which is the first token sequence output by :func:`.tokenize`. If there is no

1 0

gh-118761: Redudce the import time of ``optparse`` (#128899)
by AA-Turner Jan. 20, 2025

Jan. 20, 2025

https://github.com/python/cpython/commit/3b6a27c5607d2b199f127c2a5ef5316bbc… commit: 3b6a27c5607d2b199f127c2a5ef5316bbc30ae42 branch: main author: Eli Schwartz <eschwartz(a)gentoo.org> committer: AA-Turner <9087854+AA-Turner(a)users.noreply.github.com> date: 2025-01-20T00:03:19Z summary: gh-118761: Redudce the import time of ``optparse`` (#128899) The same change was made, and for the same reason, by ``argparse`` back in 2017. The ``textwrap`` module is only used when printing help text, so most invocations will never need it imported. Co-authored-by: Adam Turner <9087854+AA-Turner(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2025-01-15-18-54-48.gh-issue-118761.G1dv6E.rst M Lib/optparse.py diff --git a/Lib/optparse.py b/Lib/optparse.py index cbe3451ced8bc3..38cf16d21efffa 100644 --- a/Lib/optparse.py +++ b/Lib/optparse.py @@ -74,7 +74,6 @@ """ import sys, os -import textwrap from gettext import gettext as _, ngettext @@ -252,6 +251,7 @@ def _format_text(self, text): Format a paragraph of free-form text for inclusion in the help output at the current indentation level. """ + import textwrap text_width = max(self.width - self.current_indent, 11) indent = " "*self.current_indent return textwrap.fill(text, @@ -308,6 +308,7 @@ def format_option(self, option): indent_first = 0 result.append(opts) if option.help: + import textwrap help_text = self.expand_default(option) help_lines = textwrap.wrap(help_text, self.help_width) result.append("%*s%s\n" % (indent_first, "", help_lines[0])) diff --git a/Misc/NEWS.d/next/Library/2025-01-15-18-54-48.gh-issue-118761.G1dv6E.rst b/Misc/NEWS.d/next/Library/2025-01-15-18-54-48.gh-issue-118761.G1dv6E.rst new file mode 100644 index 00000000000000..4144ef8f40e6dd --- /dev/null +++ b/Misc/NEWS.d/next/Library/2025-01-15-18-54-48.gh-issue-118761.G1dv6E.rst @@ -0,0 +1,2 @@ +Reduce the import time of :mod:`optparse` when no help text is printed. +Patch by Eli Schwartz.

1 0

gh-118761: Improve the import time of ``gettext`` (#128898)
by AA-Turner Jan. 20, 2025

Jan. 20, 2025

https://github.com/python/cpython/commit/c9c9fcb8fcc3ef43e1d8bd71ae0ed3d423… commit: c9c9fcb8fcc3ef43e1d8bd71ae0ed3d4231a6013 branch: main author: Eli Schwartz <eschwartz(a)gentoo.org> committer: AA-Turner <9087854+AA-Turner(a)users.noreply.github.com> date: 2025-01-20T00:01:20Z summary: gh-118761: Improve the import time of ``gettext`` (#128898) ``gettext`` is often imported in programs that may not end up translating anything. In fact, the ``struct`` module already has a delayed import when parsing ``GNUTranslations`` to speed up the no ``.mo`` files case. The re module is also used in the same situation, but behind a function chain only called by ``GNUTranslations``. Cache the compiled regex globally the first time it is used. The finditer function is converted to a method call on the compiled object which is slightly more efficient, and necessary for the delayed re import. files: A Misc/NEWS.d/next/Library/2025-01-15-19-16-50.gh-issue-118761.cbW2ZL.rst M Lib/gettext.py diff --git a/Lib/gettext.py b/Lib/gettext.py index a0d81cf846a05c..4c1f9427459b14 100644 --- a/Lib/gettext.py +++ b/Lib/gettext.py @@ -48,7 +48,6 @@ import operator import os -import re import sys @@ -70,22 +69,26 @@ # https://www.gnu.org/software/gettext/manual/gettext.html#Plural-forms # http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-runtime/intl/plur… -_token_pattern = re.compile(r""" - (?P<WHITESPACES>[ \t]+) | # spaces and horizontal tabs - (?P<NUMBER>[0-9]+\b) | # decimal integer - (?P<NAME>n\b) | # only n is allowed - (?P<PARENTHESIS>[()]) | - (?P<OPERATOR>[-*/%+?:]|[><!]=?|==|&&|\|\|) | # !, *, /, %, +, -, <, >, - # <=, >=, ==, !=, &&, ||, - # ? : - # unary and bitwise ops - # not allowed - (?P<INVALID>\w+|.) # invalid token - """, re.VERBOSE|re.DOTALL) - +_token_pattern = None def _tokenize(plural): - for mo in re.finditer(_token_pattern, plural): + global _token_pattern + if _token_pattern is None: + import re + _token_pattern = re.compile(r""" + (?P<WHITESPACES>[ \t]+) | # spaces and horizontal tabs + (?P<NUMBER>[0-9]+\b) | # decimal integer + (?P<NAME>n\b) | # only n is allowed + (?P<PARENTHESIS>[()]) | + (?P<OPERATOR>[-*/%+?:]|[><!]=?|==|&&|\|\|) | # !, *, /, %, +, -, <, >, + # <=, >=, ==, !=, &&, ||, + # ? : + # unary and bitwise ops + # not allowed + (?P<INVALID>\w+|.) # invalid token + """, re.VERBOSE|re.DOTALL) + + for mo in _token_pattern.finditer(plural): kind = mo.lastgroup if kind == 'WHITESPACES': continue diff --git a/Misc/NEWS.d/next/Library/2025-01-15-19-16-50.gh-issue-118761.cbW2ZL.rst b/Misc/NEWS.d/next/Library/2025-01-15-19-16-50.gh-issue-118761.cbW2ZL.rst new file mode 100644 index 00000000000000..0eef8777512dd8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2025-01-15-19-16-50.gh-issue-118761.cbW2ZL.rst @@ -0,0 +1,3 @@ +Reduce import time of :mod:`gettext` by up to ten times, by importing +:mod:`re` on demand. In particular, ``re`` is no longer implicitly +exposed as ``gettext.re``. Patch by Eli Schwartz.

1 0

Complete doc of pending C API removals in Python 3.15 (#129032)
by vstinner Jan. 19, 2025

Jan. 19, 2025

https://github.com/python/cpython/commit/bbeb219354764aef85e660be6570f0f329… commit: bbeb219354764aef85e660be6570f0f329e7227e branch: main author: Victor Stinner <vstinner(a)python.org> committer: vstinner <vstinner(a)python.org> date: 2025-01-19T22:47:05+01:00 summary: Complete doc of pending C API removals in Python 3.15 (#129032) files: M Doc/c-api/init.rst M Doc/deprecations/c-api-pending-removal-in-3.15.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 97996a6f69dd22..dc44f3eaf87765 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -109,7 +109,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-b` option. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_DebugFlag @@ -123,7 +123,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-d` option and the :envvar:`PYTHONDEBUG` environment variable. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_DontWriteBytecodeFlag @@ -137,7 +137,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-B` option and the :envvar:`PYTHONDONTWRITEBYTECODE` environment variable. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_FrozenFlag @@ -150,7 +150,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Private flag used by ``_freeze_module`` and ``frozenmain`` programs. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_HashRandomizationFlag @@ -165,7 +165,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. If the flag is non-zero, read the :envvar:`PYTHONHASHSEED` environment variable to initialize the secret hash seed. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_IgnoreEnvironmentFlag @@ -178,7 +178,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-E` and :option:`-I` options. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_InspectFlag @@ -193,7 +193,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-i` option and the :envvar:`PYTHONINSPECT` environment variable. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_InteractiveFlag @@ -218,7 +218,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. .. versionadded:: 3.4 - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_LegacyWindowsFSEncodingFlag @@ -237,7 +237,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. .. availability:: Windows. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_LegacyWindowsStdioFlag @@ -255,7 +255,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. .. availability:: Windows. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_NoSiteFlag @@ -270,7 +270,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-S` option. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_NoUserSiteDirectory @@ -284,7 +284,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-s` and :option:`-I` options, and the :envvar:`PYTHONNOUSERSITE` environment variable. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_OptimizeFlag @@ -295,7 +295,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-O` option and the :envvar:`PYTHONOPTIMIZE` environment variable. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_QuietFlag @@ -309,7 +309,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. .. versionadded:: 3.2 - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_UnbufferedStdioFlag @@ -322,7 +322,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-u` option and the :envvar:`PYTHONUNBUFFERED` environment variable. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 .. c:var:: int Py_VerboseFlag @@ -338,7 +338,7 @@ to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. Set by the :option:`-v` option and the :envvar:`PYTHONVERBOSE` environment variable. - .. deprecated-removed:: 3.12 3.14 + .. deprecated-removed:: 3.12 3.15 Initializing and finalizing the interpreter @@ -606,7 +606,7 @@ Process-wide parameters Use :c:func:`Py_DecodeLocale` to decode a bytes string to get a :c:expr:`wchar_*` string. - .. deprecated:: 3.11 + .. deprecated-removed:: 3.11 3.15 .. c:function:: wchar_t* Py_GetProgramName() @@ -868,7 +868,7 @@ Process-wide parameters .. XXX impl. doesn't seem consistent in allowing ``0``/``NULL`` for the params; check w/ Guido. - .. deprecated:: 3.11 + .. deprecated-removed:: 3.11 3.15 .. c:function:: void PySys_SetArgv(int argc, wchar_t **argv) @@ -889,7 +889,7 @@ Process-wide parameters .. versionchanged:: 3.4 The *updatepath* value depends on :option:`-I`. - .. deprecated:: 3.11 + .. deprecated-removed:: 3.11 3.15 .. c:function:: void Py_SetPythonHome(const wchar_t *home) @@ -910,7 +910,7 @@ Process-wide parameters Use :c:func:`Py_DecodeLocale` to decode a bytes string to get a :c:expr:`wchar_*` string. - .. deprecated:: 3.11 + .. deprecated-removed:: 3.11 3.15 .. c:function:: wchar_t* Py_GetPythonHome() diff --git a/Doc/deprecations/c-api-pending-removal-in-3.15.rst b/Doc/deprecations/c-api-pending-removal-in-3.15.rst index ba908f81b7f800..ac31b3cc8cd451 100644 --- a/Doc/deprecations/c-api-pending-removal-in-3.15.rst +++ b/Doc/deprecations/c-api-pending-removal-in-3.15.rst @@ -5,7 +5,9 @@ Pending removal in Python 3.15 * The :c:func:`PyImport_ImportModuleNoBlock`: Use :c:func:`PyImport_ImportModule` instead. * :c:func:`PyWeakref_GetObject` and :c:func:`PyWeakref_GET_OBJECT`: - Use :c:func:`PyWeakref_GetRef` instead. + Use :c:func:`PyWeakref_GetRef` instead. The `pythoncapi-compat project + <https://github.com/python/pythoncapi-compat/>`__ can be used to get + :c:func:`PyWeakref_GetRef` on Python 3.12 and older. * :c:type:`Py_UNICODE` type and the :c:macro:`!Py_UNICODE_WIDE` macro: Use :c:type:`wchar_t` instead. * Python initialization functions: @@ -23,9 +25,11 @@ Pending removal in Python 3.15 * :c:func:`Py_GetProgramName`: Get :data:`sys.executable` instead. * :c:func:`Py_GetPythonHome`: - Get :c:member:`PyConfig.home` + Get :c:func:`PyConfig_Get("home") <PyConfig_Get>` or the :envvar:`PYTHONHOME` environment variable instead. + See also the :c:func:`PyConfig_Get` function. + * Functions to configure Python's initialization, deprecated in Python 3.11: * :c:func:`!PySys_SetArgvEx()`:

1 0

[3.12] gh-80222: Fix email address header folding with long quoted-string (GH-122753) (#129008)
by bitdancer Jan. 19, 2025

Jan. 19, 2025

https://github.com/python/cpython/commit/b8170e58849a7728ebc9b58e222aa5bd05… commit: b8170e58849a7728ebc9b58e222aa5bd051cf7c4 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: bitdancer <rdmurray(a)bitdance.com> date: 2025-01-19T16:07:39-05:00 summary: [3.12] gh-80222: Fix email address header folding with long quoted-string (GH-122753) (#129008) gh-80222: Fix email address header folding with long quoted-string (GH-122753) Email generators using email.policy.default could incorrectly omit the quote ('"') characters from a quoted-string during header refolding, leading to invalid address headers and enabling header spoofing. This change restores the quote characters on a bare-quoted-string as the header is refolded, and escapes backslash and quote chars in the string. (cherry picked from commit 5aaf41685834901e4ed0a40f4c055b92991a0bb5) Co-authored-by: Mike Edmunds <medmunds(a)gmail.com> files: A Misc/NEWS.d/next/Security/2024-08-06-11-43-08.gh-issue-80222.wfR4BU.rst M Lib/email/_header_value_parser.py M Lib/test/test_email/test__header_value_parser.py diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py index ec2215a5e5f33c..3d845c09d415f6 100644 --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -95,8 +95,16 @@ NLSET = {'\n', '\r'} SPECIALSNL = SPECIALS | NLSET + +def make_quoted_pairs(value): + """Escape dquote and backslash for use within a quoted-string.""" + return str(value).replace('\\', '\\\\').replace('"', '\\"') + + def quote_string(value): - return '"'+str(value).replace('\\', '\\\\').replace('"', r'\"')+'"' + escaped = make_quoted_pairs(value) + return f'"{escaped}"' + # Match a RFC 2047 word, looks like =?utf-8?q?someword?= rfc2047_matcher = re.compile(r''' @@ -2905,6 +2913,15 @@ def _refold_parse_tree(parse_tree, *, policy): if not hasattr(part, 'encode'): # It's not a terminal, try folding the subparts. newparts = list(part) + if part.token_type == 'bare-quoted-string': + # To fold a quoted string we need to create a list of terminal + # tokens that will render the leading and trailing quotes + # and use quoted pairs in the value as appropriate. + newparts = ( + [ValueTerminal('"', 'ptext')] + + [ValueTerminal(make_quoted_pairs(p), 'ptext') + for p in newparts] + + [ValueTerminal('"', 'ptext')]) if not part.as_ew_allowed: wrap_as_ew_blocked += 1 newparts.append(end_ew_not_allowed) diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py index 5413319a414a62..efd1695711d7c1 100644 --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -3082,13 +3082,40 @@ def test_address_list_with_list_separator_after_fold(self): self._test(parser.get_address_list(to)[0], f'{a},\n =?utf-8?q?H=C3=BCbsch?= Kaktus <beautiful(a)example.com>\n') - a = '.' * 79 + a = '.' * 79 # ('.' is a special, so must be in quoted-string.) to = f'"{a}" <xyz(a)example.com>, "Hübsch Kaktus" <beautiful(a)example.com>' self._test(parser.get_address_list(to)[0], - f'{a}\n' + f'"{a}"\n' ' <xyz(a)example.com>, =?utf-8?q?H=C3=BCbsch?= Kaktus ' '<beautiful(a)example.com>\n') + def test_address_list_with_specials_in_long_quoted_string(self): + # Regression for gh-80222. + policy = self.policy.clone(max_line_length=40) + cases = [ + # (to, folded) + ('"Exfiltrator <spy(a)example.org> (unclosed comment?" <to(a)example.com>', + '"Exfiltrator <spy(a)example.org> (unclosed\n' + ' comment?" <to(a)example.com>\n'), + ('"Escaped \\" chars \\\\ in quoted-string stay escaped" <to(a)example.com>', + '"Escaped \\" chars \\\\ in quoted-string\n' + ' stay escaped" <to(a)example.com>\n'), + ('This long display name does not need quotes <to(a)example.com>', + 'This long display name does not need\n' + ' quotes <to(a)example.com>\n'), + ('"Quotes are not required but are retained here" <to(a)example.com>', + '"Quotes are not required but are\n' + ' retained here" <to(a)example.com>\n'), + ('"A quoted-string, it can be a valid local-part"@example.com', + '"A quoted-string, it can be a valid\n' + ' local-part"@example.com\n'), + ('"local-part-with-specials(a)but-no-fws.cannot-fold"@example.com', + '"local-part-with-specials(a)but-no-fws.cannot-fold"@example.com\n'), + ] + for (to, folded) in cases: + with self.subTest(to=to): + self._test(parser.get_address_list(to)[0], folded, policy=policy) + # XXX Need tests with comments on various sides of a unicode token, # and with unicode tokens in the comments. Spaces inside the quotes # currently don't do the right thing. diff --git a/Misc/NEWS.d/next/Security/2024-08-06-11-43-08.gh-issue-80222.wfR4BU.rst b/Misc/NEWS.d/next/Security/2024-08-06-11-43-08.gh-issue-80222.wfR4BU.rst new file mode 100644 index 00000000000000..0f0661d0b1cf4a --- /dev/null +++ b/Misc/NEWS.d/next/Security/2024-08-06-11-43-08.gh-issue-80222.wfR4BU.rst @@ -0,0 +1,6 @@ +Fix bug in the folding of quoted strings when flattening an email message using +a modern email policy. Previously when a quoted string was folded so that +it spanned more than one line, the surrounding quotes and internal escapes +would be omitted. This could theoretically be used to spoof header lines +using a carefully constructed quoted string if the resulting rendered email +was transmitted or re-parsed.

1 0

[3.13] gh-80222: Fix email address header folding with long quoted-string (GH-122753) (#129007)
by bitdancer Jan. 19, 2025

Jan. 19, 2025

https://github.com/python/cpython/commit/21200895478b8076fc0bda6ee2c26f0d87… commit: 21200895478b8076fc0bda6ee2c26f0d87c9404b branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: bitdancer <rdmurray(a)bitdance.com> date: 2025-01-19T16:06:28-05:00 summary: [3.13] gh-80222: Fix email address header folding with long quoted-string (GH-122753) (#129007) gh-80222: Fix email address header folding with long quoted-string (GH-122753) Email generators using email.policy.default could incorrectly omit the quote ('"') characters from a quoted-string during header refolding, leading to invalid address headers and enabling header spoofing. This change restores the quote characters on a bare-quoted-string as the header is refolded, and escapes backslash and quote chars in the string. (cherry picked from commit 5aaf41685834901e4ed0a40f4c055b92991a0bb5) Co-authored-by: Mike Edmunds <medmunds(a)gmail.com> files: A Misc/NEWS.d/next/Security/2024-08-06-11-43-08.gh-issue-80222.wfR4BU.rst M Lib/email/_header_value_parser.py M Lib/test/test_email/test__header_value_parser.py diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py index ec2215a5e5f33c..3d845c09d415f6 100644 --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -95,8 +95,16 @@ NLSET = {'\n', '\r'} SPECIALSNL = SPECIALS | NLSET + +def make_quoted_pairs(value): + """Escape dquote and backslash for use within a quoted-string.""" + return str(value).replace('\\', '\\\\').replace('"', '\\"') + + def quote_string(value): - return '"'+str(value).replace('\\', '\\\\').replace('"', r'\"')+'"' + escaped = make_quoted_pairs(value) + return f'"{escaped}"' + # Match a RFC 2047 word, looks like =?utf-8?q?someword?= rfc2047_matcher = re.compile(r''' @@ -2905,6 +2913,15 @@ def _refold_parse_tree(parse_tree, *, policy): if not hasattr(part, 'encode'): # It's not a terminal, try folding the subparts. newparts = list(part) + if part.token_type == 'bare-quoted-string': + # To fold a quoted string we need to create a list of terminal + # tokens that will render the leading and trailing quotes + # and use quoted pairs in the value as appropriate. + newparts = ( + [ValueTerminal('"', 'ptext')] + + [ValueTerminal(make_quoted_pairs(p), 'ptext') + for p in newparts] + + [ValueTerminal('"', 'ptext')]) if not part.as_ew_allowed: wrap_as_ew_blocked += 1 newparts.append(end_ew_not_allowed) diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py index 5413319a414a62..efd1695711d7c1 100644 --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -3082,13 +3082,40 @@ def test_address_list_with_list_separator_after_fold(self): self._test(parser.get_address_list(to)[0], f'{a},\n =?utf-8?q?H=C3=BCbsch?= Kaktus <beautiful(a)example.com>\n') - a = '.' * 79 + a = '.' * 79 # ('.' is a special, so must be in quoted-string.) to = f'"{a}" <xyz(a)example.com>, "Hübsch Kaktus" <beautiful(a)example.com>' self._test(parser.get_address_list(to)[0], - f'{a}\n' + f'"{a}"\n' ' <xyz(a)example.com>, =?utf-8?q?H=C3=BCbsch?= Kaktus ' '<beautiful(a)example.com>\n') + def test_address_list_with_specials_in_long_quoted_string(self): + # Regression for gh-80222. + policy = self.policy.clone(max_line_length=40) + cases = [ + # (to, folded) + ('"Exfiltrator <spy(a)example.org> (unclosed comment?" <to(a)example.com>', + '"Exfiltrator <spy(a)example.org> (unclosed\n' + ' comment?" <to(a)example.com>\n'), + ('"Escaped \\" chars \\\\ in quoted-string stay escaped" <to(a)example.com>', + '"Escaped \\" chars \\\\ in quoted-string\n' + ' stay escaped" <to(a)example.com>\n'), + ('This long display name does not need quotes <to(a)example.com>', + 'This long display name does not need\n' + ' quotes <to(a)example.com>\n'), + ('"Quotes are not required but are retained here" <to(a)example.com>', + '"Quotes are not required but are\n' + ' retained here" <to(a)example.com>\n'), + ('"A quoted-string, it can be a valid local-part"@example.com', + '"A quoted-string, it can be a valid\n' + ' local-part"@example.com\n'), + ('"local-part-with-specials(a)but-no-fws.cannot-fold"@example.com', + '"local-part-with-specials(a)but-no-fws.cannot-fold"@example.com\n'), + ] + for (to, folded) in cases: + with self.subTest(to=to): + self._test(parser.get_address_list(to)[0], folded, policy=policy) + # XXX Need tests with comments on various sides of a unicode token, # and with unicode tokens in the comments. Spaces inside the quotes # currently don't do the right thing. diff --git a/Misc/NEWS.d/next/Security/2024-08-06-11-43-08.gh-issue-80222.wfR4BU.rst b/Misc/NEWS.d/next/Security/2024-08-06-11-43-08.gh-issue-80222.wfR4BU.rst new file mode 100644 index 00000000000000..0f0661d0b1cf4a --- /dev/null +++ b/Misc/NEWS.d/next/Security/2024-08-06-11-43-08.gh-issue-80222.wfR4BU.rst @@ -0,0 +1,6 @@ +Fix bug in the folding of quoted strings when flattening an email message using +a modern email policy. Previously when a quoted string was folded so that +it spanned more than one line, the surrounding quotes and internal escapes +would be omitted. This could theoretically be used to spoof header lines +using a carefully constructed quoted string if the resulting rendered email +was transmitted or re-parsed.

1 0

gh-77782: Postpone Py_VerboseFlag removal to Python 3.15 (#129024)
by vstinner Jan. 19, 2025

Jan. 19, 2025

https://github.com/python/cpython/commit/d57b2d83dd470c5d3a391f4388e4bb41c9… commit: d57b2d83dd470c5d3a391f4388e4bb41c9843fad branch: main author: Victor Stinner <vstinner(a)python.org> committer: vstinner <vstinner(a)python.org> date: 2025-01-19T20:51:17+01:00 summary: gh-77782: Postpone Py_VerboseFlag removal to Python 3.15 (#129024) Postpone the global configuration variables removal to Python 3.15. Do the same for PySys_SetArgvEx(), PySys_SetArgv(), Py_SetProgramName() and Py_SetPythonHome(). files: M Doc/deprecations/c-api-pending-removal-in-3.14.rst M Doc/deprecations/c-api-pending-removal-in-3.15.rst diff --git a/Doc/deprecations/c-api-pending-removal-in-3.14.rst b/Doc/deprecations/c-api-pending-removal-in-3.14.rst index 9e10bf2691e5c8..c805074669811a 100644 --- a/Doc/deprecations/c-api-pending-removal-in-3.14.rst +++ b/Doc/deprecations/c-api-pending-removal-in-3.14.rst @@ -6,67 +6,3 @@ Pending removal in Python 3.14 * Creating :c:data:`immutable types <Py_TPFLAGS_IMMUTABLETYPE>` with mutable bases (:gh:`95388`). - -* Functions to configure Python's initialization, deprecated in Python 3.11: - - * :c:func:`!PySys_SetArgvEx()`: - Set :c:member:`PyConfig.argv` instead. - * :c:func:`!PySys_SetArgv()`: - Set :c:member:`PyConfig.argv` instead. - * :c:func:`!Py_SetProgramName()`: - Set :c:member:`PyConfig.program_name` instead. - * :c:func:`!Py_SetPythonHome()`: - Set :c:member:`PyConfig.home` instead. - - The :c:func:`Py_InitializeFromConfig` API should be used with - :c:type:`PyConfig` instead. - -* Global configuration variables: - - * :c:var:`Py_DebugFlag`: - Use :c:member:`PyConfig.parser_debug` instead. - * :c:var:`Py_VerboseFlag`: - Use :c:member:`PyConfig.verbose` instead. - * :c:var:`Py_QuietFlag`: - Use :c:member:`PyConfig.quiet` instead. - * :c:var:`Py_InteractiveFlag`: - Use :c:member:`PyConfig.interactive` instead. - * :c:var:`Py_InspectFlag`: - Use :c:member:`PyConfig.inspect` instead. - * :c:var:`Py_OptimizeFlag`: - Use :c:member:`PyConfig.optimization_level` instead. - * :c:var:`Py_NoSiteFlag`: - Use :c:member:`PyConfig.site_import` instead. - * :c:var:`Py_BytesWarningFlag`: - Use :c:member:`PyConfig.bytes_warning` instead. - * :c:var:`Py_FrozenFlag`: - Use :c:member:`PyConfig.pathconfig_warnings` instead. - * :c:var:`Py_IgnoreEnvironmentFlag`: - Use :c:member:`PyConfig.use_environment` instead. - * :c:var:`Py_DontWriteBytecodeFlag`: - Use :c:member:`PyConfig.write_bytecode` instead. - * :c:var:`Py_NoUserSiteDirectory`: - Use :c:member:`PyConfig.user_site_directory` instead. - * :c:var:`Py_UnbufferedStdioFlag`: - Use :c:member:`PyConfig.buffered_stdio` instead. - * :c:var:`Py_HashRandomizationFlag`: - Use :c:member:`PyConfig.use_hash_seed` - and :c:member:`PyConfig.hash_seed` instead. - * :c:var:`Py_IsolatedFlag`: - Use :c:member:`PyConfig.isolated` instead. - * :c:var:`Py_LegacyWindowsFSEncodingFlag`: - Use :c:member:`PyPreConfig.legacy_windows_fs_encoding` instead. - * :c:var:`Py_LegacyWindowsStdioFlag`: - Use :c:member:`PyConfig.legacy_windows_stdio` instead. - * :c:var:`!Py_FileSystemDefaultEncoding`: - Use :c:member:`PyConfig.filesystem_encoding` instead. - * :c:var:`!Py_HasFileSystemDefaultEncoding`: - Use :c:member:`PyConfig.filesystem_encoding` instead. - * :c:var:`!Py_FileSystemDefaultEncodeErrors`: - Use :c:member:`PyConfig.filesystem_errors` instead. - * :c:var:`!Py_UTF8Mode`: - Use :c:member:`PyPreConfig.utf8_mode` instead. - (see :c:func:`Py_PreInitialize`) - - The :c:func:`Py_InitializeFromConfig` API should be used with - :c:type:`PyConfig` instead. diff --git a/Doc/deprecations/c-api-pending-removal-in-3.15.rst b/Doc/deprecations/c-api-pending-removal-in-3.15.rst index 0ce0f9c118c094..ba908f81b7f800 100644 --- a/Doc/deprecations/c-api-pending-removal-in-3.15.rst +++ b/Doc/deprecations/c-api-pending-removal-in-3.15.rst @@ -25,3 +25,86 @@ Pending removal in Python 3.15 * :c:func:`Py_GetPythonHome`: Get :c:member:`PyConfig.home` or the :envvar:`PYTHONHOME` environment variable instead. + +* Functions to configure Python's initialization, deprecated in Python 3.11: + + * :c:func:`!PySys_SetArgvEx()`: + Set :c:member:`PyConfig.argv` instead. + * :c:func:`!PySys_SetArgv()`: + Set :c:member:`PyConfig.argv` instead. + * :c:func:`!Py_SetProgramName()`: + Set :c:member:`PyConfig.program_name` instead. + * :c:func:`!Py_SetPythonHome()`: + Set :c:member:`PyConfig.home` instead. + + The :c:func:`Py_InitializeFromConfig` API should be used with + :c:type:`PyConfig` instead. + +* Global configuration variables: + + * :c:var:`Py_DebugFlag`: + Use :c:member:`PyConfig.parser_debug` or + :c:func:`PyConfig_Get("parser_debug") <PyConfig_Get>` instead. + * :c:var:`Py_VerboseFlag`: + Use :c:member:`PyConfig.verbose` or + :c:func:`PyConfig_Get("verbose") <PyConfig_Get>` instead. + * :c:var:`Py_QuietFlag`: + Use :c:member:`PyConfig.quiet` or + :c:func:`PyConfig_Get("quiet") <PyConfig_Get>` instead. + * :c:var:`Py_InteractiveFlag`: + Use :c:member:`PyConfig.interactive` or + :c:func:`PyConfig_Get("interactive") <PyConfig_Get>` instead. + * :c:var:`Py_InspectFlag`: + Use :c:member:`PyConfig.inspect` or + :c:func:`PyConfig_Get("inspect") <PyConfig_Get>` instead. + * :c:var:`Py_OptimizeFlag`: + Use :c:member:`PyConfig.optimization_level` or + :c:func:`PyConfig_Get("optimization_level") <PyConfig_Get>` instead. + * :c:var:`Py_NoSiteFlag`: + Use :c:member:`PyConfig.site_import` or + :c:func:`PyConfig_Get("site_import") <PyConfig_Get>` instead. + * :c:var:`Py_BytesWarningFlag`: + Use :c:member:`PyConfig.bytes_warning` or + :c:func:`PyConfig_Get("bytes_warning") <PyConfig_Get>` instead. + * :c:var:`Py_FrozenFlag`: + Use :c:member:`PyConfig.pathconfig_warnings` or + :c:func:`PyConfig_Get("pathconfig_warnings") <PyConfig_Get>` instead. + * :c:var:`Py_IgnoreEnvironmentFlag`: + Use :c:member:`PyConfig.use_environment` or + :c:func:`PyConfig_Get("use_environment") <PyConfig_Get>` instead. + * :c:var:`Py_DontWriteBytecodeFlag`: + Use :c:member:`PyConfig.write_bytecode` or + :c:func:`PyConfig_Get("write_bytecode") <PyConfig_Get>` instead. + * :c:var:`Py_NoUserSiteDirectory`: + Use :c:member:`PyConfig.user_site_directory` or + :c:func:`PyConfig_Get("user_site_directory") <PyConfig_Get>` instead. + * :c:var:`Py_UnbufferedStdioFlag`: + Use :c:member:`PyConfig.buffered_stdio` or + :c:func:`PyConfig_Get("buffered_stdio") <PyConfig_Get>` instead. + * :c:var:`Py_HashRandomizationFlag`: + Use :c:member:`PyConfig.use_hash_seed` + and :c:member:`PyConfig.hash_seed` or + :c:func:`PyConfig_Get("hash_seed") <PyConfig_Get>` instead. + * :c:var:`Py_IsolatedFlag`: + Use :c:member:`PyConfig.isolated` or + :c:func:`PyConfig_Get("isolated") <PyConfig_Get>` instead. + * :c:var:`Py_LegacyWindowsFSEncodingFlag`: + Use :c:member:`PyPreConfig.legacy_windows_fs_encoding` or + :c:func:`PyConfig_Get("legacy_windows_fs_encoding") <PyConfig_Get>` instead. + * :c:var:`Py_LegacyWindowsStdioFlag`: + Use :c:member:`PyConfig.legacy_windows_stdio` or + :c:func:`PyConfig_Get("legacy_windows_stdio") <PyConfig_Get>` instead. + * :c:var:`!Py_FileSystemDefaultEncoding`, :c:var:`!Py_HasFileSystemDefaultEncoding`: + Use :c:member:`PyConfig.filesystem_encoding` or + :c:func:`PyConfig_Get("filesystem_encoding") <PyConfig_Get>` instead. + * :c:var:`!Py_FileSystemDefaultEncodeErrors`: + Use :c:member:`PyConfig.filesystem_errors` or + :c:func:`PyConfig_Get("filesystem_errors") <PyConfig_Get>` instead. + * :c:var:`!Py_UTF8Mode`: + Use :c:member:`PyPreConfig.utf8_mode` or + :c:func:`PyConfig_Get("utf8_mode") <PyConfig_Get>` instead. + (see :c:func:`Py_PreInitialize`) + + The :c:func:`Py_InitializeFromConfig` API should be used with + :c:type:`PyConfig` to set these options. Or :c:func:`PyConfig_Get` can be + used to get these options at runtime.

1 0

[3.12] gh-111495: Add more tests on PyEval C APIs (#122789) (#128987) (#129023)
by vstinner Jan. 19, 2025

Jan. 19, 2025

https://github.com/python/cpython/commit/fceb8c31ddf8db2fddd92c8c6437094081… commit: fceb8c31ddf8db2fddd92c8c6437094081c91ac8 branch: 3.12 author: Victor Stinner <vstinner(a)python.org> committer: vstinner <vstinner(a)python.org> date: 2025-01-19T13:51:53Z summary: [3.12] gh-111495: Add more tests on PyEval C APIs (#122789) (#128987) (#129023) * Add Lib/test/test_capi/test_eval.py * Add Modules/_testlimitedcapi/eval.c (cherry picked from commit bf8b3746398ea756c97e3cf263d63ca3ce3a544e) * gh-111495: Fix refleaks in test_capi.test_eval tests (#122851) (cherry picked from commit b4a316087c32d83e375087fd35fc511bc430ee8b) (cherry picked from commit 430ccbc009aa7a2da92b85d7aeadd39e1666e875) files: A Lib/test/test_capi/test_eval.py A Modules/_testcapi/eval.c M Lib/test/test_capi/test_misc.py M Modules/Setup.stdlib.in M Modules/_testcapi/parts.h M Modules/_testcapimodule.c M PCbuild/_testcapi.vcxproj M PCbuild/_testcapi.vcxproj.filters diff --git a/Lib/test/test_capi/test_eval.py b/Lib/test/test_capi/test_eval.py new file mode 100644 index 00000000000000..1fedf2f2b4b23b --- /dev/null +++ b/Lib/test/test_capi/test_eval.py @@ -0,0 +1,88 @@ +import sys +import unittest +from test.support import import_helper + +_testcapi = import_helper.import_module('_testcapi') + + +class Tests(unittest.TestCase): + def test_eval_get_func_name(self): + eval_get_func_name = _testcapi.eval_get_func_name + + def function_example(): ... + + class A: + def method_example(self): ... + + self.assertEqual(eval_get_func_name(function_example), + "function_example") + self.assertEqual(eval_get_func_name(A.method_example), + "method_example") + self.assertEqual(eval_get_func_name(A().method_example), + "method_example") + self.assertEqual(eval_get_func_name(sum), "sum") # c function + self.assertEqual(eval_get_func_name(A), "type") + + def test_eval_get_func_desc(self): + eval_get_func_desc = _testcapi.eval_get_func_desc + + def function_example(): ... + + class A: + def method_example(self): ... + + self.assertEqual(eval_get_func_desc(function_example), + "()") + self.assertEqual(eval_get_func_desc(A.method_example), + "()") + self.assertEqual(eval_get_func_desc(A().method_example), + "()") + self.assertEqual(eval_get_func_desc(sum), "()") # c function + self.assertEqual(eval_get_func_desc(A), " object") + + def test_eval_getlocals(self): + # Test PyEval_GetLocals() + x = 1 + self.assertEqual(_testcapi.eval_getlocals(), + {'self': self, + 'x': 1}) + + y = 2 + self.assertEqual(_testcapi.eval_getlocals(), + {'self': self, + 'x': 1, + 'y': 2}) + + def test_eval_getglobals(self): + # Test PyEval_GetGlobals() + self.assertEqual(_testcapi.eval_getglobals(), + globals()) + + def test_eval_getbuiltins(self): + # Test PyEval_GetBuiltins() + self.assertEqual(_testcapi.eval_getbuiltins(), + globals()['__builtins__']) + + def test_eval_getframe(self): + # Test PyEval_GetFrame() + self.assertEqual(_testcapi.eval_getframe(), + sys._getframe()) + + def test_eval_get_recursion_limit(self): + # Test Py_GetRecursionLimit() + self.assertEqual(_testcapi.eval_get_recursion_limit(), + sys.getrecursionlimit()) + + def test_eval_set_recursion_limit(self): + # Test Py_SetRecursionLimit() + old_limit = sys.getrecursionlimit() + try: + limit = old_limit + 123 + _testcapi.eval_set_recursion_limit(limit) + self.assertEqual(sys.getrecursionlimit(), limit) + finally: + sys.setrecursionlimit(old_limit) + + +if __name__ == "__main__": + unittest.main() diff --git a/Lib/test/test_capi/test_misc.py b/Lib/test/test_capi/test_misc.py index 98c74a44e4cb27..f284e665d6fde0 100644 --- a/Lib/test/test_capi/test_misc.py +++ b/Lib/test/test_capi/test_misc.py @@ -879,36 +879,6 @@ def __init__(self): _testcapi.clear_managed_dict(c) self.assertEqual(c.__dict__, {}) - def test_eval_get_func_name(self): - def function_example(): ... - - class A: - def method_example(self): ... - - self.assertEqual(_testcapi.eval_get_func_name(function_example), - "function_example") - self.assertEqual(_testcapi.eval_get_func_name(A.method_example), - "method_example") - self.assertEqual(_testcapi.eval_get_func_name(A().method_example), - "method_example") - self.assertEqual(_testcapi.eval_get_func_name(sum), "sum") # c function - self.assertEqual(_testcapi.eval_get_func_name(A), "type") - - def test_eval_get_func_desc(self): - def function_example(): ... - - class A: - def method_example(self): ... - - self.assertEqual(_testcapi.eval_get_func_desc(function_example), - "()") - self.assertEqual(_testcapi.eval_get_func_desc(A.method_example), - "()") - self.assertEqual(_testcapi.eval_get_func_desc(A().method_example), - "()") - self.assertEqual(_testcapi.eval_get_func_desc(sum), "()") # c function - self.assertEqual(_testcapi.eval_get_func_desc(A), " object") - def test_function_get_code(self): import types diff --git a/Modules/Setup.stdlib.in b/Modules/Setup.stdlib.in index d60afcf55dba4a..5e587dd3a5a397 100644 --- a/Modules/Setup.stdlib.in +++ b/Modules/Setup.stdlib.in @@ -168,7 +168,7 @@ @MODULE__XXTESTFUZZ_TRUE@_xxtestfuzz _xxtestfuzz/_xxtestfuzz.c _xxtestfuzz/fuzzer.c @MODULE__TESTBUFFER_TRUE@_testbuffer _testbuffer.c @MODULE__TESTINTERNALCAPI_TRUE@_testinternalcapi _testinternalcapi.c -@MODULE__TESTCAPI_TRUE@_testcapi _testcapimodule.c _testcapi/vectorcall.c _testcapi/vectorcall_limited.c _testcapi/heaptype.c _testcapi/abstract.c _testcapi/bytearray.c _testcapi/bytes.c _testcapi/unicode.c _testcapi/dict.c _testcapi/set.c _testcapi/list.c _testcapi/tuple.c _testcapi/getargs.c _testcapi/pytime.c _testcapi/datetime.c _testcapi/docstring.c _testcapi/mem.c _testcapi/watchers.c _testcapi/long.c _testcapi/float.c _testcapi/complex.c _testcapi/numbers.c _testcapi/structmember.c _testcapi/exceptions.c _testcapi/code.c _testcapi/buffer.c _testcapi/pyos.c _testcapi/run.c _testcapi/file.c _testcapi/codec.c _testcapi/immortal.c _testcapi/heaptype_relative.c _testcapi/gc.c _testcapi/sys.c _testcapi/import.c +@MODULE__TESTCAPI_TRUE@_testcapi _testcapimodule.c _testcapi/vectorcall.c _testcapi/vectorcall_limited.c _testcapi/heaptype.c _testcapi/abstract.c _testcapi/bytearray.c _testcapi/bytes.c _testcapi/unicode.c _testcapi/dict.c _testcapi/set.c _testcapi/list.c _testcapi/tuple.c _testcapi/getargs.c _testcapi/pytime.c _testcapi/datetime.c _testcapi/docstring.c _testcapi/mem.c _testcapi/watchers.c _testcapi/long.c _testcapi/float.c _testcapi/complex.c _testcapi/numbers.c _testcapi/structmember.c _testcapi/exceptions.c _testcapi/code.c _testcapi/buffer.c _testcapi/pyos.c _testcapi/run.c _testcapi/file.c _testcapi/codec.c _testcapi/immortal.c _testcapi/heaptype_relative.c _testcapi/gc.c _testcapi/sys.c _testcapi/import.c _testcapi/eval.c @MODULE__TESTCLINIC_TRUE@_testclinic _testclinic.c # Some testing modules MUST be built as shared libraries. diff --git a/Modules/_testcapi/eval.c b/Modules/_testcapi/eval.c new file mode 100644 index 00000000000000..581b5579178910 --- /dev/null +++ b/Modules/_testcapi/eval.c @@ -0,0 +1,74 @@ +#include "parts.h" +#include "util.h" + +static PyObject * +eval_get_func_name(PyObject *self, PyObject *func) +{ + return PyUnicode_FromString(PyEval_GetFuncName(func)); +} + +static PyObject * +eval_get_func_desc(PyObject *self, PyObject *func) +{ + return PyUnicode_FromString(PyEval_GetFuncDesc(func)); +} + +static PyObject * +eval_getlocals(PyObject *module, PyObject *Py_UNUSED(args)) +{ + return Py_XNewRef(PyEval_GetLocals()); +} + +static PyObject * +eval_getglobals(PyObject *module, PyObject *Py_UNUSED(args)) +{ + return Py_XNewRef(PyEval_GetGlobals()); +} + +static PyObject * +eval_getbuiltins(PyObject *module, PyObject *Py_UNUSED(args)) +{ + return Py_XNewRef(PyEval_GetBuiltins()); +} + +static PyObject * +eval_getframe(PyObject *module, PyObject *Py_UNUSED(args)) +{ + return Py_XNewRef(PyEval_GetFrame()); +} + +static PyObject * +eval_get_recursion_limit(PyObject *module, PyObject *Py_UNUSED(args)) +{ + int limit = Py_GetRecursionLimit(); + return PyLong_FromLong(limit); +} + +static PyObject * +eval_set_recursion_limit(PyObject *module, PyObject *args) +{ + int limit; + if (!PyArg_ParseTuple(args, "i", &limit)) { + return NULL; + } + Py_SetRecursionLimit(limit); + Py_RETURN_NONE; +} + +static PyMethodDef test_methods[] = { + {"eval_get_func_name", eval_get_func_name, METH_O, NULL}, + {"eval_get_func_desc", eval_get_func_desc, METH_O, NULL}, + {"eval_getlocals", eval_getlocals, METH_NOARGS}, + {"eval_getglobals", eval_getglobals, METH_NOARGS}, + {"eval_getbuiltins", eval_getbuiltins, METH_NOARGS}, + {"eval_getframe", eval_getframe, METH_NOARGS}, + {"eval_get_recursion_limit", eval_get_recursion_limit, METH_NOARGS}, + {"eval_set_recursion_limit", eval_set_recursion_limit, METH_VARARGS}, + {NULL}, +}; + +int +_PyTestCapi_Init_Eval(PyObject *m) +{ + return PyModule_AddFunctions(m, test_methods); +} diff --git a/Modules/_testcapi/parts.h b/Modules/_testcapi/parts.h index bc348dc122c525..4ca46c800aa9eb 100644 --- a/Modules/_testcapi/parts.h +++ b/Modules/_testcapi/parts.h @@ -56,6 +56,7 @@ int _PyTestCapi_Init_Immortal(PyObject *module); int _PyTestCapi_Init_GC(PyObject *module); int _PyTestCapi_Init_Sys(PyObject *module); int _PyTestCapi_Init_Import(PyObject *module); +int _PyTestCapi_Init_Eval(PyObject *module); #ifdef LIMITED_API_AVAILABLE int _PyTestCapi_Init_VectorcallLimited(PyObject *module); diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 21d1228d79fe44..1fbeb58e3d404b 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -2797,18 +2797,6 @@ test_frame_getvarstring(PyObject *self, PyObject *args) } -static PyObject * -eval_get_func_name(PyObject *self, PyObject *func) -{ - return PyUnicode_FromString(PyEval_GetFuncName(func)); -} - -static PyObject * -eval_get_func_desc(PyObject *self, PyObject *func) -{ - return PyUnicode_FromString(PyEval_GetFuncDesc(func)); -} - static PyObject * gen_get_code(PyObject *self, PyObject *gen) { @@ -3460,8 +3448,6 @@ static PyMethodDef TestMethods[] = { {"frame_new", frame_new, METH_VARARGS, NULL}, {"frame_getvar", test_frame_getvar, METH_VARARGS, NULL}, {"frame_getvarstring", test_frame_getvarstring, METH_VARARGS, NULL}, - {"eval_get_func_name", eval_get_func_name, METH_O, NULL}, - {"eval_get_func_desc", eval_get_func_desc, METH_O, NULL}, {"gen_get_code", gen_get_code, METH_O, NULL}, {"get_feature_macros", get_feature_macros, METH_NOARGS, NULL}, {"test_code_api", test_code_api, METH_NOARGS, NULL}, @@ -4174,6 +4160,9 @@ PyInit__testcapi(void) if (_PyTestCapi_Init_Import(m) < 0) { return NULL; } + if (_PyTestCapi_Init_Eval(m) < 0) { + return NULL; + } #ifndef LIMITED_API_AVAILABLE PyModule_AddObjectRef(m, "LIMITED_API_AVAILABLE", Py_False); diff --git a/PCbuild/_testcapi.vcxproj b/PCbuild/_testcapi.vcxproj index ec24c62903af90..06be7d291b17f3 100644 --- a/PCbuild/_testcapi.vcxproj +++ b/PCbuild/_testcapi.vcxproj @@ -128,6 +128,7 @@ <ClCompile Include="..\Modules\_testcapi\immortal.c" /> <ClCompile Include="..\Modules\_testcapi\gc.c" /> <ClCompile Include="..\Modules\_testcapi\run.c" /> + <ClCompile Include="..\Modules\_testcapi\eval.c" /> </ItemGroup> <ItemGroup> <ResourceCompile Include="..\PC\python_nt.rc" /> diff --git a/PCbuild/_testcapi.vcxproj.filters b/PCbuild/_testcapi.vcxproj.filters index 99eefb7b565393..309a016da2d587 100644 --- a/PCbuild/_testcapi.vcxproj.filters +++ b/PCbuild/_testcapi.vcxproj.filters @@ -111,6 +111,9 @@ <ClCompile Include="..\Modules\_testcapi\import.c"> <Filter>Source Files</Filter> </ClCompile> + <ClCompile Include="..\Modules\_testcapi\eval.c"> + <Filter>Source Files</Filter> + </ClCompile> </ItemGroup> <ItemGroup> <ResourceCompile Include="..\PC\python_nt.rc">

1 0