Mailman 3 January 2025 - Python-checkins

[3.12] gh-114990: Add missing mixin methods in collections.abc's document (GH-114991) (#128536)
by hugovk 06 Jan '25

06 Jan '25

https://github.com/python/cpython/commit/8ba72caf2644f749ad92a419e24ba95ebf… commit: 8ba72caf2644f749ad92a419e24ba95ebf8c9444 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: hugovk <1324225+hugovk(a)users.noreply.github.com> date: 2025-01-06T10:04:57Z summary: [3.12] gh-114990: Add missing mixin methods in collections.abc's document (GH-114991) (#128536) Co-authored-by: AN Long <aisk(a)users.noreply.github.com> Co-authored-by: Carol Willing <carolcode(a)willingconsulting.com> files: M Doc/library/collections.abc.rst diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst index 0adbd305b468f4..850853346991b5 100644 --- a/Doc/library/collections.abc.rst +++ b/Doc/library/collections.abc.rst @@ -146,7 +146,8 @@ ABC Inherits from Abstract Methods Mi :class:`Set` :class:`Collection` ``__contains__``, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``, ``__iter__``, ``__gt__``, ``__ge__``, ``__and__``, ``__or__``, - ``__len__`` ``__sub__``, ``__xor__``, and ``isdisjoint`` + ``__len__`` ``__sub__``, ``__rsub__``, ``__xor__``, ``__rxor__`` + and ``isdisjoint`` :class:`MutableSet` :class:`Set` ``__contains__``, Inherited :class:`Set` methods and ``__iter__``, ``clear``, ``pop``, ``remove``, ``__ior__``, @@ -165,7 +166,7 @@ ABC Inherits from Abstract Methods Mi ``__len__`` -:class:`MappingView` :class:`Sized` ``__len__`` +:class:`MappingView` :class:`Sized` ``__init__``, ``__len__`` and ``__repr__`` :class:`ItemsView` :class:`MappingView`, ``__contains__``, :class:`Set` ``__iter__`` :class:`KeysView` :class:`MappingView`, ``__contains__``,

1 0

[3.13] gh-114990: Add missing mixin methods in collections.abc's document (GH-114991) (#128535)
by hugovk 06 Jan '25

06 Jan '25

https://github.com/python/cpython/commit/aa519e4caeacd5fbfdb3fe2d48b337d2c2… commit: aa519e4caeacd5fbfdb3fe2d48b337d2c252afc9 branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: hugovk <1324225+hugovk(a)users.noreply.github.com> date: 2025-01-06T10:03:05Z summary: [3.13] gh-114990: Add missing mixin methods in collections.abc's document (GH-114991) (#128535) Co-authored-by: AN Long <aisk(a)users.noreply.github.com> Co-authored-by: Carol Willing <carolcode(a)willingconsulting.com> files: M Doc/library/collections.abc.rst diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst index 0adbd305b468f4..850853346991b5 100644 --- a/Doc/library/collections.abc.rst +++ b/Doc/library/collections.abc.rst @@ -146,7 +146,8 @@ ABC Inherits from Abstract Methods Mi :class:`Set` :class:`Collection` ``__contains__``, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``, ``__iter__``, ``__gt__``, ``__ge__``, ``__and__``, ``__or__``, - ``__len__`` ``__sub__``, ``__xor__``, and ``isdisjoint`` + ``__len__`` ``__sub__``, ``__rsub__``, ``__xor__``, ``__rxor__`` + and ``isdisjoint`` :class:`MutableSet` :class:`Set` ``__contains__``, Inherited :class:`Set` methods and ``__iter__``, ``clear``, ``pop``, ``remove``, ``__ior__``, @@ -165,7 +166,7 @@ ABC Inherits from Abstract Methods Mi ``__len__`` -:class:`MappingView` :class:`Sized` ``__len__`` +:class:`MappingView` :class:`Sized` ``__init__``, ``__len__`` and ``__repr__`` :class:`ItemsView` :class:`MappingView`, ``__contains__``, :class:`Set` ``__iter__`` :class:`KeysView` :class:`MappingView`, ``__contains__``,

1 0

[3.13] gh-128519: Align the docstring of untokenize() to match the docs (GH-128521) (#128531)
by erlend-aasland 06 Jan '25

06 Jan '25

https://github.com/python/cpython/commit/9c5a86d23ad7d0df8611520dbf2c174652… commit: 9c5a86d23ad7d0df8611520dbf2c1746523ac1be branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2025-01-06T09:05:54Z summary: [3.13] gh-128519: Align the docstring of untokenize() to match the docs (GH-128521) (#128531) (cherry picked from commit aef52ca8b334ff90e8032da39f4d06e7b5130eb9) Co-authored-by: Tomas R <tomas.roun8(a)gmail.com> files: M Lib/tokenize.py diff --git a/Lib/tokenize.py b/Lib/tokenize.py index 4b4c3cfe16999b..430447d35e1adf 100644 --- a/Lib/tokenize.py +++ b/Lib/tokenize.py @@ -318,16 +318,10 @@ def untokenize(iterable): with at least two elements, a token number and token value. If only two tokens are passed, the resulting output is poor. - Round-trip invariant for full input: - Untokenized source will match input source exactly - - Round-trip invariant for limited input: - # Output bytes will tokenize back to the input - t1 = [tok[:2] for tok in tokenize(f.readline)] - newcode = untokenize(t1) - readline = BytesIO(newcode).readline - t2 = [tok[:2] for tok in tokenize(readline)] - assert t1 == t2 + The result is guaranteed to tokenize back to match the input so + that the conversion is lossless and round-trips are assured. + The guarantee applies only to the token type and token string as + the spacing between tokens (column positions) may change. """ ut = Untokenizer() out = ut.untokenize(iterable)

1 0

[3.12] gh-128519: Align the docstring of untokenize() to match the docs (GH-128521) (#128532)
by erlend-aasland 06 Jan '25

06 Jan '25

https://github.com/python/cpython/commit/f9aac4a76d68c8eef78058b4da1252beaa… commit: f9aac4a76d68c8eef78058b4da1252beaadd1570 branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2025-01-06T08:59:24Z summary: [3.12] gh-128519: Align the docstring of untokenize() to match the docs (GH-128521) (#128532) (cherry picked from commit aef52ca8b334ff90e8032da39f4d06e7b5130eb9) Co-authored-by: Tomas R <tomas.roun8(a)gmail.com> files: M Lib/tokenize.py diff --git a/Lib/tokenize.py b/Lib/tokenize.py index b2dff8e6967094..553c1ca4388974 100644 --- a/Lib/tokenize.py +++ b/Lib/tokenize.py @@ -320,16 +320,10 @@ def untokenize(iterable): with at least two elements, a token number and token value. If only two tokens are passed, the resulting output is poor. - Round-trip invariant for full input: - Untokenized source will match input source exactly - - Round-trip invariant for limited input: - # Output bytes will tokenize back to the input - t1 = [tok[:2] for tok in tokenize(f.readline)] - newcode = untokenize(t1) - readline = BytesIO(newcode).readline - t2 = [tok[:2] for tok in tokenize(readline)] - assert t1 == t2 + The result is guaranteed to tokenize back to match the input so + that the conversion is lossless and round-trips are assured. + The guarantee applies only to the token type and token string as + the spacing between tokens (column positions) may change. """ ut = Untokenizer() out = ut.untokenize(iterable)

1 0

gh-128519: Align the docstring of untokenize() to match the docs (#128521)
by erlend-aasland 06 Jan '25

06 Jan '25

https://github.com/python/cpython/commit/aef52ca8b334ff90e8032da39f4d06e7b5… commit: aef52ca8b334ff90e8032da39f4d06e7b5130eb9 branch: main author: Tomas R. <tomas.roun8(a)gmail.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2025-01-06T08:42:26Z summary: gh-128519: Align the docstring of untokenize() to match the docs (#128521) files: M Lib/tokenize.py diff --git a/Lib/tokenize.py b/Lib/tokenize.py index 7ece4e9b70d31b..1a60fd32a77ea4 100644 --- a/Lib/tokenize.py +++ b/Lib/tokenize.py @@ -318,16 +318,10 @@ def untokenize(iterable): with at least two elements, a token number and token value. If only two tokens are passed, the resulting output is poor. - Round-trip invariant for full input: - Untokenized source will match input source exactly - - Round-trip invariant for limited input: - # Output bytes will tokenize back to the input - t1 = [tok[:2] for tok in tokenize(f.readline)] - newcode = untokenize(t1) - readline = BytesIO(newcode).readline - t2 = [tok[:2] for tok in tokenize(readline)] - assert t1 == t2 + The result is guaranteed to tokenize back to match the input so + that the conversion is lossless and round-trips are assured. + The guarantee applies only to the token type and token string as + the spacing between tokens (column positions) may change. """ ut = Untokenizer() out = ut.untokenize(iterable)

1 0

gh-98188: Fix EmailMessage.get_payload to decode data when CTE value has extra text (#127547)
by bitdancer 06 Jan '25

06 Jan '25

https://github.com/python/cpython/commit/a62ba52f1439c1f878a3ff9b8544caf9ae… commit: a62ba52f1439c1f878a3ff9b8544caf9aeef9b90 branch: main author: RanKKI <hliu86.me(a)gmail.com> committer: bitdancer <rdmurray(a)bitdance.com> date: 2025-01-05T20:32:16-05:00 summary: gh-98188: Fix EmailMessage.get_payload to decode data when CTE value has extra text (#127547) Up to this point message handling has been very strict with regards to content encoding values: mixed case was accepted, but trailing blanks or other text would cause decoding failure, even if the first token was a valid encoding. By Postel's Rule we should go ahead and decode as long as we can recognize that first token. We have not thought of any security or backward compatibility concerns with this fix. This fix does introduce a new technique/pattern to the Message code: we look to see if the header has a 'cte' attribute, and if so we use that. This effectively promotes the header API exposed by HeaderRegistry to an API that any header parser "should" support. This seems like a reasonable thing to do. It is not, however, a requirement, as the string value of the header is still used if there is no cte attribute. The full fix (ignore any trailing blanks or blank-separated trailing text) applies only to the non-compat32 API. compat32 is only fixed to the extent that it now ignores trailing spaces. Note that the HeaderRegistry parsing still records a HeaderDefect if there is extra text. Co-authored-by: Bénédikt Tran <10796600+picnixz(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2024-12-03-14-45-16.gh-issue-98188.GX9i2b.rst M Lib/email/message.py M Lib/test/test_email/test_email.py M Lib/test/test_email/test_headerregistry.py M Misc/ACKS diff --git a/Lib/email/message.py b/Lib/email/message.py index a58afc5fe5f68e..87fcab68868d46 100644 --- a/Lib/email/message.py +++ b/Lib/email/message.py @@ -286,8 +286,12 @@ def get_payload(self, i=None, decode=False): if i is not None and not isinstance(self._payload, list): raise TypeError('Expected list, got %s' % type(self._payload)) payload = self._payload - # cte might be a Header, so for now stringify it. - cte = str(self.get('content-transfer-encoding', '')).lower() + cte = self.get('content-transfer-encoding', '') + if hasattr(cte, 'cte'): + cte = cte.cte + else: + # cte might be a Header, so for now stringify it. + cte = str(cte).strip().lower() # payload may be bytes here. if not decode: if isinstance(payload, str) and utils._has_surrogates(payload): diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index abe9ef2e94409f..2deb35721576b8 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -810,6 +810,16 @@ def test_unicode_body_defaults_to_utf8_encoding(self): w4kgdGVzdGFiYwo= """)) + def test_string_payload_with_base64_cte(self): + msg = email.message_from_string(textwrap.dedent("""\ + Content-Transfer-Encoding: base64 + + SGVsbG8uIFRlc3Rpbmc= + """), policy=email.policy.default) + self.assertEqual(msg.get_payload(decode=True), b"Hello. Testing") + self.assertDefectsEqual(msg['content-transfer-encoding'].defects, []) + + # Test the email.encoders module class TestEncoders(unittest.TestCase): @@ -2352,6 +2362,40 @@ def test_missing_header_body_separator(self): self.assertDefectsEqual(msg.defects, [errors.MissingHeaderBodySeparatorDefect]) + def test_string_payload_with_extra_space_after_cte(self): + # https://github.com/python/cpython/issues/98188 + cte = "base64 " + msg = email.message_from_string(textwrap.dedent(f"""\ + Content-Transfer-Encoding: {cte} + + SGVsbG8uIFRlc3Rpbmc= + """), policy=email.policy.default) + self.assertEqual(msg.get_payload(decode=True), b"Hello. Testing") + self.assertDefectsEqual(msg['content-transfer-encoding'].defects, []) + + def test_string_payload_with_extra_text_after_cte(self): + msg = email.message_from_string(textwrap.dedent("""\ + Content-Transfer-Encoding: base64 some text + + SGVsbG8uIFRlc3Rpbmc= + """), policy=email.policy.default) + self.assertEqual(msg.get_payload(decode=True), b"Hello. Testing") + cte = msg['content-transfer-encoding'] + self.assertDefectsEqual(cte.defects, [email.errors.InvalidHeaderDefect]) + + def test_string_payload_with_extra_space_after_cte_compat32(self): + cte = "base64 " + msg = email.message_from_string(textwrap.dedent(f"""\ + Content-Transfer-Encoding: {cte} + + SGVsbG8uIFRlc3Rpbmc= + """), policy=email.policy.compat32) + pasted_cte = msg['content-transfer-encoding'] + self.assertEqual(pasted_cte, cte) + self.assertEqual(msg.get_payload(decode=True), b"Hello. Testing") + self.assertDefectsEqual(msg.defects, []) + + # Test RFC 2047 header encoding and decoding class TestRFC2047(TestEmailBase): diff --git a/Lib/test/test_email/test_headerregistry.py b/Lib/test/test_email/test_headerregistry.py index 4c0523f410332f..ff7a6da644d572 100644 --- a/Lib/test/test_email/test_headerregistry.py +++ b/Lib/test/test_email/test_headerregistry.py @@ -837,6 +837,11 @@ def cte_as_value(self, '7bit', [errors.InvalidHeaderDefect]), + 'extra_space_after_cte': ( + 'base64 ', + 'base64', + []), + } diff --git a/Misc/ACKS b/Misc/ACKS index c6e53317b37d78..d7585c16c8169c 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1129,6 +1129,7 @@ Gregor Lingl Everett Lipman Mirko Liss Alexander Liu +Hui Liu Yuan Liu Nick Lockwood Stephanie Lockwood diff --git a/Misc/NEWS.d/next/Library/2024-12-03-14-45-16.gh-issue-98188.GX9i2b.rst b/Misc/NEWS.d/next/Library/2024-12-03-14-45-16.gh-issue-98188.GX9i2b.rst new file mode 100644 index 00000000000000..30ab8cfc3f0bc6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-12-03-14-45-16.gh-issue-98188.GX9i2b.rst @@ -0,0 +1,3 @@ +Fix an issue in :meth:`email.message.Message.get_payload` where data +cannot be decoded if the Content Transfer Encoding mechanism contains +trailing whitespaces or additional junk text. Patch by Hui Liu.

1 0

[3.12] Docs: fix `MessageDefect` references in email.policy docs (GH-128468) (#128527)
by erlend-aasland 05 Jan '25

05 Jan '25

https://github.com/python/cpython/commit/4016be27755ef6d0cb8cd11e0625b97faf… commit: 4016be27755ef6d0cb8cd11e0625b97fafaccf0f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2025-01-05T22:05:48Z summary: [3.12] Docs: fix `MessageDefect` references in email.policy docs (GH-128468) (#128527) (cherry picked from commit 3b231be8f000ae59faa04d5a2f1af11beafee866) Co-authored-by: Yuki Kobayashi <drsuaimqjgar(a)gmail.com> files: M Doc/library/email.policy.rst M Doc/tools/.nitignore diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst index 51d65dc5ba94ee..152350b0b02659 100644 --- a/Doc/library/email.policy.rst +++ b/Doc/library/email.policy.rst @@ -267,7 +267,7 @@ added matters. To illustrate:: Handle a *defect* found on *obj*. When the email package calls this method, *defect* will always be a subclass of - :class:`~email.errors.Defect`. + :class:`~email.errors.MessageDefect`. The default implementation checks the :attr:`raise_on_defect` flag. If it is ``True``, *defect* is raised as an exception. If it is ``False`` @@ -277,7 +277,7 @@ added matters. To illustrate:: .. method:: register_defect(obj, defect) Register a *defect* on *obj*. In the email package, *defect* will always - be a subclass of :class:`~email.errors.Defect`. + be a subclass of :class:`~email.errors.MessageDefect`. The default implementation calls the ``append`` method of the ``defects`` attribute of *obj*. When the email package calls :attr:`handle_defect`, diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index e69e6100945672..a721b494d750eb 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -31,7 +31,6 @@ Doc/library/email.charset.rst Doc/library/email.compat32-message.rst Doc/library/email.errors.rst Doc/library/email.parser.rst -Doc/library/email.policy.rst Doc/library/exceptions.rst Doc/library/functools.rst Doc/library/getopt.rst

1 0

[3.13] Docs: fix `MessageDefect` references in email.policy docs (GH-128468) (#128526)
by erlend-aasland 05 Jan '25

05 Jan '25

https://github.com/python/cpython/commit/838e8a2718c187199668e6b868c1c6844a… commit: 838e8a2718c187199668e6b868c1c6844a419b3d branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2025-01-05T22:04:39Z summary: [3.13] Docs: fix `MessageDefect` references in email.policy docs (GH-128468) (#128526) (cherry picked from commit 3b231be8f000ae59faa04d5a2f1af11beafee866) Co-authored-by: Yuki Kobayashi <drsuaimqjgar(a)gmail.com> files: M Doc/library/email.policy.rst M Doc/tools/.nitignore diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst index 314767d0802a08..6b997ee784f6e4 100644 --- a/Doc/library/email.policy.rst +++ b/Doc/library/email.policy.rst @@ -267,7 +267,7 @@ added matters. To illustrate:: Handle a *defect* found on *obj*. When the email package calls this method, *defect* will always be a subclass of - :class:`~email.errors.Defect`. + :class:`~email.errors.MessageDefect`. The default implementation checks the :attr:`raise_on_defect` flag. If it is ``True``, *defect* is raised as an exception. If it is ``False`` @@ -277,7 +277,7 @@ added matters. To illustrate:: .. method:: register_defect(obj, defect) Register a *defect* on *obj*. In the email package, *defect* will always - be a subclass of :class:`~email.errors.Defect`. + be a subclass of :class:`~email.errors.MessageDefect`. The default implementation calls the ``append`` method of the ``defects`` attribute of *obj*. When the email package calls :attr:`handle_defect`, diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index d827ea29a98d16..8485766034ca77 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -24,7 +24,6 @@ Doc/library/email.charset.rst Doc/library/email.compat32-message.rst Doc/library/email.errors.rst Doc/library/email.parser.rst -Doc/library/email.policy.rst Doc/library/exceptions.rst Doc/library/functools.rst Doc/library/http.cookiejar.rst

1 0

Docs: fix `MessageDefect` references in email.policy docs (#128468)
by erlend-aasland 05 Jan '25

05 Jan '25

https://github.com/python/cpython/commit/3b231be8f000ae59faa04d5a2f1af11bea… commit: 3b231be8f000ae59faa04d5a2f1af11beafee866 branch: main author: Yuki Kobayashi <drsuaimqjgar(a)gmail.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2025-01-05T22:58:31+01:00 summary: Docs: fix `MessageDefect` references in email.policy docs (#128468) files: M Doc/library/email.policy.rst M Doc/tools/.nitignore diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst index 314767d0802a08..6b997ee784f6e4 100644 --- a/Doc/library/email.policy.rst +++ b/Doc/library/email.policy.rst @@ -267,7 +267,7 @@ added matters. To illustrate:: Handle a *defect* found on *obj*. When the email package calls this method, *defect* will always be a subclass of - :class:`~email.errors.Defect`. + :class:`~email.errors.MessageDefect`. The default implementation checks the :attr:`raise_on_defect` flag. If it is ``True``, *defect* is raised as an exception. If it is ``False`` @@ -277,7 +277,7 @@ added matters. To illustrate:: .. method:: register_defect(obj, defect) Register a *defect* on *obj*. In the email package, *defect* will always - be a subclass of :class:`~email.errors.Defect`. + be a subclass of :class:`~email.errors.MessageDefect`. The default implementation calls the ``append`` method of the ``defects`` attribute of *obj*. When the email package calls :attr:`handle_defect`, diff --git a/Doc/tools/.nitignore b/Doc/tools/.nitignore index 7d50aec56a9bf7..6940c95ab2c9a1 100644 --- a/Doc/tools/.nitignore +++ b/Doc/tools/.nitignore @@ -23,7 +23,6 @@ Doc/library/email.charset.rst Doc/library/email.compat32-message.rst Doc/library/email.errors.rst Doc/library/email.parser.rst -Doc/library/email.policy.rst Doc/library/exceptions.rst Doc/library/functools.rst Doc/library/http.cookiejar.rst

1 0

[3.12] gh-123085: Fix issue in inferred caller when resources package has no source (GH-123102) (#124021)
by jaraco 05 Jan '25

05 Jan '25

https://github.com/python/cpython/commit/d712ece43feeed7f930954d40b046cdaee… commit: d712ece43feeed7f930954d40b046cdaeed7c414 branch: 3.12 author: Jason R. Coombs <jaraco(a)jaraco.com> committer: jaraco <jaraco(a)jaraco.com> date: 2025-01-05T16:53:55-05:00 summary: [3.12] gh-123085: Fix issue in inferred caller when resources package has no source (GH-123102) (#124021) gh-123085: Fix issue in inferred caller when resources package has no source. >From importlib_resources 6.4.3 (python/importlib_resourcesGH-314). (cherry picked from commit a53812df126b99bca25187441a123c7785ee82a0) files: A Misc/NEWS.d/next/Library/2024-08-17-08-17-20.gh-issue-123085.7Io2yH.rst M Lib/importlib/resources/_common.py M Lib/test/test_importlib/resources/test_files.py diff --git a/Lib/importlib/resources/_common.py b/Lib/importlib/resources/_common.py index a3902535342612..a85df4b399fe61 100644 --- a/Lib/importlib/resources/_common.py +++ b/Lib/importlib/resources/_common.py @@ -93,12 +93,13 @@ def _infer_caller(): """ def is_this_file(frame_info): - return frame_info.filename == __file__ + return frame_info.filename == stack[0].filename def is_wrapper(frame_info): return frame_info.function == 'wrapper' - not_this_file = itertools.filterfalse(is_this_file, inspect.stack()) + stack = inspect.stack() + not_this_file = itertools.filterfalse(is_this_file, stack) # also exclude 'wrapper' due to singledispatch in the call stack callers = itertools.filterfalse(is_wrapper, not_this_file) return next(callers).frame diff --git a/Lib/test/test_importlib/resources/test_files.py b/Lib/test/test_importlib/resources/test_files.py index 038aac0a3fe773..d34ea51b150f1e 100644 --- a/Lib/test/test_importlib/resources/test_files.py +++ b/Lib/test/test_importlib/resources/test_files.py @@ -1,4 +1,8 @@ import typing +import os +import pathlib +import py_compile +import shutil import textwrap import unittest import warnings @@ -10,8 +14,7 @@ from . import data01 from . import util from . import _path -from test.support import os_helper -from test.support import import_helper +from test.support import os_helper, import_helper @contextlib.contextmanager @@ -144,6 +147,45 @@ def create_zip_from_directory(source_dir, zip_filename): self.fixtures.enter_context(import_helper.DirsOnSysPath(zip_file)) assert importlib.import_module('somepkg.submod').val == 'resources are the best' + def _compile_importlib(self): + """ + Make a compiled-only copy of the importlib resources package. + """ + bin_site = self.fixtures.enter_context(os_helper.temp_dir()) + c_resources = pathlib.Path(bin_site, 'c_resources') + sources = pathlib.Path(resources.__file__).parent + shutil.copytree(sources, c_resources, ignore=lambda *_: ['__pycache__']) + + for dirpath, _, filenames in os.walk(c_resources): + for filename in filenames: + source_path = pathlib.Path(dirpath) / filename + cfile = source_path.with_suffix('.pyc') + py_compile.compile(source_path, cfile) + pathlib.Path.unlink(source_path) + self.fixtures.enter_context(import_helper.DirsOnSysPath(bin_site)) + + def test_implicit_files_with_compiled_importlib(self): + """ + Caller detection works for compiled-only resources module. + + python/cpython#123085 + """ + set_val = textwrap.dedent( + f""" + import {resources.__name__} as res + val = res.files().joinpath('res.txt').read_text(encoding='utf-8') + """ + ) + spec = { + 'frozenpkg': { + '__init__.py': set_val.replace(resources.__name__, 'c_resources'), + 'res.txt': 'resources are the best', + }, + } + _path.build(spec, self.site_dir) + self._compile_importlib() + assert importlib.import_module('frozenpkg').val == 'resources are the best' + if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS.d/next/Library/2024-08-17-08-17-20.gh-issue-123085.7Io2yH.rst b/Misc/NEWS.d/next/Library/2024-08-17-08-17-20.gh-issue-123085.7Io2yH.rst new file mode 100644 index 00000000000000..2e09401ceb5b56 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-08-17-08-17-20.gh-issue-123085.7Io2yH.rst @@ -0,0 +1,3 @@ +In a bare call to :func:`importlib.resources.files`, ensure the caller's +frame is properly detected when ``importlib.resources`` is itself available +as a compiled module only (no source).

1 0