Mailman 3 June 2022 - Python-checkins

[3.11] gh-89038: [doc] update dis.findlinestarts documentation for changes related to PEP-626 (GH-94247) (GH-94450)
by iritkatriel June 30, 2022

June 30, 2022

https://github.com/python/cpython/commit/ecc8e77cc178707495dad7539348756fbe… commit: ecc8e77cc178707495dad7539348756fbe941bc5 branch: 3.11 author: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2022-06-30T16:18:30+01:00 summary: [3.11] gh-89038: [doc] update dis.findlinestarts documentation for changes related to PEP-626 (GH-94247) (GH-94450) (cherry picked from commit d68f2d27bbf85f3573a08fc7554889e1733a30f0) Co-authored-by: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> files: M Doc/library/dis.rst diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 505aa8188560f..6f882b6d712ab 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -268,15 +268,17 @@ operation is being performed, so the intermediate analysis object isn't useful: .. function:: findlinestarts(code) - This generator function uses the ``co_firstlineno`` and ``co_lnotab`` - attributes of the code object *code* to find the offsets which are starts of + This generator function uses the ``co_lines`` method + of the code object *code* to find the offsets which are starts of lines in the source code. They are generated as ``(offset, lineno)`` pairs. - See :source:`Objects/lnotab_notes.txt` for the ``co_lnotab`` format and - how to decode it. .. versionchanged:: 3.6 Line numbers can be decreasing. Before, they were always increasing. + .. versionchanged:: 3.10 + The :pep:`626` ``co_lines`` method is used instead of the ``co_firstlineno`` + and ``co_lnotab`` attributes of the code object. + .. function:: findlabels(code)

1 0

gh-89038: [doc] update dis.findlinestarts documentation for changes related to PEP-626 (GH-94247)
by iritkatriel June 30, 2022

June 30, 2022

https://github.com/python/cpython/commit/d68f2d27bbf85f3573a08fc7554889e173… commit: d68f2d27bbf85f3573a08fc7554889e1733a30f0 branch: main author: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2022-06-30T15:57:35+01:00 summary: gh-89038: [doc] update dis.findlinestarts documentation for changes related to PEP-626 (GH-94247) files: M Doc/library/dis.rst diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 4ec2fb0de98fb..d2c3c1a0966d7 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -266,15 +266,17 @@ operation is being performed, so the intermediate analysis object isn't useful: .. function:: findlinestarts(code) - This generator function uses the ``co_firstlineno`` and ``co_lnotab`` - attributes of the code object *code* to find the offsets which are starts of + This generator function uses the ``co_lines`` method + of the code object *code* to find the offsets which are starts of lines in the source code. They are generated as ``(offset, lineno)`` pairs. - See :source:`Objects/lnotab_notes.txt` for the ``co_lnotab`` format and - how to decode it. .. versionchanged:: 3.6 Line numbers can be decreasing. Before, they were always increasing. + .. versionchanged:: 3.10 + The :pep:`626` ``co_lines`` method is used instead of the ``co_firstlineno`` + and ``co_lnotab`` attributes of the code object. + .. function:: findlabels(code)

1 0

GH-94329: Don't raise on excessive stack consumption (GH-94421) (GH-94446)
by markshannon June 30, 2022

June 30, 2022

https://github.com/python/cpython/commit/f58c366a730ae3e68028dbf63aba7c4033… commit: f58c366a730ae3e68028dbf63aba7c4033eaec9b branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: markshannon <mark(a)hotpy.org> date: 2022-06-30T15:53:20+01:00 summary: GH-94329: Don't raise on excessive stack consumption (GH-94421) (GH-94446) (cherry picked from commit b152bf448b321e3a4c0a7280e0b608840f5ac661) files: A Misc/NEWS.d/next/Core and Builtins/2022-06-29-15-45-04.gh-issue-94329.olUQyk.rst M Lib/test/test_compile.py M Python/compile.c diff --git a/Lib/test/test_compile.py b/Lib/test/test_compile.py index c32c27f33b447..d7c536e4ab20d 100644 --- a/Lib/test/test_compile.py +++ b/Lib/test/test_compile.py @@ -1207,6 +1207,12 @@ def test_func_and(self): code += " x and x\n" * self.N self.check_stack_size(code) + def test_stack_3050(self): + M = 3050 + code = "x," * M + "=t" + # This raised on 3.10.0 to 3.10.5 + compile(code, "<foo>", "single") + class TestStackSizeStability(unittest.TestCase): # Check that repeating certain snippets doesn't increase the stack size diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-06-29-15-45-04.gh-issue-94329.olUQyk.rst b/Misc/NEWS.d/next/Core and Builtins/2022-06-29-15-45-04.gh-issue-94329.olUQyk.rst new file mode 100644 index 0000000000000..afd31b6e4dc44 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-06-29-15-45-04.gh-issue-94329.olUQyk.rst @@ -0,0 +1,2 @@ +Compile and run code with unpacking of extremely large sequences (1000s of elements). +Such code failed to compile. It now compiles and runs correctly. diff --git a/Python/compile.c b/Python/compile.c index 70a754b16d888..1bb0d96207d1d 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -8391,12 +8391,7 @@ assemble(struct compiler *c, int addNone) if (maxdepth < 0) { goto error; } - if (maxdepth > MAX_ALLOWED_STACK_USE) { - PyErr_Format(PyExc_SystemError, - "excessive stack use: stack is %d deep", - maxdepth); - goto error; - } + /* TO DO -- For 3.12, make sure that `maxdepth <= MAX_ALLOWED_STACK_USE` */ if (label_exception_targets(entryblock)) { goto error;

1 0

gh-94332: make it safe to call assemble_free when assemble_init has not been called (GH-94389) (GH-94442)
by iritkatriel June 30, 2022

June 30, 2022

https://github.com/python/cpython/commit/48a739ec106dffabe89d67e79736c7dcf2… commit: 48a739ec106dffabe89d67e79736c7dcf2f8f4a6 branch: 3.11 author: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2022-06-30T15:30:12+01:00 summary: gh-94332: make it safe to call assemble_free when assemble_init has not been called (GH-94389) (GH-94442) (cherry picked from commit be82d26570343dafc8a89be5a1a0e2f58d51a904) files: M Python/compile.c diff --git a/Python/compile.c b/Python/compile.c index 6d800b21c162f..70a754b16d888 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -8299,6 +8299,7 @@ assemble(struct compiler *c, int addNone) int j, nblocks; PyCodeObject *co = NULL; PyObject *consts = NULL; + memset(&a, 0, sizeof(struct assembler)); /* Make sure every block that falls off the end returns None. */ if (!c->u->u_curblock->b_return) {

1 0

GH-94329: Don't raise on excessive stack consumption (GH-94421)
by markshannon June 30, 2022

June 30, 2022

https://github.com/python/cpython/commit/b152bf448b321e3a4c0a7280e0b608840f… commit: b152bf448b321e3a4c0a7280e0b608840f5ac661 branch: main author: Mark Shannon <mark(a)hotpy.org> committer: markshannon <mark(a)hotpy.org> date: 2022-06-30T15:27:14+01:00 summary: GH-94329: Don't raise on excessive stack consumption (GH-94421) files: A Misc/NEWS.d/next/Core and Builtins/2022-06-29-15-45-04.gh-issue-94329.olUQyk.rst M Lib/test/test_compile.py M Python/compile.c diff --git a/Lib/test/test_compile.py b/Lib/test/test_compile.py index 85b2d82518409..46b16e753ceee 100644 --- a/Lib/test/test_compile.py +++ b/Lib/test/test_compile.py @@ -1243,6 +1243,12 @@ def test_func_and(self): code += " x and x\n" * self.N self.check_stack_size(code) + def test_stack_3050(self): + M = 3050 + code = "x," * M + "=t" + # This raised on 3.10.0 to 3.10.5 + compile(code, "<foo>", "single") + class TestStackSizeStability(unittest.TestCase): # Check that repeating certain snippets doesn't increase the stack size diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-06-29-15-45-04.gh-issue-94329.olUQyk.rst b/Misc/NEWS.d/next/Core and Builtins/2022-06-29-15-45-04.gh-issue-94329.olUQyk.rst new file mode 100644 index 0000000000000..afd31b6e4dc44 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2022-06-29-15-45-04.gh-issue-94329.olUQyk.rst @@ -0,0 +1,2 @@ +Compile and run code with unpacking of extremely large sequences (1000s of elements). +Such code failed to compile. It now compiles and runs correctly. diff --git a/Python/compile.c b/Python/compile.c index 59f88965bb29c..2d3b5717eeab5 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -8652,12 +8652,7 @@ assemble(struct compiler *c, int addNone) if (maxdepth < 0) { goto error; } - if (maxdepth > MAX_ALLOWED_STACK_USE) { - PyErr_Format(PyExc_SystemError, - "excessive stack use: stack is %d deep", - maxdepth); - goto error; - } + /* TO DO -- For 3.12, make sure that `maxdepth <= MAX_ALLOWED_STACK_USE` */ if (label_exception_targets(entryblock)) { goto error;

1 0

Delete DOS-only PC/testpy.py (GH-94419)
by zooba June 30, 2022

June 30, 2022

https://github.com/python/cpython/commit/9ef50c1d46dc2a52d83dcc7d0c4f527134… commit: 9ef50c1d46dc2a52d83dcc7d0c4f5271344d2f66 branch: main author: Oleg Iarygin <oleg(a)arhadthedev.net> committer: zooba <steve.dower(a)microsoft.com> date: 2022-06-30T14:17:35+01:00 summary: Delete DOS-only PC/testpy.py (GH-94419) files: D PC/testpy.py M PC/readme.txt diff --git a/PC/readme.txt b/PC/readme.txt index 4e6dcf98c937f..bef5111c59182 100644 --- a/PC/readme.txt +++ b/PC/readme.txt @@ -51,12 +51,6 @@ config.c The list of C modules to include in the Python PC version. Manually edit this file to add or remove Python modules. -testpy.py A Python test program. Run this to test your - Python port. It should produce copious output, - ending in a report on how many tests were OK, how many - failed, and how many were skipped. Don't worry about - skipped tests (these test unavailable optional features). - Additional files and subdirectories for 32-bit Windows ====================================================== diff --git a/PC/testpy.py b/PC/testpy.py deleted file mode 100644 index 709f35c452587..0000000000000 --- a/PC/testpy.py +++ /dev/null @@ -1,30 +0,0 @@ -import sys - -# This is a test module for Python. It looks in the standard -# places for various *.py files. If these are moved, you must -# change this module too. - -try: - import os -except: - print("""Could not import the standard "os" module. - Please check your PYTHONPATH environment variable.""") - sys.exit(1) - -try: - import symbol -except: - print("""Could not import the standard "symbol" module. If this is - a PC, you should add the dos_8x3 directory to your PYTHONPATH.""") - sys.exit(1) - -for dir in sys.path: - file = os.path.join(dir, "os.py") - if os.path.isfile(file): - test = os.path.join(dir, "test") - if os.path.isdir(test): - # Add the "test" directory to PYTHONPATH. - sys.path = sys.path + [test] - -import libregrtest # Standard Python tester. -libregrtest.main()

1 0

gh-94332: make it safe to call assemble_free when assemble_init has not been called (GH-94389)
by iritkatriel June 30, 2022

June 30, 2022

https://github.com/python/cpython/commit/be82d26570343dafc8a89be5a1a0e2f58d… commit: be82d26570343dafc8a89be5a1a0e2f58d51a904 branch: main author: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2022-06-30T12:26:29+01:00 summary: gh-94332: make it safe to call assemble_free when assemble_init has not been called (GH-94389) files: M Python/compile.c diff --git a/Python/compile.c b/Python/compile.c index 74c21fd69f80c..59f88965bb29c 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -8554,6 +8554,8 @@ assemble(struct compiler *c, int addNone) { PyCodeObject *co = NULL; PyObject *consts = NULL; + struct assembler a; + memset(&a, 0, sizeof(struct assembler)); int code_flags = compute_code_flags(c); if (code_flags < 0) { @@ -8683,7 +8685,6 @@ assemble(struct compiler *c, int addNone) /* Create assembler */ - struct assembler a; if (!assemble_init(&a, c->u->u_firstlineno)) goto error;

1 0

gh-92336: linecache.getline should not raise exceptions on decoding errors (GH-94410)
by miss-islington June 30, 2022

June 30, 2022

https://github.com/python/cpython/commit/20b8c9eee0978b8f621293d5dbf2d574d9… commit: 20b8c9eee0978b8f621293d5dbf2d574d9b83d1b branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-06-30T02:59:33-07:00 summary: gh-92336: linecache.getline should not raise exceptions on decoding errors (GH-94410) (cherry picked from commit 21cbdae90ffdac047d27d1b83a5442fabcf89f7c) Co-authored-by: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2022-06-29-09-48-37.gh-issue-92336.otA6c6.rst M Lib/linecache.py M Lib/test/test_linecache.py diff --git a/Lib/linecache.py b/Lib/linecache.py index 23191d6501d2a..97644a8e3794e 100644 --- a/Lib/linecache.py +++ b/Lib/linecache.py @@ -135,7 +135,7 @@ def updatecache(filename, module_globals=None): try: with tokenize.open(fullname) as fp: lines = fp.readlines() - except OSError: + except (OSError, UnicodeDecodeError, SyntaxError): return [] if lines and not lines[-1].endswith('\n'): lines[-1] += '\n' diff --git a/Lib/test/test_linecache.py b/Lib/test/test_linecache.py index c6e2dadbb25e1..72dd40136cfdb 100644 --- a/Lib/test/test_linecache.py +++ b/Lib/test/test_linecache.py @@ -73,12 +73,10 @@ class GetLineTestsBadData(TempFile): # file_byte_string = b'Bad data goes here' def test_getline(self): - self.assertRaises((SyntaxError, UnicodeDecodeError), - linecache.getline, self.file_name, 1) + self.assertEqual(linecache.getline(self.file_name, 1), '') def test_getlines(self): - self.assertRaises((SyntaxError, UnicodeDecodeError), - linecache.getlines, self.file_name) + self.assertEqual(linecache.getlines(self.file_name), []) class EmptyFile(GetLineTestsGoodData, unittest.TestCase): @@ -92,9 +90,11 @@ class SingleEmptyLine(GetLineTestsGoodData, unittest.TestCase): class GoodUnicode(GetLineTestsGoodData, unittest.TestCase): file_list = ['á\n', 'b\n', 'abcdef\n', 'ááááá\n'] +class BadUnicode_NoDeclaration(GetLineTestsBadData, unittest.TestCase): + file_byte_string = b'\n\x80abc' -class BadUnicode(GetLineTestsBadData, unittest.TestCase): - file_byte_string = b'\x80abc' +class BadUnicode_WithDeclaration(GetLineTestsBadData, unittest.TestCase): + file_byte_string = b'# coding=utf-8\n\x80abc' class LineCacheTests(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Library/2022-06-29-09-48-37.gh-issue-92336.otA6c6.rst b/Misc/NEWS.d/next/Library/2022-06-29-09-48-37.gh-issue-92336.otA6c6.rst new file mode 100644 index 0000000000000..eb74e0ceb7446 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2022-06-29-09-48-37.gh-issue-92336.otA6c6.rst @@ -0,0 +1 @@ +Fix bug where :meth:`linecache.getline` fails on bad files with :exc:`UnicodeDecodeError` or :exc:`SyntaxError`. It now returns an empty string as per the documentation.

1 0

[3.10] GH-77403: Fix tests which fail when PYTHONUSERBASE is not normalized (GH-93917) (GH-93970)
by miss-islington June 30, 2022

June 30, 2022

https://github.com/python/cpython/commit/226fdce45d217d6e9375d784d1e6aba00c… commit: 226fdce45d217d6e9375d784d1e6aba00c7c491c branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-06-30T02:48:13-07:00 summary: [3.10] GH-77403: Fix tests which fail when PYTHONUSERBASE is not normalized (GH-93917) (GH-93970) (cherry picked from commit b1ae4af5e82e7275cebcfb383690b816a388a785) Co-authored-by: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> Automerge-Triggered-By: GH:iritkatriel files: M Lib/test/test_site.py M Lib/test/test_sysconfig.py diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py index aff3db5d9f49f..93349ed8bb16a 100644 --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -205,14 +205,14 @@ def test_get_path(self): scheme = 'osx_framework_user' else: scheme = os.name + '_user' - self.assertEqual(site._get_path(site._getuserbase()), + self.assertEqual(os.path.normpath(site._get_path(site._getuserbase())), sysconfig.get_path('purelib', scheme)) @unittest.skipUnless(site.ENABLE_USER_SITE, "requires access to PEP 370 " "user-site (site.ENABLE_USER_SITE)") def test_s_option(self): # (ncoghlan) Change this to use script_helper... - usersite = site.USER_SITE + usersite = os.path.normpath(site.USER_SITE) self.assertIn(usersite, sys.path) env = os.environ.copy() diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py index 9408657c91886..5ee9839c0487d 100644 --- a/Lib/test/test_sysconfig.py +++ b/Lib/test/test_sysconfig.py @@ -296,7 +296,7 @@ def test_user_similar(self): base = base.replace(sys.base_prefix, sys.prefix) if HAS_USER_BASE: user_path = get_path(name, 'posix_user') - expected = global_path.replace(base, user, 1) + expected = os.path.normpath(global_path.replace(base, user, 1)) # bpo-44860: platlib of posix_user doesn't use sys.platlibdir, # whereas posix_prefix does. if name == 'platlib':

1 0

[3.11] GH-77403: Fix tests which fail when PYTHONUSERBASE is not normalized (GH-93917) (GH-93969)
by miss-islington June 30, 2022

June 30, 2022

https://github.com/python/cpython/commit/c5ecfa0a1e5334be21ca3448abf87aabc5… commit: c5ecfa0a1e5334be21ca3448abf87aabc534a97e branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2022-06-30T02:47:16-07:00 summary: [3.11] GH-77403: Fix tests which fail when PYTHONUSERBASE is not normalized (GH-93917) (GH-93969) (cherry picked from commit b1ae4af5e82e7275cebcfb383690b816a388a785) Co-authored-by: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> Automerge-Triggered-By: GH:iritkatriel files: M Lib/test/test_site.py M Lib/test/test_sysconfig.py diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py index 9a148fe2f4e98..b5dc381a2f3bb 100644 --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -206,7 +206,7 @@ def test_get_path(self): scheme = 'osx_framework_user' else: scheme = os.name + '_user' - self.assertEqual(site._get_path(site._getuserbase()), + self.assertEqual(os.path.normpath(site._get_path(site._getuserbase())), sysconfig.get_path('purelib', scheme)) @unittest.skipUnless(site.ENABLE_USER_SITE, "requires access to PEP 370 " @@ -214,7 +214,7 @@ def test_get_path(self): @support.requires_subprocess() def test_s_option(self): # (ncoghlan) Change this to use script_helper... - usersite = site.USER_SITE + usersite = os.path.normpath(site.USER_SITE) self.assertIn(usersite, sys.path) env = os.environ.copy() diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py index 1679700f59296..83b492e9219ac 100644 --- a/Lib/test/test_sysconfig.py +++ b/Lib/test/test_sysconfig.py @@ -370,7 +370,7 @@ def test_user_similar(self): base = base.replace(sys.base_prefix, sys.prefix) if HAS_USER_BASE: user_path = get_path(name, 'posix_user') - expected = global_path.replace(base, user, 1) + expected = os.path.normpath(global_path.replace(base, user, 1)) # bpo-44860: platlib of posix_user doesn't use sys.platlibdir, # whereas posix_prefix does. if name == 'platlib':

1 0