Mailman 3 July 2019 - Python-checkins

Don't skip pickle check_frame_opcodes() (GH-15027)
by Neil Schemenauer July 30, 2019

July 30, 2019

https://github.com/python/cpython/commit/0f9efbcf9a051198dc9e226528d5c5d24a… commit: 0f9efbcf9a051198dc9e226528d5c5d24a52d23a branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: Neil Schemenauer <nas-github(a)arctrix.com> date: 2019-07-30T13:10:15-07:00 summary: Don't skip pickle check_frame_opcodes() (GH-15027) This looks like the only place that proto 4 framing gets exercised so leave it as part of the PGO task. (cherry picked … [View More]

1 0

Don't skip pickle check_frame_opcodes() (GH-15025)
by Neil Schemenauer July 30, 2019

July 30, 2019

https://github.com/python/cpython/commit/eca7ffc61cf925eae5def8c64b05d47f24… commit: eca7ffc61cf925eae5def8c64b05d47f24e60e1f branch: master author: Neil Schemenauer <nas-github(a)arctrix.com> committer: GitHub <noreply(a)github.com> date: 2019-07-30T12:08:17-07:00 summary: Don't skip pickle check_frame_opcodes() (GH-15025) This looks like the only place that proto 4 framing gets exercised so leave it as part of the PGO task. files: M Lib/test/pickletester.py diff --git a/Lib/… [View More]

1 0

bpo-37707: Exclude expensive unit tests from PGO task (GH-15009) (#15024)
by Neil Schemenauer July 30, 2019

July 30, 2019

https://github.com/python/cpython/commit/382cb85401bb010ead411c0532499ffe16… commit: 382cb85401bb010ead411c0532499ffe16c3cf27 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: Neil Schemenauer <nas-github(a)arctrix.com> date: 2019-07-30T11:34:32-07:00 summary: bpo-37707: Exclude expensive unit tests from PGO task (GH-15009) (#15024) Mark some individual tests to skip when --pgo is used. The tests marked increase the PGO task … [View More]time significantly and likely don't help improve optimization of the final executable. (cherry picked from commit 52a48e62c6a94577152f9301bbe5f3bc806cfcf1) Co-authored-by: Neil Schemenauer <nas-github(a)arctrix.com> files: A Misc/NEWS.d/next/Build/2019-07-29-11-36-16.bpo-37707.Sm-dGk.rst M Lib/test/libregrtest/main.py M Lib/test/pickletester.py M Lib/test/support/__init__.py M Lib/test/test_bz2.py M Lib/test/test_itertools.py M Lib/test/test_lzma.py M Lib/test/test_statistics.py diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py index 2b6607df15dd..87278d0c76a2 100644 --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -643,6 +643,7 @@ def _main(self, tests, kwargs): input("Press any key to continue...") support.PGO = self.ns.pgo + support.PGO_EXTENDED = self.ns.pgo_extended setup_tests(self.ns) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py index 2bc99e6becfd..5d402a4711bf 100644 --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -2281,6 +2281,7 @@ def test_setitems_on_non_dicts(self): FRAME_SIZE_MIN = 4 FRAME_SIZE_TARGET = 64 * 1024 + @support.skip_if_pgo_task def check_frame_opcodes(self, pickled): """ Check the arguments of FRAME opcodes in a protocol 4+ pickle. @@ -2328,6 +2329,7 @@ def check_frame_opcodes(self, pickled): elif frameless_start is not None: self.assertLess(pos - frameless_start, self.FRAME_SIZE_MIN) + @support.skip_if_pgo_task def test_framing_many_objects(self): obj = list(range(10**5)) for proto in range(4, pickle.HIGHEST_PROTOCOL + 1): @@ -2417,6 +2419,7 @@ def remove_frames(pickled, keep_frame=None): count_opcode(pickle.FRAME, pickled)) self.assertEqual(obj, self.loads(some_frames_pickle)) + @support.skip_if_pgo_task def test_framed_write_sizes_with_delayed_writer(self): class ChunkAccumulator: """Accumulate pickler output in a list of raw chunks.""" diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 1ada1f927d9c..b88eb5804a96 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -971,6 +971,10 @@ def dec(*args, **kwargs): # useful for PGO PGO = False +# Set by libregrtest/main.py if we are running the extended (time consuming) +# PGO task. If this is True, PGO is also True. +PGO_EXTENDED = False + @contextlib.contextmanager def temp_dir(path=None, quiet=False): """Return a context manager that creates a temporary directory. @@ -2636,6 +2640,12 @@ def skip_unless_xattr(test): msg = "no non-broken extended attribute support" return test if ok else unittest.skip(msg)(test) +def skip_if_pgo_task(test): + """Skip decorator for tests not run in (non-extended) PGO task""" + ok = not PGO or PGO_EXTENDED + msg = "Not run for (non-extended) PGO task" + return test if ok else unittest.skip(msg)(test) + _bind_nix_socket_error = None def skip_unless_bind_unix_socket(test): """Decorator for tests requiring a functional bind() for unix sockets.""" diff --git a/Lib/test/test_bz2.py b/Lib/test/test_bz2.py index f158b901b9c9..eb2f72ee4a5d 100644 --- a/Lib/test/test_bz2.py +++ b/Lib/test/test_bz2.py @@ -643,6 +643,7 @@ def testCompressChunks10(self): data += bz2c.flush() self.assertEqual(ext_decompress(data), self.TEXT) + @support.skip_if_pgo_task @bigmemtest(size=_4G + 100, memuse=2) def testCompress4G(self, size): # "Test BZ2Compressor.compress()/flush() with >4GiB input" @@ -701,6 +702,7 @@ def testEOFError(self): self.assertRaises(EOFError, bz2d.decompress, b"anything") self.assertRaises(EOFError, bz2d.decompress, b"") + @support.skip_if_pgo_task @bigmemtest(size=_4G + 100, memuse=3.3) def testDecompress4G(self, size): # "Test BZ2Decompressor.decompress() with >4GiB input" diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py index ea060a98a5ee..573739fde14c 100644 --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -2062,6 +2062,7 @@ def gen2(x): self.assertRaises(AssertionError, list, cycle(gen1())) self.assertEqual(hist, [0,1]) + @support.skip_if_pgo_task def test_long_chain_of_empty_iterables(self): # Make sure itertools.chain doesn't run into recursion limits when # dealing with long chains of empty iterables. Even with a high diff --git a/Lib/test/test_lzma.py b/Lib/test/test_lzma.py index 3dc2c1e7e3b7..117de0a1b554 100644 --- a/Lib/test/test_lzma.py +++ b/Lib/test/test_lzma.py @@ -333,6 +333,7 @@ def test_decompressor_multistream(self): # Test with inputs larger than 4GiB. + @support.skip_if_pgo_task @bigmemtest(size=_4G + 100, memuse=2) def test_compressor_bigmem(self, size): lzc = LZMACompressor() @@ -344,6 +345,7 @@ def test_compressor_bigmem(self, size): finally: ddata = None + @support.skip_if_pgo_task @bigmemtest(size=_4G + 100, memuse=3) def test_decompressor_bigmem(self, size): lzd = LZMADecompressor() diff --git a/Lib/test/test_statistics.py b/Lib/test/test_statistics.py index ed2f6579b0b9..104718ed7db6 100644 --- a/Lib/test/test_statistics.py +++ b/Lib/test/test_statistics.py @@ -14,6 +14,7 @@ import random import sys import unittest +from test import support from decimal import Decimal from fractions import Fraction @@ -2462,6 +2463,7 @@ def test_cdf(self): self.assertEqual(X.cdf(float('Inf')), 1.0) self.assertTrue(math.isnan(X.cdf(float('NaN')))) + @support.skip_if_pgo_task def test_inv_cdf(self): NormalDist = statistics.NormalDist diff --git a/Misc/NEWS.d/next/Build/2019-07-29-11-36-16.bpo-37707.Sm-dGk.rst b/Misc/NEWS.d/next/Build/2019-07-29-11-36-16.bpo-37707.Sm-dGk.rst new file mode 100644 index 000000000000..c0d58ab747b2 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2019-07-29-11-36-16.bpo-37707.Sm-dGk.rst @@ -0,0 +1,3 @@ +Mark some individual tests to skip when --pgo is used. The tests marked +increase the PGO task time significantly and likely don't help improve +optimization of the final executable. [View Less]

1 0

bpo-37707: Exclude expensive unit tests from PGO task (GH-15009)
by Neil Schemenauer July 30, 2019

July 30, 2019

https://github.com/python/cpython/commit/52a48e62c6a94577152f9301bbe5f3bc80… commit: 52a48e62c6a94577152f9301bbe5f3bc806cfcf1 branch: master author: Neil Schemenauer <nas-github(a)arctrix.com> committer: GitHub <noreply(a)github.com> date: 2019-07-30T11:08:18-07:00 summary: bpo-37707: Exclude expensive unit tests from PGO task (GH-15009) Mark some individual tests to skip when --pgo is used. The tests marked increase the PGO task time significantly and likely don't help improve … [View More]optimization of the final executable. files: A Misc/NEWS.d/next/Build/2019-07-29-11-36-16.bpo-37707.Sm-dGk.rst M Lib/test/libregrtest/main.py M Lib/test/pickletester.py M Lib/test/support/__init__.py M Lib/test/test_bz2.py M Lib/test/test_itertools.py M Lib/test/test_lzma.py M Lib/test/test_statistics.py diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py index 2b6607df15dd..87278d0c76a2 100644 --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -643,6 +643,7 @@ def _main(self, tests, kwargs): input("Press any key to continue...") support.PGO = self.ns.pgo + support.PGO_EXTENDED = self.ns.pgo_extended setup_tests(self.ns) diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py index 2bc99e6becfd..5d402a4711bf 100644 --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -2281,6 +2281,7 @@ def test_setitems_on_non_dicts(self): FRAME_SIZE_MIN = 4 FRAME_SIZE_TARGET = 64 * 1024 + @support.skip_if_pgo_task def check_frame_opcodes(self, pickled): """ Check the arguments of FRAME opcodes in a protocol 4+ pickle. @@ -2328,6 +2329,7 @@ def check_frame_opcodes(self, pickled): elif frameless_start is not None: self.assertLess(pos - frameless_start, self.FRAME_SIZE_MIN) + @support.skip_if_pgo_task def test_framing_many_objects(self): obj = list(range(10**5)) for proto in range(4, pickle.HIGHEST_PROTOCOL + 1): @@ -2417,6 +2419,7 @@ def remove_frames(pickled, keep_frame=None): count_opcode(pickle.FRAME, pickled)) self.assertEqual(obj, self.loads(some_frames_pickle)) + @support.skip_if_pgo_task def test_framed_write_sizes_with_delayed_writer(self): class ChunkAccumulator: """Accumulate pickler output in a list of raw chunks.""" diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 9efb3d91705f..dbbbdb0ee339 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -973,6 +973,10 @@ def dec(*args, **kwargs): # useful for PGO PGO = False +# Set by libregrtest/main.py if we are running the extended (time consuming) +# PGO task. If this is True, PGO is also True. +PGO_EXTENDED = False + @contextlib.contextmanager def temp_dir(path=None, quiet=False): """Return a context manager that creates a temporary directory. @@ -2638,6 +2642,12 @@ def skip_unless_xattr(test): msg = "no non-broken extended attribute support" return test if ok else unittest.skip(msg)(test) +def skip_if_pgo_task(test): + """Skip decorator for tests not run in (non-extended) PGO task""" + ok = not PGO or PGO_EXTENDED + msg = "Not run for (non-extended) PGO task" + return test if ok else unittest.skip(msg)(test) + _bind_nix_socket_error = None def skip_unless_bind_unix_socket(test): """Decorator for tests requiring a functional bind() for unix sockets.""" diff --git a/Lib/test/test_bz2.py b/Lib/test/test_bz2.py index f158b901b9c9..eb2f72ee4a5d 100644 --- a/Lib/test/test_bz2.py +++ b/Lib/test/test_bz2.py @@ -643,6 +643,7 @@ def testCompressChunks10(self): data += bz2c.flush() self.assertEqual(ext_decompress(data), self.TEXT) + @support.skip_if_pgo_task @bigmemtest(size=_4G + 100, memuse=2) def testCompress4G(self, size): # "Test BZ2Compressor.compress()/flush() with >4GiB input" @@ -701,6 +702,7 @@ def testEOFError(self): self.assertRaises(EOFError, bz2d.decompress, b"anything") self.assertRaises(EOFError, bz2d.decompress, b"") + @support.skip_if_pgo_task @bigmemtest(size=_4G + 100, memuse=3.3) def testDecompress4G(self, size): # "Test BZ2Decompressor.decompress() with >4GiB input" diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py index ea060a98a5ee..573739fde14c 100644 --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -2062,6 +2062,7 @@ def gen2(x): self.assertRaises(AssertionError, list, cycle(gen1())) self.assertEqual(hist, [0,1]) + @support.skip_if_pgo_task def test_long_chain_of_empty_iterables(self): # Make sure itertools.chain doesn't run into recursion limits when # dealing with long chains of empty iterables. Even with a high diff --git a/Lib/test/test_lzma.py b/Lib/test/test_lzma.py index 3dc2c1e7e3b7..117de0a1b554 100644 --- a/Lib/test/test_lzma.py +++ b/Lib/test/test_lzma.py @@ -333,6 +333,7 @@ def test_decompressor_multistream(self): # Test with inputs larger than 4GiB. + @support.skip_if_pgo_task @bigmemtest(size=_4G + 100, memuse=2) def test_compressor_bigmem(self, size): lzc = LZMACompressor() @@ -344,6 +345,7 @@ def test_compressor_bigmem(self, size): finally: ddata = None + @support.skip_if_pgo_task @bigmemtest(size=_4G + 100, memuse=3) def test_decompressor_bigmem(self, size): lzd = LZMADecompressor() diff --git a/Lib/test/test_statistics.py b/Lib/test/test_statistics.py index ed2f6579b0b9..104718ed7db6 100644 --- a/Lib/test/test_statistics.py +++ b/Lib/test/test_statistics.py @@ -14,6 +14,7 @@ import random import sys import unittest +from test import support from decimal import Decimal from fractions import Fraction @@ -2462,6 +2463,7 @@ def test_cdf(self): self.assertEqual(X.cdf(float('Inf')), 1.0) self.assertTrue(math.isnan(X.cdf(float('NaN')))) + @support.skip_if_pgo_task def test_inv_cdf(self): NormalDist = statistics.NormalDist diff --git a/Misc/NEWS.d/next/Build/2019-07-29-11-36-16.bpo-37707.Sm-dGk.rst b/Misc/NEWS.d/next/Build/2019-07-29-11-36-16.bpo-37707.Sm-dGk.rst new file mode 100644 index 000000000000..c0d58ab747b2 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2019-07-29-11-36-16.bpo-37707.Sm-dGk.rst @@ -0,0 +1,3 @@ +Mark some individual tests to skip when --pgo is used. The tests marked +increase the PGO task time significantly and likely don't help improve +optimization of the final executable. [View Less]

1 0

bpo-37704: Remove Tools/scripts/h2py.py (GH-15000)
by Victor Stinner July 30, 2019

July 30, 2019

https://github.com/python/cpython/commit/27eeaf0f2c9bd53a0fbdfdf38ee759e971… commit: 27eeaf0f2c9bd53a0fbdfdf38ee759e971221f8d branch: master author: Victor Stinner <vstinner(a)redhat.com> committer: GitHub <noreply(a)github.com> date: 2019-07-30T17:45:09+02:00 summary: bpo-37704: Remove Tools/scripts/h2py.py (GH-15000) Use cffi to access a C API in Python. files: A Misc/NEWS.d/next/Tools-Demos/2019-07-29-13-59-19.bpo-37704.xxGUz_.rst D Tools/scripts/h2py.py M Lib/test/test_tools/… [View More]test_sundry.py M Tools/scripts/README diff --git a/Lib/test/test_tools/test_sundry.py b/Lib/test/test_tools/test_sundry.py index f5fed01491e3..10eb6941b3be 100644 --- a/Lib/test/test_tools/test_sundry.py +++ b/Lib/test/test_tools/test_sundry.py @@ -2,7 +2,7 @@ This file contains extremely basic regression tests for the scripts found in the Tools directory of a Python checkout or tarball which don't have separate -tests of their own, such as h2py.py. +tests of their own. """ import os diff --git a/Misc/NEWS.d/next/Tools-Demos/2019-07-29-13-59-19.bpo-37704.xxGUz_.rst b/Misc/NEWS.d/next/Tools-Demos/2019-07-29-13-59-19.bpo-37704.xxGUz_.rst new file mode 100644 index 000000000000..52ca4895dee9 --- /dev/null +++ b/Misc/NEWS.d/next/Tools-Demos/2019-07-29-13-59-19.bpo-37704.xxGUz_.rst @@ -0,0 +1 @@ +Remove ``Tools/scripts/h2py.py``: use cffi to access a C API in Python. diff --git a/Tools/scripts/README b/Tools/scripts/README index d4ac2ade25ef..bee5b0bd9b20 100644 --- a/Tools/scripts/README +++ b/Tools/scripts/README @@ -30,7 +30,6 @@ ftpmirror.py FTP mirror script get-remote-certificate.py Fetch the certificate that the server(s) are providing in PEM form google.py Open a webbrowser with Google gprof2html.py Transform gprof(1) output into useful HTML -h2py.py Translate #define's into Python assignments highlight.py Python syntax highlighting with HTML output idle3 Main program to start IDLE ifdef.py Remove #if(n)def groups from C sources diff --git a/Tools/scripts/h2py.py b/Tools/scripts/h2py.py deleted file mode 100755 index ea37c04d4c56..000000000000 --- a/Tools/scripts/h2py.py +++ /dev/null @@ -1,171 +0,0 @@ -#! /usr/bin/env python3 - -# Read #define's and translate to Python code. -# Handle #include statements. -# Handle #define macros with one argument. -# Anything that isn't recognized or doesn't translate into valid -# Python is ignored. - -# Without filename arguments, acts as a filter. -# If one or more filenames are given, output is written to corresponding -# filenames in the local directory, translated to all uppercase, with -# the extension replaced by ".py". - -# By passing one or more options of the form "-i regular_expression" -# you can specify additional strings to be ignored. This is useful -# e.g. to ignore casts to u_long: simply specify "-i '(u_long)'". - -# XXX To do: -# - turn trailing C comments into Python comments -# - turn C Boolean operators "&& || !" into Python "and or not" -# - what to do about #if(def)? -# - what to do about macros with multiple parameters? - -import sys, re, getopt, os - -p_define = re.compile(r'^[\t ]*#[\t ]*define[\t ]+([a-zA-Z0-9_]+)[\t ]+') - -p_macro = re.compile( - r'^[\t ]*#[\t ]*define[\t ]+' - r'([a-zA-Z0-9_]+)\(([_a-zA-Z][_a-zA-Z0-9]*)\)[\t ]+') - -p_include = re.compile(r'^[\t ]*#[\t ]*include[\t ]+<([^>\n]+)>') - -p_comment = re.compile(r'/\*([^*]+|\*+[^/])*(\*+/)?') -p_cpp_comment = re.compile('//.*') - -ignores = [p_comment, p_cpp_comment] - -p_char = re.compile(r"'(\\.[^\\]*|[^\\])'") - -p_hex = re.compile(r"0x([0-9a-fA-F]+)L?") - -filedict = {} -importable = {} - -try: - searchdirs=os.environ['include'].split(';') -except KeyError: - try: - searchdirs=os.environ['INCLUDE'].split(';') - except KeyError: - searchdirs=['/usr/include'] - try: - searchdirs.insert(0, os.path.join('/usr/include', - os.environ['MULTIARCH'])) - except KeyError: - pass - -def main(): - global filedict - opts, args = getopt.getopt(sys.argv[1:], 'i:') - for o, a in opts: - if o == '-i': - ignores.append(re.compile(a)) - if not args: - args = ['-'] - for filename in args: - if filename == '-': - sys.stdout.write('# Generated by h2py from stdin\n') - process(sys.stdin, sys.stdout) - else: - with open(filename) as fp: - outfile = os.path.basename(filename) - i = outfile.rfind('.') - if i > 0: outfile = outfile[:i] - modname = outfile.upper() - outfile = modname + '.py' - with open(outfile, 'w') as outfp: - outfp.write('# Generated by h2py from %s\n' % filename) - filedict = {} - for dir in searchdirs: - if filename[:len(dir)] == dir: - filedict[filename[len(dir)+1:]] = None # no '/' trailing - importable[filename[len(dir)+1:]] = modname - break - process(fp, outfp) - -def pytify(body): - # replace ignored patterns by spaces - for p in ignores: - body = p.sub(' ', body) - # replace char literals by ord(...) - body = p_char.sub("ord('\\1')", body) - # Compute negative hexadecimal constants - start = 0 - UMAX = 2*(sys.maxsize+1) - while 1: - m = p_hex.search(body, start) - if not m: break - s,e = m.span() - val = int(body[slice(*m.span(1))], 16) - if val > sys.maxsize: - val -= UMAX - body = body[:s] + "(" + str(val) + ")" + body[e:] - start = s + 1 - return body - -def process(fp, outfp, env = {}): - lineno = 0 - while 1: - line = fp.readline() - if not line: break - lineno = lineno + 1 - match = p_define.match(line) - if match: - # gobble up continuation lines - while line[-2:] == '\\\n': - nextline = fp.readline() - if not nextline: break - lineno = lineno + 1 - line = line + nextline - name = match.group(1) - body = line[match.end():] - body = pytify(body) - ok = 0 - stmt = '%s = %s\n' % (name, body.strip()) - try: - exec(stmt, env) - except: - sys.stderr.write('Skipping: %s' % stmt) - else: - outfp.write(stmt) - match = p_macro.match(line) - if match: - macro, arg = match.group(1, 2) - body = line[match.end():] - body = pytify(body) - stmt = 'def %s(%s): return %s\n' % (macro, arg, body) - try: - exec(stmt, env) - except: - sys.stderr.write('Skipping: %s' % stmt) - else: - outfp.write(stmt) - match = p_include.match(line) - if match: - regs = match.regs - a, b = regs[1] - filename = line[a:b] - if filename in importable: - outfp.write('from %s import *\n' % importable[filename]) - elif filename not in filedict: - filedict[filename] = None - inclfp = None - for dir in searchdirs: - try: - inclfp = open(dir + '/' + filename) - break - except IOError: - pass - if inclfp: - with inclfp: - outfp.write( - '\n# Included from %s\n' % filename) - process(inclfp, outfp, env) - else: - sys.stderr.write('Warning - could not find file %s\n' % - filename) - -if __name__ == '__main__': - main() [View Less]

1 0

bpo-37587: Make json.loads faster for long strings (GH-14752)
by Miss Islington (bot) July 30, 2019

July 30, 2019

https://github.com/python/cpython/commit/9265a877426af4fa5c44cc8482e0198806… commit: 9265a877426af4fa5c44cc8482e0198806889350 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2019-07-30T07:37:28-07:00 summary: bpo-37587: Make json.loads faster for long strings (GH-14752) When scanning the string, most characters are valid, so checking for invalid characters first means never needing to … [View More]

1 0

bpo-37587: Make json.loads faster for long strings (GH-14752)
by Nick Coghlan July 30, 2019

July 30, 2019

https://github.com/python/cpython/commit/8a758f5b99c5fc3fd32edeac049d7d4a4b… commit: 8a758f5b99c5fc3fd32edeac049d7d4a4b7cc163 branch: master author: Marco Paolini <mpaolini(a)users.noreply.github.com> committer: Nick Coghlan <ncoghlan(a)gmail.com> date: 2019-07-31T00:16:34+10:00 summary: bpo-37587: Make json.loads faster for long strings (GH-14752) When scanning the string, most characters are valid, so checking for invalid characters first means never needing to check the value … [View More]

1 0

bpo-37268: Add deprecation notice and a DeprecationWarning for the parser module (GH-15017)
by Miss Islington (bot) July 30, 2019

July 30, 2019

https://github.com/python/cpython/commit/9211e2fd81fe1db6f73ded70752b144cc9… commit: 9211e2fd81fe1db6f73ded70752b144cc9691ab6 branch: master author: Pablo Galindo <Pablogsal(a)gmail.com> committer: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> date: 2019-07-30T04:04:01-07:00 summary: bpo-37268: Add deprecation notice and a DeprecationWarning for the parser module (GH-15017) Deprecate the parser module and add a deprecation warning triggered on import … [View More]and a warning block in the documentation. https://bugs.python.org/issue37268 Automerge-Triggered-By: @pablogsal files: A Misc/NEWS.d/next/Library/2019-07-30-01-27-29.bpo-37268.QDmA44.rst M Doc/library/parser.rst M Doc/whatsnew/3.9.rst M Lib/test/test_parser.py M Modules/parsermodule.c diff --git a/Doc/library/parser.rst b/Doc/library/parser.rst index a302681eca05..c55cd653b371 100644 --- a/Doc/library/parser.rst +++ b/Doc/library/parser.rst @@ -25,11 +25,11 @@ from this. This is better than trying to parse and modify an arbitrary Python code fragment as a string because parsing is performed in a manner identical to the code forming the application. It is also faster. -.. note:: +.. warning:: - From Python 2.5 onward, it's much more convenient to cut in at the Abstract - Syntax Tree (AST) generation and compilation stage, using the :mod:`ast` - module. + The parser module is deprecated and will be removed in future versions of + Python. For the majority of use cases you can leverage the Abstract Syntax + Tree (AST) generation and compilation stage, using the :mod:`ast` module. There are a few things to note about this module which are important to making use of the data structures created. This is not a tutorial on editing the parse diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 85e254f061eb..273fd2b50d59 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -142,6 +142,10 @@ Deprecated Python versions it will raise a :exc:`TypeError` for all floats. (Contributed by Serhiy Storchaka in :issue:`37315`.) +* The :mod:`parser` module is deprecated and will be removed in future versions + of Python. For the majority of use cases users can leverage the Abstract Syntax + Tree (AST) generation and compilation stage, using the :mod:`ast` module. + Removed ======= diff --git a/Lib/test/test_parser.py b/Lib/test/test_parser.py index e5285c636022..ec1845d7fe9a 100644 --- a/Lib/test/test_parser.py +++ b/Lib/test/test_parser.py @@ -6,6 +6,7 @@ import struct from test import support from test.support.script_helper import assert_python_failure +from test.support.script_helper import assert_python_ok # # First, we test that we can generate trees from valid source fragments, @@ -987,5 +988,13 @@ def test_two_args_to_expr(self): with self.assertRaises(TypeError): parser.expr("a", "b") + +class TestDeprecation(unittest.TestCase): + def test_deprecation_message(self): + code = "def f():\n import parser\n\nf()" + rc, out, err = assert_python_ok('-c', code) + self.assertIn(b'<string>:2: DeprecationWarning', err) + + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/Library/2019-07-30-01-27-29.bpo-37268.QDmA44.rst b/Misc/NEWS.d/next/Library/2019-07-30-01-27-29.bpo-37268.QDmA44.rst new file mode 100644 index 000000000000..d5c7b1d2fc66 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-30-01-27-29.bpo-37268.QDmA44.rst @@ -0,0 +1,2 @@ +The :mod:`parser` module is deprecated and will be removed in future +versions of Python. diff --git a/Modules/parsermodule.c b/Modules/parsermodule.c index 079d00f32aa6..b2495fc8b322 100644 --- a/Modules/parsermodule.c +++ b/Modules/parsermodule.c @@ -1158,6 +1158,12 @@ PyInit_parser(void) { PyObject *module, *copyreg; + if (PyErr_WarnEx(PyExc_DeprecationWarning, + "The parser module is deprecated and will be removed " + "in future versions of Python", 7) != 0) { + return NULL; + } + if (PyType_Ready(&PyST_Type) < 0) return NULL; module = PyModule_Create(&parsermodule); [View Less]

1 0

bpo-34162: Update idlelib/news.txt for 3.7.5 (GH-15016)
by Terry Jan Reedy July 29, 2019

July 29, 2019

https://github.com/python/cpython/commit/e941cbcf8c2fce59a9afa8043948f9a147… commit: e941cbcf8c2fce59a9afa8043948f9a147e09177 branch: 3.7 author: Terry Jan Reedy <tjreedy(a)udel.edu> committer: GitHub <noreply(a)github.com> date: 2019-07-29T19:21:11-04:00 summary: bpo-34162: Update idlelib/news.txt for 3.7.5 (GH-15016) files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index fc9f0fdee2c3..2b223e1c8b7a 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/… [View More]

1 0

bpo-34162: Update idlelib/news.txt. (GH-15011)
by Miss Islington (bot) July 29, 2019

July 29, 2019

https://github.com/python/cpython/commit/8b50e3e2729190d4b65ee9510d81f01bd3… commit: 8b50e3e2729190d4b65ee9510d81f01bd31f2f7c branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2019-07-29T15:39:49-07:00 summary: bpo-34162: Update idlelib/news.txt. (GH-15011) (cherry picked from commit f35c51d2eadd297bcf06d4f7c536bd1d8682b724) Co-authored-by: Terry Jan Reedy <tjreedy(a)udel.edu> files: … [View More]

1 0