From webhook-mailer at python.org Thu Aug 1 01:43:11 2019 From: webhook-mailer at python.org (Ronald Oussoren) Date: Thu, 01 Aug 2019 05:43:11 -0000 Subject: [Python-checkins] bpo-18049: Sync thread stack size to main thread size on macOS (GH-14748) Message-ID: https://github.com/python/cpython/commit/1a057bab0f18d6ad843ce321d1d77a4819497ae4 commit: 1a057bab0f18d6ad843ce321d1d77a4819497ae4 branch: master author: Ronald Oussoren committer: GitHub date: 2019-08-01T07:43:07+02:00 summary: bpo-18049: Sync thread stack size to main thread size on macOS (GH-14748) This changeset increases the default size of the stack for threads on macOS to the size of the stack of the main thread and reenables the relevant recursion test. files: A Misc/NEWS.d/next/macOS/2019-07-13-15-58-18.bpo-18049.MklhQQ.rst M Lib/test/test_threading.py M Python/thread_pthread.h M configure M configure.ac diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index a99b8eca2be7..1466d25e9482 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -1057,8 +1057,6 @@ def test_releasing_unacquired_lock(self): lock = threading.Lock() self.assertRaises(RuntimeError, lock.release) - @unittest.skipUnless(sys.platform == 'darwin' and test.support.python_is_optimized(), - 'test macosx problem') def test_recursion_limit(self): # Issue 9670 # test that excessive recursion within a non-main thread causes diff --git a/Misc/NEWS.d/next/macOS/2019-07-13-15-58-18.bpo-18049.MklhQQ.rst b/Misc/NEWS.d/next/macOS/2019-07-13-15-58-18.bpo-18049.MklhQQ.rst new file mode 100644 index 000000000000..5af07cdb4119 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2019-07-13-15-58-18.bpo-18049.MklhQQ.rst @@ -0,0 +1,3 @@ +Increase the default stack size of threads from 5MB to 16MB on macOS, to +match the stack size of the main thread. This avoids crashes on deep recursion +in threads. diff --git a/Python/thread_pthread.h b/Python/thread_pthread.h index a36d16c19ead..994e35b2cc08 100644 --- a/Python/thread_pthread.h +++ b/Python/thread_pthread.h @@ -40,7 +40,8 @@ */ #if defined(__APPLE__) && defined(THREAD_STACK_SIZE) && THREAD_STACK_SIZE == 0 #undef THREAD_STACK_SIZE -#define THREAD_STACK_SIZE 0x500000 +/* Note: This matches the value of -Wl,-stack_size in configure.ac */ +#define THREAD_STACK_SIZE 0x1000000 #endif #if defined(__FreeBSD__) && defined(THREAD_STACK_SIZE) && THREAD_STACK_SIZE == 0 #undef THREAD_STACK_SIZE diff --git a/configure b/configure index 4cea98e36528..bffb849b797b 100755 --- a/configure +++ b/configure @@ -9542,6 +9542,8 @@ then # Issue #18075: the default maximum stack size (8MBytes) is too # small for the default recursion limit. Increase the stack size # to ensure that tests don't crash + # Note: This matches the value of THREAD_STACK_SIZE in + # thread_pthread.h LINKFORSHARED="-Wl,-stack_size,1000000 $LINKFORSHARED" if test "$enable_framework" diff --git a/configure.ac b/configure.ac index b9759e12f89f..8566d4338469 100644 --- a/configure.ac +++ b/configure.ac @@ -2694,6 +2694,8 @@ then # Issue #18075: the default maximum stack size (8MBytes) is too # small for the default recursion limit. Increase the stack size # to ensure that tests don't crash + # Note: This matches the value of THREAD_STACK_SIZE in + # thread_pthread.h LINKFORSHARED="-Wl,-stack_size,1000000 $LINKFORSHARED" if test "$enable_framework" From webhook-mailer at python.org Thu Aug 1 09:18:11 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Thu, 01 Aug 2019 13:18:11 -0000 Subject: [Python-checkins] bpo-36302: Sort list of sources (GH-12341) Message-ID: https://github.com/python/cpython/commit/0d30ae1a03102de07758650af9243fd31211325a commit: 0d30ae1a03102de07758650af9243fd31211325a branch: master author: Bernhard M. Wiedemann committer: Victor Stinner date: 2019-08-01T15:18:03+02:00 summary: bpo-36302: Sort list of sources (GH-12341) When building packages (e.g. for openSUSE Linux) (random) filesystem order of input files influences ordering of functions in the output .so files. Thus without the patch, builds (in disposable VMs) would usually differ. Without this patch, all callers have to be patched individually https://github.com/dugsong/libdnet/pull/42 https://github.com/sass/libsass-python/pull/212 https://github.com/tahoe-lafs/pycryptopp/pull/41 https://github.com/yt-project/yt/pull/2206 https://github.com/pyproj4/pyproj/pull/142 https://github.com/pytries/datrie/pull/49 https://github.com/Roche/pyreadstat/pull/37 but that is an infinite effort. See https://reproducible-builds.org/ for why this matters. files: A Misc/NEWS.d/next/Library/2019-03-21-19-23-46.bpo-36302.Yc591g.rst M Lib/distutils/command/build_ext.py diff --git a/Lib/distutils/command/build_ext.py b/Lib/distutils/command/build_ext.py index 2d7cdf063f01..38bb8fd93c27 100644 --- a/Lib/distutils/command/build_ext.py +++ b/Lib/distutils/command/build_ext.py @@ -490,7 +490,8 @@ def build_extension(self, ext): "in 'ext_modules' option (extension '%s'), " "'sources' must be present and must be " "a list of source filenames" % ext.name) - sources = list(sources) + # sort to make the resulting .so file build reproducible + sources = sorted(sources) ext_path = self.get_ext_fullpath(ext.name) depends = sources + ext.depends diff --git a/Misc/NEWS.d/next/Library/2019-03-21-19-23-46.bpo-36302.Yc591g.rst b/Misc/NEWS.d/next/Library/2019-03-21-19-23-46.bpo-36302.Yc591g.rst new file mode 100644 index 000000000000..fe01b5915d5d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-03-21-19-23-46.bpo-36302.Yc591g.rst @@ -0,0 +1,2 @@ +distutils sorts source file lists so that Extension .so files +build more reproducibly by default From webhook-mailer at python.org Thu Aug 1 10:17:58 2019 From: webhook-mailer at python.org (Guido van Rossum) Date: Thu, 01 Aug 2019 14:17:58 -0000 Subject: [Python-checkins] bpo-37726: Prefer argparse over getopt in stdlib tutorial (#15052) Message-ID: https://github.com/python/cpython/commit/2491134029b195d3159a489e1803ee22a7839b41 commit: 2491134029b195d3159a489e1803ee22a7839b41 branch: master author: mental committer: Guido van Rossum date: 2019-08-01T07:17:30-07:00 summary: bpo-37726: Prefer argparse over getopt in stdlib tutorial (#15052) files: A Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst M Doc/tutorial/stdlib.rst diff --git a/Doc/tutorial/stdlib.rst b/Doc/tutorial/stdlib.rst index e030f8f19b48..f32063e7f096 100644 --- a/Doc/tutorial/stdlib.rst +++ b/Doc/tutorial/stdlib.rst @@ -72,10 +72,21 @@ three`` at the command line:: >>> print(sys.argv) ['demo.py', 'one', 'two', 'three'] -The :mod:`getopt` module processes *sys.argv* using the conventions of the Unix -:func:`getopt` function. More powerful and flexible command line processing is -provided by the :mod:`argparse` module. - +The :mod:`argparse` module provides a mechanism to process command line arguments. +It should always be preferred over directly processing ``sys.argv`` manually. + +Take, for example, the below snippet of code:: + + >>> import argparse + >>> from getpass import getuser + >>> parser = argparse.ArgumentParser(description='An argparse example.') + >>> parser.add_argument('name', nargs='?', default=getuser(), help='The name of someone to greet.') + >>> parser.add_argument('--verbose', '-v', action='count') + >>> args = parser.parse_args() + >>> greeting = ["Hi", "Hello", "Greetings! its very nice to meet you"][args.verbose % 3] + >>> print(f'{greeting}, {args.name}') + >>> if not args.verbose: + >>> print('Try running this again with multiple "-v" flags!') .. _tut-stderr: diff --git a/Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst b/Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst new file mode 100644 index 000000000000..195e9755a43c --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst @@ -0,0 +1,2 @@ +Stop recommending getopt in the tutorial for command line argument parsing +and promote argparse. From webhook-mailer at python.org Thu Aug 1 10:35:05 2019 From: webhook-mailer at python.org (Guido van Rossum) Date: Thu, 01 Aug 2019 14:35:05 -0000 Subject: [Python-checkins] bpo-37726: Prefer argparse over getopt in stdlib tutorial (GH-15052) (#15070) Message-ID: https://github.com/python/cpython/commit/dcc53ebbff384076b0763c1f842966c189417071 commit: dcc53ebbff384076b0763c1f842966c189417071 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Guido van Rossum date: 2019-08-01T07:34:57-07:00 summary: bpo-37726: Prefer argparse over getopt in stdlib tutorial (GH-15052) (#15070) (cherry picked from commit 2491134029b195d3159a489e1803ee22a7839b41) Co-authored-by: mental files: A Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst M Doc/tutorial/stdlib.rst diff --git a/Doc/tutorial/stdlib.rst b/Doc/tutorial/stdlib.rst index 82261a651348..20ad3563ca05 100644 --- a/Doc/tutorial/stdlib.rst +++ b/Doc/tutorial/stdlib.rst @@ -72,10 +72,21 @@ three`` at the command line:: >>> print(sys.argv) ['demo.py', 'one', 'two', 'three'] -The :mod:`getopt` module processes *sys.argv* using the conventions of the Unix -:func:`getopt` function. More powerful and flexible command line processing is -provided by the :mod:`argparse` module. - +The :mod:`argparse` module provides a mechanism to process command line arguments. +It should always be preferred over directly processing ``sys.argv`` manually. + +Take, for example, the below snippet of code:: + + >>> import argparse + >>> from getpass import getuser + >>> parser = argparse.ArgumentParser(description='An argparse example.') + >>> parser.add_argument('name', nargs='?', default=getuser(), help='The name of someone to greet.') + >>> parser.add_argument('--verbose', '-v', action='count') + >>> args = parser.parse_args() + >>> greeting = ["Hi", "Hello", "Greetings! its very nice to meet you"][args.verbose % 3] + >>> print(f'{greeting}, {args.name}') + >>> if not args.verbose: + >>> print('Try running this again with multiple "-v" flags!') .. _tut-stderr: diff --git a/Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst b/Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst new file mode 100644 index 000000000000..195e9755a43c --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst @@ -0,0 +1,2 @@ +Stop recommending getopt in the tutorial for command line argument parsing +and promote argparse. From webhook-mailer at python.org Thu Aug 1 10:35:24 2019 From: webhook-mailer at python.org (Guido van Rossum) Date: Thu, 01 Aug 2019 14:35:24 -0000 Subject: [Python-checkins] bpo-37726: Prefer argparse over getopt in stdlib tutorial (GH-15052) (#15069) Message-ID: https://github.com/python/cpython/commit/8990ac0ab0398bfb9c62031288030fe7c630c2c7 commit: 8990ac0ab0398bfb9c62031288030fe7c630c2c7 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Guido van Rossum date: 2019-08-01T07:35:20-07:00 summary: bpo-37726: Prefer argparse over getopt in stdlib tutorial (GH-15052) (#15069) (cherry picked from commit 2491134029b195d3159a489e1803ee22a7839b41) Co-authored-by: mental files: A Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst M Doc/tutorial/stdlib.rst diff --git a/Doc/tutorial/stdlib.rst b/Doc/tutorial/stdlib.rst index f5ec8acf58ad..26f11950e79e 100644 --- a/Doc/tutorial/stdlib.rst +++ b/Doc/tutorial/stdlib.rst @@ -72,10 +72,21 @@ three`` at the command line:: >>> print(sys.argv) ['demo.py', 'one', 'two', 'three'] -The :mod:`getopt` module processes *sys.argv* using the conventions of the Unix -:func:`getopt` function. More powerful and flexible command line processing is -provided by the :mod:`argparse` module. - +The :mod:`argparse` module provides a mechanism to process command line arguments. +It should always be preferred over directly processing ``sys.argv`` manually. + +Take, for example, the below snippet of code:: + + >>> import argparse + >>> from getpass import getuser + >>> parser = argparse.ArgumentParser(description='An argparse example.') + >>> parser.add_argument('name', nargs='?', default=getuser(), help='The name of someone to greet.') + >>> parser.add_argument('--verbose', '-v', action='count') + >>> args = parser.parse_args() + >>> greeting = ["Hi", "Hello", "Greetings! its very nice to meet you"][args.verbose % 3] + >>> print(f'{greeting}, {args.name}') + >>> if not args.verbose: + >>> print('Try running this again with multiple "-v" flags!') .. _tut-stderr: diff --git a/Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst b/Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst new file mode 100644 index 000000000000..195e9755a43c --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-07-31-11-40-06.bpo-37726.h-3o9a.rst @@ -0,0 +1,2 @@ +Stop recommending getopt in the tutorial for command line argument parsing +and promote argparse. From webhook-mailer at python.org Thu Aug 1 10:39:01 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 01 Aug 2019 14:39:01 -0000 Subject: [Python-checkins] bpo-18049: Sync thread stack size to main thread size on macOS (GH-14748) Message-ID: https://github.com/python/cpython/commit/8399641c34d8136c3151fda6461cc4727a20b28e commit: 8399641c34d8136c3151fda6461cc4727a20b28e branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-01T07:38:57-07:00 summary: bpo-18049: Sync thread stack size to main thread size on macOS (GH-14748) This changeset increases the default size of the stack for threads on macOS to the size of the stack of the main thread and reenables the relevant recursion test. (cherry picked from commit 1a057bab0f18d6ad843ce321d1d77a4819497ae4) Co-authored-by: Ronald Oussoren files: A Misc/NEWS.d/next/macOS/2019-07-13-15-58-18.bpo-18049.MklhQQ.rst M Lib/test/test_threading.py M Python/thread_pthread.h M configure M configure.ac diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index 0a0a62bdf9bf..ac4e7a7e0f53 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -1049,8 +1049,6 @@ def test_releasing_unacquired_lock(self): lock = threading.Lock() self.assertRaises(RuntimeError, lock.release) - @unittest.skipUnless(sys.platform == 'darwin' and test.support.python_is_optimized(), - 'test macosx problem') def test_recursion_limit(self): # Issue 9670 # test that excessive recursion within a non-main thread causes diff --git a/Misc/NEWS.d/next/macOS/2019-07-13-15-58-18.bpo-18049.MklhQQ.rst b/Misc/NEWS.d/next/macOS/2019-07-13-15-58-18.bpo-18049.MklhQQ.rst new file mode 100644 index 000000000000..5af07cdb4119 --- /dev/null +++ b/Misc/NEWS.d/next/macOS/2019-07-13-15-58-18.bpo-18049.MklhQQ.rst @@ -0,0 +1,3 @@ +Increase the default stack size of threads from 5MB to 16MB on macOS, to +match the stack size of the main thread. This avoids crashes on deep recursion +in threads. diff --git a/Python/thread_pthread.h b/Python/thread_pthread.h index a36d16c19ead..994e35b2cc08 100644 --- a/Python/thread_pthread.h +++ b/Python/thread_pthread.h @@ -40,7 +40,8 @@ */ #if defined(__APPLE__) && defined(THREAD_STACK_SIZE) && THREAD_STACK_SIZE == 0 #undef THREAD_STACK_SIZE -#define THREAD_STACK_SIZE 0x500000 +/* Note: This matches the value of -Wl,-stack_size in configure.ac */ +#define THREAD_STACK_SIZE 0x1000000 #endif #if defined(__FreeBSD__) && defined(THREAD_STACK_SIZE) && THREAD_STACK_SIZE == 0 #undef THREAD_STACK_SIZE diff --git a/configure b/configure index cb5f130d38e0..3cd9b8866c72 100755 --- a/configure +++ b/configure @@ -9542,6 +9542,8 @@ then # Issue #18075: the default maximum stack size (8MBytes) is too # small for the default recursion limit. Increase the stack size # to ensure that tests don't crash + # Note: This matches the value of THREAD_STACK_SIZE in + # thread_pthread.h LINKFORSHARED="-Wl,-stack_size,1000000 $LINKFORSHARED" if test "$enable_framework" diff --git a/configure.ac b/configure.ac index b31ed242f1a8..a2088897fc13 100644 --- a/configure.ac +++ b/configure.ac @@ -2694,6 +2694,8 @@ then # Issue #18075: the default maximum stack size (8MBytes) is too # small for the default recursion limit. Increase the stack size # to ensure that tests don't crash + # Note: This matches the value of THREAD_STACK_SIZE in + # thread_pthread.h LINKFORSHARED="-Wl,-stack_size,1000000 $LINKFORSHARED" if test "$enable_framework" From webhook-mailer at python.org Thu Aug 1 12:36:51 2019 From: webhook-mailer at python.org (Ned Deily) Date: Thu, 01 Aug 2019 16:36:51 -0000 Subject: [Python-checkins] bpo-37461: Fix infinite loop in parsing of specially crafted email headers (GH-14794) (GH-14817) Message-ID: https://github.com/python/cpython/commit/1789bbdd3e03023951a39933ef12dee0a03be616 commit: 1789bbdd3e03023951a39933ef12dee0a03be616 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Ned Deily date: 2019-08-01T12:36:46-04:00 summary: bpo-37461: Fix infinite loop in parsing of specially crafted email headers (GH-14794) (GH-14817) Some crafted email header would cause the get_parameter method to run in an infinite loop causing a DoS attack surface when parsing those headers. This patch fixes that by making sure the DQUOTE character is handled to prevent going into an infinite loop. (cherry picked from commit a4a994bd3e619cbaff97610a1cee8ffa87c672f5) Co-authored-by: Abhilash Raj files: A Misc/NEWS.d/next/Security/2019-07-16-08-11-00.bpo-37461.1Ahz7O.rst M Lib/email/_header_value_parser.py M Lib/test/test_email/test__header_value_parser.py diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py index e3c343dffb42..737951e4b1b1 100644 --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -2366,6 +2366,9 @@ def get_parameter(value): while value: if value[0] in WSP: token, value = get_fws(value) + elif value[0] == '"': + token = ValueTerminal('"', 'DQUOTE') + value = value[1:] else: token, value = get_qcontent(value) v.append(token) diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py index 676732bb3d02..a2c900fa7fd2 100644 --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -2601,6 +2601,13 @@ def mime_parameters_as_value(self, # Defects are apparent missing *0*, and two 'out of sequence'. [errors.InvalidHeaderDefect]*3), + # bpo-37461: Check that we don't go into an infinite loop. + 'extra_dquote': ( + 'r*="\'a\'\\"', + ' r="\\""', + 'r*=\'a\'"', + [('r', '"')], + [errors.InvalidHeaderDefect]*2), } @parameterize diff --git a/Misc/NEWS.d/next/Security/2019-07-16-08-11-00.bpo-37461.1Ahz7O.rst b/Misc/NEWS.d/next/Security/2019-07-16-08-11-00.bpo-37461.1Ahz7O.rst new file mode 100644 index 000000000000..4bfd350c0b40 --- /dev/null +++ b/Misc/NEWS.d/next/Security/2019-07-16-08-11-00.bpo-37461.1Ahz7O.rst @@ -0,0 +1,2 @@ +Fix an inifite loop when parsing specially crafted email headers. Patch by +Abhilash Raj. From webhook-mailer at python.org Fri Aug 2 00:57:31 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 02 Aug 2019 04:57:31 -0000 Subject: [Python-checkins] bpo-16970: Adding error message for invalid args (GH-14844) Message-ID: https://github.com/python/cpython/commit/4b3e97592376d5f8a3b75192b399a2da1be642cb commit: 4b3e97592376d5f8a3b75192b399a2da1be642cb branch: master author: tmblweed committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-01T21:57:13-07:00 summary: bpo-16970: Adding error message for invalid args (GH-14844) BPO -16970: Adding error message for invalid args Applied the patch argparse-v2 patch issue 16970, ran patch check and the test suite, test_argparse with 0 errors https://bugs.python.org/issue16970 files: A Misc/NEWS.d/next/Library/2019-07-19-01-46-56.bpo-16970.GEASf5.rst M Lib/argparse.py M Lib/test/test_argparse.py M Misc/ACKS diff --git a/Lib/argparse.py b/Lib/argparse.py index e45b67b67704..a300828f9e3d 100644 --- a/Lib/argparse.py +++ b/Lib/argparse.py @@ -593,7 +593,10 @@ def _format_args(self, action, default_metavar): elif action.nargs == SUPPRESS: result = '' else: - formats = ['%s' for _ in range(action.nargs)] + try: + formats = ['%s' for _ in range(action.nargs)] + except TypeError: + raise ValueError("invalid nargs value") from None result = ' '.join(formats) % get_metavar(action.nargs) return result @@ -850,7 +853,7 @@ def __init__(self, help=None, metavar=None): if nargs == 0: - raise ValueError('nargs for store actions must be > 0; if you ' + raise ValueError('nargs for store actions must be != 0; if you ' 'have nothing to store, actions such as store ' 'true or store const may be more appropriate') if const is not None and nargs != OPTIONAL: @@ -942,7 +945,7 @@ def __init__(self, help=None, metavar=None): if nargs == 0: - raise ValueError('nargs for append actions must be > 0; if arg ' + raise ValueError('nargs for append actions must be != 0; if arg ' 'strings are not supplying the value to append, ' 'the append const action may be more appropriate') if const is not None and nargs != OPTIONAL: diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py index 5128dc5e1be4..d6d16090eb02 100644 --- a/Lib/test/test_argparse.py +++ b/Lib/test/test_argparse.py @@ -4263,7 +4263,6 @@ class TestHelpSubparsersWithHelpOrdering(HelpTestCase): class TestHelpMetavarTypeFormatter(HelpTestCase): - """""" def custom_type(string): return string @@ -5150,6 +5149,35 @@ def test_nargs_3_metavar_length2(self): def test_nargs_3_metavar_length3(self): self.do_test_no_exception(nargs=3, metavar=("1", "2", "3")) + +class TestInvalidNargs(TestCase): + + EXPECTED_INVALID_MESSAGE = "invalid nargs value" + EXPECTED_RANGE_MESSAGE = ("nargs for store actions must be != 0; if you " + "have nothing to store, actions such as store " + "true or store const may be more appropriate") + + def do_test_range_exception(self, nargs): + parser = argparse.ArgumentParser() + with self.assertRaises(ValueError) as cm: + parser.add_argument("--foo", nargs=nargs) + self.assertEqual(cm.exception.args[0], self.EXPECTED_RANGE_MESSAGE) + + def do_test_invalid_exception(self, nargs): + parser = argparse.ArgumentParser() + with self.assertRaises(ValueError) as cm: + parser.add_argument("--foo", nargs=nargs) + self.assertEqual(cm.exception.args[0], self.EXPECTED_INVALID_MESSAGE) + + # Unit tests for different values of nargs + + def test_nargs_alphabetic(self): + self.do_test_invalid_exception(nargs='a') + self.do_test_invalid_exception(nargs="abcd") + + def test_nargs_zero(self): + self.do_test_range_exception(nargs=0) + # ============================ # from argparse import * tests # ============================ diff --git a/Misc/ACKS b/Misc/ACKS index 6463b535e17c..82fc51c98c40 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1878,3 +1878,4 @@ Edison Abahurire Geoff Shannon Batuhan Taskaya Aleksandr Balezin +Robert Leenders diff --git a/Misc/NEWS.d/next/Library/2019-07-19-01-46-56.bpo-16970.GEASf5.rst b/Misc/NEWS.d/next/Library/2019-07-19-01-46-56.bpo-16970.GEASf5.rst new file mode 100644 index 000000000000..7285b8176032 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-19-01-46-56.bpo-16970.GEASf5.rst @@ -0,0 +1,2 @@ +Adding a value error when an invalid value in passed to nargs +Patch by Robert Leenders From webhook-mailer at python.org Fri Aug 2 01:16:49 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 02 Aug 2019 05:16:49 -0000 Subject: [Python-checkins] bpo-16970: Adding error message for invalid args (GH-14844) Message-ID: https://github.com/python/cpython/commit/1cc70322c99b80c123f9ff23a415d3da28b4ec74 commit: 1cc70322c99b80c123f9ff23a415d3da28b4ec74 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-01T22:16:44-07:00 summary: bpo-16970: Adding error message for invalid args (GH-14844) BPO -16970: Adding error message for invalid args Applied the patch argparse-v2 patch issue 16970, ran patch check and the test suite, test_argparse with 0 errors https://bugs.python.org/issue16970 (cherry picked from commit 4b3e97592376d5f8a3b75192b399a2da1be642cb) Co-authored-by: tmblweed files: A Misc/NEWS.d/next/Library/2019-07-19-01-46-56.bpo-16970.GEASf5.rst M Lib/argparse.py M Lib/test/test_argparse.py M Misc/ACKS diff --git a/Lib/argparse.py b/Lib/argparse.py index 5820d0d8ca99..6f0b37c1c5ea 100644 --- a/Lib/argparse.py +++ b/Lib/argparse.py @@ -593,7 +593,10 @@ def _format_args(self, action, default_metavar): elif action.nargs == SUPPRESS: result = '' else: - formats = ['%s' for _ in range(action.nargs)] + try: + formats = ['%s' for _ in range(action.nargs)] + except TypeError: + raise ValueError("invalid nargs value") from None result = ' '.join(formats) % get_metavar(action.nargs) return result @@ -850,7 +853,7 @@ def __init__(self, help=None, metavar=None): if nargs == 0: - raise ValueError('nargs for store actions must be > 0; if you ' + raise ValueError('nargs for store actions must be != 0; if you ' 'have nothing to store, actions such as store ' 'true or store const may be more appropriate') if const is not None and nargs != OPTIONAL: @@ -942,7 +945,7 @@ def __init__(self, help=None, metavar=None): if nargs == 0: - raise ValueError('nargs for append actions must be > 0; if arg ' + raise ValueError('nargs for append actions must be != 0; if arg ' 'strings are not supplying the value to append, ' 'the append const action may be more appropriate') if const is not None and nargs != OPTIONAL: diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py index 5128dc5e1be4..d6d16090eb02 100644 --- a/Lib/test/test_argparse.py +++ b/Lib/test/test_argparse.py @@ -4263,7 +4263,6 @@ class TestHelpSubparsersWithHelpOrdering(HelpTestCase): class TestHelpMetavarTypeFormatter(HelpTestCase): - """""" def custom_type(string): return string @@ -5150,6 +5149,35 @@ def test_nargs_3_metavar_length2(self): def test_nargs_3_metavar_length3(self): self.do_test_no_exception(nargs=3, metavar=("1", "2", "3")) + +class TestInvalidNargs(TestCase): + + EXPECTED_INVALID_MESSAGE = "invalid nargs value" + EXPECTED_RANGE_MESSAGE = ("nargs for store actions must be != 0; if you " + "have nothing to store, actions such as store " + "true or store const may be more appropriate") + + def do_test_range_exception(self, nargs): + parser = argparse.ArgumentParser() + with self.assertRaises(ValueError) as cm: + parser.add_argument("--foo", nargs=nargs) + self.assertEqual(cm.exception.args[0], self.EXPECTED_RANGE_MESSAGE) + + def do_test_invalid_exception(self, nargs): + parser = argparse.ArgumentParser() + with self.assertRaises(ValueError) as cm: + parser.add_argument("--foo", nargs=nargs) + self.assertEqual(cm.exception.args[0], self.EXPECTED_INVALID_MESSAGE) + + # Unit tests for different values of nargs + + def test_nargs_alphabetic(self): + self.do_test_invalid_exception(nargs='a') + self.do_test_invalid_exception(nargs="abcd") + + def test_nargs_zero(self): + self.do_test_range_exception(nargs=0) + # ============================ # from argparse import * tests # ============================ diff --git a/Misc/ACKS b/Misc/ACKS index ad2e0a1996aa..3e53429bf237 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1872,3 +1872,4 @@ Edison Abahurire Geoff Shannon Batuhan Taskaya Aleksandr Balezin +Robert Leenders diff --git a/Misc/NEWS.d/next/Library/2019-07-19-01-46-56.bpo-16970.GEASf5.rst b/Misc/NEWS.d/next/Library/2019-07-19-01-46-56.bpo-16970.GEASf5.rst new file mode 100644 index 000000000000..7285b8176032 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-19-01-46-56.bpo-16970.GEASf5.rst @@ -0,0 +1,2 @@ +Adding a value error when an invalid value in passed to nargs +Patch by Robert Leenders From webhook-mailer at python.org Fri Aug 2 03:25:53 2019 From: webhook-mailer at python.org (Inada Naoki) Date: Fri, 02 Aug 2019 07:25:53 -0000 Subject: [Python-checkins] bpo-37729: gc: write stats at once (GH-15050) Message-ID: https://github.com/python/cpython/commit/bf8162c8c45338470bbe487c8769bba20bde66c2 commit: bf8162c8c45338470bbe487c8769bba20bde66c2 branch: master author: Inada Naoki committer: GitHub date: 2019-08-02T16:25:29+09:00 summary: bpo-37729: gc: write stats at once (GH-15050) gc used several PySys_WriteStderr() calls to write stats. It caused stats mixed up when stderr is shared by multiple processes like this: gc: collecting generation 2... gc: objects in each generation: 0 0gc: collecting generation 2... gc: objects in each generation: 0 0 126077 126077 gc: objects in permanent generation: 0 gc: objects in permanent generation: 0 gc: done, 112575 unreachable, 0 uncollectablegc: done, 112575 unreachable, 0 uncollectable, 0.2223s elapsed , 0.2344s elapsed files: M Modules/gcmodule.c diff --git a/Modules/gcmodule.c b/Modules/gcmodule.c index 2ee765140b92..7586cd3e0515 100644 --- a/Modules/gcmodule.c +++ b/Modules/gcmodule.c @@ -962,6 +962,25 @@ clear_freelists(void) (void)PyContext_ClearFreeList(); } +// Show stats for objects in each gennerations. +static void +show_stats_each_generations(struct _gc_runtime_state *state) +{ + char buf[100]; + size_t pos = 0; + + for (int i = 0; i < NUM_GENERATIONS && pos < sizeof(buf); i++) { + pos += PyOS_snprintf(buf+pos, sizeof(buf)-pos, + " %"PY_FORMAT_SIZE_T"d", + gc_list_size(GEN_HEAD(state, i))); + } + + PySys_FormatStderr( + "gc: objects in each generation:%s\n" + "gc: objects in permanent generation: %zd\n", + buf, gc_list_size(&state->permanent_generation.head)); +} + /* This is the main function. Read this to understand how the * collection process works. */ static Py_ssize_t @@ -979,17 +998,9 @@ collect(struct _gc_runtime_state *state, int generation, _PyTime_t t1 = 0; /* initialize to prevent a compiler warning */ if (state->debug & DEBUG_STATS) { - PySys_WriteStderr("gc: collecting generation %d...\n", - generation); - PySys_WriteStderr("gc: objects in each generation:"); - for (i = 0; i < NUM_GENERATIONS; i++) - PySys_FormatStderr(" %zd", - gc_list_size(GEN_HEAD(state, i))); - PySys_WriteStderr("\ngc: objects in permanent generation: %zd", - gc_list_size(&state->permanent_generation.head)); + PySys_WriteStderr("gc: collecting generation %d...\n", generation); + show_stats_each_generations(state); t1 = _PyTime_GetMonotonicClock(); - - PySys_WriteStderr("\n"); } if (PyDTrace_GC_START_ENABLED()) @@ -1103,16 +1114,10 @@ collect(struct _gc_runtime_state *state, int generation, debug_cycle("uncollectable", FROM_GC(gc)); } if (state->debug & DEBUG_STATS) { - _PyTime_t t2 = _PyTime_GetMonotonicClock(); - - if (m == 0 && n == 0) - PySys_WriteStderr("gc: done"); - else - PySys_FormatStderr( - "gc: done, %zd unreachable, %zd uncollectable", - n+m, n); - PySys_WriteStderr(", %.4fs elapsed\n", - _PyTime_AsSecondsDouble(t2 - t1)); + double d = _PyTime_AsSecondsDouble(_PyTime_GetMonotonicClock() - t1); + PySys_FormatStderr( + "gc: done, %zd unreachable, %zd uncollectable, %.4fs elapsed\n", + n+m, n, d); } /* Append instances in the uncollectable set to a Python From webhook-mailer at python.org Fri Aug 2 11:50:31 2019 From: webhook-mailer at python.org (Eric Snow) Date: Fri, 02 Aug 2019 15:50:31 -0000 Subject: [Python-checkins] bpo-36487: Make C-API docs clear about what the main interpreter is. (gh-12666) Message-ID: https://github.com/python/cpython/commit/854d0a4b98b13629252e21edaf2b785b429e5135 commit: 854d0a4b98b13629252e21edaf2b785b429e5135 branch: master author: Joannah Nanjekye <33177550+nanjekyejoannah at users.noreply.github.com> committer: Eric Snow date: 2019-08-02T09:50:22-06:00 summary: bpo-36487: Make C-API docs clear about what the main interpreter is. (gh-12666) files: A Misc/NEWS.d/next/Documentation/2019-04-02-19-23-00.bpo-36487.Jg6-MG.rst M Doc/c-api/init.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index a0ac4d21d139..4985a1d867e2 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1141,10 +1141,18 @@ Sub-interpreter support While in most uses, you will only embed a single Python interpreter, there are cases where you need to create several independent interpreters in the -same process and perhaps even in the same thread. Sub-interpreters allow -you to do that. You can switch between sub-interpreters using the -:c:func:`PyThreadState_Swap` function. You can create and destroy them -using the following functions: +same process and perhaps even in the same thread. Sub-interpreters allow +you to do that. + +The "main" interpreter is the first one created when the runtime initializes. +It is usually the only Python interpreter in a process. Unlike sub-interpreters, +the main interpreter has unique process-global responsibilities like signal +handling. It is also responsible for execution during runtime initialization and +is usually the active interpreter during runtime finalization. The +:c:func:`PyInterpreterState_Main` funtion returns a pointer to its state. + +You can switch between sub-interpreters using the :c:func:`PyThreadState_Swap` +function. You can create and destroy them using the following functions: .. c:function:: PyThreadState* Py_NewInterpreter() diff --git a/Misc/NEWS.d/next/Documentation/2019-04-02-19-23-00.bpo-36487.Jg6-MG.rst b/Misc/NEWS.d/next/Documentation/2019-04-02-19-23-00.bpo-36487.Jg6-MG.rst new file mode 100755 index 000000000000..c8eb05b6c79c --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-04-02-19-23-00.bpo-36487.Jg6-MG.rst @@ -0,0 +1 @@ +Make C-API docs clear about what the "main" interpreter is. \ No newline at end of file From webhook-mailer at python.org Fri Aug 2 11:53:05 2019 From: webhook-mailer at python.org (Vinay Sajip) Date: Fri, 02 Aug 2019 15:53:05 -0000 Subject: [Python-checkins] =?utf-8?q?bpo-37742=3A_Return_the_root_logger_?= =?utf-8?b?d2hlbiBsb2dnaW5nLmdldExvZ2dlcigncm9vdCcpIGlzIGPigKYgKCMxNTA3?= =?utf-8?q?7=29?= Message-ID: https://github.com/python/cpython/commit/cb65b3a4f484ce71dcb76a918af98c7015513025 commit: cb65b3a4f484ce71dcb76a918af98c7015513025 branch: master author: Vinay Sajip committer: GitHub date: 2019-08-02T16:53:00+01:00 summary: bpo-37742: Return the root logger when logging.getLogger('root') is c? (#15077) * bpo-37742: Return the root logger when logging.getLogger('root') is called. * Added type check guard on logger name in logging.getLogger() and refined a test. files: A Misc/NEWS.d/next/Library/2019-08-02-14-01-25.bpo-37742.f4Xn9S.rst M Lib/logging/__init__.py M Lib/test/test_logging.py diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py index 645e0b3c3a67..62a87a71b1a3 100644 --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -2024,10 +2024,9 @@ def getLogger(name=None): If no name is specified, return the root logger. """ - if name: - return Logger.manager.getLogger(name) - else: + if not name or isinstance(name, str) and name == root.name: return root + return Logger.manager.getLogger(name) def critical(msg, *args, **kwargs): """ diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index 6507d79742a4..dca744c59092 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -4856,6 +4856,7 @@ def test_root_logger_aliases(self): self.assertIs(root, logging.root) self.assertIs(root, logging.getLogger(None)) self.assertIs(root, logging.getLogger('')) + self.assertIs(root, logging.getLogger('root')) self.assertIs(root, logging.getLogger('foo').root) self.assertIs(root, logging.getLogger('foo.bar').root) self.assertIs(root, logging.getLogger('foo').parent) diff --git a/Misc/NEWS.d/next/Library/2019-08-02-14-01-25.bpo-37742.f4Xn9S.rst b/Misc/NEWS.d/next/Library/2019-08-02-14-01-25.bpo-37742.f4Xn9S.rst new file mode 100644 index 000000000000..300ced3fec6b --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-02-14-01-25.bpo-37742.f4Xn9S.rst @@ -0,0 +1,5 @@ +The logging.getLogger() API now returns the root logger when passed the name +'root', whereas previously it returned a non-root logger named 'root'. This +could affect cases where user code explicitly wants a non-root logger named +'root', or instantiates a logger using logging.getLogger(__name__) in some +top-level module called 'root.py'. From webhook-mailer at python.org Fri Aug 2 13:49:42 2019 From: webhook-mailer at python.org (Eric Snow) Date: Fri, 02 Aug 2019 17:49:42 -0000 Subject: [Python-checkins] bpo-36487: Make C-API docs clear about what the main interpreter is. (gh-15080) Message-ID: https://github.com/python/cpython/commit/375f35be0688da0fc0f27afc4faea76590d7c24d commit: 375f35be0688da0fc0f27afc4faea76590d7c24d branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Eric Snow date: 2019-08-02T11:49:38-06:00 summary: bpo-36487: Make C-API docs clear about what the main interpreter is. (gh-15080) (cherry picked from commit 854d0a4b98b13629252e21edaf2b785b429e5135) (gh-12666) Co-authored-by: Joannah Nanjekye <33177550+nanjekyejoannah at users.noreply.github.com> files: A Misc/NEWS.d/next/Documentation/2019-04-02-19-23-00.bpo-36487.Jg6-MG.rst M Doc/c-api/init.rst diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index a0ac4d21d139..4985a1d867e2 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -1141,10 +1141,18 @@ Sub-interpreter support While in most uses, you will only embed a single Python interpreter, there are cases where you need to create several independent interpreters in the -same process and perhaps even in the same thread. Sub-interpreters allow -you to do that. You can switch between sub-interpreters using the -:c:func:`PyThreadState_Swap` function. You can create and destroy them -using the following functions: +same process and perhaps even in the same thread. Sub-interpreters allow +you to do that. + +The "main" interpreter is the first one created when the runtime initializes. +It is usually the only Python interpreter in a process. Unlike sub-interpreters, +the main interpreter has unique process-global responsibilities like signal +handling. It is also responsible for execution during runtime initialization and +is usually the active interpreter during runtime finalization. The +:c:func:`PyInterpreterState_Main` funtion returns a pointer to its state. + +You can switch between sub-interpreters using the :c:func:`PyThreadState_Swap` +function. You can create and destroy them using the following functions: .. c:function:: PyThreadState* Py_NewInterpreter() diff --git a/Misc/NEWS.d/next/Documentation/2019-04-02-19-23-00.bpo-36487.Jg6-MG.rst b/Misc/NEWS.d/next/Documentation/2019-04-02-19-23-00.bpo-36487.Jg6-MG.rst new file mode 100755 index 000000000000..c8eb05b6c79c --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-04-02-19-23-00.bpo-36487.Jg6-MG.rst @@ -0,0 +1 @@ +Make C-API docs clear about what the "main" interpreter is. \ No newline at end of file From webhook-mailer at python.org Fri Aug 2 16:30:02 2019 From: webhook-mailer at python.org (Steve Dower) Date: Fri, 02 Aug 2019 20:30:02 -0000 Subject: [Python-checkins] bpo-36590: Add Bluetooth RFCOMM and support for Windows. (GH-12767) Message-ID: https://github.com/python/cpython/commit/8fbece135d7615e836a845ca39223097046c8b8b commit: 8fbece135d7615e836a845ca39223097046c8b8b branch: master author: Greg Bowser committer: Steve Dower date: 2019-08-02T13:29:52-07:00 summary: bpo-36590: Add Bluetooth RFCOMM and support for Windows. (GH-12767) Support for RFCOMM, L2CAP, HCI, SCO is based on the BTPROTO_* macros being defined. Winsock only supports RFCOMM, even though it has a BTHPROTO_L2CAP macro. L2CAP support would build on windows, but not necessarily work. This also adds some basic unittests for constants (all of which existed prior to this commit, just not on windows) and creating sockets. pair: Nate Duarte files: A Misc/NEWS.d/next/Windows/2019-04-10-21-13-26.bpo-36590.ZTaKcu.rst M Lib/test/test_socket.py M Modules/socketmodule.c M Modules/socketmodule.h diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py index 11b2a38ad76d..ce816cd603ec 100644 --- a/Lib/test/test_socket.py +++ b/Lib/test/test_socket.py @@ -114,6 +114,19 @@ def _have_socket_vsock(): return ret +def _have_socket_bluetooth(): + """Check whether AF_BLUETOOTH sockets are supported on this host.""" + try: + # RFCOMM is supported by all platforms with bluetooth support. Windows + # does not support omitting the protocol. + s = socket.socket(socket.AF_BLUETOOTH, socket.SOCK_STREAM, socket.BTPROTO_RFCOMM) + except (AttributeError, OSError): + return False + else: + s.close() + return True + + @contextlib.contextmanager def socket_setdefaulttimeout(timeout): old_timeout = socket.getdefaulttimeout() @@ -138,6 +151,8 @@ def socket_setdefaulttimeout(timeout): HAVE_SOCKET_UDPLITE = hasattr(socket, "IPPROTO_UDPLITE") +HAVE_SOCKET_BLUETOOTH = _have_socket_bluetooth() + # Size in bytes of the int type SIZEOF_INT = array.array("i").itemsize @@ -2257,6 +2272,45 @@ def testSocketBufferSize(self): socket.SO_VM_SOCKETS_BUFFER_MIN_SIZE)) + at unittest.skipUnless(HAVE_SOCKET_BLUETOOTH, + 'Bluetooth sockets required for this test.') +class BasicBluetoothTest(unittest.TestCase): + + def testBluetoothConstants(self): + socket.BDADDR_ANY + socket.BDADDR_LOCAL + socket.AF_BLUETOOTH + socket.BTPROTO_RFCOMM + + if sys.platform != "win32": + socket.BTPROTO_HCI + socket.SOL_HCI + socket.BTPROTO_L2CAP + + if not sys.platform.startswith("freebsd"): + socket.BTPROTO_SCO + + def testCreateRfcommSocket(self): + with socket.socket(socket.AF_BLUETOOTH, socket.SOCK_STREAM, socket.BTPROTO_RFCOMM) as s: + pass + + @unittest.skipIf(sys.platform == "win32", "windows does not support L2CAP sockets") + def testCreateL2capSocket(self): + with socket.socket(socket.AF_BLUETOOTH, socket.SOCK_SEQPACKET, socket.BTPROTO_L2CAP) as s: + pass + + @unittest.skipIf(sys.platform == "win32", "windows does not support HCI sockets") + def testCreateHciSocket(self): + with socket.socket(socket.AF_BLUETOOTH, socket.SOCK_RAW, socket.BTPROTO_HCI) as s: + pass + + @unittest.skipIf(sys.platform == "win32" or sys.platform.startswith("freebsd"), + "windows and freebsd do not support SCO sockets") + def testCreateScoSocket(self): + with socket.socket(socket.AF_BLUETOOTH, socket.SOCK_SEQPACKET, socket.BTPROTO_SCO) as s: + pass + + class BasicTCPTest(SocketConnectedTest): def __init__(self, methodName='runTest'): @@ -6416,6 +6470,7 @@ def test_main(): BasicVSOCKTest, ThreadedVSOCKSocketStreamTest, ]) + tests.append(BasicBluetoothTest) tests.extend([ CmsgMacroTests, SendmsgUDPTest, diff --git a/Misc/NEWS.d/next/Windows/2019-04-10-21-13-26.bpo-36590.ZTaKcu.rst b/Misc/NEWS.d/next/Windows/2019-04-10-21-13-26.bpo-36590.ZTaKcu.rst new file mode 100644 index 000000000000..6a186bbeed74 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-04-10-21-13-26.bpo-36590.ZTaKcu.rst @@ -0,0 +1 @@ +Add native Bluetooth RFCOMM support to socket module. \ No newline at end of file diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c old mode 100644 new mode 100755 index 99e350b4333b..f220c2636319 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -520,6 +520,15 @@ remove_unusable_flags(PyObject *m) #endif #endif +#ifdef MS_WINDOWS +#define sockaddr_rc SOCKADDR_BTH_REDEF + +#define USE_BLUETOOTH 1 +#define AF_BLUETOOTH AF_BTH +#define BTPROTO_RFCOMM BTHPROTO_RFCOMM +#define _BT_RC_MEMB(sa, memb) ((sa)->memb) +#endif + /* Convert "sock_addr_t *" to "struct sockaddr *". */ #define SAS2SA(x) (&((x)->sa)) @@ -1256,12 +1265,23 @@ setbdaddr(const char *name, bdaddr_t *bdaddr) n = sscanf(name, "%X:%X:%X:%X:%X:%X%c", &b5, &b4, &b3, &b2, &b1, &b0, &ch); if (n == 6 && (b0 | b1 | b2 | b3 | b4 | b5) < 256) { + +#ifdef MS_WINDOWS + *bdaddr = (ULONGLONG)(b0 & 0xFF); + *bdaddr |= ((ULONGLONG)(b1 & 0xFF) << 8); + *bdaddr |= ((ULONGLONG)(b2 & 0xFF) << 16); + *bdaddr |= ((ULONGLONG)(b3 & 0xFF) << 24); + *bdaddr |= ((ULONGLONG)(b4 & 0xFF) << 32); + *bdaddr |= ((ULONGLONG)(b5 & 0xFF) << 40); +#else bdaddr->b[0] = b0; bdaddr->b[1] = b1; bdaddr->b[2] = b2; bdaddr->b[3] = b3; bdaddr->b[4] = b4; bdaddr->b[5] = b5; +#endif + return 6; } else { PyErr_SetString(PyExc_OSError, "bad bluetooth address"); @@ -1278,9 +1298,23 @@ makebdaddr(bdaddr_t *bdaddr) { char buf[(6 * 2) + 5 + 1]; +#ifdef MS_WINDOWS + int i; + unsigned int octets[6]; + + for (i = 0; i < 6; ++i) { + octets[i] = ((*bdaddr) >> (8 * i)) & 0xFF; + } + + sprintf(buf, "%02X:%02X:%02X:%02X:%02X:%02X", + octets[5], octets[4], octets[3], + octets[2], octets[1], octets[0]); +#else sprintf(buf, "%02X:%02X:%02X:%02X:%02X:%02X", bdaddr->b[5], bdaddr->b[4], bdaddr->b[3], bdaddr->b[2], bdaddr->b[1], bdaddr->b[0]); +#endif + return PyUnicode_FromString(buf); } #endif @@ -1378,6 +1412,7 @@ makesockaddr(SOCKET_T sockfd, struct sockaddr *addr, size_t addrlen, int proto) case AF_BLUETOOTH: switch (proto) { +#ifdef BTPROTO_L2CAP case BTPROTO_L2CAP: { struct sockaddr_l2 *a = (struct sockaddr_l2 *) addr; @@ -1392,6 +1427,8 @@ makesockaddr(SOCKET_T sockfd, struct sockaddr *addr, size_t addrlen, int proto) return ret; } +#endif /* BTPROTO_L2CAP */ + case BTPROTO_RFCOMM: { struct sockaddr_rc *a = (struct sockaddr_rc *) addr; @@ -1406,6 +1443,7 @@ makesockaddr(SOCKET_T sockfd, struct sockaddr *addr, size_t addrlen, int proto) return ret; } +#ifdef BTPROTO_HCI case BTPROTO_HCI: { struct sockaddr_hci *a = (struct sockaddr_hci *) addr; @@ -1425,6 +1463,7 @@ makesockaddr(SOCKET_T sockfd, struct sockaddr *addr, size_t addrlen, int proto) return makebdaddr(&_BT_SCO_MEMB(a, bdaddr)); } #endif /* !__FreeBSD__ */ +#endif /* BTPROTO_HCI */ default: PyErr_SetString(PyExc_ValueError, @@ -1879,6 +1918,7 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, case AF_BLUETOOTH: { switch (s->sock_proto) { +#ifdef BTPROTO_L2CAP case BTPROTO_L2CAP: { struct sockaddr_l2 *addr; @@ -1899,6 +1939,7 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, *len_ret = sizeof *addr; return 1; } +#endif /* BTPROTO_L2CAP */ case BTPROTO_RFCOMM: { struct sockaddr_rc *addr; @@ -1918,6 +1959,7 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, *len_ret = sizeof *addr; return 1; } +#ifdef BTPROTO_HCI case BTPROTO_HCI: { struct sockaddr_hci *addr = (struct sockaddr_hci *)addr_ret; @@ -1964,6 +2006,7 @@ getsockaddrarg(PySocketSockObject *s, PyObject *args, return 1; } #endif /* !__FreeBSD__ */ +#endif /* BTPROTO_HCI */ default: PyErr_Format(PyExc_OSError, "%s(): unknown Bluetooth protocol", caller); @@ -2385,12 +2428,15 @@ getsockaddrlen(PySocketSockObject *s, socklen_t *len_ret) switch(s->sock_proto) { +#ifdef BTPROTO_L2CAP case BTPROTO_L2CAP: *len_ret = sizeof (struct sockaddr_l2); return 1; +#endif /* BTPROTO_L2CAP */ case BTPROTO_RFCOMM: *len_ret = sizeof (struct sockaddr_rc); return 1; +#ifdef BTPROTO_HCI case BTPROTO_HCI: *len_ret = sizeof (struct sockaddr_hci); return 1; @@ -2399,6 +2445,7 @@ getsockaddrlen(PySocketSockObject *s, socklen_t *len_ret) *len_ret = sizeof (struct sockaddr_sco); return 1; #endif /* !__FreeBSD__ */ +#endif /* BTPROTO_HCI */ default: PyErr_SetString(PyExc_OSError, "getsockaddrlen: " "unknown BT protocol"); @@ -7273,23 +7320,29 @@ PyInit__socket(void) #ifdef USE_BLUETOOTH PyModule_AddIntMacro(m, AF_BLUETOOTH); +#ifdef BTPROTO_L2CAP PyModule_AddIntMacro(m, BTPROTO_L2CAP); +#endif /* BTPROTO_L2CAP */ +#ifdef BTPROTO_HCI PyModule_AddIntMacro(m, BTPROTO_HCI); PyModule_AddIntMacro(m, SOL_HCI); #if !defined(__NetBSD__) && !defined(__DragonFly__) PyModule_AddIntMacro(m, HCI_FILTER); -#endif #if !defined(__FreeBSD__) -#if !defined(__NetBSD__) && !defined(__DragonFly__) PyModule_AddIntMacro(m, HCI_TIME_STAMP); -#endif PyModule_AddIntMacro(m, HCI_DATA_DIR); - PyModule_AddIntMacro(m, BTPROTO_SCO); -#endif +#endif /* !__FreeBSD__ */ +#endif /* !__NetBSD__ && !__DragonFly__ */ +#endif /* BTPROTO_HCI */ +#ifdef BTPROTO_RFCOMM PyModule_AddIntMacro(m, BTPROTO_RFCOMM); +#endif /* BTPROTO_RFCOMM */ PyModule_AddStringConstant(m, "BDADDR_ANY", "00:00:00:00:00:00"); PyModule_AddStringConstant(m, "BDADDR_LOCAL", "00:00:00:FF:FF:FF"); -#endif +#ifdef BTPROTO_SCO + PyModule_AddIntMacro(m, BTPROTO_SCO); +#endif /* BTPROTO_SCO */ +#endif /* USE_BLUETOOTH */ #ifdef AF_CAN /* Controller Area Network */ diff --git a/Modules/socketmodule.h b/Modules/socketmodule.h old mode 100644 new mode 100755 index dff1f8f4e9ce..3d95fe709ac7 --- a/Modules/socketmodule.h +++ b/Modules/socketmodule.h @@ -14,6 +14,47 @@ #else /* MS_WINDOWS */ # include + +/* + * If Windows has bluetooth support, include bluetooth constants. + */ +#ifdef AF_BTH +# include +# include + +/* + * The current implementation assumes the bdaddr in the sockaddr structs + * will be a bdaddr_t. We treat this as an opaque type: on *nix systems, it + * will be a struct with a single member (an array of six bytes). On windows, + * we typedef this to ULONGLONG to match the Windows definition. + */ +typedef ULONGLONG bdaddr_t; + +/* + * Redefine SOCKADDR_BTH to provide names compatible with _BT_RC_MEMB() macros. + */ +struct SOCKADDR_BTH_REDEF { + union { + USHORT addressFamily; + USHORT family; + }; + + union { + ULONGLONG btAddr; + bdaddr_t bdaddr; + }; + + GUID serviceClassId; + + union { + ULONG port; + ULONG channel; + }; + +}; +# include +#endif + /* Windows 'supports' CMSG_LEN, but does not follow the POSIX standard * interface at all, so there is no point including the code that * attempts to use it. @@ -199,6 +240,8 @@ typedef union sock_addr { struct sockaddr_rc bt_rc; struct sockaddr_sco bt_sco; struct sockaddr_hci bt_hci; +#elif defined(MS_WINDOWS) + struct SOCKADDR_BTH_REDEF bt_rc; #endif #ifdef HAVE_NETPACKET_PACKET_H struct sockaddr_ll ll; From webhook-mailer at python.org Fri Aug 2 18:20:18 2019 From: webhook-mailer at python.org (Steve Dower) Date: Fri, 02 Aug 2019 22:20:18 -0000 Subject: [Python-checkins] bpo-20523: pdb searches for .pdbrc in ~ instead of $HOME (GH-11847) Message-ID: https://github.com/python/cpython/commit/7ea9a85f132b32347fcbd2cbe1b553a2e9890b56 commit: 7ea9a85f132b32347fcbd2cbe1b553a2e9890b56 branch: master author: Timothy Hopper committer: Steve Dower date: 2019-08-02T15:20:14-07:00 summary: bpo-20523: pdb searches for .pdbrc in ~ instead of $HOME (GH-11847) Previously pdb checked the $HOME environmental variable to find the user .pdbrc. If $HOME is not set, the user .pdbrc would not be found. Change pdb to use `os.path.expanduser('~')` to determine the user's home directory. Thus, if $HOME is not set (as in tox or on Windows), os.path.expanduser('~') falls back on other techniques for locating the user's home directory. This follows pip's implementation for loading .piprc. Co-authored-by: Dan Lidral-Porter files: A Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst M Lib/pdb.py M Lib/test/test_pdb.py diff --git a/Lib/pdb.py b/Lib/pdb.py index 5e62f392d932..69fd8bd6efb0 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -160,16 +160,14 @@ def __init__(self, completekey='tab', stdin=None, stdout=None, skip=None, self.allow_kbdint = False self.nosigint = nosigint - # Read $HOME/.pdbrc and ./.pdbrc + # Read ~/.pdbrc and ./.pdbrc self.rcLines = [] if readrc: - if 'HOME' in os.environ: - envHome = os.environ['HOME'] - try: - with open(os.path.join(envHome, ".pdbrc")) as rcFile: - self.rcLines.extend(rcFile) - except OSError: - pass + try: + with open(os.path.expanduser('~/.pdbrc')) as rcFile: + self.rcLines.extend(rcFile) + except OSError: + pass try: with open(".pdbrc") as rcFile: self.rcLines.extend(rcFile) diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py index 1e464df28340..646bdb16e53b 100644 --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -1377,6 +1377,19 @@ def test_readrc_kwarg(self): if save_home is not None: os.environ['HOME'] = save_home + def test_readrc_homedir(self): + save_home = os.environ.pop("HOME", None) + with support.temp_dir() as temp_dir, patch("os.path.expanduser"): + rc_path = os.path.join(temp_dir, ".pdbrc") + os.path.expanduser.return_value = rc_path + try: + with open(rc_path, "w") as f: + f.write("invalid") + self.assertEqual(pdb.Pdb().rcLines[0], "invalid") + finally: + if save_home is not None: + os.environ["HOME"] = save_home + def test_header(self): stdout = StringIO() header = 'Nobody expects... blah, blah, blah' diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst b/Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst new file mode 100644 index 000000000000..91397c243b9f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst @@ -0,0 +1,2 @@ +``pdb.Pdb`` supports ~/.pdbrc in Windows 7. Patch by Tim Hopper and Dan +Lidral-Porter. \ No newline at end of file From webhook-mailer at python.org Fri Aug 2 18:40:19 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 02 Aug 2019 22:40:19 -0000 Subject: [Python-checkins] bpo-20523: pdb searches for .pdbrc in ~ instead of $HOME (GH-11847) Message-ID: https://github.com/python/cpython/commit/1ff7dd681c7f3e31524bfada6d6d2786d4e37704 commit: 1ff7dd681c7f3e31524bfada6d6d2786d4e37704 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-02T15:40:14-07:00 summary: bpo-20523: pdb searches for .pdbrc in ~ instead of $HOME (GH-11847) Previously pdb checked the $HOME environmental variable to find the user .pdbrc. If $HOME is not set, the user .pdbrc would not be found. Change pdb to use `os.path.expanduser('~')` to determine the user's home directory. Thus, if $HOME is not set (as in tox or on Windows), os.path.expanduser('~') falls back on other techniques for locating the user's home directory. This follows pip's implementation for loading .piprc. Co-authored-by: Dan Lidral-Porter (cherry picked from commit 7ea9a85f132b32347fcbd2cbe1b553a2e9890b56) Co-authored-by: Timothy Hopper files: A Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst M Lib/pdb.py M Lib/test/test_pdb.py diff --git a/Lib/pdb.py b/Lib/pdb.py index 59b23dfc8bea..11d763994458 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -159,16 +159,14 @@ def __init__(self, completekey='tab', stdin=None, stdout=None, skip=None, self.allow_kbdint = False self.nosigint = nosigint - # Read $HOME/.pdbrc and ./.pdbrc + # Read ~/.pdbrc and ./.pdbrc self.rcLines = [] if readrc: - if 'HOME' in os.environ: - envHome = os.environ['HOME'] - try: - with open(os.path.join(envHome, ".pdbrc")) as rcFile: - self.rcLines.extend(rcFile) - except OSError: - pass + try: + with open(os.path.expanduser('~/.pdbrc')) as rcFile: + self.rcLines.extend(rcFile) + except OSError: + pass try: with open(".pdbrc") as rcFile: self.rcLines.extend(rcFile) diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py index de6c651b370a..63909e21f246 100644 --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -1350,6 +1350,19 @@ def test_readrc_kwarg(self): if save_home is not None: os.environ['HOME'] = save_home + def test_readrc_homedir(self): + save_home = os.environ.pop("HOME", None) + with support.temp_dir() as temp_dir, patch("os.path.expanduser"): + rc_path = os.path.join(temp_dir, ".pdbrc") + os.path.expanduser.return_value = rc_path + try: + with open(rc_path, "w") as f: + f.write("invalid") + self.assertEqual(pdb.Pdb().rcLines[0], "invalid") + finally: + if save_home is not None: + os.environ["HOME"] = save_home + def test_header(self): stdout = StringIO() header = 'Nobody expects... blah, blah, blah' diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst b/Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst new file mode 100644 index 000000000000..91397c243b9f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst @@ -0,0 +1,2 @@ +``pdb.Pdb`` supports ~/.pdbrc in Windows 7. Patch by Tim Hopper and Dan +Lidral-Porter. \ No newline at end of file From webhook-mailer at python.org Fri Aug 2 18:42:54 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 02 Aug 2019 22:42:54 -0000 Subject: [Python-checkins] bpo-20523: pdb searches for .pdbrc in ~ instead of $HOME (GH-11847) Message-ID: https://github.com/python/cpython/commit/79af3bd1d170ed6a72a5c126e862590cdbf192d7 commit: 79af3bd1d170ed6a72a5c126e862590cdbf192d7 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-02T15:42:50-07:00 summary: bpo-20523: pdb searches for .pdbrc in ~ instead of $HOME (GH-11847) Previously pdb checked the $HOME environmental variable to find the user .pdbrc. If $HOME is not set, the user .pdbrc would not be found. Change pdb to use `os.path.expanduser('~')` to determine the user's home directory. Thus, if $HOME is not set (as in tox or on Windows), os.path.expanduser('~') falls back on other techniques for locating the user's home directory. This follows pip's implementation for loading .piprc. Co-authored-by: Dan Lidral-Porter (cherry picked from commit 7ea9a85f132b32347fcbd2cbe1b553a2e9890b56) Co-authored-by: Timothy Hopper files: A Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst M Lib/pdb.py M Lib/test/test_pdb.py diff --git a/Lib/pdb.py b/Lib/pdb.py index 5e62f392d932..69fd8bd6efb0 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -160,16 +160,14 @@ def __init__(self, completekey='tab', stdin=None, stdout=None, skip=None, self.allow_kbdint = False self.nosigint = nosigint - # Read $HOME/.pdbrc and ./.pdbrc + # Read ~/.pdbrc and ./.pdbrc self.rcLines = [] if readrc: - if 'HOME' in os.environ: - envHome = os.environ['HOME'] - try: - with open(os.path.join(envHome, ".pdbrc")) as rcFile: - self.rcLines.extend(rcFile) - except OSError: - pass + try: + with open(os.path.expanduser('~/.pdbrc')) as rcFile: + self.rcLines.extend(rcFile) + except OSError: + pass try: with open(".pdbrc") as rcFile: self.rcLines.extend(rcFile) diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py index d03f1b284300..16d245a5602a 100644 --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -1377,6 +1377,19 @@ def test_readrc_kwarg(self): if save_home is not None: os.environ['HOME'] = save_home + def test_readrc_homedir(self): + save_home = os.environ.pop("HOME", None) + with support.temp_dir() as temp_dir, patch("os.path.expanduser"): + rc_path = os.path.join(temp_dir, ".pdbrc") + os.path.expanduser.return_value = rc_path + try: + with open(rc_path, "w") as f: + f.write("invalid") + self.assertEqual(pdb.Pdb().rcLines[0], "invalid") + finally: + if save_home is not None: + os.environ["HOME"] = save_home + def test_header(self): stdout = StringIO() header = 'Nobody expects... blah, blah, blah' diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst b/Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst new file mode 100644 index 000000000000..91397c243b9f --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-02-15-20-42-36.bpo-20523.rRLrvr.rst @@ -0,0 +1,2 @@ +``pdb.Pdb`` supports ~/.pdbrc in Windows 7. Patch by Tim Hopper and Dan +Lidral-Porter. \ No newline at end of file From webhook-mailer at python.org Fri Aug 2 18:44:28 2019 From: webhook-mailer at python.org (Steve Dower) Date: Fri, 02 Aug 2019 22:44:28 -0000 Subject: [Python-checkins] bpo-30974: Change os.path.samefile docstring to match docs (GH-7337) Message-ID: https://github.com/python/cpython/commit/8e568ef266a2805f9a6042003723d9c050830461 commit: 8e568ef266a2805f9a6042003723d9c050830461 branch: master author: Timo Furrer committer: Steve Dower date: 2019-08-02T15:44:25-07:00 summary: bpo-30974: Change os.path.samefile docstring to match docs (GH-7337) files: M Lib/genericpath.py diff --git a/Lib/genericpath.py b/Lib/genericpath.py index 5dd703d736c5..db11f67d54cc 100644 --- a/Lib/genericpath.py +++ b/Lib/genericpath.py @@ -92,7 +92,11 @@ def samestat(s1, s2): # Are two filenames really pointing to the same file? def samefile(f1, f2): - """Test whether two pathnames reference the same actual file""" + """Test whether two pathnames reference the same actual file or directory + + This is determined by the device number and i-node number and + raises an exception if an os.stat() call on either pathname fails. + """ s1 = os.stat(f1) s2 = os.stat(f2) return samestat(s1, s2) From webhook-mailer at python.org Fri Aug 2 19:04:57 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 02 Aug 2019 23:04:57 -0000 Subject: [Python-checkins] bpo-30974: Change os.path.samefile docstring to match docs (GH-7337) Message-ID: https://github.com/python/cpython/commit/6b833901fe9053937c289c6371bb731c9aceb58e commit: 6b833901fe9053937c289c6371bb731c9aceb58e branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-02T16:04:53-07:00 summary: bpo-30974: Change os.path.samefile docstring to match docs (GH-7337) (cherry picked from commit 8e568ef266a2805f9a6042003723d9c050830461) Co-authored-by: Timo Furrer files: M Lib/genericpath.py diff --git a/Lib/genericpath.py b/Lib/genericpath.py index 5dd703d736c5..db11f67d54cc 100644 --- a/Lib/genericpath.py +++ b/Lib/genericpath.py @@ -92,7 +92,11 @@ def samestat(s1, s2): # Are two filenames really pointing to the same file? def samefile(f1, f2): - """Test whether two pathnames reference the same actual file""" + """Test whether two pathnames reference the same actual file or directory + + This is determined by the device number and i-node number and + raises an exception if an os.stat() call on either pathname fails. + """ s1 = os.stat(f1) s2 = os.stat(f2) return samestat(s1, s2) From webhook-mailer at python.org Fri Aug 2 19:11:37 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 02 Aug 2019 23:11:37 -0000 Subject: [Python-checkins] bpo-30974: Change os.path.samefile docstring to match docs (GH-7337) Message-ID: https://github.com/python/cpython/commit/a49f203e052c6fb1244d4e55c3fccc439dda0e2e commit: a49f203e052c6fb1244d4e55c3fccc439dda0e2e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-02T16:11:33-07:00 summary: bpo-30974: Change os.path.samefile docstring to match docs (GH-7337) (cherry picked from commit 8e568ef266a2805f9a6042003723d9c050830461) Co-authored-by: Timo Furrer files: M Lib/genericpath.py diff --git a/Lib/genericpath.py b/Lib/genericpath.py index 303b3b349a9f..85f4a57582f2 100644 --- a/Lib/genericpath.py +++ b/Lib/genericpath.py @@ -92,7 +92,11 @@ def samestat(s1, s2): # Are two filenames really pointing to the same file? def samefile(f1, f2): - """Test whether two pathnames reference the same actual file""" + """Test whether two pathnames reference the same actual file or directory + + This is determined by the device number and i-node number and + raises an exception if an os.stat() call on either pathname fails. + """ s1 = os.stat(f1) s2 = os.stat(f2) return samestat(s1, s2) From webhook-mailer at python.org Sat Aug 3 01:46:06 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 03 Aug 2019 05:46:06 -0000 Subject: [Python-checkins] bpo-37444: Update differing exception between builtins and importlib (GH-14869) Message-ID: https://github.com/python/cpython/commit/c5fa44944ee0a31a12b9a70776c7cb56c4dc39a2 commit: c5fa44944ee0a31a12b9a70776c7cb56c4dc39a2 branch: master author: Ngalim Siregar committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-02T22:46:02-07:00 summary: bpo-37444: Update differing exception between builtins and importlib (GH-14869) Imports now raise `TypeError` instead of `ValueError` for relative import failures. This makes things consistent between `builtins.__import__` and `importlib.__import__` as well as using a more natural import for the failure. https://bugs.python.org/issue37444 Automerge-Triggered-By: @brettcannon files: A Misc/NEWS.d/next/Core and Builtins/2019-07-20-22-34-42.bpo-37444.UOd3Xs.rst M Doc/library/importlib.rst M Doc/whatsnew/3.9.rst M Lib/importlib/_bootstrap.py M Lib/importlib/util.py M Lib/test/test_importlib/import_/test_relative_imports.py M Lib/test/test_importlib/test_util.py M Python/import.c M Python/importlib.h diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index 65a685022e92..8be6172d4c76 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -1433,13 +1433,18 @@ an :term:`importer`. ``importlib.util.resolve_name('sys', __package__)`` without doing a check to see if the **package** argument is needed. - :exc:`ValueError` is raised if **name** is a relative module name but - package is a false value (e.g. ``None`` or the empty string). - :exc:`ValueError` is also raised a relative name would escape its containing + :exc:`ImportError` is raised if **name** is a relative module name but + **package** is a false value (e.g. ``None`` or the empty string). + :exc:`ImportError` is also raised a relative name would escape its containing package (e.g. requesting ``..bacon`` from within the ``spam`` package). .. versionadded:: 3.3 + .. versionchanged:: 3.9 + To improve consistency with import statements, raise + :exc:`ImportError` instead of :exc:`ValueError` for invalid relative + import attempts. + .. function:: find_spec(name, package=None) Find the :term:`spec ` for a module, optionally relative to diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 273fd2b50d59..61d9e745e87c 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -75,6 +75,12 @@ New Features Other Language Changes ====================== +* :func:`builtins.__import__` now raises :exc:`ImportError` instead of + :exc:`ValueError` as used to occur when a relative import went past + its top-level package. + (Contributed by Ngalim Siregar in :issue:`37444`.) + + * Python now gets the absolute path of the script filename specified on the command line (ex: ``python3 script.py``): the ``__file__`` attribute of the ``__main__`` module, ``sys.argv[0]`` and ``sys.path[0]`` become an @@ -118,6 +124,13 @@ pprint :mod:`pprint` can now pretty-print :class:`types.SimpleNamespace`. (Contributed by Carl Bordum Hansen in :issue:`37376`.) +importlib +--------- + +To improve consistency with import statements, :func:`importlib.util.resolve_name` +now raises :exc:`ImportError` instead of :exc:`ValueError` for invalid relative +import attempts. +(Contributed by Ngalim Siregar in :issue:`37444`.) Optimizations ============= @@ -180,4 +193,11 @@ Porting to Python 3.9 This section lists previously described changes and other bugfixes that may require changes to your code. +Changes in the Python API +------------------------- +* :func:`builtins.__import__` and :func:`importlib.util.resolve_name` now raise + :exc:`ImportError` where it previously raised :exc:`ValueError`. Callers + catching the specific exception type and supporting both Python 3.9 and + earlier versions will need to catch both: + ``except (ImportError, ValueError):`` diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py index 32deef10af9b..5e2f520c0ef8 100644 --- a/Lib/importlib/_bootstrap.py +++ b/Lib/importlib/_bootstrap.py @@ -873,7 +873,7 @@ def _resolve_name(name, package, level): """Resolve a relative module name to an absolute one.""" bits = package.rsplit('.', level - 1) if len(bits) < level: - raise ValueError('attempted relative import beyond top-level package') + raise ImportError('attempted relative import beyond top-level package') base = bits[0] return '{}.{}'.format(base, name) if name else base diff --git a/Lib/importlib/util.py b/Lib/importlib/util.py index 201e0f4cb89d..269a6fa930aa 100644 --- a/Lib/importlib/util.py +++ b/Lib/importlib/util.py @@ -29,8 +29,8 @@ def resolve_name(name, package): if not name.startswith('.'): return name elif not package: - raise ValueError(f'no package specified for {repr(name)} ' - '(required for relative module names)') + raise ImportError(f'no package specified for {repr(name)} ' + '(required for relative module names)') level = 0 for character in name: if character != '.': diff --git a/Lib/test/test_importlib/import_/test_relative_imports.py b/Lib/test/test_importlib/import_/test_relative_imports.py index 8a95a32109d2..586a9bf4bc12 100644 --- a/Lib/test/test_importlib/import_/test_relative_imports.py +++ b/Lib/test/test_importlib/import_/test_relative_imports.py @@ -156,7 +156,7 @@ def test_too_high_from_package(self): {'__name__': 'pkg', '__path__': ['blah']}) def callback(global_): self.__import__('pkg') - with self.assertRaises(ValueError): + with self.assertRaises(ImportError): self.__import__('', global_, fromlist=['top_level'], level=2) self.relative_import_test(create, globals_, callback) @@ -167,7 +167,7 @@ def test_too_high_from_module(self): globals_ = {'__package__': 'pkg'}, {'__name__': 'pkg.module'} def callback(global_): self.__import__('pkg') - with self.assertRaises(ValueError): + with self.assertRaises(ImportError): self.__import__('', global_, fromlist=['top_level'], level=2) self.relative_import_test(create, globals_, callback) diff --git a/Lib/test/test_importlib/test_util.py b/Lib/test/test_importlib/test_util.py index 0350a5a5cc05..8489c198728a 100644 --- a/Lib/test/test_importlib/test_util.py +++ b/Lib/test/test_importlib/test_util.py @@ -375,7 +375,7 @@ def test_absolute_within_package(self): def test_no_package(self): # .bacon in '' - with self.assertRaises(ValueError): + with self.assertRaises(ImportError): self.util.resolve_name('.bacon', '') def test_in_package(self): @@ -390,7 +390,7 @@ def test_other_package(self): def test_escape(self): # ..bacon in spam - with self.assertRaises(ValueError): + with self.assertRaises(ImportError): self.util.resolve_name('..bacon', 'spam') @@ -518,7 +518,7 @@ def test_find_relative_module_missing_package(self): with util.temp_module(name, pkg=True) as pkg_dir: fullname, _ = util.submodule(name, subname, pkg_dir) relname = '.' + subname - with self.assertRaises(ValueError): + with self.assertRaises(ImportError): self.util.find_spec(relname) self.assertNotIn(name, sorted(sys.modules)) self.assertNotIn(fullname, sorted(sys.modules)) diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-07-20-22-34-42.bpo-37444.UOd3Xs.rst b/Misc/NEWS.d/next/Core and Builtins/2019-07-20-22-34-42.bpo-37444.UOd3Xs.rst new file mode 100644 index 000000000000..67c68071a839 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-07-20-22-34-42.bpo-37444.UOd3Xs.rst @@ -0,0 +1 @@ +Update differing exception between :meth:`builtins.__import__` and :meth:`importlib.__import__`. diff --git a/Python/import.c b/Python/import.c index 9f5ec284ae15..41c2f34f12c6 100644 --- a/Python/import.c +++ b/Python/import.c @@ -1671,7 +1671,7 @@ resolve_name(PyThreadState *tstate, PyObject *name, PyObject *globals, int level goto error; } else if (last_dot == -1) { - _PyErr_SetString(tstate, PyExc_ValueError, + _PyErr_SetString(tstate, PyExc_ImportError, "attempted relative import beyond top-level " "package"); goto error; diff --git a/Python/importlib.h b/Python/importlib.h index a285a31e2065..a6952af22b4f 100644 --- a/Python/importlib.h +++ b/Python/importlib.h @@ -1351,89 +1351,89 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 112,111,114,116,32,98,101,121,111,110,100,32,116,111,112,45, 108,101,118,101,108,32,112,97,99,107,97,103,101,114,22,0, 0,0,250,5,123,125,46,123,125,41,4,218,6,114,115,112, - 108,105,116,218,3,108,101,110,218,10,86,97,108,117,101,69, - 114,114,111,114,114,45,0,0,0,41,5,114,17,0,0,0, - 218,7,112,97,99,107,97,103,101,218,5,108,101,118,101,108, - 90,4,98,105,116,115,90,4,98,97,115,101,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,218,13,95,114,101, - 115,111,108,118,101,95,110,97,109,101,104,3,0,0,115,10, - 0,0,0,0,2,16,1,12,1,8,1,8,1,114,188,0, - 0,0,99,3,0,0,0,0,0,0,0,0,0,0,0,4, - 0,0,0,4,0,0,0,67,0,0,0,115,34,0,0,0, - 124,0,160,0,124,1,124,2,161,2,125,3,124,3,100,0, - 107,8,114,24,100,0,83,0,116,1,124,1,124,3,131,2, - 83,0,114,13,0,0,0,41,2,114,167,0,0,0,114,91, - 0,0,0,41,4,218,6,102,105,110,100,101,114,114,17,0, - 0,0,114,164,0,0,0,114,109,0,0,0,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,218,17,95,102,105, - 110,100,95,115,112,101,99,95,108,101,103,97,99,121,113,3, - 0,0,115,8,0,0,0,0,3,12,1,8,1,4,1,114, - 190,0,0,0,99,3,0,0,0,0,0,0,0,0,0,0, - 0,10,0,0,0,10,0,0,0,67,0,0,0,115,12,1, - 0,0,116,0,106,1,125,3,124,3,100,1,107,8,114,22, - 116,2,100,2,131,1,130,1,124,3,115,38,116,3,160,4, - 100,3,116,5,161,2,1,0,124,0,116,0,106,6,107,6, - 125,4,124,3,68,0,93,210,125,5,116,7,131,0,143,84, - 1,0,122,10,124,5,106,8,125,6,87,0,110,54,4,0, - 116,9,107,10,114,128,1,0,1,0,1,0,116,10,124,5, - 124,0,124,1,131,3,125,7,124,7,100,1,107,8,114,124, - 89,0,87,0,53,0,81,0,82,0,163,0,113,52,89,0, - 110,14,88,0,124,6,124,0,124,1,124,2,131,3,125,7, - 87,0,53,0,81,0,82,0,88,0,124,7,100,1,107,9, - 114,52,124,4,144,0,115,254,124,0,116,0,106,6,107,6, - 144,0,114,254,116,0,106,6,124,0,25,0,125,8,122,10, - 124,8,106,11,125,9,87,0,110,28,4,0,116,9,107,10, - 114,226,1,0,1,0,1,0,124,7,6,0,89,0,2,0, - 1,0,83,0,88,0,124,9,100,1,107,8,114,244,124,7, - 2,0,1,0,83,0,124,9,2,0,1,0,83,0,113,52, - 124,7,2,0,1,0,83,0,113,52,100,1,83,0,41,4, - 122,21,70,105,110,100,32,97,32,109,111,100,117,108,101,39, - 115,32,115,112,101,99,46,78,122,53,115,121,115,46,109,101, - 116,97,95,112,97,116,104,32,105,115,32,78,111,110,101,44, - 32,80,121,116,104,111,110,32,105,115,32,108,105,107,101,108, - 121,32,115,104,117,116,116,105,110,103,32,100,111,119,110,122, - 22,115,121,115,46,109,101,116,97,95,112,97,116,104,32,105, - 115,32,101,109,112,116,121,41,12,114,15,0,0,0,218,9, - 109,101,116,97,95,112,97,116,104,114,79,0,0,0,218,9, - 95,119,97,114,110,105,110,103,115,218,4,119,97,114,110,218, - 13,73,109,112,111,114,116,87,97,114,110,105,110,103,114,92, - 0,0,0,114,178,0,0,0,114,166,0,0,0,114,106,0, - 0,0,114,190,0,0,0,114,105,0,0,0,41,10,114,17, - 0,0,0,114,164,0,0,0,114,165,0,0,0,114,191,0, - 0,0,90,9,105,115,95,114,101,108,111,97,100,114,189,0, - 0,0,114,166,0,0,0,114,95,0,0,0,114,96,0,0, - 0,114,105,0,0,0,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,218,10,95,102,105,110,100,95,115,112,101, - 99,122,3,0,0,115,54,0,0,0,0,2,6,1,8,2, - 8,3,4,1,12,5,10,1,8,1,8,1,2,1,10,1, - 14,1,12,1,8,1,20,2,22,1,8,2,18,1,10,1, - 2,1,10,1,14,4,14,2,8,1,8,2,10,2,10,2, - 114,195,0,0,0,99,3,0,0,0,0,0,0,0,0,0, - 0,0,3,0,0,0,5,0,0,0,67,0,0,0,115,108, - 0,0,0,116,0,124,0,116,1,131,2,115,28,116,2,100, - 1,160,3,116,4,124,0,131,1,161,1,131,1,130,1,124, - 2,100,2,107,0,114,44,116,5,100,3,131,1,130,1,124, - 2,100,2,107,4,114,84,116,0,124,1,116,1,131,2,115, - 72,116,2,100,4,131,1,130,1,110,12,124,1,115,84,116, - 6,100,5,131,1,130,1,124,0,115,104,124,2,100,2,107, - 2,114,104,116,5,100,6,131,1,130,1,100,7,83,0,41, - 8,122,28,86,101,114,105,102,121,32,97,114,103,117,109,101, - 110,116,115,32,97,114,101,32,34,115,97,110,101,34,46,122, - 31,109,111,100,117,108,101,32,110,97,109,101,32,109,117,115, - 116,32,98,101,32,115,116,114,44,32,110,111,116,32,123,125, - 114,22,0,0,0,122,18,108,101,118,101,108,32,109,117,115, - 116,32,98,101,32,62,61,32,48,122,31,95,95,112,97,99, - 107,97,103,101,95,95,32,110,111,116,32,115,101,116,32,116, - 111,32,97,32,115,116,114,105,110,103,122,54,97,116,116,101, - 109,112,116,101,100,32,114,101,108,97,116,105,118,101,32,105, - 109,112,111,114,116,32,119,105,116,104,32,110,111,32,107,110, - 111,119,110,32,112,97,114,101,110,116,32,112,97,99,107,97, - 103,101,122,17,69,109,112,116,121,32,109,111,100,117,108,101, - 32,110,97,109,101,78,41,7,218,10,105,115,105,110,115,116, - 97,110,99,101,218,3,115,116,114,218,9,84,121,112,101,69, - 114,114,111,114,114,45,0,0,0,114,14,0,0,0,114,185, - 0,0,0,114,79,0,0,0,169,3,114,17,0,0,0,114, - 186,0,0,0,114,187,0,0,0,114,10,0,0,0,114,10, + 108,105,116,218,3,108,101,110,114,79,0,0,0,114,45,0, + 0,0,41,5,114,17,0,0,0,218,7,112,97,99,107,97, + 103,101,218,5,108,101,118,101,108,90,4,98,105,116,115,90, + 4,98,97,115,101,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,218,13,95,114,101,115,111,108,118,101,95,110, + 97,109,101,104,3,0,0,115,10,0,0,0,0,2,16,1, + 12,1,8,1,8,1,114,187,0,0,0,99,3,0,0,0, + 0,0,0,0,0,0,0,0,4,0,0,0,4,0,0,0, + 67,0,0,0,115,34,0,0,0,124,0,160,0,124,1,124, + 2,161,2,125,3,124,3,100,0,107,8,114,24,100,0,83, + 0,116,1,124,1,124,3,131,2,83,0,114,13,0,0,0, + 41,2,114,167,0,0,0,114,91,0,0,0,41,4,218,6, + 102,105,110,100,101,114,114,17,0,0,0,114,164,0,0,0, + 114,109,0,0,0,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,218,17,95,102,105,110,100,95,115,112,101,99, + 95,108,101,103,97,99,121,113,3,0,0,115,8,0,0,0, + 0,3,12,1,8,1,4,1,114,189,0,0,0,99,3,0, + 0,0,0,0,0,0,0,0,0,0,10,0,0,0,10,0, + 0,0,67,0,0,0,115,12,1,0,0,116,0,106,1,125, + 3,124,3,100,1,107,8,114,22,116,2,100,2,131,1,130, + 1,124,3,115,38,116,3,160,4,100,3,116,5,161,2,1, + 0,124,0,116,0,106,6,107,6,125,4,124,3,68,0,93, + 210,125,5,116,7,131,0,143,84,1,0,122,10,124,5,106, + 8,125,6,87,0,110,54,4,0,116,9,107,10,114,128,1, + 0,1,0,1,0,116,10,124,5,124,0,124,1,131,3,125, + 7,124,7,100,1,107,8,114,124,89,0,87,0,53,0,81, + 0,82,0,163,0,113,52,89,0,110,14,88,0,124,6,124, + 0,124,1,124,2,131,3,125,7,87,0,53,0,81,0,82, + 0,88,0,124,7,100,1,107,9,114,52,124,4,144,0,115, + 254,124,0,116,0,106,6,107,6,144,0,114,254,116,0,106, + 6,124,0,25,0,125,8,122,10,124,8,106,11,125,9,87, + 0,110,28,4,0,116,9,107,10,114,226,1,0,1,0,1, + 0,124,7,6,0,89,0,2,0,1,0,83,0,88,0,124, + 9,100,1,107,8,114,244,124,7,2,0,1,0,83,0,124, + 9,2,0,1,0,83,0,113,52,124,7,2,0,1,0,83, + 0,113,52,100,1,83,0,41,4,122,21,70,105,110,100,32, + 97,32,109,111,100,117,108,101,39,115,32,115,112,101,99,46, + 78,122,53,115,121,115,46,109,101,116,97,95,112,97,116,104, + 32,105,115,32,78,111,110,101,44,32,80,121,116,104,111,110, + 32,105,115,32,108,105,107,101,108,121,32,115,104,117,116,116, + 105,110,103,32,100,111,119,110,122,22,115,121,115,46,109,101, + 116,97,95,112,97,116,104,32,105,115,32,101,109,112,116,121, + 41,12,114,15,0,0,0,218,9,109,101,116,97,95,112,97, + 116,104,114,79,0,0,0,218,9,95,119,97,114,110,105,110, + 103,115,218,4,119,97,114,110,218,13,73,109,112,111,114,116, + 87,97,114,110,105,110,103,114,92,0,0,0,114,178,0,0, + 0,114,166,0,0,0,114,106,0,0,0,114,189,0,0,0, + 114,105,0,0,0,41,10,114,17,0,0,0,114,164,0,0, + 0,114,165,0,0,0,114,190,0,0,0,90,9,105,115,95, + 114,101,108,111,97,100,114,188,0,0,0,114,166,0,0,0, + 114,95,0,0,0,114,96,0,0,0,114,105,0,0,0,114, + 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,10, + 95,102,105,110,100,95,115,112,101,99,122,3,0,0,115,54, + 0,0,0,0,2,6,1,8,2,8,3,4,1,12,5,10, + 1,8,1,8,1,2,1,10,1,14,1,12,1,8,1,20, + 2,22,1,8,2,18,1,10,1,2,1,10,1,14,4,14, + 2,8,1,8,2,10,2,10,2,114,194,0,0,0,99,3, + 0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,5, + 0,0,0,67,0,0,0,115,108,0,0,0,116,0,124,0, + 116,1,131,2,115,28,116,2,100,1,160,3,116,4,124,0, + 131,1,161,1,131,1,130,1,124,2,100,2,107,0,114,44, + 116,5,100,3,131,1,130,1,124,2,100,2,107,4,114,84, + 116,0,124,1,116,1,131,2,115,72,116,2,100,4,131,1, + 130,1,110,12,124,1,115,84,116,6,100,5,131,1,130,1, + 124,0,115,104,124,2,100,2,107,2,114,104,116,5,100,6, + 131,1,130,1,100,7,83,0,41,8,122,28,86,101,114,105, + 102,121,32,97,114,103,117,109,101,110,116,115,32,97,114,101, + 32,34,115,97,110,101,34,46,122,31,109,111,100,117,108,101, + 32,110,97,109,101,32,109,117,115,116,32,98,101,32,115,116, + 114,44,32,110,111,116,32,123,125,114,22,0,0,0,122,18, + 108,101,118,101,108,32,109,117,115,116,32,98,101,32,62,61, + 32,48,122,31,95,95,112,97,99,107,97,103,101,95,95,32, + 110,111,116,32,115,101,116,32,116,111,32,97,32,115,116,114, + 105,110,103,122,54,97,116,116,101,109,112,116,101,100,32,114, + 101,108,97,116,105,118,101,32,105,109,112,111,114,116,32,119, + 105,116,104,32,110,111,32,107,110,111,119,110,32,112,97,114, + 101,110,116,32,112,97,99,107,97,103,101,122,17,69,109,112, + 116,121,32,109,111,100,117,108,101,32,110,97,109,101,78,41, + 7,218,10,105,115,105,110,115,116,97,110,99,101,218,3,115, + 116,114,218,9,84,121,112,101,69,114,114,111,114,114,45,0, + 0,0,114,14,0,0,0,218,10,86,97,108,117,101,69,114, + 114,111,114,114,79,0,0,0,169,3,114,17,0,0,0,114, + 185,0,0,0,114,186,0,0,0,114,10,0,0,0,114,10, 0,0,0,114,11,0,0,0,218,13,95,115,97,110,105,116, 121,95,99,104,101,99,107,169,3,0,0,115,22,0,0,0, 0,2,10,1,18,1,8,1,8,1,8,1,10,1,10,1, @@ -1462,7 +1462,7 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 0,0,0,114,141,0,0,0,114,106,0,0,0,218,8,95, 69,82,82,95,77,83,71,114,45,0,0,0,218,19,77,111, 100,117,108,101,78,111,116,70,111,117,110,100,69,114,114,111, - 114,114,195,0,0,0,114,159,0,0,0,114,5,0,0,0, + 114,114,194,0,0,0,114,159,0,0,0,114,5,0,0,0, 41,8,114,17,0,0,0,218,7,105,109,112,111,114,116,95, 114,164,0,0,0,114,130,0,0,0,90,13,112,97,114,101, 110,116,95,109,111,100,117,108,101,114,157,0,0,0,114,95, @@ -1520,7 +1520,7 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 97,103,101,95,95,32,105,102,10,32,32,32,32,116,104,101, 32,108,111,97,100,101,114,32,100,105,100,32,110,111,116,46, 10,10,32,32,32,32,114,22,0,0,0,41,4,114,200,0, - 0,0,114,188,0,0,0,114,207,0,0,0,218,11,95,103, + 0,0,114,187,0,0,0,114,207,0,0,0,218,11,95,103, 99,100,95,105,109,112,111,114,116,114,199,0,0,0,114,10, 0,0,0,114,10,0,0,0,114,11,0,0,0,114,208,0, 0,0,234,3,0,0,115,8,0,0,0,0,9,12,1,8, @@ -1561,8 +1561,8 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 122,8,73,116,101,109,32,105,110,32,122,18,32,109,117,115, 116,32,98,101,32,115,116,114,44,32,110,111,116,32,250,1, 42,218,7,95,95,97,108,108,95,95,84,114,209,0,0,0, - 114,182,0,0,0,78,41,16,114,196,0,0,0,114,197,0, - 0,0,114,1,0,0,0,114,198,0,0,0,114,14,0,0, + 114,182,0,0,0,78,41,16,114,195,0,0,0,114,196,0, + 0,0,114,1,0,0,0,114,197,0,0,0,114,14,0,0, 0,114,4,0,0,0,218,16,95,104,97,110,100,108,101,95, 102,114,111,109,108,105,115,116,114,212,0,0,0,114,45,0, 0,0,114,67,0,0,0,114,203,0,0,0,114,17,0,0, @@ -1609,9 +1609,9 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 110,32,95,95,110,97,109,101,95,95,32,97,110,100,32,95, 95,112,97,116,104,95,95,114,1,0,0,0,114,141,0,0, 0,114,128,0,0,0,114,22,0,0,0,41,6,114,34,0, - 0,0,114,130,0,0,0,114,192,0,0,0,114,193,0,0, - 0,114,194,0,0,0,114,129,0,0,0,41,3,218,7,103, - 108,111,98,97,108,115,114,186,0,0,0,114,95,0,0,0, + 0,0,114,130,0,0,0,114,191,0,0,0,114,192,0,0, + 0,114,193,0,0,0,114,129,0,0,0,41,3,218,7,103, + 108,111,98,97,108,115,114,185,0,0,0,114,95,0,0,0, 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, 17,95,99,97,108,99,95,95,95,112,97,99,107,97,103,101, 95,95,30,4,0,0,115,38,0,0,0,0,7,10,1,10, @@ -1666,8 +1666,8 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 111,110,114,184,0,0,0,114,15,0,0,0,114,92,0,0, 0,114,1,0,0,0,114,4,0,0,0,114,213,0,0,0, 41,9,114,17,0,0,0,114,218,0,0,0,218,6,108,111, - 99,97,108,115,114,214,0,0,0,114,187,0,0,0,114,96, - 0,0,0,90,8,103,108,111,98,97,108,115,95,114,186,0, + 99,97,108,115,114,214,0,0,0,114,186,0,0,0,114,96, + 0,0,0,90,8,103,108,111,98,97,108,115,95,114,185,0, 0,0,90,7,99,117,116,95,111,102,102,114,10,0,0,0, 114,10,0,0,0,114,11,0,0,0,218,10,95,95,105,109, 112,111,114,116,95,95,57,4,0,0,115,30,0,0,0,0, @@ -1713,10 +1713,10 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 101,32,116,119,111,32,109,111,100,117,108,101,115,32,109,117, 115,116,32,98,101,32,101,120,112,108,105,99,105,116,108,121, 32,112,97,115,115,101,100,32,105,110,46,10,10,32,32,32, - 32,41,3,114,23,0,0,0,114,192,0,0,0,114,64,0, + 32,41,3,114,23,0,0,0,114,191,0,0,0,114,64,0, 0,0,78,41,15,114,57,0,0,0,114,15,0,0,0,114, 14,0,0,0,114,92,0,0,0,218,5,105,116,101,109,115, - 114,196,0,0,0,114,78,0,0,0,114,160,0,0,0,114, + 114,195,0,0,0,114,78,0,0,0,114,160,0,0,0,114, 88,0,0,0,114,173,0,0,0,114,142,0,0,0,114,148, 0,0,0,114,1,0,0,0,114,223,0,0,0,114,5,0, 0,0,41,10,218,10,115,121,115,95,109,111,100,117,108,101, @@ -1738,7 +1738,7 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 108,32,105,109,112,111,114,116,101,114,115,32,102,111,114,32, 98,117,105,108,116,105,110,32,97,110,100,32,102,114,111,122, 101,110,32,109,111,100,117,108,101,115,78,41,6,114,227,0, - 0,0,114,15,0,0,0,114,191,0,0,0,114,120,0,0, + 0,0,114,15,0,0,0,114,190,0,0,0,114,120,0,0, 0,114,160,0,0,0,114,173,0,0,0,41,2,114,225,0, 0,0,114,226,0,0,0,114,10,0,0,0,114,10,0,0, 0,114,11,0,0,0,218,8,95,105,110,115,116,97,108,108, @@ -1771,7 +1771,7 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 114,148,0,0,0,114,152,0,0,0,114,107,0,0,0,114, 93,0,0,0,114,158,0,0,0,114,159,0,0,0,114,94, 0,0,0,114,160,0,0,0,114,173,0,0,0,114,178,0, - 0,0,114,188,0,0,0,114,190,0,0,0,114,195,0,0, + 0,0,114,187,0,0,0,114,189,0,0,0,114,194,0,0, 0,114,200,0,0,0,90,15,95,69,82,82,95,77,83,71, 95,80,82,69,70,73,88,114,202,0,0,0,114,205,0,0, 0,218,6,111,98,106,101,99,116,114,206,0,0,0,114,207, From webhook-mailer at python.org Sat Aug 3 02:12:31 2019 From: webhook-mailer at python.org (Ronald Oussoren) Date: Sat, 03 Aug 2019 06:12:31 -0000 Subject: [Python-checkins] bpo-18049: Define THREAD_STACK_SIZE for AIX to pass default recursion limit test (GH-15081) Message-ID: https://github.com/python/cpython/commit/9670ce76b83bde950020f8d89c4d27168aaaf912 commit: 9670ce76b83bde950020f8d89c4d27168aaaf912 branch: master author: Michael Felt committer: Ronald Oussoren date: 2019-08-03T08:12:26+02:00 summary: bpo-18049: Define THREAD_STACK_SIZE for AIX to pass default recursion limit test (GH-15081) * Define THREAD_STACK_SIZE for AIX to pass default recursion limit test files: A Misc/NEWS.d/next/Library/2019-08-02-16-44-42.bpo-18049.OA4qBL.rst M Python/thread_pthread.h diff --git a/Misc/NEWS.d/next/Library/2019-08-02-16-44-42.bpo-18049.OA4qBL.rst b/Misc/NEWS.d/next/Library/2019-08-02-16-44-42.bpo-18049.OA4qBL.rst new file mode 100644 index 000000000000..36a4de384cc0 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-02-16-44-42.bpo-18049.OA4qBL.rst @@ -0,0 +1,3 @@ +Add definition of THREAD_STACK_SIZE for AIX in Python/thread_pthread.h +The default thread stacksize caused crashes with the default recursion limit +Patch by M Felt diff --git a/Python/thread_pthread.h b/Python/thread_pthread.h index 994e35b2cc08..5678b05ced36 100644 --- a/Python/thread_pthread.h +++ b/Python/thread_pthread.h @@ -47,6 +47,10 @@ #undef THREAD_STACK_SIZE #define THREAD_STACK_SIZE 0x400000 #endif +#if defined(_AIX) && defined(THREAD_STACK_SIZE) && THREAD_STACK_SIZE == 0 +#undef THREAD_STACK_SIZE +#define THREAD_STACK_SIZE 0x200000 +#endif /* for safety, ensure a viable minimum stacksize */ #define THREAD_STACK_MIN 0x8000 /* 32 KiB */ #else /* !_POSIX_THREAD_ATTR_STACKSIZE */ From webhook-mailer at python.org Sat Aug 3 14:16:39 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Sat, 03 Aug 2019 18:16:39 -0000 Subject: [Python-checkins] Correct description of HTTP status code 308. (GH-15078) Message-ID: https://github.com/python/cpython/commit/5c72badd06a962fe0018ceb9916f3ae66314ea8e commit: 5c72badd06a962fe0018ceb9916f3ae66314ea8e branch: master author: Florian Wendelborn <1133858+FlorianWendelborn at users.noreply.github.com> committer: Benjamin Peterson date: 2019-08-03T11:16:34-07:00 summary: Correct description of HTTP status code 308. (GH-15078) Permanent redirect was explained as a temporary redirect. files: M Lib/http/__init__.py diff --git a/Lib/http/__init__.py b/Lib/http/__init__.py index aed94a585046..e14a1eb074c5 100644 --- a/Lib/http/__init__.py +++ b/Lib/http/__init__.py @@ -59,7 +59,7 @@ def __new__(cls, value, phrase, description=''): TEMPORARY_REDIRECT = (307, 'Temporary Redirect', 'Object moved temporarily -- see URI list') PERMANENT_REDIRECT = (308, 'Permanent Redirect', - 'Object moved temporarily -- see URI list') + 'Object moved permanently -- see URI list') # client error BAD_REQUEST = (400, 'Bad Request', From webhook-mailer at python.org Sat Aug 3 14:39:50 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Sat, 03 Aug 2019 18:39:50 -0000 Subject: [Python-checkins] Correct description of HTTP status code 308. (GH-15098) Message-ID: https://github.com/python/cpython/commit/4e402d37eb4009140421365acbeff78104b108e0 commit: 4e402d37eb4009140421365acbeff78104b108e0 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Benjamin Peterson date: 2019-08-03T11:39:46-07:00 summary: Correct description of HTTP status code 308. (GH-15098) Permanent redirect was explained as a temporary redirect. (cherry picked from commit 5c72badd06a962fe0018ceb9916f3ae66314ea8e) Co-authored-by: Florian Wendelborn <1133858+FlorianWendelborn at users.noreply.github.com> files: M Lib/http/__init__.py diff --git a/Lib/http/__init__.py b/Lib/http/__init__.py index aed94a585046..e14a1eb074c5 100644 --- a/Lib/http/__init__.py +++ b/Lib/http/__init__.py @@ -59,7 +59,7 @@ def __new__(cls, value, phrase, description=''): TEMPORARY_REDIRECT = (307, 'Temporary Redirect', 'Object moved temporarily -- see URI list') PERMANENT_REDIRECT = (308, 'Permanent Redirect', - 'Object moved temporarily -- see URI list') + 'Object moved permanently -- see URI list') # client error BAD_REQUEST = (400, 'Bad Request', From webhook-mailer at python.org Sat Aug 3 14:40:13 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Sat, 03 Aug 2019 18:40:13 -0000 Subject: [Python-checkins] Correct description of HTTP status code 308. (GH-15097) Message-ID: https://github.com/python/cpython/commit/0bb8f22abd81751eab1125132cdaa31b2083c735 commit: 0bb8f22abd81751eab1125132cdaa31b2083c735 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Benjamin Peterson date: 2019-08-03T11:40:09-07:00 summary: Correct description of HTTP status code 308. (GH-15097) Permanent redirect was explained as a temporary redirect. (cherry picked from commit 5c72badd06a962fe0018ceb9916f3ae66314ea8e) Co-authored-by: Florian Wendelborn <1133858+FlorianWendelborn at users.noreply.github.com> files: M Lib/http/__init__.py diff --git a/Lib/http/__init__.py b/Lib/http/__init__.py index aed94a585046..e14a1eb074c5 100644 --- a/Lib/http/__init__.py +++ b/Lib/http/__init__.py @@ -59,7 +59,7 @@ def __new__(cls, value, phrase, description=''): TEMPORARY_REDIRECT = (307, 'Temporary Redirect', 'Object moved temporarily -- see URI list') PERMANENT_REDIRECT = (308, 'Permanent Redirect', - 'Object moved temporarily -- see URI list') + 'Object moved permanently -- see URI list') # client error BAD_REQUEST = (400, 'Bad Request', From webhook-mailer at python.org Sun Aug 4 05:38:50 2019 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sun, 04 Aug 2019 09:38:50 -0000 Subject: [Python-checkins] bpo-37685: Fixed comparisons of datetime.timedelta and datetime.timezone. (GH-14996) Message-ID: https://github.com/python/cpython/commit/17e52649c0e7e9389f1cc2444a53f059e24e6bca commit: 17e52649c0e7e9389f1cc2444a53f059e24e6bca branch: master author: Serhiy Storchaka committer: GitHub date: 2019-08-04T12:38:46+03:00 summary: bpo-37685: Fixed comparisons of datetime.timedelta and datetime.timezone. (GH-14996) There was a discrepancy between the Python and C implementations. Add singletons ALWAYS_EQ, LARGEST and SMALLEST in test.support to test mixed type comparison. files: A Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst M Doc/library/test.rst M Lib/datetime.py M Lib/test/datetimetester.py M Lib/test/support/__init__.py M Lib/test/test_ipaddress.py M Modules/_datetimemodule.c diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 7d62a94d839b..da6a85d340be 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -356,11 +356,28 @@ The :mod:`test.support` module defines the following constants: Check for presence of docstrings. + .. data:: TEST_HTTP_URL Define the URL of a dedicated HTTP server for the network tests. +.. data:: ALWAYS_EQ + + Object that is equal to anything. Used to test mixed type comparison. + + +.. data:: LARGEST + + Object that is greater than anything (except itself). + Used to test mixed type comparison. + + +.. data:: SMALLEST + + Object that is less than anything (except itself). + Used to test mixed type comparison. + The :mod:`test.support` module defines the following functions: diff --git a/Lib/datetime.py b/Lib/datetime.py index e35ee0554c1f..d4c7a1ff9004 100644 --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -739,25 +739,25 @@ def __le__(self, other): if isinstance(other, timedelta): return self._cmp(other) <= 0 else: - _cmperror(self, other) + return NotImplemented def __lt__(self, other): if isinstance(other, timedelta): return self._cmp(other) < 0 else: - _cmperror(self, other) + return NotImplemented def __ge__(self, other): if isinstance(other, timedelta): return self._cmp(other) >= 0 else: - _cmperror(self, other) + return NotImplemented def __gt__(self, other): if isinstance(other, timedelta): return self._cmp(other) > 0 else: - _cmperror(self, other) + return NotImplemented def _cmp(self, other): assert isinstance(other, timedelta) @@ -1316,25 +1316,25 @@ def __le__(self, other): if isinstance(other, time): return self._cmp(other) <= 0 else: - _cmperror(self, other) + return NotImplemented def __lt__(self, other): if isinstance(other, time): return self._cmp(other) < 0 else: - _cmperror(self, other) + return NotImplemented def __ge__(self, other): if isinstance(other, time): return self._cmp(other) >= 0 else: - _cmperror(self, other) + return NotImplemented def __gt__(self, other): if isinstance(other, time): return self._cmp(other) > 0 else: - _cmperror(self, other) + return NotImplemented def _cmp(self, other, allow_mixed=False): assert isinstance(other, time) @@ -2210,9 +2210,9 @@ def __getinitargs__(self): return (self._offset, self._name) def __eq__(self, other): - if type(other) != timezone: - return False - return self._offset == other._offset + if isinstance(other, timezone): + return self._offset == other._offset + return NotImplemented def __hash__(self): return hash(self._offset) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 37ddd4b1a8bc..99b620ce2f41 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -2,11 +2,8 @@ See http://www.zope.org/Members/fdrake/DateTimeWiki/TestCases """ -from test.support import is_resource_enabled - import itertools import bisect - import copy import decimal import sys @@ -22,6 +19,7 @@ from operator import lt, le, gt, ge, eq, ne, truediv, floordiv, mod from test import support +from test.support import is_resource_enabled, ALWAYS_EQ, LARGEST, SMALLEST import datetime as datetime_module from datetime import MINYEAR, MAXYEAR @@ -54,18 +52,6 @@ NAN = float("nan") -class ComparesEqualClass(object): - """ - A class that is always equal to whatever you compare it to. - """ - - def __eq__(self, other): - return True - - def __ne__(self, other): - return False - - ############################################################################# # module tests @@ -353,6 +339,18 @@ def test_comparison(self): self.assertTrue(timezone(ZERO) != None) self.assertFalse(timezone(ZERO) == None) + tz = timezone(ZERO) + self.assertTrue(tz == ALWAYS_EQ) + self.assertFalse(tz != ALWAYS_EQ) + self.assertTrue(tz < LARGEST) + self.assertFalse(tz > LARGEST) + self.assertTrue(tz <= LARGEST) + self.assertFalse(tz >= LARGEST) + self.assertFalse(tz < SMALLEST) + self.assertTrue(tz > SMALLEST) + self.assertFalse(tz <= SMALLEST) + self.assertTrue(tz >= SMALLEST) + def test_aware_datetime(self): # test that timezone instances can be used by datetime t = datetime(1, 1, 1) @@ -414,10 +412,21 @@ def test_harmless_mixed_comparison(self): # Comparison to objects of unsupported types should return # NotImplemented which falls back to the right hand side's __eq__ - # method. In this case, ComparesEqualClass.__eq__ always returns True. - # ComparesEqualClass.__ne__ always returns False. - self.assertTrue(me == ComparesEqualClass()) - self.assertFalse(me != ComparesEqualClass()) + # method. In this case, ALWAYS_EQ.__eq__ always returns True. + # ALWAYS_EQ.__ne__ always returns False. + self.assertTrue(me == ALWAYS_EQ) + self.assertFalse(me != ALWAYS_EQ) + + # If the other class explicitly defines ordering + # relative to our class, it is allowed to do so + self.assertTrue(me < LARGEST) + self.assertFalse(me > LARGEST) + self.assertTrue(me <= LARGEST) + self.assertFalse(me >= LARGEST) + self.assertFalse(me < SMALLEST) + self.assertTrue(me > SMALLEST) + self.assertFalse(me <= SMALLEST) + self.assertTrue(me >= SMALLEST) def test_harmful_mixed_comparison(self): me = self.theclass(1, 1, 1) @@ -1582,29 +1591,6 @@ class SomeClass: self.assertRaises(TypeError, lambda: our < their) self.assertRaises(TypeError, lambda: their < our) - # However, if the other class explicitly defines ordering - # relative to our class, it is allowed to do so - - class LargerThanAnything: - def __lt__(self, other): - return False - def __le__(self, other): - return isinstance(other, LargerThanAnything) - def __eq__(self, other): - return isinstance(other, LargerThanAnything) - def __gt__(self, other): - return not isinstance(other, LargerThanAnything) - def __ge__(self, other): - return True - - their = LargerThanAnything() - self.assertEqual(our == their, False) - self.assertEqual(their == our, False) - self.assertEqual(our != their, True) - self.assertEqual(their != our, True) - self.assertEqual(our < their, True) - self.assertEqual(their < our, False) - def test_bool(self): # All dates are considered true. self.assertTrue(self.theclass.min) @@ -3781,8 +3767,8 @@ def test_replace(self): self.assertRaises(ValueError, base.replace, microsecond=1000000) def test_mixed_compare(self): - t1 = time(1, 2, 3) - t2 = time(1, 2, 3) + t1 = self.theclass(1, 2, 3) + t2 = self.theclass(1, 2, 3) self.assertEqual(t1, t2) t2 = t2.replace(tzinfo=None) self.assertEqual(t1, t2) diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index dbbbdb0ee339..c82037eea523 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -113,6 +113,7 @@ "run_with_locale", "swap_item", "swap_attr", "Matcher", "set_memlimit", "SuppressCrashReport", "sortdict", "run_with_tz", "PGO", "missing_compiler_executable", "fd_count", + "ALWAYS_EQ", "LARGEST", "SMALLEST" ] class Error(Exception): @@ -3103,6 +3104,41 @@ def __fspath__(self): return self.path +class _ALWAYS_EQ: + """ + Object that is equal to anything. + """ + def __eq__(self, other): + return True + def __ne__(self, other): + return False + +ALWAYS_EQ = _ALWAYS_EQ() + + at functools.total_ordering +class _LARGEST: + """ + Object that is greater than anything (except itself). + """ + def __eq__(self, other): + return isinstance(other, _LARGEST) + def __lt__(self, other): + return False + +LARGEST = _LARGEST() + + at functools.total_ordering +class _SMALLEST: + """ + Object that is less than anything (except itself). + """ + def __eq__(self, other): + return isinstance(other, _SMALLEST) + def __gt__(self, other): + return False + +SMALLEST = _SMALLEST() + def maybe_get_event_loop_policy(): """Return the global event loop policy if one is set, else return None.""" return asyncio.events._event_loop_policy diff --git a/Lib/test/test_ipaddress.py b/Lib/test/test_ipaddress.py index 9e17ea0c7aac..de77111705b6 100644 --- a/Lib/test/test_ipaddress.py +++ b/Lib/test/test_ipaddress.py @@ -12,6 +12,7 @@ import pickle import ipaddress import weakref +from test.support import LARGEST, SMALLEST class BaseTestCase(unittest.TestCase): @@ -673,20 +674,6 @@ def test_ip_network(self): self.assertFactoryError(ipaddress.ip_network, "network") - at functools.total_ordering -class LargestObject: - def __eq__(self, other): - return isinstance(other, LargestObject) - def __lt__(self, other): - return False - - at functools.total_ordering -class SmallestObject: - def __eq__(self, other): - return isinstance(other, SmallestObject) - def __gt__(self, other): - return False - class ComparisonTests(unittest.TestCase): v4addr = ipaddress.IPv4Address(1) @@ -775,8 +762,6 @@ def test_mixed_type_ordering(self): def test_foreign_type_ordering(self): other = object() - smallest = SmallestObject() - largest = LargestObject() for obj in self.objects: with self.assertRaises(TypeError): obj < other @@ -786,14 +771,14 @@ def test_foreign_type_ordering(self): obj <= other with self.assertRaises(TypeError): obj >= other - self.assertTrue(obj < largest) - self.assertFalse(obj > largest) - self.assertTrue(obj <= largest) - self.assertFalse(obj >= largest) - self.assertFalse(obj < smallest) - self.assertTrue(obj > smallest) - self.assertFalse(obj <= smallest) - self.assertTrue(obj >= smallest) + self.assertTrue(obj < LARGEST) + self.assertFalse(obj > LARGEST) + self.assertTrue(obj <= LARGEST) + self.assertFalse(obj >= LARGEST) + self.assertFalse(obj < SMALLEST) + self.assertTrue(obj > SMALLEST) + self.assertFalse(obj <= SMALLEST) + self.assertTrue(obj >= SMALLEST) def test_mixed_type_key(self): # with get_mixed_type_key, you can sort addresses and network. diff --git a/Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst b/Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst new file mode 100644 index 000000000000..ba60057e6fb6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst @@ -0,0 +1,2 @@ +Fixed comparisons of :class:`datetime.timedelta` and +:class:`datetime.timezone`. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index 19d3d7e848f5..b55922cd961a 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3741,11 +3741,8 @@ timezone_richcompare(PyDateTime_TimeZone *self, { if (op != Py_EQ && op != Py_NE) Py_RETURN_NOTIMPLEMENTED; - if (Py_TYPE(other) != &PyDateTime_TimeZoneType) { - if (op == Py_EQ) - Py_RETURN_FALSE; - else - Py_RETURN_TRUE; + if (!PyTZInfo_Check(other)) { + Py_RETURN_NOTIMPLEMENTED; } return delta_richcompare(self->offset, other->offset, op); } From webhook-mailer at python.org Sun Aug 4 06:02:01 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 04 Aug 2019 10:02:01 -0000 Subject: [Python-checkins] bpo-37685: Fixed comparisons of datetime.timedelta and datetime.timezone. (GH-14996) Message-ID: https://github.com/python/cpython/commit/dde944f9df8dea28c07935ebd6de06db7e575c12 commit: dde944f9df8dea28c07935ebd6de06db7e575c12 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-04T03:01:55-07:00 summary: bpo-37685: Fixed comparisons of datetime.timedelta and datetime.timezone. (GH-14996) There was a discrepancy between the Python and C implementations. Add singletons ALWAYS_EQ, LARGEST and SMALLEST in test.support to test mixed type comparison. (cherry picked from commit 17e52649c0e7e9389f1cc2444a53f059e24e6bca) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst M Doc/library/test.rst M Lib/datetime.py M Lib/test/datetimetester.py M Lib/test/support/__init__.py M Lib/test/test_ipaddress.py M Modules/_datetimemodule.c diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 7d62a94d839b..da6a85d340be 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -356,11 +356,28 @@ The :mod:`test.support` module defines the following constants: Check for presence of docstrings. + .. data:: TEST_HTTP_URL Define the URL of a dedicated HTTP server for the network tests. +.. data:: ALWAYS_EQ + + Object that is equal to anything. Used to test mixed type comparison. + + +.. data:: LARGEST + + Object that is greater than anything (except itself). + Used to test mixed type comparison. + + +.. data:: SMALLEST + + Object that is less than anything (except itself). + Used to test mixed type comparison. + The :mod:`test.support` module defines the following functions: diff --git a/Lib/datetime.py b/Lib/datetime.py index e35ee0554c1f..d4c7a1ff9004 100644 --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -739,25 +739,25 @@ def __le__(self, other): if isinstance(other, timedelta): return self._cmp(other) <= 0 else: - _cmperror(self, other) + return NotImplemented def __lt__(self, other): if isinstance(other, timedelta): return self._cmp(other) < 0 else: - _cmperror(self, other) + return NotImplemented def __ge__(self, other): if isinstance(other, timedelta): return self._cmp(other) >= 0 else: - _cmperror(self, other) + return NotImplemented def __gt__(self, other): if isinstance(other, timedelta): return self._cmp(other) > 0 else: - _cmperror(self, other) + return NotImplemented def _cmp(self, other): assert isinstance(other, timedelta) @@ -1316,25 +1316,25 @@ def __le__(self, other): if isinstance(other, time): return self._cmp(other) <= 0 else: - _cmperror(self, other) + return NotImplemented def __lt__(self, other): if isinstance(other, time): return self._cmp(other) < 0 else: - _cmperror(self, other) + return NotImplemented def __ge__(self, other): if isinstance(other, time): return self._cmp(other) >= 0 else: - _cmperror(self, other) + return NotImplemented def __gt__(self, other): if isinstance(other, time): return self._cmp(other) > 0 else: - _cmperror(self, other) + return NotImplemented def _cmp(self, other, allow_mixed=False): assert isinstance(other, time) @@ -2210,9 +2210,9 @@ def __getinitargs__(self): return (self._offset, self._name) def __eq__(self, other): - if type(other) != timezone: - return False - return self._offset == other._offset + if isinstance(other, timezone): + return self._offset == other._offset + return NotImplemented def __hash__(self): return hash(self._offset) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 37ddd4b1a8bc..99b620ce2f41 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -2,11 +2,8 @@ See http://www.zope.org/Members/fdrake/DateTimeWiki/TestCases """ -from test.support import is_resource_enabled - import itertools import bisect - import copy import decimal import sys @@ -22,6 +19,7 @@ from operator import lt, le, gt, ge, eq, ne, truediv, floordiv, mod from test import support +from test.support import is_resource_enabled, ALWAYS_EQ, LARGEST, SMALLEST import datetime as datetime_module from datetime import MINYEAR, MAXYEAR @@ -54,18 +52,6 @@ NAN = float("nan") -class ComparesEqualClass(object): - """ - A class that is always equal to whatever you compare it to. - """ - - def __eq__(self, other): - return True - - def __ne__(self, other): - return False - - ############################################################################# # module tests @@ -353,6 +339,18 @@ def test_comparison(self): self.assertTrue(timezone(ZERO) != None) self.assertFalse(timezone(ZERO) == None) + tz = timezone(ZERO) + self.assertTrue(tz == ALWAYS_EQ) + self.assertFalse(tz != ALWAYS_EQ) + self.assertTrue(tz < LARGEST) + self.assertFalse(tz > LARGEST) + self.assertTrue(tz <= LARGEST) + self.assertFalse(tz >= LARGEST) + self.assertFalse(tz < SMALLEST) + self.assertTrue(tz > SMALLEST) + self.assertFalse(tz <= SMALLEST) + self.assertTrue(tz >= SMALLEST) + def test_aware_datetime(self): # test that timezone instances can be used by datetime t = datetime(1, 1, 1) @@ -414,10 +412,21 @@ def test_harmless_mixed_comparison(self): # Comparison to objects of unsupported types should return # NotImplemented which falls back to the right hand side's __eq__ - # method. In this case, ComparesEqualClass.__eq__ always returns True. - # ComparesEqualClass.__ne__ always returns False. - self.assertTrue(me == ComparesEqualClass()) - self.assertFalse(me != ComparesEqualClass()) + # method. In this case, ALWAYS_EQ.__eq__ always returns True. + # ALWAYS_EQ.__ne__ always returns False. + self.assertTrue(me == ALWAYS_EQ) + self.assertFalse(me != ALWAYS_EQ) + + # If the other class explicitly defines ordering + # relative to our class, it is allowed to do so + self.assertTrue(me < LARGEST) + self.assertFalse(me > LARGEST) + self.assertTrue(me <= LARGEST) + self.assertFalse(me >= LARGEST) + self.assertFalse(me < SMALLEST) + self.assertTrue(me > SMALLEST) + self.assertFalse(me <= SMALLEST) + self.assertTrue(me >= SMALLEST) def test_harmful_mixed_comparison(self): me = self.theclass(1, 1, 1) @@ -1582,29 +1591,6 @@ class SomeClass: self.assertRaises(TypeError, lambda: our < their) self.assertRaises(TypeError, lambda: their < our) - # However, if the other class explicitly defines ordering - # relative to our class, it is allowed to do so - - class LargerThanAnything: - def __lt__(self, other): - return False - def __le__(self, other): - return isinstance(other, LargerThanAnything) - def __eq__(self, other): - return isinstance(other, LargerThanAnything) - def __gt__(self, other): - return not isinstance(other, LargerThanAnything) - def __ge__(self, other): - return True - - their = LargerThanAnything() - self.assertEqual(our == their, False) - self.assertEqual(their == our, False) - self.assertEqual(our != their, True) - self.assertEqual(their != our, True) - self.assertEqual(our < their, True) - self.assertEqual(their < our, False) - def test_bool(self): # All dates are considered true. self.assertTrue(self.theclass.min) @@ -3781,8 +3767,8 @@ def test_replace(self): self.assertRaises(ValueError, base.replace, microsecond=1000000) def test_mixed_compare(self): - t1 = time(1, 2, 3) - t2 = time(1, 2, 3) + t1 = self.theclass(1, 2, 3) + t2 = self.theclass(1, 2, 3) self.assertEqual(t1, t2) t2 = t2.replace(tzinfo=None) self.assertEqual(t1, t2) diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index b88eb5804a96..e73c65bb9f34 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -113,6 +113,7 @@ "run_with_locale", "swap_item", "swap_attr", "Matcher", "set_memlimit", "SuppressCrashReport", "sortdict", "run_with_tz", "PGO", "missing_compiler_executable", "fd_count", + "ALWAYS_EQ", "LARGEST", "SMALLEST" ] class Error(Exception): @@ -3101,6 +3102,41 @@ def __fspath__(self): return self.path +class _ALWAYS_EQ: + """ + Object that is equal to anything. + """ + def __eq__(self, other): + return True + def __ne__(self, other): + return False + +ALWAYS_EQ = _ALWAYS_EQ() + + at functools.total_ordering +class _LARGEST: + """ + Object that is greater than anything (except itself). + """ + def __eq__(self, other): + return isinstance(other, _LARGEST) + def __lt__(self, other): + return False + +LARGEST = _LARGEST() + + at functools.total_ordering +class _SMALLEST: + """ + Object that is less than anything (except itself). + """ + def __eq__(self, other): + return isinstance(other, _SMALLEST) + def __gt__(self, other): + return False + +SMALLEST = _SMALLEST() + def maybe_get_event_loop_policy(): """Return the global event loop policy if one is set, else return None.""" return asyncio.events._event_loop_policy diff --git a/Lib/test/test_ipaddress.py b/Lib/test/test_ipaddress.py index 9e17ea0c7aac..de77111705b6 100644 --- a/Lib/test/test_ipaddress.py +++ b/Lib/test/test_ipaddress.py @@ -12,6 +12,7 @@ import pickle import ipaddress import weakref +from test.support import LARGEST, SMALLEST class BaseTestCase(unittest.TestCase): @@ -673,20 +674,6 @@ def test_ip_network(self): self.assertFactoryError(ipaddress.ip_network, "network") - at functools.total_ordering -class LargestObject: - def __eq__(self, other): - return isinstance(other, LargestObject) - def __lt__(self, other): - return False - - at functools.total_ordering -class SmallestObject: - def __eq__(self, other): - return isinstance(other, SmallestObject) - def __gt__(self, other): - return False - class ComparisonTests(unittest.TestCase): v4addr = ipaddress.IPv4Address(1) @@ -775,8 +762,6 @@ def test_mixed_type_ordering(self): def test_foreign_type_ordering(self): other = object() - smallest = SmallestObject() - largest = LargestObject() for obj in self.objects: with self.assertRaises(TypeError): obj < other @@ -786,14 +771,14 @@ def test_foreign_type_ordering(self): obj <= other with self.assertRaises(TypeError): obj >= other - self.assertTrue(obj < largest) - self.assertFalse(obj > largest) - self.assertTrue(obj <= largest) - self.assertFalse(obj >= largest) - self.assertFalse(obj < smallest) - self.assertTrue(obj > smallest) - self.assertFalse(obj <= smallest) - self.assertTrue(obj >= smallest) + self.assertTrue(obj < LARGEST) + self.assertFalse(obj > LARGEST) + self.assertTrue(obj <= LARGEST) + self.assertFalse(obj >= LARGEST) + self.assertFalse(obj < SMALLEST) + self.assertTrue(obj > SMALLEST) + self.assertFalse(obj <= SMALLEST) + self.assertTrue(obj >= SMALLEST) def test_mixed_type_key(self): # with get_mixed_type_key, you can sort addresses and network. diff --git a/Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst b/Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst new file mode 100644 index 000000000000..ba60057e6fb6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst @@ -0,0 +1,2 @@ +Fixed comparisons of :class:`datetime.timedelta` and +:class:`datetime.timezone`. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index 4d3562cbe64f..80ecfc3fff0b 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3744,11 +3744,8 @@ timezone_richcompare(PyDateTime_TimeZone *self, { if (op != Py_EQ && op != Py_NE) Py_RETURN_NOTIMPLEMENTED; - if (Py_TYPE(other) != &PyDateTime_TimeZoneType) { - if (op == Py_EQ) - Py_RETURN_FALSE; - else - Py_RETURN_TRUE; + if (!PyTZInfo_Check(other)) { + Py_RETURN_NOTIMPLEMENTED; } return delta_richcompare(self->offset, other->offset, op); } From webhook-mailer at python.org Sun Aug 4 07:12:52 2019 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sun, 04 Aug 2019 11:12:52 -0000 Subject: [Python-checkins] bpo-37648: Fixed minor inconsistency in some __contains__. (GH-14904) Message-ID: https://github.com/python/cpython/commit/18b711c5a7f90d88fb74748f18fa8ef49d8486c7 commit: 18b711c5a7f90d88fb74748f18fa8ef49d8486c7 branch: master author: Serhiy Storchaka committer: GitHub date: 2019-08-04T14:12:48+03:00 summary: bpo-37648: Fixed minor inconsistency in some __contains__. (GH-14904) The collection's item is now always at the left and the needle is on the right of ==. files: A Misc/NEWS.d/next/Core and Builtins/2019-07-22-11-05-05.bpo-37648.6TY2L-.rst M Doc/library/test.rst M Lib/test/list_tests.py M Lib/test/seq_tests.py M Lib/test/support/__init__.py M Lib/test/test_iter.py M Modules/_asynciomodule.c M Modules/_ssl.c M Objects/abstract.c M Objects/dictobject.c M Objects/listobject.c M Objects/tupleobject.c diff --git a/Doc/library/test.rst b/Doc/library/test.rst index da6a85d340be..6eef5c65499a 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -367,6 +367,12 @@ The :mod:`test.support` module defines the following constants: Object that is equal to anything. Used to test mixed type comparison. +.. data:: NEVER_EQ + + Object that is not equal to anything (even to :data:`ALWAYS_EQ`). + Used to test mixed type comparison. + + .. data:: LARGEST Object that is greater than anything (except itself). diff --git a/Lib/test/list_tests.py b/Lib/test/list_tests.py index 40316de220fa..44bc2ae6573c 100644 --- a/Lib/test/list_tests.py +++ b/Lib/test/list_tests.py @@ -7,6 +7,7 @@ from functools import cmp_to_key from test import support, seq_tests +from test.support import ALWAYS_EQ, NEVER_EQ class CommonTest(seq_tests.CommonTest): @@ -329,6 +330,20 @@ def test_remove(self): self.assertRaises(TypeError, a.remove) + a = self.type2test([1, 2]) + self.assertRaises(ValueError, a.remove, NEVER_EQ) + self.assertEqual(a, [1, 2]) + a.remove(ALWAYS_EQ) + self.assertEqual(a, [2]) + a = self.type2test([ALWAYS_EQ]) + a.remove(1) + self.assertEqual(a, []) + a = self.type2test([ALWAYS_EQ]) + a.remove(NEVER_EQ) + self.assertEqual(a, []) + a = self.type2test([NEVER_EQ]) + self.assertRaises(ValueError, a.remove, ALWAYS_EQ) + class BadExc(Exception): pass diff --git a/Lib/test/seq_tests.py b/Lib/test/seq_tests.py index 65b110ef7818..1d9ad588987f 100644 --- a/Lib/test/seq_tests.py +++ b/Lib/test/seq_tests.py @@ -6,6 +6,7 @@ import sys import pickle from test import support +from test.support import ALWAYS_EQ, NEVER_EQ # Various iterables # This is used for checking the constructor (here and in test_deque.py) @@ -221,15 +222,15 @@ def test_contains(self): self.assertRaises(TypeError, u.__contains__) def test_contains_fake(self): - class AllEq: - # Sequences must use rich comparison against each item - # (unless "is" is true, or an earlier item answered) - # So instances of AllEq must be found in all non-empty sequences. - def __eq__(self, other): - return True - __hash__ = None # Can't meet hash invariant requirements - self.assertNotIn(AllEq(), self.type2test([])) - self.assertIn(AllEq(), self.type2test([1])) + # Sequences must use rich comparison against each item + # (unless "is" is true, or an earlier item answered) + # So ALWAYS_EQ must be found in all non-empty sequences. + self.assertNotIn(ALWAYS_EQ, self.type2test([])) + self.assertIn(ALWAYS_EQ, self.type2test([1])) + self.assertIn(1, self.type2test([ALWAYS_EQ])) + self.assertNotIn(NEVER_EQ, self.type2test([])) + self.assertNotIn(ALWAYS_EQ, self.type2test([NEVER_EQ])) + self.assertIn(NEVER_EQ, self.type2test([ALWAYS_EQ])) def test_contains_order(self): # Sequences must test in-order. If a rich comparison has side @@ -350,6 +351,11 @@ def test_count(self): self.assertEqual(a.count(1), 3) self.assertEqual(a.count(3), 0) + self.assertEqual(a.count(ALWAYS_EQ), 9) + self.assertEqual(self.type2test([ALWAYS_EQ, ALWAYS_EQ]).count(1), 2) + self.assertEqual(self.type2test([ALWAYS_EQ, ALWAYS_EQ]).count(NEVER_EQ), 2) + self.assertEqual(self.type2test([NEVER_EQ, NEVER_EQ]).count(ALWAYS_EQ), 0) + self.assertRaises(TypeError, a.count) class BadExc(Exception): @@ -378,6 +384,11 @@ def test_index(self): self.assertEqual(u.index(0, 3, 4), 3) self.assertRaises(ValueError, u.index, 2, 0, -10) + self.assertEqual(u.index(ALWAYS_EQ), 0) + self.assertEqual(self.type2test([ALWAYS_EQ, ALWAYS_EQ]).index(1), 0) + self.assertEqual(self.type2test([ALWAYS_EQ, ALWAYS_EQ]).index(NEVER_EQ), 0) + self.assertRaises(ValueError, self.type2test([NEVER_EQ, NEVER_EQ]).index, ALWAYS_EQ) + self.assertRaises(TypeError, u.index) class BadExc(Exception): diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index c82037eea523..d34f2efaed53 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -113,7 +113,7 @@ "run_with_locale", "swap_item", "swap_attr", "Matcher", "set_memlimit", "SuppressCrashReport", "sortdict", "run_with_tz", "PGO", "missing_compiler_executable", "fd_count", - "ALWAYS_EQ", "LARGEST", "SMALLEST" + "ALWAYS_EQ", "NEVER_EQ", "LARGEST", "SMALLEST" ] class Error(Exception): @@ -3115,6 +3115,17 @@ def __ne__(self, other): ALWAYS_EQ = _ALWAYS_EQ() +class _NEVER_EQ: + """ + Object that is not equal to anything. + """ + def __eq__(self, other): + return False + def __ne__(self, other): + return True + +NEVER_EQ = _NEVER_EQ() + @functools.total_ordering class _LARGEST: """ diff --git a/Lib/test/test_iter.py b/Lib/test/test_iter.py index 542b28419e2c..6aceda23e9bd 100644 --- a/Lib/test/test_iter.py +++ b/Lib/test/test_iter.py @@ -3,7 +3,7 @@ import sys import unittest from test.support import run_unittest, TESTFN, unlink, cpython_only -from test.support import check_free_after_iterating +from test.support import check_free_after_iterating, ALWAYS_EQ, NEVER_EQ import pickle import collections.abc @@ -41,6 +41,14 @@ def __init__(self, n): def __iter__(self): return BasicIterClass(self.n) +class IteratorProxyClass: + def __init__(self, i): + self.i = i + def __next__(self): + return next(self.i) + def __iter__(self): + return self + class SequenceClass: def __init__(self, n): self.n = n @@ -50,6 +58,12 @@ def __getitem__(self, i): else: raise IndexError +class SequenceProxyClass: + def __init__(self, s): + self.s = s + def __getitem__(self, i): + return self.s[i] + class UnlimitedSequenceClass: def __getitem__(self, i): return i @@ -635,6 +649,13 @@ def test_in_and_not_in(self): for i in "abc", -1, 5, 42.42, (3, 4), [], {1: 1}, 3-12j, sc5: self.assertNotIn(i, sc5) + self.assertIn(ALWAYS_EQ, IteratorProxyClass(iter([1]))) + self.assertIn(ALWAYS_EQ, SequenceProxyClass([1])) + self.assertNotIn(ALWAYS_EQ, IteratorProxyClass(iter([NEVER_EQ]))) + self.assertNotIn(ALWAYS_EQ, SequenceProxyClass([NEVER_EQ])) + self.assertIn(NEVER_EQ, IteratorProxyClass(iter([ALWAYS_EQ]))) + self.assertIn(NEVER_EQ, SequenceProxyClass([ALWAYS_EQ])) + self.assertRaises(TypeError, lambda: 3 in 12) self.assertRaises(TypeError, lambda: 3 not in map) diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-07-22-11-05-05.bpo-37648.6TY2L-.rst b/Misc/NEWS.d/next/Core and Builtins/2019-07-22-11-05-05.bpo-37648.6TY2L-.rst new file mode 100644 index 000000000000..3c11d3d6008e --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-07-22-11-05-05.bpo-37648.6TY2L-.rst @@ -0,0 +1,3 @@ +Fixed minor inconsistency in :meth:`list.__contains__`, +:meth:`tuple.__contains__` and a few other places. The collection's item is +now always at the left and the needle is on the right of ``==``. diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index e9e6c5682d62..4d503a418a2e 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -937,7 +937,7 @@ _asyncio_Future_remove_done_callback(FutureObj *self, PyObject *fn) ENSURE_FUTURE_ALIVE(self) if (self->fut_callback0 != NULL) { - int cmp = PyObject_RichCompareBool(fn, self->fut_callback0, Py_EQ); + int cmp = PyObject_RichCompareBool(self->fut_callback0, fn, Py_EQ); if (cmp == -1) { return NULL; } @@ -962,7 +962,7 @@ _asyncio_Future_remove_done_callback(FutureObj *self, PyObject *fn) if (len == 1) { PyObject *cb_tup = PyList_GET_ITEM(self->fut_callbacks, 0); int cmp = PyObject_RichCompareBool( - fn, PyTuple_GET_ITEM(cb_tup, 0), Py_EQ); + PyTuple_GET_ITEM(cb_tup, 0), fn, Py_EQ); if (cmp == -1) { return NULL; } @@ -984,7 +984,7 @@ _asyncio_Future_remove_done_callback(FutureObj *self, PyObject *fn) int ret; PyObject *item = PyList_GET_ITEM(self->fut_callbacks, i); Py_INCREF(item); - ret = PyObject_RichCompareBool(fn, PyTuple_GET_ITEM(item, 0), Py_EQ); + ret = PyObject_RichCompareBool(PyTuple_GET_ITEM(item, 0), fn, Py_EQ); if (ret == 0) { if (j < len) { PyList_SET_ITEM(newlist, j, item); diff --git a/Modules/_ssl.c b/Modules/_ssl.c index da30cbb758e2..3d54b844fe07 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -5600,8 +5600,7 @@ list_contains(PyListObject *a, PyObject *el) int cmp; for (i = 0, cmp = 0 ; cmp == 0 && i < Py_SIZE(a); ++i) - cmp = PyObject_RichCompareBool(el, PyList_GET_ITEM(a, i), - Py_EQ); + cmp = PyObject_RichCompareBool(PyList_GET_ITEM(a, i), el, Py_EQ); return cmp; } diff --git a/Objects/abstract.c b/Objects/abstract.c index db1c3064db6f..f93d73fa7571 100644 --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2016,7 +2016,7 @@ _PySequence_IterSearch(PyObject *seq, PyObject *obj, int operation) break; } - cmp = PyObject_RichCompareBool(obj, item, Py_EQ); + cmp = PyObject_RichCompareBool(item, obj, Py_EQ); Py_DECREF(item); if (cmp < 0) goto Fail; diff --git a/Objects/dictobject.c b/Objects/dictobject.c index b6205d93ca14..f168ad5d2f00 100644 --- a/Objects/dictobject.c +++ b/Objects/dictobject.c @@ -4392,7 +4392,7 @@ dictitems_contains(_PyDictViewObject *dv, PyObject *obj) return 0; } Py_INCREF(found); - result = PyObject_RichCompareBool(value, found, Py_EQ); + result = PyObject_RichCompareBool(found, value, Py_EQ); Py_DECREF(found); return result; } diff --git a/Objects/listobject.c b/Objects/listobject.c index d012ab933a9e..cea9b24a3b2f 100644 --- a/Objects/listobject.c +++ b/Objects/listobject.c @@ -449,8 +449,7 @@ list_contains(PyListObject *a, PyObject *el) int cmp; for (i = 0, cmp = 0 ; cmp == 0 && i < Py_SIZE(a); ++i) - cmp = PyObject_RichCompareBool(el, PyList_GET_ITEM(a, i), - Py_EQ); + cmp = PyObject_RichCompareBool(PyList_GET_ITEM(a, i), el, Py_EQ); return cmp; } diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c index fc2d2742dd2c..aeaf845d74cf 100644 --- a/Objects/tupleobject.c +++ b/Objects/tupleobject.c @@ -403,8 +403,7 @@ tuplecontains(PyTupleObject *a, PyObject *el) int cmp; for (i = 0, cmp = 0 ; cmp == 0 && i < Py_SIZE(a); ++i) - cmp = PyObject_RichCompareBool(el, PyTuple_GET_ITEM(a, i), - Py_EQ); + cmp = PyObject_RichCompareBool(PyTuple_GET_ITEM(a, i), el, Py_EQ); return cmp; } From webhook-mailer at python.org Sun Aug 4 08:28:26 2019 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sun, 04 Aug 2019 12:28:26 -0000 Subject: [Python-checkins] [3.7] bpo-37685: Fixed comparisons of datetime.timedelta and datetime.timezone. (GH-14996) (GH-15104) Message-ID: https://github.com/python/cpython/commit/6ed20e54e4c110e9adcfb70aba85310625e3edb4 commit: 6ed20e54e4c110e9adcfb70aba85310625e3edb4 branch: 3.7 author: Serhiy Storchaka committer: GitHub date: 2019-08-04T15:28:21+03:00 summary: [3.7] bpo-37685: Fixed comparisons of datetime.timedelta and datetime.timezone. (GH-14996) (GH-15104) There was a discrepancy between the Python and C implementations. Add singletons ALWAYS_EQ, LARGEST and SMALLEST in test.support to test mixed type comparison. (cherry picked from commit 17e52649c0e7e9389f1cc2444a53f059e24e6bca) files: A Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst M Doc/library/test.rst M Lib/datetime.py M Lib/test/datetimetester.py M Lib/test/support/__init__.py M Lib/test/test_ipaddress.py M Modules/_datetimemodule.c diff --git a/Doc/library/test.rst b/Doc/library/test.rst index de79cdfc5fa9..e93ef450f022 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -356,11 +356,28 @@ The :mod:`test.support` module defines the following constants: Check for presence of docstrings. + .. data:: TEST_HTTP_URL Define the URL of a dedicated HTTP server for the network tests. +.. data:: ALWAYS_EQ + + Object that is equal to anything. Used to test mixed type comparison. + + +.. data:: LARGEST + + Object that is greater than anything (except itself). + Used to test mixed type comparison. + + +.. data:: SMALLEST + + Object that is less than anything (except itself). + Used to test mixed type comparison. + The :mod:`test.support` module defines the following functions: diff --git a/Lib/datetime.py b/Lib/datetime.py index 03beb055a00b..0485b0523e8d 100644 --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -724,25 +724,25 @@ def __le__(self, other): if isinstance(other, timedelta): return self._cmp(other) <= 0 else: - _cmperror(self, other) + return NotImplemented def __lt__(self, other): if isinstance(other, timedelta): return self._cmp(other) < 0 else: - _cmperror(self, other) + return NotImplemented def __ge__(self, other): if isinstance(other, timedelta): return self._cmp(other) >= 0 else: - _cmperror(self, other) + return NotImplemented def __gt__(self, other): if isinstance(other, timedelta): return self._cmp(other) > 0 else: - _cmperror(self, other) + return NotImplemented def _cmp(self, other): assert isinstance(other, timedelta) @@ -1267,25 +1267,25 @@ def __le__(self, other): if isinstance(other, time): return self._cmp(other) <= 0 else: - _cmperror(self, other) + return NotImplemented def __lt__(self, other): if isinstance(other, time): return self._cmp(other) < 0 else: - _cmperror(self, other) + return NotImplemented def __ge__(self, other): if isinstance(other, time): return self._cmp(other) >= 0 else: - _cmperror(self, other) + return NotImplemented def __gt__(self, other): if isinstance(other, time): return self._cmp(other) > 0 else: - _cmperror(self, other) + return NotImplemented def _cmp(self, other, allow_mixed=False): assert isinstance(other, time) @@ -2167,9 +2167,9 @@ def __getinitargs__(self): return (self._offset, self._name) def __eq__(self, other): - if type(other) != timezone: - return False - return self._offset == other._offset + if isinstance(other, timezone): + return self._offset == other._offset + return NotImplemented def __hash__(self): return hash(self._offset) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 581441738658..8a5716979716 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -2,11 +2,8 @@ See http://www.zope.org/Members/fdrake/DateTimeWiki/TestCases """ -from test.support import is_resource_enabled - import itertools import bisect - import copy import decimal import sys @@ -22,6 +19,7 @@ from operator import lt, le, gt, ge, eq, ne, truediv, floordiv, mod from test import support +from test.support import is_resource_enabled, ALWAYS_EQ, LARGEST, SMALLEST import datetime as datetime_module from datetime import MINYEAR, MAXYEAR @@ -54,18 +52,6 @@ NAN = float("nan") -class ComparesEqualClass(object): - """ - A class that is always equal to whatever you compare it to. - """ - - def __eq__(self, other): - return True - - def __ne__(self, other): - return False - - ############################################################################# # module tests @@ -353,6 +339,18 @@ def test_comparison(self): self.assertTrue(timezone(ZERO) != None) self.assertFalse(timezone(ZERO) == None) + tz = timezone(ZERO) + self.assertTrue(tz == ALWAYS_EQ) + self.assertFalse(tz != ALWAYS_EQ) + self.assertTrue(tz < LARGEST) + self.assertFalse(tz > LARGEST) + self.assertTrue(tz <= LARGEST) + self.assertFalse(tz >= LARGEST) + self.assertFalse(tz < SMALLEST) + self.assertTrue(tz > SMALLEST) + self.assertFalse(tz <= SMALLEST) + self.assertTrue(tz >= SMALLEST) + def test_aware_datetime(self): # test that timezone instances can be used by datetime t = datetime(1, 1, 1) @@ -414,10 +412,21 @@ def test_harmless_mixed_comparison(self): # Comparison to objects of unsupported types should return # NotImplemented which falls back to the right hand side's __eq__ - # method. In this case, ComparesEqualClass.__eq__ always returns True. - # ComparesEqualClass.__ne__ always returns False. - self.assertTrue(me == ComparesEqualClass()) - self.assertFalse(me != ComparesEqualClass()) + # method. In this case, ALWAYS_EQ.__eq__ always returns True. + # ALWAYS_EQ.__ne__ always returns False. + self.assertTrue(me == ALWAYS_EQ) + self.assertFalse(me != ALWAYS_EQ) + + # If the other class explicitly defines ordering + # relative to our class, it is allowed to do so + self.assertTrue(me < LARGEST) + self.assertFalse(me > LARGEST) + self.assertTrue(me <= LARGEST) + self.assertFalse(me >= LARGEST) + self.assertFalse(me < SMALLEST) + self.assertTrue(me > SMALLEST) + self.assertFalse(me <= SMALLEST) + self.assertTrue(me >= SMALLEST) def test_harmful_mixed_comparison(self): me = self.theclass(1, 1, 1) @@ -1544,29 +1553,6 @@ class SomeClass: self.assertRaises(TypeError, lambda: our < their) self.assertRaises(TypeError, lambda: their < our) - # However, if the other class explicitly defines ordering - # relative to our class, it is allowed to do so - - class LargerThanAnything: - def __lt__(self, other): - return False - def __le__(self, other): - return isinstance(other, LargerThanAnything) - def __eq__(self, other): - return isinstance(other, LargerThanAnything) - def __gt__(self, other): - return not isinstance(other, LargerThanAnything) - def __ge__(self, other): - return True - - their = LargerThanAnything() - self.assertEqual(our == their, False) - self.assertEqual(their == our, False) - self.assertEqual(our != their, True) - self.assertEqual(their != our, True) - self.assertEqual(our < their, True) - self.assertEqual(their < our, False) - def test_bool(self): # All dates are considered true. self.assertTrue(self.theclass.min) @@ -3642,8 +3628,8 @@ def test_replace(self): self.assertRaises(ValueError, base.replace, microsecond=1000000) def test_mixed_compare(self): - t1 = time(1, 2, 3) - t2 = time(1, 2, 3) + t1 = self.theclass(1, 2, 3) + t2 = self.theclass(1, 2, 3) self.assertEqual(t1, t2) t2 = t2.replace(tzinfo=None) self.assertEqual(t1, t2) diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 6e40a143c274..bf1f0d2120ac 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -109,6 +109,7 @@ "run_with_locale", "swap_item", "swap_attr", "Matcher", "set_memlimit", "SuppressCrashReport", "sortdict", "run_with_tz", "PGO", "missing_compiler_executable", "fd_count", + "ALWAYS_EQ", "LARGEST", "SMALLEST" ] class Error(Exception): @@ -2919,3 +2920,39 @@ def __fspath__(self): raise self.path else: return self.path + + +class _ALWAYS_EQ: + """ + Object that is equal to anything. + """ + def __eq__(self, other): + return True + def __ne__(self, other): + return False + +ALWAYS_EQ = _ALWAYS_EQ() + + at functools.total_ordering +class _LARGEST: + """ + Object that is greater than anything (except itself). + """ + def __eq__(self, other): + return isinstance(other, _LARGEST) + def __lt__(self, other): + return False + +LARGEST = _LARGEST() + + at functools.total_ordering +class _SMALLEST: + """ + Object that is less than anything (except itself). + """ + def __eq__(self, other): + return isinstance(other, _SMALLEST) + def __gt__(self, other): + return False + +SMALLEST = _SMALLEST() diff --git a/Lib/test/test_ipaddress.py b/Lib/test/test_ipaddress.py index a2326db0af86..455b893fb126 100644 --- a/Lib/test/test_ipaddress.py +++ b/Lib/test/test_ipaddress.py @@ -12,6 +12,7 @@ import pickle import ipaddress import weakref +from test.support import LARGEST, SMALLEST class BaseTestCase(unittest.TestCase): @@ -679,20 +680,6 @@ def test_ip_network(self): self.assertFactoryError(ipaddress.ip_network, "network") - at functools.total_ordering -class LargestObject: - def __eq__(self, other): - return isinstance(other, LargestObject) - def __lt__(self, other): - return False - - at functools.total_ordering -class SmallestObject: - def __eq__(self, other): - return isinstance(other, SmallestObject) - def __gt__(self, other): - return False - class ComparisonTests(unittest.TestCase): v4addr = ipaddress.IPv4Address(1) @@ -781,8 +768,6 @@ def test_mixed_type_ordering(self): def test_foreign_type_ordering(self): other = object() - smallest = SmallestObject() - largest = LargestObject() for obj in self.objects: with self.assertRaises(TypeError): obj < other @@ -792,14 +777,14 @@ def test_foreign_type_ordering(self): obj <= other with self.assertRaises(TypeError): obj >= other - self.assertTrue(obj < largest) - self.assertFalse(obj > largest) - self.assertTrue(obj <= largest) - self.assertFalse(obj >= largest) - self.assertFalse(obj < smallest) - self.assertTrue(obj > smallest) - self.assertFalse(obj <= smallest) - self.assertTrue(obj >= smallest) + self.assertTrue(obj < LARGEST) + self.assertFalse(obj > LARGEST) + self.assertTrue(obj <= LARGEST) + self.assertFalse(obj >= LARGEST) + self.assertFalse(obj < SMALLEST) + self.assertTrue(obj > SMALLEST) + self.assertFalse(obj <= SMALLEST) + self.assertTrue(obj >= SMALLEST) def test_mixed_type_key(self): # with get_mixed_type_key, you can sort addresses and network. diff --git a/Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst b/Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst new file mode 100644 index 000000000000..ba60057e6fb6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-28-22-25-25.bpo-37685._3bN9f.rst @@ -0,0 +1,2 @@ +Fixed comparisons of :class:`datetime.timedelta` and +:class:`datetime.timezone`. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index 9405b4610dfc..aa759b115f0e 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -3644,11 +3644,8 @@ timezone_richcompare(PyDateTime_TimeZone *self, { if (op != Py_EQ && op != Py_NE) Py_RETURN_NOTIMPLEMENTED; - if (Py_TYPE(other) != &PyDateTime_TimeZoneType) { - if (op == Py_EQ) - Py_RETURN_FALSE; - else - Py_RETURN_TRUE; + if (!PyTZInfo_Check(other)) { + Py_RETURN_NOTIMPLEMENTED; } return delta_richcompare(self->offset, other->offset, op); } From webhook-mailer at python.org Sun Aug 4 09:43:39 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 04 Aug 2019 13:43:39 -0000 Subject: [Python-checkins] bpo-37730: Fix usage of NotImplemented instead of NotImplementedError in docs. (GH-15062) Message-ID: https://github.com/python/cpython/commit/dd5f8abb54923bf2efea51b7a553ca1fca9cad68 commit: dd5f8abb54923bf2efea51b7a553ca1fca9cad68 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-04T06:43:30-07:00 summary: bpo-37730: Fix usage of NotImplemented instead of NotImplementedError in docs. (GH-15062) (cherry picked from commit ed5e8e06cbf766e89d6c58a882ee024abb5b2ed7) Co-authored-by: David H files: M Doc/library/winreg.rst M PC/clinic/winreg.c.h M PC/winreg.c diff --git a/Doc/library/winreg.rst b/Doc/library/winreg.rst index e9c026102273..cb67f2f39d76 100644 --- a/Doc/library/winreg.rst +++ b/Doc/library/winreg.rst @@ -456,7 +456,7 @@ This module offers the following functions: *key* is an already open key, or one of the predefined :ref:`HKEY_* constants `. - Will generally raise :exc:`NotImplemented` if executed on a 32-bit operating + Will generally raise :exc:`NotImplementedError` if executed on a 32-bit operating system. If the key is not on the reflection list, the function succeeds but has no @@ -471,7 +471,7 @@ This module offers the following functions: *key* is an already open key, or one of the predefined :ref:`HKEY_* constants `. - Will generally raise :exc:`NotImplemented` if executed on a 32-bit operating + Will generally raise :exc:`NotImplementedError` if executed on a 32-bit operating system. Restoring reflection for a key does not affect reflection of any subkeys. @@ -486,7 +486,7 @@ This module offers the following functions: Returns ``True`` if reflection is disabled. - Will generally raise :exc:`NotImplemented` if executed on a 32-bit + Will generally raise :exc:`NotImplementedError` if executed on a 32-bit operating system. diff --git a/PC/clinic/winreg.c.h b/PC/clinic/winreg.c.h index 50210250ed19..b7af1855ac54 100644 --- a/PC/clinic/winreg.c.h +++ b/PC/clinic/winreg.c.h @@ -1029,7 +1029,7 @@ PyDoc_STRVAR(winreg_DisableReflectionKey__doc__, " key\n" " An already open key, or any one of the predefined HKEY_* constants.\n" "\n" -"Will generally raise NotImplemented if executed on a 32bit OS.\n" +"Will generally raise NotImplementedError if executed on a 32bit OS.\n" "\n" "If the key is not on the reflection list, the function succeeds but has\n" "no effect. Disabling reflection for a key does not affect reflection\n" @@ -1065,7 +1065,7 @@ PyDoc_STRVAR(winreg_EnableReflectionKey__doc__, " key\n" " An already open key, or any one of the predefined HKEY_* constants.\n" "\n" -"Will generally raise NotImplemented if executed on a 32bit OS.\n" +"Will generally raise NotImplementedError if executed on a 32bit OS.\n" "Restoring reflection for a key does not affect reflection of any\n" "subkeys."); @@ -1099,7 +1099,7 @@ PyDoc_STRVAR(winreg_QueryReflectionKey__doc__, " key\n" " An already open key, or any one of the predefined HKEY_* constants.\n" "\n" -"Will generally raise NotImplemented if executed on a 32bit OS."); +"Will generally raise NotImplementedError if executed on a 32bit OS."); #define WINREG_QUERYREFLECTIONKEY_METHODDEF \ {"QueryReflectionKey", (PyCFunction)winreg_QueryReflectionKey, METH_O, winreg_QueryReflectionKey__doc__}, @@ -1121,4 +1121,4 @@ winreg_QueryReflectionKey(PyObject *module, PyObject *arg) exit: return return_value; } -/*[clinic end generated code: output=1204d20c543b5b4a input=a9049054013a1b77]*/ +/*[clinic end generated code: output=015afbbd690eb59d input=a9049054013a1b77]*/ diff --git a/PC/winreg.c b/PC/winreg.c index 5f5fc85d2250..d0df7ef0ad47 100644 --- a/PC/winreg.c +++ b/PC/winreg.c @@ -1702,7 +1702,7 @@ winreg.DisableReflectionKey Disables registry reflection for 32bit processes running on a 64bit OS. -Will generally raise NotImplemented if executed on a 32bit OS. +Will generally raise NotImplementedError if executed on a 32bit OS. If the key is not on the reflection list, the function succeeds but has no effect. Disabling reflection for a key does not affect reflection @@ -1711,7 +1711,7 @@ of any subkeys. static PyObject * winreg_DisableReflectionKey_impl(PyObject *module, HKEY key) -/*[clinic end generated code: output=830cce504cc764b4 input=a6c9e5ca5410193c]*/ +/*[clinic end generated code: output=830cce504cc764b4 input=70bece2dee02e073]*/ { HMODULE hMod; typedef LONG (WINAPI *RDRKFunc)(HKEY); @@ -1749,14 +1749,14 @@ winreg.EnableReflectionKey Restores registry reflection for the specified disabled key. -Will generally raise NotImplemented if executed on a 32bit OS. +Will generally raise NotImplementedError if executed on a 32bit OS. Restoring reflection for a key does not affect reflection of any subkeys. [clinic start generated code]*/ static PyObject * winreg_EnableReflectionKey_impl(PyObject *module, HKEY key) -/*[clinic end generated code: output=86fa1385fdd9ce57 input=7748abbacd1e166a]*/ +/*[clinic end generated code: output=86fa1385fdd9ce57 input=eeae770c6eb9f559]*/ { HMODULE hMod; typedef LONG (WINAPI *RERKFunc)(HKEY); @@ -1794,12 +1794,12 @@ winreg.QueryReflectionKey Returns the reflection state for the specified key as a bool. -Will generally raise NotImplemented if executed on a 32bit OS. +Will generally raise NotImplementedError if executed on a 32bit OS. [clinic start generated code]*/ static PyObject * winreg_QueryReflectionKey_impl(PyObject *module, HKEY key) -/*[clinic end generated code: output=4e774af288c3ebb9 input=9f325eacb5a65d88]*/ +/*[clinic end generated code: output=4e774af288c3ebb9 input=a98fa51d55ade186]*/ { HMODULE hMod; typedef LONG (WINAPI *RQRKFunc)(HKEY, BOOL *); From webhook-mailer at python.org Sun Aug 4 10:06:59 2019 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sun, 04 Aug 2019 14:06:59 -0000 Subject: [Python-checkins] [3.7] bpo-37730: Fix usage of NotImplemented instead of NotImplementedError in docs. (GH-15062). (GH-15106) Message-ID: https://github.com/python/cpython/commit/8cd630578451a062dcf46fbb21274520ae36c399 commit: 8cd630578451a062dcf46fbb21274520ae36c399 branch: 3.7 author: Serhiy Storchaka committer: GitHub date: 2019-08-04T17:06:51+03:00 summary: [3.7] bpo-37730: Fix usage of NotImplemented instead of NotImplementedError in docs. (GH-15062). (GH-15106) (cherry picked from commit ed5e8e06cbf766e89d6c58a882ee024abb5b2ed7) Co-authored-by: David H files: M Doc/library/winreg.rst M PC/clinic/winreg.c.h M PC/winreg.c diff --git a/Doc/library/winreg.rst b/Doc/library/winreg.rst index e9c026102273..cb67f2f39d76 100644 --- a/Doc/library/winreg.rst +++ b/Doc/library/winreg.rst @@ -456,7 +456,7 @@ This module offers the following functions: *key* is an already open key, or one of the predefined :ref:`HKEY_* constants `. - Will generally raise :exc:`NotImplemented` if executed on a 32-bit operating + Will generally raise :exc:`NotImplementedError` if executed on a 32-bit operating system. If the key is not on the reflection list, the function succeeds but has no @@ -471,7 +471,7 @@ This module offers the following functions: *key* is an already open key, or one of the predefined :ref:`HKEY_* constants `. - Will generally raise :exc:`NotImplemented` if executed on a 32-bit operating + Will generally raise :exc:`NotImplementedError` if executed on a 32-bit operating system. Restoring reflection for a key does not affect reflection of any subkeys. @@ -486,7 +486,7 @@ This module offers the following functions: Returns ``True`` if reflection is disabled. - Will generally raise :exc:`NotImplemented` if executed on a 32-bit + Will generally raise :exc:`NotImplementedError` if executed on a 32-bit operating system. diff --git a/PC/clinic/winreg.c.h b/PC/clinic/winreg.c.h index 6a5f613808c2..e9df3675b146 100644 --- a/PC/clinic/winreg.c.h +++ b/PC/clinic/winreg.c.h @@ -1003,7 +1003,7 @@ PyDoc_STRVAR(winreg_DisableReflectionKey__doc__, " key\n" " An already open key, or any one of the predefined HKEY_* constants.\n" "\n" -"Will generally raise NotImplemented if executed on a 32bit OS.\n" +"Will generally raise NotImplementedError if executed on a 32bit OS.\n" "\n" "If the key is not on the reflection list, the function succeeds but has\n" "no effect. Disabling reflection for a key does not affect reflection\n" @@ -1039,7 +1039,7 @@ PyDoc_STRVAR(winreg_EnableReflectionKey__doc__, " key\n" " An already open key, or any one of the predefined HKEY_* constants.\n" "\n" -"Will generally raise NotImplemented if executed on a 32bit OS.\n" +"Will generally raise NotImplementedError if executed on a 32bit OS.\n" "Restoring reflection for a key does not affect reflection of any\n" "subkeys."); @@ -1073,7 +1073,7 @@ PyDoc_STRVAR(winreg_QueryReflectionKey__doc__, " key\n" " An already open key, or any one of the predefined HKEY_* constants.\n" "\n" -"Will generally raise NotImplemented if executed on a 32bit OS."); +"Will generally raise NotImplementedError if executed on a 32bit OS."); #define WINREG_QUERYREFLECTIONKEY_METHODDEF \ {"QueryReflectionKey", (PyCFunction)winreg_QueryReflectionKey, METH_O, winreg_QueryReflectionKey__doc__}, @@ -1095,4 +1095,4 @@ winreg_QueryReflectionKey(PyObject *module, PyObject *arg) exit: return return_value; } -/*[clinic end generated code: output=60c92ffc7438f8cf input=a9049054013a1b77]*/ +/*[clinic end generated code: output=696f627d310ac599 input=a9049054013a1b77]*/ diff --git a/PC/winreg.c b/PC/winreg.c index 1021609e582f..5f18876895c3 100644 --- a/PC/winreg.c +++ b/PC/winreg.c @@ -1697,7 +1697,7 @@ winreg.DisableReflectionKey Disables registry reflection for 32bit processes running on a 64bit OS. -Will generally raise NotImplemented if executed on a 32bit OS. +Will generally raise NotImplementedError if executed on a 32bit OS. If the key is not on the reflection list, the function succeeds but has no effect. Disabling reflection for a key does not affect reflection @@ -1706,7 +1706,7 @@ of any subkeys. static PyObject * winreg_DisableReflectionKey_impl(PyObject *module, HKEY key) -/*[clinic end generated code: output=830cce504cc764b4 input=a6c9e5ca5410193c]*/ +/*[clinic end generated code: output=830cce504cc764b4 input=70bece2dee02e073]*/ { HMODULE hMod; typedef LONG (WINAPI *RDRKFunc)(HKEY); @@ -1742,14 +1742,14 @@ winreg.EnableReflectionKey Restores registry reflection for the specified disabled key. -Will generally raise NotImplemented if executed on a 32bit OS. +Will generally raise NotImplementedError if executed on a 32bit OS. Restoring reflection for a key does not affect reflection of any subkeys. [clinic start generated code]*/ static PyObject * winreg_EnableReflectionKey_impl(PyObject *module, HKEY key) -/*[clinic end generated code: output=86fa1385fdd9ce57 input=7748abbacd1e166a]*/ +/*[clinic end generated code: output=86fa1385fdd9ce57 input=eeae770c6eb9f559]*/ { HMODULE hMod; typedef LONG (WINAPI *RERKFunc)(HKEY); @@ -1785,12 +1785,12 @@ winreg.QueryReflectionKey Returns the reflection state for the specified key as a bool. -Will generally raise NotImplemented if executed on a 32bit OS. +Will generally raise NotImplementedError if executed on a 32bit OS. [clinic start generated code]*/ static PyObject * winreg_QueryReflectionKey_impl(PyObject *module, HKEY key) -/*[clinic end generated code: output=4e774af288c3ebb9 input=9f325eacb5a65d88]*/ +/*[clinic end generated code: output=4e774af288c3ebb9 input=a98fa51d55ade186]*/ { HMODULE hMod; typedef LONG (WINAPI *RQRKFunc)(HKEY, BOOL *); From webhook-mailer at python.org Sun Aug 4 12:25:34 2019 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Sun, 04 Aug 2019 16:25:34 -0000 Subject: [Python-checkins] bpo-37706: IDLE - fix sidebar code bug and drag tests (GH-15103) Message-ID: https://github.com/python/cpython/commit/86f1a18abfee5939452f468d80de998918e2afd2 commit: 86f1a18abfee5939452f468d80de998918e2afd2 branch: master author: Tal Einat committer: Terry Jan Reedy date: 2019-08-04T12:25:27-04:00 summary: bpo-37706: IDLE - fix sidebar code bug and drag tests (GH-15103) Convert mouse y to line number in the sidebar rather than the text. files: M Lib/idlelib/idle_test/htest.py M Lib/idlelib/idle_test/test_sidebar.py M Lib/idlelib/sidebar.py diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py index 6990af519b1f..1373b7642a6e 100644 --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -210,13 +210,20 @@ def _wrapper(parent): # htest # 'file': 'sidebar', 'kwds': {}, 'msg': textwrap.dedent("""\ - Click on the line numbers and drag down below the edge of the + 1. Click on the line numbers and drag down below the edge of the window, moving the mouse a bit and then leaving it there for a while. The text and line numbers should gradually scroll down, with the selection updated continuously. - Do the same as above, dragging to above the window. The text and line + + 2. With the lines still selected, click on a line number above the + selected lines. Only the line whose number was clicked should be + selected. + + 3. Repeat step #1, dragging to above the window. The text and line numbers should gradually scroll up, with the selection updated - continuously."""), + continuously. + + 4. Repeat step #2, clicking a line number below the selection."""), } _multi_call_spec = { diff --git a/Lib/idlelib/idle_test/test_sidebar.py b/Lib/idlelib/idle_test/test_sidebar.py index 07e8f2e96b90..0f5b4c712232 100644 --- a/Lib/idlelib/idle_test/test_sidebar.py +++ b/Lib/idlelib/idle_test/test_sidebar.py @@ -240,7 +240,6 @@ def get_width(): self.assert_sidebar_n_lines(1) self.assertEqual(get_width(), 1) - @unittest.skipIf(platform == 'darwin', 'test tk version dependent') def test_click_selection(self): self.linenumber.show_sidebar() self.text.insert('1.0', 'one\ntwo\nthree\nfour\n') @@ -254,44 +253,47 @@ def test_click_selection(self): self.assertEqual(self.get_selection(), ('2.0', '3.0')) - @unittest.skipIf(platform == 'darwin', 'test tk version dependent') - def test_drag_selection_down(self): - self.linenumber.show_sidebar() - self.text.insert('1.0', 'one\ntwo\nthree\nfour\nfive\n') - self.root.update() + def simulate_drag(self, start_line, end_line): + start_x, start_y = self.get_line_screen_position(start_line) + end_x, end_y = self.get_line_screen_position(end_line) - # Drag from the second line to the fourth line. - start_x, start_y = self.get_line_screen_position(2) - end_x, end_y = self.get_line_screen_position(4) self.linenumber.sidebar_text.event_generate('', x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=end_x, y=end_y) + self.root.update() + + def lerp(a, b, steps): + """linearly interpolate from a to b (inclusive) in equal steps""" + last_step = steps - 1 + for i in range(steps): + yield ((last_step - i) / last_step) * a + (i / last_step) * b + + for x, y in zip( + map(int, lerp(start_x, end_x, steps=11)), + map(int, lerp(start_y, end_y, steps=11)), + ): + self.linenumber.sidebar_text.event_generate('', x=x, y=y) + self.root.update() + self.linenumber.sidebar_text.event_generate('', x=end_x, y=end_y) self.root.update() + + def test_drag_selection_down(self): + self.linenumber.show_sidebar() + self.text.insert('1.0', 'one\ntwo\nthree\nfour\nfive\n') + self.root.update() + + # Drag from the second line to the fourth line. + self.simulate_drag(2, 4) self.assertEqual(self.get_selection(), ('2.0', '5.0')) - @unittest.skipIf(platform == 'darwin', 'test tk version dependent') def test_drag_selection_up(self): self.linenumber.show_sidebar() self.text.insert('1.0', 'one\ntwo\nthree\nfour\nfive\n') self.root.update() # Drag from the fourth line to the second line. - start_x, start_y = self.get_line_screen_position(4) - end_x, end_y = self.get_line_screen_position(2) - self.linenumber.sidebar_text.event_generate('', - x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=end_x, y=end_y) - self.linenumber.sidebar_text.event_generate('', - x=end_x, y=end_y) - self.root.update() + self.simulate_drag(4, 2) self.assertEqual(self.get_selection(), ('2.0', '5.0')) def test_scroll(self): diff --git a/Lib/idlelib/sidebar.py b/Lib/idlelib/sidebar.py index 76964a4cdfbe..41c09684a202 100644 --- a/Lib/idlelib/sidebar.py +++ b/Lib/idlelib/sidebar.py @@ -204,10 +204,19 @@ def bind_mouse_event(event_name, target_event_name): bind_mouse_event(event_name, target_event_name=f'') + # This is set by b1_mousedown_handler() and read by + # drag_update_selection_and_insert_mark(), to know where dragging + # began. start_line = None + # These are set by b1_motion_handler() and read by selection_handler(). + # last_y is passed this way since the mouse Y-coordinate is not + # available on selection event objects. last_yview is passed this way + # to recognize scrolling while the mouse isn't moving. + last_y = last_yview = None + def b1_mousedown_handler(event): # select the entire line - lineno = self.editwin.getlineno(f"@0,{event.y}") + lineno = int(float(self.sidebar_text.index(f"@0,{event.y}"))) self.text.tag_remove("sel", "1.0", "end") self.text.tag_add("sel", f"{lineno}.0", f"{lineno+1}.0") self.text.mark_set("insert", f"{lineno+1}.0") @@ -217,15 +226,20 @@ def b1_mousedown_handler(event): start_line = lineno self.sidebar_text.bind('', b1_mousedown_handler) - # These are set by b1_motion_handler() and read by selection_handler(); - # see below. last_y is passed this way since the mouse Y-coordinate - # is not available on selection event objects. last_yview is passed - # this way to recognize scrolling while the mouse isn't moving. - last_y = last_yview = None + def b1_mouseup_handler(event): + # On mouse up, we're no longer dragging. Set the shared persistent + # variables to None to represent this. + nonlocal start_line + nonlocal last_y + nonlocal last_yview + start_line = None + last_y = None + last_yview = None + self.sidebar_text.bind('', b1_mouseup_handler) def drag_update_selection_and_insert_mark(y_coord): """Helper function for drag and selection event handlers.""" - lineno = self.editwin.getlineno(f"@0,{y_coord}") + lineno = int(float(self.sidebar_text.index(f"@0,{y_coord}"))) a, b = sorted([start_line, lineno]) self.text.tag_remove("sel", "1.0", "end") self.text.tag_add("sel", f"{a}.0", f"{b+1}.0") @@ -253,6 +267,9 @@ def b1_drag_handler(event, *args): # while the mouse isn't moving, leading to the above fix not scrolling # properly. def selection_handler(event): + if last_yview is None: + # This logic is only needed while dragging. + return yview = self.sidebar_text.yview() if yview != last_yview: self.text.yview_moveto(yview[0]) From webhook-mailer at python.org Sun Aug 4 12:44:00 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 04 Aug 2019 16:44:00 -0000 Subject: [Python-checkins] bpo-37706: IDLE - fix sidebar code bug and drag tests (GH-15103) Message-ID: https://github.com/python/cpython/commit/1d2b4dba6ccbeb90ef797034b775c0f1ed17d1a0 commit: 1d2b4dba6ccbeb90ef797034b775c0f1ed17d1a0 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-04T09:43:56-07:00 summary: bpo-37706: IDLE - fix sidebar code bug and drag tests (GH-15103) Convert mouse y to line number in the sidebar rather than the text. (cherry picked from commit 86f1a18abfee5939452f468d80de998918e2afd2) Co-authored-by: Tal Einat files: M Lib/idlelib/idle_test/htest.py M Lib/idlelib/idle_test/test_sidebar.py M Lib/idlelib/sidebar.py diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py index 6990af519b1f..1373b7642a6e 100644 --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -210,13 +210,20 @@ def _wrapper(parent): # htest # 'file': 'sidebar', 'kwds': {}, 'msg': textwrap.dedent("""\ - Click on the line numbers and drag down below the edge of the + 1. Click on the line numbers and drag down below the edge of the window, moving the mouse a bit and then leaving it there for a while. The text and line numbers should gradually scroll down, with the selection updated continuously. - Do the same as above, dragging to above the window. The text and line + + 2. With the lines still selected, click on a line number above the + selected lines. Only the line whose number was clicked should be + selected. + + 3. Repeat step #1, dragging to above the window. The text and line numbers should gradually scroll up, with the selection updated - continuously."""), + continuously. + + 4. Repeat step #2, clicking a line number below the selection."""), } _multi_call_spec = { diff --git a/Lib/idlelib/idle_test/test_sidebar.py b/Lib/idlelib/idle_test/test_sidebar.py index 07e8f2e96b90..0f5b4c712232 100644 --- a/Lib/idlelib/idle_test/test_sidebar.py +++ b/Lib/idlelib/idle_test/test_sidebar.py @@ -240,7 +240,6 @@ def get_width(): self.assert_sidebar_n_lines(1) self.assertEqual(get_width(), 1) - @unittest.skipIf(platform == 'darwin', 'test tk version dependent') def test_click_selection(self): self.linenumber.show_sidebar() self.text.insert('1.0', 'one\ntwo\nthree\nfour\n') @@ -254,44 +253,47 @@ def test_click_selection(self): self.assertEqual(self.get_selection(), ('2.0', '3.0')) - @unittest.skipIf(platform == 'darwin', 'test tk version dependent') - def test_drag_selection_down(self): - self.linenumber.show_sidebar() - self.text.insert('1.0', 'one\ntwo\nthree\nfour\nfive\n') - self.root.update() + def simulate_drag(self, start_line, end_line): + start_x, start_y = self.get_line_screen_position(start_line) + end_x, end_y = self.get_line_screen_position(end_line) - # Drag from the second line to the fourth line. - start_x, start_y = self.get_line_screen_position(2) - end_x, end_y = self.get_line_screen_position(4) self.linenumber.sidebar_text.event_generate('', x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=end_x, y=end_y) + self.root.update() + + def lerp(a, b, steps): + """linearly interpolate from a to b (inclusive) in equal steps""" + last_step = steps - 1 + for i in range(steps): + yield ((last_step - i) / last_step) * a + (i / last_step) * b + + for x, y in zip( + map(int, lerp(start_x, end_x, steps=11)), + map(int, lerp(start_y, end_y, steps=11)), + ): + self.linenumber.sidebar_text.event_generate('', x=x, y=y) + self.root.update() + self.linenumber.sidebar_text.event_generate('', x=end_x, y=end_y) self.root.update() + + def test_drag_selection_down(self): + self.linenumber.show_sidebar() + self.text.insert('1.0', 'one\ntwo\nthree\nfour\nfive\n') + self.root.update() + + # Drag from the second line to the fourth line. + self.simulate_drag(2, 4) self.assertEqual(self.get_selection(), ('2.0', '5.0')) - @unittest.skipIf(platform == 'darwin', 'test tk version dependent') def test_drag_selection_up(self): self.linenumber.show_sidebar() self.text.insert('1.0', 'one\ntwo\nthree\nfour\nfive\n') self.root.update() # Drag from the fourth line to the second line. - start_x, start_y = self.get_line_screen_position(4) - end_x, end_y = self.get_line_screen_position(2) - self.linenumber.sidebar_text.event_generate('', - x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=end_x, y=end_y) - self.linenumber.sidebar_text.event_generate('', - x=end_x, y=end_y) - self.root.update() + self.simulate_drag(4, 2) self.assertEqual(self.get_selection(), ('2.0', '5.0')) def test_scroll(self): diff --git a/Lib/idlelib/sidebar.py b/Lib/idlelib/sidebar.py index 76964a4cdfbe..41c09684a202 100644 --- a/Lib/idlelib/sidebar.py +++ b/Lib/idlelib/sidebar.py @@ -204,10 +204,19 @@ def bind_mouse_event(event_name, target_event_name): bind_mouse_event(event_name, target_event_name=f'') + # This is set by b1_mousedown_handler() and read by + # drag_update_selection_and_insert_mark(), to know where dragging + # began. start_line = None + # These are set by b1_motion_handler() and read by selection_handler(). + # last_y is passed this way since the mouse Y-coordinate is not + # available on selection event objects. last_yview is passed this way + # to recognize scrolling while the mouse isn't moving. + last_y = last_yview = None + def b1_mousedown_handler(event): # select the entire line - lineno = self.editwin.getlineno(f"@0,{event.y}") + lineno = int(float(self.sidebar_text.index(f"@0,{event.y}"))) self.text.tag_remove("sel", "1.0", "end") self.text.tag_add("sel", f"{lineno}.0", f"{lineno+1}.0") self.text.mark_set("insert", f"{lineno+1}.0") @@ -217,15 +226,20 @@ def b1_mousedown_handler(event): start_line = lineno self.sidebar_text.bind('', b1_mousedown_handler) - # These are set by b1_motion_handler() and read by selection_handler(); - # see below. last_y is passed this way since the mouse Y-coordinate - # is not available on selection event objects. last_yview is passed - # this way to recognize scrolling while the mouse isn't moving. - last_y = last_yview = None + def b1_mouseup_handler(event): + # On mouse up, we're no longer dragging. Set the shared persistent + # variables to None to represent this. + nonlocal start_line + nonlocal last_y + nonlocal last_yview + start_line = None + last_y = None + last_yview = None + self.sidebar_text.bind('', b1_mouseup_handler) def drag_update_selection_and_insert_mark(y_coord): """Helper function for drag and selection event handlers.""" - lineno = self.editwin.getlineno(f"@0,{y_coord}") + lineno = int(float(self.sidebar_text.index(f"@0,{y_coord}"))) a, b = sorted([start_line, lineno]) self.text.tag_remove("sel", "1.0", "end") self.text.tag_add("sel", f"{a}.0", f"{b+1}.0") @@ -253,6 +267,9 @@ def b1_drag_handler(event, *args): # while the mouse isn't moving, leading to the above fix not scrolling # properly. def selection_handler(event): + if last_yview is None: + # This logic is only needed while dragging. + return yview = self.sidebar_text.yview() if yview != last_yview: self.text.yview_moveto(yview[0]) From webhook-mailer at python.org Sun Aug 4 12:47:15 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 04 Aug 2019 16:47:15 -0000 Subject: [Python-checkins] bpo-37706: IDLE - fix sidebar code bug and drag tests (GH-15103) Message-ID: https://github.com/python/cpython/commit/9e0c48a002c144d910a46a74b76b02179502b9aa commit: 9e0c48a002c144d910a46a74b76b02179502b9aa branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-04T09:47:11-07:00 summary: bpo-37706: IDLE - fix sidebar code bug and drag tests (GH-15103) Convert mouse y to line number in the sidebar rather than the text. (cherry picked from commit 86f1a18abfee5939452f468d80de998918e2afd2) Co-authored-by: Tal Einat files: M Lib/idlelib/idle_test/htest.py M Lib/idlelib/idle_test/test_sidebar.py M Lib/idlelib/sidebar.py diff --git a/Lib/idlelib/idle_test/htest.py b/Lib/idlelib/idle_test/htest.py index 6990af519b1f..1373b7642a6e 100644 --- a/Lib/idlelib/idle_test/htest.py +++ b/Lib/idlelib/idle_test/htest.py @@ -210,13 +210,20 @@ def _wrapper(parent): # htest # 'file': 'sidebar', 'kwds': {}, 'msg': textwrap.dedent("""\ - Click on the line numbers and drag down below the edge of the + 1. Click on the line numbers and drag down below the edge of the window, moving the mouse a bit and then leaving it there for a while. The text and line numbers should gradually scroll down, with the selection updated continuously. - Do the same as above, dragging to above the window. The text and line + + 2. With the lines still selected, click on a line number above the + selected lines. Only the line whose number was clicked should be + selected. + + 3. Repeat step #1, dragging to above the window. The text and line numbers should gradually scroll up, with the selection updated - continuously."""), + continuously. + + 4. Repeat step #2, clicking a line number below the selection."""), } _multi_call_spec = { diff --git a/Lib/idlelib/idle_test/test_sidebar.py b/Lib/idlelib/idle_test/test_sidebar.py index 07e8f2e96b90..0f5b4c712232 100644 --- a/Lib/idlelib/idle_test/test_sidebar.py +++ b/Lib/idlelib/idle_test/test_sidebar.py @@ -240,7 +240,6 @@ def get_width(): self.assert_sidebar_n_lines(1) self.assertEqual(get_width(), 1) - @unittest.skipIf(platform == 'darwin', 'test tk version dependent') def test_click_selection(self): self.linenumber.show_sidebar() self.text.insert('1.0', 'one\ntwo\nthree\nfour\n') @@ -254,44 +253,47 @@ def test_click_selection(self): self.assertEqual(self.get_selection(), ('2.0', '3.0')) - @unittest.skipIf(platform == 'darwin', 'test tk version dependent') - def test_drag_selection_down(self): - self.linenumber.show_sidebar() - self.text.insert('1.0', 'one\ntwo\nthree\nfour\nfive\n') - self.root.update() + def simulate_drag(self, start_line, end_line): + start_x, start_y = self.get_line_screen_position(start_line) + end_x, end_y = self.get_line_screen_position(end_line) - # Drag from the second line to the fourth line. - start_x, start_y = self.get_line_screen_position(2) - end_x, end_y = self.get_line_screen_position(4) self.linenumber.sidebar_text.event_generate('', x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=end_x, y=end_y) + self.root.update() + + def lerp(a, b, steps): + """linearly interpolate from a to b (inclusive) in equal steps""" + last_step = steps - 1 + for i in range(steps): + yield ((last_step - i) / last_step) * a + (i / last_step) * b + + for x, y in zip( + map(int, lerp(start_x, end_x, steps=11)), + map(int, lerp(start_y, end_y, steps=11)), + ): + self.linenumber.sidebar_text.event_generate('', x=x, y=y) + self.root.update() + self.linenumber.sidebar_text.event_generate('', x=end_x, y=end_y) self.root.update() + + def test_drag_selection_down(self): + self.linenumber.show_sidebar() + self.text.insert('1.0', 'one\ntwo\nthree\nfour\nfive\n') + self.root.update() + + # Drag from the second line to the fourth line. + self.simulate_drag(2, 4) self.assertEqual(self.get_selection(), ('2.0', '5.0')) - @unittest.skipIf(platform == 'darwin', 'test tk version dependent') def test_drag_selection_up(self): self.linenumber.show_sidebar() self.text.insert('1.0', 'one\ntwo\nthree\nfour\nfive\n') self.root.update() # Drag from the fourth line to the second line. - start_x, start_y = self.get_line_screen_position(4) - end_x, end_y = self.get_line_screen_position(2) - self.linenumber.sidebar_text.event_generate('', - x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=start_x, y=start_y) - self.linenumber.sidebar_text.event_generate('', - x=end_x, y=end_y) - self.linenumber.sidebar_text.event_generate('', - x=end_x, y=end_y) - self.root.update() + self.simulate_drag(4, 2) self.assertEqual(self.get_selection(), ('2.0', '5.0')) def test_scroll(self): diff --git a/Lib/idlelib/sidebar.py b/Lib/idlelib/sidebar.py index 76964a4cdfbe..41c09684a202 100644 --- a/Lib/idlelib/sidebar.py +++ b/Lib/idlelib/sidebar.py @@ -204,10 +204,19 @@ def bind_mouse_event(event_name, target_event_name): bind_mouse_event(event_name, target_event_name=f'') + # This is set by b1_mousedown_handler() and read by + # drag_update_selection_and_insert_mark(), to know where dragging + # began. start_line = None + # These are set by b1_motion_handler() and read by selection_handler(). + # last_y is passed this way since the mouse Y-coordinate is not + # available on selection event objects. last_yview is passed this way + # to recognize scrolling while the mouse isn't moving. + last_y = last_yview = None + def b1_mousedown_handler(event): # select the entire line - lineno = self.editwin.getlineno(f"@0,{event.y}") + lineno = int(float(self.sidebar_text.index(f"@0,{event.y}"))) self.text.tag_remove("sel", "1.0", "end") self.text.tag_add("sel", f"{lineno}.0", f"{lineno+1}.0") self.text.mark_set("insert", f"{lineno+1}.0") @@ -217,15 +226,20 @@ def b1_mousedown_handler(event): start_line = lineno self.sidebar_text.bind('', b1_mousedown_handler) - # These are set by b1_motion_handler() and read by selection_handler(); - # see below. last_y is passed this way since the mouse Y-coordinate - # is not available on selection event objects. last_yview is passed - # this way to recognize scrolling while the mouse isn't moving. - last_y = last_yview = None + def b1_mouseup_handler(event): + # On mouse up, we're no longer dragging. Set the shared persistent + # variables to None to represent this. + nonlocal start_line + nonlocal last_y + nonlocal last_yview + start_line = None + last_y = None + last_yview = None + self.sidebar_text.bind('', b1_mouseup_handler) def drag_update_selection_and_insert_mark(y_coord): """Helper function for drag and selection event handlers.""" - lineno = self.editwin.getlineno(f"@0,{y_coord}") + lineno = int(float(self.sidebar_text.index(f"@0,{y_coord}"))) a, b = sorted([start_line, lineno]) self.text.tag_remove("sel", "1.0", "end") self.text.tag_add("sel", f"{a}.0", f"{b+1}.0") @@ -253,6 +267,9 @@ def b1_drag_handler(event, *args): # while the mouse isn't moving, leading to the above fix not scrolling # properly. def selection_handler(event): + if last_yview is None: + # This logic is only needed while dragging. + return yview = self.sidebar_text.yview() if yview != last_yview: self.text.yview_moveto(yview[0]) From webhook-mailer at python.org Sun Aug 4 14:52:09 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sun, 04 Aug 2019 18:52:09 -0000 Subject: [Python-checkins] bpo-36324: Update comments to include the target hash sums (GH-15110) Message-ID: https://github.com/python/cpython/commit/8183bb8150edcac6a7525bfb7708d08837ecb095 commit: 8183bb8150edcac6a7525bfb7708d08837ecb095 branch: master author: Raymond Hettinger committer: GitHub date: 2019-08-04T11:52:04-07:00 summary: bpo-36324: Update comments to include the target hash sums (GH-15110) files: M Lib/statistics.py diff --git a/Lib/statistics.py b/Lib/statistics.py index 8a6be7c75901..77291dd62cb9 100644 --- a/Lib/statistics.py +++ b/Lib/statistics.py @@ -892,6 +892,7 @@ def inv_cdf(self, p): q = p - 0.5 if fabs(q) <= 0.425: r = 0.180625 - q * q + # Hash sum: 55.88319_28806_14901_4439 num = (((((((2.50908_09287_30122_6727e+3 * r + 3.34305_75583_58812_8105e+4) * r + 6.72657_70927_00870_0853e+4) * r + @@ -914,6 +915,7 @@ def inv_cdf(self, p): r = sqrt(-log(r)) if r <= 5.0: r = r - 1.6 + # Hash sum: 49.33206_50330_16102_89036 num = (((((((7.74545_01427_83414_07640e-4 * r + 2.27238_44989_26918_45833e-2) * r + 2.41780_72517_74506_11770e-1) * r + @@ -932,6 +934,7 @@ def inv_cdf(self, p): 1.0) else: r = r - 5.0 + # Hash sum: 47.52583_31754_92896_71629 num = (((((((2.01033_43992_92288_13265e-7 * r + 2.71155_55687_43487_57815e-5) * r + 1.24266_09473_88078_43860e-3) * r + From webhook-mailer at python.org Sun Aug 4 15:16:23 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sun, 04 Aug 2019 19:16:23 -0000 Subject: [Python-checkins] bpo-36324: Update comments to include the target hash sums (GH-15110) (GH-15112) Message-ID: https://github.com/python/cpython/commit/d7d607c000a28432f87d24d2a5a9315c51017041 commit: d7d607c000a28432f87d24d2a5a9315c51017041 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Raymond Hettinger date: 2019-08-04T12:16:19-07:00 summary: bpo-36324: Update comments to include the target hash sums (GH-15110) (GH-15112) (cherry picked from commit 8183bb8150edcac6a7525bfb7708d08837ecb095) Co-authored-by: Raymond Hettinger files: M Lib/statistics.py diff --git a/Lib/statistics.py b/Lib/statistics.py index 8a6be7c75901..77291dd62cb9 100644 --- a/Lib/statistics.py +++ b/Lib/statistics.py @@ -892,6 +892,7 @@ def inv_cdf(self, p): q = p - 0.5 if fabs(q) <= 0.425: r = 0.180625 - q * q + # Hash sum: 55.88319_28806_14901_4439 num = (((((((2.50908_09287_30122_6727e+3 * r + 3.34305_75583_58812_8105e+4) * r + 6.72657_70927_00870_0853e+4) * r + @@ -914,6 +915,7 @@ def inv_cdf(self, p): r = sqrt(-log(r)) if r <= 5.0: r = r - 1.6 + # Hash sum: 49.33206_50330_16102_89036 num = (((((((7.74545_01427_83414_07640e-4 * r + 2.27238_44989_26918_45833e-2) * r + 2.41780_72517_74506_11770e-1) * r + @@ -932,6 +934,7 @@ def inv_cdf(self, p): 1.0) else: r = r - 5.0 + # Hash sum: 47.52583_31754_92896_71629 num = (((((((2.01033_43992_92288_13265e-7 * r + 2.71155_55687_43487_57815e-5) * r + 1.24266_09473_88078_43860e-3) * r + From webhook-mailer at python.org Sun Aug 4 15:26:40 2019 From: webhook-mailer at python.org (Tal Einat) Date: Sun, 04 Aug 2019 19:26:40 -0000 Subject: [Python-checkins] [3.7] bpo-34621: backwards-compatible pickle UUID with is_safe=unknown (GH-14834) Message-ID: https://github.com/python/cpython/commit/a2ea9448c677706d6318eaa71101f08df7604eb9 commit: a2ea9448c677706d6318eaa71101f08df7604eb9 branch: 3.7 author: Tal Einat committer: GitHub date: 2019-08-04T22:26:32+03:00 summary: [3.7] bpo-34621: backwards-compatible pickle UUID with is_safe=unknown (GH-14834) This is a fix for a bug introduced in the original implementation of this for 3.7. files: A Misc/NEWS.d/next/Library/2019-08-04-22-06-54.bpo-34621.E2EWkw.rst M Lib/uuid.py diff --git a/Lib/uuid.py b/Lib/uuid.py index 26faa1accd09..b1abfe315d56 100644 --- a/Lib/uuid.py +++ b/Lib/uuid.py @@ -205,12 +205,14 @@ def __init__(self, hex=None, bytes=None, bytes_le=None, fields=None, self.__dict__['is_safe'] = is_safe def __getstate__(self): - state = self.__dict__ + state = self.__dict__.copy() if self.is_safe != SafeUUID.unknown: # is_safe is a SafeUUID instance. Return just its value, so that # it can be un-pickled in older Python versions without SafeUUID. - state = state.copy() state['is_safe'] = self.is_safe.value + else: + # omit is_safe when it is "unknown" + del state['is_safe'] return state def __setstate__(self, state): diff --git a/Misc/NEWS.d/next/Library/2019-08-04-22-06-54.bpo-34621.E2EWkw.rst b/Misc/NEWS.d/next/Library/2019-08-04-22-06-54.bpo-34621.E2EWkw.rst new file mode 100644 index 000000000000..1128a2fb4459 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-04-22-06-54.bpo-34621.E2EWkw.rst @@ -0,0 +1,2 @@ +Fixed unpickle-ability in older Python versions (<3.7) of UUID objects with +``is_safe`` set to ``SafeUUID.unknown``. From webhook-mailer at python.org Sun Aug 4 16:14:07 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sun, 04 Aug 2019 20:14:07 -0000 Subject: [Python-checkins] bpo-28292: Mark calendar.py helper functions as private. (GH-15113) Message-ID: https://github.com/python/cpython/commit/b1c8ec010fb4eb2654ca994e95144c8f2fea07fb commit: b1c8ec010fb4eb2654ca994e95144c8f2fea07fb branch: master author: Raymond Hettinger committer: GitHub date: 2019-08-04T13:14:03-07:00 summary: bpo-28292: Mark calendar.py helper functions as private. (GH-15113) files: A Misc/NEWS.d/next/Library/2019-08-04-11-47-58.bpo-28292.vkihH5.rst M Lib/calendar.py diff --git a/Lib/calendar.py b/Lib/calendar.py index 3828c43ed279..7550d52c0a94 100644 --- a/Lib/calendar.py +++ b/Lib/calendar.py @@ -127,18 +127,18 @@ def monthrange(year, month): return day1, ndays -def monthlen(year, month): +def _monthlen(year, month): return mdays[month] + (month == February and isleap(year)) -def prevmonth(year, month): +def _prevmonth(year, month): if month == 1: return year-1, 12 else: return year, month-1 -def nextmonth(year, month): +def _nextmonth(year, month): if month == 12: return year+1, 1 else: @@ -207,13 +207,13 @@ def itermonthdays3(self, year, month): day1, ndays = monthrange(year, month) days_before = (day1 - self.firstweekday) % 7 days_after = (self.firstweekday - day1 - ndays) % 7 - y, m = prevmonth(year, month) - end = monthlen(y, m) + 1 + y, m = _prevmonth(year, month) + end = _monthlen(y, m) + 1 for d in range(end-days_before, end): yield y, m, d for d in range(1, ndays + 1): yield year, month, d - y, m = nextmonth(year, month) + y, m = _nextmonth(year, month) for d in range(1, days_after + 1): yield y, m, d diff --git a/Misc/NEWS.d/next/Library/2019-08-04-11-47-58.bpo-28292.vkihH5.rst b/Misc/NEWS.d/next/Library/2019-08-04-11-47-58.bpo-28292.vkihH5.rst new file mode 100644 index 000000000000..478a1b03c195 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-04-11-47-58.bpo-28292.vkihH5.rst @@ -0,0 +1,3 @@ +Mark calendar.py helper functions as being private. The follows PEP 8 +guidance to maintain the style conventions in the module and it addresses a +known case of user confusion. From webhook-mailer at python.org Sun Aug 4 16:35:00 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sun, 04 Aug 2019 20:35:00 -0000 Subject: [Python-checkins] bpo-28292: Mark calendar.py helper functions as private. (GH-15113) (GH-15116) Message-ID: https://github.com/python/cpython/commit/0c16f6b307f7607e29b98b8fbb99cbca28f91a48 commit: 0c16f6b307f7607e29b98b8fbb99cbca28f91a48 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Raymond Hettinger date: 2019-08-04T13:34:56-07:00 summary: bpo-28292: Mark calendar.py helper functions as private. (GH-15113) (GH-15116) (cherry picked from commit b1c8ec010fb4eb2654ca994e95144c8f2fea07fb) Co-authored-by: Raymond Hettinger files: A Misc/NEWS.d/next/Library/2019-08-04-11-47-58.bpo-28292.vkihH5.rst M Lib/calendar.py diff --git a/Lib/calendar.py b/Lib/calendar.py index 3828c43ed279..7550d52c0a94 100644 --- a/Lib/calendar.py +++ b/Lib/calendar.py @@ -127,18 +127,18 @@ def monthrange(year, month): return day1, ndays -def monthlen(year, month): +def _monthlen(year, month): return mdays[month] + (month == February and isleap(year)) -def prevmonth(year, month): +def _prevmonth(year, month): if month == 1: return year-1, 12 else: return year, month-1 -def nextmonth(year, month): +def _nextmonth(year, month): if month == 12: return year+1, 1 else: @@ -207,13 +207,13 @@ def itermonthdays3(self, year, month): day1, ndays = monthrange(year, month) days_before = (day1 - self.firstweekday) % 7 days_after = (self.firstweekday - day1 - ndays) % 7 - y, m = prevmonth(year, month) - end = monthlen(y, m) + 1 + y, m = _prevmonth(year, month) + end = _monthlen(y, m) + 1 for d in range(end-days_before, end): yield y, m, d for d in range(1, ndays + 1): yield year, month, d - y, m = nextmonth(year, month) + y, m = _nextmonth(year, month) for d in range(1, days_after + 1): yield y, m, d diff --git a/Misc/NEWS.d/next/Library/2019-08-04-11-47-58.bpo-28292.vkihH5.rst b/Misc/NEWS.d/next/Library/2019-08-04-11-47-58.bpo-28292.vkihH5.rst new file mode 100644 index 000000000000..478a1b03c195 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-04-11-47-58.bpo-28292.vkihH5.rst @@ -0,0 +1,3 @@ +Mark calendar.py helper functions as being private. The follows PEP 8 +guidance to maintain the style conventions in the module and it addresses a +known case of user confusion. From webhook-mailer at python.org Sun Aug 4 16:36:02 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sun, 04 Aug 2019 20:36:02 -0000 Subject: [Python-checkins] Update itertools docs (GH-15114) Message-ID: https://github.com/python/cpython/commit/adf02b36b3f3745ad4ee380d88f2f6011f54fc22 commit: adf02b36b3f3745ad4ee380d88f2f6011f54fc22 branch: master author: Raymond Hettinger committer: GitHub date: 2019-08-04T13:35:58-07:00 summary: Update itertools docs (GH-15114) * Remove suggestion that is less relevant now that global lookups are much faster * Add link for installing the recipes files: M Doc/library/itertools.rst diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index b3a0a5f5192d..a3f403a5b40b 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -691,6 +691,12 @@ Itertools Recipes This section shows recipes for creating an extended toolset using the existing itertools as building blocks. +Substantially all of these recipes and many, many others can be installed from +the `more-itertools project `_ found +on the Python Package Index:: + + pip install more-itertools + The extended tools offer the same high performance as the underlying toolset. The superior memory performance is kept by processing elements one at a time rather than bringing the whole iterable into memory all at once. Code volume is @@ -913,9 +919,3 @@ which incur interpreter overhead. result.append(pool[-1-n]) return tuple(result) -Note, many of the above recipes can be optimized by replacing global lookups -with local variables defined as default values. For example, the -*dotproduct* recipe can be written as:: - - def dotproduct(vec1, vec2, sum=sum, map=map, mul=operator.mul): - return sum(map(mul, vec1, vec2)) From webhook-mailer at python.org Sun Aug 4 16:45:19 2019 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Sun, 04 Aug 2019 20:45:19 -0000 Subject: [Python-checkins] bpo-37748: Re-order the Run menu. (GH-15115) Message-ID: https://github.com/python/cpython/commit/14070299cdc0faf36975f0cc2d51824a9abf3db0 commit: 14070299cdc0faf36975f0cc2d51824a9abf3db0 branch: master author: Terry Jan Reedy committer: GitHub date: 2019-08-04T16:45:15-04:00 summary: bpo-37748: Re-order the Run menu. (GH-15115) Put the most common choice, Run Module, at the top. files: A Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst M Doc/library/idle.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/help.html M Lib/idlelib/mainmenu.py diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 5975e3bb5806..0bd248c22b18 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -207,20 +207,6 @@ Strip trailing whitespace Run menu (Editor window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. _python-shell: - -Python Shell - Open or wake up the Python Shell window. - -.. _check-module: - -Check Module - Check the syntax of the module currently open in the Editor window. If the - module has not been saved IDLE will either prompt the user to save or - autosave, as selected in the General tab of the Idle Settings dialog. If - there is a syntax error, the approximate location is indicated in the - Editor window. - .. _run-module: Run Module @@ -239,6 +225,20 @@ Run... Customized settings. *Command Line Arguments* extend :data:`sys.argv` as if passed on a command line. The module can be run in the Shell without restarting. +.. _check-module: + +Check Module + Check the syntax of the module currently open in the Editor window. If the + module has not been saved IDLE will either prompt the user to save or + autosave, as selected in the General tab of the Idle Settings dialog. If + there is a syntax error, the approximate location is indicated in the + Editor window. + +.. _python-shell: + +Python Shell + Open or wake up the Python Shell window. + Shell menu (Shell window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -295,8 +295,8 @@ Configure IDLE menu. For more details, see :ref:`Setting preferences ` under Help and preferences. - Most configuration options apply to all windows or all future windows. - The option items below only apply to the active window. +Most configuration options apply to all windows or all future windows. +The option items below only apply to the active window. Show/Hide Code Context (Editor Window only) Open a pane at the top of the edit window which shows the block context diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index e3e92bd75f09..0fff7003b9d6 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,9 @@ Released on 2019-10-20? ====================================== +bpo-37748: Reorder the Run menu. Put the most common choice, +Run Module, at the top. + bpo-37692: Improve highlight config sample with example shell interaction and better labels for shell elements. diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html index 49068df7a8b9..91e66a4b9117 100644 --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -210,16 +210,6 @@

Edit menu (Shell and Editor)

Run menu (Editor window only)?

-
-
Python Shell
Open or wake up the Python Shell window.
-
-
-
Check Module
Check the syntax of the module currently open in the Editor window. If the -module has not been saved IDLE will either prompt the user to save or -autosave, as selected in the General tab of the Idle Settings dialog. If -there is a syntax error, the approximate location is indicated in the -Editor window.
-
Run Module
Do Check Module. If no error, restart the shell to clean the environment, then execute the module. Output is displayed in the Shell @@ -234,6 +224,16 @@

Edit menu (Shell and Editor)sys.argv as if passed on a command line. The module can be run in the Shell without restarting.

+
+
Check Module
Check the syntax of the module currently open in the Editor window. If the +module has not been saved IDLE will either prompt the user to save or +autosave, as selected in the General tab of the Idle Settings dialog. If +there is a syntax error, the approximate location is indicated in the +Editor window.
+
+
+
Python Shell
Open or wake up the Python Shell window.
+

Shell menu (Shell window only)?

@@ -268,23 +268,26 @@

Options menu (Shell and Editor)
Configure IDLE
Open a configuration dialog and change preferences for the following: fonts, indentation, keybindings, text color themes, startup windows and -size, additional help sources, and extensions. On macOS, open the +size, additional help sources, and extensions. On macOS, open the configuration dialog by selecting Preferences in the application -menu. For more, see -Setting preferences under Help and preferences. -Most configuration options apply to all windows or all future windows. -The option items below only apply to the active window.
+menu. For more details, see +Setting preferences under Help and preferences. + +

Most configuration options apply to all windows or all future windows. +The option items below only apply to the active window.

+
Show/Hide Code Context (Editor Window only)
Open a pane at the top of the edit window which shows the block context of the code which has scrolled above the top of the window. See -Code Context in the Editing and Navigation section below.
-
Line Numbers (Editor Window only)
Open a column to the left of the edit window which shows the linenumber -of each line of text. The default is off unless configured on -(see Setting preferences).
+Code Context in the Editing and Navigation section +below. +
Show/Hide Line Numbers (Editor Window only)
Open a column to the left of the edit window which shows the number +of each line of text. The default is off, which may be changed in the +preferences (see Setting preferences).
Zoom/Restore Height
Toggles the window between normal size and maximum height. The initial size defaults to 40 lines by 80 chars unless changed on the General tab of the Configure IDLE dialog. The maximum height for a screen is determined by momentarily maximizing a window the first time one is zoomed on the screen. -Changing screen settings may invalidate the saved height. This toogle has +Changing screen settings may invalidate the saved height. This toggle has no effect when a window is maximized.

@@ -900,7 +903,7 @@

Navigation



- Last updated on Jul 23, 2019. + Last updated on Aug 04, 2019. Found a bug?
diff --git a/Lib/idlelib/mainmenu.py b/Lib/idlelib/mainmenu.py index fc51fb1b5d34..74edce234838 100644 --- a/Lib/idlelib/mainmenu.py +++ b/Lib/idlelib/mainmenu.py @@ -73,10 +73,10 @@ ]), ('run', [ - ('Python Shell', '<>'), - ('C_heck Module', '<>'), ('R_un Module', '<>'), ('Run... _Customized', '<>'), + ('C_heck Module', '<>'), + ('Python Shell', '<>'), ]), ('shell', [ diff --git a/Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst b/Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst new file mode 100644 index 000000000000..fc1d6b6bb357 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst @@ -0,0 +1 @@ +Reorder the Run menu. Put the most common choice, Run Module, at the top. From webhook-mailer at python.org Sun Aug 4 16:53:03 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sun, 04 Aug 2019 20:53:03 -0000 Subject: [Python-checkins] Update itertools docs (GH-15114) (GH-15118) Message-ID: https://github.com/python/cpython/commit/fc6e3bc1cdaabac89382b45459f9b3f84d929afd commit: fc6e3bc1cdaabac89382b45459f9b3f84d929afd branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Raymond Hettinger date: 2019-08-04T13:52:59-07:00 summary: Update itertools docs (GH-15114) (GH-15118) * Remove suggestion that is less relevant now that global lookups are much faster * Add link for installing the recipes (cherry picked from commit adf02b36b3f3745ad4ee380d88f2f6011f54fc22) Co-authored-by: Raymond Hettinger files: M Doc/library/itertools.rst diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index b3a0a5f5192d..a3f403a5b40b 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -691,6 +691,12 @@ Itertools Recipes This section shows recipes for creating an extended toolset using the existing itertools as building blocks. +Substantially all of these recipes and many, many others can be installed from +the `more-itertools project `_ found +on the Python Package Index:: + + pip install more-itertools + The extended tools offer the same high performance as the underlying toolset. The superior memory performance is kept by processing elements one at a time rather than bringing the whole iterable into memory all at once. Code volume is @@ -913,9 +919,3 @@ which incur interpreter overhead. result.append(pool[-1-n]) return tuple(result) -Note, many of the above recipes can be optimized by replacing global lookups -with local variables defined as default values. For example, the -*dotproduct* recipe can be written as:: - - def dotproduct(vec1, vec2, sum=sum, map=map, mul=operator.mul): - return sum(map(mul, vec1, vec2)) From webhook-mailer at python.org Sun Aug 4 17:04:53 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 04 Aug 2019 21:04:53 -0000 Subject: [Python-checkins] bpo-37748: Re-order the Run menu. (GH-15115) Message-ID: https://github.com/python/cpython/commit/9c95fc752c1465202df67fa894ef326c8ebb8cac commit: 9c95fc752c1465202df67fa894ef326c8ebb8cac branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-04T14:04:49-07:00 summary: bpo-37748: Re-order the Run menu. (GH-15115) Put the most common choice, Run Module, at the top. (cherry picked from commit 14070299cdc0faf36975f0cc2d51824a9abf3db0) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst M Doc/library/idle.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/help.html M Lib/idlelib/mainmenu.py diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 5975e3bb5806..0bd248c22b18 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -207,20 +207,6 @@ Strip trailing whitespace Run menu (Editor window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. _python-shell: - -Python Shell - Open or wake up the Python Shell window. - -.. _check-module: - -Check Module - Check the syntax of the module currently open in the Editor window. If the - module has not been saved IDLE will either prompt the user to save or - autosave, as selected in the General tab of the Idle Settings dialog. If - there is a syntax error, the approximate location is indicated in the - Editor window. - .. _run-module: Run Module @@ -239,6 +225,20 @@ Run... Customized settings. *Command Line Arguments* extend :data:`sys.argv` as if passed on a command line. The module can be run in the Shell without restarting. +.. _check-module: + +Check Module + Check the syntax of the module currently open in the Editor window. If the + module has not been saved IDLE will either prompt the user to save or + autosave, as selected in the General tab of the Idle Settings dialog. If + there is a syntax error, the approximate location is indicated in the + Editor window. + +.. _python-shell: + +Python Shell + Open or wake up the Python Shell window. + Shell menu (Shell window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -295,8 +295,8 @@ Configure IDLE menu. For more details, see :ref:`Setting preferences ` under Help and preferences. - Most configuration options apply to all windows or all future windows. - The option items below only apply to the active window. +Most configuration options apply to all windows or all future windows. +The option items below only apply to the active window. Show/Hide Code Context (Editor Window only) Open a pane at the top of the edit window which shows the block context diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index e3e92bd75f09..0fff7003b9d6 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,9 @@ Released on 2019-10-20? ====================================== +bpo-37748: Reorder the Run menu. Put the most common choice, +Run Module, at the top. + bpo-37692: Improve highlight config sample with example shell interaction and better labels for shell elements. diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html index 49068df7a8b9..91e66a4b9117 100644 --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -210,16 +210,6 @@

Edit menu (Shell and Editor)

Run menu (Editor window only)?

-
-
Python Shell
Open or wake up the Python Shell window.
-
-
-
Check Module
Check the syntax of the module currently open in the Editor window. If the -module has not been saved IDLE will either prompt the user to save or -autosave, as selected in the General tab of the Idle Settings dialog. If -there is a syntax error, the approximate location is indicated in the -Editor window.
-
Run Module
Do Check Module. If no error, restart the shell to clean the environment, then execute the module. Output is displayed in the Shell @@ -234,6 +224,16 @@

Edit menu (Shell and Editor)sys.argv as if passed on a command line. The module can be run in the Shell without restarting.

+
+
Check Module
Check the syntax of the module currently open in the Editor window. If the +module has not been saved IDLE will either prompt the user to save or +autosave, as selected in the General tab of the Idle Settings dialog. If +there is a syntax error, the approximate location is indicated in the +Editor window.
+
+
+
Python Shell
Open or wake up the Python Shell window.
+

Shell menu (Shell window only)?

@@ -268,23 +268,26 @@

Options menu (Shell and Editor)
Configure IDLE
Open a configuration dialog and change preferences for the following: fonts, indentation, keybindings, text color themes, startup windows and -size, additional help sources, and extensions. On macOS, open the +size, additional help sources, and extensions. On macOS, open the configuration dialog by selecting Preferences in the application -menu. For more, see -Setting preferences under Help and preferences. -Most configuration options apply to all windows or all future windows. -The option items below only apply to the active window.
+menu. For more details, see +Setting preferences under Help and preferences. + +

Most configuration options apply to all windows or all future windows. +The option items below only apply to the active window.

+
Show/Hide Code Context (Editor Window only)
Open a pane at the top of the edit window which shows the block context of the code which has scrolled above the top of the window. See -Code Context in the Editing and Navigation section below.
-
Line Numbers (Editor Window only)
Open a column to the left of the edit window which shows the linenumber -of each line of text. The default is off unless configured on -(see Setting preferences).
+Code Context in the Editing and Navigation section +below. +
Show/Hide Line Numbers (Editor Window only)
Open a column to the left of the edit window which shows the number +of each line of text. The default is off, which may be changed in the +preferences (see Setting preferences).
Zoom/Restore Height
Toggles the window between normal size and maximum height. The initial size defaults to 40 lines by 80 chars unless changed on the General tab of the Configure IDLE dialog. The maximum height for a screen is determined by momentarily maximizing a window the first time one is zoomed on the screen. -Changing screen settings may invalidate the saved height. This toogle has +Changing screen settings may invalidate the saved height. This toggle has no effect when a window is maximized.

@@ -900,7 +903,7 @@

Navigation



- Last updated on Jul 23, 2019. + Last updated on Aug 04, 2019. Found a bug?
diff --git a/Lib/idlelib/mainmenu.py b/Lib/idlelib/mainmenu.py index fc51fb1b5d34..74edce234838 100644 --- a/Lib/idlelib/mainmenu.py +++ b/Lib/idlelib/mainmenu.py @@ -73,10 +73,10 @@ ]), ('run', [ - ('Python Shell', '<>'), - ('C_heck Module', '<>'), ('R_un Module', '<>'), ('Run... _Customized', '<>'), + ('C_heck Module', '<>'), + ('Python Shell', '<>'), ]), ('shell', [ diff --git a/Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst b/Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst new file mode 100644 index 000000000000..fc1d6b6bb357 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst @@ -0,0 +1 @@ +Reorder the Run menu. Put the most common choice, Run Module, at the top. From webhook-mailer at python.org Sun Aug 4 17:07:29 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 04 Aug 2019 21:07:29 -0000 Subject: [Python-checkins] bpo-37748: Re-order the Run menu. (GH-15115) Message-ID: https://github.com/python/cpython/commit/a96f0367d4c84ed42f8dc80c88c10f334498f36c commit: a96f0367d4c84ed42f8dc80c88c10f334498f36c branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-04T14:07:26-07:00 summary: bpo-37748: Re-order the Run menu. (GH-15115) Put the most common choice, Run Module, at the top. (cherry picked from commit 14070299cdc0faf36975f0cc2d51824a9abf3db0) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst M Doc/library/idle.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/help.html M Lib/idlelib/mainmenu.py diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 5975e3bb5806..0bd248c22b18 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -207,20 +207,6 @@ Strip trailing whitespace Run menu (Editor window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. _python-shell: - -Python Shell - Open or wake up the Python Shell window. - -.. _check-module: - -Check Module - Check the syntax of the module currently open in the Editor window. If the - module has not been saved IDLE will either prompt the user to save or - autosave, as selected in the General tab of the Idle Settings dialog. If - there is a syntax error, the approximate location is indicated in the - Editor window. - .. _run-module: Run Module @@ -239,6 +225,20 @@ Run... Customized settings. *Command Line Arguments* extend :data:`sys.argv` as if passed on a command line. The module can be run in the Shell without restarting. +.. _check-module: + +Check Module + Check the syntax of the module currently open in the Editor window. If the + module has not been saved IDLE will either prompt the user to save or + autosave, as selected in the General tab of the Idle Settings dialog. If + there is a syntax error, the approximate location is indicated in the + Editor window. + +.. _python-shell: + +Python Shell + Open or wake up the Python Shell window. + Shell menu (Shell window only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -295,8 +295,8 @@ Configure IDLE menu. For more details, see :ref:`Setting preferences ` under Help and preferences. - Most configuration options apply to all windows or all future windows. - The option items below only apply to the active window. +Most configuration options apply to all windows or all future windows. +The option items below only apply to the active window. Show/Hide Code Context (Editor Window only) Open a pane at the top of the edit window which shows the block context diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index cf545dcc6745..d4a00027f858 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,9 @@ Released on 2019-09-30? ====================================== +bpo-37748: Reorder the Run menu. Put the most common choice, +Run Module, at the top. + bpo-37692: Improve highlight config sample with example shell interaction and better labels for shell elements. diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html index 49068df7a8b9..91e66a4b9117 100644 --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -210,16 +210,6 @@

Edit menu (Shell and Editor)

Run menu (Editor window only)?

-
-
Python Shell
Open or wake up the Python Shell window.
-
-
-
Check Module
Check the syntax of the module currently open in the Editor window. If the -module has not been saved IDLE will either prompt the user to save or -autosave, as selected in the General tab of the Idle Settings dialog. If -there is a syntax error, the approximate location is indicated in the -Editor window.
-
Run Module
Do Check Module. If no error, restart the shell to clean the environment, then execute the module. Output is displayed in the Shell @@ -234,6 +224,16 @@

Edit menu (Shell and Editor)sys.argv as if passed on a command line. The module can be run in the Shell without restarting.

+
+
Check Module
Check the syntax of the module currently open in the Editor window. If the +module has not been saved IDLE will either prompt the user to save or +autosave, as selected in the General tab of the Idle Settings dialog. If +there is a syntax error, the approximate location is indicated in the +Editor window.
+
+
+
Python Shell
Open or wake up the Python Shell window.
+

Shell menu (Shell window only)?

@@ -268,23 +268,26 @@

Options menu (Shell and Editor)
Configure IDLE
Open a configuration dialog and change preferences for the following: fonts, indentation, keybindings, text color themes, startup windows and -size, additional help sources, and extensions. On macOS, open the +size, additional help sources, and extensions. On macOS, open the configuration dialog by selecting Preferences in the application -menu. For more, see -Setting preferences under Help and preferences. -Most configuration options apply to all windows or all future windows. -The option items below only apply to the active window.
+menu. For more details, see +Setting preferences under Help and preferences. + +

Most configuration options apply to all windows or all future windows. +The option items below only apply to the active window.

+
Show/Hide Code Context (Editor Window only)
Open a pane at the top of the edit window which shows the block context of the code which has scrolled above the top of the window. See -Code Context in the Editing and Navigation section below.
-
Line Numbers (Editor Window only)
Open a column to the left of the edit window which shows the linenumber -of each line of text. The default is off unless configured on -(see Setting preferences).
+Code Context in the Editing and Navigation section +below. +
Show/Hide Line Numbers (Editor Window only)
Open a column to the left of the edit window which shows the number +of each line of text. The default is off, which may be changed in the +preferences (see Setting preferences).
Zoom/Restore Height
Toggles the window between normal size and maximum height. The initial size defaults to 40 lines by 80 chars unless changed on the General tab of the Configure IDLE dialog. The maximum height for a screen is determined by momentarily maximizing a window the first time one is zoomed on the screen. -Changing screen settings may invalidate the saved height. This toogle has +Changing screen settings may invalidate the saved height. This toggle has no effect when a window is maximized.

@@ -900,7 +903,7 @@

Navigation



- Last updated on Jul 23, 2019. + Last updated on Aug 04, 2019. Found a bug?
diff --git a/Lib/idlelib/mainmenu.py b/Lib/idlelib/mainmenu.py index fc51fb1b5d34..74edce234838 100644 --- a/Lib/idlelib/mainmenu.py +++ b/Lib/idlelib/mainmenu.py @@ -73,10 +73,10 @@ ]), ('run', [ - ('Python Shell', '<>'), - ('C_heck Module', '<>'), ('R_un Module', '<>'), ('Run... _Customized', '<>'), + ('C_heck Module', '<>'), + ('Python Shell', '<>'), ]), ('shell', [ diff --git a/Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst b/Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst new file mode 100644 index 000000000000..fc1d6b6bb357 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2019-08-04-15-27-50.bpo-37748.0vf6pg.rst @@ -0,0 +1 @@ +Reorder the Run menu. Put the most common choice, Run Module, at the top. From webhook-mailer at python.org Sun Aug 4 17:23:40 2019 From: webhook-mailer at python.org (Steve Dower) Date: Sun, 04 Aug 2019 21:23:40 -0000 Subject: [Python-checkins] Adds Tim Hopper and Dan Lidral-Porter to ACKS (GH-15101) Message-ID: https://github.com/python/cpython/commit/d748a8085568cc57613ab2b437f7b6c1b6d76ae3 commit: d748a8085568cc57613ab2b437f7b6c1b6d76ae3 branch: master author: Timothy Hopper committer: Steve Dower date: 2019-08-04T14:23:29-07:00 summary: Adds Tim Hopper and Dan Lidral-Porter to ACKS (GH-15101) Tim and Dan were authors for GH-11847 files: M Misc/ACKS diff --git a/Misc/ACKS b/Misc/ACKS index 82fc51c98c40..8c834b43120b 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1879,3 +1879,5 @@ Geoff Shannon Batuhan Taskaya Aleksandr Balezin Robert Leenders +Tim Hopper +Dan Lidral-Porter From webhook-mailer at python.org Sun Aug 4 19:49:02 2019 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Sun, 04 Aug 2019 23:49:02 -0000 Subject: [Python-checkins] bpo-36419: IDLE - Refactor autocompete and improve testing. (#15121) Message-ID: https://github.com/python/cpython/commit/1213123005d9f94bb5027c0a5256ea4d3e97b61d commit: 1213123005d9f94bb5027c0a5256ea4d3e97b61d branch: master author: Terry Jan Reedy committer: GitHub date: 2019-08-04T19:48:52-04:00 summary: bpo-36419: IDLE - Refactor autocompete and improve testing. (#15121) files: A Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst M Lib/idlelib/autocomplete.py M Lib/idlelib/autocomplete_w.py M Lib/idlelib/idle_test/test_autocomplete.py diff --git a/Lib/idlelib/autocomplete.py b/Lib/idlelib/autocomplete.py index e20b757d8710..c623d45a1534 100644 --- a/Lib/idlelib/autocomplete.py +++ b/Lib/idlelib/autocomplete.py @@ -8,42 +8,45 @@ import string import sys -# These constants represent the two different types of completions. -# They must be defined here so autocomple_w can import them. -COMPLETE_ATTRIBUTES, COMPLETE_FILES = range(1, 2+1) - +# Two types of completions; defined here for autocomplete_w import below. +ATTRS, FILES = 0, 1 from idlelib import autocomplete_w from idlelib.config import idleConf from idlelib.hyperparser import HyperParser +# Tuples passed to open_completions. +# EvalFunc, Complete, WantWin, Mode +FORCE = True, False, True, None # Control-Space. +TAB = False, True, True, None # Tab. +TRY_A = False, False, False, ATTRS # '.' for attributes. +TRY_F = False, False, False, FILES # '/' in quotes for file name. + # This string includes all chars that may be in an identifier. # TODO Update this here and elsewhere. ID_CHARS = string.ascii_letters + string.digits + "_" -SEPS = os.sep -if os.altsep: # e.g. '/' on Windows... - SEPS += os.altsep - +SEPS = f"{os.sep}{os.altsep if os.altsep else ''}" +TRIGGERS = f".{SEPS}" class AutoComplete: def __init__(self, editwin=None): self.editwin = editwin - if editwin is not None: # not in subprocess or test + if editwin is not None: # not in subprocess or no-gui test self.text = editwin.text - self.autocompletewindow = None - # id of delayed call, and the index of the text insert when - # the delayed call was issued. If _delayed_completion_id is - # None, there is no delayed call. - self._delayed_completion_id = None - self._delayed_completion_index = None + self.autocompletewindow = None + # id of delayed call, and the index of the text insert when + # the delayed call was issued. If _delayed_completion_id is + # None, there is no delayed call. + self._delayed_completion_id = None + self._delayed_completion_index = None @classmethod def reload(cls): cls.popupwait = idleConf.GetOption( "extensions", "AutoComplete", "popupwait", type="int", default=0) - def _make_autocomplete_window(self): + def _make_autocomplete_window(self): # Makes mocking easier. return autocomplete_w.AutoCompleteWindow(self.text) def _remove_autocomplete_window(self, event=None): @@ -52,30 +55,12 @@ def _remove_autocomplete_window(self, event=None): self.autocompletewindow = None def force_open_completions_event(self, event): - """Happens when the user really wants to open a completion list, even - if a function call is needed. - """ - self.open_completions(True, False, True) + "(^space) Open completion list, even if a function call is needed." + self.open_completions(FORCE) return "break" - def try_open_completions_event(self, event): - """Happens when it would be nice to open a completion list, but not - really necessary, for example after a dot, so function - calls won't be made. - """ - lastchar = self.text.get("insert-1c") - if lastchar == ".": - self._open_completions_later(False, False, False, - COMPLETE_ATTRIBUTES) - elif lastchar in SEPS: - self._open_completions_later(False, False, False, - COMPLETE_FILES) - def autocomplete_event(self, event): - """Happens when the user wants to complete his word, and if necessary, - open a completion list after that (if there is more than one - completion) - """ + "(tab) Complete word or open list if multiple options." if hasattr(event, "mc_state") and event.mc_state or\ not self.text.get("insert linestart", "insert").strip(): # A modifier was pressed along with the tab or @@ -85,34 +70,34 @@ def autocomplete_event(self, event): self.autocompletewindow.complete() return "break" else: - opened = self.open_completions(False, True, True) + opened = self.open_completions(TAB) return "break" if opened else None - def _open_completions_later(self, *args): - self._delayed_completion_index = self.text.index("insert") - if self._delayed_completion_id is not None: - self.text.after_cancel(self._delayed_completion_id) - self._delayed_completion_id = \ - self.text.after(self.popupwait, self._delayed_open_completions, - *args) - - def _delayed_open_completions(self, *args): + def try_open_completions_event(self, event=None): + "(./) Open completion list after pause with no movement." + lastchar = self.text.get("insert-1c") + if lastchar in TRIGGERS: + args = TRY_A if lastchar == "." else TRY_F + self._delayed_completion_index = self.text.index("insert") + if self._delayed_completion_id is not None: + self.text.after_cancel(self._delayed_completion_id) + self._delayed_completion_id = self.text.after( + self.popupwait, self._delayed_open_completions, args) + + def _delayed_open_completions(self, args): + "Call open_completions if index unchanged." self._delayed_completion_id = None if self.text.index("insert") == self._delayed_completion_index: - self.open_completions(*args) + self.open_completions(args) - def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): + def open_completions(self, args): """Find the completions and create the AutoCompleteWindow. Return True if successful (no syntax error or so found). If complete is True, then if there's nothing to complete and no start of completion, won't open completions and return False. If mode is given, will open a completion list only in this mode. - - Action Function Eval Complete WantWin Mode - ^space force_open_completions True, False, True no - . or / try_open_completions False, False, False yes - tab autocomplete False, True, True no """ + evalfuncs, complete, wantwin, mode = args # Cancel another delayed call, if it exists. if self._delayed_completion_id is not None: self.text.after_cancel(self._delayed_completion_id) @@ -121,14 +106,14 @@ def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): hp = HyperParser(self.editwin, "insert") curline = self.text.get("insert linestart", "insert") i = j = len(curline) - if hp.is_in_string() and (not mode or mode==COMPLETE_FILES): + if hp.is_in_string() and (not mode or mode==FILES): # Find the beginning of the string. # fetch_completions will look at the file system to determine # whether the string value constitutes an actual file name # XXX could consider raw strings here and unescape the string # value if it's not raw. self._remove_autocomplete_window() - mode = COMPLETE_FILES + mode = FILES # Find last separator or string start while i and curline[i-1] not in "'\"" + SEPS: i -= 1 @@ -138,17 +123,17 @@ def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): while i and curline[i-1] not in "'\"": i -= 1 comp_what = curline[i:j] - elif hp.is_in_code() and (not mode or mode==COMPLETE_ATTRIBUTES): + elif hp.is_in_code() and (not mode or mode==ATTRS): self._remove_autocomplete_window() - mode = COMPLETE_ATTRIBUTES + mode = ATTRS while i and (curline[i-1] in ID_CHARS or ord(curline[i-1]) > 127): i -= 1 comp_start = curline[i:j] - if i and curline[i-1] == '.': + if i and curline[i-1] == '.': # Need object with attributes. hp.set_index("insert-%dc" % (len(curline)-(i-1))) comp_what = hp.get_expression() - if not comp_what or \ - (not evalfuncs and comp_what.find('(') != -1): + if (not comp_what or + (not evalfuncs and comp_what.find('(') != -1)): return None else: comp_what = "" @@ -163,7 +148,7 @@ def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): self.autocompletewindow = self._make_autocomplete_window() return not self.autocompletewindow.show_window( comp_lists, "insert-%dc" % len(comp_start), - complete, mode, userWantsWin) + complete, mode, wantwin) def fetch_completions(self, what, mode): """Return a pair of lists of completions for something. The first list @@ -185,7 +170,7 @@ def fetch_completions(self, what, mode): return rpcclt.remotecall("exec", "get_the_completion_list", (what, mode), {}) else: - if mode == COMPLETE_ATTRIBUTES: + if mode == ATTRS: if what == "": namespace = {**__main__.__builtins__.__dict__, **__main__.__dict__} @@ -207,7 +192,7 @@ def fetch_completions(self, what, mode): except: return [], [] - elif mode == COMPLETE_FILES: + elif mode == FILES: if what == "": what = "." try: diff --git a/Lib/idlelib/autocomplete_w.py b/Lib/idlelib/autocomplete_w.py index c24962527736..c69ab4a36836 100644 --- a/Lib/idlelib/autocomplete_w.py +++ b/Lib/idlelib/autocomplete_w.py @@ -6,7 +6,7 @@ from tkinter import * from tkinter.ttk import Frame, Scrollbar -from idlelib.autocomplete import COMPLETE_FILES, COMPLETE_ATTRIBUTES +from idlelib.autocomplete import FILES, ATTRS from idlelib.multicall import MC_SHIFT HIDE_VIRTUAL_EVENT_NAME = "<>" @@ -39,8 +39,7 @@ def __init__(self, widget): self.completions = None # A list with more completions, or None self.morecompletions = None - # The completion mode. Either autocomplete.COMPLETE_ATTRIBUTES or - # autocomplete.COMPLETE_FILES + # The completion mode, either autocomplete.ATTRS or .FILES. self.mode = None # The current completion start, on the text box (a string) self.start = None @@ -73,8 +72,8 @@ def _change_start(self, newstart): def _binary_search(self, s): """Find the first index in self.completions where completions[i] is - greater or equal to s, or the last index if there is no such - one.""" + greater or equal to s, or the last index if there is no such. + """ i = 0; j = len(self.completions) while j > i: m = (i + j) // 2 @@ -87,7 +86,8 @@ def _binary_search(self, s): def _complete_string(self, s): """Assuming that s is the prefix of a string in self.completions, return the longest string which is a prefix of all the strings which - s is a prefix of them. If s is not a prefix of a string, return s.""" + s is a prefix of them. If s is not a prefix of a string, return s. + """ first = self._binary_search(s) if self.completions[first][:len(s)] != s: # There is not even one completion which s is a prefix of. @@ -116,8 +116,10 @@ def _complete_string(self, s): return first_comp[:i] def _selection_changed(self): - """Should be called when the selection of the Listbox has changed. - Updates the Listbox display and calls _change_start.""" + """Call when the selection of the Listbox has changed. + + Updates the Listbox display and calls _change_start. + """ cursel = int(self.listbox.curselection()[0]) self.listbox.see(cursel) @@ -153,8 +155,10 @@ def _selection_changed(self): def show_window(self, comp_lists, index, complete, mode, userWantsWin): """Show the autocomplete list, bind events. - If complete is True, complete the text, and if there is exactly one - matching completion, don't open a list.""" + + If complete is True, complete the text, and if there is exactly + one matching completion, don't open a list. + """ # Handle the start we already have self.completions, self.morecompletions = comp_lists self.mode = mode @@ -300,7 +304,7 @@ def keypress_event(self, event): if keysym != "Tab": self.lastkey_was_tab = False if (len(keysym) == 1 or keysym in ("underscore", "BackSpace") - or (self.mode == COMPLETE_FILES and keysym in + or (self.mode == FILES and keysym in ("period", "minus"))) \ and not (state & ~MC_SHIFT): # Normal editing of text @@ -329,10 +333,10 @@ def keypress_event(self, event): self.hide_window() return 'break' - elif (self.mode == COMPLETE_ATTRIBUTES and keysym in + elif (self.mode == ATTRS and keysym in ("period", "space", "parenleft", "parenright", "bracketleft", "bracketright")) or \ - (self.mode == COMPLETE_FILES and keysym in + (self.mode == FILES and keysym in ("slash", "backslash", "quotedbl", "apostrophe")) \ and not (state & ~MC_SHIFT): # If start is a prefix of the selection, but is not '' when @@ -340,7 +344,7 @@ def keypress_event(self, event): # selected completion. Anyway, close the list. cursel = int(self.listbox.curselection()[0]) if self.completions[cursel][:len(self.start)] == self.start \ - and (self.mode == COMPLETE_ATTRIBUTES or self.start): + and (self.mode == ATTRS or self.start): self._change_start(self.completions[cursel]) self.hide_window() return None diff --git a/Lib/idlelib/idle_test/test_autocomplete.py b/Lib/idlelib/idle_test/test_autocomplete.py index 6181b29ec250..2c478cd5c2a1 100644 --- a/Lib/idlelib/idle_test/test_autocomplete.py +++ b/Lib/idlelib/idle_test/test_autocomplete.py @@ -1,4 +1,4 @@ -"Test autocomplete, coverage 87%." +"Test autocomplete, coverage 93%." import unittest from unittest.mock import Mock, patch @@ -45,127 +45,177 @@ def setUp(self): def test_init(self): self.assertEqual(self.autocomplete.editwin, self.editor) + self.assertEqual(self.autocomplete.text, self.text) def test_make_autocomplete_window(self): testwin = self.autocomplete._make_autocomplete_window() self.assertIsInstance(testwin, acw.AutoCompleteWindow) def test_remove_autocomplete_window(self): - self.autocomplete.autocompletewindow = ( - self.autocomplete._make_autocomplete_window()) - self.autocomplete._remove_autocomplete_window() - self.assertIsNone(self.autocomplete.autocompletewindow) + acp = self.autocomplete + acp.autocompletewindow = m = Mock() + acp._remove_autocomplete_window() + m.hide_window.assert_called_once() + self.assertIsNone(acp.autocompletewindow) def test_force_open_completions_event(self): - # Test that force_open_completions_event calls _open_completions. - o_cs = Func() - self.autocomplete.open_completions = o_cs - self.autocomplete.force_open_completions_event('event') - self.assertEqual(o_cs.args, (True, False, True)) - - def test_try_open_completions_event(self): - Equal = self.assertEqual - autocomplete = self.autocomplete - trycompletions = self.autocomplete.try_open_completions_event - o_c_l = Func() - autocomplete._open_completions_later = o_c_l - - # _open_completions_later should not be called with no text in editor. - trycompletions('event') - Equal(o_c_l.args, None) - - # _open_completions_later should be called with COMPLETE_ATTRIBUTES (1). - self.text.insert('1.0', 're.') - trycompletions('event') - Equal(o_c_l.args, (False, False, False, 1)) - - # _open_completions_later should be called with COMPLETE_FILES (2). - self.text.delete('1.0', 'end') - self.text.insert('1.0', '"./Lib/') - trycompletions('event') - Equal(o_c_l.args, (False, False, False, 2)) + # Call _open_completions and break. + acp = self.autocomplete + open_c = Func() + acp.open_completions = open_c + self.assertEqual(acp.force_open_completions_event('event'), 'break') + self.assertEqual(open_c.args[0], ac.FORCE) def test_autocomplete_event(self): Equal = self.assertEqual - autocomplete = self.autocomplete + acp = self.autocomplete - # Test that the autocomplete event is ignored if user is pressing a - # modifier key in addition to the tab key. + # Result of autocomplete event: If modified tab, None. ev = Event(mc_state=True) - self.assertIsNone(autocomplete.autocomplete_event(ev)) + self.assertIsNone(acp.autocomplete_event(ev)) del ev.mc_state - # Test that tab after whitespace is ignored. + # If tab after whitespace, None. self.text.insert('1.0', ' """Docstring.\n ') - self.assertIsNone(autocomplete.autocomplete_event(ev)) + self.assertIsNone(acp.autocomplete_event(ev)) self.text.delete('1.0', 'end') - # If autocomplete window is open, complete() method is called. + # If active autocomplete window, complete() and 'break'. self.text.insert('1.0', 're.') - # This must call autocomplete._make_autocomplete_window(). - Equal(self.autocomplete.autocomplete_event(ev), 'break') - - # If autocomplete window is not active or does not exist, - # open_completions is called. Return depends on its return. - autocomplete._remove_autocomplete_window() - o_cs = Func() # .result = None. - autocomplete.open_completions = o_cs - Equal(self.autocomplete.autocomplete_event(ev), None) - Equal(o_cs.args, (False, True, True)) - o_cs.result = True - Equal(self.autocomplete.autocomplete_event(ev), 'break') - Equal(o_cs.args, (False, True, True)) - - def test_open_completions_later(self): - # Test that autocomplete._delayed_completion_id is set. + acp.autocompletewindow = mock = Mock() + mock.is_active = Mock(return_value=True) + Equal(acp.autocomplete_event(ev), 'break') + mock.complete.assert_called_once() + acp.autocompletewindow = None + + # If no active autocomplete window, open_completions(), None/break. + open_c = Func(result=False) + acp.open_completions = open_c + Equal(acp.autocomplete_event(ev), None) + Equal(open_c.args[0], ac.TAB) + open_c.result = True + Equal(acp.autocomplete_event(ev), 'break') + Equal(open_c.args[0], ac.TAB) + + def test_try_open_completions_event(self): + Equal = self.assertEqual + text = self.text acp = self.autocomplete + trycompletions = acp.try_open_completions_event + after = Func(result='after1') + acp.text.after = after + + # If no text or trigger, after not called. + trycompletions() + Equal(after.called, 0) + text.insert('1.0', 're') + trycompletions() + Equal(after.called, 0) + + # Attribute needed, no existing callback. + text.insert('insert', ' re.') acp._delayed_completion_id = None - acp._open_completions_later(False, False, False, ac.COMPLETE_ATTRIBUTES) + trycompletions() + Equal(acp._delayed_completion_index, text.index('insert')) + Equal(after.args, + (acp.popupwait, acp._delayed_open_completions, ac.TRY_A)) cb1 = acp._delayed_completion_id - self.assertTrue(cb1.startswith('after')) - - # Test that cb1 is cancelled and cb2 is new. - acp._open_completions_later(False, False, False, ac.COMPLETE_FILES) - self.assertNotIn(cb1, self.root.tk.call('after', 'info')) - cb2 = acp._delayed_completion_id - self.assertTrue(cb2.startswith('after') and cb2 != cb1) - self.text.after_cancel(cb2) + Equal(cb1, 'after1') + + # File needed, existing callback cancelled. + text.insert('insert', ' "./Lib/') + after.result = 'after2' + cancel = Func() + acp.text.after_cancel = cancel + trycompletions() + Equal(acp._delayed_completion_index, text.index('insert')) + Equal(cancel.args, (cb1,)) + Equal(after.args, + (acp.popupwait, acp._delayed_open_completions, ac.TRY_F)) + Equal(acp._delayed_completion_id, 'after2') def test_delayed_open_completions(self): - # Test that autocomplete._delayed_completion_id set to None - # and that open_completions is not called if the index is not - # equal to _delayed_completion_index. + Equal = self.assertEqual acp = self.autocomplete - acp.open_completions = Func() + open_c = Func() + acp.open_completions = open_c + self.text.insert('1.0', '"dict.') + + # Set autocomplete._delayed_completion_id to None. + # Text index changed, don't call open_completions. acp._delayed_completion_id = 'after' acp._delayed_completion_index = self.text.index('insert+1c') - acp._delayed_open_completions(1, 2, 3) + acp._delayed_open_completions('dummy') self.assertIsNone(acp._delayed_completion_id) - self.assertEqual(acp.open_completions.called, 0) + Equal(open_c.called, 0) - # Test that open_completions is called if indexes match. + # Text index unchanged, call open_completions. acp._delayed_completion_index = self.text.index('insert') - acp._delayed_open_completions(1, 2, 3, ac.COMPLETE_FILES) - self.assertEqual(acp.open_completions.args, (1, 2, 3, 2)) + acp._delayed_open_completions((1, 2, 3, ac.FILES)) + self.assertEqual(open_c.args[0], (1, 2, 3, ac.FILES)) + + def test_oc_cancel_comment(self): + none = self.assertIsNone + acp = self.autocomplete + + # Comment is in neither code or string. + acp._delayed_completion_id = 'after' + after = Func(result='after') + acp.text.after_cancel = after + self.text.insert(1.0, '# comment') + none(acp.open_completions(ac.TAB)) # From 'else' after 'elif'. + none(acp._delayed_completion_id) + + def test_oc_no_list(self): + acp = self.autocomplete + fetch = Func(result=([],[])) + acp.fetch_completions = fetch + self.text.insert('1.0', 'object') + self.assertIsNone(acp.open_completions(ac.TAB)) + self.text.insert('insert', '.') + self.assertIsNone(acp.open_completions(ac.TAB)) + self.assertEqual(fetch.called, 2) + + + def test_open_completions_none(self): + # Test other two None returns. + none = self.assertIsNone + acp = self.autocomplete + + # No object for attributes or need call not allowed. + self.text.insert(1.0, '.') + none(acp.open_completions(ac.TAB)) + self.text.insert('insert', ' int().') + none(acp.open_completions(ac.TAB)) + + # Blank or quote trigger 'if complete ...'. + self.text.delete(1.0, 'end') + self.assertFalse(acp.open_completions(ac.TAB)) + self.text.insert('1.0', '"') + self.assertFalse(acp.open_completions(ac.TAB)) + self.text.delete('1.0', 'end') + + class dummy_acw(): + __init__ = Func() + show_window = Func(result=False) + hide_window = Func() def test_open_completions(self): - # Test completions of files and attributes as well as non-completion - # of errors. - self.text.insert('1.0', 'pr') - self.assertTrue(self.autocomplete.open_completions(False, True, True)) + # Test completions of files and attributes. + acp = self.autocomplete + fetch = Func(result=(['tem'],['tem', '_tem'])) + acp.fetch_completions = fetch + def make_acw(): return self.dummy_acw() + acp._make_autocomplete_window = make_acw + + self.text.insert('1.0', 'int.') + acp.open_completions(ac.TAB) + self.assertIsInstance(acp.autocompletewindow, self.dummy_acw) self.text.delete('1.0', 'end') # Test files. self.text.insert('1.0', '"t') - #self.assertTrue(self.autocomplete.open_completions(False, True, True)) - self.text.delete('1.0', 'end') - - # Test with blank will fail. - self.assertFalse(self.autocomplete.open_completions(False, True, True)) - - # Test with only string quote will fail. - self.text.insert('1.0', '"') - self.assertFalse(self.autocomplete.open_completions(False, True, True)) + self.assertTrue(acp.open_completions(ac.TAB)) self.text.delete('1.0', 'end') def test_fetch_completions(self): @@ -174,21 +224,21 @@ def test_fetch_completions(self): # a small list containing non-private variables. # For file completion, a large list containing all files in the path, # and a small list containing files that do not start with '.'. - autocomplete = self.autocomplete - small, large = self.autocomplete.fetch_completions( - '', ac.COMPLETE_ATTRIBUTES) + acp = self.autocomplete + small, large = acp.fetch_completions( + '', ac.ATTRS) if __main__.__file__ != ac.__file__: self.assertNotIn('AutoComplete', small) # See issue 36405. # Test attributes - s, b = autocomplete.fetch_completions('', ac.COMPLETE_ATTRIBUTES) + s, b = acp.fetch_completions('', ac.ATTRS) self.assertLess(len(small), len(large)) self.assertTrue(all(filter(lambda x: x.startswith('_'), s))) self.assertTrue(any(filter(lambda x: x.startswith('_'), b))) # Test smalll should respect to __all__. with patch.dict('__main__.__dict__', {'__all__': ['a', 'b']}): - s, b = autocomplete.fetch_completions('', ac.COMPLETE_ATTRIBUTES) + s, b = acp.fetch_completions('', ac.ATTRS) self.assertEqual(s, ['a', 'b']) self.assertIn('__name__', b) # From __main__.__dict__ self.assertIn('sum', b) # From __main__.__builtins__.__dict__ @@ -197,7 +247,7 @@ def test_fetch_completions(self): mock = Mock() mock._private = Mock() with patch.dict('__main__.__dict__', {'foo': mock}): - s, b = autocomplete.fetch_completions('foo', ac.COMPLETE_ATTRIBUTES) + s, b = acp.fetch_completions('foo', ac.ATTRS) self.assertNotIn('_private', s) self.assertIn('_private', b) self.assertEqual(s, [i for i in sorted(dir(mock)) if i[:1] != '_']) @@ -211,36 +261,36 @@ def _listdir(path): return ['monty', 'python', '.hidden'] with patch.object(os, 'listdir', _listdir): - s, b = autocomplete.fetch_completions('', ac.COMPLETE_FILES) + s, b = acp.fetch_completions('', ac.FILES) self.assertEqual(s, ['bar', 'foo']) self.assertEqual(b, ['.hidden', 'bar', 'foo']) - s, b = autocomplete.fetch_completions('~', ac.COMPLETE_FILES) + s, b = acp.fetch_completions('~', ac.FILES) self.assertEqual(s, ['monty', 'python']) self.assertEqual(b, ['.hidden', 'monty', 'python']) def test_get_entity(self): # Test that a name is in the namespace of sys.modules and # __main__.__dict__. - autocomplete = self.autocomplete + acp = self.autocomplete Equal = self.assertEqual - Equal(self.autocomplete.get_entity('int'), int) + Equal(acp.get_entity('int'), int) # Test name from sys.modules. mock = Mock() with patch.dict('sys.modules', {'tempfile': mock}): - Equal(autocomplete.get_entity('tempfile'), mock) + Equal(acp.get_entity('tempfile'), mock) # Test name from __main__.__dict__. di = {'foo': 10, 'bar': 20} with patch.dict('__main__.__dict__', {'d': di}): - Equal(autocomplete.get_entity('d'), di) + Equal(acp.get_entity('d'), di) # Test name not in namespace. with patch.dict('__main__.__dict__', {}): with self.assertRaises(NameError): - autocomplete.get_entity('not_exist') + acp.get_entity('not_exist') if __name__ == '__main__': diff --git a/Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst b/Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst new file mode 100644 index 000000000000..a44fd3b59b4f --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst @@ -0,0 +1 @@ +IDLE - Refactor autocompete and improve testing. From webhook-mailer at python.org Sun Aug 4 20:08:20 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 05 Aug 2019 00:08:20 -0000 Subject: [Python-checkins] bpo-36419: IDLE - Refactor autocompete and improve testing. (GH-15121) Message-ID: https://github.com/python/cpython/commit/5349f8cd784220fc6599830c56d3f0614de2b8cb commit: 5349f8cd784220fc6599830c56d3f0614de2b8cb branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-04T17:08:16-07:00 summary: bpo-36419: IDLE - Refactor autocompete and improve testing. (GH-15121) (cherry picked from commit 1213123005d9f94bb5027c0a5256ea4d3e97b61d) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst M Lib/idlelib/autocomplete.py M Lib/idlelib/autocomplete_w.py M Lib/idlelib/idle_test/test_autocomplete.py diff --git a/Lib/idlelib/autocomplete.py b/Lib/idlelib/autocomplete.py index e20b757d8710..c623d45a1534 100644 --- a/Lib/idlelib/autocomplete.py +++ b/Lib/idlelib/autocomplete.py @@ -8,42 +8,45 @@ import string import sys -# These constants represent the two different types of completions. -# They must be defined here so autocomple_w can import them. -COMPLETE_ATTRIBUTES, COMPLETE_FILES = range(1, 2+1) - +# Two types of completions; defined here for autocomplete_w import below. +ATTRS, FILES = 0, 1 from idlelib import autocomplete_w from idlelib.config import idleConf from idlelib.hyperparser import HyperParser +# Tuples passed to open_completions. +# EvalFunc, Complete, WantWin, Mode +FORCE = True, False, True, None # Control-Space. +TAB = False, True, True, None # Tab. +TRY_A = False, False, False, ATTRS # '.' for attributes. +TRY_F = False, False, False, FILES # '/' in quotes for file name. + # This string includes all chars that may be in an identifier. # TODO Update this here and elsewhere. ID_CHARS = string.ascii_letters + string.digits + "_" -SEPS = os.sep -if os.altsep: # e.g. '/' on Windows... - SEPS += os.altsep - +SEPS = f"{os.sep}{os.altsep if os.altsep else ''}" +TRIGGERS = f".{SEPS}" class AutoComplete: def __init__(self, editwin=None): self.editwin = editwin - if editwin is not None: # not in subprocess or test + if editwin is not None: # not in subprocess or no-gui test self.text = editwin.text - self.autocompletewindow = None - # id of delayed call, and the index of the text insert when - # the delayed call was issued. If _delayed_completion_id is - # None, there is no delayed call. - self._delayed_completion_id = None - self._delayed_completion_index = None + self.autocompletewindow = None + # id of delayed call, and the index of the text insert when + # the delayed call was issued. If _delayed_completion_id is + # None, there is no delayed call. + self._delayed_completion_id = None + self._delayed_completion_index = None @classmethod def reload(cls): cls.popupwait = idleConf.GetOption( "extensions", "AutoComplete", "popupwait", type="int", default=0) - def _make_autocomplete_window(self): + def _make_autocomplete_window(self): # Makes mocking easier. return autocomplete_w.AutoCompleteWindow(self.text) def _remove_autocomplete_window(self, event=None): @@ -52,30 +55,12 @@ def _remove_autocomplete_window(self, event=None): self.autocompletewindow = None def force_open_completions_event(self, event): - """Happens when the user really wants to open a completion list, even - if a function call is needed. - """ - self.open_completions(True, False, True) + "(^space) Open completion list, even if a function call is needed." + self.open_completions(FORCE) return "break" - def try_open_completions_event(self, event): - """Happens when it would be nice to open a completion list, but not - really necessary, for example after a dot, so function - calls won't be made. - """ - lastchar = self.text.get("insert-1c") - if lastchar == ".": - self._open_completions_later(False, False, False, - COMPLETE_ATTRIBUTES) - elif lastchar in SEPS: - self._open_completions_later(False, False, False, - COMPLETE_FILES) - def autocomplete_event(self, event): - """Happens when the user wants to complete his word, and if necessary, - open a completion list after that (if there is more than one - completion) - """ + "(tab) Complete word or open list if multiple options." if hasattr(event, "mc_state") and event.mc_state or\ not self.text.get("insert linestart", "insert").strip(): # A modifier was pressed along with the tab or @@ -85,34 +70,34 @@ def autocomplete_event(self, event): self.autocompletewindow.complete() return "break" else: - opened = self.open_completions(False, True, True) + opened = self.open_completions(TAB) return "break" if opened else None - def _open_completions_later(self, *args): - self._delayed_completion_index = self.text.index("insert") - if self._delayed_completion_id is not None: - self.text.after_cancel(self._delayed_completion_id) - self._delayed_completion_id = \ - self.text.after(self.popupwait, self._delayed_open_completions, - *args) - - def _delayed_open_completions(self, *args): + def try_open_completions_event(self, event=None): + "(./) Open completion list after pause with no movement." + lastchar = self.text.get("insert-1c") + if lastchar in TRIGGERS: + args = TRY_A if lastchar == "." else TRY_F + self._delayed_completion_index = self.text.index("insert") + if self._delayed_completion_id is not None: + self.text.after_cancel(self._delayed_completion_id) + self._delayed_completion_id = self.text.after( + self.popupwait, self._delayed_open_completions, args) + + def _delayed_open_completions(self, args): + "Call open_completions if index unchanged." self._delayed_completion_id = None if self.text.index("insert") == self._delayed_completion_index: - self.open_completions(*args) + self.open_completions(args) - def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): + def open_completions(self, args): """Find the completions and create the AutoCompleteWindow. Return True if successful (no syntax error or so found). If complete is True, then if there's nothing to complete and no start of completion, won't open completions and return False. If mode is given, will open a completion list only in this mode. - - Action Function Eval Complete WantWin Mode - ^space force_open_completions True, False, True no - . or / try_open_completions False, False, False yes - tab autocomplete False, True, True no """ + evalfuncs, complete, wantwin, mode = args # Cancel another delayed call, if it exists. if self._delayed_completion_id is not None: self.text.after_cancel(self._delayed_completion_id) @@ -121,14 +106,14 @@ def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): hp = HyperParser(self.editwin, "insert") curline = self.text.get("insert linestart", "insert") i = j = len(curline) - if hp.is_in_string() and (not mode or mode==COMPLETE_FILES): + if hp.is_in_string() and (not mode or mode==FILES): # Find the beginning of the string. # fetch_completions will look at the file system to determine # whether the string value constitutes an actual file name # XXX could consider raw strings here and unescape the string # value if it's not raw. self._remove_autocomplete_window() - mode = COMPLETE_FILES + mode = FILES # Find last separator or string start while i and curline[i-1] not in "'\"" + SEPS: i -= 1 @@ -138,17 +123,17 @@ def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): while i and curline[i-1] not in "'\"": i -= 1 comp_what = curline[i:j] - elif hp.is_in_code() and (not mode or mode==COMPLETE_ATTRIBUTES): + elif hp.is_in_code() and (not mode or mode==ATTRS): self._remove_autocomplete_window() - mode = COMPLETE_ATTRIBUTES + mode = ATTRS while i and (curline[i-1] in ID_CHARS or ord(curline[i-1]) > 127): i -= 1 comp_start = curline[i:j] - if i and curline[i-1] == '.': + if i and curline[i-1] == '.': # Need object with attributes. hp.set_index("insert-%dc" % (len(curline)-(i-1))) comp_what = hp.get_expression() - if not comp_what or \ - (not evalfuncs and comp_what.find('(') != -1): + if (not comp_what or + (not evalfuncs and comp_what.find('(') != -1)): return None else: comp_what = "" @@ -163,7 +148,7 @@ def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): self.autocompletewindow = self._make_autocomplete_window() return not self.autocompletewindow.show_window( comp_lists, "insert-%dc" % len(comp_start), - complete, mode, userWantsWin) + complete, mode, wantwin) def fetch_completions(self, what, mode): """Return a pair of lists of completions for something. The first list @@ -185,7 +170,7 @@ def fetch_completions(self, what, mode): return rpcclt.remotecall("exec", "get_the_completion_list", (what, mode), {}) else: - if mode == COMPLETE_ATTRIBUTES: + if mode == ATTRS: if what == "": namespace = {**__main__.__builtins__.__dict__, **__main__.__dict__} @@ -207,7 +192,7 @@ def fetch_completions(self, what, mode): except: return [], [] - elif mode == COMPLETE_FILES: + elif mode == FILES: if what == "": what = "." try: diff --git a/Lib/idlelib/autocomplete_w.py b/Lib/idlelib/autocomplete_w.py index c24962527736..c69ab4a36836 100644 --- a/Lib/idlelib/autocomplete_w.py +++ b/Lib/idlelib/autocomplete_w.py @@ -6,7 +6,7 @@ from tkinter import * from tkinter.ttk import Frame, Scrollbar -from idlelib.autocomplete import COMPLETE_FILES, COMPLETE_ATTRIBUTES +from idlelib.autocomplete import FILES, ATTRS from idlelib.multicall import MC_SHIFT HIDE_VIRTUAL_EVENT_NAME = "<>" @@ -39,8 +39,7 @@ def __init__(self, widget): self.completions = None # A list with more completions, or None self.morecompletions = None - # The completion mode. Either autocomplete.COMPLETE_ATTRIBUTES or - # autocomplete.COMPLETE_FILES + # The completion mode, either autocomplete.ATTRS or .FILES. self.mode = None # The current completion start, on the text box (a string) self.start = None @@ -73,8 +72,8 @@ def _change_start(self, newstart): def _binary_search(self, s): """Find the first index in self.completions where completions[i] is - greater or equal to s, or the last index if there is no such - one.""" + greater or equal to s, or the last index if there is no such. + """ i = 0; j = len(self.completions) while j > i: m = (i + j) // 2 @@ -87,7 +86,8 @@ def _binary_search(self, s): def _complete_string(self, s): """Assuming that s is the prefix of a string in self.completions, return the longest string which is a prefix of all the strings which - s is a prefix of them. If s is not a prefix of a string, return s.""" + s is a prefix of them. If s is not a prefix of a string, return s. + """ first = self._binary_search(s) if self.completions[first][:len(s)] != s: # There is not even one completion which s is a prefix of. @@ -116,8 +116,10 @@ def _complete_string(self, s): return first_comp[:i] def _selection_changed(self): - """Should be called when the selection of the Listbox has changed. - Updates the Listbox display and calls _change_start.""" + """Call when the selection of the Listbox has changed. + + Updates the Listbox display and calls _change_start. + """ cursel = int(self.listbox.curselection()[0]) self.listbox.see(cursel) @@ -153,8 +155,10 @@ def _selection_changed(self): def show_window(self, comp_lists, index, complete, mode, userWantsWin): """Show the autocomplete list, bind events. - If complete is True, complete the text, and if there is exactly one - matching completion, don't open a list.""" + + If complete is True, complete the text, and if there is exactly + one matching completion, don't open a list. + """ # Handle the start we already have self.completions, self.morecompletions = comp_lists self.mode = mode @@ -300,7 +304,7 @@ def keypress_event(self, event): if keysym != "Tab": self.lastkey_was_tab = False if (len(keysym) == 1 or keysym in ("underscore", "BackSpace") - or (self.mode == COMPLETE_FILES and keysym in + or (self.mode == FILES and keysym in ("period", "minus"))) \ and not (state & ~MC_SHIFT): # Normal editing of text @@ -329,10 +333,10 @@ def keypress_event(self, event): self.hide_window() return 'break' - elif (self.mode == COMPLETE_ATTRIBUTES and keysym in + elif (self.mode == ATTRS and keysym in ("period", "space", "parenleft", "parenright", "bracketleft", "bracketright")) or \ - (self.mode == COMPLETE_FILES and keysym in + (self.mode == FILES and keysym in ("slash", "backslash", "quotedbl", "apostrophe")) \ and not (state & ~MC_SHIFT): # If start is a prefix of the selection, but is not '' when @@ -340,7 +344,7 @@ def keypress_event(self, event): # selected completion. Anyway, close the list. cursel = int(self.listbox.curselection()[0]) if self.completions[cursel][:len(self.start)] == self.start \ - and (self.mode == COMPLETE_ATTRIBUTES or self.start): + and (self.mode == ATTRS or self.start): self._change_start(self.completions[cursel]) self.hide_window() return None diff --git a/Lib/idlelib/idle_test/test_autocomplete.py b/Lib/idlelib/idle_test/test_autocomplete.py index 6181b29ec250..2c478cd5c2a1 100644 --- a/Lib/idlelib/idle_test/test_autocomplete.py +++ b/Lib/idlelib/idle_test/test_autocomplete.py @@ -1,4 +1,4 @@ -"Test autocomplete, coverage 87%." +"Test autocomplete, coverage 93%." import unittest from unittest.mock import Mock, patch @@ -45,127 +45,177 @@ def setUp(self): def test_init(self): self.assertEqual(self.autocomplete.editwin, self.editor) + self.assertEqual(self.autocomplete.text, self.text) def test_make_autocomplete_window(self): testwin = self.autocomplete._make_autocomplete_window() self.assertIsInstance(testwin, acw.AutoCompleteWindow) def test_remove_autocomplete_window(self): - self.autocomplete.autocompletewindow = ( - self.autocomplete._make_autocomplete_window()) - self.autocomplete._remove_autocomplete_window() - self.assertIsNone(self.autocomplete.autocompletewindow) + acp = self.autocomplete + acp.autocompletewindow = m = Mock() + acp._remove_autocomplete_window() + m.hide_window.assert_called_once() + self.assertIsNone(acp.autocompletewindow) def test_force_open_completions_event(self): - # Test that force_open_completions_event calls _open_completions. - o_cs = Func() - self.autocomplete.open_completions = o_cs - self.autocomplete.force_open_completions_event('event') - self.assertEqual(o_cs.args, (True, False, True)) - - def test_try_open_completions_event(self): - Equal = self.assertEqual - autocomplete = self.autocomplete - trycompletions = self.autocomplete.try_open_completions_event - o_c_l = Func() - autocomplete._open_completions_later = o_c_l - - # _open_completions_later should not be called with no text in editor. - trycompletions('event') - Equal(o_c_l.args, None) - - # _open_completions_later should be called with COMPLETE_ATTRIBUTES (1). - self.text.insert('1.0', 're.') - trycompletions('event') - Equal(o_c_l.args, (False, False, False, 1)) - - # _open_completions_later should be called with COMPLETE_FILES (2). - self.text.delete('1.0', 'end') - self.text.insert('1.0', '"./Lib/') - trycompletions('event') - Equal(o_c_l.args, (False, False, False, 2)) + # Call _open_completions and break. + acp = self.autocomplete + open_c = Func() + acp.open_completions = open_c + self.assertEqual(acp.force_open_completions_event('event'), 'break') + self.assertEqual(open_c.args[0], ac.FORCE) def test_autocomplete_event(self): Equal = self.assertEqual - autocomplete = self.autocomplete + acp = self.autocomplete - # Test that the autocomplete event is ignored if user is pressing a - # modifier key in addition to the tab key. + # Result of autocomplete event: If modified tab, None. ev = Event(mc_state=True) - self.assertIsNone(autocomplete.autocomplete_event(ev)) + self.assertIsNone(acp.autocomplete_event(ev)) del ev.mc_state - # Test that tab after whitespace is ignored. + # If tab after whitespace, None. self.text.insert('1.0', ' """Docstring.\n ') - self.assertIsNone(autocomplete.autocomplete_event(ev)) + self.assertIsNone(acp.autocomplete_event(ev)) self.text.delete('1.0', 'end') - # If autocomplete window is open, complete() method is called. + # If active autocomplete window, complete() and 'break'. self.text.insert('1.0', 're.') - # This must call autocomplete._make_autocomplete_window(). - Equal(self.autocomplete.autocomplete_event(ev), 'break') - - # If autocomplete window is not active or does not exist, - # open_completions is called. Return depends on its return. - autocomplete._remove_autocomplete_window() - o_cs = Func() # .result = None. - autocomplete.open_completions = o_cs - Equal(self.autocomplete.autocomplete_event(ev), None) - Equal(o_cs.args, (False, True, True)) - o_cs.result = True - Equal(self.autocomplete.autocomplete_event(ev), 'break') - Equal(o_cs.args, (False, True, True)) - - def test_open_completions_later(self): - # Test that autocomplete._delayed_completion_id is set. + acp.autocompletewindow = mock = Mock() + mock.is_active = Mock(return_value=True) + Equal(acp.autocomplete_event(ev), 'break') + mock.complete.assert_called_once() + acp.autocompletewindow = None + + # If no active autocomplete window, open_completions(), None/break. + open_c = Func(result=False) + acp.open_completions = open_c + Equal(acp.autocomplete_event(ev), None) + Equal(open_c.args[0], ac.TAB) + open_c.result = True + Equal(acp.autocomplete_event(ev), 'break') + Equal(open_c.args[0], ac.TAB) + + def test_try_open_completions_event(self): + Equal = self.assertEqual + text = self.text acp = self.autocomplete + trycompletions = acp.try_open_completions_event + after = Func(result='after1') + acp.text.after = after + + # If no text or trigger, after not called. + trycompletions() + Equal(after.called, 0) + text.insert('1.0', 're') + trycompletions() + Equal(after.called, 0) + + # Attribute needed, no existing callback. + text.insert('insert', ' re.') acp._delayed_completion_id = None - acp._open_completions_later(False, False, False, ac.COMPLETE_ATTRIBUTES) + trycompletions() + Equal(acp._delayed_completion_index, text.index('insert')) + Equal(after.args, + (acp.popupwait, acp._delayed_open_completions, ac.TRY_A)) cb1 = acp._delayed_completion_id - self.assertTrue(cb1.startswith('after')) - - # Test that cb1 is cancelled and cb2 is new. - acp._open_completions_later(False, False, False, ac.COMPLETE_FILES) - self.assertNotIn(cb1, self.root.tk.call('after', 'info')) - cb2 = acp._delayed_completion_id - self.assertTrue(cb2.startswith('after') and cb2 != cb1) - self.text.after_cancel(cb2) + Equal(cb1, 'after1') + + # File needed, existing callback cancelled. + text.insert('insert', ' "./Lib/') + after.result = 'after2' + cancel = Func() + acp.text.after_cancel = cancel + trycompletions() + Equal(acp._delayed_completion_index, text.index('insert')) + Equal(cancel.args, (cb1,)) + Equal(after.args, + (acp.popupwait, acp._delayed_open_completions, ac.TRY_F)) + Equal(acp._delayed_completion_id, 'after2') def test_delayed_open_completions(self): - # Test that autocomplete._delayed_completion_id set to None - # and that open_completions is not called if the index is not - # equal to _delayed_completion_index. + Equal = self.assertEqual acp = self.autocomplete - acp.open_completions = Func() + open_c = Func() + acp.open_completions = open_c + self.text.insert('1.0', '"dict.') + + # Set autocomplete._delayed_completion_id to None. + # Text index changed, don't call open_completions. acp._delayed_completion_id = 'after' acp._delayed_completion_index = self.text.index('insert+1c') - acp._delayed_open_completions(1, 2, 3) + acp._delayed_open_completions('dummy') self.assertIsNone(acp._delayed_completion_id) - self.assertEqual(acp.open_completions.called, 0) + Equal(open_c.called, 0) - # Test that open_completions is called if indexes match. + # Text index unchanged, call open_completions. acp._delayed_completion_index = self.text.index('insert') - acp._delayed_open_completions(1, 2, 3, ac.COMPLETE_FILES) - self.assertEqual(acp.open_completions.args, (1, 2, 3, 2)) + acp._delayed_open_completions((1, 2, 3, ac.FILES)) + self.assertEqual(open_c.args[0], (1, 2, 3, ac.FILES)) + + def test_oc_cancel_comment(self): + none = self.assertIsNone + acp = self.autocomplete + + # Comment is in neither code or string. + acp._delayed_completion_id = 'after' + after = Func(result='after') + acp.text.after_cancel = after + self.text.insert(1.0, '# comment') + none(acp.open_completions(ac.TAB)) # From 'else' after 'elif'. + none(acp._delayed_completion_id) + + def test_oc_no_list(self): + acp = self.autocomplete + fetch = Func(result=([],[])) + acp.fetch_completions = fetch + self.text.insert('1.0', 'object') + self.assertIsNone(acp.open_completions(ac.TAB)) + self.text.insert('insert', '.') + self.assertIsNone(acp.open_completions(ac.TAB)) + self.assertEqual(fetch.called, 2) + + + def test_open_completions_none(self): + # Test other two None returns. + none = self.assertIsNone + acp = self.autocomplete + + # No object for attributes or need call not allowed. + self.text.insert(1.0, '.') + none(acp.open_completions(ac.TAB)) + self.text.insert('insert', ' int().') + none(acp.open_completions(ac.TAB)) + + # Blank or quote trigger 'if complete ...'. + self.text.delete(1.0, 'end') + self.assertFalse(acp.open_completions(ac.TAB)) + self.text.insert('1.0', '"') + self.assertFalse(acp.open_completions(ac.TAB)) + self.text.delete('1.0', 'end') + + class dummy_acw(): + __init__ = Func() + show_window = Func(result=False) + hide_window = Func() def test_open_completions(self): - # Test completions of files and attributes as well as non-completion - # of errors. - self.text.insert('1.0', 'pr') - self.assertTrue(self.autocomplete.open_completions(False, True, True)) + # Test completions of files and attributes. + acp = self.autocomplete + fetch = Func(result=(['tem'],['tem', '_tem'])) + acp.fetch_completions = fetch + def make_acw(): return self.dummy_acw() + acp._make_autocomplete_window = make_acw + + self.text.insert('1.0', 'int.') + acp.open_completions(ac.TAB) + self.assertIsInstance(acp.autocompletewindow, self.dummy_acw) self.text.delete('1.0', 'end') # Test files. self.text.insert('1.0', '"t') - #self.assertTrue(self.autocomplete.open_completions(False, True, True)) - self.text.delete('1.0', 'end') - - # Test with blank will fail. - self.assertFalse(self.autocomplete.open_completions(False, True, True)) - - # Test with only string quote will fail. - self.text.insert('1.0', '"') - self.assertFalse(self.autocomplete.open_completions(False, True, True)) + self.assertTrue(acp.open_completions(ac.TAB)) self.text.delete('1.0', 'end') def test_fetch_completions(self): @@ -174,21 +224,21 @@ def test_fetch_completions(self): # a small list containing non-private variables. # For file completion, a large list containing all files in the path, # and a small list containing files that do not start with '.'. - autocomplete = self.autocomplete - small, large = self.autocomplete.fetch_completions( - '', ac.COMPLETE_ATTRIBUTES) + acp = self.autocomplete + small, large = acp.fetch_completions( + '', ac.ATTRS) if __main__.__file__ != ac.__file__: self.assertNotIn('AutoComplete', small) # See issue 36405. # Test attributes - s, b = autocomplete.fetch_completions('', ac.COMPLETE_ATTRIBUTES) + s, b = acp.fetch_completions('', ac.ATTRS) self.assertLess(len(small), len(large)) self.assertTrue(all(filter(lambda x: x.startswith('_'), s))) self.assertTrue(any(filter(lambda x: x.startswith('_'), b))) # Test smalll should respect to __all__. with patch.dict('__main__.__dict__', {'__all__': ['a', 'b']}): - s, b = autocomplete.fetch_completions('', ac.COMPLETE_ATTRIBUTES) + s, b = acp.fetch_completions('', ac.ATTRS) self.assertEqual(s, ['a', 'b']) self.assertIn('__name__', b) # From __main__.__dict__ self.assertIn('sum', b) # From __main__.__builtins__.__dict__ @@ -197,7 +247,7 @@ def test_fetch_completions(self): mock = Mock() mock._private = Mock() with patch.dict('__main__.__dict__', {'foo': mock}): - s, b = autocomplete.fetch_completions('foo', ac.COMPLETE_ATTRIBUTES) + s, b = acp.fetch_completions('foo', ac.ATTRS) self.assertNotIn('_private', s) self.assertIn('_private', b) self.assertEqual(s, [i for i in sorted(dir(mock)) if i[:1] != '_']) @@ -211,36 +261,36 @@ def _listdir(path): return ['monty', 'python', '.hidden'] with patch.object(os, 'listdir', _listdir): - s, b = autocomplete.fetch_completions('', ac.COMPLETE_FILES) + s, b = acp.fetch_completions('', ac.FILES) self.assertEqual(s, ['bar', 'foo']) self.assertEqual(b, ['.hidden', 'bar', 'foo']) - s, b = autocomplete.fetch_completions('~', ac.COMPLETE_FILES) + s, b = acp.fetch_completions('~', ac.FILES) self.assertEqual(s, ['monty', 'python']) self.assertEqual(b, ['.hidden', 'monty', 'python']) def test_get_entity(self): # Test that a name is in the namespace of sys.modules and # __main__.__dict__. - autocomplete = self.autocomplete + acp = self.autocomplete Equal = self.assertEqual - Equal(self.autocomplete.get_entity('int'), int) + Equal(acp.get_entity('int'), int) # Test name from sys.modules. mock = Mock() with patch.dict('sys.modules', {'tempfile': mock}): - Equal(autocomplete.get_entity('tempfile'), mock) + Equal(acp.get_entity('tempfile'), mock) # Test name from __main__.__dict__. di = {'foo': 10, 'bar': 20} with patch.dict('__main__.__dict__', {'d': di}): - Equal(autocomplete.get_entity('d'), di) + Equal(acp.get_entity('d'), di) # Test name not in namespace. with patch.dict('__main__.__dict__', {}): with self.assertRaises(NameError): - autocomplete.get_entity('not_exist') + acp.get_entity('not_exist') if __name__ == '__main__': diff --git a/Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst b/Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst new file mode 100644 index 000000000000..a44fd3b59b4f --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst @@ -0,0 +1 @@ +IDLE - Refactor autocompete and improve testing. From webhook-mailer at python.org Sun Aug 4 20:09:40 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 05 Aug 2019 00:09:40 -0000 Subject: [Python-checkins] bpo-36419: IDLE - Refactor autocompete and improve testing. (GH-15121) Message-ID: https://github.com/python/cpython/commit/4969192f99046bc0f6453185082c00f7a6e132ec commit: 4969192f99046bc0f6453185082c00f7a6e132ec branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-04T17:09:36-07:00 summary: bpo-36419: IDLE - Refactor autocompete and improve testing. (GH-15121) (cherry picked from commit 1213123005d9f94bb5027c0a5256ea4d3e97b61d) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst M Lib/idlelib/autocomplete.py M Lib/idlelib/autocomplete_w.py M Lib/idlelib/idle_test/test_autocomplete.py diff --git a/Lib/idlelib/autocomplete.py b/Lib/idlelib/autocomplete.py index e20b757d8710..c623d45a1534 100644 --- a/Lib/idlelib/autocomplete.py +++ b/Lib/idlelib/autocomplete.py @@ -8,42 +8,45 @@ import string import sys -# These constants represent the two different types of completions. -# They must be defined here so autocomple_w can import them. -COMPLETE_ATTRIBUTES, COMPLETE_FILES = range(1, 2+1) - +# Two types of completions; defined here for autocomplete_w import below. +ATTRS, FILES = 0, 1 from idlelib import autocomplete_w from idlelib.config import idleConf from idlelib.hyperparser import HyperParser +# Tuples passed to open_completions. +# EvalFunc, Complete, WantWin, Mode +FORCE = True, False, True, None # Control-Space. +TAB = False, True, True, None # Tab. +TRY_A = False, False, False, ATTRS # '.' for attributes. +TRY_F = False, False, False, FILES # '/' in quotes for file name. + # This string includes all chars that may be in an identifier. # TODO Update this here and elsewhere. ID_CHARS = string.ascii_letters + string.digits + "_" -SEPS = os.sep -if os.altsep: # e.g. '/' on Windows... - SEPS += os.altsep - +SEPS = f"{os.sep}{os.altsep if os.altsep else ''}" +TRIGGERS = f".{SEPS}" class AutoComplete: def __init__(self, editwin=None): self.editwin = editwin - if editwin is not None: # not in subprocess or test + if editwin is not None: # not in subprocess or no-gui test self.text = editwin.text - self.autocompletewindow = None - # id of delayed call, and the index of the text insert when - # the delayed call was issued. If _delayed_completion_id is - # None, there is no delayed call. - self._delayed_completion_id = None - self._delayed_completion_index = None + self.autocompletewindow = None + # id of delayed call, and the index of the text insert when + # the delayed call was issued. If _delayed_completion_id is + # None, there is no delayed call. + self._delayed_completion_id = None + self._delayed_completion_index = None @classmethod def reload(cls): cls.popupwait = idleConf.GetOption( "extensions", "AutoComplete", "popupwait", type="int", default=0) - def _make_autocomplete_window(self): + def _make_autocomplete_window(self): # Makes mocking easier. return autocomplete_w.AutoCompleteWindow(self.text) def _remove_autocomplete_window(self, event=None): @@ -52,30 +55,12 @@ def _remove_autocomplete_window(self, event=None): self.autocompletewindow = None def force_open_completions_event(self, event): - """Happens when the user really wants to open a completion list, even - if a function call is needed. - """ - self.open_completions(True, False, True) + "(^space) Open completion list, even if a function call is needed." + self.open_completions(FORCE) return "break" - def try_open_completions_event(self, event): - """Happens when it would be nice to open a completion list, but not - really necessary, for example after a dot, so function - calls won't be made. - """ - lastchar = self.text.get("insert-1c") - if lastchar == ".": - self._open_completions_later(False, False, False, - COMPLETE_ATTRIBUTES) - elif lastchar in SEPS: - self._open_completions_later(False, False, False, - COMPLETE_FILES) - def autocomplete_event(self, event): - """Happens when the user wants to complete his word, and if necessary, - open a completion list after that (if there is more than one - completion) - """ + "(tab) Complete word or open list if multiple options." if hasattr(event, "mc_state") and event.mc_state or\ not self.text.get("insert linestart", "insert").strip(): # A modifier was pressed along with the tab or @@ -85,34 +70,34 @@ def autocomplete_event(self, event): self.autocompletewindow.complete() return "break" else: - opened = self.open_completions(False, True, True) + opened = self.open_completions(TAB) return "break" if opened else None - def _open_completions_later(self, *args): - self._delayed_completion_index = self.text.index("insert") - if self._delayed_completion_id is not None: - self.text.after_cancel(self._delayed_completion_id) - self._delayed_completion_id = \ - self.text.after(self.popupwait, self._delayed_open_completions, - *args) - - def _delayed_open_completions(self, *args): + def try_open_completions_event(self, event=None): + "(./) Open completion list after pause with no movement." + lastchar = self.text.get("insert-1c") + if lastchar in TRIGGERS: + args = TRY_A if lastchar == "." else TRY_F + self._delayed_completion_index = self.text.index("insert") + if self._delayed_completion_id is not None: + self.text.after_cancel(self._delayed_completion_id) + self._delayed_completion_id = self.text.after( + self.popupwait, self._delayed_open_completions, args) + + def _delayed_open_completions(self, args): + "Call open_completions if index unchanged." self._delayed_completion_id = None if self.text.index("insert") == self._delayed_completion_index: - self.open_completions(*args) + self.open_completions(args) - def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): + def open_completions(self, args): """Find the completions and create the AutoCompleteWindow. Return True if successful (no syntax error or so found). If complete is True, then if there's nothing to complete and no start of completion, won't open completions and return False. If mode is given, will open a completion list only in this mode. - - Action Function Eval Complete WantWin Mode - ^space force_open_completions True, False, True no - . or / try_open_completions False, False, False yes - tab autocomplete False, True, True no """ + evalfuncs, complete, wantwin, mode = args # Cancel another delayed call, if it exists. if self._delayed_completion_id is not None: self.text.after_cancel(self._delayed_completion_id) @@ -121,14 +106,14 @@ def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): hp = HyperParser(self.editwin, "insert") curline = self.text.get("insert linestart", "insert") i = j = len(curline) - if hp.is_in_string() and (not mode or mode==COMPLETE_FILES): + if hp.is_in_string() and (not mode or mode==FILES): # Find the beginning of the string. # fetch_completions will look at the file system to determine # whether the string value constitutes an actual file name # XXX could consider raw strings here and unescape the string # value if it's not raw. self._remove_autocomplete_window() - mode = COMPLETE_FILES + mode = FILES # Find last separator or string start while i and curline[i-1] not in "'\"" + SEPS: i -= 1 @@ -138,17 +123,17 @@ def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): while i and curline[i-1] not in "'\"": i -= 1 comp_what = curline[i:j] - elif hp.is_in_code() and (not mode or mode==COMPLETE_ATTRIBUTES): + elif hp.is_in_code() and (not mode or mode==ATTRS): self._remove_autocomplete_window() - mode = COMPLETE_ATTRIBUTES + mode = ATTRS while i and (curline[i-1] in ID_CHARS or ord(curline[i-1]) > 127): i -= 1 comp_start = curline[i:j] - if i and curline[i-1] == '.': + if i and curline[i-1] == '.': # Need object with attributes. hp.set_index("insert-%dc" % (len(curline)-(i-1))) comp_what = hp.get_expression() - if not comp_what or \ - (not evalfuncs and comp_what.find('(') != -1): + if (not comp_what or + (not evalfuncs and comp_what.find('(') != -1)): return None else: comp_what = "" @@ -163,7 +148,7 @@ def open_completions(self, evalfuncs, complete, userWantsWin, mode=None): self.autocompletewindow = self._make_autocomplete_window() return not self.autocompletewindow.show_window( comp_lists, "insert-%dc" % len(comp_start), - complete, mode, userWantsWin) + complete, mode, wantwin) def fetch_completions(self, what, mode): """Return a pair of lists of completions for something. The first list @@ -185,7 +170,7 @@ def fetch_completions(self, what, mode): return rpcclt.remotecall("exec", "get_the_completion_list", (what, mode), {}) else: - if mode == COMPLETE_ATTRIBUTES: + if mode == ATTRS: if what == "": namespace = {**__main__.__builtins__.__dict__, **__main__.__dict__} @@ -207,7 +192,7 @@ def fetch_completions(self, what, mode): except: return [], [] - elif mode == COMPLETE_FILES: + elif mode == FILES: if what == "": what = "." try: diff --git a/Lib/idlelib/autocomplete_w.py b/Lib/idlelib/autocomplete_w.py index c24962527736..c69ab4a36836 100644 --- a/Lib/idlelib/autocomplete_w.py +++ b/Lib/idlelib/autocomplete_w.py @@ -6,7 +6,7 @@ from tkinter import * from tkinter.ttk import Frame, Scrollbar -from idlelib.autocomplete import COMPLETE_FILES, COMPLETE_ATTRIBUTES +from idlelib.autocomplete import FILES, ATTRS from idlelib.multicall import MC_SHIFT HIDE_VIRTUAL_EVENT_NAME = "<>" @@ -39,8 +39,7 @@ def __init__(self, widget): self.completions = None # A list with more completions, or None self.morecompletions = None - # The completion mode. Either autocomplete.COMPLETE_ATTRIBUTES or - # autocomplete.COMPLETE_FILES + # The completion mode, either autocomplete.ATTRS or .FILES. self.mode = None # The current completion start, on the text box (a string) self.start = None @@ -73,8 +72,8 @@ def _change_start(self, newstart): def _binary_search(self, s): """Find the first index in self.completions where completions[i] is - greater or equal to s, or the last index if there is no such - one.""" + greater or equal to s, or the last index if there is no such. + """ i = 0; j = len(self.completions) while j > i: m = (i + j) // 2 @@ -87,7 +86,8 @@ def _binary_search(self, s): def _complete_string(self, s): """Assuming that s is the prefix of a string in self.completions, return the longest string which is a prefix of all the strings which - s is a prefix of them. If s is not a prefix of a string, return s.""" + s is a prefix of them. If s is not a prefix of a string, return s. + """ first = self._binary_search(s) if self.completions[first][:len(s)] != s: # There is not even one completion which s is a prefix of. @@ -116,8 +116,10 @@ def _complete_string(self, s): return first_comp[:i] def _selection_changed(self): - """Should be called when the selection of the Listbox has changed. - Updates the Listbox display and calls _change_start.""" + """Call when the selection of the Listbox has changed. + + Updates the Listbox display and calls _change_start. + """ cursel = int(self.listbox.curselection()[0]) self.listbox.see(cursel) @@ -153,8 +155,10 @@ def _selection_changed(self): def show_window(self, comp_lists, index, complete, mode, userWantsWin): """Show the autocomplete list, bind events. - If complete is True, complete the text, and if there is exactly one - matching completion, don't open a list.""" + + If complete is True, complete the text, and if there is exactly + one matching completion, don't open a list. + """ # Handle the start we already have self.completions, self.morecompletions = comp_lists self.mode = mode @@ -300,7 +304,7 @@ def keypress_event(self, event): if keysym != "Tab": self.lastkey_was_tab = False if (len(keysym) == 1 or keysym in ("underscore", "BackSpace") - or (self.mode == COMPLETE_FILES and keysym in + or (self.mode == FILES and keysym in ("period", "minus"))) \ and not (state & ~MC_SHIFT): # Normal editing of text @@ -329,10 +333,10 @@ def keypress_event(self, event): self.hide_window() return 'break' - elif (self.mode == COMPLETE_ATTRIBUTES and keysym in + elif (self.mode == ATTRS and keysym in ("period", "space", "parenleft", "parenright", "bracketleft", "bracketright")) or \ - (self.mode == COMPLETE_FILES and keysym in + (self.mode == FILES and keysym in ("slash", "backslash", "quotedbl", "apostrophe")) \ and not (state & ~MC_SHIFT): # If start is a prefix of the selection, but is not '' when @@ -340,7 +344,7 @@ def keypress_event(self, event): # selected completion. Anyway, close the list. cursel = int(self.listbox.curselection()[0]) if self.completions[cursel][:len(self.start)] == self.start \ - and (self.mode == COMPLETE_ATTRIBUTES or self.start): + and (self.mode == ATTRS or self.start): self._change_start(self.completions[cursel]) self.hide_window() return None diff --git a/Lib/idlelib/idle_test/test_autocomplete.py b/Lib/idlelib/idle_test/test_autocomplete.py index 6181b29ec250..2c478cd5c2a1 100644 --- a/Lib/idlelib/idle_test/test_autocomplete.py +++ b/Lib/idlelib/idle_test/test_autocomplete.py @@ -1,4 +1,4 @@ -"Test autocomplete, coverage 87%." +"Test autocomplete, coverage 93%." import unittest from unittest.mock import Mock, patch @@ -45,127 +45,177 @@ def setUp(self): def test_init(self): self.assertEqual(self.autocomplete.editwin, self.editor) + self.assertEqual(self.autocomplete.text, self.text) def test_make_autocomplete_window(self): testwin = self.autocomplete._make_autocomplete_window() self.assertIsInstance(testwin, acw.AutoCompleteWindow) def test_remove_autocomplete_window(self): - self.autocomplete.autocompletewindow = ( - self.autocomplete._make_autocomplete_window()) - self.autocomplete._remove_autocomplete_window() - self.assertIsNone(self.autocomplete.autocompletewindow) + acp = self.autocomplete + acp.autocompletewindow = m = Mock() + acp._remove_autocomplete_window() + m.hide_window.assert_called_once() + self.assertIsNone(acp.autocompletewindow) def test_force_open_completions_event(self): - # Test that force_open_completions_event calls _open_completions. - o_cs = Func() - self.autocomplete.open_completions = o_cs - self.autocomplete.force_open_completions_event('event') - self.assertEqual(o_cs.args, (True, False, True)) - - def test_try_open_completions_event(self): - Equal = self.assertEqual - autocomplete = self.autocomplete - trycompletions = self.autocomplete.try_open_completions_event - o_c_l = Func() - autocomplete._open_completions_later = o_c_l - - # _open_completions_later should not be called with no text in editor. - trycompletions('event') - Equal(o_c_l.args, None) - - # _open_completions_later should be called with COMPLETE_ATTRIBUTES (1). - self.text.insert('1.0', 're.') - trycompletions('event') - Equal(o_c_l.args, (False, False, False, 1)) - - # _open_completions_later should be called with COMPLETE_FILES (2). - self.text.delete('1.0', 'end') - self.text.insert('1.0', '"./Lib/') - trycompletions('event') - Equal(o_c_l.args, (False, False, False, 2)) + # Call _open_completions and break. + acp = self.autocomplete + open_c = Func() + acp.open_completions = open_c + self.assertEqual(acp.force_open_completions_event('event'), 'break') + self.assertEqual(open_c.args[0], ac.FORCE) def test_autocomplete_event(self): Equal = self.assertEqual - autocomplete = self.autocomplete + acp = self.autocomplete - # Test that the autocomplete event is ignored if user is pressing a - # modifier key in addition to the tab key. + # Result of autocomplete event: If modified tab, None. ev = Event(mc_state=True) - self.assertIsNone(autocomplete.autocomplete_event(ev)) + self.assertIsNone(acp.autocomplete_event(ev)) del ev.mc_state - # Test that tab after whitespace is ignored. + # If tab after whitespace, None. self.text.insert('1.0', ' """Docstring.\n ') - self.assertIsNone(autocomplete.autocomplete_event(ev)) + self.assertIsNone(acp.autocomplete_event(ev)) self.text.delete('1.0', 'end') - # If autocomplete window is open, complete() method is called. + # If active autocomplete window, complete() and 'break'. self.text.insert('1.0', 're.') - # This must call autocomplete._make_autocomplete_window(). - Equal(self.autocomplete.autocomplete_event(ev), 'break') - - # If autocomplete window is not active or does not exist, - # open_completions is called. Return depends on its return. - autocomplete._remove_autocomplete_window() - o_cs = Func() # .result = None. - autocomplete.open_completions = o_cs - Equal(self.autocomplete.autocomplete_event(ev), None) - Equal(o_cs.args, (False, True, True)) - o_cs.result = True - Equal(self.autocomplete.autocomplete_event(ev), 'break') - Equal(o_cs.args, (False, True, True)) - - def test_open_completions_later(self): - # Test that autocomplete._delayed_completion_id is set. + acp.autocompletewindow = mock = Mock() + mock.is_active = Mock(return_value=True) + Equal(acp.autocomplete_event(ev), 'break') + mock.complete.assert_called_once() + acp.autocompletewindow = None + + # If no active autocomplete window, open_completions(), None/break. + open_c = Func(result=False) + acp.open_completions = open_c + Equal(acp.autocomplete_event(ev), None) + Equal(open_c.args[0], ac.TAB) + open_c.result = True + Equal(acp.autocomplete_event(ev), 'break') + Equal(open_c.args[0], ac.TAB) + + def test_try_open_completions_event(self): + Equal = self.assertEqual + text = self.text acp = self.autocomplete + trycompletions = acp.try_open_completions_event + after = Func(result='after1') + acp.text.after = after + + # If no text or trigger, after not called. + trycompletions() + Equal(after.called, 0) + text.insert('1.0', 're') + trycompletions() + Equal(after.called, 0) + + # Attribute needed, no existing callback. + text.insert('insert', ' re.') acp._delayed_completion_id = None - acp._open_completions_later(False, False, False, ac.COMPLETE_ATTRIBUTES) + trycompletions() + Equal(acp._delayed_completion_index, text.index('insert')) + Equal(after.args, + (acp.popupwait, acp._delayed_open_completions, ac.TRY_A)) cb1 = acp._delayed_completion_id - self.assertTrue(cb1.startswith('after')) - - # Test that cb1 is cancelled and cb2 is new. - acp._open_completions_later(False, False, False, ac.COMPLETE_FILES) - self.assertNotIn(cb1, self.root.tk.call('after', 'info')) - cb2 = acp._delayed_completion_id - self.assertTrue(cb2.startswith('after') and cb2 != cb1) - self.text.after_cancel(cb2) + Equal(cb1, 'after1') + + # File needed, existing callback cancelled. + text.insert('insert', ' "./Lib/') + after.result = 'after2' + cancel = Func() + acp.text.after_cancel = cancel + trycompletions() + Equal(acp._delayed_completion_index, text.index('insert')) + Equal(cancel.args, (cb1,)) + Equal(after.args, + (acp.popupwait, acp._delayed_open_completions, ac.TRY_F)) + Equal(acp._delayed_completion_id, 'after2') def test_delayed_open_completions(self): - # Test that autocomplete._delayed_completion_id set to None - # and that open_completions is not called if the index is not - # equal to _delayed_completion_index. + Equal = self.assertEqual acp = self.autocomplete - acp.open_completions = Func() + open_c = Func() + acp.open_completions = open_c + self.text.insert('1.0', '"dict.') + + # Set autocomplete._delayed_completion_id to None. + # Text index changed, don't call open_completions. acp._delayed_completion_id = 'after' acp._delayed_completion_index = self.text.index('insert+1c') - acp._delayed_open_completions(1, 2, 3) + acp._delayed_open_completions('dummy') self.assertIsNone(acp._delayed_completion_id) - self.assertEqual(acp.open_completions.called, 0) + Equal(open_c.called, 0) - # Test that open_completions is called if indexes match. + # Text index unchanged, call open_completions. acp._delayed_completion_index = self.text.index('insert') - acp._delayed_open_completions(1, 2, 3, ac.COMPLETE_FILES) - self.assertEqual(acp.open_completions.args, (1, 2, 3, 2)) + acp._delayed_open_completions((1, 2, 3, ac.FILES)) + self.assertEqual(open_c.args[0], (1, 2, 3, ac.FILES)) + + def test_oc_cancel_comment(self): + none = self.assertIsNone + acp = self.autocomplete + + # Comment is in neither code or string. + acp._delayed_completion_id = 'after' + after = Func(result='after') + acp.text.after_cancel = after + self.text.insert(1.0, '# comment') + none(acp.open_completions(ac.TAB)) # From 'else' after 'elif'. + none(acp._delayed_completion_id) + + def test_oc_no_list(self): + acp = self.autocomplete + fetch = Func(result=([],[])) + acp.fetch_completions = fetch + self.text.insert('1.0', 'object') + self.assertIsNone(acp.open_completions(ac.TAB)) + self.text.insert('insert', '.') + self.assertIsNone(acp.open_completions(ac.TAB)) + self.assertEqual(fetch.called, 2) + + + def test_open_completions_none(self): + # Test other two None returns. + none = self.assertIsNone + acp = self.autocomplete + + # No object for attributes or need call not allowed. + self.text.insert(1.0, '.') + none(acp.open_completions(ac.TAB)) + self.text.insert('insert', ' int().') + none(acp.open_completions(ac.TAB)) + + # Blank or quote trigger 'if complete ...'. + self.text.delete(1.0, 'end') + self.assertFalse(acp.open_completions(ac.TAB)) + self.text.insert('1.0', '"') + self.assertFalse(acp.open_completions(ac.TAB)) + self.text.delete('1.0', 'end') + + class dummy_acw(): + __init__ = Func() + show_window = Func(result=False) + hide_window = Func() def test_open_completions(self): - # Test completions of files and attributes as well as non-completion - # of errors. - self.text.insert('1.0', 'pr') - self.assertTrue(self.autocomplete.open_completions(False, True, True)) + # Test completions of files and attributes. + acp = self.autocomplete + fetch = Func(result=(['tem'],['tem', '_tem'])) + acp.fetch_completions = fetch + def make_acw(): return self.dummy_acw() + acp._make_autocomplete_window = make_acw + + self.text.insert('1.0', 'int.') + acp.open_completions(ac.TAB) + self.assertIsInstance(acp.autocompletewindow, self.dummy_acw) self.text.delete('1.0', 'end') # Test files. self.text.insert('1.0', '"t') - #self.assertTrue(self.autocomplete.open_completions(False, True, True)) - self.text.delete('1.0', 'end') - - # Test with blank will fail. - self.assertFalse(self.autocomplete.open_completions(False, True, True)) - - # Test with only string quote will fail. - self.text.insert('1.0', '"') - self.assertFalse(self.autocomplete.open_completions(False, True, True)) + self.assertTrue(acp.open_completions(ac.TAB)) self.text.delete('1.0', 'end') def test_fetch_completions(self): @@ -174,21 +224,21 @@ def test_fetch_completions(self): # a small list containing non-private variables. # For file completion, a large list containing all files in the path, # and a small list containing files that do not start with '.'. - autocomplete = self.autocomplete - small, large = self.autocomplete.fetch_completions( - '', ac.COMPLETE_ATTRIBUTES) + acp = self.autocomplete + small, large = acp.fetch_completions( + '', ac.ATTRS) if __main__.__file__ != ac.__file__: self.assertNotIn('AutoComplete', small) # See issue 36405. # Test attributes - s, b = autocomplete.fetch_completions('', ac.COMPLETE_ATTRIBUTES) + s, b = acp.fetch_completions('', ac.ATTRS) self.assertLess(len(small), len(large)) self.assertTrue(all(filter(lambda x: x.startswith('_'), s))) self.assertTrue(any(filter(lambda x: x.startswith('_'), b))) # Test smalll should respect to __all__. with patch.dict('__main__.__dict__', {'__all__': ['a', 'b']}): - s, b = autocomplete.fetch_completions('', ac.COMPLETE_ATTRIBUTES) + s, b = acp.fetch_completions('', ac.ATTRS) self.assertEqual(s, ['a', 'b']) self.assertIn('__name__', b) # From __main__.__dict__ self.assertIn('sum', b) # From __main__.__builtins__.__dict__ @@ -197,7 +247,7 @@ def test_fetch_completions(self): mock = Mock() mock._private = Mock() with patch.dict('__main__.__dict__', {'foo': mock}): - s, b = autocomplete.fetch_completions('foo', ac.COMPLETE_ATTRIBUTES) + s, b = acp.fetch_completions('foo', ac.ATTRS) self.assertNotIn('_private', s) self.assertIn('_private', b) self.assertEqual(s, [i for i in sorted(dir(mock)) if i[:1] != '_']) @@ -211,36 +261,36 @@ def _listdir(path): return ['monty', 'python', '.hidden'] with patch.object(os, 'listdir', _listdir): - s, b = autocomplete.fetch_completions('', ac.COMPLETE_FILES) + s, b = acp.fetch_completions('', ac.FILES) self.assertEqual(s, ['bar', 'foo']) self.assertEqual(b, ['.hidden', 'bar', 'foo']) - s, b = autocomplete.fetch_completions('~', ac.COMPLETE_FILES) + s, b = acp.fetch_completions('~', ac.FILES) self.assertEqual(s, ['monty', 'python']) self.assertEqual(b, ['.hidden', 'monty', 'python']) def test_get_entity(self): # Test that a name is in the namespace of sys.modules and # __main__.__dict__. - autocomplete = self.autocomplete + acp = self.autocomplete Equal = self.assertEqual - Equal(self.autocomplete.get_entity('int'), int) + Equal(acp.get_entity('int'), int) # Test name from sys.modules. mock = Mock() with patch.dict('sys.modules', {'tempfile': mock}): - Equal(autocomplete.get_entity('tempfile'), mock) + Equal(acp.get_entity('tempfile'), mock) # Test name from __main__.__dict__. di = {'foo': 10, 'bar': 20} with patch.dict('__main__.__dict__', {'d': di}): - Equal(autocomplete.get_entity('d'), di) + Equal(acp.get_entity('d'), di) # Test name not in namespace. with patch.dict('__main__.__dict__', {}): with self.assertRaises(NameError): - autocomplete.get_entity('not_exist') + acp.get_entity('not_exist') if __name__ == '__main__': diff --git a/Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst b/Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst new file mode 100644 index 000000000000..a44fd3b59b4f --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2019-08-04-17-10-01.bpo-36419.TJZqOc.rst @@ -0,0 +1 @@ +IDLE - Refactor autocompete and improve testing. From webhook-mailer at python.org Mon Aug 5 03:20:42 2019 From: webhook-mailer at python.org (Inada Naoki) Date: Mon, 05 Aug 2019 07:20:42 -0000 Subject: [Python-checkins] bpo-37729: gc: write stats at once (GH-15050) Message-ID: https://github.com/python/cpython/commit/e8ea34855c7635f8a84b430f17dc01a666f4c0ef commit: e8ea34855c7635f8a84b430f17dc01a666f4c0ef branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Inada Naoki date: 2019-08-05T16:20:25+09:00 summary: bpo-37729: gc: write stats at once (GH-15050) gc used several PySys_WriteStderr() calls to write stats. It caused stats mixed up when stderr is shared by multiple processes like this: gc: collecting generation 2... gc: objects in each generation: 0 0gc: collecting generation 2... gc: objects in each generation: 0 0 126077 126077 gc: objects in permanent generation: 0 gc: objects in permanent generation: 0 gc: done, 112575 unreachable, 0 uncollectablegc: done, 112575 unreachable, 0 uncollectable, 0.2223s elapsed , 0.2344s elapsed (cherry picked from commit bf8162c8c45338470bbe487c8769bba20bde66c2) Co-authored-by: Inada Naoki files: M Modules/gcmodule.c diff --git a/Modules/gcmodule.c b/Modules/gcmodule.c index 0cf00e839661..21839d95bd8c 100644 --- a/Modules/gcmodule.c +++ b/Modules/gcmodule.c @@ -962,6 +962,25 @@ clear_freelists(void) (void)PyContext_ClearFreeList(); } +// Show stats for objects in each gennerations. +static void +show_stats_each_generations(struct _gc_runtime_state *state) +{ + char buf[100]; + size_t pos = 0; + + for (int i = 0; i < NUM_GENERATIONS && pos < sizeof(buf); i++) { + pos += PyOS_snprintf(buf+pos, sizeof(buf)-pos, + " %"PY_FORMAT_SIZE_T"d", + gc_list_size(GEN_HEAD(state, i))); + } + + PySys_FormatStderr( + "gc: objects in each generation:%s\n" + "gc: objects in permanent generation: %zd\n", + buf, gc_list_size(&state->permanent_generation.head)); +} + /* This is the main function. Read this to understand how the * collection process works. */ static Py_ssize_t @@ -979,17 +998,9 @@ collect(struct _gc_runtime_state *state, int generation, _PyTime_t t1 = 0; /* initialize to prevent a compiler warning */ if (state->debug & DEBUG_STATS) { - PySys_WriteStderr("gc: collecting generation %d...\n", - generation); - PySys_WriteStderr("gc: objects in each generation:"); - for (i = 0; i < NUM_GENERATIONS; i++) - PySys_FormatStderr(" %zd", - gc_list_size(GEN_HEAD(state, i))); - PySys_WriteStderr("\ngc: objects in permanent generation: %zd", - gc_list_size(&state->permanent_generation.head)); + PySys_WriteStderr("gc: collecting generation %d...\n", generation); + show_stats_each_generations(state); t1 = _PyTime_GetMonotonicClock(); - - PySys_WriteStderr("\n"); } if (PyDTrace_GC_START_ENABLED()) @@ -1103,16 +1114,10 @@ collect(struct _gc_runtime_state *state, int generation, debug_cycle("uncollectable", FROM_GC(gc)); } if (state->debug & DEBUG_STATS) { - _PyTime_t t2 = _PyTime_GetMonotonicClock(); - - if (m == 0 && n == 0) - PySys_WriteStderr("gc: done"); - else - PySys_FormatStderr( - "gc: done, %zd unreachable, %zd uncollectable", - n+m, n); - PySys_WriteStderr(", %.4fs elapsed\n", - _PyTime_AsSecondsDouble(t2 - t1)); + double d = _PyTime_AsSecondsDouble(_PyTime_GetMonotonicClock() - t1); + PySys_FormatStderr( + "gc: done, %zd unreachable, %zd uncollectable, %.4fs elapsed\n", + n+m, n, d); } /* Append instances in the uncollectable set to a Python From webhook-mailer at python.org Mon Aug 5 05:19:32 2019 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Mon, 05 Aug 2019 09:19:32 -0000 Subject: [Python-checkins] [2.7] bpo-37730: Fix usage of NotImplemented instead of NotImplementedError in docs. (GH-15062). (GH-15133) Message-ID: https://github.com/python/cpython/commit/35f9bccd8198330579ecb4b4c503062f8b5da130 commit: 35f9bccd8198330579ecb4b4c503062f8b5da130 branch: 2.7 author: David H committer: Serhiy Storchaka date: 2019-08-05T12:19:26+03:00 summary: [2.7] bpo-37730: Fix usage of NotImplemented instead of NotImplementedError in docs. (GH-15062). (GH-15133) (cherry picked from commit ed5e8e06cbf766e89d6c58a882ee024abb5b2ed7) Co-authored-by: David H files: M Doc/library/_winreg.rst M PC/_winreg.c diff --git a/Doc/library/_winreg.rst b/Doc/library/_winreg.rst index a87cb9c51d98..177e1994f3fa 100644 --- a/Doc/library/_winreg.rst +++ b/Doc/library/_winreg.rst @@ -427,7 +427,7 @@ This module offers the following functions: *key* is an already open key, or one of the predefined :ref:`HKEY_* constants `. - Will generally raise :exc:`NotImplemented` if executed on a 32-bit + Will generally raise :exc:`NotImplementedError` if executed on a 32-bit operating system. If the key is not on the reflection list, the function succeeds but has no @@ -442,7 +442,7 @@ This module offers the following functions: *key* is an already open key, or one of the predefined :ref:`HKEY_* constants `. - Will generally raise :exc:`NotImplemented` if executed on a 32-bit + Will generally raise :exc:`NotImplementedError` if executed on a 32-bit operating system. Restoring reflection for a key does not affect reflection of any subkeys. @@ -457,7 +457,7 @@ This module offers the following functions: Returns ``True`` if reflection is disabled. - Will generally raise :exc:`NotImplemented` if executed on a 32-bit + Will generally raise :exc:`NotImplementedError` if executed on a 32-bit operating system. diff --git a/PC/_winreg.c b/PC/_winreg.c index 3b887e075334..e0fdff2250ba 100644 --- a/PC/_winreg.c +++ b/PC/_winreg.c @@ -322,19 +322,19 @@ PyDoc_STRVAR(SetValueEx_doc, PyDoc_STRVAR(DisableReflectionKey_doc, "Disables registry reflection for 32-bit processes running on a 64-bit\n" -"Operating System. Will generally raise NotImplemented if executed on\n" +"Operating System. Will generally raise NotImplementedError if executed on\n" "a 32-bit Operating System.\n" "If the key is not on the reflection list, the function succeeds but has no effect.\n" "Disabling reflection for a key does not affect reflection of any subkeys."); PyDoc_STRVAR(EnableReflectionKey_doc, "Restores registry reflection for the specified disabled key.\n" -"Will generally raise NotImplemented if executed on a 32-bit Operating System.\n" +"Will generally raise NotImplementedError if executed on a 32-bit Operating System.\n" "Restoring reflection for a key does not affect reflection of any subkeys."); PyDoc_STRVAR(QueryReflectionKey_doc, "bool = QueryReflectionKey(hkey) - Determines the reflection state for the specified key.\n" -"Will generally raise NotImplemented if executed on a 32-bit Operating System.\n"); +"Will generally raise NotImplementedError if executed on a 32-bit Operating System.\n"); /* PyHKEY docstrings */ PyDoc_STRVAR(PyHKEY_doc, From webhook-mailer at python.org Mon Aug 5 16:33:27 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Mon, 05 Aug 2019 20:33:27 -0000 Subject: [Python-checkins] bpo-37759: First round of major edits to Whatsnew 3.8 (GH-15127) Message-ID: https://github.com/python/cpython/commit/4f9ffc9d1a6a293563deaaaaf4a13331302219b4 commit: 4f9ffc9d1a6a293563deaaaaf4a13331302219b4 branch: master author: Raymond Hettinger committer: GitHub date: 2019-08-05T13:33:19-07:00 summary: bpo-37759: First round of major edits to Whatsnew 3.8 (GH-15127) files: A Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst M Doc/whatsnew/3.8.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 0455688841fb..d8062e772f6e 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -42,21 +42,25 @@ This saves the maintainer the effort of going through the Mercurial log when researching a change. -This article explains the new features in Python 3.8, compared to 3.7. +:Editor: Raymond Hettinger +This article explains the new features in Python 3.8, compared to 3.7. For full details, see the :ref:`changelog `. -.. note:: +Prerelease users should be aware that this document is currently in +draft form. It will be updated as Python 3.8 moves towards release, so +it's worth checking back even after reading earlier versions. Some +notable items not yet covered are: - Prerelease users should be aware that this document is currently in draft - form. It will be updated substantially as Python 3.8 moves towards release, - so it's worth checking back even after reading earlier versions. +* :pep:`578` - Runtime audit hooks for potentially sensitive operations +* ``python -m asyncio`` runs a natively async REPL - Some notable items not yet covered here: +.. testsetup:: - * :pep:`578` - Runtime audit hooks for potentially sensitive operations - * ``python -m asyncio`` runs a natively async REPL - * ... + from datetime import date + from math import cos, radians + import re + import math Summary -- Release highlights @@ -76,12 +80,43 @@ New Features Assignment expressions ---------------------- -There is new syntax (the "walrus operator", ``:=``) to assign values -to variables as part of an expression. Example:: +There is new syntax ``:=`` that assigns values to variables as part of a larger +expression. It is affectionately known as "walrus operator" due to +its resemblance to `the eyes and tusks of a walrus +`_. + +In this example, the assignment expression helps avoid calling +:func:`len` twice:: if (n := len(a)) > 10: print(f"List is too long ({n} elements, expected <= 10)") +A similar benefit arises during regular expression matching where +match objects are needed twice, once to test whether a match +occurred and another to extract a subgroup:: + + discount = 0.0 + if (mo := re.search(r'(\d+)% discount', advertisement)): + discount = float(mo.group(1)) / 100.0 + +The operator is also useful with while-loops that compute +a value to test loop termination and then need that same +value again in the body of the loop:: + + # Loop over fixed length blocks + while (block := f.read(256)) != '': + process(block) + +Another motivating use case arises in list comprehensions where +a value computed in a filtering condition is also needed in +the expression body:: + + [clean_name.title() for name in names + if (clean_name := normalize('NFC', name)) in allowed_names] + +Try to limit use of the walrus operator to clean cases that reduce +complexity and improve readability. + See :pep:`572` for a full description. (Contributed by Emily Morehouse in :issue:`35224`.) @@ -92,20 +127,69 @@ See :pep:`572` for a full description. Positional-only parameters -------------------------- -There is new syntax (``/``) to indicate that some function parameters -must be specified positionally (i.e., cannot be used as keyword -arguments). This is the same notation as shown by ``help()`` for -functions implemented in C (produced by Larry Hastings' "Argument -Clinic" tool). Example:: +There is a new function parameter syntax ``/`` to indicate that some +function parameters must be specified positionally and cannot be used as +keyword arguments. This is the same notation shown by ``help()`` for C +functions annotated with Larry Hastings' `Argument Clinic +`_ tool. + +In the following example, parameters *a* and *b* are positional-only, +while *c* or *d* can be positional or keyword, and *e* or *f* are +required to be keywords:: + + def f(a, b, /, c, d, *, e, f): + print(a, b, c, d, e, f) + +The following is a valid call:: + + f(10, 20, 30, d=40, e=50, f=60) + +However, these are invalid calls:: + + f(10, b=20, c=30, d=40, e=50, f=60) # b cannot be a keyword argument + f(10, 20, 30, 40, 50, f=60) # e must be a keyword argument + +One use case for this notation is that it allows pure Python functions +to fully emulate behaviors of existing C coded functions. For example, +the built-in :func:`pow` function does not accept keyword arguments:: def pow(x, y, z=None, /): - r = x**y - if z is not None: - r %= z - return r + "Emulate the built in pow() function" + r = x ** y + return r if z is None else r%z + +Another use case is to preclude keyword arguments when the parameter +name is not helpful. For example, the builtin :func:`len` function has +the signature ``len(obj, /)``. This precludes awkward calls such as:: -Now ``pow(2, 10)`` and ``pow(2, 10, 17)`` are valid calls, but -``pow(x=2, y=10)`` and ``pow(2, 10, z=17)`` are invalid. + len(obj='hello') # The "obj" keyword argument impairs readability + +A further benefit of marking a parameter as positional-only is that it +allows the parameter name to be changed in the future without risk of +breaking client code. For example, in the :mod:`statistics` module, the +parameter name *dist* may be changed in the future. This was made +possible with the following function specification:: + + def quantiles(dist, /, *, n=4, method='exclusive') + ... + +Since the parameters to the left of ``/`` are not exposed as possible +keywords, the parameters names remain available for use in ``**kwargs``:: + + >>> def f(a, b, /, **kwargs): + ... print(a, b, kwargs) + ... + >>> f(10, 20, a=1, b=2, c=3) # a and b are used in two ways + 10 20 {'a': 1, 'b': 2, 'c': 3} + +This greatly simplifies the implementation of functions and methods +that need to accept arbitrary keyword arguments. For example, here +is an except from code in the :mod:`collections` module:: + + class Counter(dict): + + def __init__(self, iterable=None, /, **kwds): + # Note "iterable" is a possible keyword argument See :pep:`570` for a full description. @@ -174,17 +258,31 @@ Android and Cygwin, whose cases are handled by the script); this change is backward incompatible on purpose. (Contributed by Victor Stinner in :issue:`36721`.) -f-strings now support = for quick and easy debugging ------------------------------------------------------ -Add ``=`` specifier to f-strings. ``f'{expr=}'`` expands -to the text of the expression, an equal sign, then the repr of the -evaluated expression. So:: +f-strings support ``=`` for self-documenting expressions and debugging +---------------------------------------------------------------------- + +Added an ``=`` specifier to :term:`f-string`\s. An f-string such as +``f'{expr=}'`` will expand to the text of the expression, an equal sign, +then the representation of the evaluated expression. For example: - x = 3 - print(f'{x*9 + 15=}') + >>> user = 'eric_idle' + >>> member_since = date(1975, 7, 31) + >>> f'{user=} {member_since=}' + "user='eric_idle' member_since=datetime.date(1975, 7, 31)" -Would print ``x*9 + 15=42``. +The usual :ref:`f-string format specifiers ` allow more +control over how the result of the expression is displayed:: + + >>> delta = date.today() - member_since + >>> f'{user=!s} {delta.days=:,d}' + 'user=eric_idle delta.days=16,075' + +The ``=`` specifier will display the whole expression so that +calculations can be shown:: + + >>> print(f'{theta=} {cos(radians(theta))=:.3f}') + theta=30 cos(radians(theta))=0.866 (Contributed by Eric V. Smith and Larry Hastings in :issue:`36817`.) @@ -295,7 +393,13 @@ Other Language Changes or :meth:`~object.__complex__` is not available. (Contributed by Serhiy Storchaka in :issue:`20092`.) -* Added support of ``\N{name}`` escapes in :mod:`regular expressions `. +* Added support of ``\N{name}`` escapes in :mod:`regular expressions `:: + + >>> notice = 'Copyright ? 2019' + >>> copyright_year_pattern = re.compile(r'\N{copyright sign}\s*(\d{4})') + >>> int(copyright_year_pattern.search(notice).group(1)) + 2019 + (Contributed by Jonathan Eunice and Serhiy Storchaka in :issue:`30688`.) * Dict and dictviews are now iterable in reversed insertion order using @@ -343,10 +447,30 @@ Other Language Changes * Added new ``replace()`` method to the code type (:class:`types.CodeType`). (Contributed by Victor Stinner in :issue:`37032`.) -* For integers, the three-argument form of the :func:`pow` function now permits - the exponent to be negative in the case where the base is relatively prime to - the modulus. It then computes a modular inverse to the base when the exponent - is ``-1``, and a suitable power of that inverse for other negative exponents. +* For integers, the three-argument form of the :func:`pow` function now + permits the exponent to be negative in the case where the base is + relatively prime to the modulus. It then computes a modular inverse to + the base when the exponent is ``-1``, and a suitable power of that + inverse for other negative exponents. For example, to compute the + `modular multiplicative inverse + `_ of 38 + modulo 137, write:: + + >>> pow(38, -1, 137) + 119 + >>> 119 * 38 % 137 + 1 + + Modular inverses arise in the solution of `linear Diophantine + equations `_. + For example, to find integer solutions for ``4258? + 147? = 369``, + first rewrite as ``4258? ? 369 (mod 147)`` then solve: + + >>> x = 369 * pow(4258, -1, 147) % 147 + >>> y = (4258 * x - 369) // -147 + >>> 4258 * x + 147 * y + 369 + (Contributed by Mark Dickinson in :issue:`36027`.) * When dictionary comprehensions are evaluated, the key is now evaluated before @@ -576,7 +700,14 @@ Formerly, it only supported the 2-D case. Added new function, :func:`math.prod`, as analogous function to :func:`sum` that returns the product of a 'start' value (default: 1) times an iterable of -numbers. (Contributed by Pablo Galindo in :issue:`35606`) +numbers:: + + >>> prior = 0.8 + >>> likelihoods = [0.625, 0.84, 0.30] + >>> (link: http://math.prod) math.prod(likelihoods, start=prior) + 0.126 + +(Contributed by Pablo Galindo in :issue:`35606`) Added new function :func:`math.isqrt` for computing integer square roots. (Contributed by Mark Dickinson in :issue:`36887`.) @@ -1357,7 +1488,7 @@ Changes in the Python API * :func:`shutil.copyfile` default buffer size on Windows was changed from 16 KiB to 1 MiB. -* ``PyGC_Head`` struct is changed completely. All code touched the +* The ``PyGC_Head`` struct has changed completely. All code that touched the struct member should be rewritten. (See :issue:`33597`) * The ``PyInterpreterState`` struct has been moved into the "internal" diff --git a/Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst b/Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst new file mode 100644 index 000000000000..90fb7213ebde --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst @@ -0,0 +1 @@ +Beginning edits to Whatsnew 3.8 From webhook-mailer at python.org Mon Aug 5 18:22:18 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Mon, 05 Aug 2019 22:22:18 -0000 Subject: [Python-checkins] bpo-37759: First round of major edits to Whatsnew 3.8 (GH-15127) (GH-15139) Message-ID: https://github.com/python/cpython/commit/26f91db5ba487033994b396011518cfc80bf8401 commit: 26f91db5ba487033994b396011518cfc80bf8401 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Raymond Hettinger date: 2019-08-05T15:22:13-07:00 summary: bpo-37759: First round of major edits to Whatsnew 3.8 (GH-15127) (GH-15139) (cherry picked from commit 4f9ffc9d1a6a293563deaaaaf4a13331302219b4) Co-authored-by: Raymond Hettinger files: A Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst M Doc/whatsnew/3.8.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 967b3f2d68b9..9f7058274514 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -42,21 +42,25 @@ This saves the maintainer the effort of going through the Mercurial log when researching a change. -This article explains the new features in Python 3.8, compared to 3.7. +:Editor: Raymond Hettinger +This article explains the new features in Python 3.8, compared to 3.7. For full details, see the :ref:`changelog `. -.. note:: +Prerelease users should be aware that this document is currently in +draft form. It will be updated as Python 3.8 moves towards release, so +it's worth checking back even after reading earlier versions. Some +notable items not yet covered are: - Prerelease users should be aware that this document is currently in draft - form. It will be updated substantially as Python 3.8 moves towards release, - so it's worth checking back even after reading earlier versions. +* :pep:`578` - Runtime audit hooks for potentially sensitive operations +* ``python -m asyncio`` runs a natively async REPL - Some notable items not yet covered here: +.. testsetup:: - * :pep:`578` - Runtime audit hooks for potentially sensitive operations - * ``python -m asyncio`` runs a natively async REPL - * ... + from datetime import date + from math import cos, radians + import re + import math Summary -- Release highlights @@ -76,12 +80,43 @@ New Features Assignment expressions ---------------------- -There is new syntax (the "walrus operator", ``:=``) to assign values -to variables as part of an expression. Example:: +There is new syntax ``:=`` that assigns values to variables as part of a larger +expression. It is affectionately known as "walrus operator" due to +its resemblance to `the eyes and tusks of a walrus +`_. + +In this example, the assignment expression helps avoid calling +:func:`len` twice:: if (n := len(a)) > 10: print(f"List is too long ({n} elements, expected <= 10)") +A similar benefit arises during regular expression matching where +match objects are needed twice, once to test whether a match +occurred and another to extract a subgroup:: + + discount = 0.0 + if (mo := re.search(r'(\d+)% discount', advertisement)): + discount = float(mo.group(1)) / 100.0 + +The operator is also useful with while-loops that compute +a value to test loop termination and then need that same +value again in the body of the loop:: + + # Loop over fixed length blocks + while (block := f.read(256)) != '': + process(block) + +Another motivating use case arises in list comprehensions where +a value computed in a filtering condition is also needed in +the expression body:: + + [clean_name.title() for name in names + if (clean_name := normalize('NFC', name)) in allowed_names] + +Try to limit use of the walrus operator to clean cases that reduce +complexity and improve readability. + See :pep:`572` for a full description. (Contributed by Emily Morehouse in :issue:`35224`.) @@ -92,20 +127,69 @@ See :pep:`572` for a full description. Positional-only parameters -------------------------- -There is new syntax (``/``) to indicate that some function parameters -must be specified positionally (i.e., cannot be used as keyword -arguments). This is the same notation as shown by ``help()`` for -functions implemented in C (produced by Larry Hastings' "Argument -Clinic" tool). Example:: +There is a new function parameter syntax ``/`` to indicate that some +function parameters must be specified positionally and cannot be used as +keyword arguments. This is the same notation shown by ``help()`` for C +functions annotated with Larry Hastings' `Argument Clinic +`_ tool. + +In the following example, parameters *a* and *b* are positional-only, +while *c* or *d* can be positional or keyword, and *e* or *f* are +required to be keywords:: + + def f(a, b, /, c, d, *, e, f): + print(a, b, c, d, e, f) + +The following is a valid call:: + + f(10, 20, 30, d=40, e=50, f=60) + +However, these are invalid calls:: + + f(10, b=20, c=30, d=40, e=50, f=60) # b cannot be a keyword argument + f(10, 20, 30, 40, 50, f=60) # e must be a keyword argument + +One use case for this notation is that it allows pure Python functions +to fully emulate behaviors of existing C coded functions. For example, +the built-in :func:`pow` function does not accept keyword arguments:: def pow(x, y, z=None, /): - r = x**y - if z is not None: - r %= z - return r + "Emulate the built in pow() function" + r = x ** y + return r if z is None else r%z + +Another use case is to preclude keyword arguments when the parameter +name is not helpful. For example, the builtin :func:`len` function has +the signature ``len(obj, /)``. This precludes awkward calls such as:: -Now ``pow(2, 10)`` and ``pow(2, 10, 17)`` are valid calls, but -``pow(x=2, y=10)`` and ``pow(2, 10, z=17)`` are invalid. + len(obj='hello') # The "obj" keyword argument impairs readability + +A further benefit of marking a parameter as positional-only is that it +allows the parameter name to be changed in the future without risk of +breaking client code. For example, in the :mod:`statistics` module, the +parameter name *dist* may be changed in the future. This was made +possible with the following function specification:: + + def quantiles(dist, /, *, n=4, method='exclusive') + ... + +Since the parameters to the left of ``/`` are not exposed as possible +keywords, the parameters names remain available for use in ``**kwargs``:: + + >>> def f(a, b, /, **kwargs): + ... print(a, b, kwargs) + ... + >>> f(10, 20, a=1, b=2, c=3) # a and b are used in two ways + 10 20 {'a': 1, 'b': 2, 'c': 3} + +This greatly simplifies the implementation of functions and methods +that need to accept arbitrary keyword arguments. For example, here +is an except from code in the :mod:`collections` module:: + + class Counter(dict): + + def __init__(self, iterable=None, /, **kwds): + # Note "iterable" is a possible keyword argument See :pep:`570` for a full description. @@ -174,17 +258,31 @@ Android and Cygwin, whose cases are handled by the script); this change is backward incompatible on purpose. (Contributed by Victor Stinner in :issue:`36721`.) -f-strings now support = for quick and easy debugging ------------------------------------------------------ -Add ``=`` specifier to f-strings. ``f'{expr=}'`` expands -to the text of the expression, an equal sign, then the repr of the -evaluated expression. So:: +f-strings support ``=`` for self-documenting expressions and debugging +---------------------------------------------------------------------- + +Added an ``=`` specifier to :term:`f-string`\s. An f-string such as +``f'{expr=}'`` will expand to the text of the expression, an equal sign, +then the representation of the evaluated expression. For example: - x = 3 - print(f'{x*9 + 15=}') + >>> user = 'eric_idle' + >>> member_since = date(1975, 7, 31) + >>> f'{user=} {member_since=}' + "user='eric_idle' member_since=datetime.date(1975, 7, 31)" -Would print ``x*9 + 15=42``. +The usual :ref:`f-string format specifiers ` allow more +control over how the result of the expression is displayed:: + + >>> delta = date.today() - member_since + >>> f'{user=!s} {delta.days=:,d}' + 'user=eric_idle delta.days=16,075' + +The ``=`` specifier will display the whole expression so that +calculations can be shown:: + + >>> print(f'{theta=} {cos(radians(theta))=:.3f}') + theta=30 cos(radians(theta))=0.866 (Contributed by Eric V. Smith and Larry Hastings in :issue:`36817`.) @@ -295,7 +393,13 @@ Other Language Changes or :meth:`~object.__complex__` is not available. (Contributed by Serhiy Storchaka in :issue:`20092`.) -* Added support of ``\N{name}`` escapes in :mod:`regular expressions `. +* Added support of ``\N{name}`` escapes in :mod:`regular expressions `:: + + >>> notice = 'Copyright ? 2019' + >>> copyright_year_pattern = re.compile(r'\N{copyright sign}\s*(\d{4})') + >>> int(copyright_year_pattern.search(notice).group(1)) + 2019 + (Contributed by Jonathan Eunice and Serhiy Storchaka in :issue:`30688`.) * Dict and dictviews are now iterable in reversed insertion order using @@ -343,10 +447,30 @@ Other Language Changes * Added new ``replace()`` method to the code type (:class:`types.CodeType`). (Contributed by Victor Stinner in :issue:`37032`.) -* For integers, the three-argument form of the :func:`pow` function now permits - the exponent to be negative in the case where the base is relatively prime to - the modulus. It then computes a modular inverse to the base when the exponent - is ``-1``, and a suitable power of that inverse for other negative exponents. +* For integers, the three-argument form of the :func:`pow` function now + permits the exponent to be negative in the case where the base is + relatively prime to the modulus. It then computes a modular inverse to + the base when the exponent is ``-1``, and a suitable power of that + inverse for other negative exponents. For example, to compute the + `modular multiplicative inverse + `_ of 38 + modulo 137, write:: + + >>> pow(38, -1, 137) + 119 + >>> 119 * 38 % 137 + 1 + + Modular inverses arise in the solution of `linear Diophantine + equations `_. + For example, to find integer solutions for ``4258? + 147? = 369``, + first rewrite as ``4258? ? 369 (mod 147)`` then solve: + + >>> x = 369 * pow(4258, -1, 147) % 147 + >>> y = (4258 * x - 369) // -147 + >>> 4258 * x + 147 * y + 369 + (Contributed by Mark Dickinson in :issue:`36027`.) * When dictionary comprehensions are evaluated, the key is now evaluated before @@ -573,7 +697,14 @@ Formerly, it only supported the 2-D case. Added new function, :func:`math.prod`, as analogous function to :func:`sum` that returns the product of a 'start' value (default: 1) times an iterable of -numbers. (Contributed by Pablo Galindo in :issue:`35606`) +numbers:: + + >>> prior = 0.8 + >>> likelihoods = [0.625, 0.84, 0.30] + >>> (link: http://math.prod) math.prod(likelihoods, start=prior) + 0.126 + +(Contributed by Pablo Galindo in :issue:`35606`) Added new function :func:`math.isqrt` for computing integer square roots. (Contributed by Mark Dickinson in :issue:`36887`.) @@ -1354,7 +1485,7 @@ Changes in the Python API * :func:`shutil.copyfile` default buffer size on Windows was changed from 16 KiB to 1 MiB. -* ``PyGC_Head`` struct is changed completely. All code touched the +* The ``PyGC_Head`` struct has changed completely. All code that touched the struct member should be rewritten. (See :issue:`33597`) * The ``PyInterpreterState`` struct has been moved into the "internal" diff --git a/Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst b/Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst new file mode 100644 index 000000000000..90fb7213ebde --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst @@ -0,0 +1 @@ +Beginning edits to Whatsnew 3.8 From webhook-mailer at python.org Tue Aug 6 17:12:33 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 06 Aug 2019 21:12:33 -0000 Subject: [Python-checkins] Improve signal documentation (GH-14274) Message-ID: https://github.com/python/cpython/commit/cfebfef2def48095aa1f4c790a35e51818d67502 commit: cfebfef2def48095aa1f4c790a35e51818d67502 branch: master author: G?ry Ogam committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-06T14:12:22-07:00 summary: Improve signal documentation (GH-14274) * add a missing ``.. availability::`` reST explicit markup; * more consistent "see man page" sentences. files: M Doc/library/signal.rst diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst index 01200b4df880..8fecc2b7eed0 100644 --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -95,7 +95,7 @@ The variables defined in the :mod:`signal` module are: All the signal numbers are defined symbolically. For example, the hangup signal is defined as :const:`signal.SIGHUP`; the variable names are identical to the - names used in C programs, as found in ````. The Unix man page for + names used in C programs, as found in ````. The Unix man page for ':c:func:`signal`' lists the existing signals (on some systems this is :manpage:`signal(2)`, on others the list is in :manpage:`signal(7)`). Note that not all systems define the same set of signal names; only those names defined by @@ -193,10 +193,10 @@ The :mod:`signal` module defines the following functions: canceled (only one alarm can be scheduled at any time). The returned value is then the number of seconds before any previously set alarm was to have been delivered. If *time* is zero, no alarm is scheduled, and any scheduled alarm is - canceled. If the return value is zero, no alarm is currently scheduled. (See - the Unix man page :manpage:`alarm(2)`.) + canceled. If the return value is zero, no alarm is currently scheduled. - .. availability:: Unix. + .. availability:: Unix. See the man page :manpage:`alarm(2)` for further + information. .. function:: getsignal(signalnum) @@ -231,8 +231,10 @@ The :mod:`signal` module defines the following functions: .. function:: pause() Cause the process to sleep until a signal is received; the appropriate handler - will then be called. Returns nothing. Not on Windows. (See the Unix man page - :manpage:`signal(2)`.) + will then be called. Returns nothing. + + .. availability:: Unix. See the man page :manpage:`signal(2)` for further + information. See also :func:`sigwait`, :func:`sigwaitinfo`, :func:`sigtimedwait` and :func:`sigpending`. @@ -262,8 +264,8 @@ The :mod:`signal` module defines the following functions: If *signalnum* is 0, then no signal is sent, but error checking is still performed; this can be used to check if the target thread is still running. - .. availability:: Unix (see the man page :manpage:`pthread_kill(3)` for further - information). + .. availability:: Unix. See the man page :manpage:`pthread_kill(3)` for further + information. See also :func:`os.kill`. @@ -293,7 +295,7 @@ The :mod:`signal` module defines the following functions: For example, ``signal.pthread_sigmask(signal.SIG_BLOCK, [])`` reads the signal mask of the calling thread. - .. availability:: Unix. See the man page :manpage:`sigprocmask(3)` and + .. availability:: Unix. See the man page :manpage:`sigprocmask(3)` and :manpage:`pthread_sigmask(3)` for further information. See also :func:`pause`, :func:`sigpending` and :func:`sigwait`. @@ -380,8 +382,8 @@ The :mod:`signal` module defines the following functions: calls will be restarted when interrupted by signal *signalnum*, otherwise system calls will be interrupted. Returns nothing. - .. availability:: Unix (see the man page :manpage:`siginterrupt(3)` - for further information). + .. availability:: Unix. See the man page :manpage:`siginterrupt(3)` + for further information. Note that installing a signal handler with :func:`signal` will reset the restart behaviour to interruptible by implicitly calling @@ -394,7 +396,7 @@ The :mod:`signal` module defines the following functions: be a callable Python object taking two arguments (see below), or one of the special values :const:`signal.SIG_IGN` or :const:`signal.SIG_DFL`. The previous signal handler will be returned (see the description of :func:`getsignal` - above). (See the Unix man page :manpage:`signal(2)`.) + above). (See the Unix man page :manpage:`signal(2)` for further information.) When threads are enabled, this function can only be called from the main thread; attempting to call it from other threads will cause a :exc:`ValueError` @@ -420,8 +422,8 @@ The :mod:`signal` module defines the following functions: thread (i.e., the signals which have been raised while blocked). Return the set of the pending signals. - .. availability:: Unix (see the man page :manpage:`sigpending(2)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigpending(2)` for further + information. See also :func:`pause`, :func:`pthread_sigmask` and :func:`sigwait`. @@ -434,8 +436,8 @@ The :mod:`signal` module defines the following functions: signals specified in the signal set *sigset*. The function accepts the signal (removes it from the pending list of signals), and returns the signal number. - .. availability:: Unix (see the man page :manpage:`sigwait(3)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigwait(3)` for further + information. See also :func:`pause`, :func:`pthread_sigmask`, :func:`sigpending`, :func:`sigwaitinfo` and :func:`sigtimedwait`. @@ -459,8 +461,8 @@ The :mod:`signal` module defines the following functions: :attr:`si_errno`, :attr:`si_pid`, :attr:`si_uid`, :attr:`si_status`, :attr:`si_band`. - .. availability:: Unix (see the man page :manpage:`sigwaitinfo(2)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigwaitinfo(2)` for further + information. See also :func:`pause`, :func:`sigwait` and :func:`sigtimedwait`. @@ -478,8 +480,8 @@ The :mod:`signal` module defines the following functions: specifying a timeout. If *timeout* is specified as :const:`0`, a poll is performed. Returns :const:`None` if a timeout occurs. - .. availability:: Unix (see the man page :manpage:`sigtimedwait(2)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigtimedwait(2)` for further + information. See also :func:`pause`, :func:`sigwait` and :func:`sigwaitinfo`. From webhook-mailer at python.org Tue Aug 6 17:53:32 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 06 Aug 2019 21:53:32 -0000 Subject: [Python-checkins] Improve signal documentation (GH-14274) Message-ID: https://github.com/python/cpython/commit/ef0b81927ab3bf2b838964e78c7b8a91c9a61e12 commit: ef0b81927ab3bf2b838964e78c7b8a91c9a61e12 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-06T14:53:27-07:00 summary: Improve signal documentation (GH-14274) * add a missing ``.. availability::`` reST explicit markup; * more consistent "see man page" sentences. (cherry picked from commit cfebfef2def48095aa1f4c790a35e51818d67502) Co-authored-by: G?ry Ogam files: M Doc/library/signal.rst diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst index 01200b4df880..8fecc2b7eed0 100644 --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -95,7 +95,7 @@ The variables defined in the :mod:`signal` module are: All the signal numbers are defined symbolically. For example, the hangup signal is defined as :const:`signal.SIGHUP`; the variable names are identical to the - names used in C programs, as found in ````. The Unix man page for + names used in C programs, as found in ````. The Unix man page for ':c:func:`signal`' lists the existing signals (on some systems this is :manpage:`signal(2)`, on others the list is in :manpage:`signal(7)`). Note that not all systems define the same set of signal names; only those names defined by @@ -193,10 +193,10 @@ The :mod:`signal` module defines the following functions: canceled (only one alarm can be scheduled at any time). The returned value is then the number of seconds before any previously set alarm was to have been delivered. If *time* is zero, no alarm is scheduled, and any scheduled alarm is - canceled. If the return value is zero, no alarm is currently scheduled. (See - the Unix man page :manpage:`alarm(2)`.) + canceled. If the return value is zero, no alarm is currently scheduled. - .. availability:: Unix. + .. availability:: Unix. See the man page :manpage:`alarm(2)` for further + information. .. function:: getsignal(signalnum) @@ -231,8 +231,10 @@ The :mod:`signal` module defines the following functions: .. function:: pause() Cause the process to sleep until a signal is received; the appropriate handler - will then be called. Returns nothing. Not on Windows. (See the Unix man page - :manpage:`signal(2)`.) + will then be called. Returns nothing. + + .. availability:: Unix. See the man page :manpage:`signal(2)` for further + information. See also :func:`sigwait`, :func:`sigwaitinfo`, :func:`sigtimedwait` and :func:`sigpending`. @@ -262,8 +264,8 @@ The :mod:`signal` module defines the following functions: If *signalnum* is 0, then no signal is sent, but error checking is still performed; this can be used to check if the target thread is still running. - .. availability:: Unix (see the man page :manpage:`pthread_kill(3)` for further - information). + .. availability:: Unix. See the man page :manpage:`pthread_kill(3)` for further + information. See also :func:`os.kill`. @@ -293,7 +295,7 @@ The :mod:`signal` module defines the following functions: For example, ``signal.pthread_sigmask(signal.SIG_BLOCK, [])`` reads the signal mask of the calling thread. - .. availability:: Unix. See the man page :manpage:`sigprocmask(3)` and + .. availability:: Unix. See the man page :manpage:`sigprocmask(3)` and :manpage:`pthread_sigmask(3)` for further information. See also :func:`pause`, :func:`sigpending` and :func:`sigwait`. @@ -380,8 +382,8 @@ The :mod:`signal` module defines the following functions: calls will be restarted when interrupted by signal *signalnum*, otherwise system calls will be interrupted. Returns nothing. - .. availability:: Unix (see the man page :manpage:`siginterrupt(3)` - for further information). + .. availability:: Unix. See the man page :manpage:`siginterrupt(3)` + for further information. Note that installing a signal handler with :func:`signal` will reset the restart behaviour to interruptible by implicitly calling @@ -394,7 +396,7 @@ The :mod:`signal` module defines the following functions: be a callable Python object taking two arguments (see below), or one of the special values :const:`signal.SIG_IGN` or :const:`signal.SIG_DFL`. The previous signal handler will be returned (see the description of :func:`getsignal` - above). (See the Unix man page :manpage:`signal(2)`.) + above). (See the Unix man page :manpage:`signal(2)` for further information.) When threads are enabled, this function can only be called from the main thread; attempting to call it from other threads will cause a :exc:`ValueError` @@ -420,8 +422,8 @@ The :mod:`signal` module defines the following functions: thread (i.e., the signals which have been raised while blocked). Return the set of the pending signals. - .. availability:: Unix (see the man page :manpage:`sigpending(2)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigpending(2)` for further + information. See also :func:`pause`, :func:`pthread_sigmask` and :func:`sigwait`. @@ -434,8 +436,8 @@ The :mod:`signal` module defines the following functions: signals specified in the signal set *sigset*. The function accepts the signal (removes it from the pending list of signals), and returns the signal number. - .. availability:: Unix (see the man page :manpage:`sigwait(3)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigwait(3)` for further + information. See also :func:`pause`, :func:`pthread_sigmask`, :func:`sigpending`, :func:`sigwaitinfo` and :func:`sigtimedwait`. @@ -459,8 +461,8 @@ The :mod:`signal` module defines the following functions: :attr:`si_errno`, :attr:`si_pid`, :attr:`si_uid`, :attr:`si_status`, :attr:`si_band`. - .. availability:: Unix (see the man page :manpage:`sigwaitinfo(2)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigwaitinfo(2)` for further + information. See also :func:`pause`, :func:`sigwait` and :func:`sigtimedwait`. @@ -478,8 +480,8 @@ The :mod:`signal` module defines the following functions: specifying a timeout. If *timeout* is specified as :const:`0`, a poll is performed. Returns :const:`None` if a timeout occurs. - .. availability:: Unix (see the man page :manpage:`sigtimedwait(2)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigtimedwait(2)` for further + information. See also :func:`pause`, :func:`sigwait` and :func:`sigwaitinfo`. From webhook-mailer at python.org Tue Aug 6 17:55:52 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 06 Aug 2019 21:55:52 -0000 Subject: [Python-checkins] Improve signal documentation (GH-14274) Message-ID: https://github.com/python/cpython/commit/18343aba78dea0c23abcb6e615e70f89764ae9a4 commit: 18343aba78dea0c23abcb6e615e70f89764ae9a4 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-06T14:55:48-07:00 summary: Improve signal documentation (GH-14274) * add a missing ``.. availability::`` reST explicit markup; * more consistent "see man page" sentences. (cherry picked from commit cfebfef2def48095aa1f4c790a35e51818d67502) Co-authored-by: G?ry Ogam files: M Doc/library/signal.rst diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst index b6716eea95fb..75008118b7ca 100644 --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -95,7 +95,7 @@ The variables defined in the :mod:`signal` module are: All the signal numbers are defined symbolically. For example, the hangup signal is defined as :const:`signal.SIGHUP`; the variable names are identical to the - names used in C programs, as found in ````. The Unix man page for + names used in C programs, as found in ````. The Unix man page for ':c:func:`signal`' lists the existing signals (on some systems this is :manpage:`signal(2)`, on others the list is in :manpage:`signal(7)`). Note that not all systems define the same set of signal names; only those names defined by @@ -193,10 +193,10 @@ The :mod:`signal` module defines the following functions: canceled (only one alarm can be scheduled at any time). The returned value is then the number of seconds before any previously set alarm was to have been delivered. If *time* is zero, no alarm is scheduled, and any scheduled alarm is - canceled. If the return value is zero, no alarm is currently scheduled. (See - the Unix man page :manpage:`alarm(2)`.) + canceled. If the return value is zero, no alarm is currently scheduled. - .. availability:: Unix. + .. availability:: Unix. See the man page :manpage:`alarm(2)` for further + information. .. function:: getsignal(signalnum) @@ -213,8 +213,10 @@ The :mod:`signal` module defines the following functions: .. function:: pause() Cause the process to sleep until a signal is received; the appropriate handler - will then be called. Returns nothing. Not on Windows. (See the Unix man page - :manpage:`signal(2)`.) + will then be called. Returns nothing. + + .. availability:: Unix. See the man page :manpage:`signal(2)` for further + information. See also :func:`sigwait`, :func:`sigwaitinfo`, :func:`sigtimedwait` and :func:`sigpending`. @@ -237,8 +239,8 @@ The :mod:`signal` module defines the following functions: If *signalnum* is 0, then no signal is sent, but error checking is still performed; this can be used to check if the target thread is still running. - .. availability:: Unix (see the man page :manpage:`pthread_kill(3)` for further - information). + .. availability:: Unix. See the man page :manpage:`pthread_kill(3)` for further + information. See also :func:`os.kill`. @@ -268,7 +270,7 @@ The :mod:`signal` module defines the following functions: For example, ``signal.pthread_sigmask(signal.SIG_BLOCK, [])`` reads the signal mask of the calling thread. - .. availability:: Unix. See the man page :manpage:`sigprocmask(3)` and + .. availability:: Unix. See the man page :manpage:`sigprocmask(3)` and :manpage:`pthread_sigmask(3)` for further information. See also :func:`pause`, :func:`sigpending` and :func:`sigwait`. @@ -355,8 +357,8 @@ The :mod:`signal` module defines the following functions: calls will be restarted when interrupted by signal *signalnum*, otherwise system calls will be interrupted. Returns nothing. - .. availability:: Unix (see the man page :manpage:`siginterrupt(3)` - for further information). + .. availability:: Unix. See the man page :manpage:`siginterrupt(3)` + for further information. Note that installing a signal handler with :func:`signal` will reset the restart behaviour to interruptible by implicitly calling @@ -369,7 +371,7 @@ The :mod:`signal` module defines the following functions: be a callable Python object taking two arguments (see below), or one of the special values :const:`signal.SIG_IGN` or :const:`signal.SIG_DFL`. The previous signal handler will be returned (see the description of :func:`getsignal` - above). (See the Unix man page :manpage:`signal(2)`.) + above). (See the Unix man page :manpage:`signal(2)` for further information.) When threads are enabled, this function can only be called from the main thread; attempting to call it from other threads will cause a :exc:`ValueError` @@ -395,8 +397,8 @@ The :mod:`signal` module defines the following functions: thread (i.e., the signals which have been raised while blocked). Return the set of the pending signals. - .. availability:: Unix (see the man page :manpage:`sigpending(2)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigpending(2)` for further + information. See also :func:`pause`, :func:`pthread_sigmask` and :func:`sigwait`. @@ -409,8 +411,8 @@ The :mod:`signal` module defines the following functions: signals specified in the signal set *sigset*. The function accepts the signal (removes it from the pending list of signals), and returns the signal number. - .. availability:: Unix (see the man page :manpage:`sigwait(3)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigwait(3)` for further + information. See also :func:`pause`, :func:`pthread_sigmask`, :func:`sigpending`, :func:`sigwaitinfo` and :func:`sigtimedwait`. @@ -434,8 +436,8 @@ The :mod:`signal` module defines the following functions: :attr:`si_errno`, :attr:`si_pid`, :attr:`si_uid`, :attr:`si_status`, :attr:`si_band`. - .. availability:: Unix (see the man page :manpage:`sigwaitinfo(2)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigwaitinfo(2)` for further + information. See also :func:`pause`, :func:`sigwait` and :func:`sigtimedwait`. @@ -453,8 +455,8 @@ The :mod:`signal` module defines the following functions: specifying a timeout. If *timeout* is specified as :const:`0`, a poll is performed. Returns :const:`None` if a timeout occurs. - .. availability:: Unix (see the man page :manpage:`sigtimedwait(2)` for further - information). + .. availability:: Unix. See the man page :manpage:`sigtimedwait(2)` for further + information. See also :func:`pause`, :func:`sigwait` and :func:`sigwaitinfo`. From webhook-mailer at python.org Tue Aug 6 19:59:10 2019 From: webhook-mailer at python.org (Jason R. Coombs) Date: Tue, 06 Aug 2019 23:59:10 -0000 Subject: [Python-checkins] Make importlib.metadata a simple module (#15153) Message-ID: https://github.com/python/cpython/commit/3a5c433fce7312748859290b9d8db5b6507660f9 commit: 3a5c433fce7312748859290b9d8db5b6507660f9 branch: master author: Barry Warsaw committer: Jason R. Coombs date: 2019-08-06T19:59:07-04:00 summary: Make importlib.metadata a simple module (#15153) files: A Lib/importlib/metadata.py D Lib/importlib/metadata/__init__.py diff --git a/Lib/importlib/metadata/__init__.py b/Lib/importlib/metadata.py similarity index 100% rename from Lib/importlib/metadata/__init__.py rename to Lib/importlib/metadata.py From webhook-mailer at python.org Tue Aug 6 20:38:35 2019 From: webhook-mailer at python.org (Inada Naoki) Date: Wed, 07 Aug 2019 00:38:35 -0000 Subject: [Python-checkins] bpo-34488: optimize BytesIO.writelines() (GH-8904) Message-ID: https://github.com/python/cpython/commit/3e41f3cabb661824a1a197116f7f5ead64eb6ced commit: 3e41f3cabb661824a1a197116f7f5ead64eb6ced branch: master author: Sergey Fedoseev committer: Inada Naoki date: 2019-08-07T09:38:31+09:00 summary: bpo-34488: optimize BytesIO.writelines() (GH-8904) Avoid the creation of unused int object for each line. files: A Misc/NEWS.d/next/Library/2019-08-06-21-30-58.bpo-34488.OqxVo8.rst M Modules/_io/bytesio.c diff --git a/Misc/NEWS.d/next/Library/2019-08-06-21-30-58.bpo-34488.OqxVo8.rst b/Misc/NEWS.d/next/Library/2019-08-06-21-30-58.bpo-34488.OqxVo8.rst new file mode 100644 index 000000000000..1e7e5a6ba0bc --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-06-21-30-58.bpo-34488.OqxVo8.rst @@ -0,0 +1,2 @@ +:meth:`writelines` method of :class:`io.BytesIO` is now slightly faster +when many small lines are passed. Patch by Sergey Fedoseev. diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c index 19e1ed8441e3..793ce920004e 100644 --- a/Modules/_io/bytesio.c +++ b/Modules/_io/bytesio.c @@ -31,17 +31,34 @@ typedef struct { * exports > 0. Py_REFCNT(buf) == 1, any modifications are forbidden. */ +static int +check_closed(bytesio *self) +{ + if (self->buf == NULL) { + PyErr_SetString(PyExc_ValueError, "I/O operation on closed file."); + return 1; + } + return 0; +} + +static int +check_exports(bytesio *self) +{ + if (self->exports > 0) { + PyErr_SetString(PyExc_BufferError, + "Existing exports of data: object cannot be re-sized"); + return 1; + } + return 0; +} + #define CHECK_CLOSED(self) \ - if ((self)->buf == NULL) { \ - PyErr_SetString(PyExc_ValueError, \ - "I/O operation on closed file."); \ + if (check_closed(self)) { \ return NULL; \ } #define CHECK_EXPORTS(self) \ - if ((self)->exports > 0) { \ - PyErr_SetString(PyExc_BufferError, \ - "Existing exports of data: object cannot be re-sized"); \ + if (check_exports(self)) { \ return NULL; \ } @@ -156,23 +173,41 @@ resize_buffer(bytesio *self, size_t size) } /* Internal routine for writing a string of bytes to the buffer of a BytesIO - object. Returns the number of bytes written, or -1 on error. */ -static Py_ssize_t -write_bytes(bytesio *self, const char *bytes, Py_ssize_t len) + object. Returns the number of bytes written, or -1 on error. + Inlining is disabled because it's significantly decreases performance + of writelines() in PGO build. */ +_Py_NO_INLINE static Py_ssize_t +write_bytes(bytesio *self, PyObject *b) { - size_t endpos; - assert(self->buf != NULL); - assert(self->pos >= 0); - assert(len >= 0); + if (check_closed(self)) { + return -1; + } + if (check_exports(self)) { + return -1; + } - endpos = (size_t)self->pos + len; + Py_buffer buf; + if (PyObject_GetBuffer(b, &buf, PyBUF_CONTIG_RO) < 0) { + return -1; + } + Py_ssize_t len = buf.len; + if (len == 0) { + goto done; + } + + assert(self->pos >= 0); + size_t endpos = (size_t)self->pos + len; if (endpos > (size_t)PyBytes_GET_SIZE(self->buf)) { - if (resize_buffer(self, endpos) < 0) - return -1; + if (resize_buffer(self, endpos) < 0) { + len = -1; + goto done; + } } else if (SHARED_BUF(self)) { - if (unshare_buffer(self, Py_MAX(endpos, (size_t)self->string_size)) < 0) - return -1; + if (unshare_buffer(self, Py_MAX(endpos, (size_t)self->string_size)) < 0) { + len = -1; + goto done; + } } if (self->pos > self->string_size) { @@ -190,7 +225,7 @@ write_bytes(bytesio *self, const char *bytes, Py_ssize_t len) /* Copy the data to the internal buffer, overwriting some of the existing data if self->pos < self->string_size. */ - memcpy(PyBytes_AS_STRING(self->buf) + self->pos, bytes, len); + memcpy(PyBytes_AS_STRING(self->buf) + self->pos, buf.buf, len); self->pos = endpos; /* Set the new length of the internal string if it has changed. */ @@ -198,6 +233,8 @@ write_bytes(bytesio *self, const char *bytes, Py_ssize_t len) self->string_size = endpos; } + done: + PyBuffer_Release(&buf); return len; } @@ -669,19 +706,7 @@ static PyObject * _io_BytesIO_write(bytesio *self, PyObject *b) /*[clinic end generated code: output=53316d99800a0b95 input=f5ec7c8c64ed720a]*/ { - Py_ssize_t n = 0; - Py_buffer buf; - - CHECK_CLOSED(self); - CHECK_EXPORTS(self); - - if (PyObject_GetBuffer(b, &buf, PyBUF_CONTIG_RO) < 0) - return NULL; - - if (buf.len != 0) - n = write_bytes(self, buf.buf, buf.len); - - PyBuffer_Release(&buf); + Py_ssize_t n = write_bytes(self, b); return n >= 0 ? PyLong_FromSsize_t(n) : NULL; } @@ -702,7 +727,6 @@ _io_BytesIO_writelines(bytesio *self, PyObject *lines) /*[clinic end generated code: output=7f33aa3271c91752 input=e972539176fc8fc1]*/ { PyObject *it, *item; - PyObject *ret; CHECK_CLOSED(self); @@ -711,13 +735,12 @@ _io_BytesIO_writelines(bytesio *self, PyObject *lines) return NULL; while ((item = PyIter_Next(it)) != NULL) { - ret = _io_BytesIO_write(self, item); + Py_ssize_t ret = write_bytes(self, item); Py_DECREF(item); - if (ret == NULL) { + if (ret < 0) { Py_DECREF(it); return NULL; } - Py_DECREF(ret); } Py_DECREF(it); From webhook-mailer at python.org Tue Aug 6 20:56:26 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Wed, 07 Aug 2019 00:56:26 -0000 Subject: [Python-checkins] bpo-37646: Document that eval() cannot access nested scopes (GH-15117) Message-ID: https://github.com/python/cpython/commit/610a4823cc0a3c2380ad0dfe64ae483ced4e5304 commit: 610a4823cc0a3c2380ad0dfe64ae483ced4e5304 branch: master author: Raymond Hettinger committer: GitHub date: 2019-08-06T17:56:22-07:00 summary: bpo-37646: Document that eval() cannot access nested scopes (GH-15117) files: M Doc/library/functions.rst diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index e146f5a95acc..c225f3dee921 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -465,12 +465,16 @@ are always available. They are listed here in alphabetical order. dictionaries as global and local namespace. If the *globals* dictionary is present and does not contain a value for the key ``__builtins__``, a reference to the dictionary of the built-in module :mod:`builtins` is - inserted under that key before *expression* is parsed. - This means that *expression* normally has full - access to the standard :mod:`builtins` module and restricted environments are - propagated. If the *locals* dictionary is omitted it defaults to the *globals* - dictionary. If both dictionaries are omitted, the expression is executed in the - environment where :func:`eval` is called. The return value is the result of + inserted under that key before *expression* is parsed. This means that + *expression* normally has full access to the standard :mod:`builtins` + module and restricted environments are propagated. If the *locals* + dictionary is omitted it defaults to the *globals* dictionary. If both + dictionaries are omitted, the expression is executed with the *globals* and + *locals* in the environment where :func:`eval` is called. Note, *eval()* + does not have access to the :term:`nested scope`\s (non-locals) in the + enclosing environment. + + The return value is the result of the evaluated expression. Syntax errors are reported as exceptions. Example: >>> x = 1 From webhook-mailer at python.org Tue Aug 6 21:08:03 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Wed, 07 Aug 2019 01:08:03 -0000 Subject: [Python-checkins] bpo-37646: Document that eval() cannot access nested scopes (GH-15117) (GH-15155) Message-ID: https://github.com/python/cpython/commit/9341dcb4b9520ab92df10d4256e93a50e1e7d19f commit: 9341dcb4b9520ab92df10d4256e93a50e1e7d19f branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Raymond Hettinger date: 2019-08-06T18:07:59-07:00 summary: bpo-37646: Document that eval() cannot access nested scopes (GH-15117) (GH-15155) (cherry picked from commit 610a4823cc0a3c2380ad0dfe64ae483ced4e5304) Co-authored-by: Raymond Hettinger files: M Doc/library/functions.rst diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index e146f5a95acc..c225f3dee921 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -465,12 +465,16 @@ are always available. They are listed here in alphabetical order. dictionaries as global and local namespace. If the *globals* dictionary is present and does not contain a value for the key ``__builtins__``, a reference to the dictionary of the built-in module :mod:`builtins` is - inserted under that key before *expression* is parsed. - This means that *expression* normally has full - access to the standard :mod:`builtins` module and restricted environments are - propagated. If the *locals* dictionary is omitted it defaults to the *globals* - dictionary. If both dictionaries are omitted, the expression is executed in the - environment where :func:`eval` is called. The return value is the result of + inserted under that key before *expression* is parsed. This means that + *expression* normally has full access to the standard :mod:`builtins` + module and restricted environments are propagated. If the *locals* + dictionary is omitted it defaults to the *globals* dictionary. If both + dictionaries are omitted, the expression is executed with the *globals* and + *locals* in the environment where :func:`eval` is called. Note, *eval()* + does not have access to the :term:`nested scope`\s (non-locals) in the + enclosing environment. + + The return value is the result of the evaluated expression. Syntax errors are reported as exceptions. Example: >>> x = 1 From webhook-mailer at python.org Tue Aug 6 21:38:26 2019 From: webhook-mailer at python.org (Jason R. Coombs) Date: Wed, 07 Aug 2019 01:38:26 -0000 Subject: [Python-checkins] Make importlib.metadata a simple module (GH-15153) (GH-15154) Message-ID: https://github.com/python/cpython/commit/e780d2f1f5e621adf5a2a8b39cb266946bcc0a18 commit: e780d2f1f5e621adf5a2a8b39cb266946bcc0a18 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Jason R. Coombs date: 2019-08-06T21:38:22-04:00 summary: Make importlib.metadata a simple module (GH-15153) (GH-15154) (cherry picked from commit 3a5c433fce7312748859290b9d8db5b6507660f9) Co-authored-by: Barry Warsaw files: A Lib/importlib/metadata.py D Lib/importlib/metadata/__init__.py diff --git a/Lib/importlib/metadata/__init__.py b/Lib/importlib/metadata.py similarity index 100% rename from Lib/importlib/metadata/__init__.py rename to Lib/importlib/metadata.py From webhook-mailer at python.org Wed Aug 7 00:37:16 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 07 Aug 2019 04:37:16 -0000 Subject: [Python-checkins] bpo-37004: Documented asymmetry of string arguments in difflib.SequenceMatcher for ratio method (GH-13482) Message-ID: https://github.com/python/cpython/commit/e9cbcd0018abd2a5f2348c45d5c9c4265c4f42dc commit: e9cbcd0018abd2a5f2348c45d5c9c4265c4f42dc branch: master author: sweeneyde <36520290+sweeneyde at users.noreply.github.com> committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-06T21:37:08-07:00 summary: bpo-37004: Documented asymmetry of string arguments in difflib.SequenceMatcher for ratio method (GH-13482) https://bugs.python.org/issue37004 files: A Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst M Doc/library/difflib.rst diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst index f044cb2d6e0a..e245ab81cfb9 100644 --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -543,6 +543,16 @@ The :class:`SequenceMatcher` class has this constructor: to try :meth:`quick_ratio` or :meth:`real_quick_ratio` first to get an upper bound. + .. note:: + + Caution: The result of a :meth:`ratio` call may depend on the order of + the arguments. For instance:: + + >>> SequenceMatcher(None, 'tide', 'diet').ratio() + 0.25 + >>> SequenceMatcher(None, 'diet', 'tide').ratio() + 0.5 + .. method:: quick_ratio() diff --git a/Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst b/Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst new file mode 100644 index 000000000000..dfc8b7ed74ca --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst @@ -0,0 +1 @@ +In the documentation for difflib, a note was added explicitly warning that the results of SequenceMatcher's ratio method may depend on the order of the input strings. \ No newline at end of file From webhook-mailer at python.org Wed Aug 7 01:02:27 2019 From: webhook-mailer at python.org (Carol Willing) Date: Wed, 07 Aug 2019 05:02:27 -0000 Subject: [Python-checkins] Update pickle.rst (GH-14128) Message-ID: https://github.com/python/cpython/commit/362f5350eb5e2c7bfb0b0a8c306a2e128c3aee93 commit: 362f5350eb5e2c7bfb0b0a8c306a2e128c3aee93 branch: master author: G?ry Ogam committer: Carol Willing date: 2019-08-06T22:02:23-07:00 summary: Update pickle.rst (GH-14128) * Edits for readability and grammar files: M Doc/library/pickle.rst diff --git a/Doc/library/pickle.rst b/Doc/library/pickle.rst index e6025aeaf476..09c9c86abbba 100644 --- a/Doc/library/pickle.rst +++ b/Doc/library/pickle.rst @@ -197,8 +197,9 @@ process more convenient: .. function:: dump(obj, file, protocol=None, \*, fix_imports=True, buffer_callback=None) - Write a pickled representation of *obj* to the open :term:`file object` *file*. - This is equivalent to ``Pickler(file, protocol).dump(obj)``. + Write the pickled representation of the object *obj* to the open + :term:`file object` *file*. This is equivalent to + ``Pickler(file, protocol).dump(obj)``. Arguments *file*, *protocol*, *fix_imports* and *buffer_callback* have the same meaning as in the :class:`Pickler` constructor. @@ -208,7 +209,7 @@ process more convenient: .. function:: dumps(obj, protocol=None, \*, fix_imports=True, buffer_callback=None) - Return the pickled representation of the object as a :class:`bytes` object, + Return the pickled representation of the object *obj* as a :class:`bytes` object, instead of writing it to a file. Arguments *protocol*, *fix_imports* and *buffer_callback* have the same @@ -219,13 +220,13 @@ process more convenient: .. function:: load(file, \*, fix_imports=True, encoding="ASCII", errors="strict", buffers=None) - Read a pickled object representation from the open :term:`file object` + Read the pickled representation of an object from the open :term:`file object` *file* and return the reconstituted object hierarchy specified therein. This is equivalent to ``Unpickler(file).load()``. The protocol version of the pickle is detected automatically, so no - protocol argument is needed. Bytes past the pickled object's - representation are ignored. + protocol argument is needed. Bytes past the pickled representation + of the object are ignored. Arguments *file*, *fix_imports*, *encoding*, *errors*, *strict* and *buffers* have the same meaning as in the :class:`Unpickler` constructor. @@ -235,12 +236,12 @@ process more convenient: .. function:: loads(bytes_object, \*, fix_imports=True, encoding="ASCII", errors="strict", buffers=None) - Read a pickled object hierarchy from a :class:`bytes` object and return the - reconstituted object hierarchy specified therein. + Return the reconstituted object hierarchy of the pickled representation + *bytes_object* of an object. The protocol version of the pickle is detected automatically, so no - protocol argument is needed. Bytes past the pickled object's - representation are ignored. + protocol argument is needed. Bytes past the pickled representation + of the object are ignored. Arguments *file*, *fix_imports*, *encoding*, *errors*, *strict* and *buffers* have the same meaning as in the :class:`Unpickler` constructor. @@ -311,7 +312,7 @@ The :mod:`pickle` module exports three classes, :class:`Pickler`, .. method:: dump(obj) - Write a pickled representation of *obj* to the open file object given in + Write the pickled representation of *obj* to the open file object given in the constructor. .. method:: persistent_id(obj) @@ -412,9 +413,10 @@ The :mod:`pickle` module exports three classes, :class:`Pickler`, .. method:: load() - Read a pickled object representation from the open file object given in - the constructor, and return the reconstituted object hierarchy specified - therein. Bytes past the pickled object's representation are ignored. + Read the pickled representation of an object from the open file object + given in the constructor, and return the reconstituted object hierarchy + specified therein. Bytes past the pickled representation of the object + are ignored. .. method:: persistent_load(pid) @@ -717,13 +719,13 @@ alphanumeric characters (for protocol 0) [#]_ or just an arbitrary object (for any newer protocol). The resolution of such persistent IDs is not defined by the :mod:`pickle` -module; it will delegate this resolution to the user defined methods on the +module; it will delegate this resolution to the user-defined methods on the pickler and unpickler, :meth:`~Pickler.persistent_id` and :meth:`~Unpickler.persistent_load` respectively. -To pickle objects that have an external persistent id, the pickler must have a +To pickle objects that have an external persistent ID, the pickler must have a custom :meth:`~Pickler.persistent_id` method that takes an object as an -argument and returns either ``None`` or the persistent id for that object. +argument and returns either ``None`` or the persistent ID for that object. When ``None`` is returned, the pickler simply pickles the object as normal. When a persistent ID string is returned, the pickler will pickle that object, along with a marker so that the unpickler will recognize it as a persistent ID. From webhook-mailer at python.org Wed Aug 7 11:39:37 2019 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Wed, 07 Aug 2019 15:39:37 -0000 Subject: [Python-checkins] bpo-37004: Documented asymmetry of string arguments in difflib.SequenceMatcher for ratio method (GH-13482) (#15157) Message-ID: https://github.com/python/cpython/commit/1a3a40c1cb582e436d568009fae2b06c0b1978ed commit: 1a3a40c1cb582e436d568009fae2b06c0b1978ed branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Terry Jan Reedy date: 2019-08-07T11:39:14-04:00 summary: bpo-37004: Documented asymmetry of string arguments in difflib.SequenceMatcher for ratio method (GH-13482) (#15157) https://bugs.python.org/issue37004 (cherry picked from commit e9cbcd0018abd2a5f2348c45d5c9c4265c4f42dc) Co-authored-by: sweeneyde <36520290+sweeneyde at users.noreply.github.com> files: A Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst M Doc/library/difflib.rst diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst index f044cb2d6e0a..e245ab81cfb9 100644 --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -543,6 +543,16 @@ The :class:`SequenceMatcher` class has this constructor: to try :meth:`quick_ratio` or :meth:`real_quick_ratio` first to get an upper bound. + .. note:: + + Caution: The result of a :meth:`ratio` call may depend on the order of + the arguments. For instance:: + + >>> SequenceMatcher(None, 'tide', 'diet').ratio() + 0.25 + >>> SequenceMatcher(None, 'diet', 'tide').ratio() + 0.5 + .. method:: quick_ratio() diff --git a/Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst b/Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst new file mode 100644 index 000000000000..dfc8b7ed74ca --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst @@ -0,0 +1 @@ +In the documentation for difflib, a note was added explicitly warning that the results of SequenceMatcher's ratio method may depend on the order of the input strings. \ No newline at end of file From webhook-mailer at python.org Wed Aug 7 11:39:53 2019 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Wed, 07 Aug 2019 15:39:53 -0000 Subject: [Python-checkins] bpo-37004: Documented asymmetry of string arguments in difflib.SequenceMatcher for ratio method (GH-13482) (#15158) Message-ID: https://github.com/python/cpython/commit/7dafbe81bd0afb8bd67bc3a4c851a6c728fd87fe commit: 7dafbe81bd0afb8bd67bc3a4c851a6c728fd87fe branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Terry Jan Reedy date: 2019-08-07T11:39:47-04:00 summary: bpo-37004: Documented asymmetry of string arguments in difflib.SequenceMatcher for ratio method (GH-13482) (#15158) https://bugs.python.org/issue37004 (cherry picked from commit e9cbcd0018abd2a5f2348c45d5c9c4265c4f42dc) Co-authored-by: sweeneyde <36520290+sweeneyde at users.noreply.github.com> files: A Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst M Doc/library/difflib.rst diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst index f044cb2d6e0a..e245ab81cfb9 100644 --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -543,6 +543,16 @@ The :class:`SequenceMatcher` class has this constructor: to try :meth:`quick_ratio` or :meth:`real_quick_ratio` first to get an upper bound. + .. note:: + + Caution: The result of a :meth:`ratio` call may depend on the order of + the arguments. For instance:: + + >>> SequenceMatcher(None, 'tide', 'diet').ratio() + 0.25 + >>> SequenceMatcher(None, 'diet', 'tide').ratio() + 0.5 + .. method:: quick_ratio() diff --git a/Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst b/Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst new file mode 100644 index 000000000000..dfc8b7ed74ca --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-05-22-04-30-07.bpo-37004.BRgxrt.rst @@ -0,0 +1 @@ +In the documentation for difflib, a note was added explicitly warning that the results of SequenceMatcher's ratio method may depend on the order of the input strings. \ No newline at end of file From webhook-mailer at python.org Wed Aug 7 13:49:47 2019 From: webhook-mailer at python.org (Steve Dower) Date: Wed, 07 Aug 2019 17:49:47 -0000 Subject: [Python-checkins] bpo-37734: Fix use of registry values to launch Python from Microsoft Store app (GH-15146) Message-ID: https://github.com/python/cpython/commit/1fab9cbfbaf19a7bc79cef382136fcf9491e3183 commit: 1fab9cbfbaf19a7bc79cef382136fcf9491e3183 branch: master author: Steve Dower committer: GitHub date: 2019-08-07T10:49:40-07:00 summary: bpo-37734: Fix use of registry values to launch Python from Microsoft Store app (GH-15146) files: A Misc/NEWS.d/next/Windows/2019-08-06-09-35-12.bpo-37734.EoJ9Nh.rst M PC/layout/main.py M PC/layout/support/appxmanifest.py diff --git a/Misc/NEWS.d/next/Windows/2019-08-06-09-35-12.bpo-37734.EoJ9Nh.rst b/Misc/NEWS.d/next/Windows/2019-08-06-09-35-12.bpo-37734.EoJ9Nh.rst new file mode 100644 index 000000000000..51feecb076ba --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-08-06-09-35-12.bpo-37734.EoJ9Nh.rst @@ -0,0 +1 @@ +Fix use of registry values to launch Python from Microsoft Store app. diff --git a/PC/layout/main.py b/PC/layout/main.py index fe934bfb1ab3..07b7e6d57429 100644 --- a/PC/layout/main.py +++ b/PC/layout/main.py @@ -153,9 +153,15 @@ def in_build(f, dest="", new_name=None): yield "libs/" + n + ".lib", lib if ns.include_appxmanifest: + yield from in_build("python_uwp.exe", new_name="python{}".format(VER_DOT)) + yield from in_build("pythonw_uwp.exe", new_name="pythonw{}".format(VER_DOT)) + # For backwards compatibility, but we don't reference these ourselves. yield from in_build("python_uwp.exe", new_name="python") yield from in_build("pythonw_uwp.exe", new_name="pythonw") else: + yield from in_build("python.exe", new_name="python{}".format(VER_DOT)) + yield from in_build("pythonw.exe", new_name="pythonw{}".format(VER_DOT)) + # For backwards compatibility, but we don't reference these ourselves. yield from in_build("python.exe", new_name="python") yield from in_build("pythonw.exe", new_name="pythonw") @@ -163,9 +169,9 @@ def in_build(f, dest="", new_name=None): if ns.include_launchers and ns.include_appxmanifest: if ns.include_pip: - yield from in_build("python_uwp.exe", new_name="pip") + yield from in_build("python_uwp.exe", new_name="pip{}".format(VER_DOT)) if ns.include_idle: - yield from in_build("pythonw_uwp.exe", new_name="idle") + yield from in_build("pythonw_uwp.exe", new_name="idle{}".format(VER_DOT)) if ns.include_stable: yield from in_build(PYTHON_STABLE_DLL_NAME) diff --git a/PC/layout/support/appxmanifest.py b/PC/layout/support/appxmanifest.py index 58fba8443f17..0a0f1fc8181c 100644 --- a/PC/layout/support/appxmanifest.py +++ b/PC/layout/support/appxmanifest.py @@ -154,9 +154,9 @@ "SysVersion": VER_DOT, "Version": "{}.{}.{}".format(VER_MAJOR, VER_MINOR, VER_MICRO), "InstallPath": { - "": "[{AppVPackageRoot}]", - "ExecutablePath": "[{AppVPackageRoot}]\\python.exe", - "WindowedExecutablePath": "[{AppVPackageRoot}]\\pythonw.exe", + "": "[{{AppVPackageRoot}}]", + "ExecutablePath": "[{{AppVPackageRoot}}]\\python{}.exe".format(VER_DOT), + "WindowedExecutablePath": "[{{AppVPackageRoot}}]\\pythonw{}.exe".format(VER_DOT), }, "Help": { "Main Python Documentation": { @@ -395,7 +395,7 @@ def get_appxmanifest(ns): ns, xml, "Python", - "python", + "python{}".format(VER_DOT), ["python", "python{}".format(VER_MAJOR), "python{}".format(VER_DOT)], PYTHON_VE_DATA, "console", @@ -406,7 +406,7 @@ def get_appxmanifest(ns): ns, xml, "PythonW", - "pythonw", + "pythonw{}".format(VER_DOT), ["pythonw", "pythonw{}".format(VER_MAJOR), "pythonw{}".format(VER_DOT)], PYTHONW_VE_DATA, "windows", @@ -418,7 +418,7 @@ def get_appxmanifest(ns): ns, xml, "Pip", - "pip", + "pip{}".format(VER_DOT), ["pip", "pip{}".format(VER_MAJOR), "pip{}".format(VER_DOT)], PIP_VE_DATA, "console", @@ -430,7 +430,7 @@ def get_appxmanifest(ns): ns, xml, "Idle", - "idle", + "idle{}".format(VER_DOT), ["idle", "idle{}".format(VER_MAJOR), "idle{}".format(VER_DOT)], IDLE_VE_DATA, "windows", From webhook-mailer at python.org Wed Aug 7 13:50:21 2019 From: webhook-mailer at python.org (Steve Dower) Date: Wed, 07 Aug 2019 17:50:21 -0000 Subject: [Python-checkins] bpo-37778: Fixes the icons used for file associations to the Microsoft Store package (GH-15150) Message-ID: https://github.com/python/cpython/commit/87ce9588ceb4b4dd625913344844390f0b991b0c commit: 87ce9588ceb4b4dd625913344844390f0b991b0c branch: master author: Steve Dower committer: GitHub date: 2019-08-07T10:50:17-07:00 summary: bpo-37778: Fixes the icons used for file associations to the Microsoft Store package (GH-15150) files: A Misc/NEWS.d/next/Windows/2019-08-06-13-54-12.bpo-37778.AY1XhH.rst A PC/icons/py.png M PC/layout/support/appxmanifest.py diff --git a/Misc/NEWS.d/next/Windows/2019-08-06-13-54-12.bpo-37778.AY1XhH.rst b/Misc/NEWS.d/next/Windows/2019-08-06-13-54-12.bpo-37778.AY1XhH.rst new file mode 100644 index 000000000000..14d81c05ee84 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-08-06-13-54-12.bpo-37778.AY1XhH.rst @@ -0,0 +1 @@ +Fixes the icons used for file associations to the Microsoft Store package. diff --git a/PC/icons/py.png b/PC/icons/py.png new file mode 100644 index 000000000000..5c184e6a745f Binary files /dev/null and b/PC/icons/py.png differ diff --git a/PC/layout/support/appxmanifest.py b/PC/layout/support/appxmanifest.py index 0a0f1fc8181c..4273b427e903 100644 --- a/PC/layout/support/appxmanifest.py +++ b/PC/layout/support/appxmanifest.py @@ -65,6 +65,8 @@ BackgroundColor="transparent", ) +PY_PNG = '_resources/py.png' + APPXMANIFEST_NS = { "": "http://schemas.microsoft.com/appx/manifest/foundation/windows10", "m": "http://schemas.microsoft.com/appx/manifest/foundation/windows10", @@ -278,12 +280,16 @@ def add_alias(xml, appid, alias, subsystem="windows"): e = find_or_add(e, "uap5:ExecutionAlias", ("Alias", alias)) -def add_file_type(xml, appid, name, suffix, parameters='"%1"'): +def add_file_type(xml, appid, name, suffix, parameters='"%1"', info=None, logo=None): app = _get_app(xml, appid) e = find_or_add(app, "m:Extensions") e = find_or_add(e, "uap3:Extension", ("Category", "windows.fileTypeAssociation")) e = find_or_add(e, "uap3:FileTypeAssociation", ("Name", name)) e.set("Parameters", parameters) + if info: + find_or_add(e, "uap:DisplayName").text = info + if logo: + find_or_add(e, "uap:Logo").text = logo e = find_or_add(e, "uap:SupportedFileTypes") if isinstance(suffix, str): suffix = [suffix] @@ -399,7 +405,7 @@ def get_appxmanifest(ns): ["python", "python{}".format(VER_MAJOR), "python{}".format(VER_DOT)], PYTHON_VE_DATA, "console", - ("python.file", [".py"]), + ("python.file", [".py"], '"%1"', 'Python File', PY_PNG), ) add_application( @@ -410,7 +416,7 @@ def get_appxmanifest(ns): ["pythonw", "pythonw{}".format(VER_MAJOR), "pythonw{}".format(VER_DOT)], PYTHONW_VE_DATA, "windows", - ("python.windowedfile", [".pyw"]), + ("python.windowedfile", [".pyw"], '"%1"', 'Python File (no console)', PY_PNG), ) if ns.include_pip and ns.include_launchers: @@ -422,7 +428,7 @@ def get_appxmanifest(ns): ["pip", "pip{}".format(VER_MAJOR), "pip{}".format(VER_DOT)], PIP_VE_DATA, "console", - ("python.wheel", [".whl"], 'install "%1"'), + ("python.wheel", [".whl"], 'install "%1"', 'Python Wheel'), ) if ns.include_idle and ns.include_launchers: @@ -459,16 +465,15 @@ def get_appx_layout(ns): yield "AppxManifest.xml", ("AppxManifest.xml", get_appxmanifest(ns)) yield "_resources.xml", ("_resources.xml", get_resources_xml(ns)) icons = ns.source / "PC" / "icons" - yield "_resources/pythonx44.png", icons / "pythonx44.png" - yield "_resources/pythonx44$targetsize-44_altform-unplated.png", icons / "pythonx44.png" - yield "_resources/pythonx50.png", icons / "pythonx50.png" - yield "_resources/pythonx50$targetsize-50_altform-unplated.png", icons / "pythonx50.png" - yield "_resources/pythonx150.png", icons / "pythonx150.png" - yield "_resources/pythonx150$targetsize-150_altform-unplated.png", icons / "pythonx150.png" - yield "_resources/pythonwx44.png", icons / "pythonwx44.png" - yield "_resources/pythonwx44$targetsize-44_altform-unplated.png", icons / "pythonwx44.png" - yield "_resources/pythonwx150.png", icons / "pythonwx150.png" - yield "_resources/pythonwx150$targetsize-150_altform-unplated.png", icons / "pythonwx150.png" + for px in [44, 50, 150]: + src = icons / "pythonx{}.png".format(px) + yield f"_resources/pythonx{px}.png", src + yield f"_resources/pythonx{px}$targetsize-{px}_altform-unplated.png", src + for px in [44, 150]: + src = icons / "pythonwx{}.png".format(px) + yield f"_resources/pythonwx{px}.png", src + yield f"_resources/pythonwx{px}$targetsize-{px}_altform-unplated.png", src + yield f"_resources/py.png", icons / "py.png" sccd = ns.source / SCCD_FILENAME if sccd.is_file(): # This should only be set for side-loading purposes. From webhook-mailer at python.org Wed Aug 7 14:07:48 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 07 Aug 2019 18:07:48 -0000 Subject: [Python-checkins] bpo-37734: Fix use of registry values to launch Python from Microsoft Store app (GH-15146) Message-ID: https://github.com/python/cpython/commit/eab76c3c75a572566862200728cc8d05b3298f12 commit: eab76c3c75a572566862200728cc8d05b3298f12 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-07T11:07:43-07:00 summary: bpo-37734: Fix use of registry values to launch Python from Microsoft Store app (GH-15146) (cherry picked from commit 1fab9cbfbaf19a7bc79cef382136fcf9491e3183) Co-authored-by: Steve Dower files: A Misc/NEWS.d/next/Windows/2019-08-06-09-35-12.bpo-37734.EoJ9Nh.rst M PC/layout/main.py M PC/layout/support/appxmanifest.py diff --git a/Misc/NEWS.d/next/Windows/2019-08-06-09-35-12.bpo-37734.EoJ9Nh.rst b/Misc/NEWS.d/next/Windows/2019-08-06-09-35-12.bpo-37734.EoJ9Nh.rst new file mode 100644 index 000000000000..51feecb076ba --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-08-06-09-35-12.bpo-37734.EoJ9Nh.rst @@ -0,0 +1 @@ +Fix use of registry values to launch Python from Microsoft Store app. diff --git a/PC/layout/main.py b/PC/layout/main.py index fe934bfb1ab3..07b7e6d57429 100644 --- a/PC/layout/main.py +++ b/PC/layout/main.py @@ -153,9 +153,15 @@ def in_build(f, dest="", new_name=None): yield "libs/" + n + ".lib", lib if ns.include_appxmanifest: + yield from in_build("python_uwp.exe", new_name="python{}".format(VER_DOT)) + yield from in_build("pythonw_uwp.exe", new_name="pythonw{}".format(VER_DOT)) + # For backwards compatibility, but we don't reference these ourselves. yield from in_build("python_uwp.exe", new_name="python") yield from in_build("pythonw_uwp.exe", new_name="pythonw") else: + yield from in_build("python.exe", new_name="python{}".format(VER_DOT)) + yield from in_build("pythonw.exe", new_name="pythonw{}".format(VER_DOT)) + # For backwards compatibility, but we don't reference these ourselves. yield from in_build("python.exe", new_name="python") yield from in_build("pythonw.exe", new_name="pythonw") @@ -163,9 +169,9 @@ def in_build(f, dest="", new_name=None): if ns.include_launchers and ns.include_appxmanifest: if ns.include_pip: - yield from in_build("python_uwp.exe", new_name="pip") + yield from in_build("python_uwp.exe", new_name="pip{}".format(VER_DOT)) if ns.include_idle: - yield from in_build("pythonw_uwp.exe", new_name="idle") + yield from in_build("pythonw_uwp.exe", new_name="idle{}".format(VER_DOT)) if ns.include_stable: yield from in_build(PYTHON_STABLE_DLL_NAME) diff --git a/PC/layout/support/appxmanifest.py b/PC/layout/support/appxmanifest.py index 58fba8443f17..0a0f1fc8181c 100644 --- a/PC/layout/support/appxmanifest.py +++ b/PC/layout/support/appxmanifest.py @@ -154,9 +154,9 @@ "SysVersion": VER_DOT, "Version": "{}.{}.{}".format(VER_MAJOR, VER_MINOR, VER_MICRO), "InstallPath": { - "": "[{AppVPackageRoot}]", - "ExecutablePath": "[{AppVPackageRoot}]\\python.exe", - "WindowedExecutablePath": "[{AppVPackageRoot}]\\pythonw.exe", + "": "[{{AppVPackageRoot}}]", + "ExecutablePath": "[{{AppVPackageRoot}}]\\python{}.exe".format(VER_DOT), + "WindowedExecutablePath": "[{{AppVPackageRoot}}]\\pythonw{}.exe".format(VER_DOT), }, "Help": { "Main Python Documentation": { @@ -395,7 +395,7 @@ def get_appxmanifest(ns): ns, xml, "Python", - "python", + "python{}".format(VER_DOT), ["python", "python{}".format(VER_MAJOR), "python{}".format(VER_DOT)], PYTHON_VE_DATA, "console", @@ -406,7 +406,7 @@ def get_appxmanifest(ns): ns, xml, "PythonW", - "pythonw", + "pythonw{}".format(VER_DOT), ["pythonw", "pythonw{}".format(VER_MAJOR), "pythonw{}".format(VER_DOT)], PYTHONW_VE_DATA, "windows", @@ -418,7 +418,7 @@ def get_appxmanifest(ns): ns, xml, "Pip", - "pip", + "pip{}".format(VER_DOT), ["pip", "pip{}".format(VER_MAJOR), "pip{}".format(VER_DOT)], PIP_VE_DATA, "console", @@ -430,7 +430,7 @@ def get_appxmanifest(ns): ns, xml, "Idle", - "idle", + "idle{}".format(VER_DOT), ["idle", "idle{}".format(VER_MAJOR), "idle{}".format(VER_DOT)], IDLE_VE_DATA, "windows", From webhook-mailer at python.org Wed Aug 7 14:15:20 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 07 Aug 2019 18:15:20 -0000 Subject: [Python-checkins] bpo-37778: Fixes the icons used for file associations to the Microsoft Store package (GH-15150) Message-ID: https://github.com/python/cpython/commit/dc6653fd06598f42b107dcffcd089d7ee2b1cd44 commit: dc6653fd06598f42b107dcffcd089d7ee2b1cd44 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-07T11:15:15-07:00 summary: bpo-37778: Fixes the icons used for file associations to the Microsoft Store package (GH-15150) (cherry picked from commit 87ce9588ceb4b4dd625913344844390f0b991b0c) Co-authored-by: Steve Dower files: A Misc/NEWS.d/next/Windows/2019-08-06-13-54-12.bpo-37778.AY1XhH.rst A PC/icons/py.png M PC/layout/support/appxmanifest.py diff --git a/Misc/NEWS.d/next/Windows/2019-08-06-13-54-12.bpo-37778.AY1XhH.rst b/Misc/NEWS.d/next/Windows/2019-08-06-13-54-12.bpo-37778.AY1XhH.rst new file mode 100644 index 000000000000..14d81c05ee84 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-08-06-13-54-12.bpo-37778.AY1XhH.rst @@ -0,0 +1 @@ +Fixes the icons used for file associations to the Microsoft Store package. diff --git a/PC/icons/py.png b/PC/icons/py.png new file mode 100644 index 000000000000..5c184e6a745f Binary files /dev/null and b/PC/icons/py.png differ diff --git a/PC/layout/support/appxmanifest.py b/PC/layout/support/appxmanifest.py index 0a0f1fc8181c..4273b427e903 100644 --- a/PC/layout/support/appxmanifest.py +++ b/PC/layout/support/appxmanifest.py @@ -65,6 +65,8 @@ BackgroundColor="transparent", ) +PY_PNG = '_resources/py.png' + APPXMANIFEST_NS = { "": "http://schemas.microsoft.com/appx/manifest/foundation/windows10", "m": "http://schemas.microsoft.com/appx/manifest/foundation/windows10", @@ -278,12 +280,16 @@ def add_alias(xml, appid, alias, subsystem="windows"): e = find_or_add(e, "uap5:ExecutionAlias", ("Alias", alias)) -def add_file_type(xml, appid, name, suffix, parameters='"%1"'): +def add_file_type(xml, appid, name, suffix, parameters='"%1"', info=None, logo=None): app = _get_app(xml, appid) e = find_or_add(app, "m:Extensions") e = find_or_add(e, "uap3:Extension", ("Category", "windows.fileTypeAssociation")) e = find_or_add(e, "uap3:FileTypeAssociation", ("Name", name)) e.set("Parameters", parameters) + if info: + find_or_add(e, "uap:DisplayName").text = info + if logo: + find_or_add(e, "uap:Logo").text = logo e = find_or_add(e, "uap:SupportedFileTypes") if isinstance(suffix, str): suffix = [suffix] @@ -399,7 +405,7 @@ def get_appxmanifest(ns): ["python", "python{}".format(VER_MAJOR), "python{}".format(VER_DOT)], PYTHON_VE_DATA, "console", - ("python.file", [".py"]), + ("python.file", [".py"], '"%1"', 'Python File', PY_PNG), ) add_application( @@ -410,7 +416,7 @@ def get_appxmanifest(ns): ["pythonw", "pythonw{}".format(VER_MAJOR), "pythonw{}".format(VER_DOT)], PYTHONW_VE_DATA, "windows", - ("python.windowedfile", [".pyw"]), + ("python.windowedfile", [".pyw"], '"%1"', 'Python File (no console)', PY_PNG), ) if ns.include_pip and ns.include_launchers: @@ -422,7 +428,7 @@ def get_appxmanifest(ns): ["pip", "pip{}".format(VER_MAJOR), "pip{}".format(VER_DOT)], PIP_VE_DATA, "console", - ("python.wheel", [".whl"], 'install "%1"'), + ("python.wheel", [".whl"], 'install "%1"', 'Python Wheel'), ) if ns.include_idle and ns.include_launchers: @@ -459,16 +465,15 @@ def get_appx_layout(ns): yield "AppxManifest.xml", ("AppxManifest.xml", get_appxmanifest(ns)) yield "_resources.xml", ("_resources.xml", get_resources_xml(ns)) icons = ns.source / "PC" / "icons" - yield "_resources/pythonx44.png", icons / "pythonx44.png" - yield "_resources/pythonx44$targetsize-44_altform-unplated.png", icons / "pythonx44.png" - yield "_resources/pythonx50.png", icons / "pythonx50.png" - yield "_resources/pythonx50$targetsize-50_altform-unplated.png", icons / "pythonx50.png" - yield "_resources/pythonx150.png", icons / "pythonx150.png" - yield "_resources/pythonx150$targetsize-150_altform-unplated.png", icons / "pythonx150.png" - yield "_resources/pythonwx44.png", icons / "pythonwx44.png" - yield "_resources/pythonwx44$targetsize-44_altform-unplated.png", icons / "pythonwx44.png" - yield "_resources/pythonwx150.png", icons / "pythonwx150.png" - yield "_resources/pythonwx150$targetsize-150_altform-unplated.png", icons / "pythonwx150.png" + for px in [44, 50, 150]: + src = icons / "pythonx{}.png".format(px) + yield f"_resources/pythonx{px}.png", src + yield f"_resources/pythonx{px}$targetsize-{px}_altform-unplated.png", src + for px in [44, 150]: + src = icons / "pythonwx{}.png".format(px) + yield f"_resources/pythonwx{px}.png", src + yield f"_resources/pythonwx{px}$targetsize-{px}_altform-unplated.png", src + yield f"_resources/py.png", icons / "py.png" sccd = ns.source / SCCD_FILENAME if sccd.is_file(): # This should only be set for side-loading purposes. From webhook-mailer at python.org Wed Aug 7 14:39:19 2019 From: webhook-mailer at python.org (Steve Dower) Date: Wed, 07 Aug 2019 18:39:19 -0000 Subject: [Python-checkins] bpo-37734: Remove unnecessary brace escapes in PC/layout script (GH-15165) Message-ID: https://github.com/python/cpython/commit/0378d98678f3617fd44d9a6266e7c17ebce62755 commit: 0378d98678f3617fd44d9a6266e7c17ebce62755 branch: master author: Steve Dower committer: GitHub date: 2019-08-07T11:39:09-07:00 summary: bpo-37734: Remove unnecessary brace escapes in PC/layout script (GH-15165) files: M PC/layout/support/appxmanifest.py diff --git a/PC/layout/support/appxmanifest.py b/PC/layout/support/appxmanifest.py index 4273b427e903..de5813a2536a 100644 --- a/PC/layout/support/appxmanifest.py +++ b/PC/layout/support/appxmanifest.py @@ -156,7 +156,7 @@ "SysVersion": VER_DOT, "Version": "{}.{}.{}".format(VER_MAJOR, VER_MINOR, VER_MICRO), "InstallPath": { - "": "[{{AppVPackageRoot}}]", + "": "[{AppVPackageRoot}]", "ExecutablePath": "[{{AppVPackageRoot}}]\\python{}.exe".format(VER_DOT), "WindowedExecutablePath": "[{{AppVPackageRoot}}]\\pythonw{}.exe".format(VER_DOT), }, From webhook-mailer at python.org Wed Aug 7 14:59:13 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 07 Aug 2019 18:59:13 -0000 Subject: [Python-checkins] bpo-37734: Remove unnecessary brace escapes in PC/layout script (GH-15165) Message-ID: https://github.com/python/cpython/commit/84d31bbf1f7e10bde1ceadcfa0d83d30a04313d5 commit: 84d31bbf1f7e10bde1ceadcfa0d83d30a04313d5 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-07T11:59:09-07:00 summary: bpo-37734: Remove unnecessary brace escapes in PC/layout script (GH-15165) (cherry picked from commit 0378d98678f3617fd44d9a6266e7c17ebce62755) Co-authored-by: Steve Dower files: M PC/layout/support/appxmanifest.py diff --git a/PC/layout/support/appxmanifest.py b/PC/layout/support/appxmanifest.py index 4273b427e903..de5813a2536a 100644 --- a/PC/layout/support/appxmanifest.py +++ b/PC/layout/support/appxmanifest.py @@ -156,7 +156,7 @@ "SysVersion": VER_DOT, "Version": "{}.{}.{}".format(VER_MAJOR, VER_MINOR, VER_MICRO), "InstallPath": { - "": "[{{AppVPackageRoot}}]", + "": "[{AppVPackageRoot}]", "ExecutablePath": "[{{AppVPackageRoot}}]\\python{}.exe".format(VER_DOT), "WindowedExecutablePath": "[{{AppVPackageRoot}}]\\pythonw{}.exe".format(VER_DOT), }, From webhook-mailer at python.org Thu Aug 8 01:41:19 2019 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Thu, 08 Aug 2019 05:41:19 -0000 Subject: [Python-checkins] bpo-34775: Return NotImplemented in PurePath division. (GH-9509) Message-ID: https://github.com/python/cpython/commit/4c69be22df3852f17873a74d015528d9a8ae92d6 commit: 4c69be22df3852f17873a74d015528d9a8ae92d6 branch: master author: aiudirog committer: Serhiy Storchaka date: 2019-08-08T08:41:10+03:00 summary: bpo-34775: Return NotImplemented in PurePath division. (GH-9509) files: A Misc/NEWS.d/next/Library/2018-09-23-03-18-52.bpo-34775.vHeuHk.rst M Lib/pathlib.py M Lib/test/test_pathlib.py diff --git a/Lib/pathlib.py b/Lib/pathlib.py index 6369c4b2218d..6355ae864108 100644 --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -907,10 +907,16 @@ def joinpath(self, *args): return self._make_child(args) def __truediv__(self, key): - return self._make_child((key,)) + try: + return self._make_child((key,)) + except TypeError: + return NotImplemented def __rtruediv__(self, key): - return self._from_parts([key] + self._parts) + try: + return self._from_parts([key] + self._parts) + except TypeError: + return NotImplemented @property def parent(self): diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index 069467a8459f..f74524d992a1 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -2329,5 +2329,46 @@ def check(): check() +class CompatiblePathTest(unittest.TestCase): + """ + Test that a type can be made compatible with PurePath + derivatives by implementing division operator overloads. + """ + + class CompatPath: + """ + Minimum viable class to test PurePath compatibility. + Simply uses the division operator to join a given + string and the string value of another object with + a forward slash. + """ + def __init__(self, string): + self.string = string + + def __truediv__(self, other): + return type(self)(f"{self.string}/{other}") + + def __rtruediv__(self, other): + return type(self)(f"{other}/{self.string}") + + def test_truediv(self): + result = pathlib.PurePath("test") / self.CompatPath("right") + self.assertIsInstance(result, self.CompatPath) + self.assertEqual(result.string, "test/right") + + with self.assertRaises(TypeError): + # Verify improper operations still raise a TypeError + pathlib.PurePath("test") / 10 + + def test_rtruediv(self): + result = self.CompatPath("left") / pathlib.PurePath("test") + self.assertIsInstance(result, self.CompatPath) + self.assertEqual(result.string, "left/test") + + with self.assertRaises(TypeError): + # Verify improper operations still raise a TypeError + 10 / pathlib.PurePath("test") + + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/Library/2018-09-23-03-18-52.bpo-34775.vHeuHk.rst b/Misc/NEWS.d/next/Library/2018-09-23-03-18-52.bpo-34775.vHeuHk.rst new file mode 100644 index 000000000000..f99bf5b39f95 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2018-09-23-03-18-52.bpo-34775.vHeuHk.rst @@ -0,0 +1,3 @@ +Division handling of PurePath now returns NotImplemented instead of raising +a TypeError when passed something other than an instance of str or PurePath. +Patch by Roger Aiudi. From webhook-mailer at python.org Thu Aug 8 01:42:58 2019 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Thu, 08 Aug 2019 05:42:58 -0000 Subject: [Python-checkins] bpo-37685: Fixed __eq__, __lt__ etc implementations in some classes. (GH-14952) Message-ID: https://github.com/python/cpython/commit/662db125cddbca1db68116c547c290eb3943d98e commit: 662db125cddbca1db68116c547c290eb3943d98e branch: master author: Serhiy Storchaka committer: GitHub date: 2019-08-08T08:42:54+03:00 summary: bpo-37685: Fixed __eq__, __lt__ etc implementations in some classes. (GH-14952) They now return NotImplemented for unsupported type of the other operand. files: A Misc/NEWS.d/next/Library/2019-07-26-00-12-29.bpo-37685.TqckMZ.rst M Lib/asyncio/events.py M Lib/distutils/tests/test_version.py M Lib/distutils/version.py M Lib/email/headerregistry.py M Lib/importlib/_bootstrap.py M Lib/test/test_asyncio/test_events.py M Lib/test/test_email/test_headerregistry.py M Lib/test/test_traceback.py M Lib/test/test_weakref.py M Lib/test/test_xmlrpc.py M Lib/tkinter/__init__.py M Lib/tkinter/font.py M Lib/tkinter/test/test_tkinter/test_font.py M Lib/tkinter/test/test_tkinter/test_variables.py M Lib/traceback.py M Lib/tracemalloc.py M Lib/unittest/mock.py M Lib/unittest/test/testmock/testmock.py M Lib/weakref.py M Lib/xmlrpc/client.py M Python/importlib.h M Tools/pynche/PyncheWidget.py diff --git a/Lib/asyncio/events.py b/Lib/asyncio/events.py index d381b1c59623..5fb546429cbe 100644 --- a/Lib/asyncio/events.py +++ b/Lib/asyncio/events.py @@ -119,20 +119,24 @@ def __hash__(self): return hash(self._when) def __lt__(self, other): - return self._when < other._when + if isinstance(other, TimerHandle): + return self._when < other._when + return NotImplemented def __le__(self, other): - if self._when < other._when: - return True - return self.__eq__(other) + if isinstance(other, TimerHandle): + return self._when < other._when or self.__eq__(other) + return NotImplemented def __gt__(self, other): - return self._when > other._when + if isinstance(other, TimerHandle): + return self._when > other._when + return NotImplemented def __ge__(self, other): - if self._when > other._when: - return True - return self.__eq__(other) + if isinstance(other, TimerHandle): + return self._when > other._when or self.__eq__(other) + return NotImplemented def __eq__(self, other): if isinstance(other, TimerHandle): @@ -142,10 +146,6 @@ def __eq__(self, other): self._cancelled == other._cancelled) return NotImplemented - def __ne__(self, other): - equal = self.__eq__(other) - return NotImplemented if equal is NotImplemented else not equal - def cancel(self): if not self._cancelled: self._loop._timer_handle_cancelled(self) diff --git a/Lib/distutils/tests/test_version.py b/Lib/distutils/tests/test_version.py index 15f14c7de3f1..8671cd2fc5c1 100644 --- a/Lib/distutils/tests/test_version.py +++ b/Lib/distutils/tests/test_version.py @@ -45,6 +45,14 @@ def test_cmp_strict(self): self.assertEqual(res, wanted, 'cmp(%s, %s) should be %s, got %s' % (v1, v2, wanted, res)) + res = StrictVersion(v1)._cmp(v2) + self.assertEqual(res, wanted, + 'cmp(%s, %s) should be %s, got %s' % + (v1, v2, wanted, res)) + res = StrictVersion(v1)._cmp(object()) + self.assertIs(res, NotImplemented, + 'cmp(%s, %s) should be NotImplemented, got %s' % + (v1, v2, res)) def test_cmp(self): @@ -63,6 +71,14 @@ def test_cmp(self): self.assertEqual(res, wanted, 'cmp(%s, %s) should be %s, got %s' % (v1, v2, wanted, res)) + res = LooseVersion(v1)._cmp(v2) + self.assertEqual(res, wanted, + 'cmp(%s, %s) should be %s, got %s' % + (v1, v2, wanted, res)) + res = LooseVersion(v1)._cmp(object()) + self.assertIs(res, NotImplemented, + 'cmp(%s, %s) should be NotImplemented, got %s' % + (v1, v2, res)) def test_suite(): return unittest.makeSuite(VersionTestCase) diff --git a/Lib/distutils/version.py b/Lib/distutils/version.py index af14cc134814..c33bebaed26a 100644 --- a/Lib/distutils/version.py +++ b/Lib/distutils/version.py @@ -166,6 +166,8 @@ def __str__ (self): def _cmp (self, other): if isinstance(other, str): other = StrictVersion(other) + elif not isinstance(other, StrictVersion): + return NotImplemented if self.version != other.version: # numeric versions don't match @@ -331,6 +333,8 @@ def __repr__ (self): def _cmp (self, other): if isinstance(other, str): other = LooseVersion(other) + elif not isinstance(other, LooseVersion): + return NotImplemented if self.version == other.version: return 0 diff --git a/Lib/email/headerregistry.py b/Lib/email/headerregistry.py index 8d1a2025271f..dcc960b2cdc1 100644 --- a/Lib/email/headerregistry.py +++ b/Lib/email/headerregistry.py @@ -97,8 +97,8 @@ def __str__(self): return self.addr_spec def __eq__(self, other): - if type(other) != type(self): - return False + if not isinstance(other, Address): + return NotImplemented return (self.display_name == other.display_name and self.username == other.username and self.domain == other.domain) @@ -150,8 +150,8 @@ def __str__(self): return "{}:{};".format(disp, adrstr) def __eq__(self, other): - if type(other) != type(self): - return False + if not isinstance(other, Group): + return NotImplemented return (self.display_name == other.display_name and self.addresses == other.addresses) diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py index 5e2f520c0ef8..e17eeb658538 100644 --- a/Lib/importlib/_bootstrap.py +++ b/Lib/importlib/_bootstrap.py @@ -371,7 +371,7 @@ def __eq__(self, other): self.cached == other.cached and self.has_location == other.has_location) except AttributeError: - return False + return NotImplemented @property def cached(self): diff --git a/Lib/test/test_asyncio/test_events.py b/Lib/test/test_asyncio/test_events.py index e5ad72fe5ba8..5bc1bc2a621b 100644 --- a/Lib/test/test_asyncio/test_events.py +++ b/Lib/test/test_asyncio/test_events.py @@ -32,6 +32,7 @@ from asyncio import selector_events from test.test_asyncio import utils as test_utils from test import support +from test.support import ALWAYS_EQ, LARGEST, SMALLEST def tearDownModule(): @@ -2364,6 +2365,28 @@ def callback(*args): self.assertIs(NotImplemented, h1.__eq__(h3)) self.assertIs(NotImplemented, h1.__ne__(h3)) + with self.assertRaises(TypeError): + h1 < () + with self.assertRaises(TypeError): + h1 > () + with self.assertRaises(TypeError): + h1 <= () + with self.assertRaises(TypeError): + h1 >= () + self.assertFalse(h1 == ()) + self.assertTrue(h1 != ()) + + self.assertTrue(h1 == ALWAYS_EQ) + self.assertFalse(h1 != ALWAYS_EQ) + self.assertTrue(h1 < LARGEST) + self.assertFalse(h1 > LARGEST) + self.assertTrue(h1 <= LARGEST) + self.assertFalse(h1 >= LARGEST) + self.assertFalse(h1 < SMALLEST) + self.assertTrue(h1 > SMALLEST) + self.assertFalse(h1 <= SMALLEST) + self.assertTrue(h1 >= SMALLEST) + class AbstractEventLoopTests(unittest.TestCase): diff --git a/Lib/test/test_email/test_headerregistry.py b/Lib/test/test_email/test_headerregistry.py index 5d9b3576d306..4758f4b1c52e 100644 --- a/Lib/test/test_email/test_headerregistry.py +++ b/Lib/test/test_email/test_headerregistry.py @@ -7,6 +7,7 @@ from test.test_email import TestEmailBase, parameterize from email import headerregistry from email.headerregistry import Address, Group +from test.support import ALWAYS_EQ DITTO = object() @@ -1525,6 +1526,24 @@ def test_set_message_header_from_group(self): self.assertEqual(m['to'], 'foo bar:;') self.assertEqual(m['to'].addresses, g.addresses) + def test_address_comparison(self): + a = Address('foo', 'bar', 'example.com') + self.assertEqual(Address('foo', 'bar', 'example.com'), a) + self.assertNotEqual(Address('baz', 'bar', 'example.com'), a) + self.assertNotEqual(Address('foo', 'baz', 'example.com'), a) + self.assertNotEqual(Address('foo', 'bar', 'baz'), a) + self.assertFalse(a == object()) + self.assertTrue(a == ALWAYS_EQ) + + def test_group_comparison(self): + a = Address('foo', 'bar', 'example.com') + g = Group('foo bar', [a]) + self.assertEqual(Group('foo bar', (a,)), g) + self.assertNotEqual(Group('baz', [a]), g) + self.assertNotEqual(Group('foo bar', []), g) + self.assertFalse(g == object()) + self.assertTrue(g == ALWAYS_EQ) + class TestFolding(TestHeaderBase): diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py index 96d85e2cb8a1..72dc7afb8c5c 100644 --- a/Lib/test/test_traceback.py +++ b/Lib/test/test_traceback.py @@ -7,7 +7,7 @@ import unittest import re from test import support -from test.support import TESTFN, Error, captured_output, unlink, cpython_only +from test.support import TESTFN, Error, captured_output, unlink, cpython_only, ALWAYS_EQ from test.support.script_helper import assert_python_ok import textwrap @@ -887,6 +887,8 @@ def test_basics(self): # operator fallbacks to FrameSummary.__eq__. self.assertEqual(tuple(f), f) self.assertIsNone(f.locals) + self.assertNotEqual(f, object()) + self.assertEqual(f, ALWAYS_EQ) def test_lazy_lines(self): linecache.clearcache() @@ -1083,6 +1085,18 @@ def test_context(self): self.assertEqual(exc_info[0], exc.exc_type) self.assertEqual(str(exc_info[1]), str(exc)) + def test_comparison(self): + try: + 1/0 + except Exception: + exc_info = sys.exc_info() + exc = traceback.TracebackException(*exc_info) + exc2 = traceback.TracebackException(*exc_info) + self.assertIsNot(exc, exc2) + self.assertEqual(exc, exc2) + self.assertNotEqual(exc, object()) + self.assertEqual(exc, ALWAYS_EQ) + def test_unhashable(self): class UnhashableException(Exception): def __eq__(self, other): diff --git a/Lib/test/test_weakref.py b/Lib/test/test_weakref.py index ce5bbfccd787..41f78e7e8383 100644 --- a/Lib/test/test_weakref.py +++ b/Lib/test/test_weakref.py @@ -11,7 +11,7 @@ import random from test import support -from test.support import script_helper +from test.support import script_helper, ALWAYS_EQ # Used in ReferencesTestCase.test_ref_created_during_del() . ref_from_del = None @@ -794,6 +794,10 @@ def test_equality(self): self.assertTrue(a != c) self.assertTrue(a == d) self.assertFalse(a != d) + self.assertFalse(a == x) + self.assertTrue(a != x) + self.assertTrue(a == ALWAYS_EQ) + self.assertFalse(a != ALWAYS_EQ) del x, y, z gc.collect() for r in a, b, c: @@ -1102,6 +1106,9 @@ def _ne(a, b): _ne(a, f) _ne(b, e) _ne(b, f) + # Compare with different types + _ne(a, x.some_method) + _eq(a, ALWAYS_EQ) del x, y, z gc.collect() # Dead WeakMethods compare by identity diff --git a/Lib/test/test_xmlrpc.py b/Lib/test/test_xmlrpc.py index 52bacc1eafa7..e5c3496ec548 100644 --- a/Lib/test/test_xmlrpc.py +++ b/Lib/test/test_xmlrpc.py @@ -15,6 +15,7 @@ import io import contextlib from test import support +from test.support import ALWAYS_EQ, LARGEST, SMALLEST try: import gzip @@ -530,14 +531,10 @@ def test_comparison(self): # some other types dbytes = dstr.encode('ascii') dtuple = now.timetuple() - with self.assertRaises(TypeError): - dtime == 1970 - with self.assertRaises(TypeError): - dtime != dbytes - with self.assertRaises(TypeError): - dtime == bytearray(dbytes) - with self.assertRaises(TypeError): - dtime != dtuple + self.assertFalse(dtime == 1970) + self.assertTrue(dtime != dbytes) + self.assertFalse(dtime == bytearray(dbytes)) + self.assertTrue(dtime != dtuple) with self.assertRaises(TypeError): dtime < float(1970) with self.assertRaises(TypeError): @@ -547,6 +544,18 @@ def test_comparison(self): with self.assertRaises(TypeError): dtime >= dtuple + self.assertTrue(dtime == ALWAYS_EQ) + self.assertFalse(dtime != ALWAYS_EQ) + self.assertTrue(dtime < LARGEST) + self.assertFalse(dtime > LARGEST) + self.assertTrue(dtime <= LARGEST) + self.assertFalse(dtime >= LARGEST) + self.assertFalse(dtime < SMALLEST) + self.assertTrue(dtime > SMALLEST) + self.assertFalse(dtime <= SMALLEST) + self.assertTrue(dtime >= SMALLEST) + + class BinaryTestCase(unittest.TestCase): # XXX What should str(Binary(b"\xff")) return? I'm chosing "\xff" diff --git a/Lib/tkinter/__init__.py b/Lib/tkinter/__init__.py index 9626a2780db6..9258484af5fd 100644 --- a/Lib/tkinter/__init__.py +++ b/Lib/tkinter/__init__.py @@ -484,6 +484,8 @@ def __eq__(self, other): Note: if the Variable's master matters to behavior also compare self._master == other._master """ + if not isinstance(other, Variable): + return NotImplemented return self.__class__.__name__ == other.__class__.__name__ \ and self._name == other._name diff --git a/Lib/tkinter/font.py b/Lib/tkinter/font.py index eeff454b5306..15ad7ab4b63a 100644 --- a/Lib/tkinter/font.py +++ b/Lib/tkinter/font.py @@ -101,7 +101,9 @@ def __str__(self): return self.name def __eq__(self, other): - return isinstance(other, Font) and self.name == other.name + if not isinstance(other, Font): + return NotImplemented + return self.name == other.name def __getitem__(self, key): return self.cget(key) diff --git a/Lib/tkinter/test/test_tkinter/test_font.py b/Lib/tkinter/test/test_tkinter/test_font.py index 97cd87ccdc81..a021ea336807 100644 --- a/Lib/tkinter/test/test_tkinter/test_font.py +++ b/Lib/tkinter/test/test_tkinter/test_font.py @@ -1,7 +1,7 @@ import unittest import tkinter from tkinter import font -from test.support import requires, run_unittest, gc_collect +from test.support import requires, run_unittest, gc_collect, ALWAYS_EQ from tkinter.test.support import AbstractTkTest requires('gui') @@ -70,6 +70,7 @@ def test_eq(self): self.assertEqual(font1, font2) self.assertNotEqual(font1, font1.copy()) self.assertNotEqual(font1, 0) + self.assertEqual(font1, ALWAYS_EQ) def test_measure(self): self.assertIsInstance(self.font.measure('abc'), int) diff --git a/Lib/tkinter/test/test_tkinter/test_variables.py b/Lib/tkinter/test/test_tkinter/test_variables.py index 2eb1e12671d2..08b7dedcaf93 100644 --- a/Lib/tkinter/test/test_tkinter/test_variables.py +++ b/Lib/tkinter/test/test_tkinter/test_variables.py @@ -2,6 +2,7 @@ import gc from tkinter import (Variable, StringVar, IntVar, DoubleVar, BooleanVar, Tcl, TclError) +from test.support import ALWAYS_EQ class Var(Variable): @@ -59,11 +60,17 @@ def test___eq__(self): # values doesn't matter, only class and name are checked v1 = Variable(self.root, name="abc") v2 = Variable(self.root, name="abc") + self.assertIsNot(v1, v2) self.assertEqual(v1, v2) - v3 = Variable(self.root, name="abc") - v4 = StringVar(self.root, name="abc") - self.assertNotEqual(v3, v4) + v3 = StringVar(self.root, name="abc") + self.assertNotEqual(v1, v3) + + V = type('Variable', (), {}) + self.assertNotEqual(v1, V()) + + self.assertNotEqual(v1, object()) + self.assertEqual(v1, ALWAYS_EQ) def test_invalid_name(self): with self.assertRaises(TypeError): diff --git a/Lib/traceback.py b/Lib/traceback.py index ab35da94b51e..7a4c8e19f989 100644 --- a/Lib/traceback.py +++ b/Lib/traceback.py @@ -538,7 +538,9 @@ def _load_lines(self): self.__cause__._load_lines() def __eq__(self, other): - return self.__dict__ == other.__dict__ + if isinstance(other, TracebackException): + return self.__dict__ == other.__dict__ + return NotImplemented def __str__(self): return self._str diff --git a/Lib/tracemalloc.py b/Lib/tracemalloc.py index 2c1ac3b39b07..80b521c2e1af 100644 --- a/Lib/tracemalloc.py +++ b/Lib/tracemalloc.py @@ -43,6 +43,8 @@ def __hash__(self): return hash((self.traceback, self.size, self.count)) def __eq__(self, other): + if not isinstance(other, Statistic): + return NotImplemented return (self.traceback == other.traceback and self.size == other.size and self.count == other.count) @@ -84,6 +86,8 @@ def __hash__(self): self.count, self.count_diff)) def __eq__(self, other): + if not isinstance(other, StatisticDiff): + return NotImplemented return (self.traceback == other.traceback and self.size == other.size and self.size_diff == other.size_diff @@ -153,9 +157,13 @@ def lineno(self): return self._frame[1] def __eq__(self, other): + if not isinstance(other, Frame): + return NotImplemented return (self._frame == other._frame) def __lt__(self, other): + if not isinstance(other, Frame): + return NotImplemented return (self._frame < other._frame) def __hash__(self): @@ -200,9 +208,13 @@ def __hash__(self): return hash(self._frames) def __eq__(self, other): + if not isinstance(other, Traceback): + return NotImplemented return (self._frames == other._frames) def __lt__(self, other): + if not isinstance(other, Traceback): + return NotImplemented return (self._frames < other._frames) def __str__(self): @@ -271,6 +283,8 @@ def traceback(self): return Traceback(self._trace[2]) def __eq__(self, other): + if not isinstance(other, Trace): + return NotImplemented return (self._trace == other._trace) def __hash__(self): @@ -303,6 +317,8 @@ def __contains__(self, trace): return trace._trace in self._traces def __eq__(self, other): + if not isinstance(other, _Traces): + return NotImplemented return (self._traces == other._traces) def __repr__(self): diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py index b3dc640f8fe5..298b41e0d7e4 100644 --- a/Lib/unittest/mock.py +++ b/Lib/unittest/mock.py @@ -2358,12 +2358,10 @@ def __init__(self, value=(), name=None, parent=None, two=False, def __eq__(self, other): - if other is ANY: - return True try: len_other = len(other) except TypeError: - return False + return NotImplemented self_name = '' if len(self) == 2: diff --git a/Lib/unittest/test/testmock/testmock.py b/Lib/unittest/test/testmock/testmock.py index 18efd311f9e6..69b34e9c4fb7 100644 --- a/Lib/unittest/test/testmock/testmock.py +++ b/Lib/unittest/test/testmock/testmock.py @@ -3,6 +3,7 @@ import sys import tempfile +from test.support import ALWAYS_EQ import unittest from unittest.test.testmock.support import is_instance from unittest import mock @@ -322,6 +323,8 @@ def test_calls_equal_with_any(self): self.assertFalse(mm != mock.ANY) self.assertTrue(mock.ANY == mm) self.assertFalse(mock.ANY != mm) + self.assertTrue(mm == ALWAYS_EQ) + self.assertFalse(mm != ALWAYS_EQ) call1 = mock.call(mock.MagicMock()) call2 = mock.call(mock.ANY) @@ -330,6 +333,11 @@ def test_calls_equal_with_any(self): self.assertTrue(call2 == call1) self.assertFalse(call2 != call1) + self.assertTrue(call1 == ALWAYS_EQ) + self.assertFalse(call1 != ALWAYS_EQ) + self.assertFalse(call1 == 1) + self.assertTrue(call1 != 1) + def test_assert_called_with(self): mock = Mock() diff --git a/Lib/weakref.py b/Lib/weakref.py index fa7559bb3dbf..560deee96c92 100644 --- a/Lib/weakref.py +++ b/Lib/weakref.py @@ -75,14 +75,14 @@ def __eq__(self, other): if not self._alive or not other._alive: return self is other return ref.__eq__(self, other) and self._func_ref == other._func_ref - return False + return NotImplemented def __ne__(self, other): if isinstance(other, WeakMethod): if not self._alive or not other._alive: return self is not other return ref.__ne__(self, other) or self._func_ref != other._func_ref - return True + return NotImplemented __hash__ = ref.__hash__ diff --git a/Lib/xmlrpc/client.py b/Lib/xmlrpc/client.py index b9875745000f..246ef27ffc24 100644 --- a/Lib/xmlrpc/client.py +++ b/Lib/xmlrpc/client.py @@ -313,31 +313,38 @@ def make_comparable(self, other): s = self.timetuple() o = other.timetuple() else: - otype = (hasattr(other, "__class__") - and other.__class__.__name__ - or type(other)) - raise TypeError("Can't compare %s and %s" % - (self.__class__.__name__, otype)) + s = self + o = NotImplemented return s, o def __lt__(self, other): s, o = self.make_comparable(other) + if o is NotImplemented: + return NotImplemented return s < o def __le__(self, other): s, o = self.make_comparable(other) + if o is NotImplemented: + return NotImplemented return s <= o def __gt__(self, other): s, o = self.make_comparable(other) + if o is NotImplemented: + return NotImplemented return s > o def __ge__(self, other): s, o = self.make_comparable(other) + if o is NotImplemented: + return NotImplemented return s >= o def __eq__(self, other): s, o = self.make_comparable(other) + if o is NotImplemented: + return NotImplemented return s == o def timetuple(self): diff --git a/Misc/NEWS.d/next/Library/2019-07-26-00-12-29.bpo-37685.TqckMZ.rst b/Misc/NEWS.d/next/Library/2019-07-26-00-12-29.bpo-37685.TqckMZ.rst new file mode 100644 index 000000000000..d1179a620c17 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-26-00-12-29.bpo-37685.TqckMZ.rst @@ -0,0 +1,4 @@ +Fixed ``__eq__``, ``__lt__`` etc implementations in some classes. They now +return :data:`NotImplemented` for unsupported type of the other operand. +This allows the other operand to play role (for example the equality +comparison with :data:`~unittest.mock.ANY` will return ``True``). diff --git a/Python/importlib.h b/Python/importlib.h index a6952af22b4f..5738af03b054 100644 --- a/Python/importlib.h +++ b/Python/importlib.h @@ -636,7 +636,7 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 124,2,124,0,95,1,124,3,124,0,95,2,124,4,124,0, 95,3,124,5,114,32,103,0,110,2,100,0,124,0,95,4, 100,1,124,0,95,5,100,0,124,0,95,6,100,0,83,0, - 169,2,78,70,41,7,114,17,0,0,0,114,109,0,0,0, + 41,2,78,70,41,7,114,17,0,0,0,114,109,0,0,0, 114,113,0,0,0,114,114,0,0,0,218,26,115,117,98,109, 111,100,117,108,101,95,115,101,97,114,99,104,95,108,111,99, 97,116,105,111,110,115,218,13,95,115,101,116,95,102,105,108, @@ -662,7 +662,7 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 97,116,105,111,110,115,61,123,125,122,6,123,125,40,123,125, 41,122,2,44,32,41,9,114,45,0,0,0,114,17,0,0, 0,114,109,0,0,0,114,113,0,0,0,218,6,97,112,112, - 101,110,100,114,117,0,0,0,218,9,95,95,99,108,97,115, + 101,110,100,114,116,0,0,0,218,9,95,95,99,108,97,115, 115,95,95,114,1,0,0,0,218,4,106,111,105,110,41,2, 114,30,0,0,0,114,55,0,0,0,114,10,0,0,0,114, 10,0,0,0,114,11,0,0,0,114,48,0,0,0,98,1, @@ -670,1120 +670,1122 @@ const unsigned char _Py_M__importlib_bootstrap[] = { 1,18,1,10,1,8,1,4,255,6,2,122,19,77,111,100, 117,108,101,83,112,101,99,46,95,95,114,101,112,114,95,95, 99,2,0,0,0,0,0,0,0,0,0,0,0,3,0,0, - 0,8,0,0,0,67,0,0,0,115,106,0,0,0,124,0, + 0,8,0,0,0,67,0,0,0,115,108,0,0,0,124,0, 106,0,125,2,122,72,124,0,106,1,124,1,106,1,107,2, 111,76,124,0,106,2,124,1,106,2,107,2,111,76,124,0, 106,3,124,1,106,3,107,2,111,76,124,2,124,1,106,0, 107,2,111,76,124,0,106,4,124,1,106,4,107,2,111,76, 124,0,106,5,124,1,106,5,107,2,87,0,83,0,4,0, - 116,6,107,10,114,100,1,0,1,0,1,0,89,0,100,1, - 83,0,88,0,100,0,83,0,114,116,0,0,0,41,7,114, - 117,0,0,0,114,17,0,0,0,114,109,0,0,0,114,113, - 0,0,0,218,6,99,97,99,104,101,100,218,12,104,97,115, - 95,108,111,99,97,116,105,111,110,114,106,0,0,0,41,3, - 114,30,0,0,0,90,5,111,116,104,101,114,90,4,115,109, - 115,108,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,218,6,95,95,101,113,95,95,108,1,0,0,115,30,0, - 0,0,0,1,6,1,2,1,12,1,10,255,2,2,10,254, - 2,3,8,253,2,4,10,252,2,5,10,251,4,6,14,1, - 122,17,77,111,100,117,108,101,83,112,101,99,46,95,95,101, - 113,95,95,99,1,0,0,0,0,0,0,0,0,0,0,0, - 1,0,0,0,3,0,0,0,67,0,0,0,115,58,0,0, - 0,124,0,106,0,100,0,107,8,114,52,124,0,106,1,100, - 0,107,9,114,52,124,0,106,2,114,52,116,3,100,0,107, - 8,114,38,116,4,130,1,116,3,160,5,124,0,106,1,161, - 1,124,0,95,0,124,0,106,0,83,0,114,13,0,0,0, - 41,6,114,119,0,0,0,114,113,0,0,0,114,118,0,0, - 0,218,19,95,98,111,111,116,115,116,114,97,112,95,101,120, - 116,101,114,110,97,108,218,19,78,111,116,73,109,112,108,101, - 109,101,110,116,101,100,69,114,114,111,114,90,11,95,103,101, - 116,95,99,97,99,104,101,100,114,47,0,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,114,123,0,0, - 0,120,1,0,0,115,12,0,0,0,0,2,10,1,16,1, - 8,1,4,1,14,1,122,17,77,111,100,117,108,101,83,112, - 101,99,46,99,97,99,104,101,100,99,2,0,0,0,0,0, - 0,0,0,0,0,0,2,0,0,0,2,0,0,0,67,0, - 0,0,115,10,0,0,0,124,1,124,0,95,0,100,0,83, - 0,114,13,0,0,0,41,1,114,119,0,0,0,41,2,114, - 30,0,0,0,114,123,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,114,123,0,0,0,129,1,0, - 0,115,2,0,0,0,0,2,99,1,0,0,0,0,0,0, - 0,0,0,0,0,1,0,0,0,3,0,0,0,67,0,0, - 0,115,36,0,0,0,124,0,106,0,100,1,107,8,114,26, - 124,0,106,1,160,2,100,2,161,1,100,3,25,0,83,0, - 124,0,106,1,83,0,100,1,83,0,41,4,122,32,84,104, - 101,32,110,97,109,101,32,111,102,32,116,104,101,32,109,111, - 100,117,108,101,39,115,32,112,97,114,101,110,116,46,78,218, - 1,46,114,22,0,0,0,41,3,114,117,0,0,0,114,17, - 0,0,0,218,10,114,112,97,114,116,105,116,105,111,110,114, + 116,6,107,10,114,102,1,0,1,0,1,0,116,7,6,0, + 89,0,83,0,88,0,100,0,83,0,114,13,0,0,0,41, + 8,114,116,0,0,0,114,17,0,0,0,114,109,0,0,0, + 114,113,0,0,0,218,6,99,97,99,104,101,100,218,12,104, + 97,115,95,108,111,99,97,116,105,111,110,114,106,0,0,0, + 218,14,78,111,116,73,109,112,108,101,109,101,110,116,101,100, + 41,3,114,30,0,0,0,90,5,111,116,104,101,114,90,4, + 115,109,115,108,114,10,0,0,0,114,10,0,0,0,114,11, + 0,0,0,218,6,95,95,101,113,95,95,108,1,0,0,115, + 30,0,0,0,0,1,6,1,2,1,12,1,10,255,2,2, + 10,254,2,3,8,253,2,4,10,252,2,5,10,251,4,6, + 14,1,122,17,77,111,100,117,108,101,83,112,101,99,46,95, + 95,101,113,95,95,99,1,0,0,0,0,0,0,0,0,0, + 0,0,1,0,0,0,3,0,0,0,67,0,0,0,115,58, + 0,0,0,124,0,106,0,100,0,107,8,114,52,124,0,106, + 1,100,0,107,9,114,52,124,0,106,2,114,52,116,3,100, + 0,107,8,114,38,116,4,130,1,116,3,160,5,124,0,106, + 1,161,1,124,0,95,0,124,0,106,0,83,0,114,13,0, + 0,0,41,6,114,118,0,0,0,114,113,0,0,0,114,117, + 0,0,0,218,19,95,98,111,111,116,115,116,114,97,112,95, + 101,120,116,101,114,110,97,108,218,19,78,111,116,73,109,112, + 108,101,109,101,110,116,101,100,69,114,114,111,114,90,11,95, + 103,101,116,95,99,97,99,104,101,100,114,47,0,0,0,114, + 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,122, + 0,0,0,120,1,0,0,115,12,0,0,0,0,2,10,1, + 16,1,8,1,4,1,14,1,122,17,77,111,100,117,108,101, + 83,112,101,99,46,99,97,99,104,101,100,99,2,0,0,0, + 0,0,0,0,0,0,0,0,2,0,0,0,2,0,0,0, + 67,0,0,0,115,10,0,0,0,124,1,124,0,95,0,100, + 0,83,0,114,13,0,0,0,41,1,114,118,0,0,0,41, + 2,114,30,0,0,0,114,122,0,0,0,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,114,122,0,0,0,129, + 1,0,0,115,2,0,0,0,0,2,99,1,0,0,0,0, + 0,0,0,0,0,0,0,1,0,0,0,3,0,0,0,67, + 0,0,0,115,36,0,0,0,124,0,106,0,100,1,107,8, + 114,26,124,0,106,1,160,2,100,2,161,1,100,3,25,0, + 83,0,124,0,106,1,83,0,100,1,83,0,41,4,122,32, + 84,104,101,32,110,97,109,101,32,111,102,32,116,104,101,32, + 109,111,100,117,108,101,39,115,32,112,97,114,101,110,116,46, + 78,218,1,46,114,22,0,0,0,41,3,114,116,0,0,0, + 114,17,0,0,0,218,10,114,112,97,114,116,105,116,105,111, + 110,114,47,0,0,0,114,10,0,0,0,114,10,0,0,0, + 114,11,0,0,0,218,6,112,97,114,101,110,116,133,1,0, + 0,115,6,0,0,0,0,3,10,1,16,2,122,17,77,111, + 100,117,108,101,83,112,101,99,46,112,97,114,101,110,116,99, + 1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0, + 1,0,0,0,67,0,0,0,115,6,0,0,0,124,0,106, + 0,83,0,114,13,0,0,0,41,1,114,117,0,0,0,114, 47,0,0,0,114,10,0,0,0,114,10,0,0,0,114,11, - 0,0,0,218,6,112,97,114,101,110,116,133,1,0,0,115, - 6,0,0,0,0,3,10,1,16,2,122,17,77,111,100,117, - 108,101,83,112,101,99,46,112,97,114,101,110,116,99,1,0, - 0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0, - 0,0,67,0,0,0,115,6,0,0,0,124,0,106,0,83, - 0,114,13,0,0,0,41,1,114,118,0,0,0,114,47,0, + 0,0,0,114,123,0,0,0,141,1,0,0,115,2,0,0, + 0,0,2,122,23,77,111,100,117,108,101,83,112,101,99,46, + 104,97,115,95,108,111,99,97,116,105,111,110,99,2,0,0, + 0,0,0,0,0,0,0,0,0,2,0,0,0,2,0,0, + 0,67,0,0,0,115,14,0,0,0,116,0,124,1,131,1, + 124,0,95,1,100,0,83,0,114,13,0,0,0,41,2,218, + 4,98,111,111,108,114,117,0,0,0,41,2,114,30,0,0, + 0,218,5,118,97,108,117,101,114,10,0,0,0,114,10,0, + 0,0,114,11,0,0,0,114,123,0,0,0,145,1,0,0, + 115,2,0,0,0,0,2,41,12,114,1,0,0,0,114,0, + 0,0,0,114,2,0,0,0,114,3,0,0,0,114,31,0, + 0,0,114,48,0,0,0,114,125,0,0,0,218,8,112,114, + 111,112,101,114,116,121,114,122,0,0,0,218,6,115,101,116, + 116,101,114,114,130,0,0,0,114,123,0,0,0,114,10,0, 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,124,0,0,0,141,1,0,0,115,2,0,0,0,0, - 2,122,23,77,111,100,117,108,101,83,112,101,99,46,104,97, - 115,95,108,111,99,97,116,105,111,110,99,2,0,0,0,0, - 0,0,0,0,0,0,0,2,0,0,0,2,0,0,0,67, - 0,0,0,115,14,0,0,0,116,0,124,1,131,1,124,0, - 95,1,100,0,83,0,114,13,0,0,0,41,2,218,4,98, - 111,111,108,114,118,0,0,0,41,2,114,30,0,0,0,218, - 5,118,97,108,117,101,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,114,124,0,0,0,145,1,0,0,115,2, - 0,0,0,0,2,41,12,114,1,0,0,0,114,0,0,0, - 0,114,2,0,0,0,114,3,0,0,0,114,31,0,0,0, - 114,48,0,0,0,114,125,0,0,0,218,8,112,114,111,112, - 101,114,116,121,114,123,0,0,0,218,6,115,101,116,116,101, - 114,114,130,0,0,0,114,124,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 112,0,0,0,49,1,0,0,115,32,0,0,0,8,1,4, - 36,4,1,2,255,12,12,8,10,8,12,2,1,10,8,4, - 1,10,3,2,1,10,7,2,1,10,3,4,1,114,112,0, - 0,0,169,2,114,113,0,0,0,114,115,0,0,0,99,2, - 0,0,0,0,0,0,0,2,0,0,0,6,0,0,0,8, - 0,0,0,67,0,0,0,115,154,0,0,0,116,0,124,1, - 100,1,131,2,114,74,116,1,100,2,107,8,114,22,116,2, - 130,1,116,1,106,3,125,4,124,3,100,2,107,8,114,48, - 124,4,124,0,124,1,100,3,141,2,83,0,124,3,114,56, - 103,0,110,2,100,2,125,5,124,4,124,0,124,1,124,5, - 100,4,141,3,83,0,124,3,100,2,107,8,114,138,116,0, - 124,1,100,5,131,2,114,134,122,14,124,1,160,4,124,0, - 161,1,125,3,87,0,113,138,4,0,116,5,107,10,114,130, - 1,0,1,0,1,0,100,2,125,3,89,0,113,138,88,0, - 110,4,100,6,125,3,116,6,124,0,124,1,124,2,124,3, - 100,7,141,4,83,0,41,8,122,53,82,101,116,117,114,110, - 32,97,32,109,111,100,117,108,101,32,115,112,101,99,32,98, - 97,115,101,100,32,111,110,32,118,97,114,105,111,117,115,32, - 108,111,97,100,101,114,32,109,101,116,104,111,100,115,46,90, - 12,103,101,116,95,102,105,108,101,110,97,109,101,78,41,1, - 114,109,0,0,0,41,2,114,109,0,0,0,114,117,0,0, - 0,114,115,0,0,0,70,114,135,0,0,0,41,7,114,4, - 0,0,0,114,126,0,0,0,114,127,0,0,0,218,23,115, - 112,101,99,95,102,114,111,109,95,102,105,108,101,95,108,111, - 99,97,116,105,111,110,114,115,0,0,0,114,79,0,0,0, - 114,112,0,0,0,41,6,114,17,0,0,0,114,109,0,0, - 0,114,113,0,0,0,114,115,0,0,0,114,136,0,0,0, - 90,6,115,101,97,114,99,104,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,114,91,0,0,0,150,1,0,0, - 115,36,0,0,0,0,2,10,1,8,1,4,1,6,2,8, - 1,12,1,12,1,6,1,2,255,6,3,8,1,10,1,2, - 1,14,1,14,1,12,3,4,2,114,91,0,0,0,99,3, - 0,0,0,0,0,0,0,0,0,0,0,8,0,0,0,8, - 0,0,0,67,0,0,0,115,56,1,0,0,122,10,124,0, - 106,0,125,3,87,0,110,20,4,0,116,1,107,10,114,30, - 1,0,1,0,1,0,89,0,110,14,88,0,124,3,100,0, - 107,9,114,44,124,3,83,0,124,0,106,2,125,4,124,1, - 100,0,107,8,114,90,122,10,124,0,106,3,125,1,87,0, - 110,20,4,0,116,1,107,10,114,88,1,0,1,0,1,0, - 89,0,110,2,88,0,122,10,124,0,106,4,125,5,87,0, - 110,24,4,0,116,1,107,10,114,124,1,0,1,0,1,0, - 100,0,125,5,89,0,110,2,88,0,124,2,100,0,107,8, - 114,184,124,5,100,0,107,8,114,180,122,10,124,1,106,5, - 125,2,87,0,113,184,4,0,116,1,107,10,114,176,1,0, - 1,0,1,0,100,0,125,2,89,0,113,184,88,0,110,4, - 124,5,125,2,122,10,124,0,106,6,125,6,87,0,110,24, - 4,0,116,1,107,10,114,218,1,0,1,0,1,0,100,0, - 125,6,89,0,110,2,88,0,122,14,116,7,124,0,106,8, - 131,1,125,7,87,0,110,26,4,0,116,1,107,10,144,1, - 114,4,1,0,1,0,1,0,100,0,125,7,89,0,110,2, - 88,0,116,9,124,4,124,1,124,2,100,1,141,3,125,3, - 124,5,100,0,107,8,144,1,114,34,100,2,110,2,100,3, - 124,3,95,10,124,6,124,3,95,11,124,7,124,3,95,12, - 124,3,83,0,41,4,78,169,1,114,113,0,0,0,70,84, - 41,13,114,105,0,0,0,114,106,0,0,0,114,1,0,0, - 0,114,98,0,0,0,114,108,0,0,0,218,7,95,79,82, - 73,71,73,78,218,10,95,95,99,97,99,104,101,100,95,95, - 218,4,108,105,115,116,218,8,95,95,112,97,116,104,95,95, - 114,112,0,0,0,114,118,0,0,0,114,123,0,0,0,114, - 117,0,0,0,41,8,114,96,0,0,0,114,109,0,0,0, - 114,113,0,0,0,114,95,0,0,0,114,17,0,0,0,90, - 8,108,111,99,97,116,105,111,110,114,123,0,0,0,114,117, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,218,17,95,115,112,101,99,95,102,114,111,109,95,109, - 111,100,117,108,101,176,1,0,0,115,72,0,0,0,0,2, - 2,1,10,1,14,1,6,2,8,1,4,2,6,1,8,1, - 2,1,10,1,14,2,6,1,2,1,10,1,14,1,10,1, - 8,1,8,1,2,1,10,1,14,1,12,2,4,1,2,1, - 10,1,14,1,10,1,2,1,14,1,16,1,10,2,14,1, - 20,1,6,1,6,1,114,142,0,0,0,70,169,1,218,8, - 111,118,101,114,114,105,100,101,99,2,0,0,0,0,0,0, - 0,1,0,0,0,5,0,0,0,8,0,0,0,67,0,0, - 0,115,226,1,0,0,124,2,115,20,116,0,124,1,100,1, - 100,0,131,3,100,0,107,8,114,54,122,12,124,0,106,1, - 124,1,95,2,87,0,110,20,4,0,116,3,107,10,114,52, - 1,0,1,0,1,0,89,0,110,2,88,0,124,2,115,74, - 116,0,124,1,100,2,100,0,131,3,100,0,107,8,114,178, - 124,0,106,4,125,3,124,3,100,0,107,8,114,146,124,0, - 106,5,100,0,107,9,114,146,116,6,100,0,107,8,114,110, - 116,7,130,1,116,6,106,8,125,4,124,4,160,9,124,4, - 161,1,125,3,124,0,106,5,124,3,95,10,124,3,124,0, - 95,4,100,0,124,1,95,11,122,10,124,3,124,1,95,12, - 87,0,110,20,4,0,116,3,107,10,114,176,1,0,1,0, - 1,0,89,0,110,2,88,0,124,2,115,198,116,0,124,1, - 100,3,100,0,131,3,100,0,107,8,114,232,122,12,124,0, - 106,13,124,1,95,14,87,0,110,20,4,0,116,3,107,10, - 114,230,1,0,1,0,1,0,89,0,110,2,88,0,122,10, - 124,0,124,1,95,15,87,0,110,22,4,0,116,3,107,10, - 144,1,114,8,1,0,1,0,1,0,89,0,110,2,88,0, - 124,2,144,1,115,34,116,0,124,1,100,4,100,0,131,3, - 100,0,107,8,144,1,114,82,124,0,106,5,100,0,107,9, - 144,1,114,82,122,12,124,0,106,5,124,1,95,16,87,0, - 110,22,4,0,116,3,107,10,144,1,114,80,1,0,1,0, - 1,0,89,0,110,2,88,0,124,0,106,17,144,1,114,222, - 124,2,144,1,115,114,116,0,124,1,100,5,100,0,131,3, - 100,0,107,8,144,1,114,150,122,12,124,0,106,18,124,1, - 95,11,87,0,110,22,4,0,116,3,107,10,144,1,114,148, - 1,0,1,0,1,0,89,0,110,2,88,0,124,2,144,1, - 115,174,116,0,124,1,100,6,100,0,131,3,100,0,107,8, - 144,1,114,222,124,0,106,19,100,0,107,9,144,1,114,222, - 122,12,124,0,106,19,124,1,95,20,87,0,110,22,4,0, - 116,3,107,10,144,1,114,220,1,0,1,0,1,0,89,0, - 110,2,88,0,124,1,83,0,41,7,78,114,1,0,0,0, - 114,98,0,0,0,218,11,95,95,112,97,99,107,97,103,101, - 95,95,114,141,0,0,0,114,108,0,0,0,114,139,0,0, - 0,41,21,114,6,0,0,0,114,17,0,0,0,114,1,0, - 0,0,114,106,0,0,0,114,109,0,0,0,114,117,0,0, - 0,114,126,0,0,0,114,127,0,0,0,218,16,95,78,97, - 109,101,115,112,97,99,101,76,111,97,100,101,114,218,7,95, - 95,110,101,119,95,95,90,5,95,112,97,116,104,114,108,0, - 0,0,114,98,0,0,0,114,130,0,0,0,114,145,0,0, - 0,114,105,0,0,0,114,141,0,0,0,114,124,0,0,0, - 114,113,0,0,0,114,123,0,0,0,114,139,0,0,0,41, - 5,114,95,0,0,0,114,96,0,0,0,114,144,0,0,0, - 114,109,0,0,0,114,146,0,0,0,114,10,0,0,0,114, - 10,0,0,0,114,11,0,0,0,218,18,95,105,110,105,116, - 95,109,111,100,117,108,101,95,97,116,116,114,115,221,1,0, - 0,115,96,0,0,0,0,4,20,1,2,1,12,1,14,1, - 6,2,20,1,6,1,8,2,10,1,8,1,4,1,6,2, - 10,1,8,1,6,11,6,1,2,1,10,1,14,1,6,2, - 20,1,2,1,12,1,14,1,6,2,2,1,10,1,16,1, - 6,2,24,1,12,1,2,1,12,1,16,1,6,2,8,1, - 24,1,2,1,12,1,16,1,6,2,24,1,12,1,2,1, - 12,1,16,1,6,1,114,148,0,0,0,99,1,0,0,0, - 0,0,0,0,0,0,0,0,2,0,0,0,3,0,0,0, - 67,0,0,0,115,82,0,0,0,100,1,125,1,116,0,124, - 0,106,1,100,2,131,2,114,30,124,0,106,1,160,2,124, - 0,161,1,125,1,110,20,116,0,124,0,106,1,100,3,131, - 2,114,50,116,3,100,4,131,1,130,1,124,1,100,1,107, - 8,114,68,116,4,124,0,106,5,131,1,125,1,116,6,124, - 0,124,1,131,2,1,0,124,1,83,0,41,5,122,43,67, - 114,101,97,116,101,32,97,32,109,111,100,117,108,101,32,98, - 97,115,101,100,32,111,110,32,116,104,101,32,112,114,111,118, - 105,100,101,100,32,115,112,101,99,46,78,218,13,99,114,101, - 97,116,101,95,109,111,100,117,108,101,218,11,101,120,101,99, - 95,109,111,100,117,108,101,122,66,108,111,97,100,101,114,115, - 32,116,104,97,116,32,100,101,102,105,110,101,32,101,120,101, - 99,95,109,111,100,117,108,101,40,41,32,109,117,115,116,32, - 97,108,115,111,32,100,101,102,105,110,101,32,99,114,101,97, - 116,101,95,109,111,100,117,108,101,40,41,41,7,114,4,0, - 0,0,114,109,0,0,0,114,149,0,0,0,114,79,0,0, - 0,114,18,0,0,0,114,17,0,0,0,114,148,0,0,0, - 169,2,114,95,0,0,0,114,96,0,0,0,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,218,16,109,111,100, - 117,108,101,95,102,114,111,109,95,115,112,101,99,37,2,0, - 0,115,18,0,0,0,0,3,4,1,12,3,14,1,12,1, - 8,2,8,1,10,1,10,1,114,152,0,0,0,99,1,0, - 0,0,0,0,0,0,0,0,0,0,2,0,0,0,4,0, - 0,0,67,0,0,0,115,106,0,0,0,124,0,106,0,100, - 1,107,8,114,14,100,2,110,4,124,0,106,0,125,1,124, - 0,106,1,100,1,107,8,114,66,124,0,106,2,100,1,107, - 8,114,50,100,3,160,3,124,1,161,1,83,0,100,4,160, - 3,124,1,124,0,106,2,161,2,83,0,110,36,124,0,106, - 4,114,86,100,5,160,3,124,1,124,0,106,1,161,2,83, - 0,100,6,160,3,124,0,106,0,124,0,106,1,161,2,83, - 0,100,1,83,0,41,7,122,38,82,101,116,117,114,110,32, - 116,104,101,32,114,101,112,114,32,116,111,32,117,115,101,32, - 102,111,114,32,116,104,101,32,109,111,100,117,108,101,46,78, - 114,100,0,0,0,114,101,0,0,0,114,102,0,0,0,114, - 103,0,0,0,250,18,60,109,111,100,117,108,101,32,123,33, - 114,125,32,40,123,125,41,62,41,5,114,17,0,0,0,114, - 113,0,0,0,114,109,0,0,0,114,45,0,0,0,114,124, - 0,0,0,41,2,114,95,0,0,0,114,17,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,114,107, - 0,0,0,54,2,0,0,115,16,0,0,0,0,3,20,1, - 10,1,10,1,10,2,16,2,6,1,14,2,114,107,0,0, - 0,99,2,0,0,0,0,0,0,0,0,0,0,0,4,0, - 0,0,10,0,0,0,67,0,0,0,115,204,0,0,0,124, - 0,106,0,125,2,116,1,124,2,131,1,143,180,1,0,116, - 2,106,3,160,4,124,2,161,1,124,1,107,9,114,54,100, - 1,160,5,124,2,161,1,125,3,116,6,124,3,124,2,100, - 2,141,2,130,1,122,106,124,0,106,7,100,3,107,8,114, - 106,124,0,106,8,100,3,107,8,114,90,116,6,100,4,124, - 0,106,0,100,2,141,2,130,1,116,9,124,0,124,1,100, - 5,100,6,141,3,1,0,110,52,116,9,124,0,124,1,100, - 5,100,6,141,3,1,0,116,10,124,0,106,7,100,7,131, - 2,115,146,124,0,106,7,160,11,124,2,161,1,1,0,110, - 12,124,0,106,7,160,12,124,1,161,1,1,0,87,0,53, - 0,116,2,106,3,160,13,124,0,106,0,161,1,125,1,124, - 1,116,2,106,3,124,0,106,0,60,0,88,0,87,0,53, - 0,81,0,82,0,88,0,124,1,83,0,41,8,122,70,69, - 120,101,99,117,116,101,32,116,104,101,32,115,112,101,99,39, - 115,32,115,112,101,99,105,102,105,101,100,32,109,111,100,117, - 108,101,32,105,110,32,97,110,32,101,120,105,115,116,105,110, - 103,32,109,111,100,117,108,101,39,115,32,110,97,109,101,115, - 112,97,99,101,46,122,30,109,111,100,117,108,101,32,123,33, - 114,125,32,110,111,116,32,105,110,32,115,121,115,46,109,111, - 100,117,108,101,115,114,16,0,0,0,78,250,14,109,105,115, - 115,105,110,103,32,108,111,97,100,101,114,84,114,143,0,0, - 0,114,150,0,0,0,41,14,114,17,0,0,0,114,50,0, - 0,0,114,15,0,0,0,114,92,0,0,0,114,34,0,0, - 0,114,45,0,0,0,114,79,0,0,0,114,109,0,0,0, - 114,117,0,0,0,114,148,0,0,0,114,4,0,0,0,218, - 11,108,111,97,100,95,109,111,100,117,108,101,114,150,0,0, - 0,218,3,112,111,112,41,4,114,95,0,0,0,114,96,0, - 0,0,114,17,0,0,0,218,3,109,115,103,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,93,0,0,0, - 71,2,0,0,115,34,0,0,0,0,2,6,1,10,1,16, - 1,10,1,12,1,2,1,10,1,10,1,14,2,16,2,14, - 1,12,4,14,2,16,4,14,1,24,1,114,93,0,0,0, - 99,1,0,0,0,0,0,0,0,0,0,0,0,2,0,0, - 0,8,0,0,0,67,0,0,0,115,26,1,0,0,122,18, - 124,0,106,0,160,1,124,0,106,2,161,1,1,0,87,0, - 110,52,1,0,1,0,1,0,124,0,106,2,116,3,106,4, - 107,6,114,64,116,3,106,4,160,5,124,0,106,2,161,1, - 125,1,124,1,116,3,106,4,124,0,106,2,60,0,130,0, - 89,0,110,2,88,0,116,3,106,4,160,5,124,0,106,2, + 0,114,112,0,0,0,49,1,0,0,115,32,0,0,0,8, + 1,4,36,4,1,2,255,12,12,8,10,8,12,2,1,10, + 8,4,1,10,3,2,1,10,7,2,1,10,3,4,1,114, + 112,0,0,0,169,2,114,113,0,0,0,114,115,0,0,0, + 99,2,0,0,0,0,0,0,0,2,0,0,0,6,0,0, + 0,8,0,0,0,67,0,0,0,115,154,0,0,0,116,0, + 124,1,100,1,131,2,114,74,116,1,100,2,107,8,114,22, + 116,2,130,1,116,1,106,3,125,4,124,3,100,2,107,8, + 114,48,124,4,124,0,124,1,100,3,141,2,83,0,124,3, + 114,56,103,0,110,2,100,2,125,5,124,4,124,0,124,1, + 124,5,100,4,141,3,83,0,124,3,100,2,107,8,114,138, + 116,0,124,1,100,5,131,2,114,134,122,14,124,1,160,4, + 124,0,161,1,125,3,87,0,113,138,4,0,116,5,107,10, + 114,130,1,0,1,0,1,0,100,2,125,3,89,0,113,138, + 88,0,110,4,100,6,125,3,116,6,124,0,124,1,124,2, + 124,3,100,7,141,4,83,0,41,8,122,53,82,101,116,117, + 114,110,32,97,32,109,111,100,117,108,101,32,115,112,101,99, + 32,98,97,115,101,100,32,111,110,32,118,97,114,105,111,117, + 115,32,108,111,97,100,101,114,32,109,101,116,104,111,100,115, + 46,90,12,103,101,116,95,102,105,108,101,110,97,109,101,78, + 41,1,114,109,0,0,0,41,2,114,109,0,0,0,114,116, + 0,0,0,114,115,0,0,0,70,114,135,0,0,0,41,7, + 114,4,0,0,0,114,126,0,0,0,114,127,0,0,0,218, + 23,115,112,101,99,95,102,114,111,109,95,102,105,108,101,95, + 108,111,99,97,116,105,111,110,114,115,0,0,0,114,79,0, + 0,0,114,112,0,0,0,41,6,114,17,0,0,0,114,109, + 0,0,0,114,113,0,0,0,114,115,0,0,0,114,136,0, + 0,0,90,6,115,101,97,114,99,104,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,114,91,0,0,0,150,1, + 0,0,115,36,0,0,0,0,2,10,1,8,1,4,1,6, + 2,8,1,12,1,12,1,6,1,2,255,6,3,8,1,10, + 1,2,1,14,1,14,1,12,3,4,2,114,91,0,0,0, + 99,3,0,0,0,0,0,0,0,0,0,0,0,8,0,0, + 0,8,0,0,0,67,0,0,0,115,56,1,0,0,122,10, + 124,0,106,0,125,3,87,0,110,20,4,0,116,1,107,10, + 114,30,1,0,1,0,1,0,89,0,110,14,88,0,124,3, + 100,0,107,9,114,44,124,3,83,0,124,0,106,2,125,4, + 124,1,100,0,107,8,114,90,122,10,124,0,106,3,125,1, + 87,0,110,20,4,0,116,1,107,10,114,88,1,0,1,0, + 1,0,89,0,110,2,88,0,122,10,124,0,106,4,125,5, + 87,0,110,24,4,0,116,1,107,10,114,124,1,0,1,0, + 1,0,100,0,125,5,89,0,110,2,88,0,124,2,100,0, + 107,8,114,184,124,5,100,0,107,8,114,180,122,10,124,1, + 106,5,125,2,87,0,113,184,4,0,116,1,107,10,114,176, + 1,0,1,0,1,0,100,0,125,2,89,0,113,184,88,0, + 110,4,124,5,125,2,122,10,124,0,106,6,125,6,87,0, + 110,24,4,0,116,1,107,10,114,218,1,0,1,0,1,0, + 100,0,125,6,89,0,110,2,88,0,122,14,116,7,124,0, + 106,8,131,1,125,7,87,0,110,26,4,0,116,1,107,10, + 144,1,114,4,1,0,1,0,1,0,100,0,125,7,89,0, + 110,2,88,0,116,9,124,4,124,1,124,2,100,1,141,3, + 125,3,124,5,100,0,107,8,144,1,114,34,100,2,110,2, + 100,3,124,3,95,10,124,6,124,3,95,11,124,7,124,3, + 95,12,124,3,83,0,41,4,78,169,1,114,113,0,0,0, + 70,84,41,13,114,105,0,0,0,114,106,0,0,0,114,1, + 0,0,0,114,98,0,0,0,114,108,0,0,0,218,7,95, + 79,82,73,71,73,78,218,10,95,95,99,97,99,104,101,100, + 95,95,218,4,108,105,115,116,218,8,95,95,112,97,116,104, + 95,95,114,112,0,0,0,114,117,0,0,0,114,122,0,0, + 0,114,116,0,0,0,41,8,114,96,0,0,0,114,109,0, + 0,0,114,113,0,0,0,114,95,0,0,0,114,17,0,0, + 0,90,8,108,111,99,97,116,105,111,110,114,122,0,0,0, + 114,116,0,0,0,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,218,17,95,115,112,101,99,95,102,114,111,109, + 95,109,111,100,117,108,101,176,1,0,0,115,72,0,0,0, + 0,2,2,1,10,1,14,1,6,2,8,1,4,2,6,1, + 8,1,2,1,10,1,14,2,6,1,2,1,10,1,14,1, + 10,1,8,1,8,1,2,1,10,1,14,1,12,2,4,1, + 2,1,10,1,14,1,10,1,2,1,14,1,16,1,10,2, + 14,1,20,1,6,1,6,1,114,142,0,0,0,70,169,1, + 218,8,111,118,101,114,114,105,100,101,99,2,0,0,0,0, + 0,0,0,1,0,0,0,5,0,0,0,8,0,0,0,67, + 0,0,0,115,226,1,0,0,124,2,115,20,116,0,124,1, + 100,1,100,0,131,3,100,0,107,8,114,54,122,12,124,0, + 106,1,124,1,95,2,87,0,110,20,4,0,116,3,107,10, + 114,52,1,0,1,0,1,0,89,0,110,2,88,0,124,2, + 115,74,116,0,124,1,100,2,100,0,131,3,100,0,107,8, + 114,178,124,0,106,4,125,3,124,3,100,0,107,8,114,146, + 124,0,106,5,100,0,107,9,114,146,116,6,100,0,107,8, + 114,110,116,7,130,1,116,6,106,8,125,4,124,4,160,9, + 124,4,161,1,125,3,124,0,106,5,124,3,95,10,124,3, + 124,0,95,4,100,0,124,1,95,11,122,10,124,3,124,1, + 95,12,87,0,110,20,4,0,116,3,107,10,114,176,1,0, + 1,0,1,0,89,0,110,2,88,0,124,2,115,198,116,0, + 124,1,100,3,100,0,131,3,100,0,107,8,114,232,122,12, + 124,0,106,13,124,1,95,14,87,0,110,20,4,0,116,3, + 107,10,114,230,1,0,1,0,1,0,89,0,110,2,88,0, + 122,10,124,0,124,1,95,15,87,0,110,22,4,0,116,3, + 107,10,144,1,114,8,1,0,1,0,1,0,89,0,110,2, + 88,0,124,2,144,1,115,34,116,0,124,1,100,4,100,0, + 131,3,100,0,107,8,144,1,114,82,124,0,106,5,100,0, + 107,9,144,1,114,82,122,12,124,0,106,5,124,1,95,16, + 87,0,110,22,4,0,116,3,107,10,144,1,114,80,1,0, + 1,0,1,0,89,0,110,2,88,0,124,0,106,17,144,1, + 114,222,124,2,144,1,115,114,116,0,124,1,100,5,100,0, + 131,3,100,0,107,8,144,1,114,150,122,12,124,0,106,18, + 124,1,95,11,87,0,110,22,4,0,116,3,107,10,144,1, + 114,148,1,0,1,0,1,0,89,0,110,2,88,0,124,2, + 144,1,115,174,116,0,124,1,100,6,100,0,131,3,100,0, + 107,8,144,1,114,222,124,0,106,19,100,0,107,9,144,1, + 114,222,122,12,124,0,106,19,124,1,95,20,87,0,110,22, + 4,0,116,3,107,10,144,1,114,220,1,0,1,0,1,0, + 89,0,110,2,88,0,124,1,83,0,41,7,78,114,1,0, + 0,0,114,98,0,0,0,218,11,95,95,112,97,99,107,97, + 103,101,95,95,114,141,0,0,0,114,108,0,0,0,114,139, + 0,0,0,41,21,114,6,0,0,0,114,17,0,0,0,114, + 1,0,0,0,114,106,0,0,0,114,109,0,0,0,114,116, + 0,0,0,114,126,0,0,0,114,127,0,0,0,218,16,95, + 78,97,109,101,115,112,97,99,101,76,111,97,100,101,114,218, + 7,95,95,110,101,119,95,95,90,5,95,112,97,116,104,114, + 108,0,0,0,114,98,0,0,0,114,130,0,0,0,114,145, + 0,0,0,114,105,0,0,0,114,141,0,0,0,114,123,0, + 0,0,114,113,0,0,0,114,122,0,0,0,114,139,0,0, + 0,41,5,114,95,0,0,0,114,96,0,0,0,114,144,0, + 0,0,114,109,0,0,0,114,146,0,0,0,114,10,0,0, + 0,114,10,0,0,0,114,11,0,0,0,218,18,95,105,110, + 105,116,95,109,111,100,117,108,101,95,97,116,116,114,115,221, + 1,0,0,115,96,0,0,0,0,4,20,1,2,1,12,1, + 14,1,6,2,20,1,6,1,8,2,10,1,8,1,4,1, + 6,2,10,1,8,1,6,11,6,1,2,1,10,1,14,1, + 6,2,20,1,2,1,12,1,14,1,6,2,2,1,10,1, + 16,1,6,2,24,1,12,1,2,1,12,1,16,1,6,2, + 8,1,24,1,2,1,12,1,16,1,6,2,24,1,12,1, + 2,1,12,1,16,1,6,1,114,148,0,0,0,99,1,0, + 0,0,0,0,0,0,0,0,0,0,2,0,0,0,3,0, + 0,0,67,0,0,0,115,82,0,0,0,100,1,125,1,116, + 0,124,0,106,1,100,2,131,2,114,30,124,0,106,1,160, + 2,124,0,161,1,125,1,110,20,116,0,124,0,106,1,100, + 3,131,2,114,50,116,3,100,4,131,1,130,1,124,1,100, + 1,107,8,114,68,116,4,124,0,106,5,131,1,125,1,116, + 6,124,0,124,1,131,2,1,0,124,1,83,0,41,5,122, + 43,67,114,101,97,116,101,32,97,32,109,111,100,117,108,101, + 32,98,97,115,101,100,32,111,110,32,116,104,101,32,112,114, + 111,118,105,100,101,100,32,115,112,101,99,46,78,218,13,99, + 114,101,97,116,101,95,109,111,100,117,108,101,218,11,101,120, + 101,99,95,109,111,100,117,108,101,122,66,108,111,97,100,101, + 114,115,32,116,104,97,116,32,100,101,102,105,110,101,32,101, + 120,101,99,95,109,111,100,117,108,101,40,41,32,109,117,115, + 116,32,97,108,115,111,32,100,101,102,105,110,101,32,99,114, + 101,97,116,101,95,109,111,100,117,108,101,40,41,41,7,114, + 4,0,0,0,114,109,0,0,0,114,149,0,0,0,114,79, + 0,0,0,114,18,0,0,0,114,17,0,0,0,114,148,0, + 0,0,169,2,114,95,0,0,0,114,96,0,0,0,114,10, + 0,0,0,114,10,0,0,0,114,11,0,0,0,218,16,109, + 111,100,117,108,101,95,102,114,111,109,95,115,112,101,99,37, + 2,0,0,115,18,0,0,0,0,3,4,1,12,3,14,1, + 12,1,8,2,8,1,10,1,10,1,114,152,0,0,0,99, + 1,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0, + 4,0,0,0,67,0,0,0,115,106,0,0,0,124,0,106, + 0,100,1,107,8,114,14,100,2,110,4,124,0,106,0,125, + 1,124,0,106,1,100,1,107,8,114,66,124,0,106,2,100, + 1,107,8,114,50,100,3,160,3,124,1,161,1,83,0,100, + 4,160,3,124,1,124,0,106,2,161,2,83,0,110,36,124, + 0,106,4,114,86,100,5,160,3,124,1,124,0,106,1,161, + 2,83,0,100,6,160,3,124,0,106,0,124,0,106,1,161, + 2,83,0,100,1,83,0,41,7,122,38,82,101,116,117,114, + 110,32,116,104,101,32,114,101,112,114,32,116,111,32,117,115, + 101,32,102,111,114,32,116,104,101,32,109,111,100,117,108,101, + 46,78,114,100,0,0,0,114,101,0,0,0,114,102,0,0, + 0,114,103,0,0,0,250,18,60,109,111,100,117,108,101,32, + 123,33,114,125,32,40,123,125,41,62,41,5,114,17,0,0, + 0,114,113,0,0,0,114,109,0,0,0,114,45,0,0,0, + 114,123,0,0,0,41,2,114,95,0,0,0,114,17,0,0, + 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, + 114,107,0,0,0,54,2,0,0,115,16,0,0,0,0,3, + 20,1,10,1,10,1,10,2,16,2,6,1,14,2,114,107, + 0,0,0,99,2,0,0,0,0,0,0,0,0,0,0,0, + 4,0,0,0,10,0,0,0,67,0,0,0,115,204,0,0, + 0,124,0,106,0,125,2,116,1,124,2,131,1,143,180,1, + 0,116,2,106,3,160,4,124,2,161,1,124,1,107,9,114, + 54,100,1,160,5,124,2,161,1,125,3,116,6,124,3,124, + 2,100,2,141,2,130,1,122,106,124,0,106,7,100,3,107, + 8,114,106,124,0,106,8,100,3,107,8,114,90,116,6,100, + 4,124,0,106,0,100,2,141,2,130,1,116,9,124,0,124, + 1,100,5,100,6,141,3,1,0,110,52,116,9,124,0,124, + 1,100,5,100,6,141,3,1,0,116,10,124,0,106,7,100, + 7,131,2,115,146,124,0,106,7,160,11,124,2,161,1,1, + 0,110,12,124,0,106,7,160,12,124,1,161,1,1,0,87, + 0,53,0,116,2,106,3,160,13,124,0,106,0,161,1,125, + 1,124,1,116,2,106,3,124,0,106,0,60,0,88,0,87, + 0,53,0,81,0,82,0,88,0,124,1,83,0,41,8,122, + 70,69,120,101,99,117,116,101,32,116,104,101,32,115,112,101, + 99,39,115,32,115,112,101,99,105,102,105,101,100,32,109,111, + 100,117,108,101,32,105,110,32,97,110,32,101,120,105,115,116, + 105,110,103,32,109,111,100,117,108,101,39,115,32,110,97,109, + 101,115,112,97,99,101,46,122,30,109,111,100,117,108,101,32, + 123,33,114,125,32,110,111,116,32,105,110,32,115,121,115,46, + 109,111,100,117,108,101,115,114,16,0,0,0,78,250,14,109, + 105,115,115,105,110,103,32,108,111,97,100,101,114,84,114,143, + 0,0,0,114,150,0,0,0,41,14,114,17,0,0,0,114, + 50,0,0,0,114,15,0,0,0,114,92,0,0,0,114,34, + 0,0,0,114,45,0,0,0,114,79,0,0,0,114,109,0, + 0,0,114,116,0,0,0,114,148,0,0,0,114,4,0,0, + 0,218,11,108,111,97,100,95,109,111,100,117,108,101,114,150, + 0,0,0,218,3,112,111,112,41,4,114,95,0,0,0,114, + 96,0,0,0,114,17,0,0,0,218,3,109,115,103,114,10, + 0,0,0,114,10,0,0,0,114,11,0,0,0,114,93,0, + 0,0,71,2,0,0,115,34,0,0,0,0,2,6,1,10, + 1,16,1,10,1,12,1,2,1,10,1,10,1,14,2,16, + 2,14,1,12,4,14,2,16,4,14,1,24,1,114,93,0, + 0,0,99,1,0,0,0,0,0,0,0,0,0,0,0,2, + 0,0,0,8,0,0,0,67,0,0,0,115,26,1,0,0, + 122,18,124,0,106,0,160,1,124,0,106,2,161,1,1,0, + 87,0,110,52,1,0,1,0,1,0,124,0,106,2,116,3, + 106,4,107,6,114,64,116,3,106,4,160,5,124,0,106,2, 161,1,125,1,124,1,116,3,106,4,124,0,106,2,60,0, - 116,6,124,1,100,1,100,0,131,3,100,0,107,8,114,148, - 122,12,124,0,106,0,124,1,95,7,87,0,110,20,4,0, - 116,8,107,10,114,146,1,0,1,0,1,0,89,0,110,2, - 88,0,116,6,124,1,100,2,100,0,131,3,100,0,107,8, - 114,226,122,40,124,1,106,9,124,1,95,10,116,11,124,1, - 100,3,131,2,115,202,124,0,106,2,160,12,100,4,161,1, - 100,5,25,0,124,1,95,10,87,0,110,20,4,0,116,8, - 107,10,114,224,1,0,1,0,1,0,89,0,110,2,88,0, - 116,6,124,1,100,6,100,0,131,3,100,0,107,8,144,1, - 114,22,122,10,124,0,124,1,95,13,87,0,110,22,4,0, - 116,8,107,10,144,1,114,20,1,0,1,0,1,0,89,0, - 110,2,88,0,124,1,83,0,41,7,78,114,98,0,0,0, - 114,145,0,0,0,114,141,0,0,0,114,128,0,0,0,114, - 22,0,0,0,114,105,0,0,0,41,14,114,109,0,0,0, - 114,155,0,0,0,114,17,0,0,0,114,15,0,0,0,114, - 92,0,0,0,114,156,0,0,0,114,6,0,0,0,114,98, - 0,0,0,114,106,0,0,0,114,1,0,0,0,114,145,0, - 0,0,114,4,0,0,0,114,129,0,0,0,114,105,0,0, - 0,114,151,0,0,0,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,218,25,95,108,111,97,100,95,98,97,99, - 107,119,97,114,100,95,99,111,109,112,97,116,105,98,108,101, - 101,2,0,0,115,54,0,0,0,0,4,2,1,18,1,6, - 1,12,1,14,1,12,1,8,3,14,1,12,1,16,1,2, - 1,12,1,14,1,6,1,16,1,2,4,8,1,10,1,22, - 1,14,1,6,1,18,1,2,1,10,1,16,1,6,1,114, - 158,0,0,0,99,1,0,0,0,0,0,0,0,0,0,0, - 0,2,0,0,0,11,0,0,0,67,0,0,0,115,220,0, - 0,0,124,0,106,0,100,0,107,9,114,30,116,1,124,0, - 106,0,100,1,131,2,115,30,116,2,124,0,131,1,83,0, - 116,3,124,0,131,1,125,1,100,2,124,0,95,4,122,162, - 124,1,116,5,106,6,124,0,106,7,60,0,122,52,124,0, - 106,0,100,0,107,8,114,96,124,0,106,8,100,0,107,8, - 114,108,116,9,100,3,124,0,106,7,100,4,141,2,130,1, - 110,12,124,0,106,0,160,10,124,1,161,1,1,0,87,0, - 110,50,1,0,1,0,1,0,122,14,116,5,106,6,124,0, - 106,7,61,0,87,0,110,20,4,0,116,11,107,10,114,152, - 1,0,1,0,1,0,89,0,110,2,88,0,130,0,89,0, - 110,2,88,0,116,5,106,6,160,12,124,0,106,7,161,1, - 125,1,124,1,116,5,106,6,124,0,106,7,60,0,116,13, - 100,5,124,0,106,7,124,0,106,0,131,3,1,0,87,0, - 53,0,100,6,124,0,95,4,88,0,124,1,83,0,41,7, - 78,114,150,0,0,0,84,114,154,0,0,0,114,16,0,0, - 0,122,18,105,109,112,111,114,116,32,123,33,114,125,32,35, - 32,123,33,114,125,70,41,14,114,109,0,0,0,114,4,0, - 0,0,114,158,0,0,0,114,152,0,0,0,90,13,95,105, - 110,105,116,105,97,108,105,122,105,110,103,114,15,0,0,0, - 114,92,0,0,0,114,17,0,0,0,114,117,0,0,0,114, - 79,0,0,0,114,150,0,0,0,114,63,0,0,0,114,156, - 0,0,0,114,76,0,0,0,114,151,0,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,218,14,95,108, - 111,97,100,95,117,110,108,111,99,107,101,100,138,2,0,0, - 115,46,0,0,0,0,2,10,2,12,1,8,2,8,5,6, - 1,2,1,12,1,2,1,10,1,10,1,16,3,16,1,6, - 1,2,1,14,1,14,1,6,1,8,5,14,1,12,1,20, - 2,8,2,114,159,0,0,0,99,1,0,0,0,0,0,0, - 0,0,0,0,0,1,0,0,0,10,0,0,0,67,0,0, - 0,115,42,0,0,0,116,0,124,0,106,1,131,1,143,22, - 1,0,116,2,124,0,131,1,87,0,2,0,53,0,81,0, - 82,0,163,0,83,0,81,0,82,0,88,0,100,1,83,0, - 41,2,122,191,82,101,116,117,114,110,32,97,32,110,101,119, - 32,109,111,100,117,108,101,32,111,98,106,101,99,116,44,32, - 108,111,97,100,101,100,32,98,121,32,116,104,101,32,115,112, - 101,99,39,115,32,108,111,97,100,101,114,46,10,10,32,32, - 32,32,84,104,101,32,109,111,100,117,108,101,32,105,115,32, - 110,111,116,32,97,100,100,101,100,32,116,111,32,105,116,115, - 32,112,97,114,101,110,116,46,10,10,32,32,32,32,73,102, - 32,97,32,109,111,100,117,108,101,32,105,115,32,97,108,114, - 101,97,100,121,32,105,110,32,115,121,115,46,109,111,100,117, - 108,101,115,44,32,116,104,97,116,32,101,120,105,115,116,105, - 110,103,32,109,111,100,117,108,101,32,103,101,116,115,10,32, - 32,32,32,99,108,111,98,98,101,114,101,100,46,10,10,32, - 32,32,32,78,41,3,114,50,0,0,0,114,17,0,0,0, - 114,159,0,0,0,41,1,114,95,0,0,0,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,94,0,0,0, - 180,2,0,0,115,4,0,0,0,0,9,12,1,114,94,0, - 0,0,99,0,0,0,0,0,0,0,0,0,0,0,0,0, - 0,0,0,4,0,0,0,64,0,0,0,115,136,0,0,0, - 101,0,90,1,100,0,90,2,100,1,90,3,101,4,100,2, - 100,3,132,0,131,1,90,5,101,6,100,19,100,5,100,6, - 132,1,131,1,90,7,101,6,100,20,100,7,100,8,132,1, - 131,1,90,8,101,6,100,9,100,10,132,0,131,1,90,9, - 101,6,100,11,100,12,132,0,131,1,90,10,101,6,101,11, - 100,13,100,14,132,0,131,1,131,1,90,12,101,6,101,11, - 100,15,100,16,132,0,131,1,131,1,90,13,101,6,101,11, - 100,17,100,18,132,0,131,1,131,1,90,14,101,6,101,15, - 131,1,90,16,100,4,83,0,41,21,218,15,66,117,105,108, - 116,105,110,73,109,112,111,114,116,101,114,122,144,77,101,116, - 97,32,112,97,116,104,32,105,109,112,111,114,116,32,102,111, - 114,32,98,117,105,108,116,45,105,110,32,109,111,100,117,108, - 101,115,46,10,10,32,32,32,32,65,108,108,32,109,101,116, - 104,111,100,115,32,97,114,101,32,101,105,116,104,101,114,32, - 99,108,97,115,115,32,111,114,32,115,116,97,116,105,99,32, - 109,101,116,104,111,100,115,32,116,111,32,97,118,111,105,100, - 32,116,104,101,32,110,101,101,100,32,116,111,10,32,32,32, - 32,105,110,115,116,97,110,116,105,97,116,101,32,116,104,101, - 32,99,108,97,115,115,46,10,10,32,32,32,32,99,1,0, - 0,0,0,0,0,0,0,0,0,0,1,0,0,0,3,0, - 0,0,67,0,0,0,115,12,0,0,0,100,1,160,0,124, - 0,106,1,161,1,83,0,41,2,250,115,82,101,116,117,114, - 110,32,114,101,112,114,32,102,111,114,32,116,104,101,32,109, - 111,100,117,108,101,46,10,10,32,32,32,32,32,32,32,32, - 84,104,101,32,109,101,116,104,111,100,32,105,115,32,100,101, - 112,114,101,99,97,116,101,100,46,32,32,84,104,101,32,105, - 109,112,111,114,116,32,109,97,99,104,105,110,101,114,121,32, - 100,111,101,115,32,116,104,101,32,106,111,98,32,105,116,115, - 101,108,102,46,10,10,32,32,32,32,32,32,32,32,122,24, - 60,109,111,100,117,108,101,32,123,33,114,125,32,40,98,117, - 105,108,116,45,105,110,41,62,41,2,114,45,0,0,0,114, - 1,0,0,0,41,1,114,96,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,114,99,0,0,0,204, - 2,0,0,115,2,0,0,0,0,7,122,27,66,117,105,108, - 116,105,110,73,109,112,111,114,116,101,114,46,109,111,100,117, - 108,101,95,114,101,112,114,78,99,4,0,0,0,0,0,0, - 0,0,0,0,0,4,0,0,0,5,0,0,0,67,0,0, - 0,115,44,0,0,0,124,2,100,0,107,9,114,12,100,0, - 83,0,116,0,160,1,124,1,161,1,114,36,116,2,124,1, - 124,0,100,1,100,2,141,3,83,0,100,0,83,0,100,0, - 83,0,41,3,78,122,8,98,117,105,108,116,45,105,110,114, - 137,0,0,0,41,3,114,57,0,0,0,90,10,105,115,95, - 98,117,105,108,116,105,110,114,91,0,0,0,169,4,218,3, - 99,108,115,114,81,0,0,0,218,4,112,97,116,104,218,6, - 116,97,114,103,101,116,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,218,9,102,105,110,100,95,115,112,101,99, - 213,2,0,0,115,10,0,0,0,0,2,8,1,4,1,10, - 1,14,2,122,25,66,117,105,108,116,105,110,73,109,112,111, - 114,116,101,114,46,102,105,110,100,95,115,112,101,99,99,3, - 0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,4, - 0,0,0,67,0,0,0,115,30,0,0,0,124,0,160,0, - 124,1,124,2,161,2,125,3,124,3,100,1,107,9,114,26, - 124,3,106,1,83,0,100,1,83,0,41,2,122,175,70,105, - 110,100,32,116,104,101,32,98,117,105,108,116,45,105,110,32, - 109,111,100,117,108,101,46,10,10,32,32,32,32,32,32,32, - 32,73,102,32,39,112,97,116,104,39,32,105,115,32,101,118, - 101,114,32,115,112,101,99,105,102,105,101,100,32,116,104,101, - 110,32,116,104,101,32,115,101,97,114,99,104,32,105,115,32, - 99,111,110,115,105,100,101,114,101,100,32,97,32,102,97,105, - 108,117,114,101,46,10,10,32,32,32,32,32,32,32,32,84, - 104,105,115,32,109,101,116,104,111,100,32,105,115,32,100,101, - 112,114,101,99,97,116,101,100,46,32,32,85,115,101,32,102, - 105,110,100,95,115,112,101,99,40,41,32,105,110,115,116,101, - 97,100,46,10,10,32,32,32,32,32,32,32,32,78,41,2, - 114,166,0,0,0,114,109,0,0,0,41,4,114,163,0,0, - 0,114,81,0,0,0,114,164,0,0,0,114,95,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, - 11,102,105,110,100,95,109,111,100,117,108,101,222,2,0,0, - 115,4,0,0,0,0,9,12,1,122,27,66,117,105,108,116, - 105,110,73,109,112,111,114,116,101,114,46,102,105,110,100,95, - 109,111,100,117,108,101,99,2,0,0,0,0,0,0,0,0, - 0,0,0,2,0,0,0,4,0,0,0,67,0,0,0,115, - 46,0,0,0,124,1,106,0,116,1,106,2,107,7,114,34, - 116,3,100,1,160,4,124,1,106,0,161,1,124,1,106,0, - 100,2,141,2,130,1,116,5,116,6,106,7,124,1,131,2, - 83,0,41,3,122,24,67,114,101,97,116,101,32,97,32,98, - 117,105,108,116,45,105,110,32,109,111,100,117,108,101,114,77, - 0,0,0,114,16,0,0,0,41,8,114,17,0,0,0,114, - 15,0,0,0,114,78,0,0,0,114,79,0,0,0,114,45, - 0,0,0,114,67,0,0,0,114,57,0,0,0,90,14,99, - 114,101,97,116,101,95,98,117,105,108,116,105,110,41,2,114, - 30,0,0,0,114,95,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,114,149,0,0,0,234,2,0, - 0,115,10,0,0,0,0,3,12,1,12,1,4,255,6,2, - 122,29,66,117,105,108,116,105,110,73,109,112,111,114,116,101, - 114,46,99,114,101,97,116,101,95,109,111,100,117,108,101,99, - 2,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0, - 3,0,0,0,67,0,0,0,115,16,0,0,0,116,0,116, - 1,106,2,124,1,131,2,1,0,100,1,83,0,41,2,122, - 22,69,120,101,99,32,97,32,98,117,105,108,116,45,105,110, - 32,109,111,100,117,108,101,78,41,3,114,67,0,0,0,114, - 57,0,0,0,90,12,101,120,101,99,95,98,117,105,108,116, - 105,110,41,2,114,30,0,0,0,114,96,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,114,150,0, - 0,0,242,2,0,0,115,2,0,0,0,0,3,122,27,66, - 117,105,108,116,105,110,73,109,112,111,114,116,101,114,46,101, - 120,101,99,95,109,111,100,117,108,101,99,2,0,0,0,0, - 0,0,0,0,0,0,0,2,0,0,0,1,0,0,0,67, - 0,0,0,115,4,0,0,0,100,1,83,0,41,2,122,57, - 82,101,116,117,114,110,32,78,111,110,101,32,97,115,32,98, - 117,105,108,116,45,105,110,32,109,111,100,117,108,101,115,32, - 100,111,32,110,111,116,32,104,97,118,101,32,99,111,100,101, - 32,111,98,106,101,99,116,115,46,78,114,10,0,0,0,169, - 2,114,163,0,0,0,114,81,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,218,8,103,101,116,95, - 99,111,100,101,247,2,0,0,115,2,0,0,0,0,4,122, - 24,66,117,105,108,116,105,110,73,109,112,111,114,116,101,114, - 46,103,101,116,95,99,111,100,101,99,2,0,0,0,0,0, - 0,0,0,0,0,0,2,0,0,0,1,0,0,0,67,0, - 0,0,115,4,0,0,0,100,1,83,0,41,2,122,56,82, - 101,116,117,114,110,32,78,111,110,101,32,97,115,32,98,117, - 105,108,116,45,105,110,32,109,111,100,117,108,101,115,32,100, - 111,32,110,111,116,32,104,97,118,101,32,115,111,117,114,99, - 101,32,99,111,100,101,46,78,114,10,0,0,0,114,168,0, + 130,0,89,0,110,2,88,0,116,3,106,4,160,5,124,0, + 106,2,161,1,125,1,124,1,116,3,106,4,124,0,106,2, + 60,0,116,6,124,1,100,1,100,0,131,3,100,0,107,8, + 114,148,122,12,124,0,106,0,124,1,95,7,87,0,110,20, + 4,0,116,8,107,10,114,146,1,0,1,0,1,0,89,0, + 110,2,88,0,116,6,124,1,100,2,100,0,131,3,100,0, + 107,8,114,226,122,40,124,1,106,9,124,1,95,10,116,11, + 124,1,100,3,131,2,115,202,124,0,106,2,160,12,100,4, + 161,1,100,5,25,0,124,1,95,10,87,0,110,20,4,0, + 116,8,107,10,114,224,1,0,1,0,1,0,89,0,110,2, + 88,0,116,6,124,1,100,6,100,0,131,3,100,0,107,8, + 144,1,114,22,122,10,124,0,124,1,95,13,87,0,110,22, + 4,0,116,8,107,10,144,1,114,20,1,0,1,0,1,0, + 89,0,110,2,88,0,124,1,83,0,41,7,78,114,98,0, + 0,0,114,145,0,0,0,114,141,0,0,0,114,128,0,0, + 0,114,22,0,0,0,114,105,0,0,0,41,14,114,109,0, + 0,0,114,155,0,0,0,114,17,0,0,0,114,15,0,0, + 0,114,92,0,0,0,114,156,0,0,0,114,6,0,0,0, + 114,98,0,0,0,114,106,0,0,0,114,1,0,0,0,114, + 145,0,0,0,114,4,0,0,0,114,129,0,0,0,114,105, + 0,0,0,114,151,0,0,0,114,10,0,0,0,114,10,0, + 0,0,114,11,0,0,0,218,25,95,108,111,97,100,95,98, + 97,99,107,119,97,114,100,95,99,111,109,112,97,116,105,98, + 108,101,101,2,0,0,115,54,0,0,0,0,4,2,1,18, + 1,6,1,12,1,14,1,12,1,8,3,14,1,12,1,16, + 1,2,1,12,1,14,1,6,1,16,1,2,4,8,1,10, + 1,22,1,14,1,6,1,18,1,2,1,10,1,16,1,6, + 1,114,158,0,0,0,99,1,0,0,0,0,0,0,0,0, + 0,0,0,2,0,0,0,11,0,0,0,67,0,0,0,115, + 220,0,0,0,124,0,106,0,100,0,107,9,114,30,116,1, + 124,0,106,0,100,1,131,2,115,30,116,2,124,0,131,1, + 83,0,116,3,124,0,131,1,125,1,100,2,124,0,95,4, + 122,162,124,1,116,5,106,6,124,0,106,7,60,0,122,52, + 124,0,106,0,100,0,107,8,114,96,124,0,106,8,100,0, + 107,8,114,108,116,9,100,3,124,0,106,7,100,4,141,2, + 130,1,110,12,124,0,106,0,160,10,124,1,161,1,1,0, + 87,0,110,50,1,0,1,0,1,0,122,14,116,5,106,6, + 124,0,106,7,61,0,87,0,110,20,4,0,116,11,107,10, + 114,152,1,0,1,0,1,0,89,0,110,2,88,0,130,0, + 89,0,110,2,88,0,116,5,106,6,160,12,124,0,106,7, + 161,1,125,1,124,1,116,5,106,6,124,0,106,7,60,0, + 116,13,100,5,124,0,106,7,124,0,106,0,131,3,1,0, + 87,0,53,0,100,6,124,0,95,4,88,0,124,1,83,0, + 41,7,78,114,150,0,0,0,84,114,154,0,0,0,114,16, + 0,0,0,122,18,105,109,112,111,114,116,32,123,33,114,125, + 32,35,32,123,33,114,125,70,41,14,114,109,0,0,0,114, + 4,0,0,0,114,158,0,0,0,114,152,0,0,0,90,13, + 95,105,110,105,116,105,97,108,105,122,105,110,103,114,15,0, + 0,0,114,92,0,0,0,114,17,0,0,0,114,116,0,0, + 0,114,79,0,0,0,114,150,0,0,0,114,63,0,0,0, + 114,156,0,0,0,114,76,0,0,0,114,151,0,0,0,114, + 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,14, + 95,108,111,97,100,95,117,110,108,111,99,107,101,100,138,2, + 0,0,115,46,0,0,0,0,2,10,2,12,1,8,2,8, + 5,6,1,2,1,12,1,2,1,10,1,10,1,16,3,16, + 1,6,1,2,1,14,1,14,1,6,1,8,5,14,1,12, + 1,20,2,8,2,114,159,0,0,0,99,1,0,0,0,0, + 0,0,0,0,0,0,0,1,0,0,0,10,0,0,0,67, + 0,0,0,115,42,0,0,0,116,0,124,0,106,1,131,1, + 143,22,1,0,116,2,124,0,131,1,87,0,2,0,53,0, + 81,0,82,0,163,0,83,0,81,0,82,0,88,0,100,1, + 83,0,41,2,122,191,82,101,116,117,114,110,32,97,32,110, + 101,119,32,109,111,100,117,108,101,32,111,98,106,101,99,116, + 44,32,108,111,97,100,101,100,32,98,121,32,116,104,101,32, + 115,112,101,99,39,115,32,108,111,97,100,101,114,46,10,10, + 32,32,32,32,84,104,101,32,109,111,100,117,108,101,32,105, + 115,32,110,111,116,32,97,100,100,101,100,32,116,111,32,105, + 116,115,32,112,97,114,101,110,116,46,10,10,32,32,32,32, + 73,102,32,97,32,109,111,100,117,108,101,32,105,115,32,97, + 108,114,101,97,100,121,32,105,110,32,115,121,115,46,109,111, + 100,117,108,101,115,44,32,116,104,97,116,32,101,120,105,115, + 116,105,110,103,32,109,111,100,117,108,101,32,103,101,116,115, + 10,32,32,32,32,99,108,111,98,98,101,114,101,100,46,10, + 10,32,32,32,32,78,41,3,114,50,0,0,0,114,17,0, + 0,0,114,159,0,0,0,41,1,114,95,0,0,0,114,10, + 0,0,0,114,10,0,0,0,114,11,0,0,0,114,94,0, + 0,0,180,2,0,0,115,4,0,0,0,0,9,12,1,114, + 94,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,4,0,0,0,64,0,0,0,115,136,0, + 0,0,101,0,90,1,100,0,90,2,100,1,90,3,101,4, + 100,2,100,3,132,0,131,1,90,5,101,6,100,19,100,5, + 100,6,132,1,131,1,90,7,101,6,100,20,100,7,100,8, + 132,1,131,1,90,8,101,6,100,9,100,10,132,0,131,1, + 90,9,101,6,100,11,100,12,132,0,131,1,90,10,101,6, + 101,11,100,13,100,14,132,0,131,1,131,1,90,12,101,6, + 101,11,100,15,100,16,132,0,131,1,131,1,90,13,101,6, + 101,11,100,17,100,18,132,0,131,1,131,1,90,14,101,6, + 101,15,131,1,90,16,100,4,83,0,41,21,218,15,66,117, + 105,108,116,105,110,73,109,112,111,114,116,101,114,122,144,77, + 101,116,97,32,112,97,116,104,32,105,109,112,111,114,116,32, + 102,111,114,32,98,117,105,108,116,45,105,110,32,109,111,100, + 117,108,101,115,46,10,10,32,32,32,32,65,108,108,32,109, + 101,116,104,111,100,115,32,97,114,101,32,101,105,116,104,101, + 114,32,99,108,97,115,115,32,111,114,32,115,116,97,116,105, + 99,32,109,101,116,104,111,100,115,32,116,111,32,97,118,111, + 105,100,32,116,104,101,32,110,101,101,100,32,116,111,10,32, + 32,32,32,105,110,115,116,97,110,116,105,97,116,101,32,116, + 104,101,32,99,108,97,115,115,46,10,10,32,32,32,32,99, + 1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0, + 3,0,0,0,67,0,0,0,115,12,0,0,0,100,1,160, + 0,124,0,106,1,161,1,83,0,41,2,250,115,82,101,116, + 117,114,110,32,114,101,112,114,32,102,111,114,32,116,104,101, + 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, + 32,32,84,104,101,32,109,101,116,104,111,100,32,105,115,32, + 100,101,112,114,101,99,97,116,101,100,46,32,32,84,104,101, + 32,105,109,112,111,114,116,32,109,97,99,104,105,110,101,114, + 121,32,100,111,101,115,32,116,104,101,32,106,111,98,32,105, + 116,115,101,108,102,46,10,10,32,32,32,32,32,32,32,32, + 122,24,60,109,111,100,117,108,101,32,123,33,114,125,32,40, + 98,117,105,108,116,45,105,110,41,62,41,2,114,45,0,0, + 0,114,1,0,0,0,41,1,114,96,0,0,0,114,10,0, + 0,0,114,10,0,0,0,114,11,0,0,0,114,99,0,0, + 0,204,2,0,0,115,2,0,0,0,0,7,122,27,66,117, + 105,108,116,105,110,73,109,112,111,114,116,101,114,46,109,111, + 100,117,108,101,95,114,101,112,114,78,99,4,0,0,0,0, + 0,0,0,0,0,0,0,4,0,0,0,5,0,0,0,67, + 0,0,0,115,44,0,0,0,124,2,100,0,107,9,114,12, + 100,0,83,0,116,0,160,1,124,1,161,1,114,36,116,2, + 124,1,124,0,100,1,100,2,141,3,83,0,100,0,83,0, + 100,0,83,0,41,3,78,122,8,98,117,105,108,116,45,105, + 110,114,137,0,0,0,41,3,114,57,0,0,0,90,10,105, + 115,95,98,117,105,108,116,105,110,114,91,0,0,0,169,4, + 218,3,99,108,115,114,81,0,0,0,218,4,112,97,116,104, + 218,6,116,97,114,103,101,116,114,10,0,0,0,114,10,0, + 0,0,114,11,0,0,0,218,9,102,105,110,100,95,115,112, + 101,99,213,2,0,0,115,10,0,0,0,0,2,8,1,4, + 1,10,1,14,2,122,25,66,117,105,108,116,105,110,73,109, + 112,111,114,116,101,114,46,102,105,110,100,95,115,112,101,99, + 99,3,0,0,0,0,0,0,0,0,0,0,0,4,0,0, + 0,4,0,0,0,67,0,0,0,115,30,0,0,0,124,0, + 160,0,124,1,124,2,161,2,125,3,124,3,100,1,107,9, + 114,26,124,3,106,1,83,0,100,1,83,0,41,2,122,175, + 70,105,110,100,32,116,104,101,32,98,117,105,108,116,45,105, + 110,32,109,111,100,117,108,101,46,10,10,32,32,32,32,32, + 32,32,32,73,102,32,39,112,97,116,104,39,32,105,115,32, + 101,118,101,114,32,115,112,101,99,105,102,105,101,100,32,116, + 104,101,110,32,116,104,101,32,115,101,97,114,99,104,32,105, + 115,32,99,111,110,115,105,100,101,114,101,100,32,97,32,102, + 97,105,108,117,114,101,46,10,10,32,32,32,32,32,32,32, + 32,84,104,105,115,32,109,101,116,104,111,100,32,105,115,32, + 100,101,112,114,101,99,97,116,101,100,46,32,32,85,115,101, + 32,102,105,110,100,95,115,112,101,99,40,41,32,105,110,115, + 116,101,97,100,46,10,10,32,32,32,32,32,32,32,32,78, + 41,2,114,166,0,0,0,114,109,0,0,0,41,4,114,163, + 0,0,0,114,81,0,0,0,114,164,0,0,0,114,95,0, 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,218,10,103,101,116,95,115,111,117,114,99,101,253,2,0, - 0,115,2,0,0,0,0,4,122,26,66,117,105,108,116,105, - 110,73,109,112,111,114,116,101,114,46,103,101,116,95,115,111, - 117,114,99,101,99,2,0,0,0,0,0,0,0,0,0,0, - 0,2,0,0,0,1,0,0,0,67,0,0,0,115,4,0, - 0,0,100,1,83,0,41,2,122,52,82,101,116,117,114,110, - 32,70,97,108,115,101,32,97,115,32,98,117,105,108,116,45, - 105,110,32,109,111,100,117,108,101,115,32,97,114,101,32,110, - 101,118,101,114,32,112,97,99,107,97,103,101,115,46,70,114, - 10,0,0,0,114,168,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,114,115,0,0,0,3,3,0, - 0,115,2,0,0,0,0,4,122,26,66,117,105,108,116,105, - 110,73,109,112,111,114,116,101,114,46,105,115,95,112,97,99, - 107,97,103,101,41,2,78,78,41,1,78,41,17,114,1,0, - 0,0,114,0,0,0,0,114,2,0,0,0,114,3,0,0, - 0,218,12,115,116,97,116,105,99,109,101,116,104,111,100,114, - 99,0,0,0,218,11,99,108,97,115,115,109,101,116,104,111, - 100,114,166,0,0,0,114,167,0,0,0,114,149,0,0,0, - 114,150,0,0,0,114,86,0,0,0,114,169,0,0,0,114, - 170,0,0,0,114,115,0,0,0,114,97,0,0,0,114,155, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,114,160,0,0,0,195,2,0,0, - 115,42,0,0,0,8,2,4,7,2,1,10,8,2,1,12, - 8,2,1,12,11,2,1,10,7,2,1,10,4,2,1,2, - 1,12,4,2,1,2,1,12,4,2,1,2,1,12,4,114, - 160,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, - 0,0,0,0,0,4,0,0,0,64,0,0,0,115,144,0, - 0,0,101,0,90,1,100,0,90,2,100,1,90,3,100,2, - 90,4,101,5,100,3,100,4,132,0,131,1,90,6,101,7, - 100,22,100,6,100,7,132,1,131,1,90,8,101,7,100,23, - 100,8,100,9,132,1,131,1,90,9,101,7,100,10,100,11, - 132,0,131,1,90,10,101,5,100,12,100,13,132,0,131,1, - 90,11,101,7,100,14,100,15,132,0,131,1,90,12,101,7, - 101,13,100,16,100,17,132,0,131,1,131,1,90,14,101,7, - 101,13,100,18,100,19,132,0,131,1,131,1,90,15,101,7, - 101,13,100,20,100,21,132,0,131,1,131,1,90,16,100,5, - 83,0,41,24,218,14,70,114,111,122,101,110,73,109,112,111, - 114,116,101,114,122,142,77,101,116,97,32,112,97,116,104,32, - 105,109,112,111,114,116,32,102,111,114,32,102,114,111,122,101, - 110,32,109,111,100,117,108,101,115,46,10,10,32,32,32,32, - 65,108,108,32,109,101,116,104,111,100,115,32,97,114,101,32, - 101,105,116,104,101,114,32,99,108,97,115,115,32,111,114,32, - 115,116,97,116,105,99,32,109,101,116,104,111,100,115,32,116, - 111,32,97,118,111,105,100,32,116,104,101,32,110,101,101,100, - 32,116,111,10,32,32,32,32,105,110,115,116,97,110,116,105, - 97,116,101,32,116,104,101,32,99,108,97,115,115,46,10,10, - 32,32,32,32,90,6,102,114,111,122,101,110,99,1,0,0, - 0,0,0,0,0,0,0,0,0,1,0,0,0,4,0,0, - 0,67,0,0,0,115,16,0,0,0,100,1,160,0,124,0, - 106,1,116,2,106,3,161,2,83,0,41,2,114,161,0,0, - 0,114,153,0,0,0,41,4,114,45,0,0,0,114,1,0, - 0,0,114,173,0,0,0,114,138,0,0,0,41,1,218,1, - 109,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 114,99,0,0,0,23,3,0,0,115,2,0,0,0,0,7, - 122,26,70,114,111,122,101,110,73,109,112,111,114,116,101,114, - 46,109,111,100,117,108,101,95,114,101,112,114,78,99,4,0, - 0,0,0,0,0,0,0,0,0,0,4,0,0,0,5,0, - 0,0,67,0,0,0,115,34,0,0,0,116,0,160,1,124, - 1,161,1,114,26,116,2,124,1,124,0,124,0,106,3,100, - 1,141,3,83,0,100,0,83,0,100,0,83,0,41,2,78, - 114,137,0,0,0,41,4,114,57,0,0,0,114,88,0,0, - 0,114,91,0,0,0,114,138,0,0,0,114,162,0,0,0, + 0,218,11,102,105,110,100,95,109,111,100,117,108,101,222,2, + 0,0,115,4,0,0,0,0,9,12,1,122,27,66,117,105, + 108,116,105,110,73,109,112,111,114,116,101,114,46,102,105,110, + 100,95,109,111,100,117,108,101,99,2,0,0,0,0,0,0, + 0,0,0,0,0,2,0,0,0,4,0,0,0,67,0,0, + 0,115,46,0,0,0,124,1,106,0,116,1,106,2,107,7, + 114,34,116,3,100,1,160,4,124,1,106,0,161,1,124,1, + 106,0,100,2,141,2,130,1,116,5,116,6,106,7,124,1, + 131,2,83,0,41,3,122,24,67,114,101,97,116,101,32,97, + 32,98,117,105,108,116,45,105,110,32,109,111,100,117,108,101, + 114,77,0,0,0,114,16,0,0,0,41,8,114,17,0,0, + 0,114,15,0,0,0,114,78,0,0,0,114,79,0,0,0, + 114,45,0,0,0,114,67,0,0,0,114,57,0,0,0,90, + 14,99,114,101,97,116,101,95,98,117,105,108,116,105,110,41, + 2,114,30,0,0,0,114,95,0,0,0,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,114,149,0,0,0,234, + 2,0,0,115,10,0,0,0,0,3,12,1,12,1,4,255, + 6,2,122,29,66,117,105,108,116,105,110,73,109,112,111,114, + 116,101,114,46,99,114,101,97,116,101,95,109,111,100,117,108, + 101,99,2,0,0,0,0,0,0,0,0,0,0,0,2,0, + 0,0,3,0,0,0,67,0,0,0,115,16,0,0,0,116, + 0,116,1,106,2,124,1,131,2,1,0,100,1,83,0,41, + 2,122,22,69,120,101,99,32,97,32,98,117,105,108,116,45, + 105,110,32,109,111,100,117,108,101,78,41,3,114,67,0,0, + 0,114,57,0,0,0,90,12,101,120,101,99,95,98,117,105, + 108,116,105,110,41,2,114,30,0,0,0,114,96,0,0,0, 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 166,0,0,0,32,3,0,0,115,6,0,0,0,0,2,10, - 1,16,2,122,24,70,114,111,122,101,110,73,109,112,111,114, - 116,101,114,46,102,105,110,100,95,115,112,101,99,99,3,0, - 0,0,0,0,0,0,0,0,0,0,3,0,0,0,3,0, - 0,0,67,0,0,0,115,18,0,0,0,116,0,160,1,124, - 1,161,1,114,14,124,0,83,0,100,1,83,0,41,2,122, - 93,70,105,110,100,32,97,32,102,114,111,122,101,110,32,109, - 111,100,117,108,101,46,10,10,32,32,32,32,32,32,32,32, - 84,104,105,115,32,109,101,116,104,111,100,32,105,115,32,100, - 101,112,114,101,99,97,116,101,100,46,32,32,85,115,101,32, - 102,105,110,100,95,115,112,101,99,40,41,32,105,110,115,116, - 101,97,100,46,10,10,32,32,32,32,32,32,32,32,78,41, - 2,114,57,0,0,0,114,88,0,0,0,41,3,114,163,0, - 0,0,114,81,0,0,0,114,164,0,0,0,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,167,0,0,0, - 39,3,0,0,115,2,0,0,0,0,7,122,26,70,114,111, - 122,101,110,73,109,112,111,114,116,101,114,46,102,105,110,100, - 95,109,111,100,117,108,101,99,2,0,0,0,0,0,0,0, - 0,0,0,0,2,0,0,0,1,0,0,0,67,0,0,0, - 115,4,0,0,0,100,1,83,0,41,2,122,42,85,115,101, - 32,100,101,102,97,117,108,116,32,115,101,109,97,110,116,105, - 99,115,32,102,111,114,32,109,111,100,117,108,101,32,99,114, - 101,97,116,105,111,110,46,78,114,10,0,0,0,41,2,114, - 163,0,0,0,114,95,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,114,149,0,0,0,48,3,0, - 0,115,2,0,0,0,0,2,122,28,70,114,111,122,101,110, - 73,109,112,111,114,116,101,114,46,99,114,101,97,116,101,95, - 109,111,100,117,108,101,99,1,0,0,0,0,0,0,0,0, - 0,0,0,3,0,0,0,4,0,0,0,67,0,0,0,115, - 64,0,0,0,124,0,106,0,106,1,125,1,116,2,160,3, - 124,1,161,1,115,36,116,4,100,1,160,5,124,1,161,1, - 124,1,100,2,141,2,130,1,116,6,116,2,106,7,124,1, - 131,2,125,2,116,8,124,2,124,0,106,9,131,2,1,0, - 100,0,83,0,114,87,0,0,0,41,10,114,105,0,0,0, - 114,17,0,0,0,114,57,0,0,0,114,88,0,0,0,114, - 79,0,0,0,114,45,0,0,0,114,67,0,0,0,218,17, - 103,101,116,95,102,114,111,122,101,110,95,111,98,106,101,99, - 116,218,4,101,120,101,99,114,7,0,0,0,41,3,114,96, - 0,0,0,114,17,0,0,0,218,4,99,111,100,101,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,114,150,0, - 0,0,52,3,0,0,115,14,0,0,0,0,2,8,1,10, - 1,10,1,2,255,6,2,12,1,122,26,70,114,111,122,101, - 110,73,109,112,111,114,116,101,114,46,101,120,101,99,95,109, - 111,100,117,108,101,99,2,0,0,0,0,0,0,0,0,0, - 0,0,2,0,0,0,3,0,0,0,67,0,0,0,115,10, - 0,0,0,116,0,124,0,124,1,131,2,83,0,41,1,122, - 95,76,111,97,100,32,97,32,102,114,111,122,101,110,32,109, - 111,100,117,108,101,46,10,10,32,32,32,32,32,32,32,32, - 84,104,105,115,32,109,101,116,104,111,100,32,105,115,32,100, - 101,112,114,101,99,97,116,101,100,46,32,32,85,115,101,32, - 101,120,101,99,95,109,111,100,117,108,101,40,41,32,105,110, + 150,0,0,0,242,2,0,0,115,2,0,0,0,0,3,122, + 27,66,117,105,108,116,105,110,73,109,112,111,114,116,101,114, + 46,101,120,101,99,95,109,111,100,117,108,101,99,2,0,0, + 0,0,0,0,0,0,0,0,0,2,0,0,0,1,0,0, + 0,67,0,0,0,115,4,0,0,0,100,1,83,0,41,2, + 122,57,82,101,116,117,114,110,32,78,111,110,101,32,97,115, + 32,98,117,105,108,116,45,105,110,32,109,111,100,117,108,101, + 115,32,100,111,32,110,111,116,32,104,97,118,101,32,99,111, + 100,101,32,111,98,106,101,99,116,115,46,78,114,10,0,0, + 0,169,2,114,163,0,0,0,114,81,0,0,0,114,10,0, + 0,0,114,10,0,0,0,114,11,0,0,0,218,8,103,101, + 116,95,99,111,100,101,247,2,0,0,115,2,0,0,0,0, + 4,122,24,66,117,105,108,116,105,110,73,109,112,111,114,116, + 101,114,46,103,101,116,95,99,111,100,101,99,2,0,0,0, + 0,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, + 67,0,0,0,115,4,0,0,0,100,1,83,0,41,2,122, + 56,82,101,116,117,114,110,32,78,111,110,101,32,97,115,32, + 98,117,105,108,116,45,105,110,32,109,111,100,117,108,101,115, + 32,100,111,32,110,111,116,32,104,97,118,101,32,115,111,117, + 114,99,101,32,99,111,100,101,46,78,114,10,0,0,0,114, + 168,0,0,0,114,10,0,0,0,114,10,0,0,0,114,11, + 0,0,0,218,10,103,101,116,95,115,111,117,114,99,101,253, + 2,0,0,115,2,0,0,0,0,4,122,26,66,117,105,108, + 116,105,110,73,109,112,111,114,116,101,114,46,103,101,116,95, + 115,111,117,114,99,101,99,2,0,0,0,0,0,0,0,0, + 0,0,0,2,0,0,0,1,0,0,0,67,0,0,0,115, + 4,0,0,0,100,1,83,0,41,2,122,52,82,101,116,117, + 114,110,32,70,97,108,115,101,32,97,115,32,98,117,105,108, + 116,45,105,110,32,109,111,100,117,108,101,115,32,97,114,101, + 32,110,101,118,101,114,32,112,97,99,107,97,103,101,115,46, + 70,114,10,0,0,0,114,168,0,0,0,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,114,115,0,0,0,3, + 3,0,0,115,2,0,0,0,0,4,122,26,66,117,105,108, + 116,105,110,73,109,112,111,114,116,101,114,46,105,115,95,112, + 97,99,107,97,103,101,41,2,78,78,41,1,78,41,17,114, + 1,0,0,0,114,0,0,0,0,114,2,0,0,0,114,3, + 0,0,0,218,12,115,116,97,116,105,99,109,101,116,104,111, + 100,114,99,0,0,0,218,11,99,108,97,115,115,109,101,116, + 104,111,100,114,166,0,0,0,114,167,0,0,0,114,149,0, + 0,0,114,150,0,0,0,114,86,0,0,0,114,169,0,0, + 0,114,170,0,0,0,114,115,0,0,0,114,97,0,0,0, + 114,155,0,0,0,114,10,0,0,0,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,114,160,0,0,0,195,2, + 0,0,115,42,0,0,0,8,2,4,7,2,1,10,8,2, + 1,12,8,2,1,12,11,2,1,10,7,2,1,10,4,2, + 1,2,1,12,4,2,1,2,1,12,4,2,1,2,1,12, + 4,114,160,0,0,0,99,0,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,4,0,0,0,64,0,0,0,115, + 144,0,0,0,101,0,90,1,100,0,90,2,100,1,90,3, + 100,2,90,4,101,5,100,3,100,4,132,0,131,1,90,6, + 101,7,100,22,100,6,100,7,132,1,131,1,90,8,101,7, + 100,23,100,8,100,9,132,1,131,1,90,9,101,7,100,10, + 100,11,132,0,131,1,90,10,101,5,100,12,100,13,132,0, + 131,1,90,11,101,7,100,14,100,15,132,0,131,1,90,12, + 101,7,101,13,100,16,100,17,132,0,131,1,131,1,90,14, + 101,7,101,13,100,18,100,19,132,0,131,1,131,1,90,15, + 101,7,101,13,100,20,100,21,132,0,131,1,131,1,90,16, + 100,5,83,0,41,24,218,14,70,114,111,122,101,110,73,109, + 112,111,114,116,101,114,122,142,77,101,116,97,32,112,97,116, + 104,32,105,109,112,111,114,116,32,102,111,114,32,102,114,111, + 122,101,110,32,109,111,100,117,108,101,115,46,10,10,32,32, + 32,32,65,108,108,32,109,101,116,104,111,100,115,32,97,114, + 101,32,101,105,116,104,101,114,32,99,108,97,115,115,32,111, + 114,32,115,116,97,116,105,99,32,109,101,116,104,111,100,115, + 32,116,111,32,97,118,111,105,100,32,116,104,101,32,110,101, + 101,100,32,116,111,10,32,32,32,32,105,110,115,116,97,110, + 116,105,97,116,101,32,116,104,101,32,99,108,97,115,115,46, + 10,10,32,32,32,32,90,6,102,114,111,122,101,110,99,1, + 0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,4, + 0,0,0,67,0,0,0,115,16,0,0,0,100,1,160,0, + 124,0,106,1,116,2,106,3,161,2,83,0,41,2,114,161, + 0,0,0,114,153,0,0,0,41,4,114,45,0,0,0,114, + 1,0,0,0,114,173,0,0,0,114,138,0,0,0,41,1, + 218,1,109,114,10,0,0,0,114,10,0,0,0,114,11,0, + 0,0,114,99,0,0,0,23,3,0,0,115,2,0,0,0, + 0,7,122,26,70,114,111,122,101,110,73,109,112,111,114,116, + 101,114,46,109,111,100,117,108,101,95,114,101,112,114,78,99, + 4,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0, + 5,0,0,0,67,0,0,0,115,34,0,0,0,116,0,160, + 1,124,1,161,1,114,26,116,2,124,1,124,0,124,0,106, + 3,100,1,141,3,83,0,100,0,83,0,100,0,83,0,41, + 2,78,114,137,0,0,0,41,4,114,57,0,0,0,114,88, + 0,0,0,114,91,0,0,0,114,138,0,0,0,114,162,0, + 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, + 0,114,166,0,0,0,32,3,0,0,115,6,0,0,0,0, + 2,10,1,16,2,122,24,70,114,111,122,101,110,73,109,112, + 111,114,116,101,114,46,102,105,110,100,95,115,112,101,99,99, + 3,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0, + 3,0,0,0,67,0,0,0,115,18,0,0,0,116,0,160, + 1,124,1,161,1,114,14,124,0,83,0,100,1,83,0,41, + 2,122,93,70,105,110,100,32,97,32,102,114,111,122,101,110, + 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, + 32,32,84,104,105,115,32,109,101,116,104,111,100,32,105,115, + 32,100,101,112,114,101,99,97,116,101,100,46,32,32,85,115, + 101,32,102,105,110,100,95,115,112,101,99,40,41,32,105,110, 115,116,101,97,100,46,10,10,32,32,32,32,32,32,32,32, - 41,1,114,97,0,0,0,114,168,0,0,0,114,10,0,0, - 0,114,10,0,0,0,114,11,0,0,0,114,155,0,0,0, - 61,3,0,0,115,2,0,0,0,0,7,122,26,70,114,111, - 122,101,110,73,109,112,111,114,116,101,114,46,108,111,97,100, - 95,109,111,100,117,108,101,99,2,0,0,0,0,0,0,0, - 0,0,0,0,2,0,0,0,3,0,0,0,67,0,0,0, - 115,10,0,0,0,116,0,160,1,124,1,161,1,83,0,41, - 1,122,45,82,101,116,117,114,110,32,116,104,101,32,99,111, - 100,101,32,111,98,106,101,99,116,32,102,111,114,32,116,104, - 101,32,102,114,111,122,101,110,32,109,111,100,117,108,101,46, - 41,2,114,57,0,0,0,114,175,0,0,0,114,168,0,0, - 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 114,169,0,0,0,70,3,0,0,115,2,0,0,0,0,4, - 122,23,70,114,111,122,101,110,73,109,112,111,114,116,101,114, - 46,103,101,116,95,99,111,100,101,99,2,0,0,0,0,0, + 78,41,2,114,57,0,0,0,114,88,0,0,0,41,3,114, + 163,0,0,0,114,81,0,0,0,114,164,0,0,0,114,10, + 0,0,0,114,10,0,0,0,114,11,0,0,0,114,167,0, + 0,0,39,3,0,0,115,2,0,0,0,0,7,122,26,70, + 114,111,122,101,110,73,109,112,111,114,116,101,114,46,102,105, + 110,100,95,109,111,100,117,108,101,99,2,0,0,0,0,0, 0,0,0,0,0,0,2,0,0,0,1,0,0,0,67,0, - 0,0,115,4,0,0,0,100,1,83,0,41,2,122,54,82, - 101,116,117,114,110,32,78,111,110,101,32,97,115,32,102,114, - 111,122,101,110,32,109,111,100,117,108,101,115,32,100,111,32, - 110,111,116,32,104,97,118,101,32,115,111,117,114,99,101,32, - 99,111,100,101,46,78,114,10,0,0,0,114,168,0,0,0, + 0,0,115,4,0,0,0,100,1,83,0,41,2,122,42,85, + 115,101,32,100,101,102,97,117,108,116,32,115,101,109,97,110, + 116,105,99,115,32,102,111,114,32,109,111,100,117,108,101,32, + 99,114,101,97,116,105,111,110,46,78,114,10,0,0,0,41, + 2,114,163,0,0,0,114,95,0,0,0,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,114,149,0,0,0,48, + 3,0,0,115,2,0,0,0,0,2,122,28,70,114,111,122, + 101,110,73,109,112,111,114,116,101,114,46,99,114,101,97,116, + 101,95,109,111,100,117,108,101,99,1,0,0,0,0,0,0, + 0,0,0,0,0,3,0,0,0,4,0,0,0,67,0,0, + 0,115,64,0,0,0,124,0,106,0,106,1,125,1,116,2, + 160,3,124,1,161,1,115,36,116,4,100,1,160,5,124,1, + 161,1,124,1,100,2,141,2,130,1,116,6,116,2,106,7, + 124,1,131,2,125,2,116,8,124,2,124,0,106,9,131,2, + 1,0,100,0,83,0,114,87,0,0,0,41,10,114,105,0, + 0,0,114,17,0,0,0,114,57,0,0,0,114,88,0,0, + 0,114,79,0,0,0,114,45,0,0,0,114,67,0,0,0, + 218,17,103,101,116,95,102,114,111,122,101,110,95,111,98,106, + 101,99,116,218,4,101,120,101,99,114,7,0,0,0,41,3, + 114,96,0,0,0,114,17,0,0,0,218,4,99,111,100,101, 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 170,0,0,0,76,3,0,0,115,2,0,0,0,0,4,122, - 25,70,114,111,122,101,110,73,109,112,111,114,116,101,114,46, - 103,101,116,95,115,111,117,114,99,101,99,2,0,0,0,0, - 0,0,0,0,0,0,0,2,0,0,0,3,0,0,0,67, - 0,0,0,115,10,0,0,0,116,0,160,1,124,1,161,1, - 83,0,41,1,122,46,82,101,116,117,114,110,32,84,114,117, - 101,32,105,102,32,116,104,101,32,102,114,111,122,101,110,32, - 109,111,100,117,108,101,32,105,115,32,97,32,112,97,99,107, - 97,103,101,46,41,2,114,57,0,0,0,90,17,105,115,95, - 102,114,111,122,101,110,95,112,97,99,107,97,103,101,114,168, + 150,0,0,0,52,3,0,0,115,14,0,0,0,0,2,8, + 1,10,1,10,1,2,255,6,2,12,1,122,26,70,114,111, + 122,101,110,73,109,112,111,114,116,101,114,46,101,120,101,99, + 95,109,111,100,117,108,101,99,2,0,0,0,0,0,0,0, + 0,0,0,0,2,0,0,0,3,0,0,0,67,0,0,0, + 115,10,0,0,0,116,0,124,0,124,1,131,2,83,0,41, + 1,122,95,76,111,97,100,32,97,32,102,114,111,122,101,110, + 32,109,111,100,117,108,101,46,10,10,32,32,32,32,32,32, + 32,32,84,104,105,115,32,109,101,116,104,111,100,32,105,115, + 32,100,101,112,114,101,99,97,116,101,100,46,32,32,85,115, + 101,32,101,120,101,99,95,109,111,100,117,108,101,40,41,32, + 105,110,115,116,101,97,100,46,10,10,32,32,32,32,32,32, + 32,32,41,1,114,97,0,0,0,114,168,0,0,0,114,10, + 0,0,0,114,10,0,0,0,114,11,0,0,0,114,155,0, + 0,0,61,3,0,0,115,2,0,0,0,0,7,122,26,70, + 114,111,122,101,110,73,109,112,111,114,116,101,114,46,108,111, + 97,100,95,109,111,100,117,108,101,99,2,0,0,0,0,0, + 0,0,0,0,0,0,2,0,0,0,3,0,0,0,67,0, + 0,0,115,10,0,0,0,116,0,160,1,124,1,161,1,83, + 0,41,1,122,45,82,101,116,117,114,110,32,116,104,101,32, + 99,111,100,101,32,111,98,106,101,99,116,32,102,111,114,32, + 116,104,101,32,102,114,111,122,101,110,32,109,111,100,117,108, + 101,46,41,2,114,57,0,0,0,114,175,0,0,0,114,168, 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,114,115,0,0,0,82,3,0,0,115,2,0,0,0, - 0,4,122,25,70,114,111,122,101,110,73,109,112,111,114,116, - 101,114,46,105,115,95,112,97,99,107,97,103,101,41,2,78, - 78,41,1,78,41,17,114,1,0,0,0,114,0,0,0,0, - 114,2,0,0,0,114,3,0,0,0,114,138,0,0,0,114, - 171,0,0,0,114,99,0,0,0,114,172,0,0,0,114,166, - 0,0,0,114,167,0,0,0,114,149,0,0,0,114,150,0, - 0,0,114,155,0,0,0,114,90,0,0,0,114,169,0,0, - 0,114,170,0,0,0,114,115,0,0,0,114,10,0,0,0, + 0,0,114,169,0,0,0,70,3,0,0,115,2,0,0,0, + 0,4,122,23,70,114,111,122,101,110,73,109,112,111,114,116, + 101,114,46,103,101,116,95,99,111,100,101,99,2,0,0,0, + 0,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0, + 67,0,0,0,115,4,0,0,0,100,1,83,0,41,2,122, + 54,82,101,116,117,114,110,32,78,111,110,101,32,97,115,32, + 102,114,111,122,101,110,32,109,111,100,117,108,101,115,32,100, + 111,32,110,111,116,32,104,97,118,101,32,115,111,117,114,99, + 101,32,99,111,100,101,46,78,114,10,0,0,0,114,168,0, + 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, + 0,114,170,0,0,0,76,3,0,0,115,2,0,0,0,0, + 4,122,25,70,114,111,122,101,110,73,109,112,111,114,116,101, + 114,46,103,101,116,95,115,111,117,114,99,101,99,2,0,0, + 0,0,0,0,0,0,0,0,0,2,0,0,0,3,0,0, + 0,67,0,0,0,115,10,0,0,0,116,0,160,1,124,1, + 161,1,83,0,41,1,122,46,82,101,116,117,114,110,32,84, + 114,117,101,32,105,102,32,116,104,101,32,102,114,111,122,101, + 110,32,109,111,100,117,108,101,32,105,115,32,97,32,112,97, + 99,107,97,103,101,46,41,2,114,57,0,0,0,90,17,105, + 115,95,102,114,111,122,101,110,95,112,97,99,107,97,103,101, + 114,168,0,0,0,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,114,115,0,0,0,82,3,0,0,115,2,0, + 0,0,0,4,122,25,70,114,111,122,101,110,73,109,112,111, + 114,116,101,114,46,105,115,95,112,97,99,107,97,103,101,41, + 2,78,78,41,1,78,41,17,114,1,0,0,0,114,0,0, + 0,0,114,2,0,0,0,114,3,0,0,0,114,138,0,0, + 0,114,171,0,0,0,114,99,0,0,0,114,172,0,0,0, + 114,166,0,0,0,114,167,0,0,0,114,149,0,0,0,114, + 150,0,0,0,114,155,0,0,0,114,90,0,0,0,114,169, + 0,0,0,114,170,0,0,0,114,115,0,0,0,114,10,0, + 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, + 0,114,173,0,0,0,12,3,0,0,115,46,0,0,0,8, + 2,4,7,4,2,2,1,10,8,2,1,12,6,2,1,12, + 8,2,1,10,3,2,1,10,8,2,1,10,8,2,1,2, + 1,12,4,2,1,2,1,12,4,2,1,2,1,114,173,0, + 0,0,99,0,0,0,0,0,0,0,0,0,0,0,0,0, + 0,0,0,2,0,0,0,64,0,0,0,115,32,0,0,0, + 101,0,90,1,100,0,90,2,100,1,90,3,100,2,100,3, + 132,0,90,4,100,4,100,5,132,0,90,5,100,6,83,0, + 41,7,218,18,95,73,109,112,111,114,116,76,111,99,107,67, + 111,110,116,101,120,116,122,36,67,111,110,116,101,120,116,32, + 109,97,110,97,103,101,114,32,102,111,114,32,116,104,101,32, + 105,109,112,111,114,116,32,108,111,99,107,46,99,1,0,0, + 0,0,0,0,0,0,0,0,0,1,0,0,0,2,0,0, + 0,67,0,0,0,115,12,0,0,0,116,0,160,1,161,0, + 1,0,100,1,83,0,41,2,122,24,65,99,113,117,105,114, + 101,32,116,104,101,32,105,109,112,111,114,116,32,108,111,99, + 107,46,78,41,2,114,57,0,0,0,114,58,0,0,0,114, + 47,0,0,0,114,10,0,0,0,114,10,0,0,0,114,11, + 0,0,0,114,54,0,0,0,95,3,0,0,115,2,0,0, + 0,0,2,122,28,95,73,109,112,111,114,116,76,111,99,107, + 67,111,110,116,101,120,116,46,95,95,101,110,116,101,114,95, + 95,99,4,0,0,0,0,0,0,0,0,0,0,0,4,0, + 0,0,2,0,0,0,67,0,0,0,115,12,0,0,0,116, + 0,160,1,161,0,1,0,100,1,83,0,41,2,122,60,82, + 101,108,101,97,115,101,32,116,104,101,32,105,109,112,111,114, + 116,32,108,111,99,107,32,114,101,103,97,114,100,108,101,115, + 115,32,111,102,32,97,110,121,32,114,97,105,115,101,100,32, + 101,120,99,101,112,116,105,111,110,115,46,78,41,2,114,57, + 0,0,0,114,60,0,0,0,41,4,114,30,0,0,0,218, + 8,101,120,99,95,116,121,112,101,218,9,101,120,99,95,118, + 97,108,117,101,218,13,101,120,99,95,116,114,97,99,101,98, + 97,99,107,114,10,0,0,0,114,10,0,0,0,114,11,0, + 0,0,114,56,0,0,0,99,3,0,0,115,2,0,0,0, + 0,2,122,27,95,73,109,112,111,114,116,76,111,99,107,67, + 111,110,116,101,120,116,46,95,95,101,120,105,116,95,95,78, + 41,6,114,1,0,0,0,114,0,0,0,0,114,2,0,0, + 0,114,3,0,0,0,114,54,0,0,0,114,56,0,0,0, + 114,10,0,0,0,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,114,178,0,0,0,91,3,0,0,115,6,0, + 0,0,8,2,4,2,8,4,114,178,0,0,0,99,3,0, + 0,0,0,0,0,0,0,0,0,0,5,0,0,0,5,0, + 0,0,67,0,0,0,115,64,0,0,0,124,1,160,0,100, + 1,124,2,100,2,24,0,161,2,125,3,116,1,124,3,131, + 1,124,2,107,0,114,36,116,2,100,3,131,1,130,1,124, + 3,100,4,25,0,125,4,124,0,114,60,100,5,160,3,124, + 4,124,0,161,2,83,0,124,4,83,0,41,6,122,50,82, + 101,115,111,108,118,101,32,97,32,114,101,108,97,116,105,118, + 101,32,109,111,100,117,108,101,32,110,97,109,101,32,116,111, + 32,97,110,32,97,98,115,111,108,117,116,101,32,111,110,101, + 46,114,128,0,0,0,114,37,0,0,0,122,50,97,116,116, + 101,109,112,116,101,100,32,114,101,108,97,116,105,118,101,32, + 105,109,112,111,114,116,32,98,101,121,111,110,100,32,116,111, + 112,45,108,101,118,101,108,32,112,97,99,107,97,103,101,114, + 22,0,0,0,250,5,123,125,46,123,125,41,4,218,6,114, + 115,112,108,105,116,218,3,108,101,110,114,79,0,0,0,114, + 45,0,0,0,41,5,114,17,0,0,0,218,7,112,97,99, + 107,97,103,101,218,5,108,101,118,101,108,90,4,98,105,116, + 115,90,4,98,97,115,101,114,10,0,0,0,114,10,0,0, + 0,114,11,0,0,0,218,13,95,114,101,115,111,108,118,101, + 95,110,97,109,101,104,3,0,0,115,10,0,0,0,0,2, + 16,1,12,1,8,1,8,1,114,187,0,0,0,99,3,0, + 0,0,0,0,0,0,0,0,0,0,4,0,0,0,4,0, + 0,0,67,0,0,0,115,34,0,0,0,124,0,160,0,124, + 1,124,2,161,2,125,3,124,3,100,0,107,8,114,24,100, + 0,83,0,116,1,124,1,124,3,131,2,83,0,114,13,0, + 0,0,41,2,114,167,0,0,0,114,91,0,0,0,41,4, + 218,6,102,105,110,100,101,114,114,17,0,0,0,114,164,0, + 0,0,114,109,0,0,0,114,10,0,0,0,114,10,0,0, + 0,114,11,0,0,0,218,17,95,102,105,110,100,95,115,112, + 101,99,95,108,101,103,97,99,121,113,3,0,0,115,8,0, + 0,0,0,3,12,1,8,1,4,1,114,189,0,0,0,99, + 3,0,0,0,0,0,0,0,0,0,0,0,10,0,0,0, + 10,0,0,0,67,0,0,0,115,12,1,0,0,116,0,106, + 1,125,3,124,3,100,1,107,8,114,22,116,2,100,2,131, + 1,130,1,124,3,115,38,116,3,160,4,100,3,116,5,161, + 2,1,0,124,0,116,0,106,6,107,6,125,4,124,3,68, + 0,93,210,125,5,116,7,131,0,143,84,1,0,122,10,124, + 5,106,8,125,6,87,0,110,54,4,0,116,9,107,10,114, + 128,1,0,1,0,1,0,116,10,124,5,124,0,124,1,131, + 3,125,7,124,7,100,1,107,8,114,124,89,0,87,0,53, + 0,81,0,82,0,163,0,113,52,89,0,110,14,88,0,124, + 6,124,0,124,1,124,2,131,3,125,7,87,0,53,0,81, + 0,82,0,88,0,124,7,100,1,107,9,114,52,124,4,144, + 0,115,254,124,0,116,0,106,6,107,6,144,0,114,254,116, + 0,106,6,124,0,25,0,125,8,122,10,124,8,106,11,125, + 9,87,0,110,28,4,0,116,9,107,10,114,226,1,0,1, + 0,1,0,124,7,6,0,89,0,2,0,1,0,83,0,88, + 0,124,9,100,1,107,8,114,244,124,7,2,0,1,0,83, + 0,124,9,2,0,1,0,83,0,113,52,124,7,2,0,1, + 0,83,0,113,52,100,1,83,0,41,4,122,21,70,105,110, + 100,32,97,32,109,111,100,117,108,101,39,115,32,115,112,101, + 99,46,78,122,53,115,121,115,46,109,101,116,97,95,112,97, + 116,104,32,105,115,32,78,111,110,101,44,32,80,121,116,104, + 111,110,32,105,115,32,108,105,107,101,108,121,32,115,104,117, + 116,116,105,110,103,32,100,111,119,110,122,22,115,121,115,46, + 109,101,116,97,95,112,97,116,104,32,105,115,32,101,109,112, + 116,121,41,12,114,15,0,0,0,218,9,109,101,116,97,95, + 112,97,116,104,114,79,0,0,0,218,9,95,119,97,114,110, + 105,110,103,115,218,4,119,97,114,110,218,13,73,109,112,111, + 114,116,87,97,114,110,105,110,103,114,92,0,0,0,114,178, + 0,0,0,114,166,0,0,0,114,106,0,0,0,114,189,0, + 0,0,114,105,0,0,0,41,10,114,17,0,0,0,114,164, + 0,0,0,114,165,0,0,0,114,190,0,0,0,90,9,105, + 115,95,114,101,108,111,97,100,114,188,0,0,0,114,166,0, + 0,0,114,95,0,0,0,114,96,0,0,0,114,105,0,0, + 0,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, + 218,10,95,102,105,110,100,95,115,112,101,99,122,3,0,0, + 115,54,0,0,0,0,2,6,1,8,2,8,3,4,1,12, + 5,10,1,8,1,8,1,2,1,10,1,14,1,12,1,8, + 1,20,2,22,1,8,2,18,1,10,1,2,1,10,1,14, + 4,14,2,8,1,8,2,10,2,10,2,114,194,0,0,0, + 99,3,0,0,0,0,0,0,0,0,0,0,0,3,0,0, + 0,5,0,0,0,67,0,0,0,115,108,0,0,0,116,0, + 124,0,116,1,131,2,115,28,116,2,100,1,160,3,116,4, + 124,0,131,1,161,1,131,1,130,1,124,2,100,2,107,0, + 114,44,116,5,100,3,131,1,130,1,124,2,100,2,107,4, + 114,84,116,0,124,1,116,1,131,2,115,72,116,2,100,4, + 131,1,130,1,110,12,124,1,115,84,116,6,100,5,131,1, + 130,1,124,0,115,104,124,2,100,2,107,2,114,104,116,5, + 100,6,131,1,130,1,100,7,83,0,41,8,122,28,86,101, + 114,105,102,121,32,97,114,103,117,109,101,110,116,115,32,97, + 114,101,32,34,115,97,110,101,34,46,122,31,109,111,100,117, + 108,101,32,110,97,109,101,32,109,117,115,116,32,98,101,32, + 115,116,114,44,32,110,111,116,32,123,125,114,22,0,0,0, + 122,18,108,101,118,101,108,32,109,117,115,116,32,98,101,32, + 62,61,32,48,122,31,95,95,112,97,99,107,97,103,101,95, + 95,32,110,111,116,32,115,101,116,32,116,111,32,97,32,115, + 116,114,105,110,103,122,54,97,116,116,101,109,112,116,101,100, + 32,114,101,108,97,116,105,118,101,32,105,109,112,111,114,116, + 32,119,105,116,104,32,110,111,32,107,110,111,119,110,32,112, + 97,114,101,110,116,32,112,97,99,107,97,103,101,122,17,69, + 109,112,116,121,32,109,111,100,117,108,101,32,110,97,109,101, + 78,41,7,218,10,105,115,105,110,115,116,97,110,99,101,218, + 3,115,116,114,218,9,84,121,112,101,69,114,114,111,114,114, + 45,0,0,0,114,14,0,0,0,218,10,86,97,108,117,101, + 69,114,114,111,114,114,79,0,0,0,169,3,114,17,0,0, + 0,114,185,0,0,0,114,186,0,0,0,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,218,13,95,115,97,110, + 105,116,121,95,99,104,101,99,107,169,3,0,0,115,22,0, + 0,0,0,2,10,1,18,1,8,1,8,1,8,1,10,1, + 10,1,4,1,8,2,12,1,114,200,0,0,0,122,16,78, + 111,32,109,111,100,117,108,101,32,110,97,109,101,100,32,122, + 4,123,33,114,125,99,2,0,0,0,0,0,0,0,0,0, + 0,0,8,0,0,0,8,0,0,0,67,0,0,0,115,220, + 0,0,0,100,0,125,2,124,0,160,0,100,1,161,1,100, + 2,25,0,125,3,124,3,114,134,124,3,116,1,106,2,107, + 7,114,42,116,3,124,1,124,3,131,2,1,0,124,0,116, + 1,106,2,107,6,114,62,116,1,106,2,124,0,25,0,83, + 0,116,1,106,2,124,3,25,0,125,4,122,10,124,4,106, + 4,125,2,87,0,110,50,4,0,116,5,107,10,114,132,1, + 0,1,0,1,0,116,6,100,3,23,0,160,7,124,0,124, + 3,161,2,125,5,116,8,124,5,124,0,100,4,141,2,100, + 0,130,2,89,0,110,2,88,0,116,9,124,0,124,2,131, + 2,125,6,124,6,100,0,107,8,114,172,116,8,116,6,160, + 7,124,0,161,1,124,0,100,4,141,2,130,1,110,8,116, + 10,124,6,131,1,125,7,124,3,114,216,116,1,106,2,124, + 3,25,0,125,4,116,11,124,4,124,0,160,0,100,1,161, + 1,100,5,25,0,124,7,131,3,1,0,124,7,83,0,41, + 6,78,114,128,0,0,0,114,22,0,0,0,122,23,59,32, + 123,33,114,125,32,105,115,32,110,111,116,32,97,32,112,97, + 99,107,97,103,101,114,16,0,0,0,233,2,0,0,0,41, + 12,114,129,0,0,0,114,15,0,0,0,114,92,0,0,0, + 114,67,0,0,0,114,141,0,0,0,114,106,0,0,0,218, + 8,95,69,82,82,95,77,83,71,114,45,0,0,0,218,19, + 77,111,100,117,108,101,78,111,116,70,111,117,110,100,69,114, + 114,111,114,114,194,0,0,0,114,159,0,0,0,114,5,0, + 0,0,41,8,114,17,0,0,0,218,7,105,109,112,111,114, + 116,95,114,164,0,0,0,114,130,0,0,0,90,13,112,97, + 114,101,110,116,95,109,111,100,117,108,101,114,157,0,0,0, + 114,95,0,0,0,114,96,0,0,0,114,10,0,0,0,114, + 10,0,0,0,114,11,0,0,0,218,23,95,102,105,110,100, + 95,97,110,100,95,108,111,97,100,95,117,110,108,111,99,107, + 101,100,188,3,0,0,115,42,0,0,0,0,1,4,1,14, + 1,4,1,10,1,10,2,10,1,10,1,10,1,2,1,10, + 1,14,1,16,1,20,1,10,1,8,1,20,2,8,1,4, + 2,10,1,22,1,114,205,0,0,0,99,2,0,0,0,0, + 0,0,0,0,0,0,0,4,0,0,0,10,0,0,0,67, + 0,0,0,115,106,0,0,0,116,0,124,0,131,1,143,50, + 1,0,116,1,106,2,160,3,124,0,116,4,161,2,125,2, + 124,2,116,4,107,8,114,54,116,5,124,0,124,1,131,2, + 87,0,2,0,53,0,81,0,82,0,163,0,83,0,87,0, + 53,0,81,0,82,0,88,0,124,2,100,1,107,8,114,94, + 100,2,160,6,124,0,161,1,125,3,116,7,124,3,124,0, + 100,3,141,2,130,1,116,8,124,0,131,1,1,0,124,2, + 83,0,41,4,122,25,70,105,110,100,32,97,110,100,32,108, + 111,97,100,32,116,104,101,32,109,111,100,117,108,101,46,78, + 122,40,105,109,112,111,114,116,32,111,102,32,123,125,32,104, + 97,108,116,101,100,59,32,78,111,110,101,32,105,110,32,115, + 121,115,46,109,111,100,117,108,101,115,114,16,0,0,0,41, + 9,114,50,0,0,0,114,15,0,0,0,114,92,0,0,0, + 114,34,0,0,0,218,14,95,78,69,69,68,83,95,76,79, + 65,68,73,78,71,114,205,0,0,0,114,45,0,0,0,114, + 203,0,0,0,114,65,0,0,0,41,4,114,17,0,0,0, + 114,204,0,0,0,114,96,0,0,0,114,75,0,0,0,114, + 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,14, + 95,102,105,110,100,95,97,110,100,95,108,111,97,100,218,3, + 0,0,115,22,0,0,0,0,2,10,1,14,1,8,1,32, + 2,8,1,4,1,2,255,4,2,12,2,8,1,114,207,0, + 0,0,114,22,0,0,0,99,3,0,0,0,0,0,0,0, + 0,0,0,0,3,0,0,0,4,0,0,0,67,0,0,0, + 115,42,0,0,0,116,0,124,0,124,1,124,2,131,3,1, + 0,124,2,100,1,107,4,114,32,116,1,124,0,124,1,124, + 2,131,3,125,0,116,2,124,0,116,3,131,2,83,0,41, + 2,97,50,1,0,0,73,109,112,111,114,116,32,97,110,100, + 32,114,101,116,117,114,110,32,116,104,101,32,109,111,100,117, + 108,101,32,98,97,115,101,100,32,111,110,32,105,116,115,32, + 110,97,109,101,44,32,116,104,101,32,112,97,99,107,97,103, + 101,32,116,104,101,32,99,97,108,108,32,105,115,10,32,32, + 32,32,98,101,105,110,103,32,109,97,100,101,32,102,114,111, + 109,44,32,97,110,100,32,116,104,101,32,108,101,118,101,108, + 32,97,100,106,117,115,116,109,101,110,116,46,10,10,32,32, + 32,32,84,104,105,115,32,102,117,110,99,116,105,111,110,32, + 114,101,112,114,101,115,101,110,116,115,32,116,104,101,32,103, + 114,101,97,116,101,115,116,32,99,111,109,109,111,110,32,100, + 101,110,111,109,105,110,97,116,111,114,32,111,102,32,102,117, + 110,99,116,105,111,110,97,108,105,116,121,10,32,32,32,32, + 98,101,116,119,101,101,110,32,105,109,112,111,114,116,95,109, + 111,100,117,108,101,32,97,110,100,32,95,95,105,109,112,111, + 114,116,95,95,46,32,84,104,105,115,32,105,110,99,108,117, + 100,101,115,32,115,101,116,116,105,110,103,32,95,95,112,97, + 99,107,97,103,101,95,95,32,105,102,10,32,32,32,32,116, + 104,101,32,108,111,97,100,101,114,32,100,105,100,32,110,111, + 116,46,10,10,32,32,32,32,114,22,0,0,0,41,4,114, + 200,0,0,0,114,187,0,0,0,114,207,0,0,0,218,11, + 95,103,99,100,95,105,109,112,111,114,116,114,199,0,0,0, 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,114, - 173,0,0,0,12,3,0,0,115,46,0,0,0,8,2,4, - 7,4,2,2,1,10,8,2,1,12,6,2,1,12,8,2, - 1,10,3,2,1,10,8,2,1,10,8,2,1,2,1,12, - 4,2,1,2,1,12,4,2,1,2,1,114,173,0,0,0, - 99,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, - 0,2,0,0,0,64,0,0,0,115,32,0,0,0,101,0, - 90,1,100,0,90,2,100,1,90,3,100,2,100,3,132,0, - 90,4,100,4,100,5,132,0,90,5,100,6,83,0,41,7, - 218,18,95,73,109,112,111,114,116,76,111,99,107,67,111,110, - 116,101,120,116,122,36,67,111,110,116,101,120,116,32,109,97, - 110,97,103,101,114,32,102,111,114,32,116,104,101,32,105,109, - 112,111,114,116,32,108,111,99,107,46,99,1,0,0,0,0, - 0,0,0,0,0,0,0,1,0,0,0,2,0,0,0,67, - 0,0,0,115,12,0,0,0,116,0,160,1,161,0,1,0, - 100,1,83,0,41,2,122,24,65,99,113,117,105,114,101,32, - 116,104,101,32,105,109,112,111,114,116,32,108,111,99,107,46, - 78,41,2,114,57,0,0,0,114,58,0,0,0,114,47,0, + 208,0,0,0,234,3,0,0,115,8,0,0,0,0,9,12, + 1,8,1,12,1,114,208,0,0,0,169,1,218,9,114,101, + 99,117,114,115,105,118,101,99,3,0,0,0,0,0,0,0, + 1,0,0,0,8,0,0,0,11,0,0,0,67,0,0,0, + 115,226,0,0,0,124,1,68,0,93,216,125,4,116,0,124, + 4,116,1,131,2,115,66,124,3,114,34,124,0,106,2,100, + 1,23,0,125,5,110,4,100,2,125,5,116,3,100,3,124, + 5,155,0,100,4,116,4,124,4,131,1,106,2,155,0,157, + 4,131,1,130,1,113,4,124,4,100,5,107,2,114,108,124, + 3,115,220,116,5,124,0,100,6,131,2,114,220,116,6,124, + 0,124,0,106,7,124,2,100,7,100,8,141,4,1,0,113, + 4,116,5,124,0,124,4,131,2,115,4,100,9,160,8,124, + 0,106,2,124,4,161,2,125,6,122,14,116,9,124,2,124, + 6,131,2,1,0,87,0,113,4,4,0,116,10,107,10,114, + 218,1,0,125,7,1,0,122,42,124,7,106,11,124,6,107, + 2,114,200,116,12,106,13,160,14,124,6,116,15,161,2,100, + 10,107,9,114,200,87,0,89,0,162,8,113,4,130,0,87, + 0,53,0,100,10,125,7,126,7,88,0,89,0,113,4,88, + 0,113,4,124,0,83,0,41,11,122,238,70,105,103,117,114, + 101,32,111,117,116,32,119,104,97,116,32,95,95,105,109,112, + 111,114,116,95,95,32,115,104,111,117,108,100,32,114,101,116, + 117,114,110,46,10,10,32,32,32,32,84,104,101,32,105,109, + 112,111,114,116,95,32,112,97,114,97,109,101,116,101,114,32, + 105,115,32,97,32,99,97,108,108,97,98,108,101,32,119,104, + 105,99,104,32,116,97,107,101,115,32,116,104,101,32,110,97, + 109,101,32,111,102,32,109,111,100,117,108,101,32,116,111,10, + 32,32,32,32,105,109,112,111,114,116,46,32,73,116,32,105, + 115,32,114,101,113,117,105,114,101,100,32,116,111,32,100,101, + 99,111,117,112,108,101,32,116,104,101,32,102,117,110,99,116, + 105,111,110,32,102,114,111,109,32,97,115,115,117,109,105,110, + 103,32,105,109,112,111,114,116,108,105,98,39,115,10,32,32, + 32,32,105,109,112,111,114,116,32,105,109,112,108,101,109,101, + 110,116,97,116,105,111,110,32,105,115,32,100,101,115,105,114, + 101,100,46,10,10,32,32,32,32,122,8,46,95,95,97,108, + 108,95,95,122,13,96,96,102,114,111,109,32,108,105,115,116, + 39,39,122,8,73,116,101,109,32,105,110,32,122,18,32,109, + 117,115,116,32,98,101,32,115,116,114,44,32,110,111,116,32, + 250,1,42,218,7,95,95,97,108,108,95,95,84,114,209,0, + 0,0,114,182,0,0,0,78,41,16,114,195,0,0,0,114, + 196,0,0,0,114,1,0,0,0,114,197,0,0,0,114,14, + 0,0,0,114,4,0,0,0,218,16,95,104,97,110,100,108, + 101,95,102,114,111,109,108,105,115,116,114,212,0,0,0,114, + 45,0,0,0,114,67,0,0,0,114,203,0,0,0,114,17, + 0,0,0,114,15,0,0,0,114,92,0,0,0,114,34,0, + 0,0,114,206,0,0,0,41,8,114,96,0,0,0,218,8, + 102,114,111,109,108,105,115,116,114,204,0,0,0,114,210,0, + 0,0,218,1,120,90,5,119,104,101,114,101,90,9,102,114, + 111,109,95,110,97,109,101,90,3,101,120,99,114,10,0,0, + 0,114,10,0,0,0,114,11,0,0,0,114,213,0,0,0, + 249,3,0,0,115,44,0,0,0,0,10,8,1,10,1,4, + 1,12,2,4,1,28,2,8,1,14,1,10,1,2,255,8, + 2,10,1,14,1,2,1,14,1,16,4,10,1,16,255,2, + 2,8,1,22,1,114,213,0,0,0,99,1,0,0,0,0, + 0,0,0,0,0,0,0,3,0,0,0,6,0,0,0,67, + 0,0,0,115,146,0,0,0,124,0,160,0,100,1,161,1, + 125,1,124,0,160,0,100,2,161,1,125,2,124,1,100,3, + 107,9,114,82,124,2,100,3,107,9,114,78,124,1,124,2, + 106,1,107,3,114,78,116,2,106,3,100,4,124,1,155,2, + 100,5,124,2,106,1,155,2,100,6,157,5,116,4,100,7, + 100,8,141,3,1,0,124,1,83,0,124,2,100,3,107,9, + 114,96,124,2,106,1,83,0,116,2,106,3,100,9,116,4, + 100,7,100,8,141,3,1,0,124,0,100,10,25,0,125,1, + 100,11,124,0,107,7,114,142,124,1,160,5,100,12,161,1, + 100,13,25,0,125,1,124,1,83,0,41,14,122,167,67,97, + 108,99,117,108,97,116,101,32,119,104,97,116,32,95,95,112, + 97,99,107,97,103,101,95,95,32,115,104,111,117,108,100,32, + 98,101,46,10,10,32,32,32,32,95,95,112,97,99,107,97, + 103,101,95,95,32,105,115,32,110,111,116,32,103,117,97,114, + 97,110,116,101,101,100,32,116,111,32,98,101,32,100,101,102, + 105,110,101,100,32,111,114,32,99,111,117,108,100,32,98,101, + 32,115,101,116,32,116,111,32,78,111,110,101,10,32,32,32, + 32,116,111,32,114,101,112,114,101,115,101,110,116,32,116,104, + 97,116,32,105,116,115,32,112,114,111,112,101,114,32,118,97, + 108,117,101,32,105,115,32,117,110,107,110,111,119,110,46,10, + 10,32,32,32,32,114,145,0,0,0,114,105,0,0,0,78, + 122,32,95,95,112,97,99,107,97,103,101,95,95,32,33,61, + 32,95,95,115,112,101,99,95,95,46,112,97,114,101,110,116, + 32,40,122,4,32,33,61,32,250,1,41,233,3,0,0,0, + 41,1,90,10,115,116,97,99,107,108,101,118,101,108,122,89, + 99,97,110,39,116,32,114,101,115,111,108,118,101,32,112,97, + 99,107,97,103,101,32,102,114,111,109,32,95,95,115,112,101, + 99,95,95,32,111,114,32,95,95,112,97,99,107,97,103,101, + 95,95,44,32,102,97,108,108,105,110,103,32,98,97,99,107, + 32,111,110,32,95,95,110,97,109,101,95,95,32,97,110,100, + 32,95,95,112,97,116,104,95,95,114,1,0,0,0,114,141, + 0,0,0,114,128,0,0,0,114,22,0,0,0,41,6,114, + 34,0,0,0,114,130,0,0,0,114,191,0,0,0,114,192, + 0,0,0,114,193,0,0,0,114,129,0,0,0,41,3,218, + 7,103,108,111,98,97,108,115,114,185,0,0,0,114,95,0, 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, - 0,114,54,0,0,0,95,3,0,0,115,2,0,0,0,0, - 2,122,28,95,73,109,112,111,114,116,76,111,99,107,67,111, - 110,116,101,120,116,46,95,95,101,110,116,101,114,95,95,99, - 4,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0, - 2,0,0,0,67,0,0,0,115,12,0,0,0,116,0,160, - 1,161,0,1,0,100,1,83,0,41,2,122,60,82,101,108, - 101,97,115,101,32,116,104,101,32,105,109,112,111,114,116,32, - 108,111,99,107,32,114,101,103,97,114,100,108,101,115,115,32, - 111,102,32,97,110,121,32,114,97,105,115,101,100,32,101,120, - 99,101,112,116,105,111,110,115,46,78,41,2,114,57,0,0, - 0,114,60,0,0,0,41,4,114,30,0,0,0,218,8,101, - 120,99,95,116,121,112,101,218,9,101,120,99,95,118,97,108, - 117,101,218,13,101,120,99,95,116,114,97,99,101,98,97,99, - 107,114,10,0,0,0,114,10,0,0,0,114,11,0,0,0, - 114,56,0,0,0,99,3,0,0,115,2,0,0,0,0,2, - 122,27,95,73,109,112,111,114,116,76,111,99,107,67,111,110, - 116,101,120,116,46,95,95,101,120,105,116,95,95,78,41,6, - 114,1,0,0,0,114,0,0,0,0,114,2,0,0,0,114, - 3,0,0,0,114,54,0,0,0,114,56,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,114,178,0,0,0,91,3,0,0,115,6,0,0,0, - 8,2,4,2,8,4,114,178,0,0,0,99,3,0,0,0, - 0,0,0,0,0,0,0,0,5,0,0,0,5,0,0,0, - 67,0,0,0,115,64,0,0,0,124,1,160,0,100,1,124, - 2,100,2,24,0,161,2,125,3,116,1,124,3,131,1,124, - 2,107,0,114,36,116,2,100,3,131,1,130,1,124,3,100, - 4,25,0,125,4,124,0,114,60,100,5,160,3,124,4,124, - 0,161,2,83,0,124,4,83,0,41,6,122,50,82,101,115, - 111,108,118,101,32,97,32,114,101,108,97,116,105,118,101,32, - 109,111,100,117,108,101,32,110,97,109,101,32,116,111,32,97, - 110,32,97,98,115,111,108,117,116,101,32,111,110,101,46,114, - 128,0,0,0,114,37,0,0,0,122,50,97,116,116,101,109, - 112,116,101,100,32,114,101,108,97,116,105,118,101,32,105,109, - 112,111,114,116,32,98,101,121,111,110,100,32,116,111,112,45, - 108,101,118,101,108,32,112,97,99,107,97,103,101,114,22,0, - 0,0,250,5,123,125,46,123,125,41,4,218,6,114,115,112, - 108,105,116,218,3,108,101,110,114,79,0,0,0,114,45,0, - 0,0,41,5,114,17,0,0,0,218,7,112,97,99,107,97, - 103,101,218,5,108,101,118,101,108,90,4,98,105,116,115,90, - 4,98,97,115,101,114,10,0,0,0,114,10,0,0,0,114, - 11,0,0,0,218,13,95,114,101,115,111,108,118,101,95,110, - 97,109,101,104,3,0,0,115,10,0,0,0,0,2,16,1, - 12,1,8,1,8,1,114,187,0,0,0,99,3,0,0,0, - 0,0,0,0,0,0,0,0,4,0,0,0,4,0,0,0, - 67,0,0,0,115,34,0,0,0,124,0,160,0,124,1,124, - 2,161,2,125,3,124,3,100,0,107,8,114,24,100,0,83, - 0,116,1,124,1,124,3,131,2,83,0,114,13,0,0,0, - 41,2,114,167,0,0,0,114,91,0,0,0,41,4,218,6, - 102,105,110,100,101,114,114,17,0,0,0,114,164,0,0,0, - 114,109,0,0,0,114,10,0,0,0,114,10,0,0,0,114, - 11,0,0,0,218,17,95,102,105,110,100,95,115,112,101,99, - 95,108,101,103,97,99,121,113,3,0,0,115,8,0,0,0, - 0,3,12,1,8,1,4,1,114,189,0,0,0,99,3,0, - 0,0,0,0,0,0,0,0,0,0,10,0,0,0,10,0, - 0,0,67,0,0,0,115,12,1,0,0,116,0,106,1,125, - 3,124,3,100,1,107,8,114,22,116,2,100,2,131,1,130, - 1,124,3,115,38,116,3,160,4,100,3,116,5,161,2,1, - 0,124,0,116,0,106,6,107,6,125,4,124,3,68,0,93, - 210,125,5,116,7,131,0,143,84,1,0,122,10,124,5,106, - 8,125,6,87,0,110,54,4,0,116,9,107,10,114,128,1, - 0,1,0,1,0,116,10,124,5,124,0,124,1,131,3,125, - 7,124,7,100,1,107,8,114,124,89,0,87,0,53,0,81, - 0,82,0,163,0,113,52,89,0,110,14,88,0,124,6,124, - 0,124,1,124,2,131,3,125,7,87,0,53,0,81,0,82, - 0,88,0,124,7,100,1,107,9,114,52,124,4,144,0,115, - 254,124,0,116,0,106,6,107,6,144,0,114,254,116,0,106, - 6,124,0,25,0,125,8,122,10,124,8,106,11,125,9,87, - 0,110,28,4,0,116,9,107,10,114,226,1,0,1,0,1, - 0,124,7,6,0,89,0,2,0,1,0,83,0,88,0,124, - 9,100,1,107,8,114,244,124,7,2,0,1,0,83,0,124, - 9,2,0,1,0,83,0,113,52,124,7,2,0,1,0,83, - 0,113,52,100,1,83,0,41,4,122,21,70,105,110,100,32, - 97,32,109,111,100,117,108,101,39,115,32,115,112,101,99,46, - 78,122,53,115,121,115,46,109,101,116,97,95,112,97,116,104, - 32,105,115,32,78,111,110,101,44,32,80,121,116,104,111,110, - 32,105,115,32,108,105,107,101,108,121,32,115,104,117,116,116, - 105,110,103,32,100,111,119,110,122,22,115,121,115,46,109,101, - 116,97,95,112,97,116,104,32,105,115,32,101,109,112,116,121, - 41,12,114,15,0,0,0,218,9,109,101,116,97,95,112,97, - 116,104,114,79,0,0,0,218,9,95,119,97,114,110,105,110, - 103,115,218,4,119,97,114,110,218,13,73,109,112,111,114,116, - 87,97,114,110,105,110,103,114,92,0,0,0,114,178,0,0, - 0,114,166,0,0,0,114,106,0,0,0,114,189,0,0,0, - 114,105,0,0,0,41,10,114,17,0,0,0,114,164,0,0, - 0,114,165,0,0,0,114,190,0,0,0,90,9,105,115,95, - 114,101,108,111,97,100,114,188,0,0,0,114,166,0,0,0, - 114,95,0,0,0,114,96,0,0,0,114,105,0,0,0,114, - 10,0,0,0,114,10,0,0,0,114,11,0,0,0,218,10, - 95,102,105,110,100,95,115,112,101,99,122,3,0,0,115,54, - 0,0,0,0,2,6,1,8,2,8,3,4,1,12,5,10, - 1,8,1,8,1,2,1,10,1,14,1,12,1,8,1,20, - 2,22,1,8,2,18,1,10,1,2,1,10,1,14,4,14, - 2,8,1,8,2,10,2,10,2,114,194,0,0,0,99,3, - 0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,5, - 0,0,0,67,0,0,0,115,108,0,0,0,116,0,124,0, - 116,1,131,2,115,28,116,2,100,1,160,3,116,4,124,0, - 131,1,161,1,131,1,130,1,124,2,100,2,107,0,114,44, - 116,5,100,3,131,1,130,1,124,2,100,2,107,4,114,84, - 116,0,124,1,116,1,131,2,115,72,116,2,100,4,131,1, - 130,1,110,12,124,1,115,84,116,6,100,5,131,1,130,1, - 124,0,115,104,124,2,100,2,107,2,114,104,116,5,100,6, - 131,1,130,1,100,7,83,0,41,8,122,28,86,101,114,105, - 102,121,32,97,114,103,117,109,101,110,116,115,32,97,114,101, - 32,34,115,97,110,101,34,46,122,31,109,111,100,117,108,101, - 32,110,97,109,101,32,109,117,115,116,32,98,101,32,115,116, - 114,44,32,110,111,116,32,123,125,114,22,0,0,0,122,18, - 108,101,118,101,108,32,109,117,115,116,32,98,101,32,62,61, - 32,48,122,31,95,95,112,97,99,107,97,103,101,95,95,32, - 110,111,116,32,115,101,116,32,116,111,32,97,32,115,116,114, - 105,110,103,122,54,97,116,116,101,109,112,116,101,100,32,114, - 101,108,97,116,105,118,101,32,105,109,112,111,114,116,32,119, - 105,116,104,32,110,111,32,107,110,111,119,110,32,112,97,114, - 101,110,116,32,112,97,99,107,97,103,101,122,17,69,109,112, - 116,121,32,109,111,100,117,108,101,32,110,97,109,101,78,41, - 7,218,10,105,115,105,110,115,116,97,110,99,101,218,3,115, - 116,114,218,9,84,121,112,101,69,114,114,111,114,114,45,0, - 0,0,114,14,0,0,0,218,10,86,97,108,117,101,69,114, - 114,111,114,114,79,0,0,0,169,3,114,17,0,0,0,114, - 185,0,0,0,114,186,0,0,0,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,218,13,95,115,97,110,105,116, - 121,95,99,104,101,99,107,169,3,0,0,115,22,0,0,0, - 0,2,10,1,18,1,8,1,8,1,8,1,10,1,10,1, - 4,1,8,2,12,1,114,200,0,0,0,122,16,78,111,32, - 109,111,100,117,108,101,32,110,97,109,101,100,32,122,4,123, - 33,114,125,99,2,0,0,0,0,0,0,0,0,0,0,0, - 8,0,0,0,8,0,0,0,67,0,0,0,115,220,0,0, - 0,100,0,125,2,124,0,160,0,100,1,161,1,100,2,25, - 0,125,3,124,3,114,134,124,3,116,1,106,2,107,7,114, - 42,116,3,124,1,124,3,131,2,1,0,124,0,116,1,106, - 2,107,6,114,62,116,1,106,2,124,0,25,0,83,0,116, - 1,106,2,124,3,25,0,125,4,122,10,124,4,106,4,125, - 2,87,0,110,50,4,0,116,5,107,10,114,132,1,0,1, - 0,1,0,116,6,100,3,23,0,160,7,124,0,124,3,161, - 2,125,5,116,8,124,5,124,0,100,4,141,2,100,0,130, - 2,89,0,110,2,88,0,116,9,124,0,124,2,131,2,125, - 6,124,6,100,0,107,8,114,172,116,8,116,6,160,7,124, - 0,161,1,124,0,100,4,141,2,130,1,110,8,116,10,124, - 6,131,1,125,7,124,3,114,216,116,1,106,2,124,3,25, - 0,125,4,116,11,124,4,124,0,160,0,100,1,161,1,100, - 5,25,0,124,7,131,3,1,0,124,7,83,0,41,6,78, - 114,128,0,0,0,114,22,0,0,0,122,23,59,32,123,33, - 114,125,32,105,115,32,110,111,116,32,97,32,112,97,99,107, - 97,103,101,114,16,0,0,0,233,2,0,0,0,41,12,114, - 129,0,0,0,114,15,0,0,0,114,92,0,0,0,114,67, - 0,0,0,114,141,0,0,0,114,106,0,0,0,218,8,95, - 69,82,82,95,77,83,71,114,45,0,0,0,218,19,77,111, - 100,117,108,101,78,111,116,70,111,117,110,100,69,114,114,111, - 114,114,194,0,0,0,114,159,0,0,0,114,5,0,0,0, - 41,8,114,17,0,0,0,218,7,105,109,112,111,114,116,95, - 114,164,0,0,0,114,130,0,0,0,90,13,112,97,114,101, - 110,116,95,109,111,100,117,108,101,114,157,0,0,0,114,95, - 0,0,0,114,96,0,0,0,114,10,0,0,0,114,10,0, - 0,0,114,11,0,0,0,218,23,95,102,105,110,100,95,97, - 110,100,95,108,111,97,100,95,117,110,108,111,99,107,101,100, - 188,3,0,0,115,42,0,0,0,0,1,4,1,14,1,4, - 1,10,1,10,2,10,1,10,1,10,1,2,1,10,1,14, - 1,16,1,20,1,10,1,8,1,20,2,8,1,4,2,10, - 1,22,1,114,205,0,0,0,99,2,0,0,0,0,0,0, - 0,0,0,0,0,4,0,0,0,10,0,0,0,67,0,0, - 0,115,106,0,0,0,116,0,124,0,131,1,143,50,1,0, - 116,1,106,2,160,3,124,0,116,4,161,2,125,2,124,2, - 116,4,107,8,114,54,116,5,124,0,124,1,131,2,87,0, - 2,0,53,0,81,0,82,0,163,0,83,0,87,0,53,0, - 81,0,82,0,88,0,124,2,100,1,107,8,114,94,100,2, - 160,6,124,0,161,1,125,3,116,7,124,3,124,0,100,3, - 141,2,130,1,116,8,124,0,131,1,1,0,124,2,83,0, - 41,4,122,25,70,105,110,100,32,97,110,100,32,108,111,97, - 100,32,116,104,101,32,109,111,100,117,108,101,46,78,122,40, - 105,109,112,111,114,116,32,111,102,32,123,125,32,104,97,108, - 116,101,100,59,32,78,111,110,101,32,105,110,32,115,121,115, - 46,109,111,100,117,108,101,115,114,16,0,0,0,41,9,114, - 50,0,0,0,114,15,0,0,0,114,92,0,0,0,114,34, - 0,0,0,218,14,95,78,69,69,68,83,95,76,79,65,68, - 73,78,71,114,205,0,0,0,114,45,0,0,0,114,203,0, - 0,0,114,65,0,0,0,41,4,114,17,0,0,0,114,204, - 0,0,0,114,96,0,0,0,114,75,0,0,0,114,10,0, - 0,0,114,10,0,0,0,114,11,0,0,0,218,14,95,102, - 105,110,100,95,97,110,100,95,108,111,97,100,218,3,0,0, - 115,22,0,0,0,0,2,10,1,14,1,8,1,32,2,8, - 1,4,1,2,255,4,2,12,2,8,1,114,207,0,0,0, - 114,22,0,0,0,99,3,0,0,0,0,0,0,0,0,0, - 0,0,3,0,0,0,4,0,0,0,67,0,0,0,115,42, - 0,0,0,116,0,124,0,124,1,124,2,131,3,1,0,124, - 2,100,1,107,4,114,32,116,1,124,0,124,1,124,2,131, - 3,125,0,116,2,124,0,116,3,131,2,83,0,41,2,97, - 50,1,0,0,73,109,112,111,114,116,32,97,110,100,32,114, - 101,116,117,114,110,32,116,104,101,32,109,111,100,117,108,101, - 32,98,97,115,101,100,32,111,110,32,105,116,115,32,110,97, - 109,101,44,32,116,104,101,32,112,97,99,107,97,103,101,32, - 116,104,101,32,99,97,108,108,32,105,115,10,32,32,32,32, - 98,101,105,110,103,32,109,97,100,101,32,102,114,111,109,44, - 32,97,110,100,32,116,104,101,32,108,101,118,101,108,32,97, - 100,106,117,115,116,109,101,110,116,46,10,10,32,32,32,32, - 84,104,105,115,32,102,117,110,99,116,105,111,110,32,114,101, - 112,114,101,115,101,110,116,115,32,116,104,101,32,103,114,101, - 97,116,101,115,116,32,99,111,109,109,111,110,32,100,101,110, - 111,109,105,110,97,116,111,114,32,111,102,32,102,117,110,99, - 116,105,111,110,97,108,105,116,121,10,32,32,32,32,98,101, - 116,119,101,101,110,32,105,109,112,111,114,116,95,109,111,100, - 117,108,101,32,97,110,100,32,95,95,105,109,112,111,114,116, - 95,95,46,32,84,104,105,115,32,105,110,99,108,117,100,101, - 115,32,115,101,116,116,105,110,103,32,95,95,112,97,99,107, - 97,103,101,95,95,32,105,102,10,32,32,32,32,116,104,101, - 32,108,111,97,100,101,114,32,100,105,100,32,110,111,116,46, - 10,10,32,32,32,32,114,22,0,0,0,41,4,114,200,0, - 0,0,114,187,0,0,0,114,207,0,0,0,218,11,95,103, - 99,100,95,105,109,112,111,114,116,114,199,0,0,0,114,10, - 0,0,0,114,10,0,0,0,114,11,0,0,0,114,208,0, - 0,0,234,3,0,0,115,8,0,0,0,0,9,12,1,8, - 1,12,1,114,208,0,0,0,169,1,218,9,114,101,99,117, - 114,115,105,118,101,99,3,0,0,0,0,0,0,0,1,0, - 0,0,8,0,0,0,11,0,0,0,67,0,0,0,115,226, - 0,0,0,124,1,68,0,93,216,125,4,116,0,124,4,116, - 1,131,2,115,66,124,3,114,34,124,0,106,2,100,1,23, - 0,125,5,110,4,100,2,125,5,116,3,100,3,124,5,155, - 0,100,4,116,4,124,4,131,1,106,2,155,0,157,4,131, - 1,130,1,113,4,124,4,100,5,107,2,114,108,124,3,115, - 220,116,5,124,0,100,6,131,2,114,220,116,6,124,0,124, - 0,106,7,124,2,100,7,100,8,141,4,1,0,113,4,116, - 5,124,0,124,4,131,2,115,4,100,9,160,8,124,0,106, - 2,124,4,161,2,125,6,122,14,116,9,124,2,124,6,131, - 2,1,0,87,0,113,4,4,0,116,10,107,10,114,218,1, - 0,125,7,1,0,122,42,124,7,106,11,124,6,107,2,114, - 200,116,12,106,13,160,14,124,6,116,15,161,2,100,10,107, - 9,114,200,87,0,89,0,162,8,113,4,130,0,87,0,53, - 0,100,10,125,7,126,7,88,0,89,0,113,4,88,0,113, - 4,124,0,83,0,41,11,122,238,70,105,103,117,114,101,32, - 111,117,116,32,119,104,97,116,32,95,95,105,109,112,111,114, - 116,95,95,32,115,104,111,117,108,100,32,114,101,116,117,114, - 110,46,10,10,32,32,32,32,84,104,101,32,105,109,112,111, - 114,116,95,32,112,97,114,97,109,101,116,101,114,32,105,115, - 32,97,32,99,97,108,108,97,98,108,101,32,119,104,105,99, - 104,32,116,97,107,101,115,32,116,104,101,32,110,97,109,101, - 32,111,102,32,109,111,100,117,108,101,32,116,111,10,32,32, - 32,32,105,109,112,111,114,116,46,32,73,116,32,105,115,32, - 114,101,113,117,105,114,101,100,32,116,111,32,100,101,99,111, - 117,112,108,101,32,116,104,101,32,102,117,110,99,116,105,111, - 110,32,102,114,111,109,32,97,115,115,117,109,105,110,103,32, - 105,109,112,111,114,116,108,105,98,39,115,10,32,32,32,32, - 105,109,112,111,114,116,32,105,109,112,108,101,109,101,110,116, - 97,116,105,111,110,32,105,115,32,100,101,115,105,114,101,100, - 46,10,10,32,32,32,32,122,8,46,95,95,97,108,108,95, - 95,122,13,96,96,102,114,111,109,32,108,105,115,116,39,39, - 122,8,73,116,101,109,32,105,110,32,122,18,32,109,117,115, - 116,32,98,101,32,115,116,114,44,32,110,111,116,32,250,1, - 42,218,7,95,95,97,108,108,95,95,84,114,209,0,0,0, - 114,182,0,0,0,78,41,16,114,195,0,0,0,114,196,0, - 0,0,114,1,0,0,0,114,197,0,0,0,114,14,0,0, - 0,114,4,0,0,0,218,16,95,104,97,110,100,108,101,95, - 102,114,111,109,108,105,115,116,114,212,0,0,0,114,45,0, - 0,0,114,67,0,0,0,114,203,0,0,0,114,17,0,0, - 0,114,15,0,0,0,114,92,0,0,0,114,34,0,0,0, - 114,206,0,0,0,41,8,114,96,0,0,0,218,8,102,114, - 111,109,108,105,115,116,114,204,0,0,0,114,210,0,0,0, - 218,1,120,90,5,119,104,101,114,101,90,9,102,114,111,109, - 95,110,97,109,101,90,3,101,120,99,114,10,0,0,0,114, - 10,0,0,0,114,11,0,0,0,114,213,0,0,0,249,3, - 0,0,115,44,0,0,0,0,10,8,1,10,1,4,1,12, - 2,4,1,28,2,8,1,14,1,10,1,2,255,8,2,10, - 1,14,1,2,1,14,1,16,4,10,1,16,255,2,2,8, - 1,22,1,114,213,0,0,0,99,1,0,0,0,0,0,0, - 0,0,0,0,0,3,0,0,0,6,0,0,0,67,0,0, - 0,115,146,0,0,0,124,0,160,0,100,1,161,1,125,1, - 124,0,160,0,100,2,161,1,125,2,124,1,100,3,107,9, - 114,82,124,2,100,3,107,9,114,78,124,1,124,2,106,1, - 107,3,114,78,116,2,106,3,100,4,124,1,155,2,100,5, - 124,2,106,1,155,2,100,6,157,5,116,4,100,7,100,8, - 141,3,1,0,124,1,83,0,124,2,100,3,107,9,114,96, - 124,2,106,1,83,0,116,2,106,3,100,9,116,4,100,7, - 100,8,141,3,1,0,124,0,100,10,25,0,125,1,100,11, - 124,0,107,7,114,142,124,1,160,5,100,12,161,1,100,13, - 25,0,125,1,124,1,83,0,41,14,122,167,67,97,108,99, - 117,108,97,116,101,32,119,104,97,116,32,95,95,112,97,99, - 107,97,103,101,95,95,32,115,104,111,117,108,100,32,98,101, - 46,10,10,32,32,32,32,95,95,112,97,99,107,97,103,101, - 95,95,32,105,115,32,110,111,116,32,103,117,97,114,97,110, - 116,101,101,100,32,116,111,32,98,101,32,100,101,102,105,110, - 101,100,32,111,114,32,99,111,117,108,100,32,98,101,32,115, - 101,116,32,116,111,32,78,111,110,101,10,32,32,32,32,116, - 111,32,114,101,112,114,101,115,101,110,116,32,116,104,97,116, - 32,105,116,115,32,112,114,111,112,101,114,32,118,97,108,117, - 101,32,105,115,32,117,110,107,110,111,119,110,46,10,10,32, - 32,32,32,114,145,0,0,0,114,105,0,0,0,78,122,32, - 95,95,112,97,99,107,97,103,101,95,95,32,33,61,32,95, - 95,115,112,101,99,95,95,46,112,97,114,101,110,116,32,40, - 122,4,32,33,61,32,250,1,41,233,3,0,0,0,41,1, - 90,10,115,116,97,99,107,108,101,118,101,108,122,89,99,97, - 110,39,116,32,114,101,115,111,108,118,101,32,112,97,99,107, - 97,103,101,32,102,114,111,109,32,95,95,115,112,101,99,95, - 95,32,111,114,32,95,95,112,97,99,107,97,103,101,95,95, - 44,32,102,97,108,108,105,110,103,32,98,97,99,107,32,111, - 110,32,95,95,110,97,109,101,95,95,32,97,110,100,32,95, - 95,112,97,116,104,95,95,114,1,0,0,0,114,141,0,0, - 0,114,128,0,0,0,114,22,0,0,0,41,6,114,34,0, - 0,0,114,130,0,0,0,114,191,0,0,0,114,192,0,0, - 0,114,193,0,0,0,114,129,0,0,0,41,3,218,7,103, - 108,111,98,97,108,115,114,185,0,0,0,114,95,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, - 17,95,99,97,108,99,95,95,95,112,97,99,107,97,103,101, - 95,95,30,4,0,0,115,38,0,0,0,0,7,10,1,10, - 1,8,1,18,1,22,2,2,0,2,254,6,3,4,1,8, - 1,6,2,6,2,2,0,2,254,6,3,8,1,8,1,14, - 1,114,219,0,0,0,114,10,0,0,0,99,5,0,0,0, - 0,0,0,0,0,0,0,0,9,0,0,0,5,0,0,0, - 67,0,0,0,115,180,0,0,0,124,4,100,1,107,2,114, - 18,116,0,124,0,131,1,125,5,110,36,124,1,100,2,107, - 9,114,30,124,1,110,2,105,0,125,6,116,1,124,6,131, - 1,125,7,116,0,124,0,124,7,124,4,131,3,125,5,124, - 3,115,150,124,4,100,1,107,2,114,84,116,0,124,0,160, - 2,100,3,161,1,100,1,25,0,131,1,83,0,124,0,115, - 92,124,5,83,0,116,3,124,0,131,1,116,3,124,0,160, - 2,100,3,161,1,100,1,25,0,131,1,24,0,125,8,116, - 4,106,5,124,5,106,6,100,2,116,3,124,5,106,6,131, - 1,124,8,24,0,133,2,25,0,25,0,83,0,110,26,116, - 7,124,5,100,4,131,2,114,172,116,8,124,5,124,3,116, - 0,131,3,83,0,124,5,83,0,100,2,83,0,41,5,97, - 215,1,0,0,73,109,112,111,114,116,32,97,32,109,111,100, - 117,108,101,46,10,10,32,32,32,32,84,104,101,32,39,103, - 108,111,98,97,108,115,39,32,97,114,103,117,109,101,110,116, - 32,105,115,32,117,115,101,100,32,116,111,32,105,110,102,101, - 114,32,119,104,101,114,101,32,116,104,101,32,105,109,112,111, - 114,116,32,105,115,32,111,99,99,117,114,114,105,110,103,32, - 102,114,111,109,10,32,32,32,32,116,111,32,104,97,110,100, - 108,101,32,114,101,108,97,116,105,118,101,32,105,109,112,111, - 114,116,115,46,32,84,104,101,32,39,108,111,99,97,108,115, - 39,32,97,114,103,117,109,101,110,116,32,105,115,32,105,103, - 110,111,114,101,100,46,32,84,104,101,10,32,32,32,32,39, - 102,114,111,109,108,105,115,116,39,32,97,114,103,117,109,101, - 110,116,32,115,112,101,99,105,102,105,101,115,32,119,104,97, - 116,32,115,104,111,117,108,100,32,101,120,105,115,116,32,97, - 115,32,97,116,116,114,105,98,117,116,101,115,32,111,110,32, - 116,104,101,32,109,111,100,117,108,101,10,32,32,32,32,98, - 101,105,110,103,32,105,109,112,111,114,116,101,100,32,40,101, - 46,103,46,32,96,96,102,114,111,109,32,109,111,100,117,108, - 101,32,105,109,112,111,114,116,32,60,102,114,111,109,108,105, - 115,116,62,96,96,41,46,32,32,84,104,101,32,39,108,101, - 118,101,108,39,10,32,32,32,32,97,114,103,117,109,101,110, - 116,32,114,101,112,114,101,115,101,110,116,115,32,116,104,101, - 32,112,97,99,107,97,103,101,32,108,111,99,97,116,105,111, - 110,32,116,111,32,105,109,112,111,114,116,32,102,114,111,109, - 32,105,110,32,97,32,114,101,108,97,116,105,118,101,10,32, - 32,32,32,105,109,112,111,114,116,32,40,101,46,103,46,32, - 96,96,102,114,111,109,32,46,46,112,107,103,32,105,109,112, - 111,114,116,32,109,111,100,96,96,32,119,111,117,108,100,32, - 104,97,118,101,32,97,32,39,108,101,118,101,108,39,32,111, - 102,32,50,41,46,10,10,32,32,32,32,114,22,0,0,0, - 78,114,128,0,0,0,114,141,0,0,0,41,9,114,208,0, - 0,0,114,219,0,0,0,218,9,112,97,114,116,105,116,105, - 111,110,114,184,0,0,0,114,15,0,0,0,114,92,0,0, - 0,114,1,0,0,0,114,4,0,0,0,114,213,0,0,0, - 41,9,114,17,0,0,0,114,218,0,0,0,218,6,108,111, - 99,97,108,115,114,214,0,0,0,114,186,0,0,0,114,96, - 0,0,0,90,8,103,108,111,98,97,108,115,95,114,185,0, - 0,0,90,7,99,117,116,95,111,102,102,114,10,0,0,0, - 114,10,0,0,0,114,11,0,0,0,218,10,95,95,105,109, - 112,111,114,116,95,95,57,4,0,0,115,30,0,0,0,0, - 11,8,1,10,2,16,1,8,1,12,1,4,3,8,1,18, - 1,4,1,4,4,26,3,32,1,10,1,12,2,114,222,0, - 0,0,99,1,0,0,0,0,0,0,0,0,0,0,0,2, - 0,0,0,3,0,0,0,67,0,0,0,115,38,0,0,0, - 116,0,160,1,124,0,161,1,125,1,124,1,100,0,107,8, - 114,30,116,2,100,1,124,0,23,0,131,1,130,1,116,3, - 124,1,131,1,83,0,41,2,78,122,25,110,111,32,98,117, - 105,108,116,45,105,110,32,109,111,100,117,108,101,32,110,97, - 109,101,100,32,41,4,114,160,0,0,0,114,166,0,0,0, - 114,79,0,0,0,114,159,0,0,0,41,2,114,17,0,0, - 0,114,95,0,0,0,114,10,0,0,0,114,10,0,0,0, - 114,11,0,0,0,218,18,95,98,117,105,108,116,105,110,95, - 102,114,111,109,95,110,97,109,101,94,4,0,0,115,8,0, - 0,0,0,1,10,1,8,1,12,1,114,223,0,0,0,99, - 2,0,0,0,0,0,0,0,0,0,0,0,10,0,0,0, - 5,0,0,0,67,0,0,0,115,166,0,0,0,124,1,97, - 0,124,0,97,1,116,2,116,1,131,1,125,2,116,1,106, - 3,160,4,161,0,68,0,93,72,92,2,125,3,125,4,116, - 5,124,4,124,2,131,2,114,26,124,3,116,1,106,6,107, - 6,114,60,116,7,125,5,110,18,116,0,160,8,124,3,161, - 1,114,26,116,9,125,5,110,2,113,26,116,10,124,4,124, - 5,131,2,125,6,116,11,124,6,124,4,131,2,1,0,113, - 26,116,1,106,3,116,12,25,0,125,7,100,1,68,0,93, - 46,125,8,124,8,116,1,106,3,107,7,114,138,116,13,124, - 8,131,1,125,9,110,10,116,1,106,3,124,8,25,0,125, - 9,116,14,124,7,124,8,124,9,131,3,1,0,113,114,100, - 2,83,0,41,3,122,250,83,101,116,117,112,32,105,109,112, - 111,114,116,108,105,98,32,98,121,32,105,109,112,111,114,116, - 105,110,103,32,110,101,101,100,101,100,32,98,117,105,108,116, - 45,105,110,32,109,111,100,117,108,101,115,32,97,110,100,32, - 105,110,106,101,99,116,105,110,103,32,116,104,101,109,10,32, - 32,32,32,105,110,116,111,32,116,104,101,32,103,108,111,98, - 97,108,32,110,97,109,101,115,112,97,99,101,46,10,10,32, - 32,32,32,65,115,32,115,121,115,32,105,115,32,110,101,101, - 100,101,100,32,102,111,114,32,115,121,115,46,109,111,100,117, - 108,101,115,32,97,99,99,101,115,115,32,97,110,100,32,95, - 105,109,112,32,105,115,32,110,101,101,100,101,100,32,116,111, - 32,108,111,97,100,32,98,117,105,108,116,45,105,110,10,32, - 32,32,32,109,111,100,117,108,101,115,44,32,116,104,111,115, - 101,32,116,119,111,32,109,111,100,117,108,101,115,32,109,117, - 115,116,32,98,101,32,101,120,112,108,105,99,105,116,108,121, - 32,112,97,115,115,101,100,32,105,110,46,10,10,32,32,32, - 32,41,3,114,23,0,0,0,114,191,0,0,0,114,64,0, - 0,0,78,41,15,114,57,0,0,0,114,15,0,0,0,114, - 14,0,0,0,114,92,0,0,0,218,5,105,116,101,109,115, - 114,195,0,0,0,114,78,0,0,0,114,160,0,0,0,114, - 88,0,0,0,114,173,0,0,0,114,142,0,0,0,114,148, - 0,0,0,114,1,0,0,0,114,223,0,0,0,114,5,0, - 0,0,41,10,218,10,115,121,115,95,109,111,100,117,108,101, - 218,11,95,105,109,112,95,109,111,100,117,108,101,90,11,109, - 111,100,117,108,101,95,116,121,112,101,114,17,0,0,0,114, - 96,0,0,0,114,109,0,0,0,114,95,0,0,0,90,11, - 115,101,108,102,95,109,111,100,117,108,101,90,12,98,117,105, - 108,116,105,110,95,110,97,109,101,90,14,98,117,105,108,116, - 105,110,95,109,111,100,117,108,101,114,10,0,0,0,114,10, - 0,0,0,114,11,0,0,0,218,6,95,115,101,116,117,112, - 101,4,0,0,115,36,0,0,0,0,9,4,1,4,3,8, - 1,18,1,10,1,10,1,6,1,10,1,6,2,2,1,10, - 1,12,3,10,1,8,1,10,1,10,2,10,1,114,227,0, - 0,0,99,2,0,0,0,0,0,0,0,0,0,0,0,2, - 0,0,0,3,0,0,0,67,0,0,0,115,38,0,0,0, - 116,0,124,0,124,1,131,2,1,0,116,1,106,2,160,3, - 116,4,161,1,1,0,116,1,106,2,160,3,116,5,161,1, - 1,0,100,1,83,0,41,2,122,48,73,110,115,116,97,108, - 108,32,105,109,112,111,114,116,101,114,115,32,102,111,114,32, - 98,117,105,108,116,105,110,32,97,110,100,32,102,114,111,122, - 101,110,32,109,111,100,117,108,101,115,78,41,6,114,227,0, - 0,0,114,15,0,0,0,114,190,0,0,0,114,120,0,0, - 0,114,160,0,0,0,114,173,0,0,0,41,2,114,225,0, - 0,0,114,226,0,0,0,114,10,0,0,0,114,10,0,0, - 0,114,11,0,0,0,218,8,95,105,110,115,116,97,108,108, - 136,4,0,0,115,6,0,0,0,0,2,10,2,12,1,114, - 228,0,0,0,99,0,0,0,0,0,0,0,0,0,0,0, - 0,1,0,0,0,4,0,0,0,67,0,0,0,115,32,0, - 0,0,100,1,100,2,108,0,125,0,124,0,97,1,124,0, - 160,2,116,3,106,4,116,5,25,0,161,1,1,0,100,2, - 83,0,41,3,122,57,73,110,115,116,97,108,108,32,105,109, - 112,111,114,116,101,114,115,32,116,104,97,116,32,114,101,113, - 117,105,114,101,32,101,120,116,101,114,110,97,108,32,102,105, - 108,101,115,121,115,116,101,109,32,97,99,99,101,115,115,114, - 22,0,0,0,78,41,6,218,26,95,102,114,111,122,101,110, - 95,105,109,112,111,114,116,108,105,98,95,101,120,116,101,114, - 110,97,108,114,126,0,0,0,114,228,0,0,0,114,15,0, - 0,0,114,92,0,0,0,114,1,0,0,0,41,1,114,229, - 0,0,0,114,10,0,0,0,114,10,0,0,0,114,11,0, - 0,0,218,27,95,105,110,115,116,97,108,108,95,101,120,116, - 101,114,110,97,108,95,105,109,112,111,114,116,101,114,115,144, - 4,0,0,115,6,0,0,0,0,3,8,1,4,1,114,230, - 0,0,0,41,2,78,78,41,1,78,41,2,78,114,22,0, - 0,0,41,4,78,78,114,10,0,0,0,114,22,0,0,0, - 41,50,114,3,0,0,0,114,126,0,0,0,114,12,0,0, - 0,114,18,0,0,0,114,59,0,0,0,114,33,0,0,0, - 114,42,0,0,0,114,19,0,0,0,114,20,0,0,0,114, - 49,0,0,0,114,50,0,0,0,114,53,0,0,0,114,65, - 0,0,0,114,67,0,0,0,114,76,0,0,0,114,86,0, - 0,0,114,90,0,0,0,114,97,0,0,0,114,111,0,0, - 0,114,112,0,0,0,114,91,0,0,0,114,142,0,0,0, - 114,148,0,0,0,114,152,0,0,0,114,107,0,0,0,114, - 93,0,0,0,114,158,0,0,0,114,159,0,0,0,114,94, - 0,0,0,114,160,0,0,0,114,173,0,0,0,114,178,0, - 0,0,114,187,0,0,0,114,189,0,0,0,114,194,0,0, - 0,114,200,0,0,0,90,15,95,69,82,82,95,77,83,71, - 95,80,82,69,70,73,88,114,202,0,0,0,114,205,0,0, - 0,218,6,111,98,106,101,99,116,114,206,0,0,0,114,207, - 0,0,0,114,208,0,0,0,114,213,0,0,0,114,219,0, - 0,0,114,222,0,0,0,114,223,0,0,0,114,227,0,0, - 0,114,228,0,0,0,114,230,0,0,0,114,10,0,0,0, - 114,10,0,0,0,114,10,0,0,0,114,11,0,0,0,218, - 8,60,109,111,100,117,108,101,62,1,0,0,0,115,94,0, - 0,0,4,24,4,2,8,8,8,8,4,2,4,3,16,4, - 14,68,14,21,14,16,8,37,8,17,8,11,14,8,8,11, - 8,12,8,16,8,36,14,101,16,26,10,45,14,72,8,17, - 8,17,8,30,8,37,8,42,8,15,14,73,14,79,14,13, - 8,9,8,9,10,47,8,16,4,1,8,2,8,27,6,3, - 8,16,10,15,14,37,8,27,10,37,8,7,8,35,8,8, + 0,218,17,95,99,97,108,99,95,95,95,112,97,99,107,97, + 103,101,95,95,30,4,0,0,115,38,0,0,0,0,7,10, + 1,10,1,8,1,18,1,22,2,2,0,2,254,6,3,4, + 1,8,1,6,2,6,2,2,0,2,254,6,3,8,1,8, + 1,14,1,114,219,0,0,0,114,10,0,0,0,99,5,0, + 0,0,0,0,0,0,0,0,0,0,9,0,0,0,5,0, + 0,0,67,0,0,0,115,180,0,0,0,124,4,100,1,107, + 2,114,18,116,0,124,0,131,1,125,5,110,36,124,1,100, + 2,107,9,114,30,124,1,110,2,105,0,125,6,116,1,124, + 6,131,1,125,7,116,0,124,0,124,7,124,4,131,3,125, + 5,124,3,115,150,124,4,100,1,107,2,114,84,116,0,124, + 0,160,2,100,3,161,1,100,1,25,0,131,1,83,0,124, + 0,115,92,124,5,83,0,116,3,124,0,131,1,116,3,124, + 0,160,2,100,3,161,1,100,1,25,0,131,1,24,0,125, + 8,116,4,106,5,124,5,106,6,100,2,116,3,124,5,106, + 6,131,1,124,8,24,0,133,2,25,0,25,0,83,0,110, + 26,116,7,124,5,100,4,131,2,114,172,116,8,124,5,124, + 3,116,0,131,3,83,0,124,5,83,0,100,2,83,0,41, + 5,97,215,1,0,0,73,109,112,111,114,116,32,97,32,109, + 111,100,117,108,101,46,10,10,32,32,32,32,84,104,101,32, + 39,103,108,111,98,97,108,115,39,32,97,114,103,117,109,101, + 110,116,32,105,115,32,117,115,101,100,32,116,111,32,105,110, + 102,101,114,32,119,104,101,114,101,32,116,104,101,32,105,109, + 112,111,114,116,32,105,115,32,111,99,99,117,114,114,105,110, + 103,32,102,114,111,109,10,32,32,32,32,116,111,32,104,97, + 110,100,108,101,32,114,101,108,97,116,105,118,101,32,105,109, + 112,111,114,116,115,46,32,84,104,101,32,39,108,111,99,97, + 108,115,39,32,97,114,103,117,109,101,110,116,32,105,115,32, + 105,103,110,111,114,101,100,46,32,84,104,101,10,32,32,32, + 32,39,102,114,111,109,108,105,115,116,39,32,97,114,103,117, + 109,101,110,116,32,115,112,101,99,105,102,105,101,115,32,119, + 104,97,116,32,115,104,111,117,108,100,32,101,120,105,115,116, + 32,97,115,32,97,116,116,114,105,98,117,116,101,115,32,111, + 110,32,116,104,101,32,109,111,100,117,108,101,10,32,32,32, + 32,98,101,105,110,103,32,105,109,112,111,114,116,101,100,32, + 40,101,46,103,46,32,96,96,102,114,111,109,32,109,111,100, + 117,108,101,32,105,109,112,111,114,116,32,60,102,114,111,109, + 108,105,115,116,62,96,96,41,46,32,32,84,104,101,32,39, + 108,101,118,101,108,39,10,32,32,32,32,97,114,103,117,109, + 101,110,116,32,114,101,112,114,101,115,101,110,116,115,32,116, + 104,101,32,112,97,99,107,97,103,101,32,108,111,99,97,116, + 105,111,110,32,116,111,32,105,109,112,111,114,116,32,102,114, + 111,109,32,105,110,32,97,32,114,101,108,97,116,105,118,101, + 10,32,32,32,32,105,109,112,111,114,116,32,40,101,46,103, + 46,32,96,96,102,114,111,109,32,46,46,112,107,103,32,105, + 109,112,111,114,116,32,109,111,100,96,96,32,119,111,117,108, + 100,32,104,97,118,101,32,97,32,39,108,101,118,101,108,39, + 32,111,102,32,50,41,46,10,10,32,32,32,32,114,22,0, + 0,0,78,114,128,0,0,0,114,141,0,0,0,41,9,114, + 208,0,0,0,114,219,0,0,0,218,9,112,97,114,116,105, + 116,105,111,110,114,184,0,0,0,114,15,0,0,0,114,92, + 0,0,0,114,1,0,0,0,114,4,0,0,0,114,213,0, + 0,0,41,9,114,17,0,0,0,114,218,0,0,0,218,6, + 108,111,99,97,108,115,114,214,0,0,0,114,186,0,0,0, + 114,96,0,0,0,90,8,103,108,111,98,97,108,115,95,114, + 185,0,0,0,90,7,99,117,116,95,111,102,102,114,10,0, + 0,0,114,10,0,0,0,114,11,0,0,0,218,10,95,95, + 105,109,112,111,114,116,95,95,57,4,0,0,115,30,0,0, + 0,0,11,8,1,10,2,16,1,8,1,12,1,4,3,8, + 1,18,1,4,1,4,4,26,3,32,1,10,1,12,2,114, + 222,0,0,0,99,1,0,0,0,0,0,0,0,0,0,0, + 0,2,0,0,0,3,0,0,0,67,0,0,0,115,38,0, + 0,0,116,0,160,1,124,0,161,1,125,1,124,1,100,0, + 107,8,114,30,116,2,100,1,124,0,23,0,131,1,130,1, + 116,3,124,1,131,1,83,0,41,2,78,122,25,110,111,32, + 98,117,105,108,116,45,105,110,32,109,111,100,117,108,101,32, + 110,97,109,101,100,32,41,4,114,160,0,0,0,114,166,0, + 0,0,114,79,0,0,0,114,159,0,0,0,41,2,114,17, + 0,0,0,114,95,0,0,0,114,10,0,0,0,114,10,0, + 0,0,114,11,0,0,0,218,18,95,98,117,105,108,116,105, + 110,95,102,114,111,109,95,110,97,109,101,94,4,0,0,115, + 8,0,0,0,0,1,10,1,8,1,12,1,114,223,0,0, + 0,99,2,0,0,0,0,0,0,0,0,0,0,0,10,0, + 0,0,5,0,0,0,67,0,0,0,115,166,0,0,0,124, + 1,97,0,124,0,97,1,116,2,116,1,131,1,125,2,116, + 1,106,3,160,4,161,0,68,0,93,72,92,2,125,3,125, + 4,116,5,124,4,124,2,131,2,114,26,124,3,116,1,106, + 6,107,6,114,60,116,7,125,5,110,18,116,0,160,8,124, + 3,161,1,114,26,116,9,125,5,110,2,113,26,116,10,124, + 4,124,5,131,2,125,6,116,11,124,6,124,4,131,2,1, + 0,113,26,116,1,106,3,116,12,25,0,125,7,100,1,68, + 0,93,46,125,8,124,8,116,1,106,3,107,7,114,138,116, + 13,124,8,131,1,125,9,110,10,116,1,106,3,124,8,25, + 0,125,9,116,14,124,7,124,8,124,9,131,3,1,0,113, + 114,100,2,83,0,41,3,122,250,83,101,116,117,112,32,105, + 109,112,111,114,116,108,105,98,32,98,121,32,105,109,112,111, + 114,116,105,110,103,32,110,101,101,100,101,100,32,98,117,105, + 108,116,45,105,110,32,109,111,100,117,108,101,115,32,97,110, + 100,32,105,110,106,101,99,116,105,110,103,32,116,104,101,109, + 10,32,32,32,32,105,110,116,111,32,116,104,101,32,103,108, + 111,98,97,108,32,110,97,109,101,115,112,97,99,101,46,10, + 10,32,32,32,32,65,115,32,115,121,115,32,105,115,32,110, + 101,101,100,101,100,32,102,111,114,32,115,121,115,46,109,111, + 100,117,108,101,115,32,97,99,99,101,115,115,32,97,110,100, + 32,95,105,109,112,32,105,115,32,110,101,101,100,101,100,32, + 116,111,32,108,111,97,100,32,98,117,105,108,116,45,105,110, + 10,32,32,32,32,109,111,100,117,108,101,115,44,32,116,104, + 111,115,101,32,116,119,111,32,109,111,100,117,108,101,115,32, + 109,117,115,116,32,98,101,32,101,120,112,108,105,99,105,116, + 108,121,32,112,97,115,115,101,100,32,105,110,46,10,10,32, + 32,32,32,41,3,114,23,0,0,0,114,191,0,0,0,114, + 64,0,0,0,78,41,15,114,57,0,0,0,114,15,0,0, + 0,114,14,0,0,0,114,92,0,0,0,218,5,105,116,101, + 109,115,114,195,0,0,0,114,78,0,0,0,114,160,0,0, + 0,114,88,0,0,0,114,173,0,0,0,114,142,0,0,0, + 114,148,0,0,0,114,1,0,0,0,114,223,0,0,0,114, + 5,0,0,0,41,10,218,10,115,121,115,95,109,111,100,117, + 108,101,218,11,95,105,109,112,95,109,111,100,117,108,101,90, + 11,109,111,100,117,108,101,95,116,121,112,101,114,17,0,0, + 0,114,96,0,0,0,114,109,0,0,0,114,95,0,0,0, + 90,11,115,101,108,102,95,109,111,100,117,108,101,90,12,98, + 117,105,108,116,105,110,95,110,97,109,101,90,14,98,117,105, + 108,116,105,110,95,109,111,100,117,108,101,114,10,0,0,0, + 114,10,0,0,0,114,11,0,0,0,218,6,95,115,101,116, + 117,112,101,4,0,0,115,36,0,0,0,0,9,4,1,4, + 3,8,1,18,1,10,1,10,1,6,1,10,1,6,2,2, + 1,10,1,12,3,10,1,8,1,10,1,10,2,10,1,114, + 227,0,0,0,99,2,0,0,0,0,0,0,0,0,0,0, + 0,2,0,0,0,3,0,0,0,67,0,0,0,115,38,0, + 0,0,116,0,124,0,124,1,131,2,1,0,116,1,106,2, + 160,3,116,4,161,1,1,0,116,1,106,2,160,3,116,5, + 161,1,1,0,100,1,83,0,41,2,122,48,73,110,115,116, + 97,108,108,32,105,109,112,111,114,116,101,114,115,32,102,111, + 114,32,98,117,105,108,116,105,110,32,97,110,100,32,102,114, + 111,122,101,110,32,109,111,100,117,108,101,115,78,41,6,114, + 227,0,0,0,114,15,0,0,0,114,190,0,0,0,114,119, + 0,0,0,114,160,0,0,0,114,173,0,0,0,41,2,114, + 225,0,0,0,114,226,0,0,0,114,10,0,0,0,114,10, + 0,0,0,114,11,0,0,0,218,8,95,105,110,115,116,97, + 108,108,136,4,0,0,115,6,0,0,0,0,2,10,2,12, + 1,114,228,0,0,0,99,0,0,0,0,0,0,0,0,0, + 0,0,0,1,0,0,0,4,0,0,0,67,0,0,0,115, + 32,0,0,0,100,1,100,2,108,0,125,0,124,0,97,1, + 124,0,160,2,116,3,106,4,116,5,25,0,161,1,1,0, + 100,2,83,0,41,3,122,57,73,110,115,116,97,108,108,32, + 105,109,112,111,114,116,101,114,115,32,116,104,97,116,32,114, + 101,113,117,105,114,101,32,101,120,116,101,114,110,97,108,32, + 102,105,108,101,115,121,115,116,101,109,32,97,99,99,101,115, + 115,114,22,0,0,0,78,41,6,218,26,95,102,114,111,122, + 101,110,95,105,109,112,111,114,116,108,105,98,95,101,120,116, + 101,114,110,97,108,114,126,0,0,0,114,228,0,0,0,114, + 15,0,0,0,114,92,0,0,0,114,1,0,0,0,41,1, + 114,229,0,0,0,114,10,0,0,0,114,10,0,0,0,114, + 11,0,0,0,218,27,95,105,110,115,116,97,108,108,95,101, + 120,116,101,114,110,97,108,95,105,109,112,111,114,116,101,114, + 115,144,4,0,0,115,6,0,0,0,0,3,8,1,4,1, + 114,230,0,0,0,41,2,78,78,41,1,78,41,2,78,114, + 22,0,0,0,41,4,78,78,114,10,0,0,0,114,22,0, + 0,0,41,50,114,3,0,0,0,114,126,0,0,0,114,12, + 0,0,0,114,18,0,0,0,114,59,0,0,0,114,33,0, + 0,0,114,42,0,0,0,114,19,0,0,0,114,20,0,0, + 0,114,49,0,0,0,114,50,0,0,0,114,53,0,0,0, + 114,65,0,0,0,114,67,0,0,0,114,76,0,0,0,114, + 86,0,0,0,114,90,0,0,0,114,97,0,0,0,114,111, + 0,0,0,114,112,0,0,0,114,91,0,0,0,114,142,0, + 0,0,114,148,0,0,0,114,152,0,0,0,114,107,0,0, + 0,114,93,0,0,0,114,158,0,0,0,114,159,0,0,0, + 114,94,0,0,0,114,160,0,0,0,114,173,0,0,0,114, + 178,0,0,0,114,187,0,0,0,114,189,0,0,0,114,194, + 0,0,0,114,200,0,0,0,90,15,95,69,82,82,95,77, + 83,71,95,80,82,69,70,73,88,114,202,0,0,0,114,205, + 0,0,0,218,6,111,98,106,101,99,116,114,206,0,0,0, + 114,207,0,0,0,114,208,0,0,0,114,213,0,0,0,114, + 219,0,0,0,114,222,0,0,0,114,223,0,0,0,114,227, + 0,0,0,114,228,0,0,0,114,230,0,0,0,114,10,0, + 0,0,114,10,0,0,0,114,10,0,0,0,114,11,0,0, + 0,218,8,60,109,111,100,117,108,101,62,1,0,0,0,115, + 94,0,0,0,4,24,4,2,8,8,8,8,4,2,4,3, + 16,4,14,68,14,21,14,16,8,37,8,17,8,11,14,8, + 8,11,8,12,8,16,8,36,14,101,16,26,10,45,14,72, + 8,17,8,17,8,30,8,37,8,42,8,15,14,73,14,79, + 14,13,8,9,8,9,10,47,8,16,4,1,8,2,8,27, + 6,3,8,16,10,15,14,37,8,27,10,37,8,7,8,35, + 8,8, }; diff --git a/Tools/pynche/PyncheWidget.py b/Tools/pynche/PyncheWidget.py index 364f22b0b209..ef12198a2183 100644 --- a/Tools/pynche/PyncheWidget.py +++ b/Tools/pynche/PyncheWidget.py @@ -281,10 +281,14 @@ def popup(self, event=None): self.__window.deiconify() def __eq__(self, other): - return self.__menutext == other.__menutext + if isinstance(self, PopupViewer): + return self.__menutext == other.__menutext + return NotImplemented def __lt__(self, other): - return self.__menutext < other.__menutext + if isinstance(self, PopupViewer): + return self.__menutext < other.__menutext + return NotImplemented def make_view_popups(switchboard, root, extrapath): From webhook-mailer at python.org Thu Aug 8 01:43:22 2019 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Thu, 08 Aug 2019 05:43:22 -0000 Subject: [Python-checkins] bpo-37685: Use singletons ALWAYS_EQ and NEVER_EQ in more tests. (GH-15167) Message-ID: https://github.com/python/cpython/commit/7d44e7a4563072d0fad00427b76b94cad61c38ae commit: 7d44e7a4563072d0fad00427b76b94cad61c38ae branch: master author: Serhiy Storchaka committer: GitHub date: 2019-08-08T08:43:18+03:00 summary: bpo-37685: Use singletons ALWAYS_EQ and NEVER_EQ in more tests. (GH-15167) files: M Lib/test/support/__init__.py M Lib/test/test_collections.py M Lib/test/test_compare.py M Lib/test/test_contains.py M Lib/test/test_dict_version.py M Lib/test/test_enum.py M Lib/test/test_inspect.py M Lib/test/test_range.py M Lib/test/test_urllib2.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index d34f2efaed53..00e734e7059f 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -3123,6 +3123,8 @@ def __eq__(self, other): return False def __ne__(self, other): return True + def __hash__(self): + return 1 NEVER_EQ = _NEVER_EQ() diff --git a/Lib/test/test_collections.py b/Lib/test/test_collections.py index 0119c77ac02a..7f01de6f433d 100644 --- a/Lib/test/test_collections.py +++ b/Lib/test/test_collections.py @@ -1472,9 +1472,6 @@ def __lt__(self, x): def test_issue26915(self): # Container membership test should check identity first - class CustomEqualObject: - def __eq__(self, other): - return False class CustomSequence(Sequence): def __init__(self, seq): self._seq = seq @@ -1484,7 +1481,7 @@ def __len__(self): return len(self._seq) nan = float('nan') - obj = CustomEqualObject() + obj = support.NEVER_EQ seq = CustomSequence([nan, obj, nan]) containers = [ seq, diff --git a/Lib/test/test_compare.py b/Lib/test/test_compare.py index 471c8dae767a..2b3faed7965b 100644 --- a/Lib/test/test_compare.py +++ b/Lib/test/test_compare.py @@ -1,4 +1,5 @@ import unittest +from test.support import ALWAYS_EQ class Empty: def __repr__(self): @@ -14,13 +15,6 @@ def __repr__(self): def __eq__(self, other): return self.arg == other -class Anything: - def __eq__(self, other): - return True - - def __ne__(self, other): - return False - class ComparisonTest(unittest.TestCase): set1 = [2, 2.0, 2, 2+0j, Cmp(2.0)] set2 = [[1], (3,), None, Empty()] @@ -113,11 +107,11 @@ class C: def test_issue_1393(self): x = lambda: None - self.assertEqual(x, Anything()) - self.assertEqual(Anything(), x) + self.assertEqual(x, ALWAYS_EQ) + self.assertEqual(ALWAYS_EQ, x) y = object() - self.assertEqual(y, Anything()) - self.assertEqual(Anything(), y) + self.assertEqual(y, ALWAYS_EQ) + self.assertEqual(ALWAYS_EQ, y) if __name__ == '__main__': diff --git a/Lib/test/test_contains.py b/Lib/test/test_contains.py index 036a1d012dbd..471d04a76ca4 100644 --- a/Lib/test/test_contains.py +++ b/Lib/test/test_contains.py @@ -1,5 +1,6 @@ from collections import deque import unittest +from test.support import NEVER_EQ class base_set: @@ -69,13 +70,7 @@ def test_nonreflexive(self): # containment and equality tests involving elements that are # not necessarily equal to themselves - class MyNonReflexive(object): - def __eq__(self, other): - return False - def __hash__(self): - return 28 - - values = float('nan'), 1, None, 'abc', MyNonReflexive() + values = float('nan'), 1, None, 'abc', NEVER_EQ constructors = list, tuple, dict.fromkeys, set, frozenset, deque for constructor in constructors: container = constructor(values) diff --git a/Lib/test/test_dict_version.py b/Lib/test/test_dict_version.py index cb79e7434c29..b23786514f82 100644 --- a/Lib/test/test_dict_version.py +++ b/Lib/test/test_dict_version.py @@ -98,20 +98,25 @@ def __eq__(self, other): value2 = AlwaysEqual() self.assertTrue(value1 == value2) self.assertFalse(value1 != value2) + self.assertIsNot(value1, value2) d = self.new_dict() self.check_version_changed(d, d.__setitem__, 'key', value1) + self.assertIs(d['key'], value1) # setting a key to a value equal to the current value # with dict.__setitem__() must change the version self.check_version_changed(d, d.__setitem__, 'key', value2) + self.assertIs(d['key'], value2) # setting a key to a value equal to the current value # with dict.update() must change the version self.check_version_changed(d, d.update, key=value1) + self.assertIs(d['key'], value1) d2 = self.new_dict(key=value2) self.check_version_changed(d, d.update, d2) + self.assertIs(d['key'], value2) def test_setdefault(self): d = self.new_dict() diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 7bccd6660b52..ec1cfeab12d7 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -9,6 +9,7 @@ from io import StringIO from pickle import dumps, loads, PicklingError, HIGHEST_PROTOCOL from test import support +from test.support import ALWAYS_EQ from datetime import timedelta @@ -1509,13 +1510,10 @@ class Color(AutoNumber): self.assertEqual(list(map(int, Color)), [1, 2, 3]) def test_equality(self): - class AlwaysEqual: - def __eq__(self, other): - return True class OrdinaryEnum(Enum): a = 1 - self.assertEqual(AlwaysEqual(), OrdinaryEnum.a) - self.assertEqual(OrdinaryEnum.a, AlwaysEqual()) + self.assertEqual(ALWAYS_EQ, OrdinaryEnum.a) + self.assertEqual(OrdinaryEnum.a, ALWAYS_EQ) def test_ordered_mixin(self): class OrderedEnum(Enum): diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py index 1cd4ea28939d..6c9f52977807 100644 --- a/Lib/test/test_inspect.py +++ b/Lib/test/test_inspect.py @@ -25,7 +25,7 @@ ThreadPoolExecutor = None from test.support import run_unittest, TESTFN, DirsOnSysPath, cpython_only -from test.support import MISSING_C_DOCSTRINGS, cpython_only +from test.support import MISSING_C_DOCSTRINGS, ALWAYS_EQ from test.support.script_helper import assert_python_ok, assert_python_failure from test import inspect_fodder as mod from test import inspect_fodder2 as mod2 @@ -118,10 +118,6 @@ def gen_coroutine_function_example(self): yield return 'spam' -class EqualsToAll: - def __eq__(self, other): - return True - class TestPredicates(IsTestBase): def test_excluding_predicates(self): @@ -2978,8 +2974,8 @@ def test_signature_equality(self): def foo(a, *, b:int) -> float: pass self.assertFalse(inspect.signature(foo) == 42) self.assertTrue(inspect.signature(foo) != 42) - self.assertTrue(inspect.signature(foo) == EqualsToAll()) - self.assertFalse(inspect.signature(foo) != EqualsToAll()) + self.assertTrue(inspect.signature(foo) == ALWAYS_EQ) + self.assertFalse(inspect.signature(foo) != ALWAYS_EQ) def bar(a, *, b:int) -> float: pass self.assertTrue(inspect.signature(foo) == inspect.signature(bar)) @@ -3246,8 +3242,8 @@ def test_signature_parameter_equality(self): self.assertFalse(p != p) self.assertFalse(p == 42) self.assertTrue(p != 42) - self.assertTrue(p == EqualsToAll()) - self.assertFalse(p != EqualsToAll()) + self.assertTrue(p == ALWAYS_EQ) + self.assertFalse(p != ALWAYS_EQ) self.assertTrue(p == P('foo', default=42, kind=inspect.Parameter.KEYWORD_ONLY)) @@ -3584,8 +3580,8 @@ def foo(a): pass ba = inspect.signature(foo).bind(1) self.assertTrue(ba == ba) self.assertFalse(ba != ba) - self.assertTrue(ba == EqualsToAll()) - self.assertFalse(ba != EqualsToAll()) + self.assertTrue(ba == ALWAYS_EQ) + self.assertFalse(ba != ALWAYS_EQ) ba2 = inspect.signature(foo).bind(1) self.assertTrue(ba == ba2) diff --git a/Lib/test/test_range.py b/Lib/test/test_range.py index 94c96a941b12..73cbcc4717d7 100644 --- a/Lib/test/test_range.py +++ b/Lib/test/test_range.py @@ -4,6 +4,7 @@ import sys import pickle import itertools +from test.support import ALWAYS_EQ # pure Python implementations (3 args only), for comparison def pyrange(start, stop, step): @@ -289,11 +290,7 @@ def __eq__(self, other): self.assertRaises(ValueError, range(1, 2**100, 2).index, 2**87) self.assertEqual(range(1, 2**100, 2).index(2**87+1), 2**86) - class AlwaysEqual(object): - def __eq__(self, other): - return True - always_equal = AlwaysEqual() - self.assertEqual(range(10).index(always_equal), 0) + self.assertEqual(range(10).index(ALWAYS_EQ), 0) def test_user_index_method(self): bignum = 2*sys.maxsize @@ -344,11 +341,7 @@ def test_count(self): self.assertEqual(range(1, 2**100, 2).count(2**87), 0) self.assertEqual(range(1, 2**100, 2).count(2**87+1), 1) - class AlwaysEqual(object): - def __eq__(self, other): - return True - always_equal = AlwaysEqual() - self.assertEqual(range(10).count(always_equal), 10) + self.assertEqual(range(10).count(ALWAYS_EQ), 10) self.assertEqual(len(range(sys.maxsize, sys.maxsize+10)), 10) @@ -429,9 +422,7 @@ def test_types(self): self.assertIn(True, range(3)) self.assertIn(1+0j, range(3)) - class C1: - def __eq__(self, other): return True - self.assertIn(C1(), range(3)) + self.assertIn(ALWAYS_EQ, range(3)) # Objects are never coerced into other types for comparison. class C2: diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index debb3c2400a8..7b576db4e3aa 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -634,17 +634,12 @@ def test_http_error(self): [("http_error_302")], ] handlers = add_ordered_mock_handlers(o, meth_spec) - - class Unknown: - def __eq__(self, other): - return True - req = Request("http://example.com/") o.open(req) assert len(o.calls) == 2 calls = [(handlers[0], "http_open", (req,)), (handlers[2], "http_error_302", - (req, Unknown(), 302, "", {}))] + (req, support.ALWAYS_EQ, 302, "", {}))] for expected, got in zip(calls, o.calls): handler, method_name, args = expected self.assertEqual((handler, method_name), got[:2]) From webhook-mailer at python.org Thu Aug 8 04:23:12 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Thu, 08 Aug 2019 08:23:12 -0000 Subject: [Python-checkins] bpo-35892: Add usage note to mode() (GH-15122) Message-ID: https://github.com/python/cpython/commit/e43e7ed36480190083740fd75e2b9cdca72f1a68 commit: e43e7ed36480190083740fd75e2b9cdca72f1a68 branch: master author: Raymond Hettinger committer: GitHub date: 2019-08-08T01:23:05-07:00 summary: bpo-35892: Add usage note to mode() (GH-15122) files: M Doc/library/statistics.rst diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst index 3a2a1f94db47..bb77228ceac1 100644 --- a/Doc/library/statistics.rst +++ b/Doc/library/statistics.rst @@ -313,7 +313,9 @@ However, for reading convenience, most of the examples show sorted sequences. measure of central location. If there are multiple modes, returns the first one encountered in the *data*. - If *data* is empty, :exc:`StatisticsError` is raised. + If the smallest or largest of multiple modes is desired instead, use + ``min(multimode(data))`` or ``max(multimode(data))``. If the input *data* is + empty, :exc:`StatisticsError` is raised. ``mode`` assumes discrete data, and returns a single value. This is the standard treatment of the mode as commonly taught in schools: From webhook-mailer at python.org Thu Aug 8 04:37:01 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Thu, 08 Aug 2019 08:37:01 -0000 Subject: [Python-checkins] bpo-35892: Add usage note to mode() (GH-15122) (GH-15176) Message-ID: https://github.com/python/cpython/commit/5925b7d555bc36bd43ee8704ae75cc51900cf2d4 commit: 5925b7d555bc36bd43ee8704ae75cc51900cf2d4 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Raymond Hettinger date: 2019-08-08T01:36:55-07:00 summary: bpo-35892: Add usage note to mode() (GH-15122) (GH-15176) (cherry picked from commit e43e7ed36480190083740fd75e2b9cdca72f1a68) Co-authored-by: Raymond Hettinger files: M Doc/library/statistics.rst diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst index a906a591e62c..1a19e3741921 100644 --- a/Doc/library/statistics.rst +++ b/Doc/library/statistics.rst @@ -313,7 +313,9 @@ However, for reading convenience, most of the examples show sorted sequences. measure of central location. If there are multiple modes, returns the first one encountered in the *data*. - If *data* is empty, :exc:`StatisticsError` is raised. + If the smallest or largest of multiple modes is desired instead, use + ``min(multimode(data))`` or ``max(multimode(data))``. If the input *data* is + empty, :exc:`StatisticsError` is raised. ``mode`` assumes discrete data, and returns a single value. This is the standard treatment of the mode as commonly taught in schools: From webhook-mailer at python.org Thu Aug 8 04:57:14 2019 From: webhook-mailer at python.org (Inada Naoki) Date: Thu, 08 Aug 2019 08:57:14 -0000 Subject: [Python-checkins] bpo-37587: optimize json.loads (GH-15134) Message-ID: https://github.com/python/cpython/commit/2a570af12ac5e4ac5575a68f8739b31c24d01367 commit: 2a570af12ac5e4ac5575a68f8739b31c24d01367 branch: master author: Inada Naoki committer: GitHub date: 2019-08-08T17:57:10+09:00 summary: bpo-37587: optimize json.loads (GH-15134) Use a tighter scope temporary variable to help register allocation. 1% speedup for large string. Use PyDict_SetItemDefault() for memoizing keys. At most 4% speedup when the cache hit ratio is low. files: M Modules/_json.c diff --git a/Modules/_json.c b/Modules/_json.c index 76da1d345e9d..112903ea577a 100644 --- a/Modules/_json.c +++ b/Modules/_json.c @@ -433,16 +433,21 @@ scanstring_unicode(PyObject *pystr, Py_ssize_t end, int strict, Py_ssize_t *next } while (1) { /* Find the end of the string or the next escape */ - Py_UCS4 c = 0; - for (next = end; next < len; next++) { - c = PyUnicode_READ(kind, buf, next); - if (c == '"' || c == '\\') { - break; - } - else if (c <= 0x1f && strict) { - raise_errmsg("Invalid control character at", pystr, next); - goto bail; + Py_UCS4 c; + { + // Use tight scope variable to help register allocation. + Py_UCS4 d = 0; + for (next = end; next < len; next++) { + d = PyUnicode_READ(kind, buf, next); + if (d == '"' || d == '\\') { + break; + } + if (d <= 0x1f && strict) { + raise_errmsg("Invalid control character at", pystr, next); + goto bail; + } } + c = d; } if (!(c == '"' || c == '\\')) { raise_errmsg("Unterminated string starting at", pystr, begin); @@ -749,19 +754,13 @@ _parse_object_unicode(PyScannerObject *s, PyObject *pystr, Py_ssize_t idx, Py_ss key = scanstring_unicode(pystr, idx + 1, s->strict, &next_idx); if (key == NULL) goto bail; - memokey = PyDict_GetItemWithError(s->memo, key); - if (memokey != NULL) { - Py_INCREF(memokey); - Py_DECREF(key); - key = memokey; - } - else if (PyErr_Occurred()) { + memokey = PyDict_SetDefault(s->memo, key, key); + if (memokey == NULL) { goto bail; } - else { - if (PyDict_SetItem(s->memo, key, key) < 0) - goto bail; - } + Py_INCREF(memokey); + Py_DECREF(key); + key = memokey; idx = next_idx; /* skip whitespace between key and : delimiter, read :, skip whitespace */ From webhook-mailer at python.org Thu Aug 8 16:02:58 2019 From: webhook-mailer at python.org (Paul Moore) Date: Thu, 08 Aug 2019 20:02:58 -0000 Subject: [Python-checkins] bpo-25172: Raise appropriate ImportError msg when crypt module used on Windows (GH-15149) Message-ID: https://github.com/python/cpython/commit/f4e725f224b864bf9bf405ff7f863cda46fca1cd commit: f4e725f224b864bf9bf405ff7f863cda46fca1cd branch: master author: shireenrao committer: Paul Moore date: 2019-08-08T21:02:49+01:00 summary: bpo-25172: Raise appropriate ImportError msg when crypt module used on Windows (GH-15149) files: A Misc/NEWS.d/next/Windows/2019-08-06-18-09-18.bpo-25172.Akreij.rst M Lib/crypt.py diff --git a/Lib/crypt.py b/Lib/crypt.py index b0e47f430c3c..8846602d7613 100644 --- a/Lib/crypt.py +++ b/Lib/crypt.py @@ -1,6 +1,15 @@ """Wrapper to the POSIX crypt library call and associated functionality.""" -import _crypt +import sys as _sys + +try: + import _crypt +except ModuleNotFoundError: + if _sys.platform == 'win32': + raise ImportError("The crypt module is not supported on Windows") + else: + raise ImportError("The required _crypt module was not built as part of CPython") + import string as _string from random import SystemRandom as _SystemRandom from collections import namedtuple as _namedtuple diff --git a/Misc/NEWS.d/next/Windows/2019-08-06-18-09-18.bpo-25172.Akreij.rst b/Misc/NEWS.d/next/Windows/2019-08-06-18-09-18.bpo-25172.Akreij.rst new file mode 100644 index 000000000000..47106d887921 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-08-06-18-09-18.bpo-25172.Akreij.rst @@ -0,0 +1 @@ +Trying to import the :mod:`crypt` module on Windows will result in an :exc:`ImportError` with a message explaining that the module isn't supported on Windows. On other platforms, if the underlying ``_crypt`` module is not available, the ImportError will include a message explaining the problem. From webhook-mailer at python.org Thu Aug 8 17:48:05 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 08 Aug 2019 21:48:05 -0000 Subject: [Python-checkins] bpo-37268: test_parser fails when run with -Werror (GH-15183) Message-ID: https://github.com/python/cpython/commit/10a0a093231ea82a3bfd33fd63322aebd8406866 commit: 10a0a093231ea82a3bfd33fd63322aebd8406866 branch: master author: Zackery Spytz committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-08T14:48:00-07:00 summary: bpo-37268: test_parser fails when run with -Werror (GH-15183) Use warnings.filterwarnings() when importing the deprecated parser module. @pablogsal https://bugs.python.org/issue37268 Automerge-Triggered-By: @pablogsal files: M Lib/test/test_parser.py diff --git a/Lib/test/test_parser.py b/Lib/test/test_parser.py index ec1845d7fe9a..7295f66d393a 100644 --- a/Lib/test/test_parser.py +++ b/Lib/test/test_parser.py @@ -1,5 +1,9 @@ import copy -import parser +import warnings +with warnings.catch_warnings(): + warnings.filterwarnings('ignore', 'The parser module is deprecated', + DeprecationWarning) + import parser import pickle import unittest import operator From webhook-mailer at python.org Thu Aug 8 18:25:49 2019 From: webhook-mailer at python.org (Pablo Galindo) Date: Thu, 08 Aug 2019 22:25:49 -0000 Subject: [Python-checkins] bpo-37795: Capture DeprecationWarnings in the test suite (GH-15184) Message-ID: https://github.com/python/cpython/commit/aa542c2cf26c5af9298dda6064576b18906cdfbf commit: aa542c2cf26c5af9298dda6064576b18906cdfbf branch: master author: Pablo Galindo committer: GitHub date: 2019-08-08T23:25:46+01:00 summary: bpo-37795: Capture DeprecationWarnings in the test suite (GH-15184) files: M Lib/distutils/tests/test_bdist.py M Lib/test/test_httplib.py diff --git a/Lib/distutils/tests/test_bdist.py b/Lib/distutils/tests/test_bdist.py index c80b3edc0220..130d8bf155a4 100644 --- a/Lib/distutils/tests/test_bdist.py +++ b/Lib/distutils/tests/test_bdist.py @@ -2,6 +2,7 @@ import os import unittest from test.support import run_unittest +import warnings from distutils.command.bdist import bdist from distutils.tests import support @@ -38,7 +39,10 @@ def test_skip_build(self): names.append('bdist_msi') for name in names: - subcmd = cmd.get_finalized_command(name) + with warnings.catch_warnings(): + warnings.filterwarnings('ignore', 'bdist_wininst command is deprecated', + DeprecationWarning) + subcmd = cmd.get_finalized_command(name) if getattr(subcmd, '_unsupported', False): # command is not supported on this build continue diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py index 9148169cc7c2..656932fbaab7 100644 --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -7,6 +7,7 @@ import re import socket import threading +import warnings import unittest TestCase = unittest.TestCase @@ -1759,8 +1760,11 @@ def test_tls13_pha(self): self.assertIs(h._context, context) self.assertFalse(h._context.post_handshake_auth) - h = client.HTTPSConnection('localhost', 443, context=context, - cert_file=CERT_localhost) + with warnings.catch_warnings(): + warnings.filterwarnings('ignore', 'key_file, cert_file and check_hostname are deprecated', + DeprecationWarning) + h = client.HTTPSConnection('localhost', 443, context=context, + cert_file=CERT_localhost) self.assertTrue(h._context.post_handshake_auth) From webhook-mailer at python.org Thu Aug 8 19:12:37 2019 From: webhook-mailer at python.org (Steve Dower) Date: Thu, 08 Aug 2019 23:12:37 -0000 Subject: [Python-checkins] bpo-36511: Fix failures in Windows ARM32 buildbot (GH-15181) Message-ID: https://github.com/python/cpython/commit/ed70a344b5fbddea85726ebc1964ee0cfdef9c40 commit: ed70a344b5fbddea85726ebc1964ee0cfdef9c40 branch: master author: Paul Monson committer: Steve Dower date: 2019-08-08T16:12:33-07:00 summary: bpo-36511: Fix failures in Windows ARM32 buildbot (GH-15181) files: M Lib/test/libregrtest/win_utils.py M Tools/buildbot/remoteDeploy.bat diff --git a/Lib/test/libregrtest/win_utils.py b/Lib/test/libregrtest/win_utils.py index 0e6bfa8e10f7..ec2d6c663e83 100644 --- a/Lib/test/libregrtest/win_utils.py +++ b/Lib/test/libregrtest/win_utils.py @@ -25,6 +25,7 @@ class WindowsLoadTracker(): def __init__(self): self.load = 0.0 + self.p = None self.start() def start(self): diff --git a/Tools/buildbot/remoteDeploy.bat b/Tools/buildbot/remoteDeploy.bat index 8b0ce239e1a4..6b86e1e59b07 100644 --- a/Tools/buildbot/remoteDeploy.bat +++ b/Tools/buildbot/remoteDeploy.bat @@ -36,6 +36,7 @@ for /f "USEBACKQ" %%i in (`dir PCbuild\arm32\*.pyd /b`) do @scp PCBuild\arm32\%% for /f "USEBACKQ" %%i in (`dir PCbuild\arm32\*.dll /b`) do @scp PCBuild\arm32\%%i "%SSH_SERVER%:%REMOTE_PYTHON_DIR%PCBuild\arm32" scp -r "%PYTHON_SOURCE%Include" "%SSH_SERVER%:%REMOTE_PYTHON_DIR%Include" scp -r "%PYTHON_SOURCE%Lib" "%SSH_SERVER%:%REMOTE_PYTHON_DIR%Lib" +scp -r "%PYTHON_SOURCE%Parser" "%SSH_SERVER%:%REMOTE_PYTHON_DIR%Parser" scp -r "%PYTHON_SOURCE%Tools" "%SSH_SERVER%:%REMOTE_PYTHON_DIR%Tools" scp "%PYTHON_SOURCE%Modules\Setup" "%SSH_SERVER%:%REMOTE_PYTHON_DIR%Modules" scp "%PYTHON_SOURCE%PC\pyconfig.h" "%SSH_SERVER%:%REMOTE_PYTHON_DIR%PC" From webhook-mailer at python.org Thu Aug 8 20:23:02 2019 From: webhook-mailer at python.org (Pablo Galindo) Date: Fri, 09 Aug 2019 00:23:02 -0000 Subject: [Python-checkins] [3.8] bpo-37795: Capture DeprecationWarnings in the test suite (GH-15184) (GH-15188) Message-ID: https://github.com/python/cpython/commit/162d45c531552d0699f945d2c22a763941dca3c1 commit: 162d45c531552d0699f945d2c22a763941dca3c1 branch: 3.8 author: Pablo Galindo committer: GitHub date: 2019-08-09T01:22:59+01:00 summary: [3.8] bpo-37795: Capture DeprecationWarnings in the test suite (GH-15184) (GH-15188) (cherry picked from commit aa542c2) Co-authored-by: Pablo Galindo files: M Lib/distutils/tests/test_bdist.py M Lib/test/test_httplib.py diff --git a/Lib/distutils/tests/test_bdist.py b/Lib/distutils/tests/test_bdist.py index c80b3edc0220..130d8bf155a4 100644 --- a/Lib/distutils/tests/test_bdist.py +++ b/Lib/distutils/tests/test_bdist.py @@ -2,6 +2,7 @@ import os import unittest from test.support import run_unittest +import warnings from distutils.command.bdist import bdist from distutils.tests import support @@ -38,7 +39,10 @@ def test_skip_build(self): names.append('bdist_msi') for name in names: - subcmd = cmd.get_finalized_command(name) + with warnings.catch_warnings(): + warnings.filterwarnings('ignore', 'bdist_wininst command is deprecated', + DeprecationWarning) + subcmd = cmd.get_finalized_command(name) if getattr(subcmd, '_unsupported', False): # command is not supported on this build continue diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py index 9148169cc7c2..656932fbaab7 100644 --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -7,6 +7,7 @@ import re import socket import threading +import warnings import unittest TestCase = unittest.TestCase @@ -1759,8 +1760,11 @@ def test_tls13_pha(self): self.assertIs(h._context, context) self.assertFalse(h._context.post_handshake_auth) - h = client.HTTPSConnection('localhost', 443, context=context, - cert_file=CERT_localhost) + with warnings.catch_warnings(): + warnings.filterwarnings('ignore', 'key_file, cert_file and check_hostname are deprecated', + DeprecationWarning) + h = client.HTTPSConnection('localhost', 443, context=context, + cert_file=CERT_localhost) self.assertTrue(h._context.post_handshake_auth) From webhook-mailer at python.org Fri Aug 9 04:30:51 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 09 Aug 2019 08:30:51 -0000 Subject: [Python-checkins] bpo-34155: Dont parse domains containing @ (GH-13079) Message-ID: https://github.com/python/cpython/commit/c48d606adcef395e59fd555496c42203b01dd3e8 commit: c48d606adcef395e59fd555496c42203b01dd3e8 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-09T01:30:33-07:00 summary: bpo-34155: Dont parse domains containing @ (GH-13079) Before: >>> email.message_from_string('From: a at malicious.org@important.com', policy=email.policy.default)['from'].addresses (Address(display_name='', username='a', domain='malicious.org'),) >>> parseaddr('a at malicious.org@important.com') ('', 'a at malicious.org') After: >>> email.message_from_string('From: a at malicious.org@important.com', policy=email.policy.default)['from'].addresses (Address(display_name='', username='', domain=''),) >>> parseaddr('a at malicious.org@important.com') ('', 'a@') https://bugs.python.org/issue34155 (cherry picked from commit 8cb65d1381b027f0b09ee36bfed7f35bb4dec9a9) Co-authored-by: jpic files: A Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst M Lib/email/_header_value_parser.py M Lib/email/_parseaddr.py M Lib/test/test_email/test__header_value_parser.py M Lib/test/test_email/test_email.py diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py index 801ae728dd13..c09f4f121ffb 100644 --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -1585,6 +1585,8 @@ def get_domain(value): token, value = get_dot_atom(value) except errors.HeaderParseError: token, value = get_atom(value) + if value and value[0] == '@': + raise errors.HeaderParseError('Invalid Domain') if leader is not None: token[:0] = [leader] domain.append(token) diff --git a/Lib/email/_parseaddr.py b/Lib/email/_parseaddr.py index cdfa3729adc7..41ff6f8c000d 100644 --- a/Lib/email/_parseaddr.py +++ b/Lib/email/_parseaddr.py @@ -379,7 +379,12 @@ def getaddrspec(self): aslist.append('@') self.pos += 1 self.gotonext() - return EMPTYSTRING.join(aslist) + self.getdomain() + domain = self.getdomain() + if not domain: + # Invalid domain, return an empty address instead of returning a + # local part to denote failed parsing. + return EMPTYSTRING + return EMPTYSTRING.join(aslist) + domain def getdomain(self): """Get the complete domain name from an address.""" @@ -394,6 +399,10 @@ def getdomain(self): elif self.field[self.pos] == '.': self.pos += 1 sdlist.append('.') + elif self.field[self.pos] == '@': + # bpo-34155: Don't parse domains with two `@` like + # `a at malicious.org@important.com`. + return EMPTYSTRING elif self.field[self.pos] in self.atomends: break else: diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py index 9e862feab10c..0f19f8bcc2e0 100644 --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -1448,6 +1448,16 @@ def test_get_addr_spec_dot_atom(self): self.assertEqual(addr_spec.domain, 'example.com') self.assertEqual(addr_spec.addr_spec, 'star.a.star at example.com') + def test_get_addr_spec_multiple_domains(self): + with self.assertRaises(errors.HeaderParseError): + parser.get_addr_spec('star at a.star@example.com') + + with self.assertRaises(errors.HeaderParseError): + parser.get_addr_spec('star at a@example.com') + + with self.assertRaises(errors.HeaderParseError): + parser.get_addr_spec('star at 172.17.0.1@example.com') + # get_obs_route def test_get_obs_route_simple(self): diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index c29cc56203b1..aa775881c552 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3041,6 +3041,20 @@ def test_parseaddr_empty(self): self.assertEqual(utils.parseaddr('<>'), ('', '')) self.assertEqual(utils.formataddr(utils.parseaddr('<>')), '') + def test_parseaddr_multiple_domains(self): + self.assertEqual( + utils.parseaddr('a at b@c'), + ('', '') + ) + self.assertEqual( + utils.parseaddr('a at b.c@c'), + ('', '') + ) + self.assertEqual( + utils.parseaddr('a at 172.17.0.1@c'), + ('', '') + ) + def test_noquote_dump(self): self.assertEqual( utils.formataddr(('A Silly Person', 'person at dom.ain')), diff --git a/Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst b/Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst new file mode 100644 index 000000000000..50292e29ed1d --- /dev/null +++ b/Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst @@ -0,0 +1 @@ +Fix parsing of invalid email addresses with more than one ``@`` (e.g. a at b@c.com.) to not return the part before 2nd ``@`` as valid email address. Patch by maxking & jpic. From webhook-mailer at python.org Fri Aug 9 04:31:33 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 09 Aug 2019 08:31:33 -0000 Subject: [Python-checkins] bpo-34155: Dont parse domains containing @ (GH-13079) Message-ID: https://github.com/python/cpython/commit/217077440a6938a0b428f67cfef6e053c4f8673c commit: 217077440a6938a0b428f67cfef6e053c4f8673c branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-09T01:31:27-07:00 summary: bpo-34155: Dont parse domains containing @ (GH-13079) Before: >>> email.message_from_string('From: a at malicious.org@important.com', policy=email.policy.default)['from'].addresses (Address(display_name='', username='a', domain='malicious.org'),) >>> parseaddr('a at malicious.org@important.com') ('', 'a at malicious.org') After: >>> email.message_from_string('From: a at malicious.org@important.com', policy=email.policy.default)['from'].addresses (Address(display_name='', username='', domain=''),) >>> parseaddr('a at malicious.org@important.com') ('', 'a@') https://bugs.python.org/issue34155 (cherry picked from commit 8cb65d1381b027f0b09ee36bfed7f35bb4dec9a9) Co-authored-by: jpic files: A Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst M Lib/email/_header_value_parser.py M Lib/email/_parseaddr.py M Lib/test/test_email/test__header_value_parser.py M Lib/test/test_email/test_email.py diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py index 7dfd9780a6e9..a93079278893 100644 --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -1566,6 +1566,8 @@ def get_domain(value): token, value = get_dot_atom(value) except errors.HeaderParseError: token, value = get_atom(value) + if value and value[0] == '@': + raise errors.HeaderParseError('Invalid Domain') if leader is not None: token[:0] = [leader] domain.append(token) diff --git a/Lib/email/_parseaddr.py b/Lib/email/_parseaddr.py index cdfa3729adc7..41ff6f8c000d 100644 --- a/Lib/email/_parseaddr.py +++ b/Lib/email/_parseaddr.py @@ -379,7 +379,12 @@ def getaddrspec(self): aslist.append('@') self.pos += 1 self.gotonext() - return EMPTYSTRING.join(aslist) + self.getdomain() + domain = self.getdomain() + if not domain: + # Invalid domain, return an empty address instead of returning a + # local part to denote failed parsing. + return EMPTYSTRING + return EMPTYSTRING.join(aslist) + domain def getdomain(self): """Get the complete domain name from an address.""" @@ -394,6 +399,10 @@ def getdomain(self): elif self.field[self.pos] == '.': self.pos += 1 sdlist.append('.') + elif self.field[self.pos] == '@': + # bpo-34155: Don't parse domains with two `@` like + # `a at malicious.org@important.com`. + return EMPTYSTRING elif self.field[self.pos] in self.atomends: break else: diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py index f4aad851c677..010be9fa4ae6 100644 --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -1428,6 +1428,16 @@ def test_get_addr_spec_dot_atom(self): self.assertEqual(addr_spec.domain, 'example.com') self.assertEqual(addr_spec.addr_spec, 'star.a.star at example.com') + def test_get_addr_spec_multiple_domains(self): + with self.assertRaises(errors.HeaderParseError): + parser.get_addr_spec('star at a.star@example.com') + + with self.assertRaises(errors.HeaderParseError): + parser.get_addr_spec('star at a@example.com') + + with self.assertRaises(errors.HeaderParseError): + parser.get_addr_spec('star at 172.17.0.1@example.com') + # get_obs_route def test_get_obs_route_simple(self): diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index c29cc56203b1..aa775881c552 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3041,6 +3041,20 @@ def test_parseaddr_empty(self): self.assertEqual(utils.parseaddr('<>'), ('', '')) self.assertEqual(utils.formataddr(utils.parseaddr('<>')), '') + def test_parseaddr_multiple_domains(self): + self.assertEqual( + utils.parseaddr('a at b@c'), + ('', '') + ) + self.assertEqual( + utils.parseaddr('a at b.c@c'), + ('', '') + ) + self.assertEqual( + utils.parseaddr('a at 172.17.0.1@c'), + ('', '') + ) + def test_noquote_dump(self): self.assertEqual( utils.formataddr(('A Silly Person', 'person at dom.ain')), diff --git a/Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst b/Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst new file mode 100644 index 000000000000..50292e29ed1d --- /dev/null +++ b/Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst @@ -0,0 +1 @@ +Fix parsing of invalid email addresses with more than one ``@`` (e.g. a at b@c.com.) to not return the part before 2nd ``@`` as valid email address. Patch by maxking & jpic. From webhook-mailer at python.org Fri Aug 9 10:22:22 2019 From: webhook-mailer at python.org (Paul Ganssle) Date: Fri, 09 Aug 2019 14:22:22 -0000 Subject: [Python-checkins] bpo-37642: Update acceptable offsets in timezone (GH-14878) Message-ID: https://github.com/python/cpython/commit/92c7e30adf5c81a54d6e5e555a6bdfaa60157a0d commit: 92c7e30adf5c81a54d6e5e555a6bdfaa60157a0d branch: master author: Ngalim Siregar committer: Paul Ganssle date: 2019-08-09T10:22:16-04:00 summary: bpo-37642: Update acceptable offsets in timezone (GH-14878) This fixes an inconsistency between the Python and C implementations of the datetime module. The pure python version of the code was not accepting offsets greater than 23:59 but less than 24:00. This is an accidental legacy of the original implementation, which was put in place before tzinfo allowed sub-minute time zone offsets. GH-14878 files: A Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst M Lib/datetime.py M Lib/test/datetimetester.py M Misc/ACKS M Modules/_datetimemodule.c diff --git a/Lib/datetime.py b/Lib/datetime.py index d4c7a1ff9004..0adf1dd67dfb 100644 --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -2269,7 +2269,7 @@ def fromutc(self, dt): raise TypeError("fromutc() argument must be a datetime instance" " or None") - _maxoffset = timedelta(hours=23, minutes=59) + _maxoffset = timedelta(hours=24, microseconds=-1) _minoffset = -_maxoffset @staticmethod @@ -2293,8 +2293,11 @@ def _name_from_offset(delta): return f'UTC{sign}{hours:02d}:{minutes:02d}' timezone.utc = timezone._create(timedelta(0)) -timezone.min = timezone._create(timezone._minoffset) -timezone.max = timezone._create(timezone._maxoffset) +# bpo-37642: These attributes are rounded to the nearest minute for backwards +# compatibility, even though the constructor will accept a wider range of +# values. This may change in the future. +timezone.min = timezone._create(-timedelta(hours=23, minutes=59)) +timezone.max = timezone._create(timedelta(hours=23, minutes=59)) _EPOCH = datetime(1970, 1, 1, tzinfo=timezone.utc) # Some time zone algebra. For a datetime x, let diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 99b620ce2f41..d0101c98bc76 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -388,6 +388,31 @@ def test_deepcopy(self): tz_copy = copy.deepcopy(tz) self.assertIs(tz_copy, tz) + def test_offset_boundaries(self): + # Test timedeltas close to the boundaries + time_deltas = [ + timedelta(hours=23, minutes=59), + timedelta(hours=23, minutes=59, seconds=59), + timedelta(hours=23, minutes=59, seconds=59, microseconds=999999), + ] + time_deltas.extend([-delta for delta in time_deltas]) + + for delta in time_deltas: + with self.subTest(test_type='good', delta=delta): + timezone(delta) + + # Test timedeltas on and outside the boundaries + bad_time_deltas = [ + timedelta(hours=24), + timedelta(hours=24, microseconds=1), + ] + bad_time_deltas.extend([-delta for delta in bad_time_deltas]) + + for delta in bad_time_deltas: + with self.subTest(test_type='bad', delta=delta): + with self.assertRaises(ValueError): + timezone(delta) + ############################################################################# # Base class for testing a particular aspect of timedelta, time, date and diff --git a/Misc/ACKS b/Misc/ACKS index 8c834b43120b..3b4cf85c75fe 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1881,3 +1881,4 @@ Aleksandr Balezin Robert Leenders Tim Hopper Dan Lidral-Porter +Ngalim Siregar diff --git a/Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst b/Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst new file mode 100644 index 000000000000..09ff257597e8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst @@ -0,0 +1,3 @@ +Allowed the pure Python implementation of :class:`datetime.timezone` to represent +sub-minute offsets close to minimum and maximum boundaries, specifically in the +ranges (23:59, 24:00) and (-23:59, 24:00). Patch by Ngalim Siregar diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index b55922cd961a..6d28b3e511c2 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -1099,7 +1099,9 @@ new_timezone(PyObject *offset, PyObject *name) Py_INCREF(PyDateTime_TimeZone_UTC); return PyDateTime_TimeZone_UTC; } - if ((GET_TD_DAYS(offset) == -1 && GET_TD_SECONDS(offset) == 0) || + if ((GET_TD_DAYS(offset) == -1 && + GET_TD_SECONDS(offset) == 0 && + GET_TD_MICROSECONDS(offset) < 1) || GET_TD_DAYS(offset) < -1 || GET_TD_DAYS(offset) >= 1) { PyErr_Format(PyExc_ValueError, "offset must be a timedelta" " strictly between -timedelta(hours=24) and" @@ -1169,7 +1171,9 @@ call_tzinfo_method(PyObject *tzinfo, const char *name, PyObject *tzinfoarg) if (offset == Py_None || offset == NULL) return offset; if (PyDelta_Check(offset)) { - if ((GET_TD_DAYS(offset) == -1 && GET_TD_SECONDS(offset) == 0) || + if ((GET_TD_DAYS(offset) == -1 && + GET_TD_SECONDS(offset) == 0 && + GET_TD_MICROSECONDS(offset) < 1) || GET_TD_DAYS(offset) < -1 || GET_TD_DAYS(offset) >= 1) { Py_DECREF(offset); PyErr_Format(PyExc_ValueError, "offset must be a timedelta" @@ -6481,6 +6485,9 @@ PyInit__datetime(void) PyDateTime_TimeZone_UTC = x; CAPI.TimeZone_UTC = PyDateTime_TimeZone_UTC; + /* bpo-37642: These attributes are rounded to the nearest minute for backwards + * compatibility, even though the constructor will accept a wider range of + * values. This may change in the future.*/ delta = new_delta(-1, 60, 0, 1); /* -23:59 */ if (delta == NULL) return NULL; From webhook-mailer at python.org Fri Aug 9 11:22:26 2019 From: webhook-mailer at python.org (Ned Deily) Date: Fri, 09 Aug 2019 15:22:26 -0000 Subject: [Python-checkins] bpo-34155: Dont parse domains containing @ (GH-13079) (GH-14826) Message-ID: https://github.com/python/cpython/commit/13a19139b5e76175bc95294d54afc9425e4f36c9 commit: 13a19139b5e76175bc95294d54afc9425e4f36c9 branch: 3.6 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Ned Deily date: 2019-08-09T11:22:19-04:00 summary: bpo-34155: Dont parse domains containing @ (GH-13079) (GH-14826) Before: >>> email.message_from_string('From: a at malicious.org@important.com', policy=email.policy.default)['from'].addresses (Address(display_name='', username='a', domain='malicious.org'),) >>> parseaddr('a at malicious.org@important.com') ('', 'a at malicious.org') After: >>> email.message_from_string('From: a at malicious.org@important.com', policy=email.policy.default)['from'].addresses (Address(display_name='', username='', domain=''),) >>> parseaddr('a at malicious.org@important.com') ('', 'a@') https://bugs.python.org/issue34155 (cherry picked from commit 8cb65d1381b027f0b09ee36bfed7f35bb4dec9a9) Co-authored-by: jpic files: A Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst M Lib/email/_header_value_parser.py M Lib/email/_parseaddr.py M Lib/test/test_email/test__header_value_parser.py M Lib/test/test_email/test_email.py diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py index 737951e4b1b1..bc9c9b6241d4 100644 --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -1561,6 +1561,8 @@ def get_domain(value): token, value = get_dot_atom(value) except errors.HeaderParseError: token, value = get_atom(value) + if value and value[0] == '@': + raise errors.HeaderParseError('Invalid Domain') if leader is not None: token[:0] = [leader] domain.append(token) diff --git a/Lib/email/_parseaddr.py b/Lib/email/_parseaddr.py index cdfa3729adc7..41ff6f8c000d 100644 --- a/Lib/email/_parseaddr.py +++ b/Lib/email/_parseaddr.py @@ -379,7 +379,12 @@ def getaddrspec(self): aslist.append('@') self.pos += 1 self.gotonext() - return EMPTYSTRING.join(aslist) + self.getdomain() + domain = self.getdomain() + if not domain: + # Invalid domain, return an empty address instead of returning a + # local part to denote failed parsing. + return EMPTYSTRING + return EMPTYSTRING.join(aslist) + domain def getdomain(self): """Get the complete domain name from an address.""" @@ -394,6 +399,10 @@ def getdomain(self): elif self.field[self.pos] == '.': self.pos += 1 sdlist.append('.') + elif self.field[self.pos] == '@': + # bpo-34155: Don't parse domains with two `@` like + # `a at malicious.org@important.com`. + return EMPTYSTRING elif self.field[self.pos] in self.atomends: break else: diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py index a2c900fa7fd2..02ef3e1006c6 100644 --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -1418,6 +1418,16 @@ def test_get_addr_spec_dot_atom(self): self.assertEqual(addr_spec.domain, 'example.com') self.assertEqual(addr_spec.addr_spec, 'star.a.star at example.com') + def test_get_addr_spec_multiple_domains(self): + with self.assertRaises(errors.HeaderParseError): + parser.get_addr_spec('star at a.star@example.com') + + with self.assertRaises(errors.HeaderParseError): + parser.get_addr_spec('star at a@example.com') + + with self.assertRaises(errors.HeaderParseError): + parser.get_addr_spec('star at 172.17.0.1@example.com') + # get_obs_route def test_get_obs_route_simple(self): diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index f97ccc6711cc..68d052279987 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -3035,6 +3035,20 @@ def test_parseaddr_empty(self): self.assertEqual(utils.parseaddr('<>'), ('', '')) self.assertEqual(utils.formataddr(utils.parseaddr('<>')), '') + def test_parseaddr_multiple_domains(self): + self.assertEqual( + utils.parseaddr('a at b@c'), + ('', '') + ) + self.assertEqual( + utils.parseaddr('a at b.c@c'), + ('', '') + ) + self.assertEqual( + utils.parseaddr('a at 172.17.0.1@c'), + ('', '') + ) + def test_noquote_dump(self): self.assertEqual( utils.formataddr(('A Silly Person', 'person at dom.ain')), diff --git a/Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst b/Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst new file mode 100644 index 000000000000..50292e29ed1d --- /dev/null +++ b/Misc/NEWS.d/next/Security/2019-05-04-13-33-37.bpo-34155.MJll68.rst @@ -0,0 +1 @@ +Fix parsing of invalid email addresses with more than one ``@`` (e.g. a at b@c.com.) to not return the part before 2nd ``@`` as valid email address. Patch by maxking & jpic. From webhook-mailer at python.org Fri Aug 9 18:34:28 2019 From: webhook-mailer at python.org (Gregory P. Smith) Date: Fri, 09 Aug 2019 22:34:28 -0000 Subject: [Python-checkins] [3.8] bpo-32912: Revert SyntaxWarning on invalid escape sequences (GH-15142) Message-ID: https://github.com/python/cpython/commit/4c5b6bac2408f879231c7cd38d67657dd4804e7c commit: 4c5b6bac2408f879231c7cd38d67657dd4804e7c branch: 3.8 author: Serhiy Storchaka committer: Gregory P. Smith date: 2019-08-09T15:34:22-07:00 summary: [3.8] bpo-32912: Revert SyntaxWarning on invalid escape sequences (GH-15142) * bpo-32912: Revert warnings for invalid escape sequences. DeprecationWarning will continue to be emitted for invalid escape sequences in string and bytes literals in 3.8 just as it did in 3.7. SyntaxWarning may be emitted in the future. But per mailing list discussion, we don't yet know when because we haven't settled on how to do so in a non-disruptive manner. files: A Misc/NEWS.d/next/Core and Builtins/2019-08-06-14-03-59.bpo-32912.UDwSMJ.rst M Doc/reference/lexical_analysis.rst M Doc/whatsnew/3.8.rst M Lib/test/test_fstring.py M Lib/test/test_string_literals.py M Python/ast.c diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index cc1b2f57a70e..7e1e17edb2d8 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -594,11 +594,9 @@ escape sequences only recognized in string literals fall into the category of unrecognized escapes for bytes literals. .. versionchanged:: 3.6 - Unrecognized escape sequences produce a :exc:`DeprecationWarning`. - - .. versionchanged:: 3.8 - Unrecognized escape sequences produce a :exc:`SyntaxWarning`. In - some future version of Python they will be a :exc:`SyntaxError`. + Unrecognized escape sequences produce a :exc:`DeprecationWarning`. In + a future Python version they will be a :exc:`SyntaxWarning` and + eventually a :exc:`SyntaxError`. Even in a raw literal, quotes can be escaped with a backslash, but the backslash remains in the result; for example, ``r"\""`` is a valid string diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 9f7058274514..83caa2cc5abc 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -414,11 +414,6 @@ Other Language Changes and :keyword:`return` statements. (Contributed by David Cuthbert and Jordan Chapman in :issue:`32117`.) -* A backslash-character pair that is not a valid escape sequence generates - a :exc:`DeprecationWarning` since Python 3.6. In Python 3.8 it generates - a :exc:`SyntaxWarning` instead. - (Contributed by Serhiy Storchaka in :issue:`32912`.) - * The compiler now produces a :exc:`SyntaxWarning` in some cases when a comma is missed before tuple or list. For example:: diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py index fb761441fcee..49663923e7f5 100644 --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -649,7 +649,7 @@ def test_backslashes_in_string_part(self): self.assertEqual(f'2\x203', '2 3') self.assertEqual(f'\x203', ' 3') - with self.assertWarns(SyntaxWarning): # invalid escape sequence + with self.assertWarns(DeprecationWarning): # invalid escape sequence value = eval(r"f'\{6*7}'") self.assertEqual(value, '\\42') self.assertEqual(f'\\{6*7}', '\\42') diff --git a/Lib/test/test_string_literals.py b/Lib/test/test_string_literals.py index 5961d591c448..0cea2edc32af 100644 --- a/Lib/test/test_string_literals.py +++ b/Lib/test/test_string_literals.py @@ -32,6 +32,7 @@ import shutil import tempfile import unittest +import warnings TEMPLATE = r"""# coding: %s @@ -110,10 +111,24 @@ def test_eval_str_invalid_escape(self): for b in range(1, 128): if b in b"""\n\r"'01234567NU\\abfnrtuvx""": continue - with self.assertWarns(SyntaxWarning): + with self.assertWarns(DeprecationWarning): self.assertEqual(eval(r"'\%c'" % b), '\\' + chr(b)) - self.check_syntax_warning("'''\n\\z'''") + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter('always', category=DeprecationWarning) + eval("'''\n\\z'''") + self.assertEqual(len(w), 1) + self.assertEqual(w[0].filename, '') + self.assertEqual(w[0].lineno, 1) + + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter('error', category=DeprecationWarning) + with self.assertRaises(SyntaxError) as cm: + eval("'''\n\\z'''") + exc = cm.exception + self.assertEqual(w, []) + self.assertEqual(exc.filename, '') + self.assertEqual(exc.lineno, 1) def test_eval_str_raw(self): self.assertEqual(eval(""" r'x' """), 'x') @@ -145,10 +160,24 @@ def test_eval_bytes_invalid_escape(self): for b in range(1, 128): if b in b"""\n\r"'01234567\\abfnrtvx""": continue - with self.assertWarns(SyntaxWarning): + with self.assertWarns(DeprecationWarning): self.assertEqual(eval(r"b'\%c'" % b), b'\\' + bytes([b])) - self.check_syntax_warning("b'''\n\\z'''") + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter('always', category=DeprecationWarning) + eval("b'''\n\\z'''") + self.assertEqual(len(w), 1) + self.assertEqual(w[0].filename, '') + self.assertEqual(w[0].lineno, 1) + + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter('error', category=DeprecationWarning) + with self.assertRaises(SyntaxError) as cm: + eval("b'''\n\\z'''") + exc = cm.exception + self.assertEqual(w, []) + self.assertEqual(exc.filename, '') + self.assertEqual(exc.lineno, 1) def test_eval_bytes_raw(self): self.assertEqual(eval(""" br'x' """), b'x') diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-08-06-14-03-59.bpo-32912.UDwSMJ.rst b/Misc/NEWS.d/next/Core and Builtins/2019-08-06-14-03-59.bpo-32912.UDwSMJ.rst new file mode 100644 index 000000000000..e18d8adfbee9 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-08-06-14-03-59.bpo-32912.UDwSMJ.rst @@ -0,0 +1,3 @@ +Reverted :issue:`32912`: emitting :exc:`SyntaxWarning` instead of +:exc:`DeprecationWarning` for invalid escape sequences in string and bytes +literals. diff --git a/Python/ast.c b/Python/ast.c index f6c2049ae2cf..9947824de744 100644 --- a/Python/ast.c +++ b/Python/ast.c @@ -4674,12 +4674,12 @@ warn_invalid_escape_sequence(struct compiling *c, const node *n, if (msg == NULL) { return -1; } - if (PyErr_WarnExplicitObject(PyExc_SyntaxWarning, msg, + if (PyErr_WarnExplicitObject(PyExc_DeprecationWarning, msg, c->c_filename, LINENO(n), NULL, NULL) < 0) { - if (PyErr_ExceptionMatches(PyExc_SyntaxWarning)) { - /* Replace the SyntaxWarning exception with a SyntaxError + if (PyErr_ExceptionMatches(PyExc_DeprecationWarning)) { + /* Replace the DeprecationWarning exception with a SyntaxError to get a more accurate error report */ PyErr_Clear(); ast_error(c, n, "%U", msg); From webhook-mailer at python.org Sat Aug 10 03:19:16 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 10 Aug 2019 07:19:16 -0000 Subject: [Python-checkins] bpo-32912: Revert SyntaxWarning on invalid escape sequences. (GH-15195) Message-ID: https://github.com/python/cpython/commit/b4be87a04a2a8ccfd2480e19dc527589fce53555 commit: b4be87a04a2a8ccfd2480e19dc527589fce53555 branch: master author: Gregory P. Smith committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-10T00:19:07-07:00 summary: bpo-32912: Revert SyntaxWarning on invalid escape sequences. (GH-15195) DeprecationWarning will continue to be emitted for invalid escape sequences in string and bytes literals just as it did in 3.7. SyntaxWarning may be emitted in the future. But per mailing list discussion, we don't yet know when because we haven't settled on how to do so in a non-disruptive manner. (Applies 4c5b6bac2408f879231c7cd38d67657dd4804e7c to the master branch). (This is https://github.com/python/cpython/pull/15142 for master/3.9) https://bugs.python.org/issue32912 Automerge-Triggered-By: @gpshead files: A Misc/NEWS.d/next/Core and Builtins/2019-08-06-14-03-59.bpo-32912.UDwSMJ.rst M Doc/reference/lexical_analysis.rst M Doc/whatsnew/3.8.rst M Lib/test/test_fstring.py M Lib/test/test_string_literals.py M Python/ast.c diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index cc1b2f57a70e..7e1e17edb2d8 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -594,11 +594,9 @@ escape sequences only recognized in string literals fall into the category of unrecognized escapes for bytes literals. .. versionchanged:: 3.6 - Unrecognized escape sequences produce a :exc:`DeprecationWarning`. - - .. versionchanged:: 3.8 - Unrecognized escape sequences produce a :exc:`SyntaxWarning`. In - some future version of Python they will be a :exc:`SyntaxError`. + Unrecognized escape sequences produce a :exc:`DeprecationWarning`. In + a future Python version they will be a :exc:`SyntaxWarning` and + eventually a :exc:`SyntaxError`. Even in a raw literal, quotes can be escaped with a backslash, but the backslash remains in the result; for example, ``r"\""`` is a valid string diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index d8062e772f6e..82da10cc3be8 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -414,11 +414,6 @@ Other Language Changes and :keyword:`return` statements. (Contributed by David Cuthbert and Jordan Chapman in :issue:`32117`.) -* A backslash-character pair that is not a valid escape sequence generates - a :exc:`DeprecationWarning` since Python 3.6. In Python 3.8 it generates - a :exc:`SyntaxWarning` instead. - (Contributed by Serhiy Storchaka in :issue:`32912`.) - * The compiler now produces a :exc:`SyntaxWarning` in some cases when a comma is missed before tuple or list. For example:: diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py index fb761441fcee..49663923e7f5 100644 --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -649,7 +649,7 @@ def test_backslashes_in_string_part(self): self.assertEqual(f'2\x203', '2 3') self.assertEqual(f'\x203', ' 3') - with self.assertWarns(SyntaxWarning): # invalid escape sequence + with self.assertWarns(DeprecationWarning): # invalid escape sequence value = eval(r"f'\{6*7}'") self.assertEqual(value, '\\42') self.assertEqual(f'\\{6*7}', '\\42') diff --git a/Lib/test/test_string_literals.py b/Lib/test/test_string_literals.py index 5961d591c448..0cea2edc32af 100644 --- a/Lib/test/test_string_literals.py +++ b/Lib/test/test_string_literals.py @@ -32,6 +32,7 @@ import shutil import tempfile import unittest +import warnings TEMPLATE = r"""# coding: %s @@ -110,10 +111,24 @@ def test_eval_str_invalid_escape(self): for b in range(1, 128): if b in b"""\n\r"'01234567NU\\abfnrtuvx""": continue - with self.assertWarns(SyntaxWarning): + with self.assertWarns(DeprecationWarning): self.assertEqual(eval(r"'\%c'" % b), '\\' + chr(b)) - self.check_syntax_warning("'''\n\\z'''") + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter('always', category=DeprecationWarning) + eval("'''\n\\z'''") + self.assertEqual(len(w), 1) + self.assertEqual(w[0].filename, '') + self.assertEqual(w[0].lineno, 1) + + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter('error', category=DeprecationWarning) + with self.assertRaises(SyntaxError) as cm: + eval("'''\n\\z'''") + exc = cm.exception + self.assertEqual(w, []) + self.assertEqual(exc.filename, '') + self.assertEqual(exc.lineno, 1) def test_eval_str_raw(self): self.assertEqual(eval(""" r'x' """), 'x') @@ -145,10 +160,24 @@ def test_eval_bytes_invalid_escape(self): for b in range(1, 128): if b in b"""\n\r"'01234567\\abfnrtvx""": continue - with self.assertWarns(SyntaxWarning): + with self.assertWarns(DeprecationWarning): self.assertEqual(eval(r"b'\%c'" % b), b'\\' + bytes([b])) - self.check_syntax_warning("b'''\n\\z'''") + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter('always', category=DeprecationWarning) + eval("b'''\n\\z'''") + self.assertEqual(len(w), 1) + self.assertEqual(w[0].filename, '') + self.assertEqual(w[0].lineno, 1) + + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter('error', category=DeprecationWarning) + with self.assertRaises(SyntaxError) as cm: + eval("b'''\n\\z'''") + exc = cm.exception + self.assertEqual(w, []) + self.assertEqual(exc.filename, '') + self.assertEqual(exc.lineno, 1) def test_eval_bytes_raw(self): self.assertEqual(eval(""" br'x' """), b'x') diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-08-06-14-03-59.bpo-32912.UDwSMJ.rst b/Misc/NEWS.d/next/Core and Builtins/2019-08-06-14-03-59.bpo-32912.UDwSMJ.rst new file mode 100644 index 000000000000..e18d8adfbee9 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-08-06-14-03-59.bpo-32912.UDwSMJ.rst @@ -0,0 +1,3 @@ +Reverted :issue:`32912`: emitting :exc:`SyntaxWarning` instead of +:exc:`DeprecationWarning` for invalid escape sequences in string and bytes +literals. diff --git a/Python/ast.c b/Python/ast.c index 976be70d4d70..8b3dbead2fdc 100644 --- a/Python/ast.c +++ b/Python/ast.c @@ -4671,12 +4671,12 @@ warn_invalid_escape_sequence(struct compiling *c, const node *n, if (msg == NULL) { return -1; } - if (PyErr_WarnExplicitObject(PyExc_SyntaxWarning, msg, + if (PyErr_WarnExplicitObject(PyExc_DeprecationWarning, msg, c->c_filename, LINENO(n), NULL, NULL) < 0) { - if (PyErr_ExceptionMatches(PyExc_SyntaxWarning)) { - /* Replace the SyntaxWarning exception with a SyntaxError + if (PyErr_ExceptionMatches(PyExc_DeprecationWarning)) { + /* Replace the DeprecationWarning exception with a SyntaxError to get a more accurate error report */ PyErr_Clear(); ast_error(c, n, "%U", msg); From webhook-mailer at python.org Sat Aug 10 03:20:31 2019 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Sat, 10 Aug 2019 07:20:31 -0000 Subject: [Python-checkins] Delete leftover clinic-generated file for C zipimport. (GH-15174) Message-ID: https://github.com/python/cpython/commit/51aac15f6d525595e200e3580409c4b8656e8a96 commit: 51aac15f6d525595e200e3580409c4b8656e8a96 branch: master author: Greg Price committer: Serhiy Storchaka date: 2019-08-10T10:20:27+03:00 summary: Delete leftover clinic-generated file for C zipimport. (GH-15174) files: D Modules/clinic/zipimport.c.h diff --git a/Modules/clinic/zipimport.c.h b/Modules/clinic/zipimport.c.h deleted file mode 100644 index aabe7a05ae9d..000000000000 --- a/Modules/clinic/zipimport.c.h +++ /dev/null @@ -1,325 +0,0 @@ -/*[clinic input] -preserve -[clinic start generated code]*/ - -PyDoc_STRVAR(zipimport_zipimporter___init____doc__, -"zipimporter(archivepath, /)\n" -"--\n" -"\n" -"Create a new zipimporter instance.\n" -"\n" -" archivepath\n" -" A path-like object to a zipfile, or to a specific path inside\n" -" a zipfile.\n" -"\n" -"\'archivepath\' must be a path-like object to a zipfile, or to a specific path\n" -"inside a zipfile. For example, it can be \'/tmp/myimport.zip\', or\n" -"\'/tmp/myimport.zip/mydirectory\', if mydirectory is a valid directory inside\n" -"the archive.\n" -"\n" -"\'ZipImportError\' is raised if \'archivepath\' doesn\'t point to a valid Zip\n" -"archive.\n" -"\n" -"The \'archive\' attribute of the zipimporter object contains the name of the\n" -"zipfile targeted."); - -static int -zipimport_zipimporter___init___impl(ZipImporter *self, PyObject *path); - -static int -zipimport_zipimporter___init__(PyObject *self, PyObject *args, PyObject *kwargs) -{ - int return_value = -1; - PyObject *path; - - if ((Py_TYPE(self) == &ZipImporter_Type) && - !_PyArg_NoKeywords("zipimporter", kwargs)) { - goto exit; - } - if (!PyArg_ParseTuple(args, "O&:zipimporter", - PyUnicode_FSDecoder, &path)) { - goto exit; - } - return_value = zipimport_zipimporter___init___impl((ZipImporter *)self, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_find_module__doc__, -"find_module($self, fullname, path=None, /)\n" -"--\n" -"\n" -"Search for a module specified by \'fullname\'.\n" -"\n" -"\'fullname\' must be the fully qualified (dotted) module name. It returns the\n" -"zipimporter instance itself if the module was found, or None if it wasn\'t.\n" -"The optional \'path\' argument is ignored -- it\'s there for compatibility\n" -"with the importer protocol."); - -#define ZIPIMPORT_ZIPIMPORTER_FIND_MODULE_METHODDEF \ - {"find_module", (PyCFunction)(void *)zipimport_zipimporter_find_module, METH_FASTCALL, zipimport_zipimporter_find_module__doc__}, - -static PyObject * -zipimport_zipimporter_find_module_impl(ZipImporter *self, PyObject *fullname, - PyObject *path); - -static PyObject * -zipimport_zipimporter_find_module(ZipImporter *self, PyObject *const *args, Py_ssize_t nargs) -{ - PyObject *return_value = NULL; - PyObject *fullname; - PyObject *path = Py_None; - - if (!_PyArg_ParseStack(args, nargs, "U|O:find_module", - &fullname, &path)) { - goto exit; - } - return_value = zipimport_zipimporter_find_module_impl(self, fullname, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_find_loader__doc__, -"find_loader($self, fullname, path=None, /)\n" -"--\n" -"\n" -"Search for a module specified by \'fullname\'.\n" -"\n" -"\'fullname\' must be the fully qualified (dotted) module name. It returns the\n" -"zipimporter instance itself if the module was found, a string containing the\n" -"full path name if it\'s possibly a portion of a namespace package,\n" -"or None otherwise. The optional \'path\' argument is ignored -- it\'s\n" -"there for compatibility with the importer protocol."); - -#define ZIPIMPORT_ZIPIMPORTER_FIND_LOADER_METHODDEF \ - {"find_loader", (PyCFunction)(void *)zipimport_zipimporter_find_loader, METH_FASTCALL, zipimport_zipimporter_find_loader__doc__}, - -static PyObject * -zipimport_zipimporter_find_loader_impl(ZipImporter *self, PyObject *fullname, - PyObject *path); - -static PyObject * -zipimport_zipimporter_find_loader(ZipImporter *self, PyObject *const *args, Py_ssize_t nargs) -{ - PyObject *return_value = NULL; - PyObject *fullname; - PyObject *path = Py_None; - - if (!_PyArg_ParseStack(args, nargs, "U|O:find_loader", - &fullname, &path)) { - goto exit; - } - return_value = zipimport_zipimporter_find_loader_impl(self, fullname, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_load_module__doc__, -"load_module($self, fullname, /)\n" -"--\n" -"\n" -"Load the module specified by \'fullname\'.\n" -"\n" -"\'fullname\' must be the fully qualified (dotted) module name. It returns the\n" -"imported module, or raises ZipImportError if it wasn\'t found."); - -#define ZIPIMPORT_ZIPIMPORTER_LOAD_MODULE_METHODDEF \ - {"load_module", (PyCFunction)zipimport_zipimporter_load_module, METH_O, zipimport_zipimporter_load_module__doc__}, - -static PyObject * -zipimport_zipimporter_load_module_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_load_module(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:load_module", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_load_module_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_filename__doc__, -"get_filename($self, fullname, /)\n" -"--\n" -"\n" -"Return the filename for the specified module."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_FILENAME_METHODDEF \ - {"get_filename", (PyCFunction)zipimport_zipimporter_get_filename, METH_O, zipimport_zipimporter_get_filename__doc__}, - -static PyObject * -zipimport_zipimporter_get_filename_impl(ZipImporter *self, - PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_filename(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_filename", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_filename_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_is_package__doc__, -"is_package($self, fullname, /)\n" -"--\n" -"\n" -"Return True if the module specified by fullname is a package.\n" -"\n" -"Raise ZipImportError if the module couldn\'t be found."); - -#define ZIPIMPORT_ZIPIMPORTER_IS_PACKAGE_METHODDEF \ - {"is_package", (PyCFunction)zipimport_zipimporter_is_package, METH_O, zipimport_zipimporter_is_package__doc__}, - -static PyObject * -zipimport_zipimporter_is_package_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_is_package(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:is_package", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_is_package_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_data__doc__, -"get_data($self, pathname, /)\n" -"--\n" -"\n" -"Return the data associated with \'pathname\'.\n" -"\n" -"Raise OSError if the file was not found."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_DATA_METHODDEF \ - {"get_data", (PyCFunction)zipimport_zipimporter_get_data, METH_O, zipimport_zipimporter_get_data__doc__}, - -static PyObject * -zipimport_zipimporter_get_data_impl(ZipImporter *self, PyObject *path); - -static PyObject * -zipimport_zipimporter_get_data(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *path; - - if (!PyArg_Parse(arg, "U:get_data", &path)) { - goto exit; - } - return_value = zipimport_zipimporter_get_data_impl(self, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_code__doc__, -"get_code($self, fullname, /)\n" -"--\n" -"\n" -"Return the code object for the specified module.\n" -"\n" -"Raise ZipImportError if the module couldn\'t be found."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_CODE_METHODDEF \ - {"get_code", (PyCFunction)zipimport_zipimporter_get_code, METH_O, zipimport_zipimporter_get_code__doc__}, - -static PyObject * -zipimport_zipimporter_get_code_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_code(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_code", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_code_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_source__doc__, -"get_source($self, fullname, /)\n" -"--\n" -"\n" -"Return the source code for the specified module.\n" -"\n" -"Raise ZipImportError if the module couldn\'t be found, return None if the\n" -"archive does contain the module, but has no source for it."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_SOURCE_METHODDEF \ - {"get_source", (PyCFunction)zipimport_zipimporter_get_source, METH_O, zipimport_zipimporter_get_source__doc__}, - -static PyObject * -zipimport_zipimporter_get_source_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_source(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_source", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_source_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_resource_reader__doc__, -"get_resource_reader($self, fullname, /)\n" -"--\n" -"\n" -"Return the ResourceReader for a package in a zip file.\n" -"\n" -"If \'fullname\' is a package within the zip file, return the \'ResourceReader\'\n" -"object for the package. Otherwise return None."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_RESOURCE_READER_METHODDEF \ - {"get_resource_reader", (PyCFunction)zipimport_zipimporter_get_resource_reader, METH_O, zipimport_zipimporter_get_resource_reader__doc__}, - -static PyObject * -zipimport_zipimporter_get_resource_reader_impl(ZipImporter *self, - PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_resource_reader(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_resource_reader", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_resource_reader_impl(self, fullname); - -exit: - return return_value; -} -/*[clinic end generated code: output=854ce3502180dc1f input=a9049054013a1b77]*/ From webhook-mailer at python.org Sat Aug 10 03:37:39 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 10 Aug 2019 07:37:39 -0000 Subject: [Python-checkins] Delete leftover clinic-generated file for C zipimport. (GH-15174) Message-ID: https://github.com/python/cpython/commit/c61f9b57cfce9e96bdea5674634e3a9c185b675f commit: c61f9b57cfce9e96bdea5674634e3a9c185b675f branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-10T00:37:35-07:00 summary: Delete leftover clinic-generated file for C zipimport. (GH-15174) (cherry picked from commit 51aac15f6d525595e200e3580409c4b8656e8a96) Co-authored-by: Greg Price files: D Modules/clinic/zipimport.c.h diff --git a/Modules/clinic/zipimport.c.h b/Modules/clinic/zipimport.c.h deleted file mode 100644 index aabe7a05ae9d..000000000000 --- a/Modules/clinic/zipimport.c.h +++ /dev/null @@ -1,325 +0,0 @@ -/*[clinic input] -preserve -[clinic start generated code]*/ - -PyDoc_STRVAR(zipimport_zipimporter___init____doc__, -"zipimporter(archivepath, /)\n" -"--\n" -"\n" -"Create a new zipimporter instance.\n" -"\n" -" archivepath\n" -" A path-like object to a zipfile, or to a specific path inside\n" -" a zipfile.\n" -"\n" -"\'archivepath\' must be a path-like object to a zipfile, or to a specific path\n" -"inside a zipfile. For example, it can be \'/tmp/myimport.zip\', or\n" -"\'/tmp/myimport.zip/mydirectory\', if mydirectory is a valid directory inside\n" -"the archive.\n" -"\n" -"\'ZipImportError\' is raised if \'archivepath\' doesn\'t point to a valid Zip\n" -"archive.\n" -"\n" -"The \'archive\' attribute of the zipimporter object contains the name of the\n" -"zipfile targeted."); - -static int -zipimport_zipimporter___init___impl(ZipImporter *self, PyObject *path); - -static int -zipimport_zipimporter___init__(PyObject *self, PyObject *args, PyObject *kwargs) -{ - int return_value = -1; - PyObject *path; - - if ((Py_TYPE(self) == &ZipImporter_Type) && - !_PyArg_NoKeywords("zipimporter", kwargs)) { - goto exit; - } - if (!PyArg_ParseTuple(args, "O&:zipimporter", - PyUnicode_FSDecoder, &path)) { - goto exit; - } - return_value = zipimport_zipimporter___init___impl((ZipImporter *)self, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_find_module__doc__, -"find_module($self, fullname, path=None, /)\n" -"--\n" -"\n" -"Search for a module specified by \'fullname\'.\n" -"\n" -"\'fullname\' must be the fully qualified (dotted) module name. It returns the\n" -"zipimporter instance itself if the module was found, or None if it wasn\'t.\n" -"The optional \'path\' argument is ignored -- it\'s there for compatibility\n" -"with the importer protocol."); - -#define ZIPIMPORT_ZIPIMPORTER_FIND_MODULE_METHODDEF \ - {"find_module", (PyCFunction)(void *)zipimport_zipimporter_find_module, METH_FASTCALL, zipimport_zipimporter_find_module__doc__}, - -static PyObject * -zipimport_zipimporter_find_module_impl(ZipImporter *self, PyObject *fullname, - PyObject *path); - -static PyObject * -zipimport_zipimporter_find_module(ZipImporter *self, PyObject *const *args, Py_ssize_t nargs) -{ - PyObject *return_value = NULL; - PyObject *fullname; - PyObject *path = Py_None; - - if (!_PyArg_ParseStack(args, nargs, "U|O:find_module", - &fullname, &path)) { - goto exit; - } - return_value = zipimport_zipimporter_find_module_impl(self, fullname, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_find_loader__doc__, -"find_loader($self, fullname, path=None, /)\n" -"--\n" -"\n" -"Search for a module specified by \'fullname\'.\n" -"\n" -"\'fullname\' must be the fully qualified (dotted) module name. It returns the\n" -"zipimporter instance itself if the module was found, a string containing the\n" -"full path name if it\'s possibly a portion of a namespace package,\n" -"or None otherwise. The optional \'path\' argument is ignored -- it\'s\n" -"there for compatibility with the importer protocol."); - -#define ZIPIMPORT_ZIPIMPORTER_FIND_LOADER_METHODDEF \ - {"find_loader", (PyCFunction)(void *)zipimport_zipimporter_find_loader, METH_FASTCALL, zipimport_zipimporter_find_loader__doc__}, - -static PyObject * -zipimport_zipimporter_find_loader_impl(ZipImporter *self, PyObject *fullname, - PyObject *path); - -static PyObject * -zipimport_zipimporter_find_loader(ZipImporter *self, PyObject *const *args, Py_ssize_t nargs) -{ - PyObject *return_value = NULL; - PyObject *fullname; - PyObject *path = Py_None; - - if (!_PyArg_ParseStack(args, nargs, "U|O:find_loader", - &fullname, &path)) { - goto exit; - } - return_value = zipimport_zipimporter_find_loader_impl(self, fullname, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_load_module__doc__, -"load_module($self, fullname, /)\n" -"--\n" -"\n" -"Load the module specified by \'fullname\'.\n" -"\n" -"\'fullname\' must be the fully qualified (dotted) module name. It returns the\n" -"imported module, or raises ZipImportError if it wasn\'t found."); - -#define ZIPIMPORT_ZIPIMPORTER_LOAD_MODULE_METHODDEF \ - {"load_module", (PyCFunction)zipimport_zipimporter_load_module, METH_O, zipimport_zipimporter_load_module__doc__}, - -static PyObject * -zipimport_zipimporter_load_module_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_load_module(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:load_module", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_load_module_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_filename__doc__, -"get_filename($self, fullname, /)\n" -"--\n" -"\n" -"Return the filename for the specified module."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_FILENAME_METHODDEF \ - {"get_filename", (PyCFunction)zipimport_zipimporter_get_filename, METH_O, zipimport_zipimporter_get_filename__doc__}, - -static PyObject * -zipimport_zipimporter_get_filename_impl(ZipImporter *self, - PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_filename(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_filename", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_filename_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_is_package__doc__, -"is_package($self, fullname, /)\n" -"--\n" -"\n" -"Return True if the module specified by fullname is a package.\n" -"\n" -"Raise ZipImportError if the module couldn\'t be found."); - -#define ZIPIMPORT_ZIPIMPORTER_IS_PACKAGE_METHODDEF \ - {"is_package", (PyCFunction)zipimport_zipimporter_is_package, METH_O, zipimport_zipimporter_is_package__doc__}, - -static PyObject * -zipimport_zipimporter_is_package_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_is_package(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:is_package", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_is_package_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_data__doc__, -"get_data($self, pathname, /)\n" -"--\n" -"\n" -"Return the data associated with \'pathname\'.\n" -"\n" -"Raise OSError if the file was not found."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_DATA_METHODDEF \ - {"get_data", (PyCFunction)zipimport_zipimporter_get_data, METH_O, zipimport_zipimporter_get_data__doc__}, - -static PyObject * -zipimport_zipimporter_get_data_impl(ZipImporter *self, PyObject *path); - -static PyObject * -zipimport_zipimporter_get_data(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *path; - - if (!PyArg_Parse(arg, "U:get_data", &path)) { - goto exit; - } - return_value = zipimport_zipimporter_get_data_impl(self, path); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_code__doc__, -"get_code($self, fullname, /)\n" -"--\n" -"\n" -"Return the code object for the specified module.\n" -"\n" -"Raise ZipImportError if the module couldn\'t be found."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_CODE_METHODDEF \ - {"get_code", (PyCFunction)zipimport_zipimporter_get_code, METH_O, zipimport_zipimporter_get_code__doc__}, - -static PyObject * -zipimport_zipimporter_get_code_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_code(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_code", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_code_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_source__doc__, -"get_source($self, fullname, /)\n" -"--\n" -"\n" -"Return the source code for the specified module.\n" -"\n" -"Raise ZipImportError if the module couldn\'t be found, return None if the\n" -"archive does contain the module, but has no source for it."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_SOURCE_METHODDEF \ - {"get_source", (PyCFunction)zipimport_zipimporter_get_source, METH_O, zipimport_zipimporter_get_source__doc__}, - -static PyObject * -zipimport_zipimporter_get_source_impl(ZipImporter *self, PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_source(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_source", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_source_impl(self, fullname); - -exit: - return return_value; -} - -PyDoc_STRVAR(zipimport_zipimporter_get_resource_reader__doc__, -"get_resource_reader($self, fullname, /)\n" -"--\n" -"\n" -"Return the ResourceReader for a package in a zip file.\n" -"\n" -"If \'fullname\' is a package within the zip file, return the \'ResourceReader\'\n" -"object for the package. Otherwise return None."); - -#define ZIPIMPORT_ZIPIMPORTER_GET_RESOURCE_READER_METHODDEF \ - {"get_resource_reader", (PyCFunction)zipimport_zipimporter_get_resource_reader, METH_O, zipimport_zipimporter_get_resource_reader__doc__}, - -static PyObject * -zipimport_zipimporter_get_resource_reader_impl(ZipImporter *self, - PyObject *fullname); - -static PyObject * -zipimport_zipimporter_get_resource_reader(ZipImporter *self, PyObject *arg) -{ - PyObject *return_value = NULL; - PyObject *fullname; - - if (!PyArg_Parse(arg, "U:get_resource_reader", &fullname)) { - goto exit; - } - return_value = zipimport_zipimporter_get_resource_reader_impl(self, fullname); - -exit: - return return_value; -} -/*[clinic end generated code: output=854ce3502180dc1f input=a9049054013a1b77]*/ From webhook-mailer at python.org Sun Aug 11 16:45:16 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 11 Aug 2019 20:45:16 -0000 Subject: [Python-checkins] bpo-32178: Fix IndexError trying to parse 'To' header starting with ':'. (GH-15044) Message-ID: https://github.com/python/cpython/commit/09a1872a8007048dcdf825a476816c5e3498b8f8 commit: 09a1872a8007048dcdf825a476816c5e3498b8f8 branch: master author: Abhilash Raj committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-11T13:45:09-07:00 summary: bpo-32178: Fix IndexError trying to parse 'To' header starting with ':'. (GH-15044) This should fix the IndexError trying to retrieve `DisplayName.display_name` and `DisplayName.value` when the `value` is basically an empty string. https://bugs.python.org/issue32178 files: A Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst M Lib/email/_header_value_parser.py M Lib/test/test_email/test__header_value_parser.py diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py index 641e0979f3d7..ea33bd8eda7d 100644 --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -561,6 +561,8 @@ class DisplayName(Phrase): @property def display_name(self): res = TokenList(self) + if len(res) == 0: + return res.value if res[0].token_type == 'cfws': res.pop(0) else: @@ -582,7 +584,7 @@ def value(self): for x in self: if x.token_type == 'quoted-string': quote = True - if quote: + if len(self) != 0 and quote: pre = post = '' if self[0].token_type=='cfws' or self[0][0].token_type=='cfws': pre = ' ' diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py index f6e5886f7571..b3e6b2661524 100644 --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -1700,6 +1700,14 @@ def test_get_display_name_ending_with_obsolete(self): self.assertEqual(display_name[3].comments, ['with trailing comment']) self.assertEqual(display_name.display_name, 'simple phrase.') + def test_get_display_name_for_invalid_address_field(self): + # bpo-32178: Test that address fields starting with `:` don't cause + # IndexError when parsing the display name. + display_name = self._test_get_x( + parser.get_display_name, + ':Foo ', '', '', [errors.InvalidHeaderDefect], ':Foo ') + self.assertEqual(display_name.value, '') + # get_name_addr def test_get_name_addr_angle_addr_only(self): diff --git a/Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst b/Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst new file mode 100644 index 000000000000..5e7a2e964d93 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst @@ -0,0 +1 @@ +Fix IndexError in :mod:`email` package when trying to parse invalid address fields starting with ``:``. From webhook-mailer at python.org Sun Aug 11 17:04:35 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 11 Aug 2019 21:04:35 -0000 Subject: [Python-checkins] bpo-32178: Fix IndexError trying to parse 'To' header starting with ':'. (GH-15044) Message-ID: https://github.com/python/cpython/commit/9500bbe9372f6080decc49d2fd9365f0b927a0e2 commit: 9500bbe9372f6080decc49d2fd9365f0b927a0e2 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-11T14:04:31-07:00 summary: bpo-32178: Fix IndexError trying to parse 'To' header starting with ':'. (GH-15044) This should fix the IndexError trying to retrieve `DisplayName.display_name` and `DisplayName.value` when the `value` is basically an empty string. https://bugs.python.org/issue32178 (cherry picked from commit 09a1872a8007048dcdf825a476816c5e3498b8f8) Co-authored-by: Abhilash Raj files: A Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst M Lib/email/_header_value_parser.py M Lib/test/test_email/test__header_value_parser.py diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py index a93079278893..3d369e4822cd 100644 --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -549,6 +549,8 @@ class DisplayName(Phrase): @property def display_name(self): res = TokenList(self) + if len(res) == 0: + return res.value if res[0].token_type == 'cfws': res.pop(0) else: @@ -570,7 +572,7 @@ def value(self): for x in self: if x.token_type == 'quoted-string': quote = True - if quote: + if len(self) != 0 and quote: pre = post = '' if self[0].token_type=='cfws' or self[0][0].token_type=='cfws': pre = ' ' diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py index 010be9fa4ae6..198cf17bbf3e 100644 --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -1680,6 +1680,14 @@ def test_get_display_name_ending_with_obsolete(self): self.assertEqual(display_name[3].comments, ['with trailing comment']) self.assertEqual(display_name.display_name, 'simple phrase.') + def test_get_display_name_for_invalid_address_field(self): + # bpo-32178: Test that address fields starting with `:` don't cause + # IndexError when parsing the display name. + display_name = self._test_get_x( + parser.get_display_name, + ':Foo ', '', '', [errors.InvalidHeaderDefect], ':Foo ') + self.assertEqual(display_name.value, '') + # get_name_addr def test_get_name_addr_angle_addr_only(self): diff --git a/Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst b/Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst new file mode 100644 index 000000000000..5e7a2e964d93 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst @@ -0,0 +1 @@ +Fix IndexError in :mod:`email` package when trying to parse invalid address fields starting with ``:``. From webhook-mailer at python.org Sun Aug 11 17:05:41 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 11 Aug 2019 21:05:41 -0000 Subject: [Python-checkins] bpo-32178: Fix IndexError trying to parse 'To' header starting with ':'. (GH-15044) Message-ID: https://github.com/python/cpython/commit/dec231a73c2a463b29f19c4e8357602c10a68856 commit: dec231a73c2a463b29f19c4e8357602c10a68856 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-11T14:05:37-07:00 summary: bpo-32178: Fix IndexError trying to parse 'To' header starting with ':'. (GH-15044) This should fix the IndexError trying to retrieve `DisplayName.display_name` and `DisplayName.value` when the `value` is basically an empty string. https://bugs.python.org/issue32178 (cherry picked from commit 09a1872a8007048dcdf825a476816c5e3498b8f8) Co-authored-by: Abhilash Raj files: A Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst M Lib/email/_header_value_parser.py M Lib/test/test_email/test__header_value_parser.py diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py index c09f4f121ffb..666be034420e 100644 --- a/Lib/email/_header_value_parser.py +++ b/Lib/email/_header_value_parser.py @@ -566,6 +566,8 @@ class DisplayName(Phrase): @property def display_name(self): res = TokenList(self) + if len(res) == 0: + return res.value if res[0].token_type == 'cfws': res.pop(0) else: @@ -587,7 +589,7 @@ def value(self): for x in self: if x.token_type == 'quoted-string': quote = True - if quote: + if len(self) != 0 and quote: pre = post = '' if self[0].token_type=='cfws' or self[0][0].token_type=='cfws': pre = ' ' diff --git a/Lib/test/test_email/test__header_value_parser.py b/Lib/test/test_email/test__header_value_parser.py index 0f19f8bcc2e0..dd6975784568 100644 --- a/Lib/test/test_email/test__header_value_parser.py +++ b/Lib/test/test_email/test__header_value_parser.py @@ -1700,6 +1700,14 @@ def test_get_display_name_ending_with_obsolete(self): self.assertEqual(display_name[3].comments, ['with trailing comment']) self.assertEqual(display_name.display_name, 'simple phrase.') + def test_get_display_name_for_invalid_address_field(self): + # bpo-32178: Test that address fields starting with `:` don't cause + # IndexError when parsing the display name. + display_name = self._test_get_x( + parser.get_display_name, + ':Foo ', '', '', [errors.InvalidHeaderDefect], ':Foo ') + self.assertEqual(display_name.value, '') + # get_name_addr def test_get_name_addr_angle_addr_only(self): diff --git a/Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst b/Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst new file mode 100644 index 000000000000..5e7a2e964d93 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-30-22-41-05.bpo-32178.X-IFLe.rst @@ -0,0 +1 @@ +Fix IndexError in :mod:`email` package when trying to parse invalid address fields starting with ``:``. From webhook-mailer at python.org Sun Aug 11 17:41:03 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sun, 11 Aug 2019 21:41:03 -0000 Subject: [Python-checkins] bpo-37819: Add Fraction.as_integer_ratio() (GH-15212) Message-ID: https://github.com/python/cpython/commit/f03b4c8a48f62134799d368b78da35301af466a3 commit: f03b4c8a48f62134799d368b78da35301af466a3 branch: master author: Raymond Hettinger committer: GitHub date: 2019-08-11T14:40:59-07:00 summary: bpo-37819: Add Fraction.as_integer_ratio() (GH-15212) files: A Misc/NEWS.d/next/Library/2019-08-11-10-34-19.bpo-37819.LVJls-.rst M Doc/library/fractions.rst M Lib/fractions.py M Lib/test/test_fractions.py diff --git a/Doc/library/fractions.rst b/Doc/library/fractions.rst index b5a818e1cafa..58e7126b0bf2 100644 --- a/Doc/library/fractions.rst +++ b/Doc/library/fractions.rst @@ -94,6 +94,13 @@ another rational number, or from a string. Denominator of the Fraction in lowest term. + .. method:: as_integer_ratio() + + Return a tuple of two integers, whose ratio is equal + to the Fraction and with a positive denominator. + + .. versionadded:: 3.8 + .. method:: from_float(flt) This class method constructs a :class:`Fraction` representing the exact diff --git a/Lib/fractions.py b/Lib/fractions.py index 7443bd3e0c6a..e774d58e4035 100644 --- a/Lib/fractions.py +++ b/Lib/fractions.py @@ -216,6 +216,14 @@ def from_decimal(cls, dec): (cls.__name__, dec, type(dec).__name__)) return cls(*dec.as_integer_ratio()) + def as_integer_ratio(self): + """Return the integer ratio as a tuple. + + Return a tuple of two integers, whose ratio is equal to the + Fraction and with a positive denominator. + """ + return (self._numerator, self._denominator) + def limit_denominator(self, max_denominator=1000000): """Closest Fraction to self with denominator at most max_denominator. diff --git a/Lib/test/test_fractions.py b/Lib/test/test_fractions.py index 277916220051..18ab28cfebe0 100644 --- a/Lib/test/test_fractions.py +++ b/Lib/test/test_fractions.py @@ -302,6 +302,12 @@ def testFromDecimal(self): ValueError, "cannot convert NaN to integer ratio", F.from_decimal, Decimal("snan")) + def test_as_integer_ratio(self): + self.assertEqual(F(4, 6).as_integer_ratio(), (2, 3)) + self.assertEqual(F(-4, 6).as_integer_ratio(), (-2, 3)) + self.assertEqual(F(4, -6).as_integer_ratio(), (-2, 3)) + self.assertEqual(F(0, 6).as_integer_ratio(), (0, 1)) + def testLimitDenominator(self): rpi = F('3.1415926535897932') self.assertEqual(rpi.limit_denominator(10000), F(355, 113)) diff --git a/Misc/NEWS.d/next/Library/2019-08-11-10-34-19.bpo-37819.LVJls-.rst b/Misc/NEWS.d/next/Library/2019-08-11-10-34-19.bpo-37819.LVJls-.rst new file mode 100644 index 000000000000..cfc1f1afb4f7 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-11-10-34-19.bpo-37819.LVJls-.rst @@ -0,0 +1,2 @@ +Add Fraction.as_integer_ratio() to match the corresponding methods in bool, +int, float, and decimal. From webhook-mailer at python.org Sun Aug 11 18:02:28 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sun, 11 Aug 2019 22:02:28 -0000 Subject: [Python-checkins] bpo-37819: Add Fraction.as_integer_ratio() (GH-15212) (GH-15215) Message-ID: https://github.com/python/cpython/commit/5ba1cb03939bd86d026db667580f590966b573ea commit: 5ba1cb03939bd86d026db667580f590966b573ea branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Raymond Hettinger date: 2019-08-11T15:02:23-07:00 summary: bpo-37819: Add Fraction.as_integer_ratio() (GH-15212) (GH-15215) (cherry picked from commit f03b4c8a48f62134799d368b78da35301af466a3) Co-authored-by: Raymond Hettinger files: A Misc/NEWS.d/next/Library/2019-08-11-10-34-19.bpo-37819.LVJls-.rst M Doc/library/fractions.rst M Lib/fractions.py M Lib/test/test_fractions.py diff --git a/Doc/library/fractions.rst b/Doc/library/fractions.rst index b5a818e1cafa..58e7126b0bf2 100644 --- a/Doc/library/fractions.rst +++ b/Doc/library/fractions.rst @@ -94,6 +94,13 @@ another rational number, or from a string. Denominator of the Fraction in lowest term. + .. method:: as_integer_ratio() + + Return a tuple of two integers, whose ratio is equal + to the Fraction and with a positive denominator. + + .. versionadded:: 3.8 + .. method:: from_float(flt) This class method constructs a :class:`Fraction` representing the exact diff --git a/Lib/fractions.py b/Lib/fractions.py index 7443bd3e0c6a..e774d58e4035 100644 --- a/Lib/fractions.py +++ b/Lib/fractions.py @@ -216,6 +216,14 @@ def from_decimal(cls, dec): (cls.__name__, dec, type(dec).__name__)) return cls(*dec.as_integer_ratio()) + def as_integer_ratio(self): + """Return the integer ratio as a tuple. + + Return a tuple of two integers, whose ratio is equal to the + Fraction and with a positive denominator. + """ + return (self._numerator, self._denominator) + def limit_denominator(self, max_denominator=1000000): """Closest Fraction to self with denominator at most max_denominator. diff --git a/Lib/test/test_fractions.py b/Lib/test/test_fractions.py index 277916220051..18ab28cfebe0 100644 --- a/Lib/test/test_fractions.py +++ b/Lib/test/test_fractions.py @@ -302,6 +302,12 @@ def testFromDecimal(self): ValueError, "cannot convert NaN to integer ratio", F.from_decimal, Decimal("snan")) + def test_as_integer_ratio(self): + self.assertEqual(F(4, 6).as_integer_ratio(), (2, 3)) + self.assertEqual(F(-4, 6).as_integer_ratio(), (-2, 3)) + self.assertEqual(F(4, -6).as_integer_ratio(), (-2, 3)) + self.assertEqual(F(0, 6).as_integer_ratio(), (0, 1)) + def testLimitDenominator(self): rpi = F('3.1415926535897932') self.assertEqual(rpi.limit_denominator(10000), F(355, 113)) diff --git a/Misc/NEWS.d/next/Library/2019-08-11-10-34-19.bpo-37819.LVJls-.rst b/Misc/NEWS.d/next/Library/2019-08-11-10-34-19.bpo-37819.LVJls-.rst new file mode 100644 index 000000000000..cfc1f1afb4f7 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-11-10-34-19.bpo-37819.LVJls-.rst @@ -0,0 +1,2 @@ +Add Fraction.as_integer_ratio() to match the corresponding methods in bool, +int, float, and decimal. From webhook-mailer at python.org Mon Aug 12 02:57:10 2019 From: webhook-mailer at python.org (Chris Withers) Date: Mon, 12 Aug 2019 06:57:10 -0000 Subject: [Python-checkins] Fix docs for assert_called and assert_called_once (#15197) Message-ID: https://github.com/python/cpython/commit/f9590edfeae192ba95aadaee9460dc03a366c51a commit: f9590edfeae192ba95aadaee9460dc03a366c51a branch: master author: Ismail S committer: Chris Withers date: 2019-08-12T01:57:03-05:00 summary: Fix docs for assert_called and assert_called_once (#15197) files: M Doc/library/unittest.mock.rst diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 46e8ef38ab11..19e9715102bf 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -286,7 +286,7 @@ the *new_callable* argument to :func:`patch`. used to set attributes on the mock after it is created. See the :meth:`configure_mock` method for details. - .. method:: assert_called(*args, **kwargs) + .. method:: assert_called() Assert that the mock was called at least once. @@ -297,7 +297,7 @@ the *new_callable* argument to :func:`patch`. .. versionadded:: 3.6 - .. method:: assert_called_once(*args, **kwargs) + .. method:: assert_called_once() Assert that the mock was called exactly once. From webhook-mailer at python.org Mon Aug 12 04:19:01 2019 From: webhook-mailer at python.org (Chris Withers) Date: Mon, 12 Aug 2019 08:19:01 -0000 Subject: [Python-checkins] Fix docs for assert_called and assert_called_once (GH-15218) Message-ID: https://github.com/python/cpython/commit/9286677538f3cd15aaad7628f4a95ab6aa97536b commit: 9286677538f3cd15aaad7628f4a95ab6aa97536b branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Chris Withers date: 2019-08-12T09:18:55+01:00 summary: Fix docs for assert_called and assert_called_once (GH-15218) (cherry picked from commit f9590edfeae192ba95aadaee9460dc03a366c51a) Co-authored-by: Ismail S files: M Doc/library/unittest.mock.rst diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index b76ae712a971..36cc0c2fc4bd 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -262,7 +262,7 @@ the *new_callable* argument to :func:`patch`. used to set attributes on the mock after it is created. See the :meth:`configure_mock` method for details. - .. method:: assert_called(*args, **kwargs) + .. method:: assert_called() Assert that the mock was called at least once. @@ -273,7 +273,7 @@ the *new_callable* argument to :func:`patch`. .. versionadded:: 3.6 - .. method:: assert_called_once(*args, **kwargs) + .. method:: assert_called_once() Assert that the mock was called exactly once. From webhook-mailer at python.org Mon Aug 12 04:19:51 2019 From: webhook-mailer at python.org (Chris Withers) Date: Mon, 12 Aug 2019 08:19:51 -0000 Subject: [Python-checkins] Fix docs for assert_called and assert_called_once (GH-15219) Message-ID: https://github.com/python/cpython/commit/2f087e279b94609073f630a86508b3a169c5e045 commit: 2f087e279b94609073f630a86508b3a169c5e045 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Chris Withers date: 2019-08-12T09:19:47+01:00 summary: Fix docs for assert_called and assert_called_once (GH-15219) (cherry picked from commit f9590edfeae192ba95aadaee9460dc03a366c51a) Co-authored-by: Ismail S files: M Doc/library/unittest.mock.rst diff --git a/Doc/library/unittest.mock.rst b/Doc/library/unittest.mock.rst index 46e8ef38ab11..19e9715102bf 100644 --- a/Doc/library/unittest.mock.rst +++ b/Doc/library/unittest.mock.rst @@ -286,7 +286,7 @@ the *new_callable* argument to :func:`patch`. used to set attributes on the mock after it is created. See the :meth:`configure_mock` method for details. - .. method:: assert_called(*args, **kwargs) + .. method:: assert_called() Assert that the mock was called at least once. @@ -297,7 +297,7 @@ the *new_callable* argument to :func:`patch`. .. versionadded:: 3.6 - .. method:: assert_called_once(*args, **kwargs) + .. method:: assert_called_once() Assert that the mock was called exactly once. From webhook-mailer at python.org Mon Aug 12 13:41:32 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 12 Aug 2019 17:41:32 -0000 Subject: [Python-checkins] bpo-37804: Remove the deprecated method threading.Thread.isAlive() (GH-15225) Message-ID: https://github.com/python/cpython/commit/44046fe4fc7f00a6eb855b33e6a3f953cf5233a5 commit: 44046fe4fc7f00a6eb855b33e6a3f953cf5233a5 branch: master author: Dong-hee Na committer: Victor Stinner date: 2019-08-12T19:41:08+02:00 summary: bpo-37804: Remove the deprecated method threading.Thread.isAlive() (GH-15225) files: A Misc/NEWS.d/next/Library/2019-08-12-23-07-47.bpo-37804.Ene6L-.rst M Doc/whatsnew/3.9.rst M Lib/test/test_threading.py M Lib/threading.py diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 61d9e745e87c..f09e09c2b905 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -186,6 +186,10 @@ Removed removed. They were deprecated since Python 3.7. (Contributed by Victor Stinner in :issue:`37320`.) +* The :meth:`~threading.Thread.isAlive()` method of :class:`threading.Thread` + has been removed. It was deprecated since Python 3.8. + Use :meth:`~threading.Thread.is_alive()` instead. + (Contributed by Dong-hee Na in :issue:`37804`.) Porting to Python 3.9 ===================== diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index 1466d25e9482..7c16974c1630 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -422,8 +422,6 @@ def test_old_threading_api(self): t.setDaemon(True) t.getName() t.setName("name") - with self.assertWarnsRegex(DeprecationWarning, 'use is_alive()'): - t.isAlive() e = threading.Event() e.isSet() threading.activeCount() diff --git a/Lib/threading.py b/Lib/threading.py index cec9cdb8e698..32a3d7c30336 100644 --- a/Lib/threading.py +++ b/Lib/threading.py @@ -1088,16 +1088,6 @@ def is_alive(self): self._wait_for_tstate_lock(False) return not self._is_stopped - def isAlive(self): - """Return whether the thread is alive. - - This method is deprecated, use is_alive() instead. - """ - import warnings - warnings.warn('isAlive() is deprecated, use is_alive() instead', - DeprecationWarning, stacklevel=2) - return self.is_alive() - @property def daemon(self): """A boolean value indicating whether this thread is a daemon thread. diff --git a/Misc/NEWS.d/next/Library/2019-08-12-23-07-47.bpo-37804.Ene6L-.rst b/Misc/NEWS.d/next/Library/2019-08-12-23-07-47.bpo-37804.Ene6L-.rst new file mode 100644 index 000000000000..ebbcb5aa7788 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-12-23-07-47.bpo-37804.Ene6L-.rst @@ -0,0 +1,2 @@ +Remove the deprecated method `threading.Thread.isAlive()`. Patch by Dong-hee +Na. From webhook-mailer at python.org Mon Aug 12 16:06:07 2019 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 12 Aug 2019 20:06:07 -0000 Subject: [Python-checkins] bpo-37354: Make Powershell Activate.ps1 script static to allow for signing (GH-14967) Message-ID: https://github.com/python/cpython/commit/732775d6be8062e72cf4995d5a9db0170e22c233 commit: 732775d6be8062e72cf4995d5a9db0170e22c233 branch: master author: Derek Keeler committer: Steve Dower date: 2019-08-12T13:06:02-07:00 summary: bpo-37354: Make Powershell Activate.ps1 script static to allow for signing (GH-14967) - Remove use of replacement text in the script - Make use of the pyvenv.cfg file for prompt value. - Add parameters to allow more flexibility - Make use of the current path, and assumptions about where env puts things, to compensate - Make the script a bit more 'idiomatic' Powershell - Add script documentation (Get-Help .\.venv\Scripts\Activate.ps1 shows PS help page now files: A Misc/NEWS.d/next/Library/2019-07-25-10-28-40.bpo-37354.RT3_3H.rst M Lib/venv/scripts/common/Activate.ps1 diff --git a/Lib/venv/scripts/common/Activate.ps1 b/Lib/venv/scripts/common/Activate.ps1 index de22962630aa..699c84097f1a 100644 --- a/Lib/venv/scripts/common/Activate.ps1 +++ b/Lib/venv/scripts/common/Activate.ps1 @@ -1,56 +1,231 @@ -function Script:add-bin([string]$envPath) { - $binPath = Join-Path -Path $env:VIRTUAL_ENV -ChildPath '__VENV_BIN_NAME__' - return ($binPath, $envPath) -join [IO.Path]::PathSeparator -} +<# +.Synopsis +Activate a Python virtual environment for the current Powershell session. + +.Description +Pushes the python executable for a virtual environment to the front of the +$Env:PATH environment variable and sets the prompt to signify that you are +in a Python virtual environment. Makes use of the command line switches as +well as the `pyvenv.cfg` file values present in the virtual environment. + +.Parameter VenvDir +Path to the directory that contains the virtual environment to activate. The +default value for this is the parent of the directory that the Activate.ps1 +script is located within. + +.Parameter Prompt +The prompt prefix to display when this virtual environment is activated. By +default, this prompt is the name of the virtual environment folder (VenvDir) +surrounded by parentheses and followed by a single space (ie. '(.venv) '). + +.Example +Activate.ps1 +Activates the Python virtual environment that contains the Activate.ps1 script. + +.Example +Activate.ps1 -Verbose +Activates the Python virtual environment that contains the Activate.ps1 script, +and shows extra information about the activation as it executes. + +.Example +Activate.ps1 -VenvDir C:\Users\MyUser\Common\.venv +Activates the Python virtual environment located in the specified location. + +.Example +Activate.ps1 -Prompt "MyPython" +Activates the Python virtual environment that contains the Activate.ps1 script, +and prefixes the current prompt with the specified string (surrounded in +parentheses) while the virtual environment is active. + + +#> +Param( + [Parameter(Mandatory = $false)] + [String] + $VenvDir, + [Parameter(Mandatory = $false)] + [String] + $Prompt +) + +<# Function declarations --------------------------------------------------- #> +<# +.Synopsis +Remove all shell session elements added by the Activate script, including the +addition of the virtual environment's Python executable from the beginning of +the PATH variable. + +.Parameter NonDestructive +If present, do not remove this function from the global namespace for the +session. + +#> function global:deactivate ([switch]$NonDestructive) { # Revert to original values - if (Test-Path function:_OLD_VIRTUAL_PROMPT) { - copy-item function:_OLD_VIRTUAL_PROMPT function:prompt - remove-item function:_OLD_VIRTUAL_PROMPT + + # The prior prompt: + if (Test-Path -Path Function:_OLD_VIRTUAL_PROMPT) { + Copy-Item -Path Function:_OLD_VIRTUAL_PROMPT -Destination Function:prompt + Remove-Item -Path Function:_OLD_VIRTUAL_PROMPT } - if (Test-Path env:_OLD_VIRTUAL_PYTHONHOME) { - copy-item env:_OLD_VIRTUAL_PYTHONHOME env:PYTHONHOME - remove-item env:_OLD_VIRTUAL_PYTHONHOME + # The prior PYTHONHOME: + if (Test-Path -Path Env:_OLD_VIRTUAL_PYTHONHOME) { + Copy-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME -Destination Env:PYTHONHOME + Remove-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME } - if (Test-Path env:_OLD_VIRTUAL_PATH) { - copy-item env:_OLD_VIRTUAL_PATH env:PATH - remove-item env:_OLD_VIRTUAL_PATH + # The prior PATH: + if (Test-Path -Path Env:_OLD_VIRTUAL_PATH) { + Copy-Item -Path Env:_OLD_VIRTUAL_PATH -Destination Env:PATH + Remove-Item -Path Env:_OLD_VIRTUAL_PATH } - if (Test-Path env:VIRTUAL_ENV) { - remove-item env:VIRTUAL_ENV + # Just remove the VIRTUAL_ENV altogether: + if (Test-Path -Path Env:VIRTUAL_ENV) { + Remove-Item -Path env:VIRTUAL_ENV + } + + # Just remove the _PYTHON_VENV_PROMPT_PREFIX altogether: + if (Get-Variable -Name "_PYTHON_VENV_PROMPT_PREFIX" -ErrorAction SilentlyContinue) { + Remove-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Scope Global -Force + } + + # Leave deactivate function in the global namespace if requested: + if (-not $NonDestructive) { + Remove-Item -Path function:deactivate + } +} + +<# +.Description +Get-PyVenvConfig parses the values from the pyvenv.cfg file located in the +given folder, and returns them in a map. + +For each line in the pyvenv.cfg file, if that line can be parsed into exactly +two strings separated by `=` (with any amount of whitespace surrounding the =) +then it is considered a `key = value` line. The left hand string is the key, +the right hand is the value. + +If the value starts with a `'` or a `"` then the first and last character is +stripped from the value before being captured. + +.Parameter ConfigDir +Path to the directory that contains the `pyvenv.cfg` file. +#> +function Get-PyVenvConfig( + [String] + $ConfigDir +) { + Write-Verbose "Given ConfigDir=$ConfigDir, obtain values in pyvenv.cfg" + + # Ensure the file exists, and issue a warning if it doesn't (but still allow the function to continue). + $pyvenvConfigPath = Join-Path -Resolve -Path $ConfigDir -ChildPath 'pyvenv.cfg' -ErrorAction Continue + + # An empty map will be returned if no config file is found. + $pyvenvConfig = @{ } + + if ($pyvenvConfigPath) { + + Write-Verbose "File exists, parse `key = value` lines" + $pyvenvConfigContent = Get-Content -Path $pyvenvConfigPath + + $pyvenvConfigContent | ForEach-Object { + $keyval = $PSItem -split "\s*=\s*", 2 + if ($keyval[0] -and $keyval[1]) { + $val = $keyval[1] + + # Remove extraneous quotations around a string value. + if ("'""".Contains($val.Substring(0,1))) { + $val = $val.Substring(1, $val.Length - 2) + } + + $pyvenvConfig[$keyval[0]] = $val + Write-Verbose "Adding Key: '$($keyval[0])'='$val'" + } + } } + return $pyvenvConfig +} + - if (!$NonDestructive) { - # Self destruct! - remove-item function:deactivate +<# Begin Activate script --------------------------------------------------- #> + +# Determine the containing directory of this script +$VenvExecPath = Split-Path -Parent $MyInvocation.MyCommand.Definition +$VenvExecDir = Get-Item -Path $VenvExecPath + +Write-Verbose "Activation script is located in path: '$VenvExecPath'" +Write-Verbose "VenvExecDir Fullname: '$($VenvExecDir.FullName)" +Write-Verbose "VenvExecDir Name: '$($VenvExecDir.Name)" + +# Set values required in priority: CmdLine, ConfigFile, Default +# First, get the location of the virtual environment, it might not be +# VenvExecDir if specified on the command line. +if ($VenvDir) { + Write-Verbose "VenvDir given as parameter, using '$VenvDir' to determine values" +} else { + Write-Verbose "VenvDir not given as a parameter, using parent directory name as VenvDir." + $VenvDir = $VenvExecDir.Parent.FullName.TrimEnd("\\/") + $VenvDir = $VenvDir.Insert($VenvDir.Length, "/") + Write-Verbose "VenvDir=$VenvDir" +} + +# Next, read the `pyvenv.cfg` file to determine any required value such +# as `prompt`. +$pyvenvCfg = Get-PyVenvConfig -ConfigDir $VenvDir + +# Next, set the prompt from the command line, or the config file, or +# just use the name of the virtual environment folder. +if ($Prompt) { + Write-Verbose "Prompt specified as argument, using '$Prompt'" +} else { + Write-Verbose "Prompt not specified as argument to script, checking pyvenv.cfg value" + if ($pyvenvCfg -and $pyvenvCfg['prompt']) { + Write-Verbose " Setting based on value in pyvenv.cfg='$($pyvenvCfg['prompt'])'" + $Prompt = $pyvenvCfg['prompt']; + } + else { + Write-Verbose " Setting prompt based on parent's directory's name. (Is the directory name passed to venv module when creating the virutal environment)" + Write-Verbose " Got leaf-name of $VenvDir='$(Split-Path -Path $venvDir -Leaf)'" + $Prompt = Split-Path -Path $venvDir -Leaf } } +Write-Verbose "Prompt = '$Prompt'" +Write-Verbose "VenvDir='$VenvDir'" + +# Deactivate any currently active virtual environment, but leave the +# deactivate function in place. deactivate -nondestructive -$env:VIRTUAL_ENV="__VENV_DIR__" +# Now set the environment variable VIRTUAL_ENV, used by many tools to determine +# that there is an activated venv. +$env:VIRTUAL_ENV = $VenvDir + +if (-not $Env:VIRTUAL_ENV_DISABLE_PROMPT) { + + Write-Verbose "Setting prompt to '$Prompt'" -if (! $env:VIRTUAL_ENV_DISABLE_PROMPT) { # Set the prompt to include the env name # Make sure _OLD_VIRTUAL_PROMPT is global - function global:_OLD_VIRTUAL_PROMPT {""} - copy-item function:prompt function:_OLD_VIRTUAL_PROMPT + function global:_OLD_VIRTUAL_PROMPT { "" } + Copy-Item -Path function:prompt -Destination function:_OLD_VIRTUAL_PROMPT + New-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Description "Python virtual environment prompt prefix" -Scope Global -Option ReadOnly -Visibility Public -Value $Prompt + function global:prompt { - Write-Host -NoNewline -ForegroundColor Green '__VENV_PROMPT__' + Write-Host -NoNewline -ForegroundColor Green "($_PYTHON_VENV_PROMPT_PREFIX) " _OLD_VIRTUAL_PROMPT } } # Clear PYTHONHOME -if (Test-Path env:PYTHONHOME) { - copy-item env:PYTHONHOME env:_OLD_VIRTUAL_PYTHONHOME - remove-item env:PYTHONHOME +if (Test-Path -Path Env:PYTHONHOME) { + Copy-Item -Path Env:PYTHONHOME -Destination Env:_OLD_VIRTUAL_PYTHONHOME + Remove-Item -Path Env:PYTHONHOME } # Add the venv to the PATH -copy-item env:PATH env:_OLD_VIRTUAL_PATH -$env:PATH = add-bin $env:PATH +Copy-Item -Path Env:PATH -Destination Env:_OLD_VIRTUAL_PATH +$Env:PATH = "$VenvExecDir$([System.IO.Path]::PathSeparator)$Env:PATH" diff --git a/Misc/NEWS.d/next/Library/2019-07-25-10-28-40.bpo-37354.RT3_3H.rst b/Misc/NEWS.d/next/Library/2019-07-25-10-28-40.bpo-37354.RT3_3H.rst new file mode 100644 index 000000000000..a314bcc9bf90 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-25-10-28-40.bpo-37354.RT3_3H.rst @@ -0,0 +1 @@ +Make Activate.ps1 Powershell script static to allow for signing it. From webhook-mailer at python.org Mon Aug 12 17:09:32 2019 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 12 Aug 2019 21:09:32 -0000 Subject: [Python-checkins] [3.8] bpo-37354: Make Powershell Activate.ps1 script static to allow for signing (GH-14967) Message-ID: https://github.com/python/cpython/commit/0c64b57e0155c333b7c96ec2af009c1388cd5d31 commit: 0c64b57e0155c333b7c96ec2af009c1388cd5d31 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: Steve Dower date: 2019-08-12T14:09:26-07:00 summary: [3.8] bpo-37354: Make Powershell Activate.ps1 script static to allow for signing (GH-14967) - Remove use of replacement text in the script - Make use of the pyvenv.cfg file for prompt value. - Add parameters to allow more flexibility - Make use of the current path, and assumptions about where env puts things, to compensate - Make the script a bit more 'idiomatic' Powershell - Add script documentation (Get-Help .\.venv\Scripts\Activate.ps1 shows PS help page now (cherry picked from commit 732775d6be8062e72cf4995d5a9db0170e22c233) Co-authored-by: Derek Keeler files: A Misc/NEWS.d/next/Library/2019-07-25-10-28-40.bpo-37354.RT3_3H.rst M Lib/venv/scripts/common/Activate.ps1 diff --git a/Lib/venv/scripts/common/Activate.ps1 b/Lib/venv/scripts/common/Activate.ps1 index de22962630aa..699c84097f1a 100644 --- a/Lib/venv/scripts/common/Activate.ps1 +++ b/Lib/venv/scripts/common/Activate.ps1 @@ -1,56 +1,231 @@ -function Script:add-bin([string]$envPath) { - $binPath = Join-Path -Path $env:VIRTUAL_ENV -ChildPath '__VENV_BIN_NAME__' - return ($binPath, $envPath) -join [IO.Path]::PathSeparator -} +<# +.Synopsis +Activate a Python virtual environment for the current Powershell session. + +.Description +Pushes the python executable for a virtual environment to the front of the +$Env:PATH environment variable and sets the prompt to signify that you are +in a Python virtual environment. Makes use of the command line switches as +well as the `pyvenv.cfg` file values present in the virtual environment. + +.Parameter VenvDir +Path to the directory that contains the virtual environment to activate. The +default value for this is the parent of the directory that the Activate.ps1 +script is located within. + +.Parameter Prompt +The prompt prefix to display when this virtual environment is activated. By +default, this prompt is the name of the virtual environment folder (VenvDir) +surrounded by parentheses and followed by a single space (ie. '(.venv) '). + +.Example +Activate.ps1 +Activates the Python virtual environment that contains the Activate.ps1 script. + +.Example +Activate.ps1 -Verbose +Activates the Python virtual environment that contains the Activate.ps1 script, +and shows extra information about the activation as it executes. + +.Example +Activate.ps1 -VenvDir C:\Users\MyUser\Common\.venv +Activates the Python virtual environment located in the specified location. + +.Example +Activate.ps1 -Prompt "MyPython" +Activates the Python virtual environment that contains the Activate.ps1 script, +and prefixes the current prompt with the specified string (surrounded in +parentheses) while the virtual environment is active. + + +#> +Param( + [Parameter(Mandatory = $false)] + [String] + $VenvDir, + [Parameter(Mandatory = $false)] + [String] + $Prompt +) + +<# Function declarations --------------------------------------------------- #> +<# +.Synopsis +Remove all shell session elements added by the Activate script, including the +addition of the virtual environment's Python executable from the beginning of +the PATH variable. + +.Parameter NonDestructive +If present, do not remove this function from the global namespace for the +session. + +#> function global:deactivate ([switch]$NonDestructive) { # Revert to original values - if (Test-Path function:_OLD_VIRTUAL_PROMPT) { - copy-item function:_OLD_VIRTUAL_PROMPT function:prompt - remove-item function:_OLD_VIRTUAL_PROMPT + + # The prior prompt: + if (Test-Path -Path Function:_OLD_VIRTUAL_PROMPT) { + Copy-Item -Path Function:_OLD_VIRTUAL_PROMPT -Destination Function:prompt + Remove-Item -Path Function:_OLD_VIRTUAL_PROMPT } - if (Test-Path env:_OLD_VIRTUAL_PYTHONHOME) { - copy-item env:_OLD_VIRTUAL_PYTHONHOME env:PYTHONHOME - remove-item env:_OLD_VIRTUAL_PYTHONHOME + # The prior PYTHONHOME: + if (Test-Path -Path Env:_OLD_VIRTUAL_PYTHONHOME) { + Copy-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME -Destination Env:PYTHONHOME + Remove-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME } - if (Test-Path env:_OLD_VIRTUAL_PATH) { - copy-item env:_OLD_VIRTUAL_PATH env:PATH - remove-item env:_OLD_VIRTUAL_PATH + # The prior PATH: + if (Test-Path -Path Env:_OLD_VIRTUAL_PATH) { + Copy-Item -Path Env:_OLD_VIRTUAL_PATH -Destination Env:PATH + Remove-Item -Path Env:_OLD_VIRTUAL_PATH } - if (Test-Path env:VIRTUAL_ENV) { - remove-item env:VIRTUAL_ENV + # Just remove the VIRTUAL_ENV altogether: + if (Test-Path -Path Env:VIRTUAL_ENV) { + Remove-Item -Path env:VIRTUAL_ENV + } + + # Just remove the _PYTHON_VENV_PROMPT_PREFIX altogether: + if (Get-Variable -Name "_PYTHON_VENV_PROMPT_PREFIX" -ErrorAction SilentlyContinue) { + Remove-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Scope Global -Force + } + + # Leave deactivate function in the global namespace if requested: + if (-not $NonDestructive) { + Remove-Item -Path function:deactivate + } +} + +<# +.Description +Get-PyVenvConfig parses the values from the pyvenv.cfg file located in the +given folder, and returns them in a map. + +For each line in the pyvenv.cfg file, if that line can be parsed into exactly +two strings separated by `=` (with any amount of whitespace surrounding the =) +then it is considered a `key = value` line. The left hand string is the key, +the right hand is the value. + +If the value starts with a `'` or a `"` then the first and last character is +stripped from the value before being captured. + +.Parameter ConfigDir +Path to the directory that contains the `pyvenv.cfg` file. +#> +function Get-PyVenvConfig( + [String] + $ConfigDir +) { + Write-Verbose "Given ConfigDir=$ConfigDir, obtain values in pyvenv.cfg" + + # Ensure the file exists, and issue a warning if it doesn't (but still allow the function to continue). + $pyvenvConfigPath = Join-Path -Resolve -Path $ConfigDir -ChildPath 'pyvenv.cfg' -ErrorAction Continue + + # An empty map will be returned if no config file is found. + $pyvenvConfig = @{ } + + if ($pyvenvConfigPath) { + + Write-Verbose "File exists, parse `key = value` lines" + $pyvenvConfigContent = Get-Content -Path $pyvenvConfigPath + + $pyvenvConfigContent | ForEach-Object { + $keyval = $PSItem -split "\s*=\s*", 2 + if ($keyval[0] -and $keyval[1]) { + $val = $keyval[1] + + # Remove extraneous quotations around a string value. + if ("'""".Contains($val.Substring(0,1))) { + $val = $val.Substring(1, $val.Length - 2) + } + + $pyvenvConfig[$keyval[0]] = $val + Write-Verbose "Adding Key: '$($keyval[0])'='$val'" + } + } } + return $pyvenvConfig +} + - if (!$NonDestructive) { - # Self destruct! - remove-item function:deactivate +<# Begin Activate script --------------------------------------------------- #> + +# Determine the containing directory of this script +$VenvExecPath = Split-Path -Parent $MyInvocation.MyCommand.Definition +$VenvExecDir = Get-Item -Path $VenvExecPath + +Write-Verbose "Activation script is located in path: '$VenvExecPath'" +Write-Verbose "VenvExecDir Fullname: '$($VenvExecDir.FullName)" +Write-Verbose "VenvExecDir Name: '$($VenvExecDir.Name)" + +# Set values required in priority: CmdLine, ConfigFile, Default +# First, get the location of the virtual environment, it might not be +# VenvExecDir if specified on the command line. +if ($VenvDir) { + Write-Verbose "VenvDir given as parameter, using '$VenvDir' to determine values" +} else { + Write-Verbose "VenvDir not given as a parameter, using parent directory name as VenvDir." + $VenvDir = $VenvExecDir.Parent.FullName.TrimEnd("\\/") + $VenvDir = $VenvDir.Insert($VenvDir.Length, "/") + Write-Verbose "VenvDir=$VenvDir" +} + +# Next, read the `pyvenv.cfg` file to determine any required value such +# as `prompt`. +$pyvenvCfg = Get-PyVenvConfig -ConfigDir $VenvDir + +# Next, set the prompt from the command line, or the config file, or +# just use the name of the virtual environment folder. +if ($Prompt) { + Write-Verbose "Prompt specified as argument, using '$Prompt'" +} else { + Write-Verbose "Prompt not specified as argument to script, checking pyvenv.cfg value" + if ($pyvenvCfg -and $pyvenvCfg['prompt']) { + Write-Verbose " Setting based on value in pyvenv.cfg='$($pyvenvCfg['prompt'])'" + $Prompt = $pyvenvCfg['prompt']; + } + else { + Write-Verbose " Setting prompt based on parent's directory's name. (Is the directory name passed to venv module when creating the virutal environment)" + Write-Verbose " Got leaf-name of $VenvDir='$(Split-Path -Path $venvDir -Leaf)'" + $Prompt = Split-Path -Path $venvDir -Leaf } } +Write-Verbose "Prompt = '$Prompt'" +Write-Verbose "VenvDir='$VenvDir'" + +# Deactivate any currently active virtual environment, but leave the +# deactivate function in place. deactivate -nondestructive -$env:VIRTUAL_ENV="__VENV_DIR__" +# Now set the environment variable VIRTUAL_ENV, used by many tools to determine +# that there is an activated venv. +$env:VIRTUAL_ENV = $VenvDir + +if (-not $Env:VIRTUAL_ENV_DISABLE_PROMPT) { + + Write-Verbose "Setting prompt to '$Prompt'" -if (! $env:VIRTUAL_ENV_DISABLE_PROMPT) { # Set the prompt to include the env name # Make sure _OLD_VIRTUAL_PROMPT is global - function global:_OLD_VIRTUAL_PROMPT {""} - copy-item function:prompt function:_OLD_VIRTUAL_PROMPT + function global:_OLD_VIRTUAL_PROMPT { "" } + Copy-Item -Path function:prompt -Destination function:_OLD_VIRTUAL_PROMPT + New-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Description "Python virtual environment prompt prefix" -Scope Global -Option ReadOnly -Visibility Public -Value $Prompt + function global:prompt { - Write-Host -NoNewline -ForegroundColor Green '__VENV_PROMPT__' + Write-Host -NoNewline -ForegroundColor Green "($_PYTHON_VENV_PROMPT_PREFIX) " _OLD_VIRTUAL_PROMPT } } # Clear PYTHONHOME -if (Test-Path env:PYTHONHOME) { - copy-item env:PYTHONHOME env:_OLD_VIRTUAL_PYTHONHOME - remove-item env:PYTHONHOME +if (Test-Path -Path Env:PYTHONHOME) { + Copy-Item -Path Env:PYTHONHOME -Destination Env:_OLD_VIRTUAL_PYTHONHOME + Remove-Item -Path Env:PYTHONHOME } # Add the venv to the PATH -copy-item env:PATH env:_OLD_VIRTUAL_PATH -$env:PATH = add-bin $env:PATH +Copy-Item -Path Env:PATH -Destination Env:_OLD_VIRTUAL_PATH +$Env:PATH = "$VenvExecDir$([System.IO.Path]::PathSeparator)$Env:PATH" diff --git a/Misc/NEWS.d/next/Library/2019-07-25-10-28-40.bpo-37354.RT3_3H.rst b/Misc/NEWS.d/next/Library/2019-07-25-10-28-40.bpo-37354.RT3_3H.rst new file mode 100644 index 000000000000..a314bcc9bf90 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-25-10-28-40.bpo-37354.RT3_3H.rst @@ -0,0 +1 @@ +Make Activate.ps1 Powershell script static to allow for signing it. From webhook-mailer at python.org Mon Aug 12 17:09:40 2019 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 12 Aug 2019 21:09:40 -0000 Subject: [Python-checkins] bpo-37354: Sign Activate.ps1 for release (GH-15235) Message-ID: https://github.com/python/cpython/commit/3e34a25a7a5c9ea2c46f2daeeb60f072faa5aaa1 commit: 3e34a25a7a5c9ea2c46f2daeeb60f072faa5aaa1 branch: master author: Steve Dower committer: GitHub date: 2019-08-12T14:09:36-07:00 summary: bpo-37354: Sign Activate.ps1 for release (GH-15235) files: M .azure-pipelines/windows-release/msi-steps.yml M .azure-pipelines/windows-release/stage-build.yml M .azure-pipelines/windows-release/stage-layout-full.yml M .azure-pipelines/windows-release/stage-layout-msix.yml M .azure-pipelines/windows-release/stage-layout-nuget.yml M .azure-pipelines/windows-release/stage-sign.yml diff --git a/.azure-pipelines/windows-release/msi-steps.yml b/.azure-pipelines/windows-release/msi-steps.yml index c55fa534eaec..f7bff162f8e0 100644 --- a/.azure-pipelines/windows-release/msi-steps.yml +++ b/.azure-pipelines/windows-release/msi-steps.yml @@ -51,6 +51,10 @@ steps: artifactName: tcltk_lib_amd64 targetPath: $(Build.BinariesDirectory)\tcltk_lib_amd64 + - powershell: | + copy $(Build.BinariesDirectory)\amd64\Activate.ps1 Lib\venv\scripts\common\Activate.ps1 -Force + displayName: 'Copy signed files into sources' + - script: | call Tools\msi\get_externals.bat call PCbuild\find_python.bat diff --git a/.azure-pipelines/windows-release/stage-build.yml b/.azure-pipelines/windows-release/stage-build.yml index ce7b38176935..c98576ef9705 100644 --- a/.azure-pipelines/windows-release/stage-build.yml +++ b/.azure-pipelines/windows-release/stage-build.yml @@ -122,7 +122,7 @@ jobs: displayName: Publish Tcl/Tk Library pool: - vmName: win2016-vs2017 + vmName: windows-latest workspace: clean: all diff --git a/.azure-pipelines/windows-release/stage-layout-full.yml b/.azure-pipelines/windows-release/stage-layout-full.yml index 8b412dffcc82..12c347239013 100644 --- a/.azure-pipelines/windows-release/stage-layout-full.yml +++ b/.azure-pipelines/windows-release/stage-layout-full.yml @@ -47,6 +47,10 @@ jobs: artifactName: tcltk_lib_$(Name) targetPath: $(Build.BinariesDirectory)\tcltk_lib + - powershell: | + copy $(Build.BinariesDirectory)\bin\Activate.ps1 Lib\venv\scripts\common\Activate.ps1 -Force + displayName: 'Copy signed files into sources' + - template: ./layout-command.yml - powershell: | diff --git a/.azure-pipelines/windows-release/stage-layout-msix.yml b/.azure-pipelines/windows-release/stage-layout-msix.yml index 7d66e8f9821c..ba86392f3ec6 100644 --- a/.azure-pipelines/windows-release/stage-layout-msix.yml +++ b/.azure-pipelines/windows-release/stage-layout-msix.yml @@ -40,6 +40,10 @@ jobs: artifactName: tcltk_lib_$(Name) targetPath: $(Build.BinariesDirectory)\tcltk_lib + - powershell: | + copy $(Build.BinariesDirectory)\bin\Activate.ps1 Lib\venv\scripts\common\Activate.ps1 -Force + displayName: 'Copy signed files into sources' + - template: ./layout-command.yml - powershell: | diff --git a/.azure-pipelines/windows-release/stage-layout-nuget.yml b/.azure-pipelines/windows-release/stage-layout-nuget.yml index 01512975e9db..7954c4547f50 100644 --- a/.azure-pipelines/windows-release/stage-layout-nuget.yml +++ b/.azure-pipelines/windows-release/stage-layout-nuget.yml @@ -29,6 +29,10 @@ jobs: artifactName: bin_$(Name) targetPath: $(Build.BinariesDirectory)\bin + - powershell: | + copy $(Build.BinariesDirectory)\bin\Activate.ps1 Lib\venv\scripts\common\Activate.ps1 -Force + displayName: 'Copy signed files into sources' + - template: ./layout-command.yml - powershell: | diff --git a/.azure-pipelines/windows-release/stage-sign.yml b/.azure-pipelines/windows-release/stage-sign.yml index d6984a0a137c..2307c6c9c8f9 100644 --- a/.azure-pipelines/windows-release/stage-sign.yml +++ b/.azure-pipelines/windows-release/stage-sign.yml @@ -1,3 +1,7 @@ +parameters: + Include: '*.exe, *.dll, *.pyd, *.cat, *.ps1' + Exclude: 'vcruntime*, libffi*, libcrypto*, libssl*' + jobs: - job: Sign_Python displayName: Sign Python binaries @@ -17,7 +21,7 @@ jobs: Name: amd64 steps: - - checkout: none + - template: ./checkout.yml - template: ./find-sdk.yml - powershell: | @@ -31,13 +35,18 @@ jobs: targetPath: $(Build.BinariesDirectory)\bin - powershell: | - $files = (gi *.exe, *.dll, *.pyd, *.cat -Exclude vcruntime*, libffi*, libcrypto*, libssl*) + copy "$(Build.SourcesDirectory)\Lib\venv\scripts\common\Activate.ps1" . + displayName: 'Copy files from source' + workingDirectory: $(Build.BinariesDirectory)\bin + + - powershell: | + $files = (gi ${{ parameters.Include }} -Exclude ${{ parameters.Exclude }}) signtool sign /a /n "$(SigningCertificate)" /fd sha256 /d "$(SigningDescription)" $files displayName: 'Sign binaries' workingDirectory: $(Build.BinariesDirectory)\bin - powershell: | - $files = (gi *.exe, *.dll, *.pyd, *.cat -Exclude vcruntime*, libffi*, libcrypto*, libssl*) + $files = (gi ${{ parameters.Include }} -Exclude ${{ parameters.Exclude }}) $failed = $true foreach ($retry in 1..10) { signtool timestamp /t http://timestamp.verisign.com/scripts/timestamp.dll $files From webhook-mailer at python.org Mon Aug 12 17:35:22 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 12 Aug 2019 21:35:22 -0000 Subject: [Python-checkins] bpo-37354: Sign Activate.ps1 for release (GH-15235) Message-ID: https://github.com/python/cpython/commit/2b98d8ec7ec3d41c6403ff9f6677a00ea0cb8b92 commit: 2b98d8ec7ec3d41c6403ff9f6677a00ea0cb8b92 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-12T14:35:16-07:00 summary: bpo-37354: Sign Activate.ps1 for release (GH-15235) (cherry picked from commit 3e34a25a7a5c9ea2c46f2daeeb60f072faa5aaa1) Co-authored-by: Steve Dower files: M .azure-pipelines/windows-release/msi-steps.yml M .azure-pipelines/windows-release/stage-build.yml M .azure-pipelines/windows-release/stage-layout-full.yml M .azure-pipelines/windows-release/stage-layout-msix.yml M .azure-pipelines/windows-release/stage-layout-nuget.yml M .azure-pipelines/windows-release/stage-sign.yml diff --git a/.azure-pipelines/windows-release/msi-steps.yml b/.azure-pipelines/windows-release/msi-steps.yml index c55fa534eaec..f7bff162f8e0 100644 --- a/.azure-pipelines/windows-release/msi-steps.yml +++ b/.azure-pipelines/windows-release/msi-steps.yml @@ -51,6 +51,10 @@ steps: artifactName: tcltk_lib_amd64 targetPath: $(Build.BinariesDirectory)\tcltk_lib_amd64 + - powershell: | + copy $(Build.BinariesDirectory)\amd64\Activate.ps1 Lib\venv\scripts\common\Activate.ps1 -Force + displayName: 'Copy signed files into sources' + - script: | call Tools\msi\get_externals.bat call PCbuild\find_python.bat diff --git a/.azure-pipelines/windows-release/stage-build.yml b/.azure-pipelines/windows-release/stage-build.yml index ce7b38176935..c98576ef9705 100644 --- a/.azure-pipelines/windows-release/stage-build.yml +++ b/.azure-pipelines/windows-release/stage-build.yml @@ -122,7 +122,7 @@ jobs: displayName: Publish Tcl/Tk Library pool: - vmName: win2016-vs2017 + vmName: windows-latest workspace: clean: all diff --git a/.azure-pipelines/windows-release/stage-layout-full.yml b/.azure-pipelines/windows-release/stage-layout-full.yml index 8b412dffcc82..12c347239013 100644 --- a/.azure-pipelines/windows-release/stage-layout-full.yml +++ b/.azure-pipelines/windows-release/stage-layout-full.yml @@ -47,6 +47,10 @@ jobs: artifactName: tcltk_lib_$(Name) targetPath: $(Build.BinariesDirectory)\tcltk_lib + - powershell: | + copy $(Build.BinariesDirectory)\bin\Activate.ps1 Lib\venv\scripts\common\Activate.ps1 -Force + displayName: 'Copy signed files into sources' + - template: ./layout-command.yml - powershell: | diff --git a/.azure-pipelines/windows-release/stage-layout-msix.yml b/.azure-pipelines/windows-release/stage-layout-msix.yml index 7d66e8f9821c..ba86392f3ec6 100644 --- a/.azure-pipelines/windows-release/stage-layout-msix.yml +++ b/.azure-pipelines/windows-release/stage-layout-msix.yml @@ -40,6 +40,10 @@ jobs: artifactName: tcltk_lib_$(Name) targetPath: $(Build.BinariesDirectory)\tcltk_lib + - powershell: | + copy $(Build.BinariesDirectory)\bin\Activate.ps1 Lib\venv\scripts\common\Activate.ps1 -Force + displayName: 'Copy signed files into sources' + - template: ./layout-command.yml - powershell: | diff --git a/.azure-pipelines/windows-release/stage-layout-nuget.yml b/.azure-pipelines/windows-release/stage-layout-nuget.yml index 01512975e9db..7954c4547f50 100644 --- a/.azure-pipelines/windows-release/stage-layout-nuget.yml +++ b/.azure-pipelines/windows-release/stage-layout-nuget.yml @@ -29,6 +29,10 @@ jobs: artifactName: bin_$(Name) targetPath: $(Build.BinariesDirectory)\bin + - powershell: | + copy $(Build.BinariesDirectory)\bin\Activate.ps1 Lib\venv\scripts\common\Activate.ps1 -Force + displayName: 'Copy signed files into sources' + - template: ./layout-command.yml - powershell: | diff --git a/.azure-pipelines/windows-release/stage-sign.yml b/.azure-pipelines/windows-release/stage-sign.yml index d6984a0a137c..2307c6c9c8f9 100644 --- a/.azure-pipelines/windows-release/stage-sign.yml +++ b/.azure-pipelines/windows-release/stage-sign.yml @@ -1,3 +1,7 @@ +parameters: + Include: '*.exe, *.dll, *.pyd, *.cat, *.ps1' + Exclude: 'vcruntime*, libffi*, libcrypto*, libssl*' + jobs: - job: Sign_Python displayName: Sign Python binaries @@ -17,7 +21,7 @@ jobs: Name: amd64 steps: - - checkout: none + - template: ./checkout.yml - template: ./find-sdk.yml - powershell: | @@ -31,13 +35,18 @@ jobs: targetPath: $(Build.BinariesDirectory)\bin - powershell: | - $files = (gi *.exe, *.dll, *.pyd, *.cat -Exclude vcruntime*, libffi*, libcrypto*, libssl*) + copy "$(Build.SourcesDirectory)\Lib\venv\scripts\common\Activate.ps1" . + displayName: 'Copy files from source' + workingDirectory: $(Build.BinariesDirectory)\bin + + - powershell: | + $files = (gi ${{ parameters.Include }} -Exclude ${{ parameters.Exclude }}) signtool sign /a /n "$(SigningCertificate)" /fd sha256 /d "$(SigningDescription)" $files displayName: 'Sign binaries' workingDirectory: $(Build.BinariesDirectory)\bin - powershell: | - $files = (gi *.exe, *.dll, *.pyd, *.cat -Exclude vcruntime*, libffi*, libcrypto*, libssl*) + $files = (gi ${{ parameters.Include }} -Exclude ${{ parameters.Exclude }}) $failed = $true foreach ($retry in 1..10) { signtool timestamp /t http://timestamp.verisign.com/scripts/timestamp.dll $files From webhook-mailer at python.org Mon Aug 12 17:56:44 2019 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 12 Aug 2019 21:56:44 -0000 Subject: [Python-checkins] Remove versioned executables from non-APPX packages (GH-15237) Message-ID: https://github.com/python/cpython/commit/c1aeb292d206e12b900dc4f7f816246c3a57c2ac commit: c1aeb292d206e12b900dc4f7f816246c3a57c2ac branch: master author: Steve Dower committer: GitHub date: 2019-08-12T14:56:39-07:00 summary: Remove versioned executables from non-APPX packages (GH-15237) files: M PC/layout/main.py diff --git a/PC/layout/main.py b/PC/layout/main.py index 07b7e6d57429..e59858196249 100644 --- a/PC/layout/main.py +++ b/PC/layout/main.py @@ -159,9 +159,6 @@ def in_build(f, dest="", new_name=None): yield from in_build("python_uwp.exe", new_name="python") yield from in_build("pythonw_uwp.exe", new_name="pythonw") else: - yield from in_build("python.exe", new_name="python{}".format(VER_DOT)) - yield from in_build("pythonw.exe", new_name="pythonw{}".format(VER_DOT)) - # For backwards compatibility, but we don't reference these ourselves. yield from in_build("python.exe", new_name="python") yield from in_build("pythonw.exe", new_name="pythonw") From webhook-mailer at python.org Mon Aug 12 18:16:46 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 12 Aug 2019 22:16:46 -0000 Subject: [Python-checkins] Remove versioned executables from non-APPX packages (GH-15237) Message-ID: https://github.com/python/cpython/commit/a150feeeb8efe99218e315c93f9ff1f158e18f10 commit: a150feeeb8efe99218e315c93f9ff1f158e18f10 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-12T15:16:36-07:00 summary: Remove versioned executables from non-APPX packages (GH-15237) (cherry picked from commit c1aeb292d206e12b900dc4f7f816246c3a57c2ac) Co-authored-by: Steve Dower files: M PC/layout/main.py diff --git a/PC/layout/main.py b/PC/layout/main.py index 07b7e6d57429..e59858196249 100644 --- a/PC/layout/main.py +++ b/PC/layout/main.py @@ -159,9 +159,6 @@ def in_build(f, dest="", new_name=None): yield from in_build("python_uwp.exe", new_name="python") yield from in_build("pythonw_uwp.exe", new_name="pythonw") else: - yield from in_build("python.exe", new_name="python{}".format(VER_DOT)) - yield from in_build("pythonw.exe", new_name="pythonw{}".format(VER_DOT)) - # For backwards compatibility, but we don't reference these ourselves. yield from in_build("python.exe", new_name="python") yield from in_build("pythonw.exe", new_name="pythonw") From webhook-mailer at python.org Mon Aug 12 18:55:24 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Mon, 12 Aug 2019 22:55:24 -0000 Subject: [Python-checkins] bpo-37759: Second round of edits to Whatsnew 3.8 (GH-15204) Message-ID: https://github.com/python/cpython/commit/66a34d35e4c97da9840a29ba9fba76721021c463 commit: 66a34d35e4c97da9840a29ba9fba76721021c463 branch: master author: Raymond Hettinger committer: GitHub date: 2019-08-12T15:55:18-07:00 summary: bpo-37759: Second round of edits to Whatsnew 3.8 (GH-15204) files: M Doc/whatsnew/3.8.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 82da10cc3be8..e8238251d6ea 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -39,7 +39,7 @@ module. (Contributed by P.Y. Developer in :issue:`12345`.) - This saves the maintainer the effort of going through the Mercurial log + This saves the maintainer the effort of going through the Git log when researching a change. :Editor: Raymond Hettinger @@ -59,6 +59,7 @@ notable items not yet covered are: from datetime import date from math import cos, radians + from unicodedata import normalize import re import math @@ -383,9 +384,13 @@ Other Language Changes was lifted. (Contributed by Serhiy Storchaka in :issue:`32489`.) -* The :class:`int` type now has a new :meth:`~int.as_integer_ratio` method - compatible with the existing :meth:`float.as_integer_ratio` method. - (Contributed by Lisa Roach in :issue:`33073`.) +* The :class:`bool`, :class:`int`, and :class:`fractions.Fraction` types + now have an :meth:`~int.as_integer_ratio` method like that found in + :class:`float` and :class:`decimal.Decimal`. This minor API extension + makes it possible to write ``numerator, denominator = + x.as_integer_ratio()`` and have it work across multiple numeric types. + (Contributed by Lisa Roach in :issue:`33073` and Raymond Hettinger in + :issue:`37819`.) * Constructors of :class:`int`, :class:`float` and :class:`complex` will now use the :meth:`~object.__index__` special method, if available and the @@ -410,19 +415,26 @@ Other Language Changes never intended to permit more than a bare name on the left-hand side of a keyword argument assignment term. See :issue:`34641`. -* Iterable unpacking is now allowed without parentheses in :keyword:`yield` - and :keyword:`return` statements. - (Contributed by David Cuthbert and Jordan Chapman in :issue:`32117`.) +* Generalized iterable unpacking in :keyword:`yield` and + :keyword:`return` statements no longer requires enclosing parentheses. + This brings the *yield* and *return* syntax into better agreement with + normal assignment syntax:: + + >>> def parse(family): + lastname, *members = family.split() + return lastname.upper(), *members -* The compiler now produces a :exc:`SyntaxWarning` in some cases when a comma - is missed before tuple or list. For example:: + >>> parse('simpsons homer marge bart lisa sally') + ('SIMPSONS', 'homer', 'marge', 'bart', 'lisa', 'sally') - data = [ - (1, 2, 3) # oops, missing comma! - (4, 5, 6) - ] - (Contributed by Serhiy Storchaka in :issue:`15248`.) + (Contributed by David Cuthbert and Jordan Chapman in :issue:`32117`.) + +* When a comma is missed in code such as ``[(10, 20) (30, 40)]``, the + compiler displays a :exc:`SyntaxWarning` with a helpful suggestion. + This improves on just having a :exc:`TypeError` indicating that the + first tuple was not callable. (Contributed by Serhiy Storchaka in + :issue:`15248`.) * Arithmetic operations between subclasses of :class:`datetime.date` or :class:`datetime.datetime` and :class:`datetime.timedelta` objects now return @@ -439,7 +451,25 @@ Other Language Changes and Windows use this to properly terminate scripts in interactive sessions. (Contributed by Google via Gregory P. Smith in :issue:`1054041`.) -* Added new ``replace()`` method to the code type (:class:`types.CodeType`). +* Some advanced styles of programming require updating the + :class:`types.CodeType` object for an existing function. Since code + objects are immutable, a new code object needs to be created, one + that is modeled on the existing code object. With 19 parameters, + this was somewhat tedious. Now, the new ``replace()`` method makes + it possible to create a clone with a few altered parameters. + + Here's an example that alters the :func:`statistics.mean` function to + prevent the *data* parameter from being used as a keyword argument:: + + >>> from statistics import mean + >>> mean(data=[10, 20, 90]) + 40 + >>> mean.__code__ = mean.__code__.replace(co_posonlyargcount=1) + >>> mean(data=[10, 20, 90]) + Traceback (most recent call last): + ... + TypeError: mean() got some positional-only arguments passed as keyword arguments: 'data' + (Contributed by Victor Stinner in :issue:`37032`.) * For integers, the three-argument form of the :func:`pow` function now @@ -468,17 +498,55 @@ Other Language Changes (Contributed by Mark Dickinson in :issue:`36027`.) -* When dictionary comprehensions are evaluated, the key is now evaluated before - the value, as proposed by :pep:`572`. +* Dict comprehensions have been synced-up with dict literals so that the + key is computed first and the value second:: + + >>> # Dict comprehension + >>> cast = {input('role? '): input('actor? ') for i in range(2)} + role? King Arthur + actor? Chapman + role? Black Knight + actor? Cleese + + >>> # Dict literal + >>> cast = {input('role? '): input('actor? ')} + role? Sir Robin + actor? Eric Idle + + The guaranteed execution order is helpful with assignment expressions + because variables assigned in the key expression will be available in + the value expression:: + + >>> names = ['Martin von L?wis', '?ukasz Langa', 'Walter D?rwald'] + >>> {(n := normalize('NFC', name)).casefold() : n for name in names} + {'martin von l?wis': 'Martin von L?wis', + '?ukasz langa': '?ukasz Langa', + 'walter d?rwald': 'Walter D?rwald'} New Modules =========== * The new :mod:`importlib.metadata` module provides (provisional) support for - reading metadata from third-party packages. For example, you can extract an - installed package's version number, list of entry points, and more. See - :issue:`34632` for additional details. + reading metadata from third-party packages. For example, it can extract an + installed package's version number, list of entry points, and more:: + + >>> # Note following example requires that the popular "requests" + >>> # package has been installed. + >>> + >>> from importlib.metadata import version, requires, files + >>> version('requests') + '2.22.0' + >>> list(requires('requests')) + ['chardet (<3.1.0,>=3.0.2)'] + >>> list(files('requests'))[:5] + [PackagePath('requests-2.22.0.dist-info/INSTALLER'), + PackagePath('requests-2.22.0.dist-info/LICENSE'), + PackagePath('requests-2.22.0.dist-info/METADATA'), + PackagePath('requests-2.22.0.dist-info/RECORD'), + PackagePath('requests-2.22.0.dist-info/WHEEL')] + + (Contributed in :issue:`34632` by Barry Warsaw and Jason R. Coombs.) Improved Modules From webhook-mailer at python.org Mon Aug 12 21:03:02 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Tue, 13 Aug 2019 01:03:02 -0000 Subject: [Python-checkins] bpo-37759: Second round of edits to Whatsnew 3.8 (GH-15204) (GH-15240) Message-ID: https://github.com/python/cpython/commit/9bedb8c9e60fab4f49ee12b95f8073ee9f577f81 commit: 9bedb8c9e60fab4f49ee12b95f8073ee9f577f81 branch: 3.8 author: Raymond Hettinger committer: GitHub date: 2019-08-12T18:02:58-07:00 summary: bpo-37759: Second round of edits to Whatsnew 3.8 (GH-15204) (GH-15240) (cherry picked from commit 66a34d35e4c97da9840a29ba9fba76721021c463) files: M Doc/whatsnew/3.8.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 83caa2cc5abc..e8238251d6ea 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -39,7 +39,7 @@ module. (Contributed by P.Y. Developer in :issue:`12345`.) - This saves the maintainer the effort of going through the Mercurial log + This saves the maintainer the effort of going through the Git log when researching a change. :Editor: Raymond Hettinger @@ -59,6 +59,7 @@ notable items not yet covered are: from datetime import date from math import cos, radians + from unicodedata import normalize import re import math @@ -383,9 +384,13 @@ Other Language Changes was lifted. (Contributed by Serhiy Storchaka in :issue:`32489`.) -* The :class:`int` type now has a new :meth:`~int.as_integer_ratio` method - compatible with the existing :meth:`float.as_integer_ratio` method. - (Contributed by Lisa Roach in :issue:`33073`.) +* The :class:`bool`, :class:`int`, and :class:`fractions.Fraction` types + now have an :meth:`~int.as_integer_ratio` method like that found in + :class:`float` and :class:`decimal.Decimal`. This minor API extension + makes it possible to write ``numerator, denominator = + x.as_integer_ratio()`` and have it work across multiple numeric types. + (Contributed by Lisa Roach in :issue:`33073` and Raymond Hettinger in + :issue:`37819`.) * Constructors of :class:`int`, :class:`float` and :class:`complex` will now use the :meth:`~object.__index__` special method, if available and the @@ -410,19 +415,26 @@ Other Language Changes never intended to permit more than a bare name on the left-hand side of a keyword argument assignment term. See :issue:`34641`. -* Iterable unpacking is now allowed without parentheses in :keyword:`yield` - and :keyword:`return` statements. - (Contributed by David Cuthbert and Jordan Chapman in :issue:`32117`.) +* Generalized iterable unpacking in :keyword:`yield` and + :keyword:`return` statements no longer requires enclosing parentheses. + This brings the *yield* and *return* syntax into better agreement with + normal assignment syntax:: + + >>> def parse(family): + lastname, *members = family.split() + return lastname.upper(), *members -* The compiler now produces a :exc:`SyntaxWarning` in some cases when a comma - is missed before tuple or list. For example:: + >>> parse('simpsons homer marge bart lisa sally') + ('SIMPSONS', 'homer', 'marge', 'bart', 'lisa', 'sally') - data = [ - (1, 2, 3) # oops, missing comma! - (4, 5, 6) - ] - (Contributed by Serhiy Storchaka in :issue:`15248`.) + (Contributed by David Cuthbert and Jordan Chapman in :issue:`32117`.) + +* When a comma is missed in code such as ``[(10, 20) (30, 40)]``, the + compiler displays a :exc:`SyntaxWarning` with a helpful suggestion. + This improves on just having a :exc:`TypeError` indicating that the + first tuple was not callable. (Contributed by Serhiy Storchaka in + :issue:`15248`.) * Arithmetic operations between subclasses of :class:`datetime.date` or :class:`datetime.datetime` and :class:`datetime.timedelta` objects now return @@ -439,7 +451,25 @@ Other Language Changes and Windows use this to properly terminate scripts in interactive sessions. (Contributed by Google via Gregory P. Smith in :issue:`1054041`.) -* Added new ``replace()`` method to the code type (:class:`types.CodeType`). +* Some advanced styles of programming require updating the + :class:`types.CodeType` object for an existing function. Since code + objects are immutable, a new code object needs to be created, one + that is modeled on the existing code object. With 19 parameters, + this was somewhat tedious. Now, the new ``replace()`` method makes + it possible to create a clone with a few altered parameters. + + Here's an example that alters the :func:`statistics.mean` function to + prevent the *data* parameter from being used as a keyword argument:: + + >>> from statistics import mean + >>> mean(data=[10, 20, 90]) + 40 + >>> mean.__code__ = mean.__code__.replace(co_posonlyargcount=1) + >>> mean(data=[10, 20, 90]) + Traceback (most recent call last): + ... + TypeError: mean() got some positional-only arguments passed as keyword arguments: 'data' + (Contributed by Victor Stinner in :issue:`37032`.) * For integers, the three-argument form of the :func:`pow` function now @@ -468,14 +498,55 @@ Other Language Changes (Contributed by Mark Dickinson in :issue:`36027`.) -* When dictionary comprehensions are evaluated, the key is now evaluated before - the value, as proposed by :pep:`572`. +* Dict comprehensions have been synced-up with dict literals so that the + key is computed first and the value second:: + + >>> # Dict comprehension + >>> cast = {input('role? '): input('actor? ') for i in range(2)} + role? King Arthur + actor? Chapman + role? Black Knight + actor? Cleese + + >>> # Dict literal + >>> cast = {input('role? '): input('actor? ')} + role? Sir Robin + actor? Eric Idle + + The guaranteed execution order is helpful with assignment expressions + because variables assigned in the key expression will be available in + the value expression:: + + >>> names = ['Martin von L?wis', '?ukasz Langa', 'Walter D?rwald'] + >>> {(n := normalize('NFC', name)).casefold() : n for name in names} + {'martin von l?wis': 'Martin von L?wis', + '?ukasz langa': '?ukasz Langa', + 'walter d?rwald': 'Walter D?rwald'} New Modules =========== -* None yet. +* The new :mod:`importlib.metadata` module provides (provisional) support for + reading metadata from third-party packages. For example, it can extract an + installed package's version number, list of entry points, and more:: + + >>> # Note following example requires that the popular "requests" + >>> # package has been installed. + >>> + >>> from importlib.metadata import version, requires, files + >>> version('requests') + '2.22.0' + >>> list(requires('requests')) + ['chardet (<3.1.0,>=3.0.2)'] + >>> list(files('requests'))[:5] + [PackagePath('requests-2.22.0.dist-info/INSTALLER'), + PackagePath('requests-2.22.0.dist-info/LICENSE'), + PackagePath('requests-2.22.0.dist-info/METADATA'), + PackagePath('requests-2.22.0.dist-info/RECORD'), + PackagePath('requests-2.22.0.dist-info/WHEEL')] + + (Contributed in :issue:`34632` by Barry Warsaw and Jason R. Coombs.) Improved Modules From webhook-mailer at python.org Tue Aug 13 01:21:07 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 13 Aug 2019 05:21:07 -0000 Subject: [Python-checkins] bpo-37760: Factor out the basic UCD parsing logic of makeunicodedata. (GH-15130) Message-ID: https://github.com/python/cpython/commit/ef2af1ad44be0542a47270d5173a0b920c3a450d commit: ef2af1ad44be0542a47270d5173a0b920c3a450d branch: master author: Greg Price committer: Benjamin Peterson date: 2019-08-12T22:20:56-07:00 summary: bpo-37760: Factor out the basic UCD parsing logic of makeunicodedata. (GH-15130) There were 10 copies of this, and almost as many distinct versions of exactly how it was written. They're all implementing the same standard. Pull them out to the top, so the more interesting logic that remains becomes easier to read. files: M Tools/unicode/makeunicodedata.py diff --git a/Tools/unicode/makeunicodedata.py b/Tools/unicode/makeunicodedata.py index 5b9427acd390..1be93ec479c8 100644 --- a/Tools/unicode/makeunicodedata.py +++ b/Tools/unicode/makeunicodedata.py @@ -30,8 +30,9 @@ import sys import zipfile -from textwrap import dedent from functools import partial +from textwrap import dedent +from typing import * SCRIPT = sys.argv[0] VERSION = "3.3" @@ -903,6 +904,32 @@ def open_data(template, version): return open(local, 'rb') +class UcdFile: + ''' + A file in the standard format of the UCD. + + See: https://www.unicode.org/reports/tr44/#Format_Conventions + + Note that, as described there, the Unihan data files have their + own separate format. + ''' + + def __init__(self, template: str, version: str) -> None: + self.template = template + self.version = version + + def records(self) -> Iterator[List[str]]: + with open_data(self.template, self.version) as file: + for line in file: + line = line.split('#', 1)[0].strip() + if not line: + continue + yield [field.strip() for field in line.split(';')] + + def __iter__(self) -> Iterator[List[str]]: + return self.records() + + # -------------------------------------------------------------------- # the following support code is taken from the unidb utilities # Copyright (c) 1999-2000 by Secret Labs AB @@ -922,14 +949,9 @@ def __init__(self, version, cjk_check=True): self.changed = [] table = [None] * 0x110000 - with open_data(UNICODE_DATA, version) as file: - while 1: - s = file.readline() - if not s: - break - s = s.strip().split(";") - char = int(s[0], 16) - table[char] = s + for s in UcdFile(UNICODE_DATA, version): + char = int(s[0], 16) + table[char] = s cjk_ranges_found = [] @@ -968,17 +990,12 @@ def __init__(self, version, # in order to take advantage of the compression and lookup # algorithms used for the other characters pua_index = NAME_ALIASES_START - with open_data(NAME_ALIASES, version) as file: - for s in file: - s = s.strip() - if not s or s.startswith('#'): - continue - char, name, abbrev = s.split(';') - char = int(char, 16) - self.aliases.append((name, char)) - # also store the name in the PUA 1 - self.table[pua_index][1] = name - pua_index += 1 + for char, name, abbrev in UcdFile(NAME_ALIASES, version): + char = int(char, 16) + self.aliases.append((name, char)) + # also store the name in the PUA 1 + self.table[pua_index][1] = name + pua_index += 1 assert pua_index - NAME_ALIASES_START == len(self.aliases) self.named_sequences = [] @@ -988,50 +1005,32 @@ def __init__(self, version, assert pua_index < NAMED_SEQUENCES_START pua_index = NAMED_SEQUENCES_START - with open_data(NAMED_SEQUENCES, version) as file: - for s in file: - s = s.strip() - if not s or s.startswith('#'): - continue - name, chars = s.split(';') - chars = tuple(int(char, 16) for char in chars.split()) - # check that the structure defined in makeunicodename is OK - assert 2 <= len(chars) <= 4, "change the Py_UCS2 array size" - assert all(c <= 0xFFFF for c in chars), ("use Py_UCS4 in " - "the NamedSequence struct and in unicodedata_lookup") - self.named_sequences.append((name, chars)) - # also store these in the PUA 1 - self.table[pua_index][1] = name - pua_index += 1 + for name, chars in UcdFile(NAMED_SEQUENCES, version): + chars = tuple(int(char, 16) for char in chars.split()) + # check that the structure defined in makeunicodename is OK + assert 2 <= len(chars) <= 4, "change the Py_UCS2 array size" + assert all(c <= 0xFFFF for c in chars), ("use Py_UCS4 in " + "the NamedSequence struct and in unicodedata_lookup") + self.named_sequences.append((name, chars)) + # also store these in the PUA 1 + self.table[pua_index][1] = name + pua_index += 1 assert pua_index - NAMED_SEQUENCES_START == len(self.named_sequences) self.exclusions = {} - with open_data(COMPOSITION_EXCLUSIONS, version) as file: - for s in file: - s = s.strip() - if not s: - continue - if s[0] == '#': - continue - char = int(s.split()[0],16) - self.exclusions[char] = 1 + for char, in UcdFile(COMPOSITION_EXCLUSIONS, version): + char = int(char, 16) + self.exclusions[char] = 1 widths = [None] * 0x110000 - with open_data(EASTASIAN_WIDTH, version) as file: - for s in file: - s = s.strip() - if not s: - continue - if s[0] == '#': - continue - s = s.split()[0].split(';') - if '..' in s[0]: - first, last = [int(c, 16) for c in s[0].split('..')] - chars = list(range(first, last+1)) - else: - chars = [int(s[0], 16)] - for char in chars: - widths[char] = s[1] + for s in UcdFile(EASTASIAN_WIDTH, version): + if '..' in s[0]: + first, last = [int(c, 16) for c in s[0].split('..')] + chars = list(range(first, last+1)) + else: + chars = [int(s[0], 16)] + for char in chars: + widths[char] = s[1] for i in range(0, 0x110000): if table[i] is not None: @@ -1041,38 +1040,27 @@ def __init__(self, version, if table[i] is not None: table[i].append(set()) - with open_data(DERIVED_CORE_PROPERTIES, version) as file: - for s in file: - s = s.split('#', 1)[0].strip() - if not s: - continue - - r, p = s.split(";") - r = r.strip() - p = p.strip() - if ".." in r: - first, last = [int(c, 16) for c in r.split('..')] - chars = list(range(first, last+1)) - else: - chars = [int(r, 16)] - for char in chars: - if table[char]: - # Some properties (e.g. Default_Ignorable_Code_Point) - # apply to unassigned code points; ignore them - table[char][-1].add(p) - - with open_data(LINE_BREAK, version) as file: - for s in file: - s = s.partition('#')[0] - s = [i.strip() for i in s.split(';')] - if len(s) < 2 or s[1] not in MANDATORY_LINE_BREAKS: - continue - if '..' not in s[0]: - first = last = int(s[0], 16) - else: - first, last = [int(c, 16) for c in s[0].split('..')] - for char in range(first, last+1): - table[char][-1].add('Line_Break') + for r, p in UcdFile(DERIVED_CORE_PROPERTIES, version): + if ".." in r: + first, last = [int(c, 16) for c in r.split('..')] + chars = list(range(first, last+1)) + else: + chars = [int(r, 16)] + for char in chars: + if table[char]: + # Some properties (e.g. Default_Ignorable_Code_Point) + # apply to unassigned code points; ignore them + table[char][-1].add(p) + + for s in UcdFile(LINE_BREAK, version): + if len(s) < 2 or s[1] not in MANDATORY_LINE_BREAKS: + continue + if '..' not in s[0]: + first = last = int(s[0], 16) + else: + first, last = [int(c, 16) for c in s[0].split('..')] + for char in range(first, last+1): + table[char][-1].add('Line_Break') # We only want the quickcheck properties # Format: NF?_QC; Y(es)/N(o)/M(aybe) @@ -1083,23 +1071,19 @@ def __init__(self, version, # for older versions, and no delta records will be created. quickchecks = [0] * 0x110000 qc_order = 'NFD_QC NFKD_QC NFC_QC NFKC_QC'.split() - with open_data(DERIVEDNORMALIZATION_PROPS, version) as file: - for s in file: - if '#' in s: - s = s[:s.index('#')] - s = [i.strip() for i in s.split(';')] - if len(s) < 2 or s[1] not in qc_order: - continue - quickcheck = 'MN'.index(s[2]) + 1 # Maybe or No - quickcheck_shift = qc_order.index(s[1])*2 - quickcheck <<= quickcheck_shift - if '..' not in s[0]: - first = last = int(s[0], 16) - else: - first, last = [int(c, 16) for c in s[0].split('..')] - for char in range(first, last+1): - assert not (quickchecks[char]>>quickcheck_shift)&3 - quickchecks[char] |= quickcheck + for s in UcdFile(DERIVEDNORMALIZATION_PROPS, version): + if len(s) < 2 or s[1] not in qc_order: + continue + quickcheck = 'MN'.index(s[2]) + 1 # Maybe or No + quickcheck_shift = qc_order.index(s[1])*2 + quickcheck <<= quickcheck_shift + if '..' not in s[0]: + first = last = int(s[0], 16) + else: + first, last = [int(c, 16) for c in s[0].split('..')] + for char in range(first, last+1): + assert not (quickchecks[char]>>quickcheck_shift)&3 + quickchecks[char] |= quickcheck for i in range(0, 0x110000): if table[i] is not None: table[i].append(quickchecks[i]) @@ -1122,34 +1106,26 @@ def __init__(self, version, # Patch the numeric field if table[i] is not None: table[i][8] = value + sc = self.special_casing = {} - with open_data(SPECIAL_CASING, version) as file: - for s in file: - s = s[:-1].split('#', 1)[0] - if not s: - continue - data = s.split("; ") - if data[4]: - # We ignore all conditionals (since they depend on - # languages) except for one, which is hardcoded. See - # handle_capital_sigma in unicodeobject.c. - continue - c = int(data[0], 16) - lower = [int(char, 16) for char in data[1].split()] - title = [int(char, 16) for char in data[2].split()] - upper = [int(char, 16) for char in data[3].split()] - sc[c] = (lower, title, upper) + for data in UcdFile(SPECIAL_CASING, version): + if data[4]: + # We ignore all conditionals (since they depend on + # languages) except for one, which is hardcoded. See + # handle_capital_sigma in unicodeobject.c. + continue + c = int(data[0], 16) + lower = [int(char, 16) for char in data[1].split()] + title = [int(char, 16) for char in data[2].split()] + upper = [int(char, 16) for char in data[3].split()] + sc[c] = (lower, title, upper) + cf = self.case_folding = {} if version != '3.2.0': - with open_data(CASE_FOLDING, version) as file: - for s in file: - s = s[:-1].split('#', 1)[0] - if not s: - continue - data = s.split("; ") - if data[1] in "CF": - c = int(data[0], 16) - cf[c] = [int(char, 16) for char in data[2].split()] + for data in UcdFile(CASE_FOLDING, version): + if data[1] in "CF": + c = int(data[0], 16) + cf[c] = [int(char, 16) for char in data[2].split()] def uselatin1(self): # restrict character range to ISO Latin 1 From webhook-mailer at python.org Tue Aug 13 01:23:46 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 13 Aug 2019 05:23:46 -0000 Subject: [Python-checkins] bpo-37760: Mark all generated Unicode data headers as generated. (GH-15171) Message-ID: https://github.com/python/cpython/commit/4e3dfcc4b987e683476a1b16456e57d3c9f581cb commit: 4e3dfcc4b987e683476a1b16456e57d3c9f581cb branch: master author: Greg Price committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-12T22:23:41-07:00 summary: bpo-37760: Mark all generated Unicode data headers as generated. (GH-15171) This causes them to be collapsed by default in diffs shown on GitHub. https://bugs.python.org/issue37760 Automerge-Triggered-By: @benjaminp files: M .gitattributes diff --git a/.gitattributes b/.gitattributes index c9a54fbd472e..bec16a08152e 100644 --- a/.gitattributes +++ b/.gitattributes @@ -54,7 +54,7 @@ Python/Python-ast.c linguist-generated=true Include/opcode.h linguist-generated=true Python/opcode_targets.h linguist-generated=true Objects/typeslots.inc linguist-generated=true -Modules/unicodedata_db.h linguist-generated=true +*_db.h linguist-generated=true Doc/library/token-list.inc linguist-generated=true Include/token.h linguist-generated=true Lib/token.py linguist-generated=true From webhook-mailer at python.org Tue Aug 13 01:58:05 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 13 Aug 2019 05:58:05 -0000 Subject: [Python-checkins] bpo-37758: Clean out vestigial script-bits from test_unicodedata. (GH-15126) Message-ID: https://github.com/python/cpython/commit/def97c988be8340f33869b57942a30d10fc3a1f9 commit: def97c988be8340f33869b57942a30d10fc3a1f9 branch: master author: Greg Price committer: Benjamin Peterson date: 2019-08-12T22:58:01-07:00 summary: bpo-37758: Clean out vestigial script-bits from test_unicodedata. (GH-15126) This file started life as a script, before conversion to a `unittest` test file. Clear out some legacies of that conversion that are a bit confusing about how it works. Most notably, it's unlikely there's still a good reason to try to recover from `unicodedata` failing to import -- as there was when that logic was first added, when the module was very new. So take that out entirely. Keep `self.db` working, though, to avoid a noisy diff. files: M Lib/test/test_unicodedata.py diff --git a/Lib/test/test_unicodedata.py b/Lib/test/test_unicodedata.py index a52b6de547fb..7bc196be362b 100644 --- a/Lib/test/test_unicodedata.py +++ b/Lib/test/test_unicodedata.py @@ -1,4 +1,4 @@ -""" Test script for the unicodedata module. +""" Tests for the unicodedata module. Written by Marc-Andre Lemburg (mal at lemburg.com). @@ -6,16 +6,12 @@ """ +import hashlib import sys +import unicodedata import unittest -import hashlib from test.support import script_helper -encoding = 'utf-8' -errors = 'surrogatepass' - - -### Run tests class UnicodeMethodsTest(unittest.TestCase): @@ -61,20 +57,12 @@ def test_method_checksum(self): (char + 'ABC').title(), ] - h.update(''.join(data).encode(encoding, errors)) + h.update(''.join(data).encode('utf-8', 'surrogatepass')) result = h.hexdigest() self.assertEqual(result, self.expectedchecksum) class UnicodeDatabaseTest(unittest.TestCase): - - def setUp(self): - # In case unicodedata is not available, this will raise an ImportError, - # but the other test cases will still be run - import unicodedata - self.db = unicodedata - - def tearDown(self): - del self.db + db = unicodedata class UnicodeFunctionsTest(UnicodeDatabaseTest): From webhook-mailer at python.org Tue Aug 13 01:59:34 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 13 Aug 2019 05:59:34 -0000 Subject: [Python-checkins] bpo-37760: Constant-fold some old options in makeunicodedata. (GH-15129) Message-ID: https://github.com/python/cpython/commit/99d208efed97e02d813e8166925b998bbd0d3993 commit: 99d208efed97e02d813e8166925b998bbd0d3993 branch: master author: Greg Price committer: Benjamin Peterson date: 2019-08-12T22:59:30-07:00 summary: bpo-37760: Constant-fold some old options in makeunicodedata. (GH-15129) The `expand` option was introduced in 2000 in commit fad27aee1. It appears to have been always set since it was committed, and what it does is tell the code to do something essential. So, just always do that, and cut the option. Also cut the `linebreakprops` option, which isn't consulted anymore. files: M Tools/unicode/makeunicodedata.py diff --git a/Tools/unicode/makeunicodedata.py b/Tools/unicode/makeunicodedata.py index 1be93ec479c8..38c1f19a6567 100644 --- a/Tools/unicode/makeunicodedata.py +++ b/Tools/unicode/makeunicodedata.py @@ -943,10 +943,7 @@ class UnicodeData: # ISO-comment, uppercase, lowercase, titlecase, ea-width, (16) # derived-props] (17) - def __init__(self, version, - linebreakprops=False, - expand=1, - cjk_check=True): + def __init__(self, version, cjk_check=True): self.changed = [] table = [None] * 0x110000 for s in UcdFile(UNICODE_DATA, version): @@ -956,26 +953,25 @@ def __init__(self, version, cjk_ranges_found = [] # expand first-last ranges - if expand: - field = None - for i in range(0, 0x110000): - s = table[i] - if s: - if s[1][-6:] == "First>": - s[1] = "" - field = s - elif s[1][-5:] == "Last>": - if s[1].startswith("": + s[1] = "" + field = s + elif s[1][-5:] == "Last>": + if s[1].startswith(" https://github.com/python/cpython/commit/c2b9d9f202e4a99fc0800b7a0f0944ac4c2382e3 commit: c2b9d9f202e4a99fc0800b7a0f0944ac4c2382e3 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-12T23:04:21-07:00 summary: bpo-37760: Mark all generated Unicode data headers as generated. (GH-15171) This causes them to be collapsed by default in diffs shown on GitHub. https://bugs.python.org/issue37760 Automerge-Triggered-By: @benjaminp (cherry picked from commit 4e3dfcc4b987e683476a1b16456e57d3c9f581cb) Co-authored-by: Greg Price files: M .gitattributes diff --git a/.gitattributes b/.gitattributes index c9a54fbd472e..bec16a08152e 100644 --- a/.gitattributes +++ b/.gitattributes @@ -54,7 +54,7 @@ Python/Python-ast.c linguist-generated=true Include/opcode.h linguist-generated=true Python/opcode_targets.h linguist-generated=true Objects/typeslots.inc linguist-generated=true -Modules/unicodedata_db.h linguist-generated=true +*_db.h linguist-generated=true Doc/library/token-list.inc linguist-generated=true Include/token.h linguist-generated=true Lib/token.py linguist-generated=true From webhook-mailer at python.org Tue Aug 13 02:06:20 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 13 Aug 2019 06:06:20 -0000 Subject: [Python-checkins] bpo-37760: Mark all generated Unicode data headers as generated. (GH-15171) Message-ID: https://github.com/python/cpython/commit/b02e148a0d6e7a11df93a09ea5f4e1b0ad9b77b8 commit: b02e148a0d6e7a11df93a09ea5f4e1b0ad9b77b8 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-12T23:06:16-07:00 summary: bpo-37760: Mark all generated Unicode data headers as generated. (GH-15171) This causes them to be collapsed by default in diffs shown on GitHub. https://bugs.python.org/issue37760 Automerge-Triggered-By: @benjaminp (cherry picked from commit 4e3dfcc4b987e683476a1b16456e57d3c9f581cb) Co-authored-by: Greg Price files: From webhook-mailer at python.org Tue Aug 13 13:34:12 2019 From: webhook-mailer at python.org (Steve Dower) Date: Tue, 13 Aug 2019 17:34:12 -0000 Subject: [Python-checkins] bpo-37841: Remove python_uwp dependency on msvcp140.dll (GH-15253) Message-ID: https://github.com/python/cpython/commit/b0dace3e979381426385c551b116d0f1434096ee commit: b0dace3e979381426385c551b116d0f1434096ee branch: master author: Steve Dower committer: GitHub date: 2019-08-13T10:34:07-07:00 summary: bpo-37841: Remove python_uwp dependency on msvcp140.dll (GH-15253) files: M PCbuild/python_uwp.vcxproj diff --git a/PCbuild/python_uwp.vcxproj b/PCbuild/python_uwp.vcxproj index af187dd4df30..14e138cbed3e 100644 --- a/PCbuild/python_uwp.vcxproj +++ b/PCbuild/python_uwp.vcxproj @@ -65,6 +65,15 @@ Console + + + Multithreaded + + + ucrt.lib;%(AdditionalDependencies) + libucrt;%(IgnoreSpecificDefaultLibraries) + + From webhook-mailer at python.org Tue Aug 13 13:38:08 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 13 Aug 2019 17:38:08 -0000 Subject: [Python-checkins] bpo-37583: Add err 113 to support.get_socket_conn_refused_errs() (GH-14729) Message-ID: https://github.com/python/cpython/commit/1ac2a83f30312976502fda042db5ce18d10ceec2 commit: 1ac2a83f30312976502fda042db5ce18d10ceec2 branch: master author: Hai Shi committer: Victor Stinner date: 2019-08-13T19:37:59+02:00 summary: bpo-37583: Add err 113 to support.get_socket_conn_refused_errs() (GH-14729) Add error number 113 EHOSTUNREACH to get_socket_conn_refused_errs() of test.support. files: M Lib/test/support/__init__.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 00e734e7059f..46d646fe5b76 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -1497,6 +1497,9 @@ def get_socket_conn_refused_errs(): # bpo-31910: socket.create_connection() fails randomly # with EADDRNOTAVAIL on Travis CI errors.append(errno.EADDRNOTAVAIL) + if hasattr(errno, 'EHOSTUNREACH'): + # bpo-37583: The destination host cannot be reached + errors.append(errno.EHOSTUNREACH) if not IPV6_ENABLED: errors.append(errno.EAFNOSUPPORT) return errors From webhook-mailer at python.org Tue Aug 13 13:54:32 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 13 Aug 2019 17:54:32 -0000 Subject: [Python-checkins] bpo-37841: Remove python_uwp dependency on msvcp140.dll (GH-15253) Message-ID: https://github.com/python/cpython/commit/853eecc7692503fec8240fd9a74d9f88e0392630 commit: 853eecc7692503fec8240fd9a74d9f88e0392630 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-13T10:54:28-07:00 summary: bpo-37841: Remove python_uwp dependency on msvcp140.dll (GH-15253) (cherry picked from commit b0dace3e979381426385c551b116d0f1434096ee) Co-authored-by: Steve Dower files: M PCbuild/python_uwp.vcxproj diff --git a/PCbuild/python_uwp.vcxproj b/PCbuild/python_uwp.vcxproj index af187dd4df30..14e138cbed3e 100644 --- a/PCbuild/python_uwp.vcxproj +++ b/PCbuild/python_uwp.vcxproj @@ -65,6 +65,15 @@ Console + + + Multithreaded + + + ucrt.lib;%(AdditionalDependencies) + libucrt;%(IgnoreSpecificDefaultLibraries) + + From webhook-mailer at python.org Tue Aug 13 14:11:54 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 13 Aug 2019 18:11:54 -0000 Subject: [Python-checkins] bpo-37583: Add err 113 to support.get_socket_conn_refused_errs() (GH-14729) Message-ID: https://github.com/python/cpython/commit/ee989512528d178d6f088916aba3e67ea9487ceb commit: ee989512528d178d6f088916aba3e67ea9487ceb branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-13T11:11:49-07:00 summary: bpo-37583: Add err 113 to support.get_socket_conn_refused_errs() (GH-14729) Add error number 113 EHOSTUNREACH to get_socket_conn_refused_errs() of test.support. (cherry picked from commit 1ac2a83f30312976502fda042db5ce18d10ceec2) Co-authored-by: Hai Shi files: M Lib/test/support/__init__.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index e73c65bb9f34..e4e0481d336f 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -1495,6 +1495,9 @@ def get_socket_conn_refused_errs(): # bpo-31910: socket.create_connection() fails randomly # with EADDRNOTAVAIL on Travis CI errors.append(errno.EADDRNOTAVAIL) + if hasattr(errno, 'EHOSTUNREACH'): + # bpo-37583: The destination host cannot be reached + errors.append(errno.EHOSTUNREACH) if not IPV6_ENABLED: errors.append(errno.EAFNOSUPPORT) return errors From webhook-mailer at python.org Tue Aug 13 15:05:18 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 13 Aug 2019 19:05:18 -0000 Subject: [Python-checkins] bpo-37814: Document the empty tuple type annotation syntax (GH-15208) Message-ID: https://github.com/python/cpython/commit/8a784af750fa82c8355903309e5089eb2b60c16b commit: 8a784af750fa82c8355903309e5089eb2b60c16b branch: master author: Josh Holland committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-13T12:05:09-07:00 summary: bpo-37814: Document the empty tuple type annotation syntax (GH-15208) https://bugs.python.org/issue37814: > The empty tuple syntax in type annotations, `Tuple[()]`, is not obvious from the examples given in the documentation (I naively expected `Tuple[]` to work); it has been documented in PEP 484 and in mypy, but not in the documentation for the typing module. https://bugs.python.org/issue37814 files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index d2dd03d50fc6..12efde131657 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1245,7 +1245,8 @@ The module defines the following classes, functions and decorators: .. data:: Tuple Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items - with the first item of type X and the second of type Y. + with the first item of type X and the second of type Y. The type of + the empty tuple can be written as ``Tuple[()]``. Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple From webhook-mailer at python.org Tue Aug 13 15:12:59 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 13 Aug 2019 19:12:59 -0000 Subject: [Python-checkins] bpo-37814: Document the empty tuple type annotation syntax (GH-15208) Message-ID: https://github.com/python/cpython/commit/6ad902a08814909b4d52c4000d5a10ce58516dac commit: 6ad902a08814909b4d52c4000d5a10ce58516dac branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-13T12:12:55-07:00 summary: bpo-37814: Document the empty tuple type annotation syntax (GH-15208) https://bugs.python.org/issue37814: > The empty tuple syntax in type annotations, `Tuple[()]`, is not obvious from the examples given in the documentation (I naively expected `Tuple[]` to work); it has been documented in PEP 484 and in mypy, but not in the documentation for the typing module. https://bugs.python.org/issue37814 (cherry picked from commit 8a784af750fa82c8355903309e5089eb2b60c16b) Co-authored-by: Josh Holland files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index d2dd03d50fc6..12efde131657 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1245,7 +1245,8 @@ The module defines the following classes, functions and decorators: .. data:: Tuple Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items - with the first item of type X and the second of type Y. + with the first item of type X and the second of type Y. The type of + the empty tuple can be written as ``Tuple[()]``. Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple From webhook-mailer at python.org Tue Aug 13 15:26:20 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 13 Aug 2019 19:26:20 -0000 Subject: [Python-checkins] [3.7] bpo-37814: Document the empty tuple type annotation syntax (GH-15208) (GH-15262) Message-ID: https://github.com/python/cpython/commit/37fd9f73e2fa439554977cfba427bf94c1fedb6b commit: 37fd9f73e2fa439554977cfba427bf94c1fedb6b branch: 3.7 author: Josh Holland committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-13T12:26:12-07:00 summary: [3.7] bpo-37814: Document the empty tuple type annotation syntax (GH-15208) (GH-15262) https://bugs.python.org/issue37814: > The empty tuple syntax in type annotations, `Tuple[()]`, is not obvious from the examples given in the documentation (I naively expected `Tuple[]` to work); it has been documented in PEP 484 and in mypy, but not in the documentation for the typing module. https://bugs.python.org/issue37814 (cherry picked from commit 8a784af750fa82c8355903309e5089eb2b60c16b) Co-authored-by: Josh Holland https://bugs.python.org/issue37814 Automerge-Triggered-By: @gvanrossum files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 5adc81c1e3c8..c0b048cb2bd3 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1056,7 +1056,8 @@ The module defines the following classes, functions and decorators: .. data:: Tuple Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items - with the first item of type X and the second of type Y. + with the first item of type X and the second of type Y. The type of + the empty tuple can be written as ``Tuple[()]``. Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple From webhook-mailer at python.org Tue Aug 13 15:54:07 2019 From: webhook-mailer at python.org (Antoine Pitrou) Date: Tue, 13 Aug 2019 19:54:07 -0000 Subject: [Python-checkins] bpo-37689: add Path.is_relative_to() method (GH-14982) Message-ID: https://github.com/python/cpython/commit/82642a052dc46b2180679518bc8d87e1a28a88b5 commit: 82642a052dc46b2180679518bc8d87e1a28a88b5 branch: master author: Hai Shi committer: Antoine Pitrou date: 2019-08-13T21:54:02+02:00 summary: bpo-37689: add Path.is_relative_to() method (GH-14982) files: A Misc/NEWS.d/next/Library/2019-07-27-18-00-43.bpo-37689.glEmZi.rst M Doc/library/pathlib.rst M Lib/pathlib.py M Lib/test/test_pathlib.py diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst index 166de8de1f06..33fac5f69725 100644 --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -273,7 +273,7 @@ Methods and properties .. testsetup:: - from pathlib import PurePosixPath, PureWindowsPath + from pathlib import PurePath, PurePosixPath, PureWindowsPath Pure paths provide the following methods and properties: @@ -462,6 +462,19 @@ Pure paths provide the following methods and properties: True +.. method:: PurePath.is_relative_to(*other) + + Return whether or not this path is relative to the *other* path. + + >>> p = PurePath('/etc/passwd') + >>> p.is_relative_to('/etc') + True + >>> p.is_relative_to('/usr') + False + + .. versionadded:: 3.9 + + .. method:: PurePath.is_reserved() With :class:`PureWindowsPath`, return ``True`` if the path is considered diff --git a/Lib/pathlib.py b/Lib/pathlib.py index 6355ae864108..80923c768268 100644 --- a/Lib/pathlib.py +++ b/Lib/pathlib.py @@ -886,6 +886,15 @@ def relative_to(self, *other): return self._from_parsed_parts('', root if n == 1 else '', abs_parts[n:]) + def is_relative_to(self, *other): + """Return True if the path is relative to another path or False. + """ + try: + self.relative_to(*other) + return True + except ValueError: + return False + @property def parts(self): """An object providing sequence-like access to the diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py index f74524d992a1..cfda0a218826 100644 --- a/Lib/test/test_pathlib.py +++ b/Lib/test/test_pathlib.py @@ -619,6 +619,40 @@ def test_relative_to_common(self): self.assertRaises(ValueError, p.relative_to, '') self.assertRaises(ValueError, p.relative_to, P('a')) + def test_is_relative_to_common(self): + P = self.cls + p = P('a/b') + self.assertRaises(TypeError, p.is_relative_to) + self.assertRaises(TypeError, p.is_relative_to, b'a') + self.assertTrue(p.is_relative_to(P())) + self.assertTrue(p.is_relative_to('')) + self.assertTrue(p.is_relative_to(P('a'))) + self.assertTrue(p.is_relative_to('a/')) + self.assertTrue(p.is_relative_to(P('a/b'))) + self.assertTrue(p.is_relative_to('a/b')) + # With several args. + self.assertTrue(p.is_relative_to('a', 'b')) + # Unrelated paths. + self.assertFalse(p.is_relative_to(P('c'))) + self.assertFalse(p.is_relative_to(P('a/b/c'))) + self.assertFalse(p.is_relative_to(P('a/c'))) + self.assertFalse(p.is_relative_to(P('/a'))) + p = P('/a/b') + self.assertTrue(p.is_relative_to(P('/'))) + self.assertTrue(p.is_relative_to('/')) + self.assertTrue(p.is_relative_to(P('/a'))) + self.assertTrue(p.is_relative_to('/a')) + self.assertTrue(p.is_relative_to('/a/')) + self.assertTrue(p.is_relative_to(P('/a/b'))) + self.assertTrue(p.is_relative_to('/a/b')) + # Unrelated paths. + self.assertFalse(p.is_relative_to(P('/c'))) + self.assertFalse(p.is_relative_to(P('/a/b/c'))) + self.assertFalse(p.is_relative_to(P('/a/c'))) + self.assertFalse(p.is_relative_to(P())) + self.assertFalse(p.is_relative_to('')) + self.assertFalse(p.is_relative_to(P('a'))) + def test_pickling_common(self): P = self.cls p = P('/a/b') @@ -1062,6 +1096,59 @@ def test_relative_to(self): self.assertRaises(ValueError, p.relative_to, P('//z/Share/Foo')) self.assertRaises(ValueError, p.relative_to, P('//Server/z/Foo')) + def test_is_relative_to(self): + P = self.cls + p = P('C:Foo/Bar') + self.assertTrue(p.is_relative_to(P('c:'))) + self.assertTrue(p.is_relative_to('c:')) + self.assertTrue(p.is_relative_to(P('c:foO'))) + self.assertTrue(p.is_relative_to('c:foO')) + self.assertTrue(p.is_relative_to('c:foO/')) + self.assertTrue(p.is_relative_to(P('c:foO/baR'))) + self.assertTrue(p.is_relative_to('c:foO/baR')) + # Unrelated paths. + self.assertFalse(p.is_relative_to(P())) + self.assertFalse(p.is_relative_to('')) + self.assertFalse(p.is_relative_to(P('d:'))) + self.assertFalse(p.is_relative_to(P('/'))) + self.assertFalse(p.is_relative_to(P('Foo'))) + self.assertFalse(p.is_relative_to(P('/Foo'))) + self.assertFalse(p.is_relative_to(P('C:/Foo'))) + self.assertFalse(p.is_relative_to(P('C:Foo/Bar/Baz'))) + self.assertFalse(p.is_relative_to(P('C:Foo/Baz'))) + p = P('C:/Foo/Bar') + self.assertTrue(p.is_relative_to('c:')) + self.assertTrue(p.is_relative_to(P('c:/'))) + self.assertTrue(p.is_relative_to(P('c:/foO'))) + self.assertTrue(p.is_relative_to('c:/foO/')) + self.assertTrue(p.is_relative_to(P('c:/foO/baR'))) + self.assertTrue(p.is_relative_to('c:/foO/baR')) + # Unrelated paths. + self.assertFalse(p.is_relative_to(P('C:/Baz'))) + self.assertFalse(p.is_relative_to(P('C:/Foo/Bar/Baz'))) + self.assertFalse(p.is_relative_to(P('C:/Foo/Baz'))) + self.assertFalse(p.is_relative_to(P('C:Foo'))) + self.assertFalse(p.is_relative_to(P('d:'))) + self.assertFalse(p.is_relative_to(P('d:/'))) + self.assertFalse(p.is_relative_to(P('/'))) + self.assertFalse(p.is_relative_to(P('/Foo'))) + self.assertFalse(p.is_relative_to(P('//C/Foo'))) + # UNC paths. + p = P('//Server/Share/Foo/Bar') + self.assertTrue(p.is_relative_to(P('//sErver/sHare'))) + self.assertTrue(p.is_relative_to('//sErver/sHare')) + self.assertTrue(p.is_relative_to('//sErver/sHare/')) + self.assertTrue(p.is_relative_to(P('//sErver/sHare/Foo'))) + self.assertTrue(p.is_relative_to('//sErver/sHare/Foo')) + self.assertTrue(p.is_relative_to('//sErver/sHare/Foo/')) + self.assertTrue(p.is_relative_to(P('//sErver/sHare/Foo/Bar'))) + self.assertTrue(p.is_relative_to('//sErver/sHare/Foo/Bar')) + # Unrelated paths. + self.assertFalse(p.is_relative_to(P('/Server/Share/Foo'))) + self.assertFalse(p.is_relative_to(P('c:/Server/Share/Foo'))) + self.assertFalse(p.is_relative_to(P('//z/Share/Foo'))) + self.assertFalse(p.is_relative_to(P('//Server/z/Foo'))) + def test_is_absolute(self): P = self.cls # Under NT, only paths with both a drive and a root are absolute. diff --git a/Misc/NEWS.d/next/Library/2019-07-27-18-00-43.bpo-37689.glEmZi.rst b/Misc/NEWS.d/next/Library/2019-07-27-18-00-43.bpo-37689.glEmZi.rst new file mode 100644 index 000000000000..2787f9e849c2 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-27-18-00-43.bpo-37689.glEmZi.rst @@ -0,0 +1 @@ +Add :meth:`is_relative_to` in :class:`PurePath` to determine whether or not one path is relative to another. From webhook-mailer at python.org Tue Aug 13 17:27:19 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 13 Aug 2019 21:27:19 -0000 Subject: [Python-checkins] bpo-25172: Raise appropriate ImportError msg when crypt module used on Windows (GH-15149) Message-ID: https://github.com/python/cpython/commit/7f7f74734acd729d1f82b7cf672e064c9525fced commit: 7f7f74734acd729d1f82b7cf672e064c9525fced branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-13T14:27:14-07:00 summary: bpo-25172: Raise appropriate ImportError msg when crypt module used on Windows (GH-15149) (cherry picked from commit f4e725f224b864bf9bf405ff7f863cda46fca1cd) Co-authored-by: shireenrao files: A Misc/NEWS.d/next/Windows/2019-08-06-18-09-18.bpo-25172.Akreij.rst M Lib/crypt.py diff --git a/Lib/crypt.py b/Lib/crypt.py index b0e47f430c3c..8846602d7613 100644 --- a/Lib/crypt.py +++ b/Lib/crypt.py @@ -1,6 +1,15 @@ """Wrapper to the POSIX crypt library call and associated functionality.""" -import _crypt +import sys as _sys + +try: + import _crypt +except ModuleNotFoundError: + if _sys.platform == 'win32': + raise ImportError("The crypt module is not supported on Windows") + else: + raise ImportError("The required _crypt module was not built as part of CPython") + import string as _string from random import SystemRandom as _SystemRandom from collections import namedtuple as _namedtuple diff --git a/Misc/NEWS.d/next/Windows/2019-08-06-18-09-18.bpo-25172.Akreij.rst b/Misc/NEWS.d/next/Windows/2019-08-06-18-09-18.bpo-25172.Akreij.rst new file mode 100644 index 000000000000..47106d887921 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-08-06-18-09-18.bpo-25172.Akreij.rst @@ -0,0 +1 @@ +Trying to import the :mod:`crypt` module on Windows will result in an :exc:`ImportError` with a message explaining that the module isn't supported on Windows. On other platforms, if the underlying ``_crypt`` module is not available, the ImportError will include a message explaining the problem. From webhook-mailer at python.org Tue Aug 13 17:27:37 2019 From: webhook-mailer at python.org (Steve Dower) Date: Tue, 13 Aug 2019 21:27:37 -0000 Subject: [Python-checkins] bpo-25172: Add test for crypt ImportError on Windows (GH-15252) Message-ID: https://github.com/python/cpython/commit/243a73deee4ac61fe06602b7ed56b6df01e19f27 commit: 243a73deee4ac61fe06602b7ed56b6df01e19f27 branch: master author: shireenrao committer: Steve Dower date: 2019-08-13T14:27:34-07:00 summary: bpo-25172: Add test for crypt ImportError on Windows (GH-15252) files: M Lib/test/test_crypt.py diff --git a/Lib/test/test_crypt.py b/Lib/test/test_crypt.py index d9189fc6f28b..d29e005fdad5 100644 --- a/Lib/test/test_crypt.py +++ b/Lib/test/test_crypt.py @@ -1,9 +1,25 @@ import sys -from test import support import unittest -crypt = support.import_module('crypt') +try: + import crypt + IMPORT_ERROR = None +except ImportError as ex: + crypt = None + IMPORT_ERROR = str(ex) + + + at unittest.skipIf(crypt, 'This should only run on windows') +class TestWhyCryptDidNotImport(unittest.TestCase): + def test_failure_only_for_windows(self): + self.assertEqual(sys.platform, 'win32') + + def test_import_failure_message(self): + self.assertIn('not supported', IMPORT_ERROR) + + + at unittest.skipUnless(crypt, 'Not supported on Windows') class CryptTestCase(unittest.TestCase): def test_crypt(self): @@ -39,9 +55,13 @@ def test_methods(self): else: self.assertEqual(crypt.methods[-1], crypt.METHOD_CRYPT) - @unittest.skipUnless(crypt.METHOD_SHA256 in crypt.methods or - crypt.METHOD_SHA512 in crypt.methods, - 'requires support of SHA-2') + @unittest.skipUnless( + crypt + and ( + crypt.METHOD_SHA256 in crypt.methods or crypt.METHOD_SHA512 in crypt.methods + ), + 'requires support of SHA-2', + ) def test_sha2_rounds(self): for method in (crypt.METHOD_SHA256, crypt.METHOD_SHA512): for rounds in 1000, 10_000, 100_000: @@ -54,8 +74,9 @@ def test_sha2_rounds(self): cr2 = crypt.crypt('mypassword', cr) self.assertEqual(cr2, cr) - @unittest.skipUnless(crypt.METHOD_BLOWFISH in crypt.methods, - 'requires support of Blowfish') + @unittest.skipUnless( + crypt and crypt.METHOD_BLOWFISH in crypt.methods, 'requires support of Blowfish' + ) def test_blowfish_rounds(self): for log_rounds in range(4, 11): salt = crypt.mksalt(crypt.METHOD_BLOWFISH, rounds=1 << log_rounds) From webhook-mailer at python.org Tue Aug 13 17:52:24 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 13 Aug 2019 21:52:24 -0000 Subject: [Python-checkins] bpo-25172: Add test for crypt ImportError on Windows (GH-15252) Message-ID: https://github.com/python/cpython/commit/e7ec9e04c82be72aef621fdfba03f41cbd8599aa commit: e7ec9e04c82be72aef621fdfba03f41cbd8599aa branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-13T14:52:20-07:00 summary: bpo-25172: Add test for crypt ImportError on Windows (GH-15252) (cherry picked from commit 243a73deee4ac61fe06602b7ed56b6df01e19f27) Co-authored-by: shireenrao files: M Lib/test/test_crypt.py diff --git a/Lib/test/test_crypt.py b/Lib/test/test_crypt.py index d9189fc6f28b..d29e005fdad5 100644 --- a/Lib/test/test_crypt.py +++ b/Lib/test/test_crypt.py @@ -1,9 +1,25 @@ import sys -from test import support import unittest -crypt = support.import_module('crypt') +try: + import crypt + IMPORT_ERROR = None +except ImportError as ex: + crypt = None + IMPORT_ERROR = str(ex) + + + at unittest.skipIf(crypt, 'This should only run on windows') +class TestWhyCryptDidNotImport(unittest.TestCase): + def test_failure_only_for_windows(self): + self.assertEqual(sys.platform, 'win32') + + def test_import_failure_message(self): + self.assertIn('not supported', IMPORT_ERROR) + + + at unittest.skipUnless(crypt, 'Not supported on Windows') class CryptTestCase(unittest.TestCase): def test_crypt(self): @@ -39,9 +55,13 @@ def test_methods(self): else: self.assertEqual(crypt.methods[-1], crypt.METHOD_CRYPT) - @unittest.skipUnless(crypt.METHOD_SHA256 in crypt.methods or - crypt.METHOD_SHA512 in crypt.methods, - 'requires support of SHA-2') + @unittest.skipUnless( + crypt + and ( + crypt.METHOD_SHA256 in crypt.methods or crypt.METHOD_SHA512 in crypt.methods + ), + 'requires support of SHA-2', + ) def test_sha2_rounds(self): for method in (crypt.METHOD_SHA256, crypt.METHOD_SHA512): for rounds in 1000, 10_000, 100_000: @@ -54,8 +74,9 @@ def test_sha2_rounds(self): cr2 = crypt.crypt('mypassword', cr) self.assertEqual(cr2, cr) - @unittest.skipUnless(crypt.METHOD_BLOWFISH in crypt.methods, - 'requires support of Blowfish') + @unittest.skipUnless( + crypt and crypt.METHOD_BLOWFISH in crypt.methods, 'requires support of Blowfish' + ) def test_blowfish_rounds(self): for log_rounds in range(4, 11): salt = crypt.mksalt(crypt.METHOD_BLOWFISH, rounds=1 << log_rounds) From webhook-mailer at python.org Tue Aug 13 21:11:05 2019 From: webhook-mailer at python.org (Senthil Kumaran) Date: Wed, 14 Aug 2019 01:11:05 -0000 Subject: [Python-checkins] bpo-37256: Wording in Request class docs (#14792) Message-ID: https://github.com/python/cpython/commit/38c7199beb30ae9a5005c0f0d9df9fae0da3680a commit: 38c7199beb30ae9a5005c0f0d9df9fae0da3680a branch: master author: Ngalim Siregar committer: Senthil Kumaran date: 2019-08-13T18:10:58-07:00 summary: bpo-37256: Wording in Request class docs (#14792) * bpo-37256: Wording in Request class docs * ?? Added by blurb_it. * Update Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst Co-Authored-By: Kyle Stanley files: A Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst M Doc/library/urllib.request.rst diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst index 3b7508966691..448bc6785470 100644 --- a/Doc/library/urllib.request.rst +++ b/Doc/library/urllib.request.rst @@ -227,7 +227,7 @@ The following classes are provided: is not None, ``Content-Type: application/x-www-form-urlencoded`` will be added as a default. - The final two arguments are only of interest for correct handling + The next two arguments are only of interest for correct handling of third-party HTTP cookies: *origin_req_host* should be the request-host of the origin diff --git a/Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst b/Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst new file mode 100644 index 000000000000..480d7c87ebc4 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst @@ -0,0 +1 @@ +Fix wording of arguments for :class:`Request` in :mod:`urllib.request` From webhook-mailer at python.org Tue Aug 13 22:28:46 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 14 Aug 2019 02:28:46 -0000 Subject: [Python-checkins] bpo-37760: Factor out standard range-expanding logic in makeunicodedata. (GH-15248) Message-ID: https://github.com/python/cpython/commit/c03e698c344dfc557555b6b07a3ee2702e45f6ee commit: c03e698c344dfc557555b6b07a3ee2702e45f6ee branch: master author: Greg Price committer: Benjamin Peterson date: 2019-08-13T19:28:38-07:00 summary: bpo-37760: Factor out standard range-expanding logic in makeunicodedata. (GH-15248) Much like the lower-level logic in commit ef2af1ad4, we had 4 copies of this logic, written in a couple of different ways. They're all implementing the same standard, so write it just once. files: M Tools/unicode/makeunicodedata.py diff --git a/Tools/unicode/makeunicodedata.py b/Tools/unicode/makeunicodedata.py index 38c1f19a6567..cc2b2981ef5b 100644 --- a/Tools/unicode/makeunicodedata.py +++ b/Tools/unicode/makeunicodedata.py @@ -32,7 +32,7 @@ from functools import partial from textwrap import dedent -from typing import * +from typing import Iterator, List, Tuple SCRIPT = sys.argv[0] VERSION = "3.3" @@ -904,6 +904,19 @@ def open_data(template, version): return open(local, 'rb') +def expand_range(char_range: str) -> Iterator[int]: + ''' + Parses ranges of code points, as described in UAX #44: + https://www.unicode.org/reports/tr44/#Code_Point_Ranges + ''' + if '..' in char_range: + first, last = [int(c, 16) for c in char_range.split('..')] + else: + first = last = int(char_range, 16) + for char in range(first, last+1): + yield char + + class UcdFile: ''' A file in the standard format of the UCD. @@ -929,6 +942,12 @@ def records(self) -> Iterator[List[str]]: def __iter__(self) -> Iterator[List[str]]: return self.records() + def expanded(self) -> Iterator[Tuple[int, List[str]]]: + for record in self.records(): + char_range, rest = record[0], record[1:] + for char in expand_range(char_range): + yield char, rest + # -------------------------------------------------------------------- # the following support code is taken from the unidb utilities @@ -955,6 +974,9 @@ def __init__(self, version, cjk_check=True): # expand first-last ranges field = None for i in range(0, 0x110000): + # The file UnicodeData.txt has its own distinct way of + # expressing ranges. See: + # https://www.unicode.org/reports/tr44/#Code_Point_Ranges s = table[i] if s: if s[1][-6:] == "First>": @@ -1019,14 +1041,8 @@ def __init__(self, version, cjk_check=True): self.exclusions[char] = 1 widths = [None] * 0x110000 - for s in UcdFile(EASTASIAN_WIDTH, version): - if '..' in s[0]: - first, last = [int(c, 16) for c in s[0].split('..')] - chars = list(range(first, last+1)) - else: - chars = [int(s[0], 16)] - for char in chars: - widths[char] = s[1] + for char, (width,) in UcdFile(EASTASIAN_WIDTH, version).expanded(): + widths[char] = width for i in range(0, 0x110000): if table[i] is not None: @@ -1036,26 +1052,16 @@ def __init__(self, version, cjk_check=True): if table[i] is not None: table[i].append(set()) - for r, p in UcdFile(DERIVED_CORE_PROPERTIES, version): - if ".." in r: - first, last = [int(c, 16) for c in r.split('..')] - chars = list(range(first, last+1)) - else: - chars = [int(r, 16)] - for char in chars: - if table[char]: - # Some properties (e.g. Default_Ignorable_Code_Point) - # apply to unassigned code points; ignore them - table[char][-1].add(p) - - for s in UcdFile(LINE_BREAK, version): - if len(s) < 2 or s[1] not in MANDATORY_LINE_BREAKS: + for char, (p,) in UcdFile(DERIVED_CORE_PROPERTIES, version).expanded(): + if table[char]: + # Some properties (e.g. Default_Ignorable_Code_Point) + # apply to unassigned code points; ignore them + table[char][-1].add(p) + + for char_range, value in UcdFile(LINE_BREAK, version): + if value not in MANDATORY_LINE_BREAKS: continue - if '..' not in s[0]: - first = last = int(s[0], 16) - else: - first, last = [int(c, 16) for c in s[0].split('..')] - for char in range(first, last+1): + for char in expand_range(char_range): table[char][-1].add('Line_Break') # We only want the quickcheck properties @@ -1073,11 +1079,7 @@ def __init__(self, version, cjk_check=True): quickcheck = 'MN'.index(s[2]) + 1 # Maybe or No quickcheck_shift = qc_order.index(s[1])*2 quickcheck <<= quickcheck_shift - if '..' not in s[0]: - first = last = int(s[0], 16) - else: - first, last = [int(c, 16) for c in s[0].split('..')] - for char in range(first, last+1): + for char in expand_range(s[0]): assert not (quickchecks[char]>>quickcheck_shift)&3 quickchecks[char] |= quickcheck for i in range(0, 0x110000): From webhook-mailer at python.org Wed Aug 14 01:51:10 2019 From: webhook-mailer at python.org (Inada Naoki) Date: Wed, 14 Aug 2019 05:51:10 -0000 Subject: [Python-checkins] bpo-37337: Fix a GCC 9 warning in Objects/descrobject.c (GH-14814) Message-ID: https://github.com/python/cpython/commit/43d564c18c97421f73025ac3132a194975c76bd6 commit: 43d564c18c97421f73025ac3132a194975c76bd6 branch: master author: Zackery Spytz committer: Inada Naoki date: 2019-08-14T14:51:06+09:00 summary: bpo-37337: Fix a GCC 9 warning in Objects/descrobject.c (GH-14814) Commit b1263d5a60d3f7ab02dd28409fff59b3815a3f67 causes GCC 9.1.0 to give a warning in Objects/descrobject.c. files: M Objects/descrobject.c diff --git a/Objects/descrobject.c b/Objects/descrobject.c index edce250c7948..9e1b281c4603 100644 --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -1047,7 +1047,7 @@ mappingproxy_copy(mappingproxyobject *pp, PyObject *Py_UNUSED(ignored)) to the underlying mapping */ static PyMethodDef mappingproxy_methods[] = { - {"get", (PyCFunction)mappingproxy_get, METH_FASTCALL, + {"get", (PyCFunction)(void(*)(void))mappingproxy_get, METH_FASTCALL, PyDoc_STR("D.get(k[,d]) -> D[k] if k in D, else d." " d defaults to None.")}, {"keys", (PyCFunction)mappingproxy_keys, METH_NOARGS, From webhook-mailer at python.org Wed Aug 14 05:50:37 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 09:50:37 -0000 Subject: [Python-checkins] bpo-37681: no_sanitize_thread support from GCC 5.1 (GH-15096) Message-ID: https://github.com/python/cpython/commit/7e479c82218450255572e3f5fa1549dc283901ea commit: 7e479c82218450255572e3f5fa1549dc283901ea branch: master author: Hai Shi committer: Victor Stinner date: 2019-08-14T11:50:19+02:00 summary: bpo-37681: no_sanitize_thread support from GCC 5.1 (GH-15096) Fix the following warning with GCC 4.8.5: Objects/obmalloc.c: warning: ?no_sanitize_thread? attribute directive ignored files: M Objects/obmalloc.c diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c index 8d5c700d5c1e..6ca7cd84eda8 100644 --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -45,9 +45,9 @@ static void _PyMem_SetupDebugHooksDomain(PyMemAllocatorDomain domain); # define _Py_NO_ADDRESS_SAFETY_ANALYSIS \ __attribute__((no_address_safety_analysis)) # endif - // TSAN is supported since GCC 4.8, but __SANITIZE_THREAD__ macro + // TSAN is supported since GCC 5.1, but __SANITIZE_THREAD__ macro // is provided only since GCC 7. -# if __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8) +# if __GNUC__ > 5 || (__GNUC__ == 5 && __GNUC_MINOR__ >= 1) # define _Py_NO_SANITIZE_THREAD __attribute__((no_sanitize_thread)) # endif #endif From webhook-mailer at python.org Wed Aug 14 05:52:41 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 09:52:41 -0000 Subject: [Python-checkins] bpo-37583: Add err 113 to support.get_socket_conn_refused_errs() (GH-15259) Message-ID: https://github.com/python/cpython/commit/fbb0c032e89c42ab44f5372df40fffb34a91b575 commit: fbb0c032e89c42ab44f5372df40fffb34a91b575 branch: 3.7 author: Hai Shi committer: Victor Stinner date: 2019-08-14T11:52:36+02:00 summary: bpo-37583: Add err 113 to support.get_socket_conn_refused_errs() (GH-15259) Add error number 113 EHOSTUNREACH to get_socket_conn_refused_errs() of test.support. files: M Lib/test/support/__init__.py diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index bf1f0d2120ac..d04d19985f7a 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -1423,6 +1423,9 @@ def get_socket_conn_refused_errs(): # bpo-31910: socket.create_connection() fails randomly # with EADDRNOTAVAIL on Travis CI errors.append(errno.EADDRNOTAVAIL) + if hasattr(errno, 'EHOSTUNREACH'): + # bpo-37583: The destination host cannot be reached + errors.append(errno.EHOSTUNREACH) return errors From webhook-mailer at python.org Wed Aug 14 06:08:51 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 10:08:51 -0000 Subject: [Python-checkins] bpo-37681: no_sanitize_thread support from GCC 5.1 (GH-15096) Message-ID: https://github.com/python/cpython/commit/364a1d3125af6c0ff0a4bd2c8afe70fd38f30456 commit: 364a1d3125af6c0ff0a4bd2c8afe70fd38f30456 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-14T03:08:46-07:00 summary: bpo-37681: no_sanitize_thread support from GCC 5.1 (GH-15096) Fix the following warning with GCC 4.8.5: Objects/obmalloc.c: warning: ?no_sanitize_thread? attribute directive ignored (cherry picked from commit 7e479c82218450255572e3f5fa1549dc283901ea) Co-authored-by: Hai Shi files: M Objects/obmalloc.c diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c index fc7bef619946..0501cb992008 100644 --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -45,9 +45,9 @@ static void _PyMem_SetupDebugHooksDomain(PyMemAllocatorDomain domain); # define _Py_NO_ADDRESS_SAFETY_ANALYSIS \ __attribute__((no_address_safety_analysis)) # endif - // TSAN is supported since GCC 4.8, but __SANITIZE_THREAD__ macro + // TSAN is supported since GCC 5.1, but __SANITIZE_THREAD__ macro // is provided only since GCC 7. -# if __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8) +# if __GNUC__ > 5 || (__GNUC__ == 5 && __GNUC_MINOR__ >= 1) # define _Py_NO_SANITIZE_THREAD __attribute__((no_sanitize_thread)) # endif #endif From webhook-mailer at python.org Wed Aug 14 06:16:58 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 10:16:58 -0000 Subject: [Python-checkins] bpo-37681: no_sanitize_thread support from GCC 5.1 (GH-15096) Message-ID: https://github.com/python/cpython/commit/15b6d0a712c08557a605f49034f8ad392985144c commit: 15b6d0a712c08557a605f49034f8ad392985144c branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-14T03:16:54-07:00 summary: bpo-37681: no_sanitize_thread support from GCC 5.1 (GH-15096) Fix the following warning with GCC 4.8.5: Objects/obmalloc.c: warning: ?no_sanitize_thread? attribute directive ignored (cherry picked from commit 7e479c82218450255572e3f5fa1549dc283901ea) Co-authored-by: Hai Shi files: M Objects/obmalloc.c diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c index 885ad537ad95..8446efcbd440 100644 --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -44,9 +44,9 @@ static void _PyMem_SetupDebugHooksDomain(PyMemAllocatorDomain domain); # define _Py_NO_ADDRESS_SAFETY_ANALYSIS \ __attribute__((no_address_safety_analysis)) # endif - // TSAN is supported since GCC 4.8, but __SANITIZE_THREAD__ macro + // TSAN is supported since GCC 5.1, but __SANITIZE_THREAD__ macro // is provided only since GCC 7. -# if __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8) +# if __GNUC__ > 5 || (__GNUC__ == 5 && __GNUC_MINOR__ >= 1) # define _Py_NO_SANITIZE_THREAD __attribute__((no_sanitize_thread)) # endif #endif From webhook-mailer at python.org Wed Aug 14 06:31:58 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 10:31:58 -0000 Subject: [Python-checkins] bpo-37738: Fix curses addch(str, color_pair) (GH-15071) Message-ID: https://github.com/python/cpython/commit/077af8c2c93dd71086e2c5e5ff1e634b6da8f214 commit: 077af8c2c93dd71086e2c5e5ff1e634b6da8f214 branch: master author: Victor Stinner committer: GitHub date: 2019-08-14T12:31:43+02:00 summary: bpo-37738: Fix curses addch(str, color_pair) (GH-15071) Fix the implementation of curses addch(str, color_pair): pass the color pair to setcchar(), instead of always passing 0 as the color pair. files: A Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst M Modules/_cursesmodule.c diff --git a/Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst b/Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst new file mode 100644 index 000000000000..7e70a9c2231a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst @@ -0,0 +1,2 @@ +Fix the implementation of curses ``addch(str, color_pair)``: pass the color +pair to ``setcchar()``, instead of always passing 0 as the color pair. diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c index dbe3c99d7d59..c66f7943b695 100644 --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -176,6 +176,18 @@ static char *screen_encoding = NULL; /* Utility Functions */ +static inline int +color_pair_to_attr(short color_number) +{ + return ((int)color_number << 8); +} + +static inline short +attr_to_color_pair(int attr) +{ + return (short)((attr & A_COLOR) >> 8); +} + /* * Check the return code from a curses function and return None * or raise an exception as appropriate. These are exported using the @@ -606,7 +618,7 @@ _curses_window_addch_impl(PyCursesWindowObject *self, int group_left_1, if (type == 2) { funcname = "add_wch"; wstr[1] = L'\0'; - setcchar(&wcval, wstr, attr, 0, NULL); + setcchar(&wcval, wstr, attr, attr_to_color_pair(attr), NULL); if (coordinates_group) rtn = mvwadd_wch(self->win,y,x, &wcval); else { @@ -2621,7 +2633,7 @@ _curses_color_pair_impl(PyObject *module, short color_number) PyCursesInitialised; PyCursesInitialisedColor; - return PyLong_FromLong((long) (color_number << 8)); + return PyLong_FromLong(color_pair_to_attr(color_number)); } /*[clinic input] @@ -3644,7 +3656,7 @@ _curses_pair_number_impl(PyObject *module, int attr) PyCursesInitialised; PyCursesInitialisedColor; - return PyLong_FromLong((long) ((attr & A_COLOR) >> 8)); + return PyLong_FromLong(attr_to_color_pair(attr)); } /*[clinic input] From webhook-mailer at python.org Wed Aug 14 06:49:17 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 10:49:17 -0000 Subject: [Python-checkins] bpo-37738: Fix curses addch(str, color_pair) (GH-15071) Message-ID: https://github.com/python/cpython/commit/984226962bc35254551d92771b5c8fb074507903 commit: 984226962bc35254551d92771b5c8fb074507903 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-14T03:49:13-07:00 summary: bpo-37738: Fix curses addch(str, color_pair) (GH-15071) Fix the implementation of curses addch(str, color_pair): pass the color pair to setcchar(), instead of always passing 0 as the color pair. (cherry picked from commit 077af8c2c93dd71086e2c5e5ff1e634b6da8f214) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst M Modules/_cursesmodule.c diff --git a/Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst b/Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst new file mode 100644 index 000000000000..7e70a9c2231a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst @@ -0,0 +1,2 @@ +Fix the implementation of curses ``addch(str, color_pair)``: pass the color +pair to ``setcchar()``, instead of always passing 0 as the color pair. diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c index 8fca7fcf1c18..819e1ee41f1c 100644 --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -176,6 +176,18 @@ static char *screen_encoding = NULL; /* Utility Functions */ +static inline int +color_pair_to_attr(short color_number) +{ + return ((int)color_number << 8); +} + +static inline short +attr_to_color_pair(int attr) +{ + return (short)((attr & A_COLOR) >> 8); +} + /* * Check the return code from a curses function and return None * or raise an exception as appropriate. These are exported using the @@ -606,7 +618,7 @@ _curses_window_addch_impl(PyCursesWindowObject *self, int group_left_1, if (type == 2) { funcname = "add_wch"; wstr[1] = L'\0'; - setcchar(&wcval, wstr, attr, 0, NULL); + setcchar(&wcval, wstr, attr, attr_to_color_pair(attr), NULL); if (coordinates_group) rtn = mvwadd_wch(self->win,y,x, &wcval); else { @@ -2621,7 +2633,7 @@ _curses_color_pair_impl(PyObject *module, short color_number) PyCursesInitialised; PyCursesInitialisedColor; - return PyLong_FromLong((long) (color_number << 8)); + return PyLong_FromLong(color_pair_to_attr(color_number)); } /*[clinic input] @@ -3644,7 +3656,7 @@ _curses_pair_number_impl(PyObject *module, int attr) PyCursesInitialised; PyCursesInitialisedColor; - return PyLong_FromLong((long) ((attr & A_COLOR) >> 8)); + return PyLong_FromLong(attr_to_color_pair(attr)); } /*[clinic input] From webhook-mailer at python.org Wed Aug 14 07:00:31 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 11:00:31 -0000 Subject: [Python-checkins] bpo-37738: Fix curses addch(str, color_pair) (GH-15071) (GH-15273) Message-ID: https://github.com/python/cpython/commit/7eef81ee766c8df23e522b4e46a930cc1d360ad7 commit: 7eef81ee766c8df23e522b4e46a930cc1d360ad7 branch: 3.7 author: Victor Stinner committer: GitHub date: 2019-08-14T13:00:27+02:00 summary: bpo-37738: Fix curses addch(str, color_pair) (GH-15071) (GH-15273) Fix the implementation of curses addch(str, color_pair): pass the color pair to setcchar(), instead of always passing 0 as the color pair. (cherry picked from commit 077af8c2c93dd71086e2c5e5ff1e634b6da8f214) files: A Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst M Modules/_cursesmodule.c diff --git a/Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst b/Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst new file mode 100644 index 000000000000..7e70a9c2231a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-01-17-11-16.bpo-37738.A3WWcT.rst @@ -0,0 +1,2 @@ +Fix the implementation of curses ``addch(str, color_pair)``: pass the color +pair to ``setcchar()``, instead of always passing 0 as the color pair. diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c index 45decf96a352..b0be6bb30b8f 100644 --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -178,6 +178,18 @@ static char *screen_encoding = NULL; /* Utility Functions */ +static inline int +color_pair_to_attr(short color_number) +{ + return ((int)color_number << 8); +} + +static inline short +attr_to_color_pair(int attr) +{ + return (short)((attr & A_COLOR) >> 8); +} + /* * Check the return code from a curses function and return None * or raise an exception as appropriate. These are exported using the @@ -614,7 +626,7 @@ curses_window_addch_impl(PyCursesWindowObject *self, int group_left_1, int y, if (type == 2) { funcname = "add_wch"; wstr[1] = L'\0'; - setcchar(&wcval, wstr, attr, 0, NULL); + setcchar(&wcval, wstr, attr, attr_to_color_pair(attr), NULL); if (coordinates_group) rtn = mvwadd_wch(cwself->win,y,x, &wcval); else { @@ -2204,7 +2216,7 @@ PyCurses_color_pair(PyObject *self, PyObject *args) PyCursesInitialisedColor; if (!PyArg_ParseTuple(args, "i:color_pair", &n)) return NULL; - return PyLong_FromLong((long) (n << 8)); + return PyLong_FromLong(color_pair_to_attr(n)); } static PyObject * @@ -2802,7 +2814,7 @@ PyCurses_pair_number(PyObject *self, PyObject *args) return NULL; } - return PyLong_FromLong((long) ((n & A_COLOR) >> 8)); + return PyLong_FromLong(attr_to_color_pair(n)); } static PyObject * From webhook-mailer at python.org Wed Aug 14 07:05:24 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 11:05:24 -0000 Subject: [Python-checkins] bpo-36502: Correct documentation of str.isspace() (GH-15019) Message-ID: https://github.com/python/cpython/commit/6bccbe7dfb998af862a183f2c36f0d4603af2c29 commit: 6bccbe7dfb998af862a183f2c36f0d4603af2c29 branch: master author: Greg Price committer: Victor Stinner date: 2019-08-14T13:05:19+02:00 summary: bpo-36502: Correct documentation of str.isspace() (GH-15019) The documented definition was much broader than the real one: there are tons of characters with general category "Other", and we don't (and shouldn't) treat most of them as whitespace. Rewrite the definition to agree with the comment on _PyUnicode_IsWhitespace, and with the logic in makeunicodedata.py, which is what generates that function and so ultimately governs. Add suitable breadcrumbs so that a reader who wants to pin down exactly what this definition means (what's a "bidirectional class" of "B"?) can do so. The `unicodedata` module documentation is an appropriate central place for our references to Unicode's own copious documentation, so point there. Also add to the isspace() test a thorough check that the implementation agrees with the intended definition. files: M Doc/library/stdtypes.rst M Lib/test/test_unicode.py diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 9dd557fabaae..08c5ae876c1b 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1763,9 +1763,13 @@ expression support in the :mod:`re` module). .. method:: str.isspace() Return true if there are only whitespace characters in the string and there is - at least one character, false otherwise. Whitespace characters are those - characters defined in the Unicode character database as "Other" or "Separator" - and those with bidirectional property being one of "WS", "B", or "S". + at least one character, false otherwise. + + A character is *whitespace* if in the Unicode character database + (see :mod:`unicodedata`), either its general category is ``Zs`` + ("Separator, space"), or its bidirectional class is one of ``WS``, + ``B``, or ``S``. + .. method:: str.istitle() diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py index 8be16c8da926..7bd7f51b592b 100644 --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -12,6 +12,7 @@ import struct import sys import textwrap +import unicodedata import unittest import warnings from test import support, string_tests @@ -617,11 +618,21 @@ def test_isspace(self): self.checkequalnofix(True, '\u2000', 'isspace') self.checkequalnofix(True, '\u200a', 'isspace') self.checkequalnofix(False, '\u2014', 'isspace') - # apparently there are no non-BMP spaces chars in Unicode 6 + # There are no non-BMP whitespace chars as of Unicode 12. for ch in ['\U00010401', '\U00010427', '\U00010429', '\U0001044E', '\U0001F40D', '\U0001F46F']: self.assertFalse(ch.isspace(), '{!a} is not space.'.format(ch)) + @support.requires_resource('cpu') + def test_isspace_invariant(self): + for codepoint in range(sys.maxunicode + 1): + char = chr(codepoint) + bidirectional = unicodedata.bidirectional(char) + category = unicodedata.category(char) + self.assertEqual(char.isspace(), + (bidirectional in ('WS', 'B', 'S') + or category == 'Zs')) + def test_isalnum(self): super().test_isalnum() for ch in ['\U00010401', '\U00010427', '\U00010429', '\U0001044E', From webhook-mailer at python.org Wed Aug 14 08:18:55 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 12:18:55 -0000 Subject: [Python-checkins] bpo-37531: Fix regrtest timeout for subprocesses (GH-15072) Message-ID: https://github.com/python/cpython/commit/b0c8369c603633f445ccbb5ca7a8742145ff9eec commit: b0c8369c603633f445ccbb5ca7a8742145ff9eec branch: master author: Victor Stinner committer: GitHub date: 2019-08-14T14:18:51+02:00 summary: bpo-37531: Fix regrtest timeout for subprocesses (GH-15072) Co-Authored-By: Joannah Nanjekye files: A Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst M Lib/test/libregrtest/main.py M Lib/test/libregrtest/runtest.py M Lib/test/libregrtest/runtest_mp.py M Lib/test/test_regrtest.py diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py index 87278d0c76a2..3cfbb4017dfb 100644 --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -14,7 +14,7 @@ from test.libregrtest.runtest import ( findtests, runtest, get_abs_module, STDTESTS, NOTTESTS, PASSED, FAILED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED, - INTERRUPTED, CHILD_ERROR, TEST_DID_NOT_RUN, + INTERRUPTED, CHILD_ERROR, TEST_DID_NOT_RUN, TIMEOUT, PROGRESS_MIN_TIME, format_test_result, is_failed) from test.libregrtest.setup import setup_tests from test.libregrtest.pgo import setup_pgo_tests @@ -115,6 +115,8 @@ def accumulate_result(self, result, rerun=False): self.run_no_tests.append(test_name) elif ok == INTERRUPTED: self.interrupted = True + elif ok == TIMEOUT: + self.bad.append(test_name) else: raise ValueError("invalid test result: %r" % ok) diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py index a43b7666cd1e..e7dce180cb37 100644 --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -13,7 +13,7 @@ from test import support from test.libregrtest.refleak import dash_R, clear_caches from test.libregrtest.save_env import saved_test_environment -from test.libregrtest.utils import print_warning +from test.libregrtest.utils import format_duration, print_warning # Test result constants. @@ -25,6 +25,7 @@ INTERRUPTED = -4 CHILD_ERROR = -5 # error in a child process TEST_DID_NOT_RUN = -6 +TIMEOUT = -7 _FORMAT_TEST_RESULT = { PASSED: '%s passed', @@ -35,6 +36,7 @@ INTERRUPTED: '%s interrupted', CHILD_ERROR: '%s crashed', TEST_DID_NOT_RUN: '%s run no tests', + TIMEOUT: '%s timed out', } # Minimum duration of a test to display its duration or to mention that @@ -75,7 +77,10 @@ def is_failed(result, ns): def format_test_result(result): fmt = _FORMAT_TEST_RESULT.get(result.result, "%s") - return fmt % result.test_name + text = fmt % result.test_name + if result.result == TIMEOUT: + text = '%s (%s)' % (text, format_duration(result.test_time)) + return text def findtestdir(path=None): @@ -179,6 +184,7 @@ def runtest(ns, test_name): FAILED test failed PASSED test passed EMPTY_TEST_SUITE test ran no subtests. + TIMEOUT test timed out. If ns.xmlpath is not None, xml_data is a list containing each generated testsuite element. diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py index aa2409b4ef79..eed643291162 100644 --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -13,7 +13,7 @@ from test.libregrtest.runtest import ( runtest, INTERRUPTED, CHILD_ERROR, PROGRESS_MIN_TIME, - format_test_result, TestResult, is_failed) + format_test_result, TestResult, is_failed, TIMEOUT) from test.libregrtest.setup import setup_tests from test.libregrtest.utils import format_duration @@ -103,11 +103,12 @@ class ExitThread(Exception): class MultiprocessThread(threading.Thread): - def __init__(self, pending, output, ns): + def __init__(self, pending, output, ns, timeout): super().__init__() self.pending = pending self.output = output self.ns = ns + self.timeout = timeout self.current_test_name = None self.start_time = None self._popen = None @@ -126,6 +127,12 @@ def __repr__(self): return '<%s>' % ' '.join(info) def kill(self): + """ + Kill the current process (if any). + + This method can be called by the thread running the process, + or by another thread. + """ self._killed = True popen = self._popen @@ -136,6 +143,13 @@ def kill(self): # does not hang popen.stdout.close() popen.stderr.close() + popen.wait() + + def mp_result_error(self, test_name, error_type, stdout='', stderr='', + err_msg=None): + test_time = time.monotonic() - self.start_time + result = TestResult(test_name, error_type, test_time, None) + return MultiprocessResult(result, stdout, stderr, err_msg) def _runtest(self, test_name): try: @@ -154,7 +168,19 @@ def _runtest(self, test_name): raise ExitThread try: + stdout, stderr = popen.communicate(timeout=self.timeout) + except subprocess.TimeoutExpired: + if self._killed: + # kill() has been called: communicate() fails + # on reading closed stdout/stderr + raise ExitThread + + popen.kill() stdout, stderr = popen.communicate() + self.kill() + + return self.mp_result_error(test_name, TIMEOUT, + stdout, stderr) except OSError: if self._killed: # kill() has been called: communicate() fails @@ -163,7 +189,6 @@ def _runtest(self, test_name): raise except: self.kill() - popen.wait() raise retcode = popen.wait() @@ -191,8 +216,7 @@ def _runtest(self, test_name): err_msg = "Failed to parse worker JSON: %s" % exc if err_msg is not None: - test_time = time.monotonic() - self.start_time - result = TestResult(test_name, CHILD_ERROR, test_time, None) + return self.mp_result_error(test_name, CHILD_ERROR, stdout, stderr, err_msg) return MultiprocessResult(result, stdout, stderr, err_msg) @@ -236,13 +260,16 @@ def __init__(self, regrtest): self.output = queue.Queue() self.pending = MultiprocessIterator(self.regrtest.tests) if self.ns.timeout is not None: - self.test_timeout = self.ns.timeout * 1.5 + self.worker_timeout = self.ns.timeout * 1.5 + self.main_timeout = self.ns.timeout * 2.0 else: - self.test_timeout = None + self.worker_timeout = None + self.main_timeout = None self.workers = None def start_workers(self): - self.workers = [MultiprocessThread(self.pending, self.output, self.ns) + self.workers = [MultiprocessThread(self.pending, self.output, + self.ns, self.worker_timeout) for _ in range(self.ns.use_mp)] print("Run tests in parallel using %s child processes" % len(self.workers)) @@ -274,8 +301,8 @@ def _get_result(self): return None while True: - if self.test_timeout is not None: - faulthandler.dump_traceback_later(self.test_timeout, exit=True) + if self.main_timeout is not None: + faulthandler.dump_traceback_later(self.main_timeout, exit=True) # wait for a thread timeout = max(PROGRESS_UPDATE, PROGRESS_MIN_TIME) @@ -343,7 +370,7 @@ def run_tests(self): print() self.regrtest.interrupted = True finally: - if self.test_timeout is not None: + if self.main_timeout is not None: faulthandler.cancel_dump_traceback_later() # a test failed (and --failfast is set) or all tests completed diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py index fdab8c8ef5d1..931f125eb8f9 100644 --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -1154,6 +1154,33 @@ def test_garbage(self): env_changed=[testname], fail_env_changed=True) + def test_multiprocessing_timeout(self): + code = textwrap.dedent(r""" + import time + import unittest + try: + import faulthandler + except ImportError: + faulthandler = None + + class Tests(unittest.TestCase): + # test hangs and so should be stopped by the timeout + def test_sleep(self): + # we want to test regrtest multiprocessing timeout, + # not faulthandler timeout + if faulthandler is not None: + faulthandler.cancel_dump_traceback_later() + + time.sleep(60 * 5) + """) + testname = self.create_test(code=code) + + output = self.run_tests("-j2", "--timeout=1.0", testname, exitcode=2) + self.check_executed_tests(output, [testname], + failed=testname) + self.assertRegex(output, + re.compile('%s timed out' % testname, re.MULTILINE)) + def test_unraisable_exc(self): # --fail-env-changed must catch unraisable exception code = textwrap.dedent(r""" diff --git a/Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst b/Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst new file mode 100644 index 000000000000..aaf1052bd3c1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst @@ -0,0 +1,2 @@ +"python3 -m test -jN --timeout=TIMEOUT" now kills a worker process if it runs +longer than *TIMEOUT* seconds. From webhook-mailer at python.org Wed Aug 14 09:55:58 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 13:55:58 -0000 Subject: [Python-checkins] bpo-37256: Wording in Request class docs (GH-14792) Message-ID: https://github.com/python/cpython/commit/3e1f135b26cf5fb3583d7e75d39b7b7a9edb472c commit: 3e1f135b26cf5fb3583d7e75d39b7b7a9edb472c branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-14T06:55:53-07:00 summary: bpo-37256: Wording in Request class docs (GH-14792) * bpo-37256: Wording in Request class docs * ?? Added by blurb_it. * Update Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst Co-Authored-By: Kyle Stanley (cherry picked from commit 38c7199beb30ae9a5005c0f0d9df9fae0da3680a) Co-authored-by: Ngalim Siregar files: A Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst M Doc/library/urllib.request.rst diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst index 1bc81e05b38b..5d8c0ec4303c 100644 --- a/Doc/library/urllib.request.rst +++ b/Doc/library/urllib.request.rst @@ -220,7 +220,7 @@ The following classes are provided: is not None, ``Content-Type: application/x-www-form-urlencoded`` will be added as a default. - The final two arguments are only of interest for correct handling + The next two arguments are only of interest for correct handling of third-party HTTP cookies: *origin_req_host* should be the request-host of the origin diff --git a/Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst b/Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst new file mode 100644 index 000000000000..480d7c87ebc4 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst @@ -0,0 +1 @@ +Fix wording of arguments for :class:`Request` in :mod:`urllib.request` From webhook-mailer at python.org Wed Aug 14 10:10:40 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 14:10:40 -0000 Subject: [Python-checkins] bpo-36030: Improve performance of some tuple operations (GH-12052) Message-ID: https://github.com/python/cpython/commit/4fa10dde40356d7c71e5524233bafc221d9e2deb commit: 4fa10dde40356d7c71e5524233bafc221d9e2deb branch: master author: Sergey Fedoseev committer: Victor Stinner date: 2019-08-14T16:10:33+02:00 summary: bpo-36030: Improve performance of some tuple operations (GH-12052) files: M Objects/tupleobject.c diff --git a/Objects/tupleobject.c b/Objects/tupleobject.c index aeaf845d74cf..79a5d55749be 100644 --- a/Objects/tupleobject.c +++ b/Objects/tupleobject.c @@ -59,6 +59,15 @@ show_track(void) } #endif +static inline void +tuple_gc_track(PyTupleObject *op) +{ +#ifdef SHOW_TRACK_COUNT + count_tracked++; +#endif + _PyObject_GC_TRACK(op); +} + /* Print summary info about the state of the optimized allocator */ void _PyTuple_DebugMallocStats(FILE *out) @@ -76,25 +85,25 @@ _PyTuple_DebugMallocStats(FILE *out) #endif } -PyObject * -PyTuple_New(Py_ssize_t size) +/* Allocate an uninitialized tuple object. Before making it public following + steps must be done: + - initialize its items + - call tuple_gc_track() on it + Because the empty tuple is always reused and it's already tracked by GC, + this function must not be called with size == 0 (unless from PyTuple_New() + which wraps this function). +*/ +static PyTupleObject * +tuple_alloc(Py_ssize_t size) { PyTupleObject *op; - Py_ssize_t i; if (size < 0) { PyErr_BadInternalCall(); return NULL; } #if PyTuple_MAXSAVESIZE > 0 - if (size == 0 && free_list[0]) { - op = free_list[0]; - Py_INCREF(op); -#ifdef COUNT_ALLOCS - _Py_tuple_zero_allocs++; -#endif - return (PyObject *) op; - } if (size < PyTuple_MAXSAVESIZE && (op = free_list[size]) != NULL) { + assert(size != 0); free_list[size] = (PyTupleObject *) op->ob_item[0]; numfree[size]--; #ifdef COUNT_ALLOCS @@ -113,14 +122,33 @@ PyTuple_New(Py_ssize_t size) /* Check for overflow */ if ((size_t)size > ((size_t)PY_SSIZE_T_MAX - sizeof(PyTupleObject) - sizeof(PyObject *)) / sizeof(PyObject *)) { - return PyErr_NoMemory(); + return (PyTupleObject *)PyErr_NoMemory(); } op = PyObject_GC_NewVar(PyTupleObject, &PyTuple_Type, size); if (op == NULL) return NULL; } - for (i=0; i < size; i++) + return op; +} + +PyObject * +PyTuple_New(Py_ssize_t size) +{ + PyTupleObject *op; +#if PyTuple_MAXSAVESIZE > 0 + if (size == 0 && free_list[0]) { + op = free_list[0]; + Py_INCREF(op); +#ifdef COUNT_ALLOCS + _Py_tuple_zero_allocs++; +#endif + return (PyObject *) op; + } +#endif + op = tuple_alloc(size); + for (Py_ssize_t i = 0; i < size; i++) { op->ob_item[i] = NULL; + } #if PyTuple_MAXSAVESIZE > 0 if (size == 0) { free_list[0] = op; @@ -128,10 +156,7 @@ PyTuple_New(Py_ssize_t size) Py_INCREF(op); /* extra INCREF so that this is never freed */ } #endif -#ifdef SHOW_TRACK_COUNT - count_tracked++; -#endif - _PyObject_GC_TRACK(op); + tuple_gc_track(op); return (PyObject *) op; } @@ -211,24 +236,28 @@ PyTuple_Pack(Py_ssize_t n, ...) { Py_ssize_t i; PyObject *o; - PyObject *result; PyObject **items; va_list vargs; + if (n == 0) { + return PyTuple_New(0); + } + va_start(vargs, n); - result = PyTuple_New(n); + PyTupleObject *result = tuple_alloc(n); if (result == NULL) { va_end(vargs); return NULL; } - items = ((PyTupleObject *)result)->ob_item; + items = result->ob_item; for (i = 0; i < n; i++) { o = va_arg(vargs, PyObject *); Py_INCREF(o); items[i] = o; } va_end(vargs); - return result; + tuple_gc_track(result); + return (PyObject *)result; } @@ -421,7 +450,11 @@ tupleitem(PyTupleObject *a, Py_ssize_t i) PyObject * _PyTuple_FromArray(PyObject *const *src, Py_ssize_t n) { - PyTupleObject *tuple = (PyTupleObject *)PyTuple_New(n); + if (n == 0) { + return PyTuple_New(0); + } + + PyTupleObject *tuple = tuple_alloc(n); if (tuple == NULL) { return NULL; } @@ -431,6 +464,7 @@ _PyTuple_FromArray(PyObject *const *src, Py_ssize_t n) Py_INCREF(item); dst[i] = item; } + tuple_gc_track(tuple); return (PyObject *)tuple; } @@ -486,7 +520,11 @@ tupleconcat(PyTupleObject *a, PyObject *bb) if (Py_SIZE(a) > PY_SSIZE_T_MAX - Py_SIZE(b)) return PyErr_NoMemory(); size = Py_SIZE(a) + Py_SIZE(b); - np = (PyTupleObject *) PyTuple_New(size); + if (size == 0) { + return PyTuple_New(0); + } + + np = tuple_alloc(size); if (np == NULL) { return NULL; } @@ -504,6 +542,7 @@ tupleconcat(PyTupleObject *a, PyObject *bb) Py_INCREF(v); dest[i] = v; } + tuple_gc_track(np); return (PyObject *)np; #undef b } @@ -515,8 +554,6 @@ tuplerepeat(PyTupleObject *a, Py_ssize_t n) Py_ssize_t size; PyTupleObject *np; PyObject **p, **items; - if (n < 0) - n = 0; if (Py_SIZE(a) == 0 || n == 1) { if (PyTuple_CheckExact(a)) { /* Since tuples are immutable, we can return a shared @@ -524,13 +561,14 @@ tuplerepeat(PyTupleObject *a, Py_ssize_t n) Py_INCREF(a); return (PyObject *)a; } - if (Py_SIZE(a) == 0) - return PyTuple_New(0); + } + if (Py_SIZE(a) == 0 || n <= 0) { + return PyTuple_New(0); } if (n > PY_SSIZE_T_MAX / Py_SIZE(a)) return PyErr_NoMemory(); size = Py_SIZE(a) * n; - np = (PyTupleObject *) PyTuple_New(size); + np = tuple_alloc(size); if (np == NULL) return NULL; p = np->ob_item; @@ -542,6 +580,7 @@ tuplerepeat(PyTupleObject *a, Py_ssize_t n) p++; } } + tuple_gc_track(np); return (PyObject *) np; } @@ -754,7 +793,6 @@ tuplesubscript(PyTupleObject* self, PyObject* item) else if (PySlice_Check(item)) { Py_ssize_t start, stop, step, slicelength, i; size_t cur; - PyObject* result; PyObject* it; PyObject **src, **dest; @@ -774,11 +812,11 @@ tuplesubscript(PyTupleObject* self, PyObject* item) return (PyObject *)self; } else { - result = PyTuple_New(slicelength); + PyTupleObject* result = tuple_alloc(slicelength); if (!result) return NULL; src = self->ob_item; - dest = ((PyTupleObject *)result)->ob_item; + dest = result->ob_item; for (cur = start, i = 0; i < slicelength; cur += step, i++) { it = src[cur]; @@ -786,7 +824,8 @@ tuplesubscript(PyTupleObject* self, PyObject* item) dest[i] = it; } - return result; + tuple_gc_track(result); + return (PyObject *)result; } } else { From webhook-mailer at python.org Wed Aug 14 10:31:00 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 14:31:00 -0000 Subject: [Python-checkins] [3.7] bpo-37531: Fix regrtest timeout for subprocesses (GH-15072) (GH-15280) Message-ID: https://github.com/python/cpython/commit/93bee6a8d878ff952e245614c567c7c6bb7a0398 commit: 93bee6a8d878ff952e245614c567c7c6bb7a0398 branch: 3.7 author: Victor Stinner committer: GitHub date: 2019-08-14T16:30:54+02:00 summary: [3.7] bpo-37531: Fix regrtest timeout for subprocesses (GH-15072) (GH-15280) Co-Authored-By: Joannah Nanjekye (cherry picked from commit b0c8369c603633f445ccbb5ca7a8742145ff9eec) Backport also minor win_utils.py fixes from master. files: A Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst M Lib/test/libregrtest/main.py M Lib/test/libregrtest/runtest.py M Lib/test/libregrtest/runtest_mp.py M Lib/test/libregrtest/win_utils.py M Lib/test/test_regrtest.py diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py index 1dfbe47a2fab..ef1ecef25350 100644 --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -14,7 +14,7 @@ from test.libregrtest.runtest import ( findtests, runtest, get_abs_module, STDTESTS, NOTTESTS, PASSED, FAILED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED, - INTERRUPTED, CHILD_ERROR, TEST_DID_NOT_RUN, + INTERRUPTED, CHILD_ERROR, TEST_DID_NOT_RUN, TIMEOUT, PROGRESS_MIN_TIME, format_test_result, is_failed) from test.libregrtest.setup import setup_tests from test.libregrtest.utils import removepy, count, format_duration, printlist @@ -114,6 +114,8 @@ def accumulate_result(self, result, rerun=False): self.run_no_tests.append(test_name) elif ok == INTERRUPTED: self.interrupted = True + elif ok == TIMEOUT: + self.bad.append(test_name) else: raise ValueError("invalid test result: %r" % ok) diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py index a43b7666cd1e..e7dce180cb37 100644 --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -13,7 +13,7 @@ from test import support from test.libregrtest.refleak import dash_R, clear_caches from test.libregrtest.save_env import saved_test_environment -from test.libregrtest.utils import print_warning +from test.libregrtest.utils import format_duration, print_warning # Test result constants. @@ -25,6 +25,7 @@ INTERRUPTED = -4 CHILD_ERROR = -5 # error in a child process TEST_DID_NOT_RUN = -6 +TIMEOUT = -7 _FORMAT_TEST_RESULT = { PASSED: '%s passed', @@ -35,6 +36,7 @@ INTERRUPTED: '%s interrupted', CHILD_ERROR: '%s crashed', TEST_DID_NOT_RUN: '%s run no tests', + TIMEOUT: '%s timed out', } # Minimum duration of a test to display its duration or to mention that @@ -75,7 +77,10 @@ def is_failed(result, ns): def format_test_result(result): fmt = _FORMAT_TEST_RESULT.get(result.result, "%s") - return fmt % result.test_name + text = fmt % result.test_name + if result.result == TIMEOUT: + text = '%s (%s)' % (text, format_duration(result.test_time)) + return text def findtestdir(path=None): @@ -179,6 +184,7 @@ def runtest(ns, test_name): FAILED test failed PASSED test passed EMPTY_TEST_SUITE test ran no subtests. + TIMEOUT test timed out. If ns.xmlpath is not None, xml_data is a list containing each generated testsuite element. diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py index aa2409b4ef79..eed643291162 100644 --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -13,7 +13,7 @@ from test.libregrtest.runtest import ( runtest, INTERRUPTED, CHILD_ERROR, PROGRESS_MIN_TIME, - format_test_result, TestResult, is_failed) + format_test_result, TestResult, is_failed, TIMEOUT) from test.libregrtest.setup import setup_tests from test.libregrtest.utils import format_duration @@ -103,11 +103,12 @@ class ExitThread(Exception): class MultiprocessThread(threading.Thread): - def __init__(self, pending, output, ns): + def __init__(self, pending, output, ns, timeout): super().__init__() self.pending = pending self.output = output self.ns = ns + self.timeout = timeout self.current_test_name = None self.start_time = None self._popen = None @@ -126,6 +127,12 @@ def __repr__(self): return '<%s>' % ' '.join(info) def kill(self): + """ + Kill the current process (if any). + + This method can be called by the thread running the process, + or by another thread. + """ self._killed = True popen = self._popen @@ -136,6 +143,13 @@ def kill(self): # does not hang popen.stdout.close() popen.stderr.close() + popen.wait() + + def mp_result_error(self, test_name, error_type, stdout='', stderr='', + err_msg=None): + test_time = time.monotonic() - self.start_time + result = TestResult(test_name, error_type, test_time, None) + return MultiprocessResult(result, stdout, stderr, err_msg) def _runtest(self, test_name): try: @@ -154,7 +168,19 @@ def _runtest(self, test_name): raise ExitThread try: + stdout, stderr = popen.communicate(timeout=self.timeout) + except subprocess.TimeoutExpired: + if self._killed: + # kill() has been called: communicate() fails + # on reading closed stdout/stderr + raise ExitThread + + popen.kill() stdout, stderr = popen.communicate() + self.kill() + + return self.mp_result_error(test_name, TIMEOUT, + stdout, stderr) except OSError: if self._killed: # kill() has been called: communicate() fails @@ -163,7 +189,6 @@ def _runtest(self, test_name): raise except: self.kill() - popen.wait() raise retcode = popen.wait() @@ -191,8 +216,7 @@ def _runtest(self, test_name): err_msg = "Failed to parse worker JSON: %s" % exc if err_msg is not None: - test_time = time.monotonic() - self.start_time - result = TestResult(test_name, CHILD_ERROR, test_time, None) + return self.mp_result_error(test_name, CHILD_ERROR, stdout, stderr, err_msg) return MultiprocessResult(result, stdout, stderr, err_msg) @@ -236,13 +260,16 @@ def __init__(self, regrtest): self.output = queue.Queue() self.pending = MultiprocessIterator(self.regrtest.tests) if self.ns.timeout is not None: - self.test_timeout = self.ns.timeout * 1.5 + self.worker_timeout = self.ns.timeout * 1.5 + self.main_timeout = self.ns.timeout * 2.0 else: - self.test_timeout = None + self.worker_timeout = None + self.main_timeout = None self.workers = None def start_workers(self): - self.workers = [MultiprocessThread(self.pending, self.output, self.ns) + self.workers = [MultiprocessThread(self.pending, self.output, + self.ns, self.worker_timeout) for _ in range(self.ns.use_mp)] print("Run tests in parallel using %s child processes" % len(self.workers)) @@ -274,8 +301,8 @@ def _get_result(self): return None while True: - if self.test_timeout is not None: - faulthandler.dump_traceback_later(self.test_timeout, exit=True) + if self.main_timeout is not None: + faulthandler.dump_traceback_later(self.main_timeout, exit=True) # wait for a thread timeout = max(PROGRESS_UPDATE, PROGRESS_MIN_TIME) @@ -343,7 +370,7 @@ def run_tests(self): print() self.regrtest.interrupted = True finally: - if self.test_timeout is not None: + if self.main_timeout is not None: faulthandler.cancel_dump_traceback_later() # a test failed (and --failfast is set) or all tests completed diff --git a/Lib/test/libregrtest/win_utils.py b/Lib/test/libregrtest/win_utils.py index adfe278ba39b..ec2d6c663e83 100644 --- a/Lib/test/libregrtest/win_utils.py +++ b/Lib/test/libregrtest/win_utils.py @@ -18,13 +18,14 @@ class WindowsLoadTracker(): """ This class asynchronously interacts with the `typeperf` command to read - the system load on Windows. Mulitprocessing and threads can't be used + the system load on Windows. Multiprocessing and threads can't be used here because they interfere with the test suite's cases for those modules. """ def __init__(self): self.load = 0.0 + self.p = None self.start() def start(self): diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py index 6b0b53142c04..7336adc4f1b8 100644 --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -1121,6 +1121,33 @@ def test_garbage(self): env_changed=[testname], fail_env_changed=True) + def test_multiprocessing_timeout(self): + code = textwrap.dedent(r""" + import time + import unittest + try: + import faulthandler + except ImportError: + faulthandler = None + + class Tests(unittest.TestCase): + # test hangs and so should be stopped by the timeout + def test_sleep(self): + # we want to test regrtest multiprocessing timeout, + # not faulthandler timeout + if faulthandler is not None: + faulthandler.cancel_dump_traceback_later() + + time.sleep(60 * 5) + """) + testname = self.create_test(code=code) + + output = self.run_tests("-j2", "--timeout=1.0", testname, exitcode=2) + self.check_executed_tests(output, [testname], + failed=testname) + self.assertRegex(output, + re.compile('%s timed out' % testname, re.MULTILINE)) + def test_cleanup(self): dirname = os.path.join(self.tmptestdir, "test_python_123") os.mkdir(dirname) diff --git a/Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst b/Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst new file mode 100644 index 000000000000..aaf1052bd3c1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst @@ -0,0 +1,2 @@ +"python3 -m test -jN --timeout=TIMEOUT" now kills a worker process if it runs +longer than *TIMEOUT* seconds. From webhook-mailer at python.org Wed Aug 14 10:31:36 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 14:31:36 -0000 Subject: [Python-checkins] [3.8] bpo-37531: Fix regrtest timeout for subprocesses (GH-15072) (GH-15279) Message-ID: https://github.com/python/cpython/commit/d85c5670ff1611202a25c9e0967148e72c114de9 commit: d85c5670ff1611202a25c9e0967148e72c114de9 branch: 3.8 author: Victor Stinner committer: GitHub date: 2019-08-14T16:31:32+02:00 summary: [3.8] bpo-37531: Fix regrtest timeout for subprocesses (GH-15072) (GH-15279) * bpo-37531: Fix regrtest timeout for subprocesses (GH-15072) Co-Authored-By: Joannah Nanjekye (cherry picked from commit b0c8369c603633f445ccbb5ca7a8742145ff9eec) * bpo-36511: Fix failures in Windows ARM32 buildbot (GH-15181) (cherry picked from commit ed70a344b5fbddea85726ebc1964ee0cfdef9c40) Backport also minor fixes from master (fix typo, remove importlib import). files: A Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst M Lib/test/libregrtest/__init__.py M Lib/test/libregrtest/main.py M Lib/test/libregrtest/runtest.py M Lib/test/libregrtest/runtest_mp.py M Lib/test/libregrtest/win_utils.py M Lib/test/test_regrtest.py diff --git a/Lib/test/libregrtest/__init__.py b/Lib/test/libregrtest/__init__.py index 3427b51b60af..5e8dba5dbde7 100644 --- a/Lib/test/libregrtest/__init__.py +++ b/Lib/test/libregrtest/__init__.py @@ -1,5 +1,2 @@ -# We import importlib *ASAP* in order to test #15386 -import importlib - from test.libregrtest.cmdline import _parse_args, RESOURCE_NAMES, ALL_RESOURCES from test.libregrtest.main import main diff --git a/Lib/test/libregrtest/main.py b/Lib/test/libregrtest/main.py index 87278d0c76a2..3cfbb4017dfb 100644 --- a/Lib/test/libregrtest/main.py +++ b/Lib/test/libregrtest/main.py @@ -14,7 +14,7 @@ from test.libregrtest.runtest import ( findtests, runtest, get_abs_module, STDTESTS, NOTTESTS, PASSED, FAILED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED, - INTERRUPTED, CHILD_ERROR, TEST_DID_NOT_RUN, + INTERRUPTED, CHILD_ERROR, TEST_DID_NOT_RUN, TIMEOUT, PROGRESS_MIN_TIME, format_test_result, is_failed) from test.libregrtest.setup import setup_tests from test.libregrtest.pgo import setup_pgo_tests @@ -115,6 +115,8 @@ def accumulate_result(self, result, rerun=False): self.run_no_tests.append(test_name) elif ok == INTERRUPTED: self.interrupted = True + elif ok == TIMEOUT: + self.bad.append(test_name) else: raise ValueError("invalid test result: %r" % ok) diff --git a/Lib/test/libregrtest/runtest.py b/Lib/test/libregrtest/runtest.py index a43b7666cd1e..e7dce180cb37 100644 --- a/Lib/test/libregrtest/runtest.py +++ b/Lib/test/libregrtest/runtest.py @@ -13,7 +13,7 @@ from test import support from test.libregrtest.refleak import dash_R, clear_caches from test.libregrtest.save_env import saved_test_environment -from test.libregrtest.utils import print_warning +from test.libregrtest.utils import format_duration, print_warning # Test result constants. @@ -25,6 +25,7 @@ INTERRUPTED = -4 CHILD_ERROR = -5 # error in a child process TEST_DID_NOT_RUN = -6 +TIMEOUT = -7 _FORMAT_TEST_RESULT = { PASSED: '%s passed', @@ -35,6 +36,7 @@ INTERRUPTED: '%s interrupted', CHILD_ERROR: '%s crashed', TEST_DID_NOT_RUN: '%s run no tests', + TIMEOUT: '%s timed out', } # Minimum duration of a test to display its duration or to mention that @@ -75,7 +77,10 @@ def is_failed(result, ns): def format_test_result(result): fmt = _FORMAT_TEST_RESULT.get(result.result, "%s") - return fmt % result.test_name + text = fmt % result.test_name + if result.result == TIMEOUT: + text = '%s (%s)' % (text, format_duration(result.test_time)) + return text def findtestdir(path=None): @@ -179,6 +184,7 @@ def runtest(ns, test_name): FAILED test failed PASSED test passed EMPTY_TEST_SUITE test ran no subtests. + TIMEOUT test timed out. If ns.xmlpath is not None, xml_data is a list containing each generated testsuite element. diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py index aa2409b4ef79..eed643291162 100644 --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -13,7 +13,7 @@ from test.libregrtest.runtest import ( runtest, INTERRUPTED, CHILD_ERROR, PROGRESS_MIN_TIME, - format_test_result, TestResult, is_failed) + format_test_result, TestResult, is_failed, TIMEOUT) from test.libregrtest.setup import setup_tests from test.libregrtest.utils import format_duration @@ -103,11 +103,12 @@ class ExitThread(Exception): class MultiprocessThread(threading.Thread): - def __init__(self, pending, output, ns): + def __init__(self, pending, output, ns, timeout): super().__init__() self.pending = pending self.output = output self.ns = ns + self.timeout = timeout self.current_test_name = None self.start_time = None self._popen = None @@ -126,6 +127,12 @@ def __repr__(self): return '<%s>' % ' '.join(info) def kill(self): + """ + Kill the current process (if any). + + This method can be called by the thread running the process, + or by another thread. + """ self._killed = True popen = self._popen @@ -136,6 +143,13 @@ def kill(self): # does not hang popen.stdout.close() popen.stderr.close() + popen.wait() + + def mp_result_error(self, test_name, error_type, stdout='', stderr='', + err_msg=None): + test_time = time.monotonic() - self.start_time + result = TestResult(test_name, error_type, test_time, None) + return MultiprocessResult(result, stdout, stderr, err_msg) def _runtest(self, test_name): try: @@ -154,7 +168,19 @@ def _runtest(self, test_name): raise ExitThread try: + stdout, stderr = popen.communicate(timeout=self.timeout) + except subprocess.TimeoutExpired: + if self._killed: + # kill() has been called: communicate() fails + # on reading closed stdout/stderr + raise ExitThread + + popen.kill() stdout, stderr = popen.communicate() + self.kill() + + return self.mp_result_error(test_name, TIMEOUT, + stdout, stderr) except OSError: if self._killed: # kill() has been called: communicate() fails @@ -163,7 +189,6 @@ def _runtest(self, test_name): raise except: self.kill() - popen.wait() raise retcode = popen.wait() @@ -191,8 +216,7 @@ def _runtest(self, test_name): err_msg = "Failed to parse worker JSON: %s" % exc if err_msg is not None: - test_time = time.monotonic() - self.start_time - result = TestResult(test_name, CHILD_ERROR, test_time, None) + return self.mp_result_error(test_name, CHILD_ERROR, stdout, stderr, err_msg) return MultiprocessResult(result, stdout, stderr, err_msg) @@ -236,13 +260,16 @@ def __init__(self, regrtest): self.output = queue.Queue() self.pending = MultiprocessIterator(self.regrtest.tests) if self.ns.timeout is not None: - self.test_timeout = self.ns.timeout * 1.5 + self.worker_timeout = self.ns.timeout * 1.5 + self.main_timeout = self.ns.timeout * 2.0 else: - self.test_timeout = None + self.worker_timeout = None + self.main_timeout = None self.workers = None def start_workers(self): - self.workers = [MultiprocessThread(self.pending, self.output, self.ns) + self.workers = [MultiprocessThread(self.pending, self.output, + self.ns, self.worker_timeout) for _ in range(self.ns.use_mp)] print("Run tests in parallel using %s child processes" % len(self.workers)) @@ -274,8 +301,8 @@ def _get_result(self): return None while True: - if self.test_timeout is not None: - faulthandler.dump_traceback_later(self.test_timeout, exit=True) + if self.main_timeout is not None: + faulthandler.dump_traceback_later(self.main_timeout, exit=True) # wait for a thread timeout = max(PROGRESS_UPDATE, PROGRESS_MIN_TIME) @@ -343,7 +370,7 @@ def run_tests(self): print() self.regrtest.interrupted = True finally: - if self.test_timeout is not None: + if self.main_timeout is not None: faulthandler.cancel_dump_traceback_later() # a test failed (and --failfast is set) or all tests completed diff --git a/Lib/test/libregrtest/win_utils.py b/Lib/test/libregrtest/win_utils.py index adfe278ba39b..ec2d6c663e83 100644 --- a/Lib/test/libregrtest/win_utils.py +++ b/Lib/test/libregrtest/win_utils.py @@ -18,13 +18,14 @@ class WindowsLoadTracker(): """ This class asynchronously interacts with the `typeperf` command to read - the system load on Windows. Mulitprocessing and threads can't be used + the system load on Windows. Multiprocessing and threads can't be used here because they interfere with the test suite's cases for those modules. """ def __init__(self): self.load = 0.0 + self.p = None self.start() def start(self): diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py index b569100c6777..0f44c7206975 100644 --- a/Lib/test/test_regrtest.py +++ b/Lib/test/test_regrtest.py @@ -1150,6 +1150,33 @@ def test_garbage(self): env_changed=[testname], fail_env_changed=True) + def test_multiprocessing_timeout(self): + code = textwrap.dedent(r""" + import time + import unittest + try: + import faulthandler + except ImportError: + faulthandler = None + + class Tests(unittest.TestCase): + # test hangs and so should be stopped by the timeout + def test_sleep(self): + # we want to test regrtest multiprocessing timeout, + # not faulthandler timeout + if faulthandler is not None: + faulthandler.cancel_dump_traceback_later() + + time.sleep(60 * 5) + """) + testname = self.create_test(code=code) + + output = self.run_tests("-j2", "--timeout=1.0", testname, exitcode=2) + self.check_executed_tests(output, [testname], + failed=testname) + self.assertRegex(output, + re.compile('%s timed out' % testname, re.MULTILINE)) + def test_cleanup(self): dirname = os.path.join(self.tmptestdir, "test_python_123") os.mkdir(dirname) diff --git a/Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst b/Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst new file mode 100644 index 000000000000..aaf1052bd3c1 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-09-19-38-26.bpo-37531.GX7s8S.rst @@ -0,0 +1,2 @@ +"python3 -m test -jN --timeout=TIMEOUT" now kills a worker process if it runs +longer than *TIMEOUT* seconds. From webhook-mailer at python.org Wed Aug 14 13:06:16 2019 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Wed, 14 Aug 2019 17:06:16 -0000 Subject: [Python-checkins] bpo-37849: IDLE: fix completion window positioning above line (GH-15267) Message-ID: https://github.com/python/cpython/commit/71662dc2f12a7e77e5e1dfe64ec87c1b459c3f59 commit: 71662dc2f12a7e77e5e1dfe64ec87c1b459c3f59 branch: master author: Tal Einat committer: Terry Jan Reedy date: 2019-08-14T13:06:06-04:00 summary: bpo-37849: IDLE: fix completion window positioning above line (GH-15267) files: A Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/autocomplete_w.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 0fff7003b9d6..45918cf36b41 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,11 @@ Released on 2019-10-20? ====================================== +bpo-37849: Fix completions list appearing too high or low when shown +above the current line. + +bpo-36419: Refactor autocompete and improve testing. + bpo-37748: Reorder the Run menu. Put the most common choice, Run Module, at the top. diff --git a/Lib/idlelib/autocomplete_w.py b/Lib/idlelib/autocomplete_w.py index c69ab4a36836..5035e067392e 100644 --- a/Lib/idlelib/autocomplete_w.py +++ b/Lib/idlelib/autocomplete_w.py @@ -52,10 +52,12 @@ def __init__(self, widget): # (for example, he clicked the list) self.userwantswindow = None # event ids - self.hideid = self.keypressid = self.listupdateid = self.winconfigid \ - = self.keyreleaseid = self.doubleclickid = None + self.hideid = self.keypressid = self.listupdateid = \ + self.winconfigid = self.keyreleaseid = self.doubleclickid = None # Flag set if last keypress was a tab self.lastkey_was_tab = False + # Flag set to avoid recursive callback invocations. + self.is_configuring = False def _change_start(self, newstart): min_len = min(len(self.start), len(newstart)) @@ -223,12 +225,18 @@ def show_window(self, comp_lists, index, complete, mode, userWantsWin): self.widget.event_add(KEYRELEASE_VIRTUAL_EVENT_NAME,KEYRELEASE_SEQUENCE) self.listupdateid = listbox.bind(LISTUPDATE_SEQUENCE, self.listselect_event) + self.is_configuring = False self.winconfigid = acw.bind(WINCONFIG_SEQUENCE, self.winconfig_event) self.doubleclickid = listbox.bind(DOUBLECLICK_SEQUENCE, self.doubleclick_event) return None def winconfig_event(self, event): + if self.is_configuring: + # Avoid running on recursive callback invocations. + return + + self.is_configuring = True if not self.is_active(): return # Position the completion list window @@ -236,6 +244,7 @@ def winconfig_event(self, event): text.see(self.startindex) x, y, cx, cy = text.bbox(self.startindex) acw = self.autocompletewindow + acw.update() acw_width, acw_height = acw.winfo_width(), acw.winfo_height() text_width, text_height = text.winfo_width(), text.winfo_height() new_x = text.winfo_rootx() + min(x, max(0, text_width - acw_width)) @@ -256,6 +265,8 @@ def winconfig_event(self, event): acw.unbind(WINCONFIG_SEQUENCE, self.winconfigid) self.winconfigid = None + self.is_configuring = False + def _hide_event_check(self): if not self.autocompletewindow: return diff --git a/Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst b/Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst new file mode 100644 index 000000000000..9f700d9031f1 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst @@ -0,0 +1,2 @@ +Fixed completions list appearing too high or low when shown above +the current line. From webhook-mailer at python.org Wed Aug 14 13:24:08 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 17:24:08 -0000 Subject: [Python-checkins] bpo-37849: IDLE: fix completion window positioning above line (GH-15267) Message-ID: https://github.com/python/cpython/commit/557802dc17498557e481f2ad3d4a83ece469e489 commit: 557802dc17498557e481f2ad3d4a83ece469e489 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-14T10:24:04-07:00 summary: bpo-37849: IDLE: fix completion window positioning above line (GH-15267) (cherry picked from commit 71662dc2f12a7e77e5e1dfe64ec87c1b459c3f59) Co-authored-by: Tal Einat files: A Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/autocomplete_w.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 0fff7003b9d6..45918cf36b41 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,11 @@ Released on 2019-10-20? ====================================== +bpo-37849: Fix completions list appearing too high or low when shown +above the current line. + +bpo-36419: Refactor autocompete and improve testing. + bpo-37748: Reorder the Run menu. Put the most common choice, Run Module, at the top. diff --git a/Lib/idlelib/autocomplete_w.py b/Lib/idlelib/autocomplete_w.py index c69ab4a36836..5035e067392e 100644 --- a/Lib/idlelib/autocomplete_w.py +++ b/Lib/idlelib/autocomplete_w.py @@ -52,10 +52,12 @@ def __init__(self, widget): # (for example, he clicked the list) self.userwantswindow = None # event ids - self.hideid = self.keypressid = self.listupdateid = self.winconfigid \ - = self.keyreleaseid = self.doubleclickid = None + self.hideid = self.keypressid = self.listupdateid = \ + self.winconfigid = self.keyreleaseid = self.doubleclickid = None # Flag set if last keypress was a tab self.lastkey_was_tab = False + # Flag set to avoid recursive callback invocations. + self.is_configuring = False def _change_start(self, newstart): min_len = min(len(self.start), len(newstart)) @@ -223,12 +225,18 @@ def show_window(self, comp_lists, index, complete, mode, userWantsWin): self.widget.event_add(KEYRELEASE_VIRTUAL_EVENT_NAME,KEYRELEASE_SEQUENCE) self.listupdateid = listbox.bind(LISTUPDATE_SEQUENCE, self.listselect_event) + self.is_configuring = False self.winconfigid = acw.bind(WINCONFIG_SEQUENCE, self.winconfig_event) self.doubleclickid = listbox.bind(DOUBLECLICK_SEQUENCE, self.doubleclick_event) return None def winconfig_event(self, event): + if self.is_configuring: + # Avoid running on recursive callback invocations. + return + + self.is_configuring = True if not self.is_active(): return # Position the completion list window @@ -236,6 +244,7 @@ def winconfig_event(self, event): text.see(self.startindex) x, y, cx, cy = text.bbox(self.startindex) acw = self.autocompletewindow + acw.update() acw_width, acw_height = acw.winfo_width(), acw.winfo_height() text_width, text_height = text.winfo_width(), text.winfo_height() new_x = text.winfo_rootx() + min(x, max(0, text_width - acw_width)) @@ -256,6 +265,8 @@ def winconfig_event(self, event): acw.unbind(WINCONFIG_SEQUENCE, self.winconfigid) self.winconfigid = None + self.is_configuring = False + def _hide_event_check(self): if not self.autocompletewindow: return diff --git a/Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst b/Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst new file mode 100644 index 000000000000..9f700d9031f1 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst @@ -0,0 +1,2 @@ +Fixed completions list appearing too high or low when shown above +the current line. From webhook-mailer at python.org Wed Aug 14 13:37:53 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 17:37:53 -0000 Subject: [Python-checkins] bpo-37849: IDLE: fix completion window positioning above line (GH-15267) Message-ID: https://github.com/python/cpython/commit/88cce7b59fa503616295ca1a25bce6251f9cd317 commit: 88cce7b59fa503616295ca1a25bce6251f9cd317 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-14T10:37:49-07:00 summary: bpo-37849: IDLE: fix completion window positioning above line (GH-15267) (cherry picked from commit 71662dc2f12a7e77e5e1dfe64ec87c1b459c3f59) Co-authored-by: Tal Einat files: A Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/autocomplete_w.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index d4a00027f858..b579c2c1a6dd 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,11 @@ Released on 2019-09-30? ====================================== +bpo-37849: Fix completions list appearing too high or low when shown +above the current line. + +bpo-36419: Refactor autocompete and improve testing. + bpo-37748: Reorder the Run menu. Put the most common choice, Run Module, at the top. diff --git a/Lib/idlelib/autocomplete_w.py b/Lib/idlelib/autocomplete_w.py index c69ab4a36836..5035e067392e 100644 --- a/Lib/idlelib/autocomplete_w.py +++ b/Lib/idlelib/autocomplete_w.py @@ -52,10 +52,12 @@ def __init__(self, widget): # (for example, he clicked the list) self.userwantswindow = None # event ids - self.hideid = self.keypressid = self.listupdateid = self.winconfigid \ - = self.keyreleaseid = self.doubleclickid = None + self.hideid = self.keypressid = self.listupdateid = \ + self.winconfigid = self.keyreleaseid = self.doubleclickid = None # Flag set if last keypress was a tab self.lastkey_was_tab = False + # Flag set to avoid recursive callback invocations. + self.is_configuring = False def _change_start(self, newstart): min_len = min(len(self.start), len(newstart)) @@ -223,12 +225,18 @@ def show_window(self, comp_lists, index, complete, mode, userWantsWin): self.widget.event_add(KEYRELEASE_VIRTUAL_EVENT_NAME,KEYRELEASE_SEQUENCE) self.listupdateid = listbox.bind(LISTUPDATE_SEQUENCE, self.listselect_event) + self.is_configuring = False self.winconfigid = acw.bind(WINCONFIG_SEQUENCE, self.winconfig_event) self.doubleclickid = listbox.bind(DOUBLECLICK_SEQUENCE, self.doubleclick_event) return None def winconfig_event(self, event): + if self.is_configuring: + # Avoid running on recursive callback invocations. + return + + self.is_configuring = True if not self.is_active(): return # Position the completion list window @@ -236,6 +244,7 @@ def winconfig_event(self, event): text.see(self.startindex) x, y, cx, cy = text.bbox(self.startindex) acw = self.autocompletewindow + acw.update() acw_width, acw_height = acw.winfo_width(), acw.winfo_height() text_width, text_height = text.winfo_width(), text.winfo_height() new_x = text.winfo_rootx() + min(x, max(0, text_width - acw_width)) @@ -256,6 +265,8 @@ def winconfig_event(self, event): acw.unbind(WINCONFIG_SEQUENCE, self.winconfigid) self.winconfigid = None + self.is_configuring = False + def _hide_event_check(self): if not self.autocompletewindow: return diff --git a/Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst b/Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst new file mode 100644 index 000000000000..9f700d9031f1 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2019-08-14-09-43-15.bpo-37849.-bcYF3.rst @@ -0,0 +1,2 @@ +Fixed completions list appearing too high or low when shown above +the current line. From webhook-mailer at python.org Wed Aug 14 17:11:37 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 21:11:37 -0000 Subject: [Python-checkins] bpo-37826: Document exception chaining in Python tutorial for errors. (GH-15243) Message-ID: https://github.com/python/cpython/commit/dcfe111eb5602333135b8776996332a8dcf59392 commit: dcfe111eb5602333135b8776996332a8dcf59392 branch: master author: Abhilash Raj committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-14T14:11:32-07:00 summary: bpo-37826: Document exception chaining in Python tutorial for errors. (GH-15243) https://bugs.python.org/issue37826 files: M Doc/tutorial/errors.rst diff --git a/Doc/tutorial/errors.rst b/Doc/tutorial/errors.rst index 4e287bbd8d29..e9a63e425279 100644 --- a/Doc/tutorial/errors.rst +++ b/Doc/tutorial/errors.rst @@ -267,6 +267,53 @@ re-raise the exception:: NameError: HiThere +.. _tut-exception-chaining: + +Exception Chaining +================== + +The :keyword:`raise` statement allows an optional :keyword:`from` which enables +chaining exceptions by setting the ``__cause__`` attribute of the raised +exception. For example:: + + raise RuntimeError from OSError + +This can be useful when you are transforming exceptions. For example:: + + >>> def func(): + ... raise IOError + ... + >>> try: + ... func() + ... except IOError as exc: + ... raise RuntimeError('Failed to open database') from exc + ... + Traceback (most recent call last): + File "", line 2, in + File "", line 2, in func + OSError + + The above exception was the direct cause of the following exception: + + Traceback (most recent call last): + File "", line 4, in + RuntimeError + +The expression following the :keyword:`from` must be either an exception or +``None``. Exception chaining happens automatically when an exception is raised +inside an exception handler or :keyword:`finally` section. Exception chaining +can be disabled by using ``from None`` idiom: + + >>> try: + ... open('database.sqlite') + ... except IOError: + ... raise RuntimeError from None + ... + Traceback (most recent call last): + File "", line 4, in + RuntimeError + + .. _tut-userexceptions: User-defined Exceptions From webhook-mailer at python.org Wed Aug 14 17:21:52 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 21:21:52 -0000 Subject: [Python-checkins] bpo-37811: FreeBSD, OSX: fix poll(2) usage in sockets module (GH-15202) Message-ID: https://github.com/python/cpython/commit/28146206578ebe1b84b48e6f255738a227058c04 commit: 28146206578ebe1b84b48e6f255738a227058c04 branch: master author: Artem Khramov committer: Victor Stinner date: 2019-08-14T23:21:48+02:00 summary: bpo-37811: FreeBSD, OSX: fix poll(2) usage in sockets module (GH-15202) FreeBSD implementation of poll(2) restricts the timeout argument to be either zero, or positive, or equal to INFTIM (-1). Unless otherwise overridden, socket timeout defaults to -1. This value is then converted to milliseconds (-1000) and used as argument to the poll syscall. poll returns EINVAL (22), and the connection fails. This bug was discovered during the EINTR handling testing, and the reproduction code can be found in https://bugs.python.org/issue23618 (see connect_eintr.py, attached). On GNU/Linux, the example runs as expected. This change is trivial: If the supplied timeout value is negative, truncate it to -1. files: A Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst M Misc/ACKS M Modules/socketmodule.c diff --git a/Misc/ACKS b/Misc/ACKS index 3b4cf85c75fe..52a5d70e5e2d 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -837,6 +837,7 @@ Lawrence Kesteloot Garvit Khatri Vivek Khera Dhiru Kholia +Artem Khramov Akshit Khurana Sanyam Khurana Mads Kiilerich diff --git a/Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst b/Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst new file mode 100644 index 000000000000..662e7dc41005 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst @@ -0,0 +1,4 @@ +Fix ``socket`` module's ``socket.connect(address)`` function being unable to +establish connection in case of interrupted system call. The problem was +observed on all OSes which ``poll(2)`` system call can take only +non-negative integers and -1 as a timeout value. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index f220c2636319..d4f2098e1e6f 100755 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -789,6 +789,17 @@ internal_select(PySocketSockObject *s, int writing, _PyTime_t interval, ms = _PyTime_AsMilliseconds(interval, _PyTime_ROUND_CEILING); assert(ms <= INT_MAX); + /* On some OSes, typically BSD-based ones, the timeout parameter of the + poll() syscall, when negative, must be exactly INFTIM, where defined, + or -1. See issue 37811. */ + if (ms < 0) { +#ifdef INFTIM + ms = INFTIM; +#else + ms = -1; +#endif + } + Py_BEGIN_ALLOW_THREADS; n = poll(&pollfd, 1, (int)ms); Py_END_ALLOW_THREADS; From webhook-mailer at python.org Wed Aug 14 17:35:31 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 21:35:31 -0000 Subject: [Python-checkins] bpo-21131: Fix faulthandler.register(chain=True) stack (GH-15276) Message-ID: https://github.com/python/cpython/commit/ac827edc493d3ac3f5b9b0cc353df1d4b418a9aa commit: ac827edc493d3ac3f5b9b0cc353df1d4b418a9aa branch: master author: Victor Stinner committer: GitHub date: 2019-08-14T23:35:27+02:00 summary: bpo-21131: Fix faulthandler.register(chain=True) stack (GH-15276) faulthandler now allocates a dedicated stack of SIGSTKSZ*2 bytes, instead of just SIGSTKSZ bytes. Calling the previous signal handler in faulthandler signal handler uses more than SIGSTKSZ bytes of stack memory on some platforms. files: A Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst M Modules/faulthandler.c diff --git a/Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst b/Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst new file mode 100644 index 000000000000..d330aca1c17d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst @@ -0,0 +1,4 @@ +Fix ``faulthandler.register(chain=True)`` stack. faulthandler now allocates a +dedicated stack of ``SIGSTKSZ*2`` bytes, instead of just ``SIGSTKSZ`` bytes. +Calling the previous signal handler in faulthandler signal handler uses more +than ``SIGSTKSZ`` bytes of stack memory on some platforms. diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c index 2331051f7907..5dbbcad057e6 100644 --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -1325,7 +1325,11 @@ _PyFaulthandler_Init(int enable) * be able to allocate memory on the stack, even on a stack overflow. If it * fails, ignore the error. */ stack.ss_flags = 0; - stack.ss_size = SIGSTKSZ; + /* bpo-21131: allocate dedicated stack of SIGSTKSZ*2 bytes, instead of just + SIGSTKSZ bytes. Calling the previous signal handler in faulthandler + signal handler uses more than SIGSTKSZ bytes of stack memory on some + platforms. */ + stack.ss_size = SIGSTKSZ * 2; stack.ss_sp = PyMem_Malloc(stack.ss_size); if (stack.ss_sp != NULL) { err = sigaltstack(&stack, &old_stack); From webhook-mailer at python.org Wed Aug 14 17:47:51 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 21:47:51 -0000 Subject: [Python-checkins] bpo-37811: FreeBSD, OSX: fix poll(2) usage in sockets module (GH-15202) Message-ID: https://github.com/python/cpython/commit/123f6c4914827c4ced65d032fab74de62db31cd6 commit: 123f6c4914827c4ced65d032fab74de62db31cd6 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-14T14:47:43-07:00 summary: bpo-37811: FreeBSD, OSX: fix poll(2) usage in sockets module (GH-15202) FreeBSD implementation of poll(2) restricts the timeout argument to be either zero, or positive, or equal to INFTIM (-1). Unless otherwise overridden, socket timeout defaults to -1. This value is then converted to milliseconds (-1000) and used as argument to the poll syscall. poll returns EINVAL (22), and the connection fails. This bug was discovered during the EINTR handling testing, and the reproduction code can be found in https://bugs.python.org/issue23618 (see connect_eintr.py, attached). On GNU/Linux, the example runs as expected. This change is trivial: If the supplied timeout value is negative, truncate it to -1. (cherry picked from commit 28146206578ebe1b84b48e6f255738a227058c04) Co-authored-by: Artem Khramov files: A Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst M Misc/ACKS M Modules/socketmodule.c diff --git a/Misc/ACKS b/Misc/ACKS index 3e53429bf237..311259f1e5fa 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -832,6 +832,7 @@ Lawrence Kesteloot Garvit Khatri Vivek Khera Dhiru Kholia +Artem Khramov Akshit Khurana Sanyam Khurana Mads Kiilerich diff --git a/Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst b/Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst new file mode 100644 index 000000000000..662e7dc41005 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst @@ -0,0 +1,4 @@ +Fix ``socket`` module's ``socket.connect(address)`` function being unable to +establish connection in case of interrupted system call. The problem was +observed on all OSes which ``poll(2)`` system call can take only +non-negative integers and -1 as a timeout value. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index 0470f387e3bc..910e2bdd1319 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -780,6 +780,17 @@ internal_select(PySocketSockObject *s, int writing, _PyTime_t interval, ms = _PyTime_AsMilliseconds(interval, _PyTime_ROUND_CEILING); assert(ms <= INT_MAX); + /* On some OSes, typically BSD-based ones, the timeout parameter of the + poll() syscall, when negative, must be exactly INFTIM, where defined, + or -1. See issue 37811. */ + if (ms < 0) { +#ifdef INFTIM + ms = INFTIM; +#else + ms = -1; +#endif + } + Py_BEGIN_ALLOW_THREADS; n = poll(&pollfd, 1, (int)ms); Py_END_ALLOW_THREADS; From webhook-mailer at python.org Wed Aug 14 17:48:09 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 21:48:09 -0000 Subject: [Python-checkins] bpo-37811: FreeBSD, OSX: fix poll(2) usage in sockets module (GH-15202) Message-ID: https://github.com/python/cpython/commit/b0b178a2b80974da910ce6a344d66cc4d9a2fcfa commit: b0b178a2b80974da910ce6a344d66cc4d9a2fcfa branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-14T14:48:03-07:00 summary: bpo-37811: FreeBSD, OSX: fix poll(2) usage in sockets module (GH-15202) FreeBSD implementation of poll(2) restricts the timeout argument to be either zero, or positive, or equal to INFTIM (-1). Unless otherwise overridden, socket timeout defaults to -1. This value is then converted to milliseconds (-1000) and used as argument to the poll syscall. poll returns EINVAL (22), and the connection fails. This bug was discovered during the EINTR handling testing, and the reproduction code can be found in https://bugs.python.org/issue23618 (see connect_eintr.py, attached). On GNU/Linux, the example runs as expected. This change is trivial: If the supplied timeout value is negative, truncate it to -1. (cherry picked from commit 28146206578ebe1b84b48e6f255738a227058c04) Co-authored-by: Artem Khramov files: A Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst M Misc/ACKS M Modules/socketmodule.c diff --git a/Misc/ACKS b/Misc/ACKS index 29b6690f2174..4d8f96acfe30 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -815,6 +815,7 @@ Lawrence Kesteloot Garvit Khatri Vivek Khera Dhiru Kholia +Artem Khramov Akshit Khurana Sanyam Khurana Mads Kiilerich diff --git a/Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst b/Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst new file mode 100644 index 000000000000..662e7dc41005 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-14-21-41-07.bpo-37811.d1xYj7.rst @@ -0,0 +1,4 @@ +Fix ``socket`` module's ``socket.connect(address)`` function being unable to +establish connection in case of interrupted system call. The problem was +observed on all OSes which ``poll(2)`` system call can take only +non-negative integers and -1 as a timeout value. diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c index 65385e8974cf..a9ef7e2083d5 100644 --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -741,6 +741,17 @@ internal_select(PySocketSockObject *s, int writing, _PyTime_t interval, ms = _PyTime_AsMilliseconds(interval, _PyTime_ROUND_CEILING); assert(ms <= INT_MAX); + /* On some OSes, typically BSD-based ones, the timeout parameter of the + poll() syscall, when negative, must be exactly INFTIM, where defined, + or -1. See issue 37811. */ + if (ms < 0) { +#ifdef INFTIM + ms = INFTIM; +#else + ms = -1; +#endif + } + Py_BEGIN_ALLOW_THREADS; n = poll(&pollfd, 1, (int)ms); Py_END_ALLOW_THREADS; From webhook-mailer at python.org Wed Aug 14 18:02:16 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 22:02:16 -0000 Subject: [Python-checkins] bpo-21131: Fix faulthandler.register(chain=True) stack (GH-15276) Message-ID: https://github.com/python/cpython/commit/b8e682427a80798fec90dce31392beaf616c3e37 commit: b8e682427a80798fec90dce31392beaf616c3e37 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-14T15:02:12-07:00 summary: bpo-21131: Fix faulthandler.register(chain=True) stack (GH-15276) faulthandler now allocates a dedicated stack of SIGSTKSZ*2 bytes, instead of just SIGSTKSZ bytes. Calling the previous signal handler in faulthandler signal handler uses more than SIGSTKSZ bytes of stack memory on some platforms. (cherry picked from commit ac827edc493d3ac3f5b9b0cc353df1d4b418a9aa) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst M Modules/faulthandler.c diff --git a/Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst b/Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst new file mode 100644 index 000000000000..d330aca1c17d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst @@ -0,0 +1,4 @@ +Fix ``faulthandler.register(chain=True)`` stack. faulthandler now allocates a +dedicated stack of ``SIGSTKSZ*2`` bytes, instead of just ``SIGSTKSZ`` bytes. +Calling the previous signal handler in faulthandler signal handler uses more +than ``SIGSTKSZ`` bytes of stack memory on some platforms. diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c index f2b5a503f4ee..7b325996b268 100644 --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -1325,7 +1325,11 @@ _PyFaulthandler_Init(int enable) * be able to allocate memory on the stack, even on a stack overflow. If it * fails, ignore the error. */ stack.ss_flags = 0; - stack.ss_size = SIGSTKSZ; + /* bpo-21131: allocate dedicated stack of SIGSTKSZ*2 bytes, instead of just + SIGSTKSZ bytes. Calling the previous signal handler in faulthandler + signal handler uses more than SIGSTKSZ bytes of stack memory on some + platforms. */ + stack.ss_size = SIGSTKSZ * 2; stack.ss_sp = PyMem_Malloc(stack.ss_size); if (stack.ss_sp != NULL) { err = sigaltstack(&stack, &old_stack); From webhook-mailer at python.org Wed Aug 14 18:03:15 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 14 Aug 2019 22:03:15 -0000 Subject: [Python-checkins] bpo-37775: Update compileall doc for invalidation_mode parameter (GH-15148) Message-ID: https://github.com/python/cpython/commit/68e495df909a33e719e3f1ef5b4893ec785e10a4 commit: 68e495df909a33e719e3f1ef5b4893ec785e10a4 branch: master author: Hai Shi committer: Victor Stinner date: 2019-08-15T00:03:11+02:00 summary: bpo-37775: Update compileall doc for invalidation_mode parameter (GH-15148) files: M Doc/library/compileall.rst diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst index bb5000a0736c..9ce5ca819c68 100644 --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -120,7 +120,7 @@ runtime. Public functions ---------------- -.. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP) +.. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None) Recursively descend the directory tree named by *dir*, compiling all :file:`.py` files along the way. Return a true value if all the files compiled successfully, @@ -185,10 +185,13 @@ Public functions .. versionchanged:: 3.7 The *invalidation_mode* parameter was added. + .. versionchanged:: 3.7.2 + The *invalidation_mode* parameter's default value is updated to None. + .. versionchanged:: 3.8 Setting *workers* to 0 now chooses the optimal number of cores. -.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP) +.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None) Compile the file with path *fullname*. Return a true value if the file compiled successfully, and a false value otherwise. @@ -232,7 +235,10 @@ Public functions .. versionchanged:: 3.7 The *invalidation_mode* parameter was added. -.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP) + .. versionchanged:: 3.7.2 + The *invalidation_mode* parameter's default value is updated to None. + +.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None) Byte-compile all the :file:`.py` files found along ``sys.path``. Return a true value if all the files compiled successfully, and a false value otherwise. @@ -255,6 +261,9 @@ Public functions .. versionchanged:: 3.7 The *invalidation_mode* parameter was added. + .. versionchanged:: 3.7.2 + The *invalidation_mode* parameter's default value is updated to None. + To force a recompile of all the :file:`.py` files in the :file:`Lib/` subdirectory and all its subdirectories:: From webhook-mailer at python.org Wed Aug 14 18:09:43 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 22:09:43 -0000 Subject: [Python-checkins] bpo-21131: Fix faulthandler.register(chain=True) stack (GH-15276) Message-ID: https://github.com/python/cpython/commit/1581d9c405f140491791a07dca3f6166bc499ec1 commit: 1581d9c405f140491791a07dca3f6166bc499ec1 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-14T15:09:39-07:00 summary: bpo-21131: Fix faulthandler.register(chain=True) stack (GH-15276) faulthandler now allocates a dedicated stack of SIGSTKSZ*2 bytes, instead of just SIGSTKSZ bytes. Calling the previous signal handler in faulthandler signal handler uses more than SIGSTKSZ bytes of stack memory on some platforms. (cherry picked from commit ac827edc493d3ac3f5b9b0cc353df1d4b418a9aa) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst M Modules/faulthandler.c diff --git a/Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst b/Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst new file mode 100644 index 000000000000..d330aca1c17d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-14-15-34-23.bpo-21131.0MMQRi.rst @@ -0,0 +1,4 @@ +Fix ``faulthandler.register(chain=True)`` stack. faulthandler now allocates a +dedicated stack of ``SIGSTKSZ*2`` bytes, instead of just ``SIGSTKSZ`` bytes. +Calling the previous signal handler in faulthandler signal handler uses more +than ``SIGSTKSZ`` bytes of stack memory on some platforms. diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c index 0dbd5a3342b0..97c64f633091 100644 --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -1322,7 +1322,11 @@ _PyFaulthandler_Init(int enable) * be able to allocate memory on the stack, even on a stack overflow. If it * fails, ignore the error. */ stack.ss_flags = 0; - stack.ss_size = SIGSTKSZ; + /* bpo-21131: allocate dedicated stack of SIGSTKSZ*2 bytes, instead of just + SIGSTKSZ bytes. Calling the previous signal handler in faulthandler + signal handler uses more than SIGSTKSZ bytes of stack memory on some + platforms. */ + stack.ss_size = SIGSTKSZ * 2; stack.ss_sp = PyMem_Malloc(stack.ss_size); if (stack.ss_sp != NULL) { err = sigaltstack(&stack, &old_stack); From webhook-mailer at python.org Wed Aug 14 18:22:06 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 14 Aug 2019 22:22:06 -0000 Subject: [Python-checkins] bpo-37775: Update compileall doc for invalidation_mode parameter (GH-15148) Message-ID: https://github.com/python/cpython/commit/dbe4c286ce28402c3bce71d568ae55b91280e777 commit: dbe4c286ce28402c3bce71d568ae55b91280e777 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-14T15:22:02-07:00 summary: bpo-37775: Update compileall doc for invalidation_mode parameter (GH-15148) (cherry picked from commit 68e495df909a33e719e3f1ef5b4893ec785e10a4) Co-authored-by: Hai Shi files: M Doc/library/compileall.rst diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst index bb5000a0736c..9ce5ca819c68 100644 --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -120,7 +120,7 @@ runtime. Public functions ---------------- -.. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP) +.. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None) Recursively descend the directory tree named by *dir*, compiling all :file:`.py` files along the way. Return a true value if all the files compiled successfully, @@ -185,10 +185,13 @@ Public functions .. versionchanged:: 3.7 The *invalidation_mode* parameter was added. + .. versionchanged:: 3.7.2 + The *invalidation_mode* parameter's default value is updated to None. + .. versionchanged:: 3.8 Setting *workers* to 0 now chooses the optimal number of cores. -.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP) +.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None) Compile the file with path *fullname*. Return a true value if the file compiled successfully, and a false value otherwise. @@ -232,7 +235,10 @@ Public functions .. versionchanged:: 3.7 The *invalidation_mode* parameter was added. -.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP) + .. versionchanged:: 3.7.2 + The *invalidation_mode* parameter's default value is updated to None. + +.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None) Byte-compile all the :file:`.py` files found along ``sys.path``. Return a true value if all the files compiled successfully, and a false value otherwise. @@ -255,6 +261,9 @@ Public functions .. versionchanged:: 3.7 The *invalidation_mode* parameter was added. + .. versionchanged:: 3.7.2 + The *invalidation_mode* parameter's default value is updated to None. + To force a recompile of all the :file:`.py` files in the :file:`Lib/` subdirectory and all its subdirectories:: From webhook-mailer at python.org Wed Aug 14 21:19:00 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Thu, 15 Aug 2019 01:19:00 -0000 Subject: [Python-checkins] bpo-37760: Avoid cluttering work tree with downloaded Unicode files. (GH-15128) Message-ID: https://github.com/python/cpython/commit/3e4498d35c34aeaf4a9c3d57509b0d3277048ac6 commit: 3e4498d35c34aeaf4a9c3d57509b0d3277048ac6 branch: master author: Greg Price committer: Benjamin Peterson date: 2019-08-14T18:18:53-07:00 summary: bpo-37760: Avoid cluttering work tree with downloaded Unicode files. (GH-15128) files: M .gitignore M Tools/unicode/makeunicodedata.py diff --git a/.gitignore b/.gitignore index 9445ef1e2c52..5f1ba0b92ceb 100644 --- a/.gitignore +++ b/.gitignore @@ -74,6 +74,7 @@ PCbuild/arm32/ PCbuild/arm64/ PCbuild/obj/ PCbuild/win32/ +Tools/unicode/data/ .purify __pycache__ autom4te.cache diff --git a/Tools/unicode/makeunicodedata.py b/Tools/unicode/makeunicodedata.py index cc2b2981ef5b..464a4ebf7722 100644 --- a/Tools/unicode/makeunicodedata.py +++ b/Tools/unicode/makeunicodedata.py @@ -887,15 +887,18 @@ class Difference(Exception):pass normalization_changes)) +DATA_DIR = os.path.join('Tools', 'unicode', 'data') + def open_data(template, version): - local = template % ('-'+version,) + local = os.path.join(DATA_DIR, template % ('-'+version,)) if not os.path.exists(local): import urllib.request if version == '3.2.0': # irregular url structure - url = 'http://www.unicode.org/Public/3.2-Update/' + local + url = ('http://www.unicode.org/Public/3.2-Update/'+template) % ('-'+version,) else: url = ('http://www.unicode.org/Public/%s/ucd/'+template) % (version, '') + os.makedirs(DATA_DIR, exist_ok=True) urllib.request.urlretrieve(url, filename=local) if local.endswith('.txt'): return open(local, encoding='utf-8') From webhook-mailer at python.org Thu Aug 15 08:31:38 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 15 Aug 2019 12:31:38 -0000 Subject: [Python-checkins] Replace usage of the obscure PEM_read_bio_X509_AUX with the more standard PEM_read_bio_X509 (GH-15303) Message-ID: https://github.com/python/cpython/commit/40dad9545aad4ede89abbab1c1beef5303d9573e commit: 40dad9545aad4ede89abbab1c1beef5303d9573e branch: master author: Alex Gaynor committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-15T05:31:28-07:00 summary: Replace usage of the obscure PEM_read_bio_X509_AUX with the more standard PEM_read_bio_X509 (GH-15303) X509_AUX is an odd, note widely used, OpenSSL extension to the X509 file format. This function doesn't actually use any of the extra metadata that it parses, so just use the standard API. Automerge-Triggered-By: @tiran files: M Modules/_ssl.c diff --git a/Modules/_ssl.c b/Modules/_ssl.c index 3d54b844fe07..3d63612168b2 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1822,7 +1822,7 @@ _ssl__test_decode_cert_impl(PyObject *module, PyObject *path) goto fail0; } - x = PEM_read_bio_X509_AUX(cert,NULL, NULL, NULL); + x = PEM_read_bio_X509(cert, NULL, NULL, NULL); if (x == NULL) { PyErr_SetString(PySSLErrorObject, "Error decoding PEM-encoded file"); From webhook-mailer at python.org Thu Aug 15 08:52:57 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 15 Aug 2019 12:52:57 -0000 Subject: [Python-checkins] [3.8] Replace usage of the obscure PEM_read_bio_X509_AUX with the more standard PEM_read_bio_X509 (GH-15303) (GH-15304) Message-ID: https://github.com/python/cpython/commit/f781283ff6042fa5bc220a1572effc38b545eb20 commit: f781283ff6042fa5bc220a1572effc38b545eb20 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-15T05:52:51-07:00 summary: [3.8] Replace usage of the obscure PEM_read_bio_X509_AUX with the more standard PEM_read_bio_X509 (GH-15303) (GH-15304) X509_AUX is an odd, note widely used, OpenSSL extension to the X509 file format. This function doesn't actually use any of the extra metadata that it parses, so just use the standard API. Automerge-Triggered-By: @tiran (cherry picked from commit 40dad9545aad4ede89abbab1c1beef5303d9573e) Co-authored-by: Alex Gaynor Automerge-Triggered-By: @tiran files: M Modules/_ssl.c diff --git a/Modules/_ssl.c b/Modules/_ssl.c index da30cbb758e2..089aa3b24a02 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1822,7 +1822,7 @@ _ssl__test_decode_cert_impl(PyObject *module, PyObject *path) goto fail0; } - x = PEM_read_bio_X509_AUX(cert,NULL, NULL, NULL); + x = PEM_read_bio_X509(cert, NULL, NULL, NULL); if (x == NULL) { PyErr_SetString(PySSLErrorObject, "Error decoding PEM-encoded file"); From webhook-mailer at python.org Thu Aug 15 08:56:03 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 15 Aug 2019 12:56:03 -0000 Subject: [Python-checkins] Replace usage of the obscure PEM_read_bio_X509_AUX with the more standard PEM_read_bio_X509 (GH-15303) Message-ID: https://github.com/python/cpython/commit/2b9b70765ce4d955cc2e250878694885363770b8 commit: 2b9b70765ce4d955cc2e250878694885363770b8 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-15T05:55:59-07:00 summary: Replace usage of the obscure PEM_read_bio_X509_AUX with the more standard PEM_read_bio_X509 (GH-15303) X509_AUX is an odd, note widely used, OpenSSL extension to the X509 file format. This function doesn't actually use any of the extra metadata that it parses, so just use the standard API. Automerge-Triggered-By: @tiran (cherry picked from commit 40dad9545aad4ede89abbab1c1beef5303d9573e) Co-authored-by: Alex Gaynor files: M Modules/_ssl.c diff --git a/Modules/_ssl.c b/Modules/_ssl.c index e8955eedfa53..b079663cc223 100644 --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1789,7 +1789,7 @@ _ssl__test_decode_cert_impl(PyObject *module, PyObject *path) goto fail0; } - x = PEM_read_bio_X509_AUX(cert,NULL, NULL, NULL); + x = PEM_read_bio_X509(cert, NULL, NULL, NULL); if (x == NULL) { PyErr_SetString(PySSLErrorObject, "Error decoding PEM-encoded file"); From webhook-mailer at python.org Thu Aug 15 11:49:52 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 15 Aug 2019 15:49:52 -0000 Subject: [Python-checkins] bpo-37207: enable vectorcall for type.__call__ (GH-14588) Message-ID: https://github.com/python/cpython/commit/37806f404f57b234902f0c8de9a04647ad01b7f1 commit: 37806f404f57b234902f0c8de9a04647ad01b7f1 branch: master author: Jeroen Demeyer committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-15T08:49:46-07:00 summary: bpo-37207: enable vectorcall for type.__call__ (GH-14588) Base PR for other PRs that want to play with `type.__call__` such as #13930 and #14589. The author is really @markshannon I just made the PR. https://bugs.python.org/issue37207 Automerge-Triggered-By: @encukou files: A Misc/NEWS.d/next/C API/2019-07-07-10-37-07.bpo-37207.SlVNky.rst M Objects/typeobject.c diff --git a/Misc/NEWS.d/next/C API/2019-07-07-10-37-07.bpo-37207.SlVNky.rst b/Misc/NEWS.d/next/C API/2019-07-07-10-37-07.bpo-37207.SlVNky.rst new file mode 100644 index 000000000000..8df76614a0d7 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2019-07-07-10-37-07.bpo-37207.SlVNky.rst @@ -0,0 +1,3 @@ +The vectorcall protocol is now enabled for ``type`` objects: set +``tp_vectorcall`` to a vectorcall function to be used instead of ``tp_new`` +and ``tp_init`` when calling the class itself. diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 9e5709a74f78..0ca7dcb695bb 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3614,7 +3614,7 @@ PyTypeObject PyType_Type = { sizeof(PyHeapTypeObject), /* tp_basicsize */ sizeof(PyMemberDef), /* tp_itemsize */ (destructor)type_dealloc, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ + offsetof(PyTypeObject, tp_vectorcall), /* tp_vectorcall_offset */ 0, /* tp_getattr */ 0, /* tp_setattr */ 0, /* tp_as_async */ @@ -3629,7 +3629,8 @@ PyTypeObject PyType_Type = { (setattrofunc)type_setattro, /* tp_setattro */ 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | - Py_TPFLAGS_BASETYPE | Py_TPFLAGS_TYPE_SUBCLASS, /* tp_flags */ + Py_TPFLAGS_BASETYPE | Py_TPFLAGS_TYPE_SUBCLASS | + _Py_TPFLAGS_HAVE_VECTORCALL, /* tp_flags */ type_doc, /* tp_doc */ (traverseproc)type_traverse, /* tp_traverse */ (inquiry)type_clear, /* tp_clear */ From webhook-mailer at python.org Thu Aug 15 12:19:13 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Thu, 15 Aug 2019 16:19:13 -0000 Subject: [Python-checkins] Indent code inside if block. (GH-15284) Message-ID: https://github.com/python/cpython/commit/69f37bcb28d7cd78255828029f895958b5baf6ff commit: 69f37bcb28d7cd78255828029f895958b5baf6ff branch: master author: Hansraj Das committer: Benjamin Peterson date: 2019-08-15T09:19:07-07:00 summary: Indent code inside if block. (GH-15284) Without indendation, seems like strcpy line is parallel to `if` condition. files: M Parser/tokenizer.c diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 31fe970c9003..5763e47c4b00 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1821,7 +1821,7 @@ PyTokenizer_FindEncodingFilename(int fd, PyObject *filename) if (tok->encoding) { encoding = (char *)PyMem_MALLOC(strlen(tok->encoding) + 1); if (encoding) - strcpy(encoding, tok->encoding); + strcpy(encoding, tok->encoding); } PyTokenizer_Free(tok); return encoding; From webhook-mailer at python.org Thu Aug 15 12:38:27 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 15 Aug 2019 16:38:27 -0000 Subject: [Python-checkins] Indent code inside if block. (GH-15284) Message-ID: https://github.com/python/cpython/commit/64db5aac6be73b14396bcde1648997e208b5a8da commit: 64db5aac6be73b14396bcde1648997e208b5a8da branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-15T09:38:22-07:00 summary: Indent code inside if block. (GH-15284) Without indendation, seems like strcpy line is parallel to `if` condition. (cherry picked from commit 69f37bcb28d7cd78255828029f895958b5baf6ff) Co-authored-by: Hansraj Das files: M Parser/tokenizer.c diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 31fe970c9003..5763e47c4b00 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1821,7 +1821,7 @@ PyTokenizer_FindEncodingFilename(int fd, PyObject *filename) if (tok->encoding) { encoding = (char *)PyMem_MALLOC(strlen(tok->encoding) + 1); if (encoding) - strcpy(encoding, tok->encoding); + strcpy(encoding, tok->encoding); } PyTokenizer_Free(tok); return encoding; From webhook-mailer at python.org Thu Aug 15 12:46:51 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 15 Aug 2019 16:46:51 -0000 Subject: [Python-checkins] Indent code inside if block. (GH-15284) Message-ID: https://github.com/python/cpython/commit/6ac851fadf52ccaf8a14abe1b5e6a74bd6eec69e commit: 6ac851fadf52ccaf8a14abe1b5e6a74bd6eec69e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-15T09:46:47-07:00 summary: Indent code inside if block. (GH-15284) Without indendation, seems like strcpy line is parallel to `if` condition. (cherry picked from commit 69f37bcb28d7cd78255828029f895958b5baf6ff) Co-authored-by: Hansraj Das files: M Parser/tokenizer.c diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index e374c5a4aee6..d720f19eca9d 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1890,7 +1890,7 @@ PyTokenizer_FindEncodingFilename(int fd, PyObject *filename) if (tok->encoding) { encoding = (char *)PyMem_MALLOC(strlen(tok->encoding) + 1); if (encoding) - strcpy(encoding, tok->encoding); + strcpy(encoding, tok->encoding); } PyTokenizer_Free(tok); return encoding; From webhook-mailer at python.org Thu Aug 15 15:09:11 2019 From: webhook-mailer at python.org (Paul Ganssle) Date: Thu, 15 Aug 2019 19:09:11 -0000 Subject: [Python-checkins] bpo-37642: Update acceptable offsets in timezone (GH-14878) (#15227) Message-ID: https://github.com/python/cpython/commit/27b38b99b3a154fa5c25cd67fe01fb4fc04604b0 commit: 27b38b99b3a154fa5c25cd67fe01fb4fc04604b0 branch: 3.8 author: Paul Ganssle committer: GitHub date: 2019-08-15T15:08:57-04:00 summary: bpo-37642: Update acceptable offsets in timezone (GH-14878) (#15227) This fixes an inconsistency between the Python and C implementations of the datetime module. The pure python version of the code was not accepting offsets greater than 23:59 but less than 24:00. This is an accidental legacy of the original implementation, which was put in place before tzinfo allowed sub-minute time zone offsets. GH-14878 (cherry picked from commit 92c7e30adf5c81a54d6e5e555a6bdfaa60157a0d) files: A Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst M Lib/datetime.py M Lib/test/datetimetester.py M Misc/ACKS M Modules/_datetimemodule.c diff --git a/Lib/datetime.py b/Lib/datetime.py index d4c7a1ff9004..0adf1dd67dfb 100644 --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -2269,7 +2269,7 @@ def fromutc(self, dt): raise TypeError("fromutc() argument must be a datetime instance" " or None") - _maxoffset = timedelta(hours=23, minutes=59) + _maxoffset = timedelta(hours=24, microseconds=-1) _minoffset = -_maxoffset @staticmethod @@ -2293,8 +2293,11 @@ def _name_from_offset(delta): return f'UTC{sign}{hours:02d}:{minutes:02d}' timezone.utc = timezone._create(timedelta(0)) -timezone.min = timezone._create(timezone._minoffset) -timezone.max = timezone._create(timezone._maxoffset) +# bpo-37642: These attributes are rounded to the nearest minute for backwards +# compatibility, even though the constructor will accept a wider range of +# values. This may change in the future. +timezone.min = timezone._create(-timedelta(hours=23, minutes=59)) +timezone.max = timezone._create(timedelta(hours=23, minutes=59)) _EPOCH = datetime(1970, 1, 1, tzinfo=timezone.utc) # Some time zone algebra. For a datetime x, let diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 99b620ce2f41..d0101c98bc76 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -388,6 +388,31 @@ def test_deepcopy(self): tz_copy = copy.deepcopy(tz) self.assertIs(tz_copy, tz) + def test_offset_boundaries(self): + # Test timedeltas close to the boundaries + time_deltas = [ + timedelta(hours=23, minutes=59), + timedelta(hours=23, minutes=59, seconds=59), + timedelta(hours=23, minutes=59, seconds=59, microseconds=999999), + ] + time_deltas.extend([-delta for delta in time_deltas]) + + for delta in time_deltas: + with self.subTest(test_type='good', delta=delta): + timezone(delta) + + # Test timedeltas on and outside the boundaries + bad_time_deltas = [ + timedelta(hours=24), + timedelta(hours=24, microseconds=1), + ] + bad_time_deltas.extend([-delta for delta in bad_time_deltas]) + + for delta in bad_time_deltas: + with self.subTest(test_type='bad', delta=delta): + with self.assertRaises(ValueError): + timezone(delta) + ############################################################################# # Base class for testing a particular aspect of timedelta, time, date and diff --git a/Misc/ACKS b/Misc/ACKS index 311259f1e5fa..ab874e929931 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1874,3 +1874,4 @@ Geoff Shannon Batuhan Taskaya Aleksandr Balezin Robert Leenders +Ngalim Siregar diff --git a/Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst b/Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst new file mode 100644 index 000000000000..09ff257597e8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst @@ -0,0 +1,3 @@ +Allowed the pure Python implementation of :class:`datetime.timezone` to represent +sub-minute offsets close to minimum and maximum boundaries, specifically in the +ranges (23:59, 24:00) and (-23:59, 24:00). Patch by Ngalim Siregar diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index 80ecfc3fff0b..beb1c3cfbca9 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -1099,7 +1099,9 @@ new_timezone(PyObject *offset, PyObject *name) Py_INCREF(PyDateTime_TimeZone_UTC); return PyDateTime_TimeZone_UTC; } - if ((GET_TD_DAYS(offset) == -1 && GET_TD_SECONDS(offset) == 0) || + if ((GET_TD_DAYS(offset) == -1 && + GET_TD_SECONDS(offset) == 0 && + GET_TD_MICROSECONDS(offset) < 1) || GET_TD_DAYS(offset) < -1 || GET_TD_DAYS(offset) >= 1) { PyErr_Format(PyExc_ValueError, "offset must be a timedelta" " strictly between -timedelta(hours=24) and" @@ -1169,7 +1171,9 @@ call_tzinfo_method(PyObject *tzinfo, const char *name, PyObject *tzinfoarg) if (offset == Py_None || offset == NULL) return offset; if (PyDelta_Check(offset)) { - if ((GET_TD_DAYS(offset) == -1 && GET_TD_SECONDS(offset) == 0) || + if ((GET_TD_DAYS(offset) == -1 && + GET_TD_SECONDS(offset) == 0 && + GET_TD_MICROSECONDS(offset) < 1) || GET_TD_DAYS(offset) < -1 || GET_TD_DAYS(offset) >= 1) { Py_DECREF(offset); PyErr_Format(PyExc_ValueError, "offset must be a timedelta" @@ -6484,6 +6488,9 @@ PyInit__datetime(void) PyDateTime_TimeZone_UTC = x; CAPI.TimeZone_UTC = PyDateTime_TimeZone_UTC; + /* bpo-37642: These attributes are rounded to the nearest minute for backwards + * compatibility, even though the constructor will accept a wider range of + * values. This may change in the future.*/ delta = new_delta(-1, 60, 0, 1); /* -23:59 */ if (delta == NULL) return NULL; From webhook-mailer at python.org Thu Aug 15 15:09:41 2019 From: webhook-mailer at python.org (Paul Ganssle) Date: Thu, 15 Aug 2019 19:09:41 -0000 Subject: [Python-checkins] bpo-37642: Update acceptable offsets in timezone (GH-14878) (#15226) Message-ID: https://github.com/python/cpython/commit/ed44b84961eb0e5b97e4866c1455ac4093d27549 commit: ed44b84961eb0e5b97e4866c1455ac4093d27549 branch: 3.7 author: Paul Ganssle committer: GitHub date: 2019-08-15T15:09:37-04:00 summary: bpo-37642: Update acceptable offsets in timezone (GH-14878) (#15226) This fixes an inconsistency between the Python and C implementations of the datetime module. The pure python version of the code was not accepting offsets greater than 23:59 but less than 24:00. This is an accidental legacy of the original implementation, which was put in place before tzinfo allowed sub-minute time zone offsets. GH-14878 (cherry picked from commit 92c7e30adf5c81a54d6e5e555a6bdfaa60157a0d) files: A Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst M Lib/datetime.py M Lib/test/datetimetester.py M Misc/ACKS M Modules/_datetimemodule.c diff --git a/Lib/datetime.py b/Lib/datetime.py index 0485b0523e8d..5a2ee6a2035a 100644 --- a/Lib/datetime.py +++ b/Lib/datetime.py @@ -2226,7 +2226,7 @@ def fromutc(self, dt): raise TypeError("fromutc() argument must be a datetime instance" " or None") - _maxoffset = timedelta(hours=23, minutes=59) + _maxoffset = timedelta(hours=24, microseconds=-1) _minoffset = -_maxoffset @staticmethod @@ -2250,8 +2250,11 @@ def _name_from_offset(delta): return f'UTC{sign}{hours:02d}:{minutes:02d}' timezone.utc = timezone._create(timedelta(0)) -timezone.min = timezone._create(timezone._minoffset) -timezone.max = timezone._create(timezone._maxoffset) +# bpo-37642: These attributes are rounded to the nearest minute for backwards +# compatibility, even though the constructor will accept a wider range of +# values. This may change in the future. +timezone.min = timezone._create(-timedelta(hours=23, minutes=59)) +timezone.max = timezone._create(timedelta(hours=23, minutes=59)) _EPOCH = datetime(1970, 1, 1, tzinfo=timezone.utc) # Some time zone algebra. For a datetime x, let diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 8a5716979716..025e71de7872 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -388,6 +388,31 @@ def test_deepcopy(self): tz_copy = copy.deepcopy(tz) self.assertIs(tz_copy, tz) + def test_offset_boundaries(self): + # Test timedeltas close to the boundaries + time_deltas = [ + timedelta(hours=23, minutes=59), + timedelta(hours=23, minutes=59, seconds=59), + timedelta(hours=23, minutes=59, seconds=59, microseconds=999999), + ] + time_deltas.extend([-delta for delta in time_deltas]) + + for delta in time_deltas: + with self.subTest(test_type='good', delta=delta): + timezone(delta) + + # Test timedeltas on and outside the boundaries + bad_time_deltas = [ + timedelta(hours=24), + timedelta(hours=24, microseconds=1), + ] + bad_time_deltas.extend([-delta for delta in bad_time_deltas]) + + for delta in bad_time_deltas: + with self.subTest(test_type='bad', delta=delta): + with self.assertRaises(ValueError): + timezone(delta) + ############################################################################# # Base class for testing a particular aspect of timedelta, time, date and diff --git a/Misc/ACKS b/Misc/ACKS index 4d8f96acfe30..0283c85b174e 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1836,3 +1836,4 @@ Doug Zongker Peter ?strand Zheao Li Geoff Shannon +Ngalim Siregar diff --git a/Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst b/Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst new file mode 100644 index 000000000000..09ff257597e8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-21-20-59-31.bpo-37642.L61Bvy.rst @@ -0,0 +1,3 @@ +Allowed the pure Python implementation of :class:`datetime.timezone` to represent +sub-minute offsets close to minimum and maximum boundaries, specifically in the +ranges (23:59, 24:00) and (-23:59, 24:00). Patch by Ngalim Siregar diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index aa759b115f0e..655be364d9cb 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -1085,7 +1085,9 @@ new_timezone(PyObject *offset, PyObject *name) Py_INCREF(PyDateTime_TimeZone_UTC); return PyDateTime_TimeZone_UTC; } - if ((GET_TD_DAYS(offset) == -1 && GET_TD_SECONDS(offset) == 0) || + if ((GET_TD_DAYS(offset) == -1 && + GET_TD_SECONDS(offset) == 0 && + GET_TD_MICROSECONDS(offset) < 1) || GET_TD_DAYS(offset) < -1 || GET_TD_DAYS(offset) >= 1) { PyErr_Format(PyExc_ValueError, "offset must be a timedelta" " strictly between -timedelta(hours=24) and" @@ -1155,7 +1157,9 @@ call_tzinfo_method(PyObject *tzinfo, const char *name, PyObject *tzinfoarg) if (offset == Py_None || offset == NULL) return offset; if (PyDelta_Check(offset)) { - if ((GET_TD_DAYS(offset) == -1 && GET_TD_SECONDS(offset) == 0) || + if ((GET_TD_DAYS(offset) == -1 && + GET_TD_SECONDS(offset) == 0 && + GET_TD_MICROSECONDS(offset) < 1) || GET_TD_DAYS(offset) < -1 || GET_TD_DAYS(offset) >= 1) { Py_DECREF(offset); PyErr_Format(PyExc_ValueError, "offset must be a timedelta" @@ -6383,6 +6387,9 @@ PyInit__datetime(void) PyDateTime_TimeZone_UTC = x; CAPI.TimeZone_UTC = PyDateTime_TimeZone_UTC; + /* bpo-37642: These attributes are rounded to the nearest minute for backwards + * compatibility, even though the constructor will accept a wider range of + * values. This may change in the future.*/ delta = new_delta(-1, 60, 0, 1); /* -23:59 */ if (delta == NULL) return NULL; From webhook-mailer at python.org Thu Aug 15 23:58:40 2019 From: webhook-mailer at python.org (Raymond Hettinger) Date: Fri, 16 Aug 2019 03:58:40 -0000 Subject: [Python-checkins] bpo-37863: Optimize Fraction.__hash__() (#15298) Message-ID: https://github.com/python/cpython/commit/f3cb68f2e4c3e0c405460f9bb881f5c1db70f535 commit: f3cb68f2e4c3e0c405460f9bb881f5c1db70f535 branch: master author: Raymond Hettinger committer: GitHub date: 2019-08-15T20:58:26-07:00 summary: bpo-37863: Optimize Fraction.__hash__() (#15298) files: A Misc/NEWS.d/next/Library/2019-08-14-20-46-39.bpo-37863.CkXqgX.rst M Lib/fractions.py diff --git a/Lib/fractions.py b/Lib/fractions.py index e774d58e4035..c922c38e2441 100644 --- a/Lib/fractions.py +++ b/Lib/fractions.py @@ -556,23 +556,19 @@ def __round__(self, ndigits=None): def __hash__(self): """hash(self)""" - # XXX since this method is expensive, consider caching the result - - # In order to make sure that the hash of a Fraction agrees - # with the hash of a numerically equal integer, float or - # Decimal instance, we follow the rules for numeric hashes - # outlined in the documentation. (See library docs, 'Built-in - # Types'). - - # dinv is the inverse of self._denominator modulo the prime - # _PyHASH_MODULUS, or 0 if self._denominator is divisible by - # _PyHASH_MODULUS. - dinv = pow(self._denominator, _PyHASH_MODULUS - 2, _PyHASH_MODULUS) - if not dinv: + # To make sure that the hash of a Fraction agrees with the hash + # of a numerically equal integer, float or Decimal instance, we + # follow the rules for numeric hashes outlined in the + # documentation. (See library docs, 'Built-in Types'). + + try: + dinv = pow(self._denominator, -1, _PyHASH_MODULUS) + except ValueError: + # ValueError means there is no modular inverse hash_ = _PyHASH_INF else: - hash_ = abs(self._numerator) * dinv % _PyHASH_MODULUS - result = hash_ if self >= 0 else -hash_ + hash_ = hash(abs(self._numerator)) * dinv % _PyHASH_MODULUS + result = hash_ if self._numerator >= 0 else -hash_ return -2 if result == -1 else result def __eq__(a, b): diff --git a/Misc/NEWS.d/next/Library/2019-08-14-20-46-39.bpo-37863.CkXqgX.rst b/Misc/NEWS.d/next/Library/2019-08-14-20-46-39.bpo-37863.CkXqgX.rst new file mode 100644 index 000000000000..90df6e9cb652 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-14-20-46-39.bpo-37863.CkXqgX.rst @@ -0,0 +1 @@ +Optimizations for Fraction.__hash__ suggested by Tim Peters. From webhook-mailer at python.org Fri Aug 16 04:27:40 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 16 Aug 2019 08:27:40 -0000 Subject: [Python-checkins] bpo-37256: Wording in Request class docs (GH-14792) Message-ID: https://github.com/python/cpython/commit/f9919121460bffc04f935dbdb85f0af3ffbd3ddf commit: f9919121460bffc04f935dbdb85f0af3ffbd3ddf branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-16T01:27:35-07:00 summary: bpo-37256: Wording in Request class docs (GH-14792) * bpo-37256: Wording in Request class docs * ?? Added by blurb_it. * Update Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst Co-Authored-By: Kyle Stanley (cherry picked from commit 38c7199beb30ae9a5005c0f0d9df9fae0da3680a) Co-authored-by: Ngalim Siregar files: A Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst M Doc/library/urllib.request.rst diff --git a/Doc/library/urllib.request.rst b/Doc/library/urllib.request.rst index 3b7508966691..448bc6785470 100644 --- a/Doc/library/urllib.request.rst +++ b/Doc/library/urllib.request.rst @@ -227,7 +227,7 @@ The following classes are provided: is not None, ``Content-Type: application/x-www-form-urlencoded`` will be added as a default. - The final two arguments are only of interest for correct handling + The next two arguments are only of interest for correct handling of third-party HTTP cookies: *origin_req_host* should be the request-host of the origin diff --git a/Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst b/Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst new file mode 100644 index 000000000000..480d7c87ebc4 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-07-16-14-48-12.bpo-37256.qJTrBb.rst @@ -0,0 +1 @@ +Fix wording of arguments for :class:`Request` in :mod:`urllib.request` From webhook-mailer at python.org Fri Aug 16 06:41:33 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 16 Aug 2019 10:41:33 -0000 Subject: [Python-checkins] bpo-37540: vectorcall: keyword names must be strings (GH-14682) Message-ID: https://github.com/python/cpython/commit/0567786d26348aa7eaf0ab1b5d038fdabe409d92 commit: 0567786d26348aa7eaf0ab1b5d038fdabe409d92 branch: master author: Jeroen Demeyer committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-16T03:41:27-07:00 summary: bpo-37540: vectorcall: keyword names must be strings (GH-14682) The fact that keyword names are strings is now part of the vectorcall and `METH_FASTCALL` protocols. The biggest concrete change is that `_PyStack_UnpackDict` now checks that and raises `TypeError` if not. CC @markshannon @vstinner https://bugs.python.org/issue37540 files: A Misc/NEWS.d/next/C API/2019-07-10-12-27-28.bpo-37540.E8Z773.rst M Doc/c-api/object.rst M Doc/c-api/structures.rst M Doc/library/dis.rst M Include/cpython/abstract.h M Lib/test/test_extcall.py M Lib/test/test_unpack_ex.py M Objects/call.c M Python/ceval.c M Python/getargs.c diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 2cf032821afb..fd1e9c65aaba 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -400,8 +400,8 @@ Object Protocol :c:func:`PyVectorcall_NARGS(nargsf) `. *kwnames* can be either NULL (no keyword arguments) or a tuple of keyword - names. In the latter case, the values of the keyword arguments are stored - in *args* after the positional arguments. + names, which must be strings. In the latter case, the values of the keyword + arguments are stored in *args* after the positional arguments. The number of keyword arguments does not influence *nargsf*. *kwnames* must contain only objects of type ``str`` (not a subclass), diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index 5184ad511cd9..d4e65afef14d 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -204,6 +204,7 @@ also keyword arguments. So there are a total of 6 calling conventions: Keyword arguments are passed the same way as in the vectorcall protocol: there is an additional fourth :c:type:`PyObject\*` parameter which is a tuple representing the names of the keyword arguments + (which are guaranteed to be strings) or possibly *NULL* if there are no keywords. The values of the keyword arguments are stored in the *args* array, after the positional arguments. diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 39a3e130afd3..4a20245e3032 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -1142,8 +1142,10 @@ All of the following opcodes use their arguments. Calls a callable object with positional (if any) and keyword arguments. *argc* indicates the total number of positional and keyword arguments. - The top element on the stack contains a tuple of keyword argument names. - Below that are keyword arguments in the order corresponding to the tuple. + The top element on the stack contains a tuple with the names of the + keyword arguments, which must be strings. + Below that are the values for the keyword arguments, + in the order corresponding to the tuple. Below that are positional arguments, with the right-most parameter on top. Below the arguments is a callable object to call. ``CALL_FUNCTION_KW`` pops all arguments and the callable object off the stack, diff --git a/Include/cpython/abstract.h b/Include/cpython/abstract.h index d57aa54bc466..62a113fc00e2 100644 --- a/Include/cpython/abstract.h +++ b/Include/cpython/abstract.h @@ -88,8 +88,7 @@ _PyVectorcall_Function(PyObject *callable) of keyword arguments does not change nargsf). kwnames can also be NULL if there are no keyword arguments. - keywords must only contains str strings (no subclass), and all keys must - be unique. + keywords must only contain strings and all keys must be unique. Return the result on success. Raise an exception and return NULL on error. */ diff --git a/Lib/test/test_extcall.py b/Lib/test/test_extcall.py index 3cac3bda4253..d9dcb709f754 100644 --- a/Lib/test/test_extcall.py +++ b/Lib/test/test_extcall.py @@ -237,7 +237,7 @@ >>> f(**{1:2}) Traceback (most recent call last): ... - TypeError: f() keywords must be strings + TypeError: keywords must be strings >>> h(**{'e': 2}) Traceback (most recent call last): diff --git a/Lib/test/test_unpack_ex.py b/Lib/test/test_unpack_ex.py index 45cf051f1ec1..87fea593c020 100644 --- a/Lib/test/test_unpack_ex.py +++ b/Lib/test/test_unpack_ex.py @@ -256,7 +256,7 @@ >>> f(**{1: 3}, **{1: 5}) Traceback (most recent call last): ... - TypeError: f() keywords must be strings + TypeError: f() got multiple values for keyword argument '1' Unpacking non-sequence diff --git a/Misc/NEWS.d/next/C API/2019-07-10-12-27-28.bpo-37540.E8Z773.rst b/Misc/NEWS.d/next/C API/2019-07-10-12-27-28.bpo-37540.E8Z773.rst new file mode 100644 index 000000000000..1a09c7e1330e --- /dev/null +++ b/Misc/NEWS.d/next/C API/2019-07-10-12-27-28.bpo-37540.E8Z773.rst @@ -0,0 +1,2 @@ +The vectorcall protocol now requires that the caller passes only strings as +keyword names. diff --git a/Objects/call.c b/Objects/call.c index 7d917891bc0c..8a60b3e58e30 100644 --- a/Objects/call.c +++ b/Objects/call.c @@ -322,8 +322,7 @@ _PyFunction_Vectorcall(PyObject *func, PyObject* const* stack, assert(nargs >= 0); assert(kwnames == NULL || PyTuple_CheckExact(kwnames)); assert((nargs == 0 && nkwargs == 0) || stack != NULL); - /* kwnames must only contains str strings, no subclass, and all keys must - be unique */ + /* kwnames must only contain strings and all keys must be unique */ if (co->co_kwonlyargcount == 0 && nkwargs == 0 && (co->co_flags & ~PyCF_MASK) == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)) @@ -943,12 +942,12 @@ _PyStack_AsDict(PyObject *const *values, PyObject *kwnames) vector; return NULL with exception set on error. Return the keyword names tuple in *p_kwnames. - The newly allocated argument vector supports PY_VECTORCALL_ARGUMENTS_OFFSET. + This also checks that all keyword names are strings. If not, a TypeError is + raised. - When done, you must call _PyStack_UnpackDict_Free(stack, nargs, kwnames) + The newly allocated argument vector supports PY_VECTORCALL_ARGUMENTS_OFFSET. - The type of keyword keys is not checked, these checks should be done - later (ex: _PyArg_ParseStackAndKeywords). */ + When done, you must call _PyStack_UnpackDict_Free(stack, nargs, kwnames) */ static PyObject *const * _PyStack_UnpackDict(PyObject *const *args, Py_ssize_t nargs, PyObject *kwargs, PyObject **p_kwnames) @@ -994,7 +993,9 @@ _PyStack_UnpackDict(PyObject *const *args, Py_ssize_t nargs, PyObject *kwargs, called in the performance critical hot code. */ Py_ssize_t pos = 0, i = 0; PyObject *key, *value; + unsigned long keys_are_strings = Py_TPFLAGS_UNICODE_SUBCLASS; while (PyDict_Next(kwargs, &pos, &key, &value)) { + keys_are_strings &= Py_TYPE(key)->tp_flags; Py_INCREF(key); Py_INCREF(value); PyTuple_SET_ITEM(kwnames, i, key); @@ -1002,6 +1003,18 @@ _PyStack_UnpackDict(PyObject *const *args, Py_ssize_t nargs, PyObject *kwargs, i++; } + /* keys_are_strings has the value Py_TPFLAGS_UNICODE_SUBCLASS if that + * flag is set for all keys. Otherwise, keys_are_strings equals 0. + * We do this check once at the end instead of inside the loop above + * because it simplifies the deallocation in the failing case. + * It happens to also make the loop above slightly more efficient. */ + if (!keys_are_strings) { + PyErr_SetString(PyExc_TypeError, + "keywords must be strings"); + _PyStack_UnpackDict_Free(stack, nargs, kwnames); + return NULL; + } + *p_kwnames = kwnames; return stack; } diff --git a/Python/ceval.c b/Python/ceval.c index 7c7359166dad..ee03350031d9 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -3504,7 +3504,9 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag) PyObject **sp, *res, *names; names = POP(); - assert(PyTuple_CheckExact(names) && PyTuple_GET_SIZE(names) <= oparg); + assert(PyTuple_Check(names)); + assert(PyTuple_GET_SIZE(names) <= oparg); + /* We assume without checking that names contains only strings */ sp = stack_pointer; res = call_function(tstate, &sp, oparg, names); stack_pointer = sp; @@ -5372,20 +5374,12 @@ format_kwargs_error(PyThreadState *tstate, PyObject *func, PyObject *kwargs) _PyErr_Fetch(tstate, &exc, &val, &tb); if (val && PyTuple_Check(val) && PyTuple_GET_SIZE(val) == 1) { PyObject *key = PyTuple_GET_ITEM(val, 0); - if (!PyUnicode_Check(key)) { - _PyErr_Format(tstate, PyExc_TypeError, - "%.200s%.200s keywords must be strings", - PyEval_GetFuncName(func), - PyEval_GetFuncDesc(func)); - } - else { - _PyErr_Format(tstate, PyExc_TypeError, - "%.200s%.200s got multiple " - "values for keyword argument '%U'", - PyEval_GetFuncName(func), - PyEval_GetFuncDesc(func), - key); - } + _PyErr_Format(tstate, PyExc_TypeError, + "%.200s%.200s got multiple " + "values for keyword argument '%S'", + PyEval_GetFuncName(func), + PyEval_GetFuncDesc(func), + key); Py_XDECREF(exc); Py_XDECREF(val); Py_XDECREF(tb); diff --git a/Python/getargs.c b/Python/getargs.c index 59f0fdabb74a..cdc16d4730b5 100644 --- a/Python/getargs.c +++ b/Python/getargs.c @@ -2043,11 +2043,7 @@ find_keyword(PyObject *kwnames, PyObject *const *kwstack, PyObject *key) if (kwname == key) { return kwstack[i]; } - if (!PyUnicode_Check(kwname)) { - /* ignore non-string keyword keys: - an error will be raised below */ - continue; - } + assert(PyUnicode_Check(kwname)); if (_PyUnicode_EQ(kwname, key)) { return kwstack[i]; } @@ -2275,16 +2271,11 @@ vgetargskeywordsfast_impl(PyObject *const *args, Py_ssize_t nargs, j++; } - if (!PyUnicode_Check(keyword)) { - PyErr_SetString(PyExc_TypeError, - "keywords must be strings"); - return cleanreturn(0, &freelist); - } match = PySequence_Contains(kwtuple, keyword); if (match <= 0) { if (!match) { PyErr_Format(PyExc_TypeError, - "'%U' is an invalid keyword " + "'%S' is an invalid keyword " "argument for %.200s%s", keyword, (parser->fname == NULL) ? "this function" : parser->fname, @@ -2505,16 +2496,11 @@ _PyArg_UnpackKeywords(PyObject *const *args, Py_ssize_t nargs, j++; } - if (!PyUnicode_Check(keyword)) { - PyErr_SetString(PyExc_TypeError, - "keywords must be strings"); - return NULL; - } match = PySequence_Contains(kwtuple, keyword); if (match <= 0) { if (!match) { PyErr_Format(PyExc_TypeError, - "'%U' is an invalid keyword " + "'%S' is an invalid keyword " "argument for %.200s%s", keyword, (parser->fname == NULL) ? "this function" : parser->fname, From webhook-mailer at python.org Fri Aug 16 22:09:32 2019 From: webhook-mailer at python.org (Tim Peters) Date: Sat, 17 Aug 2019 02:09:32 -0000 Subject: [Python-checkins] Add a minor `Fraction.__hash__()` optimization (GH-15313) Message-ID: https://github.com/python/cpython/commit/29bb227a0ce6d355a2b3e5d6a25872e3702ba9bb commit: 29bb227a0ce6d355a2b3e5d6a25872e3702ba9bb branch: master author: Tim Peters committer: GitHub date: 2019-08-16T21:09:16-05:00 summary: Add a minor `Fraction.__hash__()` optimization (GH-15313) * Add a minor `Fraction.__hash__` optimization that got lost in the shuffle. Document the optimizations. files: M Lib/fractions.py diff --git a/Lib/fractions.py b/Lib/fractions.py index c922c38e2441..2e7047a81844 100644 --- a/Lib/fractions.py +++ b/Lib/fractions.py @@ -564,10 +564,25 @@ def __hash__(self): try: dinv = pow(self._denominator, -1, _PyHASH_MODULUS) except ValueError: - # ValueError means there is no modular inverse + # ValueError means there is no modular inverse. hash_ = _PyHASH_INF else: - hash_ = hash(abs(self._numerator)) * dinv % _PyHASH_MODULUS + # The general algorithm now specifies that the absolute value of + # the hash is + # (|N| * dinv) % P + # where N is self._numerator and P is _PyHASH_MODULUS. That's + # optimized here in two ways: first, for a non-negative int i, + # hash(i) == i % P, but the int hash implementation doesn't need + # to divide, and is faster than doing % P explicitly. So we do + # hash(|N| * dinv) + # instead. Second, N is unbounded, so its product with dinv may + # be arbitrarily expensive to compute. The final answer is the + # same if we use the bounded |N| % P instead, which can again + # be done with an int hash() call. If 0 <= i < P, hash(i) == i, + # so this nested hash() call wastes a bit of time making a + # redundant copy when |N| < P, but can save an arbitrarily large + # amount of computation for large |N|. + hash_ = hash(hash(abs(self._numerator)) * dinv) result = hash_ if self._numerator >= 0 else -hash_ return -2 if result == -1 else result From webhook-mailer at python.org Sat Aug 17 16:34:15 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 17 Aug 2019 20:34:15 -0000 Subject: [Python-checkins] fix link to time function from time_ns doc (GH-15285) Message-ID: https://github.com/python/cpython/commit/1b1d0514adbcdd859817c63d1410455c64660d78 commit: 1b1d0514adbcdd859817c63d1410455c64660d78 branch: master author: ?ric Araujo committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-17T13:34:08-07:00 summary: fix link to time function from time_ns doc (GH-15285) Because mod, func, class, etc all share one namespace, :func:time creates a link to the time module doc page rather than the time.time function. files: M Doc/library/time.rst diff --git a/Doc/library/time.rst b/Doc/library/time.rst index 6d0ceafa522a..4faa0bbebc5b 100644 --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -608,7 +608,7 @@ Functions .. function:: time_ns() -> int - Similar to :func:`time` but returns time as an integer number of nanoseconds + Similar to :func:`~time.time` but returns time as an integer number of nanoseconds since the epoch_. .. versionadded:: 3.7 From webhook-mailer at python.org Sat Aug 17 16:40:27 2019 From: webhook-mailer at python.org (Carol Willing) Date: Sat, 17 Aug 2019 20:40:27 -0000 Subject: [Python-checkins] Insert a missing close parenthesis (GH-15316) Message-ID: https://github.com/python/cpython/commit/455856391c2b4e2af79de55101421cd15901edaf commit: 455856391c2b4e2af79de55101421cd15901edaf branch: master author: cocoatomo committer: Carol Willing date: 2019-08-18T05:40:23+09:00 summary: Insert a missing close parenthesis (GH-15316) files: M Doc/c-api/veryhigh.rst diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 67dc11dfa9a5..cc194c200a37 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -117,7 +117,7 @@ the same library that the Python runtime is using. closed before PyRun_SimpleFileExFlags returns. .. note:: - On Windows, *fp* should be opened as binary mode (e.g. ``fopen(filename, "rb")``. + On Windows, *fp* should be opened as binary mode (e.g. ``fopen(filename, "rb")``). Otherwise, Python may not handle script file with LF line ending correctly. From webhook-mailer at python.org Sat Aug 17 16:50:43 2019 From: webhook-mailer at python.org (Steve Dower) Date: Sat, 17 Aug 2019 20:50:43 -0000 Subject: [Python-checkins] bpo-36266: Add module name in ImportError when DLL not found on Windows (GH-15180) Message-ID: https://github.com/python/cpython/commit/24fe46081be3d1c01b3d21cb39bc3492ab4485a3 commit: 24fe46081be3d1c01b3d21cb39bc3492ab4485a3 branch: master author: shireenrao committer: Steve Dower date: 2019-08-17T13:50:39-07:00 summary: bpo-36266: Add module name in ImportError when DLL not found on Windows (GH-15180) files: A Misc/NEWS.d/next/Windows/2019-08-08-18-05-27.bpo-36266.x4eZU3.rst M Python/dynload_win.c diff --git a/Misc/NEWS.d/next/Windows/2019-08-08-18-05-27.bpo-36266.x4eZU3.rst b/Misc/NEWS.d/next/Windows/2019-08-08-18-05-27.bpo-36266.x4eZU3.rst new file mode 100644 index 000000000000..86fdd6fe17f4 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-08-08-18-05-27.bpo-36266.x4eZU3.rst @@ -0,0 +1 @@ +Add the module name in the formatted error message when DLL load fail happens during module import in ``_PyImport_FindSharedFuncptrWindows()``. Patch by Srinivas Nyayapati. \ No newline at end of file diff --git a/Python/dynload_win.c b/Python/dynload_win.c index 5096555e701c..6deba1134e2a 100644 --- a/Python/dynload_win.c +++ b/Python/dynload_win.c @@ -240,8 +240,8 @@ dl_funcptr _PyImport_FindSharedFuncptrWindows(const char *prefix, This should not happen if called correctly. */ if (theLength == 0) { message = PyUnicode_FromFormat( - "DLL load failed with error code %u", - errorCode); + "DLL load failed with error code %u while importing %s", + errorCode, shortname); } else { /* For some reason a \r\n is appended to the text */ @@ -251,8 +251,8 @@ dl_funcptr _PyImport_FindSharedFuncptrWindows(const char *prefix, theLength -= 2; theInfo[theLength] = '\0'; } - message = PyUnicode_FromString( - "DLL load failed: "); + message = PyUnicode_FromFormat( + "DLL load failed while importing %s: ", shortname); PyUnicode_AppendAndDel(&message, PyUnicode_FromWideChar( From webhook-mailer at python.org Sat Aug 17 16:51:15 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 17 Aug 2019 20:51:15 -0000 Subject: [Python-checkins] fix link to time function from time_ns doc (GH-15285) Message-ID: https://github.com/python/cpython/commit/316acf27045174bf090fa5f0000eae6801e0d322 commit: 316acf27045174bf090fa5f0000eae6801e0d322 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-17T13:51:11-07:00 summary: fix link to time function from time_ns doc (GH-15285) Because mod, func, class, etc all share one namespace, :func:time creates a link to the time module doc page rather than the time.time function. (cherry picked from commit 1b1d0514adbcdd859817c63d1410455c64660d78) Co-authored-by: ?ric Araujo files: M Doc/library/time.rst diff --git a/Doc/library/time.rst b/Doc/library/time.rst index 4230c19faf10..17f8cfc54614 100644 --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -630,7 +630,7 @@ Functions .. function:: time_ns() -> int - Similar to :func:`time` but returns time as an integer number of nanoseconds + Similar to :func:`~time.time` but returns time as an integer number of nanoseconds since the epoch_. .. versionadded:: 3.7 From webhook-mailer at python.org Sat Aug 17 16:52:00 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 17 Aug 2019 20:52:00 -0000 Subject: [Python-checkins] [3.8] fix link to time function from time_ns doc (GH-15285) (GH-15321) Message-ID: https://github.com/python/cpython/commit/7309cca1471d556c139896dd8057d1feba378113 commit: 7309cca1471d556c139896dd8057d1feba378113 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-17T13:51:57-07:00 summary: [3.8] fix link to time function from time_ns doc (GH-15285) (GH-15321) Because mod, func, class, etc all share one namespace, :func:time creates a link to the time module doc page rather than the time.time function. (cherry picked from commit 1b1d0514adbcdd859817c63d1410455c64660d78) Co-authored-by: ?ric Araujo Automerge-Triggered-By: @merwok files: M Doc/library/time.rst diff --git a/Doc/library/time.rst b/Doc/library/time.rst index 6d0ceafa522a..4faa0bbebc5b 100644 --- a/Doc/library/time.rst +++ b/Doc/library/time.rst @@ -608,7 +608,7 @@ Functions .. function:: time_ns() -> int - Similar to :func:`time` but returns time as an integer number of nanoseconds + Similar to :func:`~time.time` but returns time as an integer number of nanoseconds since the epoch_. .. versionadded:: 3.7 From webhook-mailer at python.org Sat Aug 17 17:11:32 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 17 Aug 2019 21:11:32 -0000 Subject: [Python-checkins] bpo-36266: Add module name in ImportError when DLL not found on Windows (GH-15180) Message-ID: https://github.com/python/cpython/commit/786a4e1cef3eda8f434613d3801a5c7565fb5cd8 commit: 786a4e1cef3eda8f434613d3801a5c7565fb5cd8 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-17T14:11:28-07:00 summary: bpo-36266: Add module name in ImportError when DLL not found on Windows (GH-15180) (cherry picked from commit 24fe46081be3d1c01b3d21cb39bc3492ab4485a3) Co-authored-by: shireenrao files: A Misc/NEWS.d/next/Windows/2019-08-08-18-05-27.bpo-36266.x4eZU3.rst M Python/dynload_win.c diff --git a/Misc/NEWS.d/next/Windows/2019-08-08-18-05-27.bpo-36266.x4eZU3.rst b/Misc/NEWS.d/next/Windows/2019-08-08-18-05-27.bpo-36266.x4eZU3.rst new file mode 100644 index 000000000000..86fdd6fe17f4 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2019-08-08-18-05-27.bpo-36266.x4eZU3.rst @@ -0,0 +1 @@ +Add the module name in the formatted error message when DLL load fail happens during module import in ``_PyImport_FindSharedFuncptrWindows()``. Patch by Srinivas Nyayapati. \ No newline at end of file diff --git a/Python/dynload_win.c b/Python/dynload_win.c index 457d47f5eed5..2f28c3cf9249 100644 --- a/Python/dynload_win.c +++ b/Python/dynload_win.c @@ -258,8 +258,8 @@ dl_funcptr _PyImport_FindSharedFuncptrWindows(const char *prefix, This should not happen if called correctly. */ if (theLength == 0) { message = PyUnicode_FromFormat( - "DLL load failed with error code %u", - errorCode); + "DLL load failed with error code %u while importing %s", + errorCode, shortname); } else { /* For some reason a \r\n is appended to the text */ @@ -269,8 +269,8 @@ dl_funcptr _PyImport_FindSharedFuncptrWindows(const char *prefix, theLength -= 2; theInfo[theLength] = '\0'; } - message = PyUnicode_FromString( - "DLL load failed: "); + message = PyUnicode_FromFormat( + "DLL load failed while importing %s: ", shortname); PyUnicode_AppendAndDel(&message, PyUnicode_FromWideChar( From webhook-mailer at python.org Mon Aug 19 05:53:41 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 19 Aug 2019 09:53:41 -0000 Subject: [Python-checkins] bpo-36502: Correct documentation of str.isspace() (GH-15019) (GH-15296) Message-ID: https://github.com/python/cpython/commit/8c1c426a631ba02357112657193f82c58d3e08b4 commit: 8c1c426a631ba02357112657193f82c58d3e08b4 branch: 3.8 author: Greg Price committer: Victor Stinner date: 2019-08-19T10:53:22+01:00 summary: bpo-36502: Correct documentation of str.isspace() (GH-15019) (GH-15296) The documented definition was much broader than the real one: there are tons of characters with general category "Other", and we don't (and shouldn't) treat most of them as whitespace. Rewrite the definition to agree with the comment on _PyUnicode_IsWhitespace, and with the logic in makeunicodedata.py, which is what generates that function and so ultimately governs. Add suitable breadcrumbs so that a reader who wants to pin down exactly what this definition means (what's a "bidirectional class" of "B"?) can do so. The `unicodedata` module documentation is an appropriate central place for our references to Unicode's own copious documentation, so point there. Also add to the isspace() test a thorough check that the implementation agrees with the intended definition. files: M Doc/library/stdtypes.rst M Lib/test/test_unicode.py diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 965167640c98..0f7c369ea51f 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1756,9 +1756,13 @@ expression support in the :mod:`re` module). .. method:: str.isspace() Return true if there are only whitespace characters in the string and there is - at least one character, false otherwise. Whitespace characters are those - characters defined in the Unicode character database as "Other" or "Separator" - and those with bidirectional property being one of "WS", "B", or "S". + at least one character, false otherwise. + + A character is *whitespace* if in the Unicode character database + (see :mod:`unicodedata`), either its general category is ``Zs`` + ("Separator, space"), or its bidirectional class is one of ``WS``, + ``B``, or ``S``. + .. method:: str.istitle() diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py index 36b72e40c7e4..1d6aabdbbcc9 100644 --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -11,6 +11,7 @@ import operator import struct import sys +import unicodedata import unittest import warnings from test import support, string_tests @@ -615,11 +616,21 @@ def test_isspace(self): self.checkequalnofix(True, '\u2000', 'isspace') self.checkequalnofix(True, '\u200a', 'isspace') self.checkequalnofix(False, '\u2014', 'isspace') - # apparently there are no non-BMP spaces chars in Unicode 6 + # There are no non-BMP whitespace chars as of Unicode 12. for ch in ['\U00010401', '\U00010427', '\U00010429', '\U0001044E', '\U0001F40D', '\U0001F46F']: self.assertFalse(ch.isspace(), '{!a} is not space.'.format(ch)) + @support.requires_resource('cpu') + def test_isspace_invariant(self): + for codepoint in range(sys.maxunicode + 1): + char = chr(codepoint) + bidirectional = unicodedata.bidirectional(char) + category = unicodedata.category(char) + self.assertEqual(char.isspace(), + (bidirectional in ('WS', 'B', 'S') + or category == 'Zs')) + def test_isalnum(self): super().test_isalnum() for ch in ['\U00010401', '\U00010427', '\U00010429', '\U0001044E', From webhook-mailer at python.org Mon Aug 19 06:10:24 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 19 Aug 2019 10:10:24 -0000 Subject: [Python-checkins] bpo-36502: Correct documentation of str.isspace() (GH-15019) (GH-15296) Message-ID: https://github.com/python/cpython/commit/0fcdd8d6d67f57733203fc79e6a07a89b924a390 commit: 0fcdd8d6d67f57733203fc79e6a07a89b924a390 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-19T03:10:14-07:00 summary: bpo-36502: Correct documentation of str.isspace() (GH-15019) (GH-15296) The documented definition was much broader than the real one: there are tons of characters with general category "Other", and we don't (and shouldn't) treat most of them as whitespace. Rewrite the definition to agree with the comment on _PyUnicode_IsWhitespace, and with the logic in makeunicodedata.py, which is what generates that function and so ultimately governs. Add suitable breadcrumbs so that a reader who wants to pin down exactly what this definition means (what's a "bidirectional class" of "B"?) can do so. The `unicodedata` module documentation is an appropriate central place for our references to Unicode's own copious documentation, so point there. Also add to the isspace() test a thorough check that the implementation agrees with the intended definition. (cherry picked from commit 8c1c426a631ba02357112657193f82c58d3e08b4) Co-authored-by: Greg Price files: M Doc/library/stdtypes.rst M Lib/test/test_unicode.py diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index d35c171aba39..b9581ce1c9ae 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1731,9 +1731,13 @@ expression support in the :mod:`re` module). .. method:: str.isspace() Return true if there are only whitespace characters in the string and there is - at least one character, false otherwise. Whitespace characters are those - characters defined in the Unicode character database as "Other" or "Separator" - and those with bidirectional property being one of "WS", "B", or "S". + at least one character, false otherwise. + + A character is *whitespace* if in the Unicode character database + (see :mod:`unicodedata`), either its general category is ``Zs`` + ("Separator, space"), or its bidirectional class is one of ``WS``, + ``B``, or ``S``. + .. method:: str.istitle() diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py index 1aad9334074c..4ebd82d3e0c2 100644 --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -11,6 +11,7 @@ import operator import struct import sys +import unicodedata import unittest import warnings from test import support, string_tests @@ -615,11 +616,21 @@ def test_isspace(self): self.checkequalnofix(True, '\u2000', 'isspace') self.checkequalnofix(True, '\u200a', 'isspace') self.checkequalnofix(False, '\u2014', 'isspace') - # apparently there are no non-BMP spaces chars in Unicode 6 + # There are no non-BMP whitespace chars as of Unicode 12. for ch in ['\U00010401', '\U00010427', '\U00010429', '\U0001044E', '\U0001F40D', '\U0001F46F']: self.assertFalse(ch.isspace(), '{!a} is not space.'.format(ch)) + @support.requires_resource('cpu') + def test_isspace_invariant(self): + for codepoint in range(sys.maxunicode + 1): + char = chr(codepoint) + bidirectional = unicodedata.bidirectional(char) + category = unicodedata.category(char) + self.assertEqual(char.isspace(), + (bidirectional in ('WS', 'B', 'S') + or category == 'Zs')) + def test_isalnum(self): super().test_isalnum() for ch in ['\U00010401', '\U00010427', '\U00010429', '\U0001044E', From webhook-mailer at python.org Mon Aug 19 13:07:31 2019 From: webhook-mailer at python.org (Steve Dower) Date: Mon, 19 Aug 2019 17:07:31 -0000 Subject: [Python-checkins] Remove 'unstable' warning for Windows Store package in docs (GH-15334) Message-ID: https://github.com/python/cpython/commit/cf9360e524acafdce99a8a1e48947fd7da06f3d4 commit: cf9360e524acafdce99a8a1e48947fd7da06f3d4 branch: master author: Steve Dower committer: GitHub date: 2019-08-19T10:07:25-07:00 summary: Remove 'unstable' warning for Windows Store package in docs (GH-15334) files: M Doc/using/windows.rst diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index 54f14e604b79..0b87f6e04867 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -293,12 +293,6 @@ The Microsoft Store package .. versionadded:: 3.7.2 -.. note:: - The Microsoft Store package is currently considered unstable while its - interactions with other tools and other copies of Python are evaluated. - While Python itself is stable, this installation method may change its - behavior and capabilities during Python 3.7 releases. - The Microsoft Store package is an easily installable Python interpreter that is intended mainly for interactive use, for example, by students. @@ -318,7 +312,10 @@ session by typing ``python``. Further, pip and IDLE may be used by typing All three commands are also available with version number suffixes, for example, as ``python3.exe`` and ``python3.x.exe`` as well as ``python.exe`` (where ``3.x`` is the specific version you want to launch, -such as |version|). +such as |version|). Open "Manage App Execution Aliases" through Start to +select which version of Python is associated with each command. It is +recommended to make sure that ``pip`` and ``idle`` are consistent with +whichever version of ``python`` is selected. Virtual environments can be created with ``python -m venv`` and activated and used as normal. From webhook-mailer at python.org Mon Aug 19 13:14:35 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 19 Aug 2019 17:14:35 -0000 Subject: [Python-checkins] Remove 'unstable' warning for Windows Store package in docs (GH-15334) Message-ID: https://github.com/python/cpython/commit/9aa0ab1a97a9287420bef99803872879eaa45f8f commit: 9aa0ab1a97a9287420bef99803872879eaa45f8f branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-19T10:14:31-07:00 summary: Remove 'unstable' warning for Windows Store package in docs (GH-15334) (cherry picked from commit cf9360e524acafdce99a8a1e48947fd7da06f3d4) Co-authored-by: Steve Dower files: M Doc/using/windows.rst diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index 462e4c2b6c63..50eb9d3145d2 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -293,12 +293,6 @@ The Microsoft Store package .. versionadded:: 3.7.2 -.. note:: - The Microsoft Store package is currently considered unstable while its - interactions with other tools and other copies of Python are evaluated. - While Python itself is stable, this installation method may change its - behavior and capabilities during Python 3.7 releases. - The Microsoft Store package is an easily installable Python interpreter that is intended mainly for interactive use, for example, by students. @@ -318,7 +312,10 @@ session by typing ``python``. Further, pip and IDLE may be used by typing All three commands are also available with version number suffixes, for example, as ``python3.exe`` and ``python3.x.exe`` as well as ``python.exe`` (where ``3.x`` is the specific version you want to launch, -such as |version|). +such as |version|). Open "Manage App Execution Aliases" through Start to +select which version of Python is associated with each command. It is +recommended to make sure that ``pip`` and ``idle`` are consistent with +whichever version of ``python`` is selected. Virtual environments can be created with ``python -m venv`` and activated and used as normal. From webhook-mailer at python.org Mon Aug 19 18:37:21 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 19 Aug 2019 22:37:21 -0000 Subject: [Python-checkins] bpo-37788: Fix a reference leak if a thread is not joined (GH-15228) Message-ID: https://github.com/python/cpython/commit/d3dcc92778807ae8f7ebe85178f36a29711cd478 commit: d3dcc92778807ae8f7ebe85178f36a29711cd478 branch: master author: Victor Stinner committer: GitHub date: 2019-08-19T23:37:17+01:00 summary: bpo-37788: Fix a reference leak if a thread is not joined (GH-15228) Add threading.Thread.__del__() method to ensure that the thread state lock is removed from the _shutdown_locks list when a thread completes. files: A Misc/NEWS.d/next/Library/2019-08-12-17-21-10.bpo-37788.F0tR05.rst M Lib/test/test_threading.py M Lib/threading.py diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index 7c16974c1630..5e90627822f9 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -761,6 +761,14 @@ def test_shutdown_locks(self): # Daemon threads must never add it to _shutdown_locks. self.assertNotIn(tstate_lock, threading._shutdown_locks) + def test_leak_without_join(self): + # bpo-37788: Test that a thread which is not joined explicitly + # does not leak. Test written for reference leak checks. + def noop(): pass + with support.wait_threads_exit(): + threading.Thread(target=noop).start() + # Thread.join() is not called + class ThreadJoinOnShutdown(BaseTestCase): diff --git a/Lib/threading.py b/Lib/threading.py index 32a3d7c30336..67e1c4facfee 100644 --- a/Lib/threading.py +++ b/Lib/threading.py @@ -806,6 +806,16 @@ class is implemented. # For debugging and _after_fork() _dangling.add(self) + def __del__(self): + if not self._initialized: + return + lock = self._tstate_lock + if lock is not None and not self.daemon: + # ensure that self._tstate_lock is not in _shutdown_locks + # if join() was not called explicitly + with _shutdown_locks_lock: + _shutdown_locks.discard(lock) + def _reset_internal_locks(self, is_alive): # private! Called by _after_fork() to reset our internal locks as # they may be in an invalid state leading to a deadlock or crash. diff --git a/Misc/NEWS.d/next/Library/2019-08-12-17-21-10.bpo-37788.F0tR05.rst b/Misc/NEWS.d/next/Library/2019-08-12-17-21-10.bpo-37788.F0tR05.rst new file mode 100644 index 000000000000..d9b1e82b9223 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-12-17-21-10.bpo-37788.F0tR05.rst @@ -0,0 +1 @@ +Fix a reference leak if a thread is not joined. From webhook-mailer at python.org Mon Aug 19 19:47:11 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 19 Aug 2019 23:47:11 -0000 Subject: [Python-checkins] Revert "bpo-37788: Fix a reference leak if a thread is not joined (GH-15228)" (GH-15338) Message-ID: https://github.com/python/cpython/commit/d11c2c607768fa549b1aed7899edc061b2ebf19f commit: d11c2c607768fa549b1aed7899edc061b2ebf19f branch: master author: Victor Stinner committer: GitHub date: 2019-08-20T00:47:07+01:00 summary: Revert "bpo-37788: Fix a reference leak if a thread is not joined (GH-15228)" (GH-15338) This reverts commit d3dcc92778807ae8f7ebe85178f36a29711cd478. files: D Misc/NEWS.d/next/Library/2019-08-12-17-21-10.bpo-37788.F0tR05.rst M Lib/test/test_threading.py M Lib/threading.py diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index 5e90627822f9..7c16974c1630 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -761,14 +761,6 @@ def test_shutdown_locks(self): # Daemon threads must never add it to _shutdown_locks. self.assertNotIn(tstate_lock, threading._shutdown_locks) - def test_leak_without_join(self): - # bpo-37788: Test that a thread which is not joined explicitly - # does not leak. Test written for reference leak checks. - def noop(): pass - with support.wait_threads_exit(): - threading.Thread(target=noop).start() - # Thread.join() is not called - class ThreadJoinOnShutdown(BaseTestCase): diff --git a/Lib/threading.py b/Lib/threading.py index 67e1c4facfee..32a3d7c30336 100644 --- a/Lib/threading.py +++ b/Lib/threading.py @@ -806,16 +806,6 @@ class is implemented. # For debugging and _after_fork() _dangling.add(self) - def __del__(self): - if not self._initialized: - return - lock = self._tstate_lock - if lock is not None and not self.daemon: - # ensure that self._tstate_lock is not in _shutdown_locks - # if join() was not called explicitly - with _shutdown_locks_lock: - _shutdown_locks.discard(lock) - def _reset_internal_locks(self, is_alive): # private! Called by _after_fork() to reset our internal locks as # they may be in an invalid state leading to a deadlock or crash. diff --git a/Misc/NEWS.d/next/Library/2019-08-12-17-21-10.bpo-37788.F0tR05.rst b/Misc/NEWS.d/next/Library/2019-08-12-17-21-10.bpo-37788.F0tR05.rst deleted file mode 100644 index d9b1e82b9223..000000000000 --- a/Misc/NEWS.d/next/Library/2019-08-12-17-21-10.bpo-37788.F0tR05.rst +++ /dev/null @@ -1 +0,0 @@ -Fix a reference leak if a thread is not joined. From webhook-mailer at python.org Mon Aug 19 21:35:18 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 20 Aug 2019 01:35:18 -0000 Subject: [Python-checkins] Delete stale comment in Python/getopt.c. (GH-14719) Message-ID: https://github.com/python/cpython/commit/d13968b1be8f4c0035b0ea891ee0f0df026cbf6c commit: d13968b1be8f4c0035b0ea891ee0f0df026cbf6c branch: master author: Hansraj Das committer: Benjamin Peterson date: 2019-08-19T18:35:13-07:00 summary: Delete stale comment in Python/getopt.c. (GH-14719) files: M Python/getopt.c diff --git a/Python/getopt.c b/Python/getopt.c index 89f773417e31..708d9ce49628 100644 --- a/Python/getopt.c +++ b/Python/getopt.c @@ -18,10 +18,6 @@ * DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA, OR PROFITS, WHETHER * IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT * OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. - * - * Nevertheless, I would like to know about bugs in this library or - * suggestions for improvement. Send bug reports and feedback to - * davegottner at delphi.com. *---------------------------------------------------------------------------*/ /* Modified to support --help and --version, as well as /? on Windows From webhook-mailer at python.org Mon Aug 19 21:41:35 2019 From: webhook-mailer at python.org (Ethan Furman) Date: Tue, 20 Aug 2019 01:41:35 -0000 Subject: [Python-checkins] Minor documentation fixes on library/enum (GH-15234) Message-ID: https://github.com/python/cpython/commit/d3c8d735147ccdde1f9bf18ba481da67564837bf commit: d3c8d735147ccdde1f9bf18ba481da67564837bf branch: master author: Antoine <43954001+awecx at users.noreply.github.com> committer: Ethan Furman date: 2019-08-19T18:41:31-07:00 summary: Minor documentation fixes on library/enum (GH-15234) * Minor documentation fixes on library/enum files: M Doc/library/enum.rst M Misc/ACKS diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index d7d319a95134..1d6912aaf19a 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -383,8 +383,8 @@ enumeration, with the exception of special methods (:meth:`__str__`, variable names listed in :attr:`_ignore_`. Note: if your enumeration defines :meth:`__new__` and/or :meth:`__init__` then -whatever value(s) were given to the enum member will be passed into those -methods. See `Planet`_ for an example. +any value(s) given to the enum member will be passed into those methods. +See `Planet`_ for an example. Restricted Enum subclassing @@ -730,8 +730,7 @@ Some rules: 2. While :class:`Enum` can have members of any type, once you mix in an additional type, all the members must have values of that type, e.g. :class:`int` above. This restriction does not apply to mix-ins which only - add methods and don't specify another data type such as :class:`int` or - :class:`str`. + add methods and don't specify another type. 3. When another data type is mixed in, the :attr:`value` attribute is *not the same* as the enum member itself, although it is equivalent and will compare equal. @@ -1054,7 +1053,7 @@ Supported ``_sunder_`` names - ``_missing_`` -- a lookup function used when a value is not found; may be overridden -- ``_ignore_`` -- a list of names, either as a :func:`list` or a :func:`str`, +- ``_ignore_`` -- a list of names, either as a :class:`list` or a :class:`str`, that will not be transformed into members, and will be removed from the final class - ``_order_`` -- used in Python 2/3 code to ensure member order is consistent diff --git a/Misc/ACKS b/Misc/ACKS index 52a5d70e5e2d..3a796860b798 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1771,6 +1771,7 @@ Steve Weber Corran Webster Glyn Webster Phil Webster +Antoine Wecxsteen Stefan Wehr Zack Weinberg Bob Weiner From webhook-mailer at python.org Tue Aug 20 01:40:32 2019 From: webhook-mailer at python.org (Eric V. Smith) Date: Tue, 20 Aug 2019 05:40:32 -0000 Subject: [Python-checkins] bpo-37868: Improve is_dataclass for instances. (GH-15325) Message-ID: https://github.com/python/cpython/commit/b0f4dab8735f692bcfedcf0fa9a25e238a554bab commit: b0f4dab8735f692bcfedcf0fa9a25e238a554bab branch: master author: Eric V. Smith committer: GitHub date: 2019-08-20T01:40:28-04:00 summary: bpo-37868: Improve is_dataclass for instances. (GH-15325) files: A Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst M Lib/dataclasses.py M Lib/test/test_dataclasses.py diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index f778a27912de..9020c905d117 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -1015,13 +1015,14 @@ def fields(class_or_instance): def _is_dataclass_instance(obj): """Returns True if obj is an instance of a dataclass.""" - return not isinstance(obj, type) and hasattr(obj, _FIELDS) + return hasattr(type(obj), _FIELDS) def is_dataclass(obj): """Returns True if obj is a dataclass or an instance of a dataclass.""" - return hasattr(obj, _FIELDS) + cls = obj if isinstance(obj, type) else type(obj) + return hasattr(cls, _FIELDS) def asdict(obj, *, dict_factory=dict): diff --git a/Lib/test/test_dataclasses.py b/Lib/test/test_dataclasses.py index ea42904b003d..037bf4c22142 100755 --- a/Lib/test/test_dataclasses.py +++ b/Lib/test/test_dataclasses.py @@ -1300,6 +1300,32 @@ class D: self.assertTrue(is_dataclass(d.d)) self.assertFalse(is_dataclass(d.e)) + def test_is_dataclass_when_getattr_always_returns(self): + # See bpo-37868. + class A: + def __getattr__(self, key): + return 0 + self.assertFalse(is_dataclass(A)) + a = A() + + # Also test for an instance attribute. + class B: + pass + b = B() + b.__dataclass_fields__ = [] + + for obj in a, b: + with self.subTest(obj=obj): + self.assertFalse(is_dataclass(obj)) + + # Indirect tests for _is_dataclass_instance(). + with self.assertRaisesRegex(TypeError, 'should be called on dataclass instances'): + asdict(obj) + with self.assertRaisesRegex(TypeError, 'should be called on dataclass instances'): + astuple(obj) + with self.assertRaisesRegex(TypeError, 'should be called on dataclass instances'): + replace(obj, x=0) + def test_helper_fields_with_class_instance(self): # Check that we can call fields() on either a class or instance, # and get back the same thing. diff --git a/Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst b/Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst new file mode 100644 index 000000000000..7f342e1ee354 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst @@ -0,0 +1,3 @@ +Fix dataclasses.is_dataclass when given an instance that never raises +AttributeError in __getattr__. That is, an object that returns something +for __dataclass_fields__ even if it's not a dataclass. From webhook-mailer at python.org Tue Aug 20 01:59:31 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 20 Aug 2019 05:59:31 -0000 Subject: [Python-checkins] bpo-37868: Improve is_dataclass for instances. (GH-15325) Message-ID: https://github.com/python/cpython/commit/1271ee8187df31debda7c556882a51ec356ca534 commit: 1271ee8187df31debda7c556882a51ec356ca534 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-19T22:59:21-07:00 summary: bpo-37868: Improve is_dataclass for instances. (GH-15325) (cherry picked from commit b0f4dab8735f692bcfedcf0fa9a25e238a554bab) Co-authored-by: Eric V. Smith files: A Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst M Lib/dataclasses.py M Lib/test/test_dataclasses.py diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index 23712fa8709d..2f0e5ff0b6e0 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -1015,13 +1015,14 @@ def fields(class_or_instance): def _is_dataclass_instance(obj): """Returns True if obj is an instance of a dataclass.""" - return not isinstance(obj, type) and hasattr(obj, _FIELDS) + return hasattr(type(obj), _FIELDS) def is_dataclass(obj): """Returns True if obj is a dataclass or an instance of a dataclass.""" - return hasattr(obj, _FIELDS) + cls = obj if isinstance(obj, type) else type(obj) + return hasattr(cls, _FIELDS) def asdict(obj, *, dict_factory=dict): diff --git a/Lib/test/test_dataclasses.py b/Lib/test/test_dataclasses.py index facf9d2e2c05..0885c9798fe6 100755 --- a/Lib/test/test_dataclasses.py +++ b/Lib/test/test_dataclasses.py @@ -1300,6 +1300,32 @@ class D: self.assertTrue(is_dataclass(d.d)) self.assertFalse(is_dataclass(d.e)) + def test_is_dataclass_when_getattr_always_returns(self): + # See bpo-37868. + class A: + def __getattr__(self, key): + return 0 + self.assertFalse(is_dataclass(A)) + a = A() + + # Also test for an instance attribute. + class B: + pass + b = B() + b.__dataclass_fields__ = [] + + for obj in a, b: + with self.subTest(obj=obj): + self.assertFalse(is_dataclass(obj)) + + # Indirect tests for _is_dataclass_instance(). + with self.assertRaisesRegex(TypeError, 'should be called on dataclass instances'): + asdict(obj) + with self.assertRaisesRegex(TypeError, 'should be called on dataclass instances'): + astuple(obj) + with self.assertRaisesRegex(TypeError, 'should be called on dataclass instances'): + replace(obj, x=0) + def test_helper_fields_with_class_instance(self): # Check that we can call fields() on either a class or instance, # and get back the same thing. diff --git a/Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst b/Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst new file mode 100644 index 000000000000..7f342e1ee354 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst @@ -0,0 +1,3 @@ +Fix dataclasses.is_dataclass when given an instance that never raises +AttributeError in __getattr__. That is, an object that returns something +for __dataclass_fields__ even if it's not a dataclass. From webhook-mailer at python.org Tue Aug 20 02:02:00 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 20 Aug 2019 06:02:00 -0000 Subject: [Python-checkins] bpo-37868: Improve is_dataclass for instances. (GH-15325) Message-ID: https://github.com/python/cpython/commit/02c1457a036c2af3e91beb952afdb66d9c806435 commit: 02c1457a036c2af3e91beb952afdb66d9c806435 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-19T23:01:55-07:00 summary: bpo-37868: Improve is_dataclass for instances. (GH-15325) (cherry picked from commit b0f4dab8735f692bcfedcf0fa9a25e238a554bab) Co-authored-by: Eric V. Smith files: A Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst M Lib/dataclasses.py M Lib/test/test_dataclasses.py diff --git a/Lib/dataclasses.py b/Lib/dataclasses.py index 7725621e0fae..33e26460c74e 100644 --- a/Lib/dataclasses.py +++ b/Lib/dataclasses.py @@ -1011,13 +1011,14 @@ def fields(class_or_instance): def _is_dataclass_instance(obj): """Returns True if obj is an instance of a dataclass.""" - return not isinstance(obj, type) and hasattr(obj, _FIELDS) + return hasattr(type(obj), _FIELDS) def is_dataclass(obj): """Returns True if obj is a dataclass or an instance of a dataclass.""" - return hasattr(obj, _FIELDS) + cls = obj if isinstance(obj, type) else type(obj) + return hasattr(cls, _FIELDS) def asdict(obj, *, dict_factory=dict): diff --git a/Lib/test/test_dataclasses.py b/Lib/test/test_dataclasses.py index f36098c69339..99086e5f6d25 100755 --- a/Lib/test/test_dataclasses.py +++ b/Lib/test/test_dataclasses.py @@ -1294,6 +1294,32 @@ class D: self.assertTrue(is_dataclass(d.d)) self.assertFalse(is_dataclass(d.e)) + def test_is_dataclass_when_getattr_always_returns(self): + # See bpo-37868. + class A: + def __getattr__(self, key): + return 0 + self.assertFalse(is_dataclass(A)) + a = A() + + # Also test for an instance attribute. + class B: + pass + b = B() + b.__dataclass_fields__ = [] + + for obj in a, b: + with self.subTest(obj=obj): + self.assertFalse(is_dataclass(obj)) + + # Indirect tests for _is_dataclass_instance(). + with self.assertRaisesRegex(TypeError, 'should be called on dataclass instances'): + asdict(obj) + with self.assertRaisesRegex(TypeError, 'should be called on dataclass instances'): + astuple(obj) + with self.assertRaisesRegex(TypeError, 'should be called on dataclass instances'): + replace(obj, x=0) + def test_helper_fields_with_class_instance(self): # Check that we can call fields() on either a class or instance, # and get back the same thing. diff --git a/Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst b/Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst new file mode 100644 index 000000000000..7f342e1ee354 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-17-22-33-54.bpo-37868.hp64fi.rst @@ -0,0 +1,3 @@ +Fix dataclasses.is_dataclass when given an instance that never raises +AttributeError in __getattr__. That is, an object that returns something +for __dataclass_fields__ even if it's not a dataclass. From webhook-mailer at python.org Tue Aug 20 07:28:10 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 20 Aug 2019 11:28:10 -0000 Subject: [Python-checkins] bpo-37732: Fix GCC warning in _PyObject_Malloc() (GH-15333) Message-ID: https://github.com/python/cpython/commit/18f8dcfa10d8a858b152d12a9ad8fa83b7e967f0 commit: 18f8dcfa10d8a858b152d12a9ad8fa83b7e967f0 branch: master author: Victor Stinner committer: GitHub date: 2019-08-20T12:28:02+01:00 summary: bpo-37732: Fix GCC warning in _PyObject_Malloc() (GH-15333) pymalloc_alloc() now returns directly the pointer, return NULL on memory allocation error. allocate_from_new_pool() already uses NULL as marker for "allocation failed". files: M Objects/obmalloc.c diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c index 6ca7cd84eda8..40c098ddc30a 100644 --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -1580,35 +1580,35 @@ allocate_from_new_pool(uint size) /* pymalloc allocator - Return 1 if pymalloc allocated memory and wrote the pointer into *ptr_p. + Return a pointer to newly allocated memory if pymalloc allocated memory. - Return 0 if pymalloc failed to allocate the memory block: on bigger + Return NULL if pymalloc failed to allocate the memory block: on bigger requests, on error in the code below (as a last chance to serve the request) or when the max memory limit has been reached. */ -static inline int -pymalloc_alloc(void *ctx, void **ptr_p, size_t nbytes) +static inline void* +pymalloc_alloc(void *ctx, size_t nbytes) { #ifdef WITH_VALGRIND if (UNLIKELY(running_on_valgrind == -1)) { running_on_valgrind = RUNNING_ON_VALGRIND; } if (UNLIKELY(running_on_valgrind)) { - return 0; + return NULL; } #endif if (UNLIKELY(nbytes == 0)) { - return 0; + return NULL; } if (UNLIKELY(nbytes > SMALL_REQUEST_THRESHOLD)) { - return 0; + return NULL; } uint size = (uint)(nbytes - 1) >> ALIGNMENT_SHIFT; poolp pool = usedpools[size + size]; block *bp; - + if (LIKELY(pool != pool->nextpool)) { /* * There is a used pool for this size class. @@ -1616,6 +1616,7 @@ pymalloc_alloc(void *ctx, void **ptr_p, size_t nbytes) */ ++pool->ref.count; bp = pool->freeblock; + assert(bp != NULL); if (UNLIKELY((pool->freeblock = *(block **)bp) == NULL)) { // Reached the end of the free list, try to extend it. @@ -1627,22 +1628,17 @@ pymalloc_alloc(void *ctx, void **ptr_p, size_t nbytes) * available: use a free pool. */ bp = allocate_from_new_pool(size); - if (UNLIKELY(bp == NULL)) { - return 0; - } } - assert(bp != NULL); - *ptr_p = (void *)bp; - return 1; + return (void *)bp; } static void * _PyObject_Malloc(void *ctx, size_t nbytes) { - void* ptr; - if (LIKELY(pymalloc_alloc(ctx, &ptr, nbytes))) { + void* ptr = pymalloc_alloc(ctx, nbytes); + if (LIKELY(ptr != NULL)) { return ptr; } @@ -1657,12 +1653,11 @@ _PyObject_Malloc(void *ctx, size_t nbytes) static void * _PyObject_Calloc(void *ctx, size_t nelem, size_t elsize) { - void* ptr; - assert(elsize == 0 || nelem <= (size_t)PY_SSIZE_T_MAX / elsize); size_t nbytes = nelem * elsize; - if (LIKELY(pymalloc_alloc(ctx, &ptr, nbytes))) { + void* ptr = pymalloc_alloc(ctx, nbytes); + if (LIKELY(ptr != NULL)) { memset(ptr, 0, nbytes); return ptr; } @@ -1711,8 +1706,8 @@ insert_to_freepool(poolp pool) * are no arenas in usable_arenas with that value. */ struct arena_object* lastnf = nfp2lasta[nf]; - assert((nf == 0 && lastnf == NULL) || - (nf > 0 && + assert((nf == 0 && lastnf == NULL) || + (nf > 0 && lastnf != NULL && lastnf->nfreepools == nf && (lastnf->nextarena == NULL || From webhook-mailer at python.org Tue Aug 20 08:44:38 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 20 Aug 2019 12:44:38 -0000 Subject: [Python-checkins] bpo-37732: Fix GCC warning in _PyObject_Malloc() (GH-15333) (GH-15342) Message-ID: https://github.com/python/cpython/commit/30e5aff5fb0e3841107ddd4539a1f5b8521c80fb commit: 30e5aff5fb0e3841107ddd4539a1f5b8521c80fb branch: 3.8 author: Victor Stinner committer: GitHub date: 2019-08-20T13:44:32+01:00 summary: bpo-37732: Fix GCC warning in _PyObject_Malloc() (GH-15333) (GH-15342) pymalloc_alloc() now returns directly the pointer, return NULL on memory allocation error. allocate_from_new_pool() already uses NULL as marker for "allocation failed". (cherry picked from commit 18f8dcfa10d8a858b152d12a9ad8fa83b7e967f0) files: M Objects/obmalloc.c diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c index 0501cb992008..f420e1976174 100644 --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -1415,13 +1415,13 @@ address_in_range(void *p, poolp pool) block allocations typically result in a couple of instructions). Unless the optimizer reorders everything, being too smart... - Return 1 if pymalloc allocated memory and wrote the pointer into *ptr_p. + Return a pointer to newly allocated memory if pymalloc allocated memory. - Return 0 if pymalloc failed to allocate the memory block: on bigger + Return NULL if pymalloc failed to allocate the memory block: on bigger requests, on error in the code below (as a last chance to serve the request) or when the max memory limit has been reached. */ -static int -pymalloc_alloc(void *ctx, void **ptr_p, size_t nbytes) +static void* +pymalloc_alloc(void *ctx, size_t nbytes) { block *bp; poolp pool; @@ -1433,15 +1433,15 @@ pymalloc_alloc(void *ctx, void **ptr_p, size_t nbytes) running_on_valgrind = RUNNING_ON_VALGRIND; } if (UNLIKELY(running_on_valgrind)) { - return 0; + return NULL; } #endif if (nbytes == 0) { - return 0; + return NULL; } if (nbytes > SMALL_REQUEST_THRESHOLD) { - return 0; + return NULL; } /* @@ -1609,19 +1609,18 @@ pymalloc_alloc(void *ctx, void **ptr_p, size_t nbytes) success: assert(bp != NULL); - *ptr_p = (void *)bp; - return 1; + return (void *)bp; failed: - return 0; + return NULL; } static void * _PyObject_Malloc(void *ctx, size_t nbytes) { - void* ptr; - if (pymalloc_alloc(ctx, &ptr, nbytes)) { + void* ptr = pymalloc_alloc(ctx, nbytes); + if (ptr != NULL) { _Py_AllocatedBlocks++; return ptr; } @@ -1637,12 +1636,11 @@ _PyObject_Malloc(void *ctx, size_t nbytes) static void * _PyObject_Calloc(void *ctx, size_t nelem, size_t elsize) { - void* ptr; - assert(elsize == 0 || nelem <= (size_t)PY_SSIZE_T_MAX / elsize); size_t nbytes = nelem * elsize; - if (pymalloc_alloc(ctx, &ptr, nbytes)) { + void *ptr = pymalloc_alloc(ctx, nbytes); + if (ptr != NULL) { memset(ptr, 0, nbytes); _Py_AllocatedBlocks++; return ptr; @@ -1743,8 +1741,8 @@ pymalloc_free(void *ctx, void *p) * are no arenas in usable_arenas with that value. */ struct arena_object* lastnf = nfp2lasta[nf]; - assert((nf == 0 && lastnf == NULL) || - (nf > 0 && + assert((nf == 0 && lastnf == NULL) || + (nf > 0 && lastnf != NULL && lastnf->nfreepools == nf && (lastnf->nextarena == NULL || From webhook-mailer at python.org Tue Aug 20 10:29:27 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 20 Aug 2019 14:29:27 -0000 Subject: [Python-checkins] bpo-37732: Fix GCC warning in _PyObject_Malloc() (GH-15333) (GH-15342) (GH-15343) Message-ID: https://github.com/python/cpython/commit/c9a484a1e76384680b90454018389465760dbeae commit: c9a484a1e76384680b90454018389465760dbeae branch: 3.7 author: Victor Stinner committer: GitHub date: 2019-08-20T15:29:08+01:00 summary: bpo-37732: Fix GCC warning in _PyObject_Malloc() (GH-15333) (GH-15342) (GH-15343) pymalloc_alloc() now returns directly the pointer, return NULL on memory allocation error. allocate_from_new_pool() already uses NULL as marker for "allocation failed". (cherry picked from commit 18f8dcfa10d8a858b152d12a9ad8fa83b7e967f0) (cherry picked from commit 30e5aff5fb0e3841107ddd4539a1f5b8521c80fb) files: M Objects/obmalloc.c diff --git a/Objects/obmalloc.c b/Objects/obmalloc.c index 8446efcbd440..1248a3937e0d 100644 --- a/Objects/obmalloc.c +++ b/Objects/obmalloc.c @@ -1377,13 +1377,13 @@ address_in_range(void *p, poolp pool) block allocations typically result in a couple of instructions). Unless the optimizer reorders everything, being too smart... - Return 1 if pymalloc allocated memory and wrote the pointer into *ptr_p. + Return a pointer to newly allocated memory if pymalloc allocated memory. - Return 0 if pymalloc failed to allocate the memory block: on bigger + Return NULL if pymalloc failed to allocate the memory block: on bigger requests, on error in the code below (as a last chance to serve the request) or when the max memory limit has been reached. */ -static int -pymalloc_alloc(void *ctx, void **ptr_p, size_t nbytes) +static void* +pymalloc_alloc(void *ctx, size_t nbytes) { block *bp; poolp pool; @@ -1395,15 +1395,15 @@ pymalloc_alloc(void *ctx, void **ptr_p, size_t nbytes) running_on_valgrind = RUNNING_ON_VALGRIND; } if (UNLIKELY(running_on_valgrind)) { - return 0; + return NULL; } #endif if (nbytes == 0) { - return 0; + return NULL; } if (nbytes > SMALL_REQUEST_THRESHOLD) { - return 0; + return NULL; } LOCK(); @@ -1564,20 +1564,19 @@ pymalloc_alloc(void *ctx, void **ptr_p, size_t nbytes) success: UNLOCK(); assert(bp != NULL); - *ptr_p = (void *)bp; - return 1; + return (void *)bp; failed: UNLOCK(); - return 0; + return NULL; } static void * _PyObject_Malloc(void *ctx, size_t nbytes) { - void* ptr; - if (pymalloc_alloc(ctx, &ptr, nbytes)) { + void* ptr = pymalloc_alloc(ctx, nbytes); + if (ptr != NULL) { _Py_AllocatedBlocks++; return ptr; } @@ -1593,12 +1592,11 @@ _PyObject_Malloc(void *ctx, size_t nbytes) static void * _PyObject_Calloc(void *ctx, size_t nelem, size_t elsize) { - void* ptr; - assert(elsize == 0 || nelem <= (size_t)PY_SSIZE_T_MAX / elsize); size_t nbytes = nelem * elsize; - if (pymalloc_alloc(ctx, &ptr, nbytes)) { + void *ptr = pymalloc_alloc(ctx, nbytes); + if (ptr != NULL) { memset(ptr, 0, nbytes); _Py_AllocatedBlocks++; return ptr; From webhook-mailer at python.org Tue Aug 20 10:46:50 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 20 Aug 2019 14:46:50 -0000 Subject: [Python-checkins] bpo-15913: Implement PyBuffer_SizeFromFormat() (GH-13873) Message-ID: https://github.com/python/cpython/commit/9e66aba99925eebacfe137d9deb0ef1fdbc2d5db commit: 9e66aba99925eebacfe137d9deb0ef1fdbc2d5db branch: master author: Joannah Nanjekye <33177550+nanjekyejoannah at users.noreply.github.com> committer: Victor Stinner date: 2019-08-20T15:46:36+01:00 summary: bpo-15913: Implement PyBuffer_SizeFromFormat() (GH-13873) Implement PyBuffer_SizeFromFormat() function (previously documented but not implemented): call struct.calcsize(). files: A Misc/NEWS.d/next/Core and Builtins/2019-06-06-20-52-38.bpo-15913.5Sg5cv.rst M Doc/c-api/buffer.rst M Include/cpython/abstract.h M Lib/test/test_buffer.py M Modules/_testcapimodule.c M Objects/abstract.c diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst old mode 100644 new mode 100755 index fdb8bd19d218..2536c743b247 --- a/Doc/c-api/buffer.rst +++ b/Doc/c-api/buffer.rst @@ -462,10 +462,12 @@ Buffer-related functions :c:func:`PyObject_GetBuffer`. -.. c:function:: Py_ssize_t PyBuffer_SizeFromFormat(const char *) +.. c:function:: Py_ssize_t PyBuffer_SizeFromFormat(const char *format) Return the implied :c:data:`~Py_buffer.itemsize` from :c:data:`~Py_buffer.format`. - This function is not yet implemented. + On error, raise an exception and return -1. + + .. versionadded:: 3.9 .. c:function:: int PyBuffer_IsContiguous(Py_buffer *view, char order) diff --git a/Include/cpython/abstract.h b/Include/cpython/abstract.h old mode 100644 new mode 100755 index 62a113fc00e2..04e4a9e7bd27 --- a/Include/cpython/abstract.h +++ b/Include/cpython/abstract.h @@ -243,7 +243,7 @@ PyAPI_FUNC(void *) PyBuffer_GetPointer(Py_buffer *view, Py_ssize_t *indices); /* Return the implied itemsize of the data-format area from a struct-style description. */ -PyAPI_FUNC(int) PyBuffer_SizeFromFormat(const char *); +PyAPI_FUNC(Py_ssize_t) PyBuffer_SizeFromFormat(const char *format); /* Implementation in memoryobject.c */ PyAPI_FUNC(int) PyBuffer_ToContiguous(void *buf, Py_buffer *view, diff --git a/Lib/test/test_buffer.py b/Lib/test/test_buffer.py old mode 100644 new mode 100755 index 47413c03d663..5fa52bffc22b --- a/Lib/test/test_buffer.py +++ b/Lib/test/test_buffer.py @@ -43,6 +43,11 @@ except ImportError: numpy_array = None +try: + import _testcapi +except ImportError: + _testcapi = None + SHORT_TEST = True @@ -4412,6 +4417,13 @@ def test_issue_7385(self): x = ndarray([1,2,3], shape=[3], flags=ND_GETBUF_FAIL) self.assertRaises(BufferError, memoryview, x) + @support.cpython_only + def test_pybuffer_size_from_format(self): + # basic tests + for format in ('', 'ii', '3s'): + self.assertEqual(_testcapi.PyBuffer_SizeFromFormat(format), + struct.calcsize(format)) + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/Core and Builtins/2019-06-06-20-52-38.bpo-15913.5Sg5cv.rst b/Misc/NEWS.d/next/Core and Builtins/2019-06-06-20-52-38.bpo-15913.5Sg5cv.rst new file mode 100644 index 000000000000..0fbfcb37abb4 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2019-06-06-20-52-38.bpo-15913.5Sg5cv.rst @@ -0,0 +1,3 @@ +Implement :c:func:`PyBuffer_SizeFromFormat()` function (previously +documented but not implemented): call :func:`struct.calcsize`. +Patch by Joannah Nanjekye. diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c old mode 100644 new mode 100755 index 8a6e741d2810..84f2651641c7 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -3363,6 +3363,26 @@ getbuffer_with_null_view(PyObject* self, PyObject *obj) Py_RETURN_NONE; } +/* PyBuffer_SizeFromFormat() */ +static PyObject * +test_PyBuffer_SizeFromFormat(PyObject *self, PyObject *args) +{ + const char *format; + Py_ssize_t result; + + if (!PyArg_ParseTuple(args, "s:test_PyBuffer_SizeFromFormat", + &format)) { + return NULL; + } + + result = PyBuffer_SizeFromFormat(format); + if (result == -1) { + return NULL; + } + + return PyLong_FromSsize_t(result); +} + /* Test that the fatal error from not having a current thread doesn't cause an infinite loop. Run via Lib/test/test_capi.py */ static PyObject * @@ -5153,6 +5173,7 @@ static PyMethodDef TestMethods[] = { {"test_pep3118_obsolete_write_locks", (PyCFunction)test_pep3118_obsolete_write_locks, METH_NOARGS}, #endif {"getbuffer_with_null_view", getbuffer_with_null_view, METH_O}, + {"PyBuffer_SizeFromFormat", test_PyBuffer_SizeFromFormat, METH_VARARGS}, {"test_buildvalue_N", test_buildvalue_N, METH_NOARGS}, {"get_args", get_args, METH_VARARGS}, {"get_kwargs", (PyCFunction)(void(*)(void))get_kwargs, METH_VARARGS|METH_KEYWORDS}, diff --git a/Objects/abstract.c b/Objects/abstract.c old mode 100644 new mode 100755 index f93d73fa7571..3db56fab2c8d --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -495,6 +495,48 @@ _Py_add_one_to_index_C(int nd, Py_ssize_t *index, const Py_ssize_t *shape) } } +Py_ssize_t +PyBuffer_SizeFromFormat(const char *format) +{ + PyObject *structmodule = NULL; + PyObject *calcsize = NULL; + PyObject *res = NULL; + PyObject *fmt = NULL; + Py_ssize_t itemsize = -1; + + structmodule = PyImport_ImportModule("struct"); + if (structmodule == NULL) { + return itemsize; + } + + calcsize = PyObject_GetAttrString(structmodule, "calcsize"); + if (calcsize == NULL) { + goto done; + } + + fmt = PyUnicode_FromString(format); + if (fmt == NULL) { + goto done; + } + + res = PyObject_CallFunctionObjArgs(calcsize, fmt, NULL); + if (res == NULL) { + goto done; + } + + itemsize = PyLong_AsSsize_t(res); + if (itemsize < 0) { + goto done; + } + +done: + Py_DECREF(structmodule); + Py_XDECREF(calcsize); + Py_XDECREF(fmt); + Py_XDECREF(res); + return itemsize; +} + int PyBuffer_FromContiguous(Py_buffer *view, void *buf, Py_ssize_t len, char fort) { From webhook-mailer at python.org Tue Aug 20 13:52:38 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 20 Aug 2019 17:52:38 -0000 Subject: [Python-checkins] bpo-32793: Fix a duplicate debug message in smtplib (GH-15341) Message-ID: https://github.com/python/cpython/commit/46a7564578f208df1e0c54fc0520d3b7ca32c981 commit: 46a7564578f208df1e0c54fc0520d3b7ca32c981 branch: master author: Zackery Spytz committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-20T10:52:25-07:00 summary: bpo-32793: Fix a duplicate debug message in smtplib (GH-15341) _get_socket() already prints a debug message for the host and port. https://bugs.python.org/issue32793 Automerge-Triggered-By: @maxking files: A Misc/NEWS.d/next/Library/2019-08-20-05-17-32.bpo-32793.cgpXl6.rst M Lib/smtplib.py diff --git a/Lib/smtplib.py b/Lib/smtplib.py index a634f7ae7336..43a00d9a4e3c 100755 --- a/Lib/smtplib.py +++ b/Lib/smtplib.py @@ -333,8 +333,6 @@ def connect(self, host='localhost', port=0, source_address=None): raise OSError("nonnumeric port") if not port: port = self.default_port - if self.debuglevel > 0: - self._print_debug('connect:', (host, port)) sys.audit("smtplib.connect", self, host, port) self.sock = self._get_socket(host, port, self.timeout) self.file = None diff --git a/Misc/NEWS.d/next/Library/2019-08-20-05-17-32.bpo-32793.cgpXl6.rst b/Misc/NEWS.d/next/Library/2019-08-20-05-17-32.bpo-32793.cgpXl6.rst new file mode 100644 index 000000000000..f715a816efda --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-08-20-05-17-32.bpo-32793.cgpXl6.rst @@ -0,0 +1 @@ +Fix a duplicated debug message when :meth:`smtplib.SMTP.connect` is called. From webhook-mailer at python.org Tue Aug 20 15:21:00 2019 From: webhook-mailer at python.org (Brett Cannon) Date: Tue, 20 Aug 2019 19:21:00 -0000 Subject: [Python-checkins] Remove a dead comment from ossaudiodev.c (#15346) Message-ID: https://github.com/python/cpython/commit/1407038e0bcd3f2543c50cd5476d2d2f15a2a9fb commit: 1407038e0bcd3f2543c50cd5476d2d2f15a2a9fb branch: master author: Brett Cannon <54418+brettcannon at users.noreply.github.com> committer: GitHub date: 2019-08-20T12:20:47-07:00 summary: Remove a dead comment from ossaudiodev.c (#15346) files: M Modules/ossaudiodev.c diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c index 6e050e4bc4ca..c94ba9255acd 100644 --- a/Modules/ossaudiodev.c +++ b/Modules/ossaudiodev.c @@ -14,8 +14,6 @@ * (c) 2002 Gregory P. Ward. All Rights Reserved. * (c) 2002 Python Software Foundation. All Rights Reserved. * - * XXX need a license statement - * * $Id$ */ From webhook-mailer at python.org Tue Aug 20 22:19:50 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 21 Aug 2019 02:19:50 -0000 Subject: [Python-checkins] abstract.c should not be executable. (GH-15348) Message-ID: https://github.com/python/cpython/commit/d33e46d17d33f9b918846982c02ddc17d897c9bc commit: d33e46d17d33f9b918846982c02ddc17d897c9bc branch: master author: Benjamin Peterson committer: GitHub date: 2019-08-20T19:19:43-07:00 summary: abstract.c should not be executable. (GH-15348) files: M Objects/abstract.c diff --git a/Objects/abstract.c b/Objects/abstract.c old mode 100755 new mode 100644 From webhook-mailer at python.org Tue Aug 20 23:51:01 2019 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 21 Aug 2019 03:51:01 -0000 Subject: [Python-checkins] bpo-35518: Skip test that relies on a deceased network service. (GH-15349) Message-ID: https://github.com/python/cpython/commit/5b95a1507e349da5adae6d2ab57deac3bdd12f15 commit: 5b95a1507e349da5adae6d2ab57deac3bdd12f15 branch: master author: Greg Price committer: Benjamin Peterson date: 2019-08-20T20:50:50-07:00 summary: bpo-35518: Skip test that relies on a deceased network service. (GH-15349) If this service had thoroughly vanished, we could just ignore the test until someone gets around to either recreating such a service or redesigning the test to somehow work locally. The `support.transient_internet` mechanism catches the failure to resolve the domain name, and skips the test. But in fact the domain snakebite.net does still exist, as do its nameservers -- and they can be quite slow to reply. As a result this test can easily take 20-30s before it gets auto-skipped. So, skip the test explicitly up front. files: M Lib/test/test_timeout.py diff --git a/Lib/test/test_timeout.py b/Lib/test/test_timeout.py index b54fc826ae0a..b07c07cbfc4d 100644 --- a/Lib/test/test_timeout.py +++ b/Lib/test/test_timeout.py @@ -150,6 +150,7 @@ def setUp(self): def tearDown(self): self.sock.close() + @unittest.skipIf(True, 'need to replace these hosts; see bpo-35518') def testConnectTimeout(self): # Testing connect timeout is tricky: we need to have IP connectivity # to a host that silently drops our packets. We can't simulate this From webhook-mailer at python.org Wed Aug 21 00:09:01 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 21 Aug 2019 04:09:01 -0000 Subject: [Python-checkins] bpo-35518: Skip test that relies on a deceased network service. (GH-15349) Message-ID: https://github.com/python/cpython/commit/44f2c096804e8e3adc09400a59ef9c9ae843f339 commit: 44f2c096804e8e3adc09400a59ef9c9ae843f339 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-20T21:08:57-07:00 summary: bpo-35518: Skip test that relies on a deceased network service. (GH-15349) If this service had thoroughly vanished, we could just ignore the test until someone gets around to either recreating such a service or redesigning the test to somehow work locally. The `support.transient_internet` mechanism catches the failure to resolve the domain name, and skips the test. But in fact the domain snakebite.net does still exist, as do its nameservers -- and they can be quite slow to reply. As a result this test can easily take 20-30s before it gets auto-skipped. So, skip the test explicitly up front. (cherry picked from commit 5b95a1507e349da5adae6d2ab57deac3bdd12f15) Co-authored-by: Greg Price files: M Lib/test/test_timeout.py diff --git a/Lib/test/test_timeout.py b/Lib/test/test_timeout.py index b54fc826ae0a..b07c07cbfc4d 100644 --- a/Lib/test/test_timeout.py +++ b/Lib/test/test_timeout.py @@ -150,6 +150,7 @@ def setUp(self): def tearDown(self): self.sock.close() + @unittest.skipIf(True, 'need to replace these hosts; see bpo-35518') def testConnectTimeout(self): # Testing connect timeout is tricky: we need to have IP connectivity # to a host that silently drops our packets. We can't simulate this From webhook-mailer at python.org Wed Aug 21 00:11:29 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 21 Aug 2019 04:11:29 -0000 Subject: [Python-checkins] bpo-35518: Skip test that relies on a deceased network service. (GH-15349) Message-ID: https://github.com/python/cpython/commit/b9d88e771238b5098842cad8a6ad624621f3f62e commit: b9d88e771238b5098842cad8a6ad624621f3f62e branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-20T21:11:25-07:00 summary: bpo-35518: Skip test that relies on a deceased network service. (GH-15349) If this service had thoroughly vanished, we could just ignore the test until someone gets around to either recreating such a service or redesigning the test to somehow work locally. The `support.transient_internet` mechanism catches the failure to resolve the domain name, and skips the test. But in fact the domain snakebite.net does still exist, as do its nameservers -- and they can be quite slow to reply. As a result this test can easily take 20-30s before it gets auto-skipped. So, skip the test explicitly up front. (cherry picked from commit 5b95a1507e349da5adae6d2ab57deac3bdd12f15) Co-authored-by: Greg Price files: M Lib/test/test_timeout.py diff --git a/Lib/test/test_timeout.py b/Lib/test/test_timeout.py index b54fc826ae0a..b07c07cbfc4d 100644 --- a/Lib/test/test_timeout.py +++ b/Lib/test/test_timeout.py @@ -150,6 +150,7 @@ def setUp(self): def tearDown(self): self.sock.close() + @unittest.skipIf(True, 'need to replace these hosts; see bpo-35518') def testConnectTimeout(self): # Testing connect timeout is tricky: we need to have IP connectivity # to a host that silently drops our packets. We can't simulate this From webhook-mailer at python.org Wed Aug 21 00:12:21 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 21 Aug 2019 04:12:21 -0000 Subject: [Python-checkins] bpo-35518: Skip test that relies on a deceased network service. (GH-15349) Message-ID: https://github.com/python/cpython/commit/198a0d622a696a4c234aa7866d6c15e38839cc76 commit: 198a0d622a696a4c234aa7866d6c15e38839cc76 branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-20T21:12:18-07:00 summary: bpo-35518: Skip test that relies on a deceased network service. (GH-15349) If this service had thoroughly vanished, we could just ignore the test until someone gets around to either recreating such a service or redesigning the test to somehow work locally. The `support.transient_internet` mechanism catches the failure to resolve the domain name, and skips the test. But in fact the domain snakebite.net does still exist, as do its nameservers -- and they can be quite slow to reply. As a result this test can easily take 20-30s before it gets auto-skipped. So, skip the test explicitly up front. (cherry picked from commit 5b95a1507e349da5adae6d2ab57deac3bdd12f15) Co-authored-by: Greg Price files: M Lib/test/test_timeout.py diff --git a/Lib/test/test_timeout.py b/Lib/test/test_timeout.py index bb9252d1a433..51690335b70e 100644 --- a/Lib/test/test_timeout.py +++ b/Lib/test/test_timeout.py @@ -106,6 +106,7 @@ def setUp(self): def tearDown(self): self.sock.close() + @unittest.skipIf(True, 'need to replace these hosts; see bpo-35518') def testConnectTimeout(self): # Choose a private address that is unlikely to exist to prevent # failures due to the connect succeeding before the timeout. From webhook-mailer at python.org Wed Aug 21 00:54:03 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 21 Aug 2019 04:54:03 -0000 Subject: [Python-checkins] Unmark files as executable that can't actually be executed. (GH-15353) Message-ID: https://github.com/python/cpython/commit/9ece4a5057d52c42a8a064a6c0c7f923267fb3db commit: 9ece4a5057d52c42a8a064a6c0c7f923267fb3db branch: master author: Greg Price committer: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> date: 2019-08-20T21:53:59-07:00 summary: Unmark files as executable that can't actually be executed. (GH-15353) There are plenty of legitimate scripts in the tree that begin with a `#!`, but also a few that seem to be marked executable by mistake. Found them with this command -- it gets executable files known to Git, filters to the ones that don't start with a `#!`, and then unmarks them as executable: $ git ls-files --stage \ | perl -lane 'print $F[3] if (!/^100644/)' \ | while read f; do head -c2 "$f" | grep -qxF '#!' \ || chmod a-x "$f"; \ done Looking at the list by hand confirms that we didn't sweep up any files that should have the executable bit after all. In particular * The `.psd` files are images from Photoshop. * The `.bat` files sure look like things that can be run. But we have lots of other `.bat` files, and they don't have this bit set, so it must not be needed for them. Automerge-Triggered-By: @benjaminp files: M .azure-pipelines/posix-deps-apt.sh M Doc/c-api/buffer.rst M Include/cpython/abstract.h M Lib/idlelib/idle.bat M Lib/test/test_buffer.py M Lib/test/test_dataclasses.py M Lib/test/test_importlib/test_abc.py M Lib/turtledemo/two_canvases.py M Mac/Resources/iconsrc/PythonCompiled.psd M Mac/Resources/iconsrc/PythonIcon.psd M Mac/Resources/iconsrc/PythonSource.psd M Misc/NEWS.d/next/Documentation/2019-04-02-19-23-00.bpo-36487.Jg6-MG.rst M Modules/_decimal/tests/runall.bat M Modules/_testcapimodule.c M Modules/socketmodule.c M Modules/socketmodule.h diff --git a/.azure-pipelines/posix-deps-apt.sh b/.azure-pipelines/posix-deps-apt.sh index 4f489903ab56..e0f4ca5d8d8e 100755 --- a/.azure-pipelines/posix-deps-apt.sh +++ b/.azure-pipelines/posix-deps-apt.sh @@ -1,3 +1,4 @@ +#!/bin/sh apt-get update apt-get -yq install \ diff --git a/Doc/c-api/buffer.rst b/Doc/c-api/buffer.rst old mode 100755 new mode 100644 diff --git a/Include/cpython/abstract.h b/Include/cpython/abstract.h old mode 100755 new mode 100644 diff --git a/Lib/idlelib/idle.bat b/Lib/idlelib/idle.bat old mode 100755 new mode 100644 diff --git a/Lib/test/test_buffer.py b/Lib/test/test_buffer.py old mode 100755 new mode 100644 diff --git a/Lib/test/test_dataclasses.py b/Lib/test/test_dataclasses.py old mode 100755 new mode 100644 diff --git a/Lib/test/test_importlib/test_abc.py b/Lib/test/test_importlib/test_abc.py old mode 100755 new mode 100644 diff --git a/Lib/turtledemo/two_canvases.py b/Lib/turtledemo/two_canvases.py old mode 100755 new mode 100644 diff --git a/Mac/Resources/iconsrc/PythonCompiled.psd b/Mac/Resources/iconsrc/PythonCompiled.psd old mode 100755 new mode 100644 diff --git a/Mac/Resources/iconsrc/PythonIcon.psd b/Mac/Resources/iconsrc/PythonIcon.psd old mode 100755 new mode 100644 diff --git a/Mac/Resources/iconsrc/PythonSource.psd b/Mac/Resources/iconsrc/PythonSource.psd old mode 100755 new mode 100644 diff --git a/Misc/NEWS.d/next/Documentation/2019-04-02-19-23-00.bpo-36487.Jg6-MG.rst b/Misc/NEWS.d/next/Documentation/2019-04-02-19-23-00.bpo-36487.Jg6-MG.rst old mode 100755 new mode 100644 diff --git a/Modules/_decimal/tests/runall.bat b/Modules/_decimal/tests/runall.bat old mode 100755 new mode 100644 diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c old mode 100755 new mode 100644 diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c old mode 100755 new mode 100644 diff --git a/Modules/socketmodule.h b/Modules/socketmodule.h old mode 100755 new mode 100644 From webhook-mailer at python.org Wed Aug 21 05:59:24 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 21 Aug 2019 09:59:24 -0000 Subject: [Python-checkins] bpo-37531: Enhance regrtest multiprocess timeout (GH-15345) Message-ID: https://github.com/python/cpython/commit/de2d9eed8bc628533e1628b843cc4c7a5010f6e5 commit: de2d9eed8bc628533e1628b843cc4c7a5010f6e5 branch: master author: Victor Stinner committer: GitHub date: 2019-08-21T10:59:20+01:00 summary: bpo-37531: Enhance regrtest multiprocess timeout (GH-15345) * Write a message when killing a worker process * Put a timeout on the second popen.communicate() call (after killing the process) * Put a timeout on popen.wait() call * Catch popen.kill() and popen.wait() exceptions files: A Misc/NEWS.d/next/Tests/2019-08-20-19-24-19.bpo-37531.wRoXfU.rst M Lib/test/libregrtest/runtest_mp.py diff --git a/Lib/test/libregrtest/runtest_mp.py b/Lib/test/libregrtest/runtest_mp.py index eed643291162..c22479b7976f 100644 --- a/Lib/test/libregrtest/runtest_mp.py +++ b/Lib/test/libregrtest/runtest_mp.py @@ -126,6 +126,38 @@ def __repr__(self): info.append(f'pid={popen.pid}') return '<%s>' % ' '.join(info) + def _kill(self): + dt = time.monotonic() - self.start_time + + popen = self._popen + pid = popen.pid + print("Kill worker process %s running for %.1f sec" % (pid, dt), + file=sys.stderr, flush=True) + + try: + popen.kill() + return True + except OSError as exc: + print("WARNING: Failed to kill worker process %s: %r" % (pid, exc), + file=sys.stderr, flush=True) + return False + + def _close_wait(self): + popen = self._popen + + # stdout and stderr must be closed to ensure that communicate() + # does not hang + popen.stdout.close() + popen.stderr.close() + + try: + popen.wait(JOIN_TIMEOUT) + except (subprocess.TimeoutExpired, OSError) as exc: + print("WARNING: Failed to wait for worker process %s " + "completion (timeout=%.1f sec): %r" + % (popen.pid, JOIN_TIMEOUT, exc), + file=sys.stderr, flush=True) + def kill(self): """ Kill the current process (if any). @@ -135,15 +167,13 @@ def kill(self): """ self._killed = True - popen = self._popen - if popen is None: + if self._popen is None: return - popen.kill() - # stdout and stderr must be closed to ensure that communicate() - # does not hang - popen.stdout.close() - popen.stderr.close() - popen.wait() + + if not self._kill(): + return + + self._close_wait() def mp_result_error(self, test_name, error_type, stdout='', stderr='', err_msg=None): @@ -151,6 +181,23 @@ def mp_result_error(self, test_name, error_type, stdout='', stderr='', result = TestResult(test_name, error_type, test_time, None) return MultiprocessResult(result, stdout, stderr, err_msg) + def _timedout(self, test_name): + self._kill() + + stdout = sterr = '' + popen = self._popen + try: + stdout, stderr = popen.communicate(timeout=JOIN_TIMEOUT) + except (subprocess.TimeoutExpired, OSError) as exc: + print("WARNING: Failed to read worker process %s output " + "(timeout=%.1f sec): %r" + % (popen.pid, exc, timeout), + file=sys.stderr, flush=True) + + self._close_wait() + + return self.mp_result_error(test_name, TIMEOUT, stdout, stderr) + def _runtest(self, test_name): try: self.start_time = time.monotonic() @@ -158,7 +205,7 @@ def _runtest(self, test_name): self._popen = run_test_in_subprocess(test_name, self.ns) popen = self._popen - with popen: + try: try: if self._killed: # If kill() has been called before self._popen is set, @@ -175,12 +222,7 @@ def _runtest(self, test_name): # on reading closed stdout/stderr raise ExitThread - popen.kill() - stdout, stderr = popen.communicate() - self.kill() - - return self.mp_result_error(test_name, TIMEOUT, - stdout, stderr) + return self._timedout(test_name) except OSError: if self._killed: # kill() has been called: communicate() fails @@ -190,8 +232,10 @@ def _runtest(self, test_name): except: self.kill() raise + finally: + self._close_wait() - retcode = popen.wait() + retcode = popen.returncode finally: self.current_test_name = None self._popen = None @@ -286,10 +330,11 @@ def wait_workers(self): if not worker.is_alive(): break dt = time.monotonic() - start_time - print("Wait for regrtest worker %r for %.1f sec" % (worker, dt)) + print("Wait for regrtest worker %r for %.1f sec" % (worker, dt), + flush=True) if dt > JOIN_TIMEOUT: print("Warning -- failed to join a regrtest worker %s" - % worker) + % worker, flush=True) break def _get_result(self): diff --git a/Misc/NEWS.d/next/Tests/2019-08-20-19-24-19.bpo-37531.wRoXfU.rst b/Misc/NEWS.d/next/Tests/2019-08-20-19-24-19.bpo-37531.wRoXfU.rst new file mode 100644 index 000000000000..59500ce67a01 --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2019-08-20-19-24-19.bpo-37531.wRoXfU.rst @@ -0,0 +1,3 @@ +Enhance regrtest multiprocess timeout: write a message when killing a worker +process, catch popen.kill() and popen.wait() exceptions, put a timeout on the +second call to popen.communicate(). From webhook-mailer at python.org Wed Aug 21 07:13:43 2019 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 21 Aug 2019 11:13:43 -0000 Subject: [Python-checkins] bpo-37823: Fix open() link in telnetlib doc (GH-15281) Message-ID: https://github.com/python/cpython/commit/e0b6117e2723327d6741d0aa599408514add5b30 commit: e0b6117e2723327d6741d0aa599408514add5b30 branch: master author: Michael Anckaert committer: Victor Stinner date: 2019-08-21T12:13:34+01:00 summary: bpo-37823: Fix open() link in telnetlib doc (GH-15281) Fixed wrong link to Telnet.open() method in telnetlib documentation. files: M Doc/library/telnetlib.rst diff --git a/Doc/library/telnetlib.rst b/Doc/library/telnetlib.rst index 0262a9b08410..48a9aea50ddd 100644 --- a/Doc/library/telnetlib.rst +++ b/Doc/library/telnetlib.rst @@ -29,7 +29,7 @@ Character), EL (Erase Line), GA (Go Ahead), SB (Subnegotiation Begin). .. class:: Telnet(host=None, port=0[, timeout]) :class:`Telnet` represents a connection to a Telnet server. The instance is - initially not connected by default; the :meth:`open` method must be used to + initially not connected by default; the :meth:`~Telnet.open` method must be used to establish a connection. Alternatively, the host name and optional port number can be passed to the constructor too, in which case the connection to the server will be established before the constructor returns. The optional From webhook-mailer at python.org Wed Aug 21 07:38:09 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 21 Aug 2019 11:38:09 -0000 Subject: [Python-checkins] bpo-37823: Fix open() link in telnetlib doc (GH-15281) Message-ID: https://github.com/python/cpython/commit/c777dec6f47ce649f3dea88d308e36e45538fdac commit: c777dec6f47ce649f3dea88d308e36e45538fdac branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-21T04:38:04-07:00 summary: bpo-37823: Fix open() link in telnetlib doc (GH-15281) Fixed wrong link to Telnet.open() method in telnetlib documentation. (cherry picked from commit e0b6117e2723327d6741d0aa599408514add5b30) Co-authored-by: Michael Anckaert files: M Doc/library/telnetlib.rst diff --git a/Doc/library/telnetlib.rst b/Doc/library/telnetlib.rst index 0262a9b08410..48a9aea50ddd 100644 --- a/Doc/library/telnetlib.rst +++ b/Doc/library/telnetlib.rst @@ -29,7 +29,7 @@ Character), EL (Erase Line), GA (Go Ahead), SB (Subnegotiation Begin). .. class:: Telnet(host=None, port=0[, timeout]) :class:`Telnet` represents a connection to a Telnet server. The instance is - initially not connected by default; the :meth:`open` method must be used to + initially not connected by default; the :meth:`~Telnet.open` method must be used to establish a connection. Alternatively, the host name and optional port number can be passed to the constructor too, in which case the connection to the server will be established before the constructor returns. The optional From webhook-mailer at python.org Wed Aug 21 07:38:29 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 21 Aug 2019 11:38:29 -0000 Subject: [Python-checkins] bpo-37823: Fix open() link in telnetlib doc (GH-15281) Message-ID: https://github.com/python/cpython/commit/72088d56247ee422d8abeef94a89be463737b982 commit: 72088d56247ee422d8abeef94a89be463737b982 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-21T04:38:25-07:00 summary: bpo-37823: Fix open() link in telnetlib doc (GH-15281) Fixed wrong link to Telnet.open() method in telnetlib documentation. (cherry picked from commit e0b6117e2723327d6741d0aa599408514add5b30) Co-authored-by: Michael Anckaert files: M Doc/library/telnetlib.rst diff --git a/Doc/library/telnetlib.rst b/Doc/library/telnetlib.rst index 4ba426425277..931711963b16 100644 --- a/Doc/library/telnetlib.rst +++ b/Doc/library/telnetlib.rst @@ -29,7 +29,7 @@ Character), EL (Erase Line), GA (Go Ahead), SB (Subnegotiation Begin). .. class:: Telnet(host=None, port=0[, timeout]) :class:`Telnet` represents a connection to a Telnet server. The instance is - initially not connected by default; the :meth:`open` method must be used to + initially not connected by default; the :meth:`~Telnet.open` method must be used to establish a connection. Alternatively, the host name and optional port number can be passed to the constructor too, in which case the connection to the server will be established before the constructor returns. The optional From webhook-mailer at python.org Wed Aug 21 07:52:09 2019 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 21 Aug 2019 11:52:09 -0000 Subject: [Python-checkins] bpo-37823: Fix open() link in telnetlib doc (GH-15281) Message-ID: https://github.com/python/cpython/commit/98b11e1160a1749bfdb48bbf4c0dda177296b005 commit: 98b11e1160a1749bfdb48bbf4c0dda177296b005 branch: 2.7 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2019-08-21T04:52:05-07:00 summary: bpo-37823: Fix open() link in telnetlib doc (GH-15281) Fixed wrong link to Telnet.open() method in telnetlib documentation. (cherry picked from commit e0b6117e2723327d6741d0aa599408514add5b30) Co-authored-by: Michael Anckaert files: M Doc/library/telnetlib.rst diff --git a/Doc/library/telnetlib.rst b/Doc/librar