From webhook-mailer at python.org Tue Sep 1 02:07:39 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Tue, 01 Sep 2020 06:07:39 -0000 Subject: [Python-checkins] [doc] Document VIRTUAL_ENV environment variable (GH-21970) Message-ID: https://github.com/python/cpython/commit/3584d4b64a5373440f78237eac734831cfd83f79 commit: 3584d4b64a5373440f78237eac734831cfd83f79 branch: master author: Andre Delfino committer: GitHub date: 2020-09-01T07:07:29+01:00 summary: [doc] Document VIRTUAL_ENV environment variable (GH-21970) files: M Doc/using/venv-create.inc diff --git a/Doc/using/venv-create.inc b/Doc/using/venv-create.inc index c8f6e8f87d567..8f850a74d413c 100644 --- a/Doc/using/venv-create.inc +++ b/Doc/using/venv-create.inc @@ -126,6 +126,10 @@ directory containing the virtual environment): | | PowerShell | PS C:\\> \\Scripts\\Activate.ps1 | +-------------+-----------------+-----------------------------------------+ +When a virtual environment is active, the :envvar:`VIRTUAL_ENV` environment +variable is set to the path of the virtual environment. This can be used to +check if one is running inside a virtual environment. + You don't specifically *need* to activate an environment; activation just prepends the virtual environment's binary directory to your path, so that "python" invokes the virtual environment's Python interpreter and you can run From webhook-mailer at python.org Tue Sep 1 05:40:45 2020 From: webhook-mailer at python.org (Ned Deily) Date: Tue, 01 Sep 2020 09:40:45 -0000 Subject: [Python-checkins] bpo-41685: Temporarily pin setuptools to 49.2.1 in Docs venv. (GH-22038) Message-ID: https://github.com/python/cpython/commit/a4c4e17f959621ab00a98086af1704fb0c896466 commit: a4c4e17f959621ab00a98086af1704fb0c896466 branch: master author: Ned Deily committer: GitHub date: 2020-09-01T05:40:27-04:00 summary: bpo-41685: Temporarily pin setuptools to 49.2.1 in Docs venv. (GH-22038) See https://github.com/pypa/setuptools/pull/2361 files: M Doc/Makefile diff --git a/Doc/Makefile b/Doc/Makefile index b8ca1edfbc60a..87d5d07665c86 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -142,7 +142,8 @@ clean: venv: $(PYTHON) -m venv $(VENVDIR) - $(VENVDIR)/bin/python3 -m pip install -U pip setuptools +# $(VENVDIR)/bin/python3 -m pip install -U pip setuptools + $(VENVDIR)/bin/python3 -m pip install -U pip setuptools==49.2.1 $(VENVDIR)/bin/python3 -m pip install -U Sphinx==2.3.1 blurb python-docs-theme @echo "The venv has been created in the $(VENVDIR) directory" From webhook-mailer at python.org Tue Sep 1 08:59:51 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Tue, 01 Sep 2020 12:59:51 -0000 Subject: [Python-checkins] [doc] Add link to FileHandler in logging (GH-21940) Message-ID: https://github.com/python/cpython/commit/f5a16b4dbf62cb9b48c42098bd5a8cfa665456c3 commit: f5a16b4dbf62cb9b48c42098bd5a8cfa665456c3 branch: master author: Andre Delfino committer: GitHub date: 2020-09-01T13:59:46+01:00 summary: [doc] Add link to FileHandler in logging (GH-21940) Co-authored-by: Andr?s Delfino files: M Doc/library/logging.rst diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst index a446c80ece604..19691d50937a7 100644 --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -1166,9 +1166,9 @@ functions. +--------------+---------------------------------------------+ | Format | Description | +==============+=============================================+ - | *filename* | Specifies that a FileHandler be created, | - | | using the specified filename, rather than a | - | | StreamHandler. | + | *filename* | Specifies that a :class:`FileHandler` be | + | | created, using the specified filename, | + | | rather than a :class:`StreamHandler`. | +--------------+---------------------------------------------+ | *filemode* | If *filename* is specified, open the file | | | in this :ref:`mode `. Defaults | @@ -1192,9 +1192,10 @@ functions. | | :ref:`level `. | +--------------+---------------------------------------------+ | *stream* | Use the specified stream to initialize the | - | | StreamHandler. Note that this argument is | - | | incompatible with *filename* - if both | - | | are present, a ``ValueError`` is raised. | + | | :class:`StreamHandler`. Note that this | + | | argument is incompatible with *filename* - | + | | if both are present, a ``ValueError`` is | + | | raised. | +--------------+---------------------------------------------+ | *handlers* | If specified, this should be an iterable of | | | already created handlers to add to the root | @@ -1213,18 +1214,18 @@ functions. +--------------+---------------------------------------------+ | *encoding* | If this keyword argument is specified along | | | with *filename*, its value is used when the | - | | FileHandler is created, and thus used when | - | | opening the output file. | + | | :class:`FileHandler` is created, and thus | + | | used when opening the output file. | +--------------+---------------------------------------------+ | *errors* | If this keyword argument is specified along | | | with *filename*, its value is used when the | - | | FileHandler is created, and thus used when | - | | opening the output file. If not specified, | - | | the value 'backslashreplace' is used. Note | - | | that if ``None`` is specified, it will be | - | | passed as such to :func:`open`, which means | - | | that it will be treated the same as passing | - | | 'errors'. | + | | :class:`FileHandler` is created, and thus | + | | used when opening the output file. If not | + | | specified, the value 'backslashreplace' is | + | | used. Note that if ``None`` is specified, | + | | it will be passed as such to :func:`open`, | + | | which means that it will be treated the | + | | same as passing 'errors'. | +--------------+---------------------------------------------+ .. versionchanged:: 3.2 From webhook-mailer at python.org Tue Sep 1 10:34:38 2020 From: webhook-mailer at python.org (han-solo) Date: Tue, 01 Sep 2020 14:34:38 -0000 Subject: [Python-checkins] bpo-41681: Fix for `f-string/str.format` error description when using 2 `, ` in format specifier (GH-22036) Message-ID: https://github.com/python/cpython/commit/0d6aa7f0ee38eb453bc8f73bf4830e6172be2f35 commit: 0d6aa7f0ee38eb453bc8f73bf4830e6172be2f35 branch: master author: han-solo committer: GitHub date: 2020-09-01T10:34:29-04:00 summary: bpo-41681: Fix for `f-string/str.format` error description when using 2 `,` in format specifier (GH-22036) * Fixed `f-string/str.format` error description when using two `,` in format specifier. Co-authored-by: millefalcon files: A Misc/NEWS.d/next/Core and Builtins/2020-08-31-17-49-02.bpo-41681.3-VJiH.rst M Lib/test/test_format.py M Lib/test/test_fstring.py M Python/formatter_unicode.c diff --git a/Lib/test/test_format.py b/Lib/test/test_format.py index e9e5bb9cf00a6..dff8c69032187 100644 --- a/Lib/test/test_format.py +++ b/Lib/test/test_format.py @@ -1,6 +1,7 @@ from test.support import verbose, TestFailed import locale import sys +import re import test.support as support import unittest @@ -495,6 +496,25 @@ def test_g_format_has_no_trailing_zeros(self): self.assertEqual(format(12300050.0, ".6g"), "1.23e+07") self.assertEqual(format(12300050.0, "#.6g"), "1.23000e+07") + def test_with_two_commas_in_format_specifier(self): + error_msg = re.escape("Cannot specify ',' with ','.") + with self.assertRaisesRegex(ValueError, error_msg): + '{:,,}'.format(1) + + def test_with_two_underscore_in_format_specifier(self): + error_msg = re.escape("Cannot specify '_' with '_'.") + with self.assertRaisesRegex(ValueError, error_msg): + '{:__}'.format(1) + + def test_with_a_commas_and_an_underscore_in_format_specifier(self): + error_msg = re.escape("Cannot specify both ',' and '_'.") + with self.assertRaisesRegex(ValueError, error_msg): + '{:,_}'.format(1) + + def test_with_an_underscore_and_a_comma_in_format_specifier(self): + error_msg = re.escape("Cannot specify both ',' and '_'.") + with self.assertRaisesRegex(ValueError, error_msg): + '{:,_}'.format(1) if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py index 35a62a0632e2e..b9bede0d9b800 100644 --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -9,6 +9,7 @@ import ast import os +import re import types import decimal import unittest @@ -1198,6 +1199,25 @@ def test_invalid_syntax_error_message(self): with self.assertRaisesRegex(SyntaxError, "f-string: invalid syntax"): compile("f'{a $ b}'", "?", "exec") + def test_with_two_commas_in_format_specifier(self): + error_msg = re.escape("Cannot specify ',' with ','.") + with self.assertRaisesRegex(ValueError, error_msg): + f'{1:,,}' + + def test_with_two_underscore_in_format_specifier(self): + error_msg = re.escape("Cannot specify '_' with '_'.") + with self.assertRaisesRegex(ValueError, error_msg): + f'{1:__}' + + def test_with_a_commas_and_an_underscore_in_format_specifier(self): + error_msg = re.escape("Cannot specify both ',' and '_'.") + with self.assertRaisesRegex(ValueError, error_msg): + f'{1:,_}' + + def test_with_an_underscore_and_a_comma_in_format_specifier(self): + error_msg = re.escape("Cannot specify both ',' and '_'.") + with self.assertRaisesRegex(ValueError, error_msg): + f'{1:,_}' if __name__ == '__main__': unittest.main() diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-31-17-49-02.bpo-41681.3-VJiH.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-31-17-49-02.bpo-41681.3-VJiH.rst new file mode 100644 index 0000000000000..ed557f92d85ca --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-31-17-49-02.bpo-41681.3-VJiH.rst @@ -0,0 +1,2 @@ +Fixes the wrong error description in the error raised by using 2 `,` in +format string in f-string and :meth:`str.format`. diff --git a/Python/formatter_unicode.c b/Python/formatter_unicode.c index 74638ca223772..ed95f267d476c 100644 --- a/Python/formatter_unicode.c +++ b/Python/formatter_unicode.c @@ -252,8 +252,10 @@ parse_internal_render_format_spec(PyObject *format_spec, ++pos; } if (end-pos && READ_spec(pos) == ',') { - invalid_comma_and_underscore(); - return 0; + if (format->thousands_separators == LT_UNDERSCORE_LOCALE) { + invalid_comma_and_underscore(); + return 0; + } } /* Parse field precision */ From webhook-mailer at python.org Tue Sep 1 12:25:30 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 01 Sep 2020 16:25:30 -0000 Subject: [Python-checkins] bpo-41617: Fix pycore_bitutils.h to support clang 3.0 (GH-22042) Message-ID: https://github.com/python/cpython/commit/e6905e4c82cc05897dc1bf5ab2b5b94b2b043a7f commit: e6905e4c82cc05897dc1bf5ab2b5b94b2b043a7f branch: master author: Victor Stinner committer: GitHub date: 2020-09-01T18:25:14+02:00 summary: bpo-41617: Fix pycore_bitutils.h to support clang 3.0 (GH-22042) __builtin_bswap16() is not available in LLVM clang 3.0. files: A Misc/NEWS.d/next/Build/2020-08-24-18-34-01.bpo-41617.sKKXz7.rst M Include/internal/pycore_bitutils.h diff --git a/Include/internal/pycore_bitutils.h b/Include/internal/pycore_bitutils.h index 0bd3270fe82e5..1602fc68d9407 100644 --- a/Include/internal/pycore_bitutils.h +++ b/Include/internal/pycore_bitutils.h @@ -17,10 +17,12 @@ extern "C" { # error "this header requires Py_BUILD_CORE define" #endif -#if defined(__clang__) || \ - (defined(__GNUC__) && \ - ((__GNUC__ >= 5) || (__GNUC__ == 4) && (__GNUC_MINOR__ >= 8))) - /* __builtin_bswap16() is available since GCC 4.8, +#if ((defined(__GNUC__) \ + && ((__GNUC__ >= 5) || (__GNUC__ == 4) && (__GNUC_MINOR__ >= 8))) \ + || (defined(__clang__) \ + && (__clang_major__ >= 4 \ + || (__clang_major__ == 3 && __clang_minor__ >= 2)))) + /* __builtin_bswap16() is available since GCC 4.8 and clang 3.2, __builtin_bswap32() is available since GCC 4.3, __builtin_bswap64() is available since GCC 4.3. */ # define _PY_HAVE_BUILTIN_BSWAP diff --git a/Misc/NEWS.d/next/Build/2020-08-24-18-34-01.bpo-41617.sKKXz7.rst b/Misc/NEWS.d/next/Build/2020-08-24-18-34-01.bpo-41617.sKKXz7.rst new file mode 100644 index 0000000000000..715eadbee896f --- /dev/null +++ b/Misc/NEWS.d/next/Build/2020-08-24-18-34-01.bpo-41617.sKKXz7.rst @@ -0,0 +1,2 @@ +Fix ``pycore_bitutils.h`` header file to support old clang versions: +``__builtin_bswap16()`` is not available in LLVM clang 3.0. From webhook-mailer at python.org Tue Sep 1 12:42:50 2020 From: webhook-mailer at python.org (Marek Madejski) Date: Tue, 01 Sep 2020 16:42:50 -0000 Subject: [Python-checkins] bpo-41528: Use math module in turtle (GH-21837) Message-ID: https://github.com/python/cpython/commit/6844b56176c41f0a0e25fcd4fef5463bcdbc7d7c commit: 6844b56176c41f0a0e25fcd4fef5463bcdbc7d7c branch: master author: Marek Madejski committer: GitHub date: 2020-09-01T12:42:41-04:00 summary: bpo-41528: Use math module in turtle (GH-21837) Use angle-related functions from math module instead of reinventing the wheel. files: A Misc/NEWS.d/next/Library/2020-08-12-07-43-31.bpo-41528.bu83oD.rst M Lib/turtle.py diff --git a/Lib/turtle.py b/Lib/turtle.py index ee67a351b54f1..92d4e5dda9c2d 100644 --- a/Lib/turtle.py +++ b/Lib/turtle.py @@ -263,12 +263,12 @@ def __sub__(self, other): def __neg__(self): return Vec2D(-self[0], -self[1]) def __abs__(self): - return (self[0]**2 + self[1]**2)**0.5 + return math.hypot(*self) def rotate(self, angle): """rotate self counterclockwise by angle """ perp = Vec2D(-self[1], self[0]) - angle = angle * math.pi / 180.0 + angle = math.radians(angle) c, s = math.cos(angle), math.sin(angle) return Vec2D(self[0]*c+perp[0]*s, self[1]*c+perp[1]*s) def __getnewargs__(self): @@ -1597,7 +1597,7 @@ def radians(self): >>> turtle.heading() 1.5707963267948966 """ - self._setDegreesPerAU(2*math.pi) + self._setDegreesPerAU(math.tau) def _go(self, distance): """move turtle forward by specified distance""" @@ -1888,7 +1888,7 @@ def towards(self, x, y=None): elif isinstance(x, TNavigator): pos = x._position x, y = pos - self._position - result = round(math.atan2(y, x)*180.0/math.pi, 10) % 360.0 + result = round(math.degrees(math.atan2(y, x)), 10) % 360.0 result /= self._degreesPerAU return (self._angleOffset + self._angleOrient*result) % self._fullcircle @@ -1903,7 +1903,7 @@ def heading(self): 67.0 """ x, y = self._orient - result = round(math.atan2(y, x)*180.0/math.pi, 10) % 360.0 + result = round(math.degrees(math.atan2(y, x)), 10) % 360.0 result /= self._degreesPerAU return (self._angleOffset + self._angleOrient*result) % self._fullcircle @@ -1976,7 +1976,7 @@ def circle(self, radius, extent = None, steps = None): steps = 1+int(min(11+abs(radius)/6.0, 59.0)*frac) w = 1.0 * extent / steps w2 = 0.5 * w - l = 2.0 * radius * math.sin(w2*math.pi/180.0*self._degreesPerAU) + l = 2.0 * radius * math.sin(math.radians(w2)*self._degreesPerAU) if radius < 0: l, w, w2 = -l, -w, -w2 tr = self._tracer() @@ -2861,7 +2861,7 @@ def settiltangle(self, angle): >>> turtle.fd(50) """ tilt = -angle * self._degreesPerAU * self._angleOrient - tilt = (tilt * math.pi / 180.0) % (2*math.pi) + tilt = math.radians(tilt) % math.tau self.pen(resizemode="user", tilt=tilt) def tiltangle(self, angle=None): @@ -2885,7 +2885,7 @@ def tiltangle(self, angle=None): >>> turtle.tiltangle() """ if angle is None: - tilt = -self._tilt * (180.0/math.pi) * self._angleOrient + tilt = -math.degrees(self._tilt) * self._angleOrient return (tilt / self._degreesPerAU) % self._fullcircle else: self.settiltangle(angle) @@ -2939,7 +2939,7 @@ def shapetransform(self, t11=None, t12=None, t21=None, t22=None): if t11 * t22 - t12 * t21 == 0: raise TurtleGraphicsError("Bad shape transform matrix: must not be singular") self._shapetrafo = (m11, m12, m21, m22) - alfa = math.atan2(-m21, m11) % (2 * math.pi) + alfa = math.atan2(-m21, m11) % math.tau sa, ca = math.sin(alfa), math.cos(alfa) a11, a12, a21, a22 = (ca*m11 - sa*m21, ca*m12 - sa*m22, sa*m11 + ca*m21, sa*m12 + ca*m22) diff --git a/Misc/NEWS.d/next/Library/2020-08-12-07-43-31.bpo-41528.bu83oD.rst b/Misc/NEWS.d/next/Library/2020-08-12-07-43-31.bpo-41528.bu83oD.rst new file mode 100644 index 0000000000000..a4ba57c2438ed --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-08-12-07-43-31.bpo-41528.bu83oD.rst @@ -0,0 +1 @@ +turtle uses math module functions to convert degrees to radians and vice versa and to calculate vector norm \ No newline at end of file From webhook-mailer at python.org Tue Sep 1 14:40:08 2020 From: webhook-mailer at python.org (Pablo Galindo) Date: Tue, 01 Sep 2020 18:40:08 -0000 Subject: [Python-checkins] bpo-41654: Fix deallocator of MemoryError to account for subclasses (GH-22020) Message-ID: https://github.com/python/cpython/commit/9b648a95ccb4c3b14f1e87158f5c9f5dbb2f62c0 commit: 9b648a95ccb4c3b14f1e87158f5c9f5dbb2f62c0 branch: master author: Pablo Galindo committer: GitHub date: 2020-09-01T19:39:46+01:00 summary: bpo-41654: Fix deallocator of MemoryError to account for subclasses (GH-22020) When allocating MemoryError classes, there is some logic to use pre-allocated instances in a freelist only if the type that is being allocated is not a subclass of MemoryError. Unfortunately in the destructor this logic is not present so the freelist is altered even with subclasses of MemoryError. files: A Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst M Lib/test/test_exceptions.py M Objects/exceptions.c diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py index 2ffe8caa03f81..1ec446887770e 100644 --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -1,6 +1,7 @@ # Python test set -- part 5, built-in exceptions import copy +import gc import os import sys import unittest @@ -1330,6 +1331,36 @@ def test_assert_shadowing(self): del AssertionError self.fail('Expected exception') + def test_memory_error_subclasses(self): + # bpo-41654: MemoryError instances use a freelist of objects that are + # linked using the 'dict' attribute when they are inactive/dead. + # Subclasses of MemoryError should not participate in the freelist + # schema. This test creates a MemoryError object and keeps it alive + # (therefore advancing the freelist) and then it creates and destroys a + # subclass object. Finally, it checks that creating a new MemoryError + # succeeds, proving that the freelist is not corrupted. + + class TestException(MemoryError): + pass + + try: + raise MemoryError + except MemoryError as exc: + inst = exc + + try: + raise TestException + except Exception: + pass + + for _ in range(10): + try: + raise MemoryError + except MemoryError as exc: + pass + + gc_collect() + class ImportErrorTests(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst new file mode 100644 index 0000000000000..e05c3133e1262 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst @@ -0,0 +1,2 @@ +Fix a crash that occurred when destroying subclasses of +:class:`MemoryError`. Patch by Pablo Galindo. diff --git a/Objects/exceptions.c b/Objects/exceptions.c index 1195ba17922dd..b08cbdd6aed35 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -2286,8 +2286,11 @@ MemoryError_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { PyBaseExceptionObject *self; - if (type != (PyTypeObject *) PyExc_MemoryError) + /* If this is a subclass of MemoryError, don't use the freelist + * and just return a fresh object */ + if (type != (PyTypeObject *) PyExc_MemoryError) { return BaseException_new(type, args, kwds); + } struct _Py_exc_state *state = get_exc_state(); if (state->memerrors_freelist == NULL) { @@ -2313,9 +2316,16 @@ MemoryError_new(PyTypeObject *type, PyObject *args, PyObject *kwds) static void MemoryError_dealloc(PyBaseExceptionObject *self) { - _PyObject_GC_UNTRACK(self); BaseException_clear(self); + /* If this is a subclass of MemoryError, we don't need to + * do anything in the free-list*/ + if (!Py_IS_TYPE(self, (PyTypeObject *) PyExc_MemoryError)) { + return Py_TYPE(self)->tp_free((PyObject *)self); + } + + _PyObject_GC_UNTRACK(self); + struct _Py_exc_state *state = get_exc_state(); if (state->memerrors_numfree >= MEMERRORS_SAVE) { Py_TYPE(self)->tp_free((PyObject *)self); From webhook-mailer at python.org Tue Sep 1 16:41:04 2020 From: webhook-mailer at python.org (Pablo Galindo) Date: Tue, 01 Sep 2020 20:41:04 -0000 Subject: [Python-checkins] [3.8] [3.9] bpo-41654: Fix deallocator of MemoryError to account for subclasses (GH-22020) (GH-22046) Message-ID: https://github.com/python/cpython/commit/77f4000ae0d43a2685face80e7f14d4aba053973 commit: 77f4000ae0d43a2685face80e7f14d4aba053973 branch: 3.8 author: Pablo Galindo committer: GitHub date: 2020-09-01T21:40:48+01:00 summary: [3.8] [3.9] bpo-41654: Fix deallocator of MemoryError to account for subclasses (GH-22020) (GH-22046) When allocating MemoryError classes, there is some logic to use pre-allocated instances in a freelist only if the type that is being allocated is not a subclass of MemoryError. Unfortunately in the destructor this logic is not present so the freelist is altered even with subclasses of MemoryError.. (cherry picked from commit 9b648a95ccb4c3b14f1e87158f5c9f5dbb2f62c0) Co-authored-by: Pablo Galindo . (cherry picked from commit 87e91ae2e5f81e096c32839f211c68a749a4435a) Co-authored-by: Pablo Galindo files: A Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst M Lib/test/test_exceptions.py M Objects/exceptions.c diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py index 3a32253157369..457b46e01ce31 100644 --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -1,6 +1,7 @@ # Python test set -- part 5, built-in exceptions import copy +import gc import os import sys import unittest @@ -1297,6 +1298,35 @@ def g(): next(i) next(i) + def test_memory_error_subclasses(self): + # bpo-41654: MemoryError instances use a freelist of objects that are + # linked using the 'dict' attribute when they are inactive/dead. + # Subclasses of MemoryError should not participate in the freelist + # schema. This test creates a MemoryError object and keeps it alive + # (therefore advancing the freelist) and then it creates and destroys a + # subclass object. Finally, it checks that creating a new MemoryError + # succeeds, proving that the freelist is not corrupted. + + class TestException(MemoryError): + pass + + try: + raise MemoryError + except MemoryError as exc: + inst = exc + + try: + raise TestException + except Exception: + pass + + for _ in range(10): + try: + raise MemoryError + except MemoryError as exc: + pass + + gc_collect() class ImportErrorTests(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst new file mode 100644 index 0000000000000..e05c3133e1262 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst @@ -0,0 +1,2 @@ +Fix a crash that occurred when destroying subclasses of +:class:`MemoryError`. Patch by Pablo Galindo. diff --git a/Objects/exceptions.c b/Objects/exceptions.c index 38723d93653c3..966983810cd1e 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -2268,8 +2268,12 @@ MemoryError_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { PyBaseExceptionObject *self; - if (type != (PyTypeObject *) PyExc_MemoryError) + /* If this is a subclass of MemoryError, don't use the freelist + * and just return a fresh object */ + if (type != (PyTypeObject *) PyExc_MemoryError) { return BaseException_new(type, args, kwds); + } + if (memerrors_freelist == NULL) return BaseException_new(type, args, kwds); /* Fetch object from freelist and revive it */ @@ -2289,8 +2293,14 @@ MemoryError_new(PyTypeObject *type, PyObject *args, PyObject *kwds) static void MemoryError_dealloc(PyBaseExceptionObject *self) { - _PyObject_GC_UNTRACK(self); BaseException_clear(self); + + if (Py_TYPE(self) != PyExc_MemoryError) { + return Py_TYPE(self)->tp_free((PyObject *)self); + } + + _PyObject_GC_UNTRACK(self); + if (memerrors_numfree >= MEMERRORS_SAVE) Py_TYPE(self)->tp_free((PyObject *)self); else { From webhook-mailer at python.org Tue Sep 1 17:18:12 2020 From: webhook-mailer at python.org (Shantanu) Date: Tue, 01 Sep 2020 21:18:12 -0000 Subject: [Python-checkins] bpo-39349: Add cancel_futures to Executor.shutdown base class (GH-22023) Message-ID: https://github.com/python/cpython/commit/17dc1b789ecc33b4a254eb3b799085f4b3624ca5 commit: 17dc1b789ecc33b4a254eb3b799085f4b3624ca5 branch: master author: Shantanu <12621235+hauntsaninja at users.noreply.github.com> committer: GitHub date: 2020-09-01T17:18:07-04:00 summary: bpo-39349: Add cancel_futures to Executor.shutdown base class (GH-22023) * Add cancel_futures parameter to the Executor base class, since it was missed in the original PR (https://github.com/python/cpython/pull/18057) that added cancel_futures. files: M Lib/concurrent/futures/_base.py diff --git a/Lib/concurrent/futures/_base.py b/Lib/concurrent/futures/_base.py index bf546f8ae1d1c..00eb54881f295 100644 --- a/Lib/concurrent/futures/_base.py +++ b/Lib/concurrent/futures/_base.py @@ -605,7 +605,7 @@ def result_iterator(): future.cancel() return result_iterator() - def shutdown(self, wait=True): + def shutdown(self, wait=True, *, cancel_futures=False): """Clean-up the resources associated with the Executor. It is safe to call this method several times. Otherwise, no other @@ -615,6 +615,9 @@ def shutdown(self, wait=True): wait: If True then shutdown will not return until all running futures have finished executing and the resources used by the executor have been reclaimed. + cancel_futures: If True then shutdown will cancel all pending + futures. Futures that are completed or running will not be + cancelled. """ pass From webhook-mailer at python.org Tue Sep 1 21:36:50 2020 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 02 Sep 2020 01:36:50 -0000 Subject: [Python-checkins] Note the buffer slots can be set with PyType_Spec with the unlimited API. (GH-22031) Message-ID: https://github.com/python/cpython/commit/700fa1b836f315048e8b89ede5018a0c0a213946 commit: 700fa1b836f315048e8b89ede5018a0c0a213946 branch: master author: Benjamin Peterson committer: GitHub date: 2020-09-01T20:36:42-05:00 summary: Note the buffer slots can be set with PyType_Spec with the unlimited API. (GH-22031) Follow up to f7c4e236429606e1c982cacf24e10fc86ef4462f. files: M Doc/c-api/type.rst diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index 7309d7ee2cd39..73f26875d8194 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -225,7 +225,8 @@ The following functions and structs are used to create * ``Py_nb_add`` to set :c:member:`PyNumberMethods.nb_add` * ``Py_sq_length`` to set :c:member:`PySequenceMethods.sq_length` - The following fields cannot be set using :c:type:`PyType_Spec` and :c:type:`PyType_Slot`: + The following fields cannot be set at all using :c:type:`PyType_Spec` and + :c:type:`PyType_Slot`: * :c:member:`~PyTypeObject.tp_dict` * :c:member:`~PyTypeObject.tp_mro` @@ -239,6 +240,10 @@ The following functions and structs are used to create (see :ref:`PyMemberDef `) * :c:member:`~PyTypeObject.tp_vectorcall_offset` (see :ref:`PyMemberDef `) + + The following fields cannot be set using :c:type:`PyType_Spec` and + :c:type:`PyType_Slot` under the limited API: + * :c:member:`~PyBufferProcs.bf_getbuffer` * :c:member:`~PyBufferProcs.bf_releasebuffer` @@ -246,6 +251,10 @@ The following functions and structs are used to create To avoid issues, use the *bases* argument of :py:func:`PyType_FromSpecWithBases` instead. + .. versionchanged:: 3.9 + + Slots in :c:type:`PyBufferProcs` in may be set in the unlimited API. + .. c:member:: void *PyType_Slot.pfunc The desired value of the slot. In most cases, this is a pointer From webhook-mailer at python.org Tue Sep 1 23:21:16 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Wed, 02 Sep 2020 03:21:16 -0000 Subject: [Python-checkins] [doc] Remove references to PyChecker. (GH-22011) Message-ID: https://github.com/python/cpython/commit/dea82b67315a6b873f7d4e558dd00a851137dcbb commit: dea82b67315a6b873f7d4e558dd00a851137dcbb branch: master author: Andre Delfino committer: GitHub date: 2020-09-01T22:21:12-05:00 summary: [doc] Remove references to PyChecker. (GH-22011) files: M Doc/faq/design.rst M Doc/faq/programming.rst diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst index 4e3cc575ee196..8cf271c302408 100644 --- a/Doc/faq/design.rst +++ b/Doc/faq/design.rst @@ -573,8 +573,7 @@ whether an instance or a class implements a particular ABC. The :class:`~collections.abc.MutableMapping`. For Python, many of the advantages of interface specifications can be obtained -by an appropriate test discipline for components. There is also a tool, -PyChecker, which can be used to find problems due to subclassing. +by an appropriate test discipline for components. A good test suite for a module can both provide a regression test and serve as a module interface specification and a set of examples. Many Python modules can diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 0731e92f6dbc6..d6a2f2cfc67ee 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -57,22 +57,14 @@ They include: * PyCharm (https://www.jetbrains.com/pycharm/) -Is there a tool to help find bugs or perform static analysis? +Are there tools to help find bugs or perform static analysis? ------------------------------------------------------------- Yes. -PyChecker is a static analysis tool that finds bugs in Python source code and -warns about code complexity and style. You can get PyChecker from -http://pychecker.sourceforge.net/. - -`Pylint `_ is another tool that checks -if a module satisfies a coding standard, and also makes it possible to write -plug-ins to add a custom feature. In addition to the bug checking that -PyChecker performs, Pylint offers some additional features such as checking line -length, whether variable names are well-formed according to your coding -standard, whether declared interfaces are fully implemented, and more. -https://docs.pylint.org/ provides a full list of Pylint's features. +`Pylint `_ and +`Pyflakes `_ do basic checking that will +help you catch bugs sooner. Static type checkers such as `Mypy `_, `Pyre `_, and From webhook-mailer at python.org Tue Sep 1 23:23:00 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Wed, 02 Sep 2020 03:23:00 -0000 Subject: [Python-checkins] Remove reference to Boa Constructor. (GH-22012) Message-ID: https://github.com/python/cpython/commit/1d25f5bf7b795b47e753aca56d7579d4ad7ee468 commit: 1d25f5bf7b795b47e753aca56d7579d4ad7ee468 branch: master author: Andre Delfino committer: GitHub date: 2020-09-01T22:22:55-05:00 summary: Remove reference to Boa Constructor. (GH-22012) files: M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index d6a2f2cfc67ee..66d210a55fac7 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -35,12 +35,6 @@ for Windows Extensions `__ project an as a part of the ActivePython distribution (see https://www.activestate.com/activepython\ ). -`Boa Constructor `_ is an IDE and GUI -builder that uses wxWidgets. It offers visual frame creation and manipulation, -an object inspector, many views on the source like object browsers, inheritance -hierarchies, doc string generated html documentation, an advanced debugger, -integrated help, and Zope support. - `Eric `_ is an IDE built on PyQt and the Scintilla editing component. From webhook-mailer at python.org Wed Sep 2 01:00:58 2020 From: webhook-mailer at python.org (Raymond Hettinger) Date: Wed, 02 Sep 2020 05:00:58 -0000 Subject: [Python-checkins] Improve hypot() accuracy with three separate accumulators (GH-22032) Message-ID: https://github.com/python/cpython/commit/5b24d1592a990ad7cf81cd1498d255bad41a0b14 commit: 5b24d1592a990ad7cf81cd1498d255bad41a0b14 branch: master author: Raymond Hettinger committer: GitHub date: 2020-09-01T22:00:50-07:00 summary: Improve hypot() accuracy with three separate accumulators (GH-22032) files: M Modules/mathmodule.c diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index 6621951ee97d2..d227a5d15dca2 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -2456,7 +2456,7 @@ Given that csum >= 1.0, we have: Since lo**2 is less than 1/2 ulp(csum), we have csum+lo*lo == csum. To minimize loss of information during the accumulation of fractional -values, the lo**2 term has a separate accumulator. +values, each term has a separate accumulator. The square root differential correction is needed because a correctly rounded square root of a correctly rounded sum of @@ -2487,7 +2487,7 @@ static inline double vector_norm(Py_ssize_t n, double *vec, double max, int found_nan) { const double T27 = 134217729.0; /* ldexp(1.0, 27)+1.0) */ - double x, csum = 1.0, oldcsum, frac = 0.0, frac_lo = 0.0, scale; + double x, csum = 1.0, oldcsum, scale, frac=0.0, frac_mid=0.0, frac_lo=0.0; double t, hi, lo, h; int max_e; Py_ssize_t i; @@ -2529,12 +2529,12 @@ vector_norm(Py_ssize_t n, double *vec, double max, int found_nan) assert(fabs(csum) >= fabs(x)); oldcsum = csum; csum += x; - frac += (oldcsum - csum) + x; + frac_mid += (oldcsum - csum) + x; assert(csum + lo * lo == csum); frac_lo += lo * lo; } - frac += frac_lo; + frac += frac_lo + frac_mid; h = sqrt(csum - 1.0 + frac); x = h; From webhook-mailer at python.org Wed Sep 2 04:56:42 2020 From: webhook-mailer at python.org (han-solo) Date: Wed, 02 Sep 2020 08:56:42 -0000 Subject: [Python-checkins] Fixed mistake in test for f-string error description (GH-22036) (GH-22059) Message-ID: https://github.com/python/cpython/commit/749ed85e4446f548e22934931241f644a33d81ce commit: 749ed85e4446f548e22934931241f644a33d81ce branch: master author: han-solo committer: GitHub date: 2020-09-02T04:56:37-04:00 summary: Fixed mistake in test for f-string error description (GH-22036) (GH-22059) files: M Lib/test/test_format.py M Lib/test/test_fstring.py diff --git a/Lib/test/test_format.py b/Lib/test/test_format.py index dff8c69032187..d2744cdfdca60 100644 --- a/Lib/test/test_format.py +++ b/Lib/test/test_format.py @@ -514,7 +514,7 @@ def test_with_a_commas_and_an_underscore_in_format_specifier(self): def test_with_an_underscore_and_a_comma_in_format_specifier(self): error_msg = re.escape("Cannot specify both ',' and '_'.") with self.assertRaisesRegex(ValueError, error_msg): - '{:,_}'.format(1) + '{:_,}'.format(1) if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_fstring.py b/Lib/test/test_fstring.py index b9bede0d9b800..b53661aa0a46f 100644 --- a/Lib/test/test_fstring.py +++ b/Lib/test/test_fstring.py @@ -1217,7 +1217,7 @@ def test_with_a_commas_and_an_underscore_in_format_specifier(self): def test_with_an_underscore_and_a_comma_in_format_specifier(self): error_msg = re.escape("Cannot specify both ',' and '_'.") with self.assertRaisesRegex(ValueError, error_msg): - f'{1:,_}' + f'{1:_,}' if __name__ == '__main__': unittest.main() From webhook-mailer at python.org Wed Sep 2 05:45:28 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Wed, 02 Sep 2020 09:45:28 -0000 Subject: [Python-checkins] bpo-1635741: Port _blake2 module to multi-phase init (GH-21856) Message-ID: https://github.com/python/cpython/commit/a7f026870d2dab7015a94e287bec6dd46cdbf604 commit: a7f026870d2dab7015a94e287bec6dd46cdbf604 branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-02T11:45:13+02:00 summary: bpo-1635741: Port _blake2 module to multi-phase init (GH-21856) Port the _blake2 extension module to the multi-phase initialization API (PEP 489). files: A Misc/NEWS.d/next/Core and Builtins/2020-08-13-07-18-05.bpo-1635741.FC13e7.rst M Modules/_blake2/blake2b_impl.c M Modules/_blake2/blake2module.c M Modules/_blake2/blake2s_impl.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-13-07-18-05.bpo-1635741.FC13e7.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-13-07-18-05.bpo-1635741.FC13e7.rst new file mode 100644 index 0000000000000..cdfee874095fe --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-13-07-18-05.bpo-1635741.FC13e7.rst @@ -0,0 +1 @@ +Port the :mod:`_blake2` extension module to the multi-phase initialization API (:pep:`489`). diff --git a/Modules/_blake2/blake2b_impl.c b/Modules/_blake2/blake2b_impl.c index 7fb1296f8b2b9..8e1acce56b1d2 100644 --- a/Modules/_blake2/blake2b_impl.c +++ b/Modules/_blake2/blake2b_impl.c @@ -34,7 +34,7 @@ #endif -extern PyTypeObject PyBlake2_BLAKE2bType; +extern PyType_Spec blake2b_type_spec; typedef struct { PyObject_HEAD @@ -391,47 +391,24 @@ py_blake2b_dealloc(PyObject *self) PyThread_free_lock(obj->lock); obj->lock = NULL; } + + PyTypeObject *type = Py_TYPE(self); PyObject_Del(self); + Py_DECREF(type); } +static PyType_Slot blake2b_type_slots[] = { + {Py_tp_dealloc, py_blake2b_dealloc}, + {Py_tp_doc, (char *)py_blake2b_new__doc__}, + {Py_tp_methods, py_blake2b_methods}, + {Py_tp_getset, py_blake2b_getsetters}, + {Py_tp_new, py_blake2b_new}, + {0,0} +}; -PyTypeObject PyBlake2_BLAKE2bType = { - PyVarObject_HEAD_INIT(NULL, 0) - "_blake2.blake2b", /* tp_name */ - sizeof(BLAKE2bObject), /* tp_basicsize */ - 0, /* tp_itemsize */ - py_blake2b_dealloc, /* tp_dealloc */ - 0, /*tp_vectorcall_offset*/ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT, /* tp_flags */ - py_blake2b_new__doc__, /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - py_blake2b_methods, /* tp_methods */ - 0, /* tp_members */ - py_blake2b_getsetters, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - py_blake2b_new, /* tp_new */ +PyType_Spec blake2b_type_spec = { + .name = "_blake2.blake2b", + .basicsize = sizeof(BLAKE2bObject), + .flags = Py_TPFLAGS_DEFAULT, + .slots = blake2b_type_slots }; diff --git a/Modules/_blake2/blake2module.c b/Modules/_blake2/blake2module.c index ff142c9f3ed33..631de2cc0abc7 100644 --- a/Modules/_blake2/blake2module.c +++ b/Modules/_blake2/blake2module.c @@ -12,62 +12,81 @@ #include "impl/blake2.h" -extern PyTypeObject PyBlake2_BLAKE2bType; -extern PyTypeObject PyBlake2_BLAKE2sType; - +extern PyType_Spec blake2b_type_spec; +extern PyType_Spec blake2s_type_spec; PyDoc_STRVAR(blake2mod__doc__, "_blake2b provides BLAKE2b for hashlib\n" ); +typedef struct { + PyTypeObject* blake2b_type; + PyTypeObject* blake2s_type; +} Blake2State; + +static inline Blake2State* +blake2_get_state(PyObject *module) +{ + void *state = PyModule_GetState(module); + assert(state != NULL); + return (Blake2State *)state; +} static struct PyMethodDef blake2mod_functions[] = { {NULL, NULL} }; -static struct PyModuleDef blake2_module = { - PyModuleDef_HEAD_INIT, - "_blake2", - blake2mod__doc__, - -1, - blake2mod_functions, - NULL, - NULL, - NULL, - NULL -}; +static int +_blake2_traverse(PyObject *module, visitproc visit, void *arg) +{ + Blake2State *state = blake2_get_state(module); + Py_VISIT(state->blake2b_type); + Py_VISIT(state->blake2s_type); + return 0; +} + +static int +_blake2_clear(PyObject *module) +{ + Blake2State *state = blake2_get_state(module); + Py_CLEAR(state->blake2b_type); + Py_CLEAR(state->blake2s_type); + return 0; +} + +static void +_blake2_free(void *module) +{ + _blake2_clear((PyObject *)module); +} #define ADD_INT(d, name, value) do { \ PyObject *x = PyLong_FromLong(value); \ - if (!x) { \ - Py_DECREF(m); \ - return NULL; \ - } \ + if (!x) \ + return -1; \ if (PyDict_SetItemString(d, name, x) < 0) { \ - Py_DECREF(m); \ - return NULL; \ + Py_DECREF(x); \ + return -1; \ } \ Py_DECREF(x); \ } while(0) - -PyMODINIT_FUNC -PyInit__blake2(void) +static int +blake2_exec(PyObject *m) { - PyObject *m; - PyObject *d; + Blake2State* st = blake2_get_state(m); - m = PyModule_Create(&blake2_module); - if (m == NULL) - return NULL; + st->blake2b_type = (PyTypeObject *)PyType_FromModuleAndSpec( + m, &blake2b_type_spec, NULL); + if (NULL == st->blake2b_type) + return -1; /* BLAKE2b */ - Py_SET_TYPE(&PyBlake2_BLAKE2bType, &PyType_Type); - if (PyModule_AddType(m, &PyBlake2_BLAKE2bType) < 0) { - return NULL; + if (PyModule_AddType(m, st->blake2b_type) < 0) { + return -1; } - d = PyBlake2_BLAKE2bType.tp_dict; + PyObject *d = st->blake2b_type->tp_dict; ADD_INT(d, "SALT_SIZE", BLAKE2B_SALTBYTES); ADD_INT(d, "PERSON_SIZE", BLAKE2B_PERSONALBYTES); ADD_INT(d, "MAX_KEY_SIZE", BLAKE2B_KEYBYTES); @@ -79,12 +98,17 @@ PyInit__blake2(void) PyModule_AddIntConstant(m, "BLAKE2B_MAX_DIGEST_SIZE", BLAKE2B_OUTBYTES); /* BLAKE2s */ - Py_SET_TYPE(&PyBlake2_BLAKE2sType, &PyType_Type); - if (PyModule_AddType(m, &PyBlake2_BLAKE2sType) < 0) { - return NULL; + st->blake2s_type = (PyTypeObject *)PyType_FromModuleAndSpec( + m, &blake2s_type_spec, NULL); + + if (NULL == st->blake2s_type) + return -1; + + if (PyModule_AddType(m, st->blake2s_type) < 0) { + return -1; } - d = PyBlake2_BLAKE2sType.tp_dict; + d = st->blake2s_type->tp_dict; ADD_INT(d, "SALT_SIZE", BLAKE2S_SALTBYTES); ADD_INT(d, "PERSON_SIZE", BLAKE2S_PERSONALBYTES); ADD_INT(d, "MAX_KEY_SIZE", BLAKE2S_KEYBYTES); @@ -95,5 +119,28 @@ PyInit__blake2(void) PyModule_AddIntConstant(m, "BLAKE2S_MAX_KEY_SIZE", BLAKE2S_KEYBYTES); PyModule_AddIntConstant(m, "BLAKE2S_MAX_DIGEST_SIZE", BLAKE2S_OUTBYTES); - return m; + return 0; } + +static PyModuleDef_Slot _blake2_slots[] = { + {Py_mod_exec, blake2_exec}, + {0, NULL} +}; + +static struct PyModuleDef blake2_module = { + PyModuleDef_HEAD_INIT, + "_blake2", + .m_doc = blake2mod__doc__, + .m_size = sizeof(Blake2State), + .m_methods = blake2mod_functions, + .m_slots = _blake2_slots, + .m_traverse = _blake2_traverse, + .m_clear = _blake2_clear, + .m_free = _blake2_free, +}; + +PyMODINIT_FUNC +PyInit__blake2(void) +{ + return PyModuleDef_Init(&blake2_module); +} \ No newline at end of file diff --git a/Modules/_blake2/blake2s_impl.c b/Modules/_blake2/blake2s_impl.c index e3e90d0587b80..e1de5df37d098 100644 --- a/Modules/_blake2/blake2s_impl.c +++ b/Modules/_blake2/blake2s_impl.c @@ -33,8 +33,7 @@ #include "impl/blake2s-ref.c" #endif - -extern PyTypeObject PyBlake2_BLAKE2sType; +extern PyType_Spec blake2s_type_spec; typedef struct { PyObject_HEAD @@ -391,47 +390,24 @@ py_blake2s_dealloc(PyObject *self) PyThread_free_lock(obj->lock); obj->lock = NULL; } + + PyTypeObject *type = Py_TYPE(self); PyObject_Del(self); + Py_DECREF(type); } +static PyType_Slot blake2s_type_slots[] = { + {Py_tp_dealloc, py_blake2s_dealloc}, + {Py_tp_doc, (char *)py_blake2s_new__doc__}, + {Py_tp_methods, py_blake2s_methods}, + {Py_tp_getset, py_blake2s_getsetters}, + {Py_tp_new, py_blake2s_new}, + {0,0} +}; -PyTypeObject PyBlake2_BLAKE2sType = { - PyVarObject_HEAD_INIT(NULL, 0) - "_blake2.blake2s", /* tp_name */ - sizeof(BLAKE2sObject), /* tp_basicsize */ - 0, /* tp_itemsize */ - py_blake2s_dealloc, /* tp_dealloc */ - 0, /*tp_vectorcall_offset*/ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT, /* tp_flags */ - py_blake2s_new__doc__, /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - py_blake2s_methods, /* tp_methods */ - 0, /* tp_members */ - py_blake2s_getsetters, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - 0, /* tp_init */ - 0, /* tp_alloc */ - py_blake2s_new, /* tp_new */ +PyType_Spec blake2s_type_spec = { + .name = "_blake2.blake2s", + .basicsize = sizeof(BLAKE2sObject), + .flags = Py_TPFLAGS_DEFAULT, + .slots = blake2s_type_slots }; From webhook-mailer at python.org Wed Sep 2 05:55:27 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Wed, 02 Sep 2020 09:55:27 -0000 Subject: [Python-checkins] bpo-1635741: Port _sha3 module to multi-phase init (GH-21855) Message-ID: https://github.com/python/cpython/commit/93d50a6a8d0c5d332c11aef267e66573a09765ac commit: 93d50a6a8d0c5d332c11aef267e66573a09765ac branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-02T11:55:19+02:00 summary: bpo-1635741: Port _sha3 module to multi-phase init (GH-21855) Port the _sha3 extension module to multi-phase init (PEP 489). Convert static types to heap types. files: A Misc/NEWS.d/next/Core and Builtins/2020-08-13-07-19-21.bpo-1653741.fubBkb.rst M Modules/_sha3/sha3module.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-13-07-19-21.bpo-1653741.fubBkb.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-13-07-19-21.bpo-1653741.fubBkb.rst new file mode 100644 index 0000000000000..73a4fdbac48a2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-13-07-19-21.bpo-1653741.fubBkb.rst @@ -0,0 +1 @@ +Port :mod:`_sha3` to multi-phase init. Convert static types to heap types. diff --git a/Modules/_sha3/sha3module.c b/Modules/_sha3/sha3module.c index c826b42df13f9..da6dde6812f26 100644 --- a/Modules/_sha3/sha3module.c +++ b/Modules/_sha3/sha3module.c @@ -122,6 +122,28 @@ #define SHA3_squeeze Keccak_HashSqueeze #define SHA3_copystate(dest, src) memcpy(&(dest), &(src), sizeof(SHA3_state)) +typedef struct { + PyTypeObject *sha3_224_type; + PyTypeObject *sha3_256_type; + PyTypeObject *sha3_384_type; + PyTypeObject *sha3_512_type; +#ifdef PY_WITH_KECCAK + PyTypeObject *keccak_224_type; + PyTypeObject *keccak_256_type; + PyTypeObject *keccak_384_type; + PyTypeObject *keccak_512_type; +#endif + PyTypeObject *shake_128_type; + PyTypeObject *shake_256_type; +} SHA3State; + +static inline SHA3State* +sha3_get_state(PyObject *module) +{ + void *state = PyModule_GetState(module); + assert(state != NULL); + return (SHA3State *)state; +} /*[clinic input] module _sha3 @@ -142,19 +164,6 @@ typedef struct { PyThread_type_lock lock; } SHA3object; -static PyTypeObject SHA3_224type; -static PyTypeObject SHA3_256type; -static PyTypeObject SHA3_384type; -static PyTypeObject SHA3_512type; -#ifdef PY_WITH_KECCAK -static PyTypeObject Keccak_224type; -static PyTypeObject Keccak_256type; -static PyTypeObject Keccak_384type; -static PyTypeObject Keccak_512type; -#endif -static PyTypeObject SHAKE128type; -static PyTypeObject SHAKE256type; - #include "clinic/sha3module.c.h" static SHA3object * @@ -184,42 +193,43 @@ static PyObject * py_sha3_new_impl(PyTypeObject *type, PyObject *data, int usedforsecurity) /*[clinic end generated code: output=90409addc5d5e8b0 input=bcfcdf2e4368347a]*/ { - SHA3object *self = NULL; - Py_buffer buf = {NULL, NULL}; - HashReturn res; - - self = newSHA3object(type); + SHA3object *self = newSHA3object(type); if (self == NULL) { goto error; } - if (type == &SHA3_224type) { + SHA3State *state = PyType_GetModuleState(type); + assert(state != NULL); + + HashReturn res; + if (type == state->sha3_224_type) { res = Keccak_HashInitialize_SHA3_224(&self->hash_state); - } else if (type == &SHA3_256type) { + } else if (type == state->sha3_256_type) { res = Keccak_HashInitialize_SHA3_256(&self->hash_state); - } else if (type == &SHA3_384type) { + } else if (type == state->sha3_384_type) { res = Keccak_HashInitialize_SHA3_384(&self->hash_state); - } else if (type == &SHA3_512type) { + } else if (type == state->sha3_512_type) { res = Keccak_HashInitialize_SHA3_512(&self->hash_state); #ifdef PY_WITH_KECCAK - } else if (type == &Keccak_224type) { + } else if (type == state->keccak_224_type) { res = Keccak_HashInitialize(&self->hash_state, 1152, 448, 224, 0x01); - } else if (type == &Keccak_256type) { + } else if (type == state->keccak_256_type) { res = Keccak_HashInitialize(&self->hash_state, 1088, 512, 256, 0x01); - } else if (type == &Keccak_384type) { + } else if (type == state->keccak_384_type) { res = Keccak_HashInitialize(&self->hash_state, 832, 768, 384, 0x01); - } else if (type == &Keccak_512type) { + } else if (type == state->keccak_512_type) { res = Keccak_HashInitialize(&self->hash_state, 576, 1024, 512, 0x01); #endif - } else if (type == &SHAKE128type) { + } else if (type == state->shake_128_type) { res = Keccak_HashInitialize_SHAKE128(&self->hash_state); - } else if (type == &SHAKE256type) { + } else if (type == state->shake_256_type) { res = Keccak_HashInitialize_SHAKE256(&self->hash_state); } else { PyErr_BadInternalCall(); goto error; } + Py_buffer buf = {NULL, NULL}; if (data) { GET_BUFFER_VIEW_OR_ERROR(data, &buf, goto error); if (buf.len >= HASHLIB_GIL_MINSIZE) { @@ -262,7 +272,10 @@ SHA3_dealloc(SHA3object *self) if (self->lock) { PyThread_free_lock(self->lock); } + + PyTypeObject *tp = Py_TYPE(self); PyObject_Del(self); + Py_DECREF(tp); } @@ -416,27 +429,31 @@ static PyObject * SHA3_get_name(SHA3object *self, void *closure) { PyTypeObject *type = Py_TYPE(self); - if (type == &SHA3_224type) { + + SHA3State *state = PyType_GetModuleState(type); + assert(state != NULL); + + if (type == state->sha3_224_type) { return PyUnicode_FromString("sha3_224"); - } else if (type == &SHA3_256type) { + } else if (type == state->sha3_256_type) { return PyUnicode_FromString("sha3_256"); - } else if (type == &SHA3_384type) { + } else if (type == state->sha3_384_type) { return PyUnicode_FromString("sha3_384"); - } else if (type == &SHA3_512type) { + } else if (type == state->sha3_512_type) { return PyUnicode_FromString("sha3_512"); #ifdef PY_WITH_KECCAK - } else if (type == &Keccak_224type) { + } else if (type == state->keccak_224_type) { return PyUnicode_FromString("keccak_224"); - } else if (type == &Keccak_256type) { + } else if (type == state->keccak_256_type) { return PyUnicode_FromString("keccak_256"); - } else if (type == &Keccak_384type) { + } else if (type == state->keccak_384_type) { return PyUnicode_FromString("keccak_384"); - } else if (type == &Keccak_512type) { + } else if (type == state->keccak_512_type) { return PyUnicode_FromString("keccak_512"); #endif - } else if (type == &SHAKE128type) { + } else if (type == state->shake_128_type) { return PyUnicode_FromString("shake_128"); - } else if (type == &SHAKE256type) { + } else if (type == state->shake_256_type) { return PyUnicode_FromString("shake_256"); } else { PyErr_BadInternalCall(); @@ -476,7 +493,6 @@ SHA3_get_suffix(SHA3object *self, void *closure) return PyBytes_FromStringAndSize((const char *)suffix, 1); } - static PyGetSetDef SHA3_getseters[] = { {"block_size", (getter)SHA3_get_block_size, NULL, NULL, NULL}, {"name", (getter)SHA3_get_name, NULL, NULL, NULL}, @@ -487,48 +503,24 @@ static PyGetSetDef SHA3_getseters[] = { {NULL} /* Sentinel */ }; +#define SHA3_TYPE_SLOTS(type_slots_obj, type_doc, type_methods) \ + static PyType_Slot type_slots_obj[] = { \ + {Py_tp_dealloc, SHA3_dealloc}, \ + {Py_tp_doc, (char*)type_doc}, \ + {Py_tp_methods, type_methods}, \ + {Py_tp_getset, SHA3_getseters}, \ + {Py_tp_new, py_sha3_new}, \ + {0,0} \ + } -#define SHA3_TYPE(type_obj, type_name, type_doc, type_methods) \ - static PyTypeObject type_obj = { \ - PyVarObject_HEAD_INIT(NULL, 0) \ - type_name, /* tp_name */ \ - sizeof(SHA3object), /* tp_basicsize */ \ - 0, /* tp_itemsize */ \ - /* methods */ \ - (destructor)SHA3_dealloc, /* tp_dealloc */ \ - 0, /* tp_vectorcall_offset */ \ - 0, /* tp_getattr */ \ - 0, /* tp_setattr */ \ - 0, /* tp_as_async */ \ - 0, /* tp_repr */ \ - 0, /* tp_as_number */ \ - 0, /* tp_as_sequence */ \ - 0, /* tp_as_mapping */ \ - 0, /* tp_hash */ \ - 0, /* tp_call */ \ - 0, /* tp_str */ \ - 0, /* tp_getattro */ \ - 0, /* tp_setattro */ \ - 0, /* tp_as_buffer */ \ - Py_TPFLAGS_DEFAULT, /* tp_flags */ \ - type_doc, /* tp_doc */ \ - 0, /* tp_traverse */ \ - 0, /* tp_clear */ \ - 0, /* tp_richcompare */ \ - 0, /* tp_weaklistoffset */ \ - 0, /* tp_iter */ \ - 0, /* tp_iternext */ \ - type_methods, /* tp_methods */ \ - NULL, /* tp_members */ \ - SHA3_getseters, /* tp_getset */ \ - 0, /* tp_base */ \ - 0, /* tp_dict */ \ - 0, /* tp_descr_get */ \ - 0, /* tp_descr_set */ \ - 0, /* tp_dictoffset */ \ - 0, /* tp_init */ \ - 0, /* tp_alloc */ \ - py_sha3_new, /* tp_new */ \ +// Using PyType_GetModuleState() on these types is safe since they +// cannot be subclassed: it does not have the Py_TPFLAGS_BASETYPE flag. +#define SHA3_TYPE_SPEC(type_spec_obj, type_name, type_slots) \ + static PyType_Spec type_spec_obj = { \ + .name = "_sha3." type_name, \ + .basicsize = sizeof(SHA3object), \ + .flags = Py_TPFLAGS_DEFAULT, \ + .slots = type_slots \ } PyDoc_STRVAR(sha3_224__doc__, @@ -551,11 +543,6 @@ PyDoc_STRVAR(sha3_512__doc__, \n\ Return a new SHA3 hash object with a hashbit length of 64 bytes."); -SHA3_TYPE(SHA3_224type, "_sha3.sha3_224", sha3_224__doc__, SHA3_methods); -SHA3_TYPE(SHA3_256type, "_sha3.sha3_256", sha3_256__doc__, SHA3_methods); -SHA3_TYPE(SHA3_384type, "_sha3.sha3_384", sha3_384__doc__, SHA3_methods); -SHA3_TYPE(SHA3_512type, "_sha3.sha3_512", sha3_512__doc__, SHA3_methods); - #ifdef PY_WITH_KECCAK PyDoc_STRVAR(keccak_224__doc__, "keccak_224([data], *, usedforsecurity=True) -> Keccak object\n\ @@ -577,10 +564,32 @@ PyDoc_STRVAR(keccak_512__doc__, \n\ Return a new Keccak hash object with a hashbit length of 64 bytes."); -SHA3_TYPE(Keccak_224type, "_sha3.keccak_224", keccak_224__doc__, SHA3_methods); -SHA3_TYPE(Keccak_256type, "_sha3.keccak_256", keccak_256__doc__, SHA3_methods); -SHA3_TYPE(Keccak_384type, "_sha3.keccak_384", keccak_384__doc__, SHA3_methods); -SHA3_TYPE(Keccak_512type, "_sha3.keccak_512", keccak_512__doc__, SHA3_methods); +#endif + +SHA3_TYPE_SLOTS(sha3_224_slots, sha3_224__doc__, SHA3_methods); +SHA3_TYPE_SPEC(sha3_224_spec, "sha3_224", sha3_224_slots); + +SHA3_TYPE_SLOTS(sha3_256_slots, sha3_256__doc__, SHA3_methods); +SHA3_TYPE_SPEC(sha3_256_spec, "sha3_256", sha3_256_slots); + +SHA3_TYPE_SLOTS(sha3_384_slots, sha3_384__doc__, SHA3_methods); +SHA3_TYPE_SPEC(sha3_384_spec, "sha3_384", sha3_384_slots); + +SHA3_TYPE_SLOTS(sha3_512_slots, sha3_512__doc__, SHA3_methods); +SHA3_TYPE_SPEC(sha3_512_spec, "sha3_512", sha3_512_slots); + +#ifdef PY_WITH_KECCAK +SHA3_TYPE_SLOTS(Keccak_224_slots, keccak_224__doc__, SHA3_methods); +SHA3_TYPE_SPEC(Keccak_224_spec, "keccak_224", Keccak_224_slots); + +SHA3_TYPE_SLOTS(Keccak_256_slots, keccak_256__doc__, SHA3_methods); +SHA3_TYPE_SPEC(Keccak_256_spec, "keccak_256", Keccak_256_slots); + +SHA3_TYPE_SLOTS(Keccak_384_slots, keccak_384__doc__, SHA3_methods); +SHA3_TYPE_SPEC(Keccak_384_spec, "keccak_384", Keccak_384_slots); + +SHA3_TYPE_SLOTS(Keccak_512_slots, keccak_512__doc__, SHA3_methods); +SHA3_TYPE_SPEC(Keccak_512_spec, "keccak_512", Keccak_512_slots); #endif @@ -684,70 +693,118 @@ PyDoc_STRVAR(shake_256__doc__, \n\ Return a new SHAKE hash object."); -SHA3_TYPE(SHAKE128type, "_sha3.shake_128", shake_128__doc__, SHAKE_methods); -SHA3_TYPE(SHAKE256type, "_sha3.shake_256", shake_256__doc__, SHAKE_methods); +SHA3_TYPE_SLOTS(SHAKE128slots, shake_128__doc__, SHAKE_methods); +SHA3_TYPE_SPEC(SHAKE128_spec, "shake_128", SHAKE128slots); +SHA3_TYPE_SLOTS(SHAKE256slots, shake_256__doc__, SHAKE_methods); +SHA3_TYPE_SPEC(SHAKE256_spec, "shake_256", SHAKE256slots); -/* Initialize this module. */ -static struct PyModuleDef _SHA3module = { - PyModuleDef_HEAD_INIT, - "_sha3", - NULL, - -1, - NULL, - NULL, - NULL, - NULL, - NULL -}; +static int +_sha3_traverse(PyObject *module, visitproc visit, void *arg) +{ + SHA3State *state = sha3_get_state(module); + Py_VISIT(state->sha3_224_type); + Py_VISIT(state->sha3_256_type); + Py_VISIT(state->sha3_384_type); + Py_VISIT(state->sha3_512_type); +#ifdef PY_WITH_KECCAK + Py_VISIT(state->keccak_224_type); + Py_VISIT(state->keccak_256_type); + Py_VISIT(state->keccak_384_type); + Py_VISIT(state->keccak_512_type); +#endif + Py_VISIT(state->shake_128_type); + Py_VISIT(state->shake_256_type); + return 0; +} -PyMODINIT_FUNC -PyInit__sha3(void) +static int +_sha3_clear(PyObject *module) { - PyObject *m = NULL; + SHA3State *state = sha3_get_state(module); + Py_CLEAR(state->sha3_224_type); + Py_CLEAR(state->sha3_256_type); + Py_CLEAR(state->sha3_384_type); + Py_CLEAR(state->sha3_512_type); +#ifdef PY_WITH_KECCAK + Py_CLEAR(state->keccak_224_type); + Py_CLEAR(state->keccak_256_type); + Py_CLEAR(state->keccak_384_type); + Py_CLEAR(state->keccak_512_type); +#endif + Py_CLEAR(state->shake_128_type); + Py_CLEAR(state->shake_256_type); + return 0; +} - if ((m = PyModule_Create(&_SHA3module)) == NULL) { - return NULL; - } +static void +_sha3_free(void *module) +{ + _sha3_clear((PyObject *)module); +} -#define init_sha3type(name, type) \ - do { \ - Py_SET_TYPE(type, &PyType_Type); \ - if (PyType_Ready(type) < 0) { \ - goto error; \ - } \ - Py_INCREF((PyObject *)type); \ - if (PyModule_AddObject(m, name, (PyObject *)type) < 0) { \ - goto error; \ - } \ +static int +_sha3_exec(PyObject *m) +{ + SHA3State *st = sha3_get_state(m); + +#define init_sha3type(type, typespec) \ + do { \ + st->type = (PyTypeObject *)PyType_FromModuleAndSpec( \ + m, &typespec, NULL); \ + if (st->type == NULL) { \ + return -1; \ + } \ + if (PyModule_AddType(m, st->type) < 0) { \ + return -1; \ + } \ } while(0) - init_sha3type("sha3_224", &SHA3_224type); - init_sha3type("sha3_256", &SHA3_256type); - init_sha3type("sha3_384", &SHA3_384type); - init_sha3type("sha3_512", &SHA3_512type); + init_sha3type(sha3_224_type, sha3_224_spec); + init_sha3type(sha3_256_type, sha3_256_spec); + init_sha3type(sha3_384_type, sha3_384_spec); + init_sha3type(sha3_512_type, sha3_512_spec); #ifdef PY_WITH_KECCAK - init_sha3type("keccak_224", &Keccak_224type); - init_sha3type("keccak_256", &Keccak_256type); - init_sha3type("keccak_384", &Keccak_384type); - init_sha3type("keccak_512", &Keccak_512type); + init_sha3type(keccak_224_type, Keccak_224_spec); + init_sha3type(keccak_256_type, Keccak_256_spec); + init_sha3type(keccak_384_type, Keccak_384_spec); + init_sha3type(keccak_512_type, Keccak_512_spec); #endif - init_sha3type("shake_128", &SHAKE128type); - init_sha3type("shake_256", &SHAKE256type); - + init_sha3type(shake_128_type, SHAKE128_spec); + init_sha3type(shake_256_type, SHAKE256_spec); #undef init_sha3type if (PyModule_AddIntConstant(m, "keccakopt", KeccakOpt) < 0) { - goto error; + return -1; } if (PyModule_AddStringConstant(m, "implementation", KeccakP1600_implementation) < 0) { - goto error; + return -1; } - return m; - error: - Py_DECREF(m); - return NULL; + return 0; +} + +static PyModuleDef_Slot _sha3_slots[] = { + {Py_mod_exec, _sha3_exec}, + {0, NULL} +}; + +/* Initialize this module. */ +static struct PyModuleDef _sha3module = { + PyModuleDef_HEAD_INIT, + .m_name = "_sha3", + .m_size = sizeof(SHA3State), + .m_slots = _sha3_slots, + .m_traverse = _sha3_traverse, + .m_clear = _sha3_clear, + .m_free = _sha3_free, +}; + + +PyMODINIT_FUNC +PyInit__sha3(void) +{ + return PyModuleDef_Init(&_sha3module); } From webhook-mailer at python.org Wed Sep 2 06:29:36 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 02 Sep 2020 10:29:36 -0000 Subject: [Python-checkins] bpo-40204: Update Sphinx to version 3.2.1 in Doc/Makefile (GH-22043) Message-ID: https://github.com/python/cpython/commit/c0d5c131d9c1ac743058fae99f913b455fc90ac7 commit: c0d5c131d9c1ac743058fae99f913b455fc90ac7 branch: master author: Victor Stinner committer: GitHub date: 2020-09-02T12:29:31+02:00 summary: bpo-40204: Update Sphinx to version 3.2.1 in Doc/Makefile (GH-22043) files: M Doc/Makefile diff --git a/Doc/Makefile b/Doc/Makefile index 87d5d07665c86..f1d0a267dd89d 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -144,7 +144,7 @@ venv: $(PYTHON) -m venv $(VENVDIR) # $(VENVDIR)/bin/python3 -m pip install -U pip setuptools $(VENVDIR)/bin/python3 -m pip install -U pip setuptools==49.2.1 - $(VENVDIR)/bin/python3 -m pip install -U Sphinx==2.3.1 blurb python-docs-theme + $(VENVDIR)/bin/python3 -m pip install -U Sphinx==3.2.1 blurb python-docs-theme @echo "The venv has been created in the $(VENVDIR) directory" dist: From webhook-mailer at python.org Wed Sep 2 07:11:26 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 02 Sep 2020 11:11:26 -0000 Subject: [Python-checkins] bpo-41685: Don't pin setuptools version anymore in Doc/Makefile (GH-22062) Message-ID: https://github.com/python/cpython/commit/51fece1bb8d7aa22f114172a256714fe24788e31 commit: 51fece1bb8d7aa22f114172a256714fe24788e31 branch: master author: Victor Stinner committer: GitHub date: 2020-09-02T13:11:21+02:00 summary: bpo-41685: Don't pin setuptools version anymore in Doc/Makefile (GH-22062) setuptools 50.0.2 is now compatible with Python 3.10: https://github.com/pypa/setuptools/pull/2361 files: M Doc/Makefile diff --git a/Doc/Makefile b/Doc/Makefile index f1d0a267dd89d..c11a7ca5c1bcb 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -142,8 +142,7 @@ clean: venv: $(PYTHON) -m venv $(VENVDIR) -# $(VENVDIR)/bin/python3 -m pip install -U pip setuptools - $(VENVDIR)/bin/python3 -m pip install -U pip setuptools==49.2.1 + $(VENVDIR)/bin/python3 -m pip install -U pip setuptools $(VENVDIR)/bin/python3 -m pip install -U Sphinx==3.2.1 blurb python-docs-theme @echo "The venv has been created in the $(VENVDIR) directory" From webhook-mailer at python.org Wed Sep 2 09:29:05 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 02 Sep 2020 13:29:05 -0000 Subject: [Python-checkins] Remove reference to Boa Constructor. (GH-22057) Message-ID: https://github.com/python/cpython/commit/a8b05c4b9727ed7033840a44a67f2ee3d332ab24 commit: a8b05c4b9727ed7033840a44a67f2ee3d332ab24 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-02T08:28:59-05:00 summary: Remove reference to Boa Constructor. (GH-22057) (cherry picked from commit 1d25f5bf7b795b47e753aca56d7579d4ad7ee468) Co-authored-by: Andre Delfino files: M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 81048c9422f32..8af24c5869c20 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -35,12 +35,6 @@ for Windows Extensions `__ project an as a part of the ActivePython distribution (see https://www.activestate.com/activepython\ ). -`Boa Constructor `_ is an IDE and GUI -builder that uses wxWidgets. It offers visual frame creation and manipulation, -an object inspector, many views on the source like object browsers, inheritance -hierarchies, doc string generated html documentation, an advanced debugger, -integrated help, and Zope support. - `Eric `_ is an IDE built on PyQt and the Scintilla editing component. From webhook-mailer at python.org Wed Sep 2 09:29:51 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 02 Sep 2020 13:29:51 -0000 Subject: [Python-checkins] [doc] Remove references to PyChecker. (GH-22055) Message-ID: https://github.com/python/cpython/commit/5d663ededd9b1b3f2df6d671b6951166f80c0353 commit: 5d663ededd9b1b3f2df6d671b6951166f80c0353 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-02T08:29:46-05:00 summary: [doc] Remove references to PyChecker. (GH-22055) (cherry picked from commit dea82b67315a6b873f7d4e558dd00a851137dcbb) Co-authored-by: Andre Delfino files: M Doc/faq/design.rst M Doc/faq/programming.rst diff --git a/Doc/faq/design.rst b/Doc/faq/design.rst index e7921baf242f5..f91b51bf8f86c 100644 --- a/Doc/faq/design.rst +++ b/Doc/faq/design.rst @@ -571,8 +571,7 @@ whether an instance or a class implements a particular ABC. The :class:`~collections.abc.MutableMapping`. For Python, many of the advantages of interface specifications can be obtained -by an appropriate test discipline for components. There is also a tool, -PyChecker, which can be used to find problems due to subclassing. +by an appropriate test discipline for components. A good test suite for a module can both provide a regression test and serve as a module interface specification and a set of examples. Many Python modules can diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 8af24c5869c20..70e9190e5a520 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -51,22 +51,14 @@ They include: * PyCharm (https://www.jetbrains.com/pycharm/) -Is there a tool to help find bugs or perform static analysis? +Are there tools to help find bugs or perform static analysis? ------------------------------------------------------------- Yes. -PyChecker is a static analysis tool that finds bugs in Python source code and -warns about code complexity and style. You can get PyChecker from -http://pychecker.sourceforge.net/. - -`Pylint `_ is another tool that checks -if a module satisfies a coding standard, and also makes it possible to write -plug-ins to add a custom feature. In addition to the bug checking that -PyChecker performs, Pylint offers some additional features such as checking line -length, whether variable names are well-formed according to your coding -standard, whether declared interfaces are fully implemented, and more. -https://docs.pylint.org/ provides a full list of Pylint's features. +`Pylint `_ and +`Pyflakes `_ do basic checking that will +help you catch bugs sooner. Static type checkers such as `Mypy `_, `Pyre `_, and From webhook-mailer at python.org Wed Sep 2 10:29:21 2020 From: webhook-mailer at python.org (Pablo Galindo) Date: Wed, 02 Sep 2020 14:29:21 -0000 Subject: [Python-checkins] bpo-41675: Modernize siginterrupt calls (GH-22028) Message-ID: https://github.com/python/cpython/commit/f9c5e3f5f61cd380f8a17c814766fc3730b7fbdf commit: f9c5e3f5f61cd380f8a17c814766fc3730b7fbdf branch: master author: Pablo Galindo committer: GitHub date: 2020-09-02T15:29:12+01:00 summary: bpo-41675: Modernize siginterrupt calls (GH-22028) siginterrupt is deprecated: ./Modules/signalmodule.c:667:5: warning: ?siginterrupt? is deprecated: Use sigaction with SA_RESTART instead [-Wdeprecated-declarations] 667 | if (siginterrupt(signalnum, flag)<0) { files: A Misc/NEWS.d/next/Core and Builtins/2020-08-31-14-53-17.bpo-41675.VSoqWU.rst M Modules/signalmodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-31-14-53-17.bpo-41675.VSoqWU.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-31-14-53-17.bpo-41675.VSoqWU.rst new file mode 100644 index 0000000000000..aa102f8fe4384 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-31-14-53-17.bpo-41675.VSoqWU.rst @@ -0,0 +1,3 @@ +The implementation of :func:`signal.siginterrupt` now uses :c:func:`sigaction` +(if it is available in the system) instead of the deprecated :c:func:`siginterrupt`. +Patch by Pablo Galindo. diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c index 7bc1b535e6e2c..c49a3ea52e71d 100644 --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -664,7 +664,19 @@ signal_siginterrupt_impl(PyObject *module, int signalnum, int flag) "signal number out of range"); return NULL; } - if (siginterrupt(signalnum, flag)<0) { +#ifdef HAVE_SIGACTION + struct sigaction act; + (void) sigaction(signalnum, NULL, &act); + if (flag) { + act.sa_flags &= ~SA_RESTART; + } + else { + act.sa_flags |= SA_RESTART; + } + if (sigaction(signalnum, &act, NULL) < 0) { +#else + if (siginterrupt(signalnum, flag) < 0) { +#endif PyErr_SetFromErrno(PyExc_OSError); return NULL; } From webhook-mailer at python.org Wed Sep 2 10:29:46 2020 From: webhook-mailer at python.org (Pablo Galindo) Date: Wed, 02 Sep 2020 14:29:46 -0000 Subject: [Python-checkins] Fix invalid escape sequences in the peg_highlight Sphinx extension (GH-22047) Message-ID: https://github.com/python/cpython/commit/5a4a963a6c798fa9207a9998618a9c0ec3b6b6d7 commit: 5a4a963a6c798fa9207a9998618a9c0ec3b6b6d7 branch: master author: Pablo Galindo committer: GitHub date: 2020-09-02T15:29:38+01:00 summary: Fix invalid escape sequences in the peg_highlight Sphinx extension (GH-22047) files: M Doc/tools/extensions/peg_highlight.py diff --git a/Doc/tools/extensions/peg_highlight.py b/Doc/tools/extensions/peg_highlight.py index f02515d3919cf..8bc24670fbe0a 100644 --- a/Doc/tools/extensions/peg_highlight.py +++ b/Doc/tools/extensions/peg_highlight.py @@ -59,7 +59,7 @@ class PEGLexer(RegexLexer): include("variables"), (r"\b(?!(NULL|EXTRA))([A-Z_]+)\b\s*(?!\()", Text,), ( - r"^\s*" + _name + "\s*" + "(\[.*\])?" + "\s*" + "(\(.+\))?" + "\s*(:)", + r"^\s*" + _name + r"\s*" + r"(\[.*\])?" + r"\s*" + r"(\(.+\))?" + r"\s*(:)", bygroups(Name.Function, None, None, Punctuation), ), (_name, Name.Function), From webhook-mailer at python.org Wed Sep 2 12:29:15 2020 From: webhook-mailer at python.org (Benjamin Peterson) Date: Wed, 02 Sep 2020 16:29:15 -0000 Subject: [Python-checkins] closes bpo-41689: Preserve text signature from tp_doc in C heap type creation. (GH-22058) Message-ID: https://github.com/python/cpython/commit/3940333637b98a2781869977b077552514784529 commit: 3940333637b98a2781869977b077552514784529 branch: master author: Benjamin Peterson committer: GitHub date: 2020-09-02T11:29:06-05:00 summary: closes bpo-41689: Preserve text signature from tp_doc in C heap type creation. (GH-22058) files: A Misc/NEWS.d/next/C API/2020-09-01-23-39-45.bpo-41689.zxHbLB.rst M Lib/test/test_capi.py M Modules/_testcapimodule.c M Objects/typeobject.c diff --git a/Lib/test/test_capi.py b/Lib/test/test_capi.py index 892cc74ec3915..db62b47100ad3 100644 --- a/Lib/test/test_capi.py +++ b/Lib/test/test_capi.py @@ -401,6 +401,10 @@ def __del__(self): del L self.assertEqual(PyList.num, 0) + def test_heap_ctype_doc_and_text_signature(self): + self.assertEqual(_testcapi.HeapDocCType.__doc__, "somedoc") + self.assertEqual(_testcapi.HeapDocCType.__text_signature__, "(arg1, arg2)") + def test_subclass_of_heap_gc_ctype_with_tpdealloc_decrefs_once(self): class HeapGcCTypeSubclass(_testcapi.HeapGcCType): def __init__(self): diff --git a/Misc/NEWS.d/next/C API/2020-09-01-23-39-45.bpo-41689.zxHbLB.rst b/Misc/NEWS.d/next/C API/2020-09-01-23-39-45.bpo-41689.zxHbLB.rst new file mode 100644 index 0000000000000..44cf58a4b0638 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2020-09-01-23-39-45.bpo-41689.zxHbLB.rst @@ -0,0 +1,2 @@ +Types created with :c:func:`PyType_FromSpec` now make any signature in their +``tp_doc`` slot accessible from ``__text_signature__``. diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 593034ef65e2c..7536d29535038 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -6462,6 +6462,30 @@ static PyTypeObject MethodDescriptor2_Type = { .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_VECTORCALL, }; +PyDoc_STRVAR(heapdocctype__doc__, +"HeapDocCType(arg1, arg2)\n" +"--\n" +"\n" +"somedoc"); + +typedef struct { + PyObject_HEAD +} HeapDocCTypeObject; + +static PyType_Slot HeapDocCType_slots[] = { + {Py_tp_doc, (char*)heapdocctype__doc__}, + {0}, +}; + +static PyType_Spec HeapDocCType_spec = { + "_testcapi.HeapDocCType", + sizeof(HeapDocCTypeObject), + 0, + Py_TPFLAGS_DEFAULT, + HeapDocCType_slots +}; + + PyDoc_STRVAR(heapgctype__doc__, "A heap type with GC, and with overridden dealloc.\n\n" "The 'value' attribute is set to 10 in __init__."); @@ -7130,6 +7154,12 @@ PyInit__testcapi(void) Py_INCREF(TestError); PyModule_AddObject(m, "error", TestError); + PyObject *HeapDocCType = PyType_FromSpec(&HeapDocCType_spec); + if (HeapDocCType == NULL) { + return NULL; + } + PyModule_AddObject(m, "HeapDocCType", HeapDocCType); + PyObject *HeapGcCType = PyType_FromSpec(&HeapGcCType_spec); if (HeapGcCType == NULL) { return NULL; diff --git a/Objects/typeobject.c b/Objects/typeobject.c index c66f8fcec8ed5..74040757a0702 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -3018,15 +3018,14 @@ PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases) else if (slot->slot == Py_tp_doc) { /* For the docstring slot, which usually points to a static string literal, we need to make a copy */ - const char *old_doc = _PyType_DocWithoutSignature(type->tp_name, slot->pfunc); - size_t len = strlen(old_doc)+1; + size_t len = strlen(slot->pfunc)+1; char *tp_doc = PyObject_MALLOC(len); if (tp_doc == NULL) { type->tp_doc = NULL; PyErr_NoMemory(); goto fail; } - memcpy(tp_doc, old_doc, len); + memcpy(tp_doc, slot->pfunc, len); type->tp_doc = tp_doc; } else if (slot->slot == Py_tp_members) { @@ -3058,6 +3057,16 @@ PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases) res->ht_cached_keys = _PyDict_NewKeysForClass(); } + if (type->tp_doc) { + PyObject *__doc__ = PyUnicode_FromString(_PyType_DocWithoutSignature(type->tp_name, type->tp_doc)); + if (!__doc__) + goto fail; + int ret = _PyDict_SetItemId(type->tp_dict, &PyId___doc__, __doc__); + Py_DECREF(__doc__); + if (ret < 0) + goto fail; + } + if (weaklistoffset) { type->tp_weaklistoffset = weaklistoffset; if (PyDict_DelItemString((PyObject *)type->tp_dict, "__weaklistoffset__") < 0) From webhook-mailer at python.org Wed Sep 2 12:44:23 2020 From: webhook-mailer at python.org (Pablo Galindo) Date: Wed, 02 Sep 2020 16:44:23 -0000 Subject: [Python-checkins] bpo-41690: Use a loop to collect args in the parser instead of recursion (GH-22053) Message-ID: https://github.com/python/cpython/commit/4a97b1517a6b5ff22e2984b677a680b07ff0ce11 commit: 4a97b1517a6b5ff22e2984b677a680b07ff0ce11 branch: master author: Pablo Galindo committer: GitHub date: 2020-09-02T17:44:19+01:00 summary: bpo-41690: Use a loop to collect args in the parser instead of recursion (GH-22053) This program can segfault the parser by stack overflow: ``` import ast code = "f(" + ",".join(['a' for _ in range(100000)]) + ")" print("Ready!") ast.parse(code) ``` the reason is that the rule for arguments has a simple recursion when collecting args: args[expr_ty]: [...] | a=named_expression b=[',' c=args { c }] { [...] } files: A Misc/NEWS.d/next/Core and Builtins/2020-09-02-12-00-57.bpo-41690.Ny-Sfy.rst M Grammar/python.gram M Parser/parser.c M Parser/pegen.c M Parser/pegen.h diff --git a/Grammar/python.gram b/Grammar/python.gram index 1cba11407468d..84835b731c540 100644 --- a/Grammar/python.gram +++ b/Grammar/python.gram @@ -535,22 +535,11 @@ arguments[expr_ty] (memo): | a=args [','] &')' { a } | incorrect_arguments args[expr_ty]: - | a=starred_expression b=[',' c=args { c }] { - _Py_Call(_PyPegen_dummy_name(p), - (b) ? CHECK(_PyPegen_seq_insert_in_front(p, a, ((expr_ty) b)->v.Call.args)) - : CHECK(_PyPegen_singleton_seq(p, a)), - (b) ? ((expr_ty) b)->v.Call.keywords : NULL, - EXTRA) } + | a=','.(starred_expression | named_expression !'=')+ b=[',' k=kwargs {k}] { _PyPegen_collect_call_seqs(p, a, b) } | a=kwargs { _Py_Call(_PyPegen_dummy_name(p), CHECK_NULL_ALLOWED(_PyPegen_seq_extract_starred_exprs(p, a)), CHECK_NULL_ALLOWED(_PyPegen_seq_delete_starred_exprs(p, a)), EXTRA) } - | a=named_expression b=[',' c=args { c }] { - _Py_Call(_PyPegen_dummy_name(p), - (b) ? CHECK(_PyPegen_seq_insert_in_front(p, a, ((expr_ty) b)->v.Call.args)) - : CHECK(_PyPegen_singleton_seq(p, a)), - (b) ? ((expr_ty) b)->v.Call.keywords : NULL, - EXTRA) } kwargs[asdl_seq*]: | a=','.kwarg_or_starred+ ',' b=','.kwarg_or_double_starred+ { _PyPegen_join_sequences(p, a, b) } | ','.kwarg_or_starred+ diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-02-12-00-57.bpo-41690.Ny-Sfy.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-02-12-00-57.bpo-41690.Ny-Sfy.rst new file mode 100644 index 0000000000000..5711aa5a55f07 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-02-12-00-57.bpo-41690.Ny-Sfy.rst @@ -0,0 +1,2 @@ +Fix a possible stack overflow in the parser when parsing functions and +classes with a huge ammount of arguments. Patch by Pablo Galindo. diff --git a/Parser/parser.c b/Parser/parser.c index 75dc7176a5e75..3e724a260d90a 100644 --- a/Parser/parser.c +++ b/Parser/parser.c @@ -334,34 +334,34 @@ static KeywordToken *reserved_keywords[] = { #define _loop1_105_type 1265 #define _loop0_106_type 1266 #define _loop0_107_type 1267 -#define _tmp_108_type 1268 -#define _tmp_109_type 1269 -#define _loop0_111_type 1270 -#define _gather_110_type 1271 -#define _loop0_113_type 1272 -#define _gather_112_type 1273 -#define _loop0_115_type 1274 -#define _gather_114_type 1275 -#define _loop0_117_type 1276 -#define _gather_116_type 1277 -#define _loop0_118_type 1278 -#define _loop0_120_type 1279 -#define _gather_119_type 1280 -#define _tmp_121_type 1281 -#define _loop0_123_type 1282 -#define _gather_122_type 1283 -#define _loop0_125_type 1284 -#define _gather_124_type 1285 -#define _tmp_126_type 1286 -#define _loop0_127_type 1287 +#define _loop0_109_type 1268 +#define _gather_108_type 1269 +#define _tmp_110_type 1270 +#define _loop0_112_type 1271 +#define _gather_111_type 1272 +#define _loop0_114_type 1273 +#define _gather_113_type 1274 +#define _loop0_116_type 1275 +#define _gather_115_type 1276 +#define _loop0_118_type 1277 +#define _gather_117_type 1278 +#define _loop0_119_type 1279 +#define _loop0_121_type 1280 +#define _gather_120_type 1281 +#define _tmp_122_type 1282 +#define _loop0_124_type 1283 +#define _gather_123_type 1284 +#define _loop0_126_type 1285 +#define _gather_125_type 1286 +#define _tmp_127_type 1287 #define _loop0_128_type 1288 #define _loop0_129_type 1289 -#define _tmp_130_type 1290 +#define _loop0_130_type 1290 #define _tmp_131_type 1291 -#define _loop0_132_type 1292 -#define _tmp_133_type 1293 -#define _loop0_134_type 1294 -#define _tmp_135_type 1295 +#define _tmp_132_type 1292 +#define _loop0_133_type 1293 +#define _tmp_134_type 1294 +#define _loop0_135_type 1295 #define _tmp_136_type 1296 #define _tmp_137_type 1297 #define _tmp_138_type 1298 @@ -377,10 +377,12 @@ static KeywordToken *reserved_keywords[] = { #define _tmp_148_type 1308 #define _tmp_149_type 1309 #define _tmp_150_type 1310 -#define _loop1_151_type 1311 -#define _loop1_152_type 1312 -#define _tmp_153_type 1313 -#define _tmp_154_type 1314 +#define _tmp_151_type 1311 +#define _tmp_152_type 1312 +#define _loop1_153_type 1313 +#define _loop1_154_type 1314 +#define _tmp_155_type 1315 +#define _tmp_156_type 1316 static mod_ty file_rule(Parser *p); static mod_ty interactive_rule(Parser *p); @@ -650,34 +652,34 @@ static asdl_seq *_gather_103_rule(Parser *p); static asdl_seq *_loop1_105_rule(Parser *p); static asdl_seq *_loop0_106_rule(Parser *p); static asdl_seq *_loop0_107_rule(Parser *p); -static void *_tmp_108_rule(Parser *p); -static void *_tmp_109_rule(Parser *p); -static asdl_seq *_loop0_111_rule(Parser *p); -static asdl_seq *_gather_110_rule(Parser *p); -static asdl_seq *_loop0_113_rule(Parser *p); -static asdl_seq *_gather_112_rule(Parser *p); -static asdl_seq *_loop0_115_rule(Parser *p); -static asdl_seq *_gather_114_rule(Parser *p); -static asdl_seq *_loop0_117_rule(Parser *p); -static asdl_seq *_gather_116_rule(Parser *p); +static asdl_seq *_loop0_109_rule(Parser *p); +static asdl_seq *_gather_108_rule(Parser *p); +static void *_tmp_110_rule(Parser *p); +static asdl_seq *_loop0_112_rule(Parser *p); +static asdl_seq *_gather_111_rule(Parser *p); +static asdl_seq *_loop0_114_rule(Parser *p); +static asdl_seq *_gather_113_rule(Parser *p); +static asdl_seq *_loop0_116_rule(Parser *p); +static asdl_seq *_gather_115_rule(Parser *p); static asdl_seq *_loop0_118_rule(Parser *p); -static asdl_seq *_loop0_120_rule(Parser *p); -static asdl_seq *_gather_119_rule(Parser *p); -static void *_tmp_121_rule(Parser *p); -static asdl_seq *_loop0_123_rule(Parser *p); -static asdl_seq *_gather_122_rule(Parser *p); -static asdl_seq *_loop0_125_rule(Parser *p); -static asdl_seq *_gather_124_rule(Parser *p); -static void *_tmp_126_rule(Parser *p); -static asdl_seq *_loop0_127_rule(Parser *p); +static asdl_seq *_gather_117_rule(Parser *p); +static asdl_seq *_loop0_119_rule(Parser *p); +static asdl_seq *_loop0_121_rule(Parser *p); +static asdl_seq *_gather_120_rule(Parser *p); +static void *_tmp_122_rule(Parser *p); +static asdl_seq *_loop0_124_rule(Parser *p); +static asdl_seq *_gather_123_rule(Parser *p); +static asdl_seq *_loop0_126_rule(Parser *p); +static asdl_seq *_gather_125_rule(Parser *p); +static void *_tmp_127_rule(Parser *p); static asdl_seq *_loop0_128_rule(Parser *p); static asdl_seq *_loop0_129_rule(Parser *p); -static void *_tmp_130_rule(Parser *p); +static asdl_seq *_loop0_130_rule(Parser *p); static void *_tmp_131_rule(Parser *p); -static asdl_seq *_loop0_132_rule(Parser *p); -static void *_tmp_133_rule(Parser *p); -static asdl_seq *_loop0_134_rule(Parser *p); -static void *_tmp_135_rule(Parser *p); +static void *_tmp_132_rule(Parser *p); +static asdl_seq *_loop0_133_rule(Parser *p); +static void *_tmp_134_rule(Parser *p); +static asdl_seq *_loop0_135_rule(Parser *p); static void *_tmp_136_rule(Parser *p); static void *_tmp_137_rule(Parser *p); static void *_tmp_138_rule(Parser *p); @@ -693,10 +695,12 @@ static void *_tmp_147_rule(Parser *p); static void *_tmp_148_rule(Parser *p); static void *_tmp_149_rule(Parser *p); static void *_tmp_150_rule(Parser *p); -static asdl_seq *_loop1_151_rule(Parser *p); -static asdl_seq *_loop1_152_rule(Parser *p); -static void *_tmp_153_rule(Parser *p); -static void *_tmp_154_rule(Parser *p); +static void *_tmp_151_rule(Parser *p); +static void *_tmp_152_rule(Parser *p); +static asdl_seq *_loop1_153_rule(Parser *p); +static asdl_seq *_loop1_154_rule(Parser *p); +static void *_tmp_155_rule(Parser *p); +static void *_tmp_156_rule(Parser *p); // file: statements? $ @@ -12198,7 +12202,7 @@ arguments_rule(Parser *p) return _res; } -// args: starred_expression [',' args] | kwargs | named_expression [',' args] +// args: ','.(starred_expression | named_expression !'=')+ [',' kwargs] | kwargs static expr_ty args_rule(Parser *p) { @@ -12218,31 +12222,22 @@ args_rule(Parser *p) UNUSED(_start_lineno); // Only used by EXTRA macro int _start_col_offset = p->tokens[_mark]->col_offset; UNUSED(_start_col_offset); // Only used by EXTRA macro - { // starred_expression [',' args] + { // ','.(starred_expression | named_expression !'=')+ [',' kwargs] if (p->error_indicator) { D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> args[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "starred_expression [',' args]")); - expr_ty a; + D(fprintf(stderr, "%*c> args[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.(starred_expression | named_expression !'=')+ [',' kwargs]")); + asdl_seq * a; void *b; if ( - (a = starred_expression_rule(p)) // starred_expression + (a = _gather_108_rule(p)) // ','.(starred_expression | named_expression !'=')+ && - (b = _tmp_108_rule(p), 1) // [',' args] + (b = _tmp_110_rule(p), 1) // [',' kwargs] ) { - D(fprintf(stderr, "%*c+ args[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "starred_expression [',' args]")); - Token *_token = _PyPegen_get_last_nonnwhitespace_token(p); - if (_token == NULL) { - D(p->level--); - return NULL; - } - int _end_lineno = _token->end_lineno; - UNUSED(_end_lineno); // Only used by EXTRA macro - int _end_col_offset = _token->end_col_offset; - UNUSED(_end_col_offset); // Only used by EXTRA macro - _res = _Py_Call ( _PyPegen_dummy_name ( p ) , ( b ) ? CHECK ( _PyPegen_seq_insert_in_front ( p , a , ( ( expr_ty ) b ) -> v . Call . args ) ) : CHECK ( _PyPegen_singleton_seq ( p , a ) ) , ( b ) ? ( ( expr_ty ) b ) -> v . Call . keywords : NULL , EXTRA ); + D(fprintf(stderr, "%*c+ args[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.(starred_expression | named_expression !'=')+ [',' kwargs]")); + _res = _PyPegen_collect_call_seqs ( p , a , b ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -12252,7 +12247,7 @@ args_rule(Parser *p) } p->mark = _mark; D(fprintf(stderr, "%*c%s args[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "starred_expression [',' args]")); + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "','.(starred_expression | named_expression !'=')+ [',' kwargs]")); } { // kwargs if (p->error_indicator) { @@ -12287,42 +12282,6 @@ args_rule(Parser *p) D(fprintf(stderr, "%*c%s args[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "kwargs")); } - { // named_expression [',' args] - if (p->error_indicator) { - D(p->level--); - return NULL; - } - D(fprintf(stderr, "%*c> args[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "named_expression [',' args]")); - expr_ty a; - void *b; - if ( - (a = named_expression_rule(p)) // named_expression - && - (b = _tmp_109_rule(p), 1) // [',' args] - ) - { - D(fprintf(stderr, "%*c+ args[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "named_expression [',' args]")); - Token *_token = _PyPegen_get_last_nonnwhitespace_token(p); - if (_token == NULL) { - D(p->level--); - return NULL; - } - int _end_lineno = _token->end_lineno; - UNUSED(_end_lineno); // Only used by EXTRA macro - int _end_col_offset = _token->end_col_offset; - UNUSED(_end_col_offset); // Only used by EXTRA macro - _res = _Py_Call ( _PyPegen_dummy_name ( p ) , ( b ) ? CHECK ( _PyPegen_seq_insert_in_front ( p , a , ( ( expr_ty ) b ) -> v . Call . args ) ) : CHECK ( _PyPegen_singleton_seq ( p , a ) ) , ( b ) ? ( ( expr_ty ) b ) -> v . Call . keywords : NULL , EXTRA ); - if (_res == NULL && PyErr_Occurred()) { - p->error_indicator = 1; - D(p->level--); - return NULL; - } - goto done; - } - p->mark = _mark; - D(fprintf(stderr, "%*c%s args[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "named_expression [',' args]")); - } _res = NULL; done: D(p->level--); @@ -12353,11 +12312,11 @@ kwargs_rule(Parser *p) asdl_seq * a; asdl_seq * b; if ( - (a = _gather_110_rule(p)) // ','.kwarg_or_starred+ + (a = _gather_111_rule(p)) // ','.kwarg_or_starred+ && (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (b = _gather_112_rule(p)) // ','.kwarg_or_double_starred+ + (b = _gather_113_rule(p)) // ','.kwarg_or_double_starred+ ) { D(fprintf(stderr, "%*c+ kwargs[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.kwarg_or_starred+ ',' ','.kwarg_or_double_starred+")); @@ -12379,13 +12338,13 @@ kwargs_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> kwargs[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.kwarg_or_starred+")); - asdl_seq * _gather_114_var; + asdl_seq * _gather_115_var; if ( - (_gather_114_var = _gather_114_rule(p)) // ','.kwarg_or_starred+ + (_gather_115_var = _gather_115_rule(p)) // ','.kwarg_or_starred+ ) { D(fprintf(stderr, "%*c+ kwargs[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.kwarg_or_starred+")); - _res = _gather_114_var; + _res = _gather_115_var; goto done; } p->mark = _mark; @@ -12398,13 +12357,13 @@ kwargs_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> kwargs[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.kwarg_or_double_starred+")); - asdl_seq * _gather_116_var; + asdl_seq * _gather_117_var; if ( - (_gather_116_var = _gather_116_rule(p)) // ','.kwarg_or_double_starred+ + (_gather_117_var = _gather_117_rule(p)) // ','.kwarg_or_double_starred+ ) { D(fprintf(stderr, "%*c+ kwargs[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.kwarg_or_double_starred+")); - _res = _gather_116_var; + _res = _gather_117_var; goto done; } p->mark = _mark; @@ -12766,7 +12725,7 @@ star_targets_rule(Parser *p) if ( (a = star_target_rule(p)) // star_target && - (b = _loop0_118_rule(p)) // ((',' star_target))* + (b = _loop0_119_rule(p)) // ((',' star_target))* && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? ) @@ -12820,7 +12779,7 @@ star_targets_seq_rule(Parser *p) UNUSED(_opt_var); // Silence compiler warnings asdl_seq * a; if ( - (a = _gather_119_rule(p)) // ','.star_target+ + (a = _gather_120_rule(p)) // ','.star_target+ && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? ) @@ -12883,7 +12842,7 @@ star_target_rule(Parser *p) if ( (_literal = _PyPegen_expect_token(p, 16)) // token='*' && - (a = _tmp_121_rule(p)) // !'*' star_target + (a = _tmp_122_rule(p)) // !'*' star_target ) { D(fprintf(stderr, "%*c+ star_target[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' (!'*' star_target)")); @@ -13405,7 +13364,7 @@ del_targets_rule(Parser *p) UNUSED(_opt_var); // Silence compiler warnings asdl_seq * a; if ( - (a = _gather_122_rule(p)) // ','.del_target+ + (a = _gather_123_rule(p)) // ','.del_target+ && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? ) @@ -13746,7 +13705,7 @@ targets_rule(Parser *p) UNUSED(_opt_var); // Silence compiler warnings asdl_seq * a; if ( - (a = _gather_124_rule(p)) // ','.target+ + (a = _gather_125_rule(p)) // ','.target+ && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? ) @@ -14458,7 +14417,7 @@ incorrect_arguments_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_opt_var = _tmp_126_rule(p), 1) // [args | expression for_if_clauses] + (_opt_var = _tmp_127_rule(p), 1) // [args | expression for_if_clauses] ) { D(fprintf(stderr, "%*c+ incorrect_arguments[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression for_if_clauses ',' [args | expression for_if_clauses]")); @@ -14716,7 +14675,7 @@ invalid_assignment_rule(Parser *p) D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expression ',' star_named_expressions* ':' expression")); Token * _literal; Token * _literal_1; - asdl_seq * _loop0_127_var; + asdl_seq * _loop0_128_var; expr_ty a; expr_ty expression_var; if ( @@ -14724,7 +14683,7 @@ invalid_assignment_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_loop0_127_var = _loop0_127_rule(p)) // star_named_expressions* + (_loop0_128_var = _loop0_128_rule(p)) // star_named_expressions* && (_literal_1 = _PyPegen_expect_token(p, 11)) // token=':' && @@ -14781,10 +14740,10 @@ invalid_assignment_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "((star_targets '='))* star_expressions '='")); Token * _literal; - asdl_seq * _loop0_128_var; + asdl_seq * _loop0_129_var; expr_ty a; if ( - (_loop0_128_var = _loop0_128_rule(p)) // ((star_targets '='))* + (_loop0_129_var = _loop0_129_rule(p)) // ((star_targets '='))* && (a = star_expressions_rule(p)) // star_expressions && @@ -14811,10 +14770,10 @@ invalid_assignment_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "((star_targets '='))* yield_expr '='")); Token * _literal; - asdl_seq * _loop0_129_var; + asdl_seq * _loop0_130_var; expr_ty a; if ( - (_loop0_129_var = _loop0_129_rule(p)) // ((star_targets '='))* + (_loop0_130_var = _loop0_130_rule(p)) // ((star_targets '='))* && (a = yield_expr_rule(p)) // yield_expr && @@ -14840,7 +14799,7 @@ invalid_assignment_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions augassign (yield_expr | star_expressions)")); - void *_tmp_130_var; + void *_tmp_131_var; expr_ty a; AugOperator* augassign_var; if ( @@ -14848,7 +14807,7 @@ invalid_assignment_rule(Parser *p) && (augassign_var = augassign_rule(p)) // augassign && - (_tmp_130_var = _tmp_130_rule(p)) // yield_expr | star_expressions + (_tmp_131_var = _tmp_131_rule(p)) // yield_expr | star_expressions ) { D(fprintf(stderr, "%*c+ invalid_assignment[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions augassign (yield_expr | star_expressions)")); @@ -15059,11 +15018,11 @@ invalid_comprehension_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '(' | '{') starred_expression for_if_clauses")); - void *_tmp_131_var; + void *_tmp_132_var; expr_ty a; asdl_seq* for_if_clauses_var; if ( - (_tmp_131_var = _tmp_131_rule(p)) // '[' | '(' | '{' + (_tmp_132_var = _tmp_132_rule(p)) // '[' | '(' | '{' && (a = starred_expression_rule(p)) // starred_expression && @@ -15160,13 +15119,13 @@ invalid_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default* (slash_with_default | param_with_default+) param_no_default")); - asdl_seq * _loop0_132_var; - void *_tmp_133_var; + asdl_seq * _loop0_133_var; + void *_tmp_134_var; arg_ty param_no_default_var; if ( - (_loop0_132_var = _loop0_132_rule(p)) // param_no_default* + (_loop0_133_var = _loop0_133_rule(p)) // param_no_default* && - (_tmp_133_var = _tmp_133_rule(p)) // slash_with_default | param_with_default+ + (_tmp_134_var = _tmp_134_rule(p)) // slash_with_default | param_with_default+ && (param_no_default_var = param_no_default_rule(p)) // param_no_default ) @@ -15208,13 +15167,13 @@ invalid_lambda_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> invalid_lambda_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default* (lambda_slash_with_default | lambda_param_with_default+) lambda_param_no_default")); - asdl_seq * _loop0_134_var; - void *_tmp_135_var; + asdl_seq * _loop0_135_var; + void *_tmp_136_var; arg_ty lambda_param_no_default_var; if ( - (_loop0_134_var = _loop0_134_rule(p)) // lambda_param_no_default* + (_loop0_135_var = _loop0_135_rule(p)) // lambda_param_no_default* && - (_tmp_135_var = _tmp_135_rule(p)) // lambda_slash_with_default | lambda_param_with_default+ + (_tmp_136_var = _tmp_136_rule(p)) // lambda_slash_with_default | lambda_param_with_default+ && (lambda_param_no_default_var = lambda_param_no_default_rule(p)) // lambda_param_no_default ) @@ -15256,11 +15215,11 @@ invalid_star_etc_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_star_etc[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'*' (')' | ',' (')' | '**'))")); Token * _literal; - void *_tmp_136_var; + void *_tmp_137_var; if ( (_literal = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_136_var = _tmp_136_rule(p)) // ')' | ',' (')' | '**') + (_tmp_137_var = _tmp_137_rule(p)) // ')' | ',' (')' | '**') ) { D(fprintf(stderr, "%*c+ invalid_star_etc[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' (')' | ',' (')' | '**'))")); @@ -15330,11 +15289,11 @@ invalid_lambda_star_etc_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_lambda_star_etc[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'*' (':' | ',' (':' | '**'))")); Token * _literal; - void *_tmp_137_var; + void *_tmp_138_var; if ( (_literal = _PyPegen_expect_token(p, 16)) // token='*' && - (_tmp_137_var = _tmp_137_rule(p)) // ':' | ',' (':' | '**') + (_tmp_138_var = _tmp_138_rule(p)) // ':' | ',' (':' | '**') ) { D(fprintf(stderr, "%*c+ invalid_lambda_star_etc[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' (':' | ',' (':' | '**'))")); @@ -16843,12 +16802,12 @@ _loop1_22_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_22[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_138_var; + void *_tmp_139_var; while ( - (_tmp_138_var = _tmp_138_rule(p)) // star_targets '=' + (_tmp_139_var = _tmp_139_rule(p)) // star_targets '=' ) { - _res = _tmp_138_var; + _res = _tmp_139_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -17351,12 +17310,12 @@ _loop0_31_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_31[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('.' | '...')")); - void *_tmp_139_var; + void *_tmp_140_var; while ( - (_tmp_139_var = _tmp_139_rule(p)) // '.' | '...' + (_tmp_140_var = _tmp_140_rule(p)) // '.' | '...' ) { - _res = _tmp_139_var; + _res = _tmp_140_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -17417,12 +17376,12 @@ _loop1_32_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_32[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('.' | '...')")); - void *_tmp_140_var; + void *_tmp_141_var; while ( - (_tmp_140_var = _tmp_140_rule(p)) // '.' | '...' + (_tmp_141_var = _tmp_141_rule(p)) // '.' | '...' ) { - _res = _tmp_140_var; + _res = _tmp_141_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -19579,12 +19538,12 @@ _loop1_68_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_68[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('@' named_expression NEWLINE)")); - void *_tmp_141_var; + void *_tmp_142_var; while ( - (_tmp_141_var = _tmp_141_rule(p)) // '@' named_expression NEWLINE + (_tmp_142_var = _tmp_142_rule(p)) // '@' named_expression NEWLINE ) { - _res = _tmp_141_var; + _res = _tmp_142_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -19811,12 +19770,12 @@ _loop1_72_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_72[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_expression)")); - void *_tmp_142_var; + void *_tmp_143_var; while ( - (_tmp_142_var = _tmp_142_rule(p)) // ',' star_expression + (_tmp_143_var = _tmp_143_rule(p)) // ',' star_expression ) { - _res = _tmp_142_var; + _res = _tmp_143_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -19996,12 +19955,12 @@ _loop1_75_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_75[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' expression)")); - void *_tmp_143_var; + void *_tmp_144_var; while ( - (_tmp_143_var = _tmp_143_rule(p)) // ',' expression + (_tmp_144_var = _tmp_144_rule(p)) // ',' expression ) { - _res = _tmp_143_var; + _res = _tmp_144_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -21026,12 +20985,12 @@ _loop1_90_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_90[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('or' conjunction)")); - void *_tmp_144_var; + void *_tmp_145_var; while ( - (_tmp_144_var = _tmp_144_rule(p)) // 'or' conjunction + (_tmp_145_var = _tmp_145_rule(p)) // 'or' conjunction ) { - _res = _tmp_144_var; + _res = _tmp_145_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -21097,12 +21056,12 @@ _loop1_91_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_91[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('and' inversion)")); - void *_tmp_145_var; + void *_tmp_146_var; while ( - (_tmp_145_var = _tmp_145_rule(p)) // 'and' inversion + (_tmp_146_var = _tmp_146_rule(p)) // 'and' inversion ) { - _res = _tmp_145_var; + _res = _tmp_146_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -22018,12 +21977,12 @@ _loop0_106_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_106[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('if' disjunction)")); - void *_tmp_146_var; + void *_tmp_147_var; while ( - (_tmp_146_var = _tmp_146_rule(p)) // 'if' disjunction + (_tmp_147_var = _tmp_147_rule(p)) // 'if' disjunction ) { - _res = _tmp_146_var; + _res = _tmp_147_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -22084,12 +22043,12 @@ _loop0_107_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_107[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('if' disjunction)")); - void *_tmp_147_var; + void *_tmp_148_var; while ( - (_tmp_147_var = _tmp_147_rule(p)) // 'if' disjunction + (_tmp_148_var = _tmp_148_rule(p)) // 'if' disjunction ) { - _res = _tmp_147_var; + _res = _tmp_148_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -22123,43 +22082,113 @@ _loop0_107_rule(Parser *p) return _seq; } -// _tmp_108: ',' args -static void * -_tmp_108_rule(Parser *p) +// _loop0_109: ',' (starred_expression | named_expression !'=') +static asdl_seq * +_loop0_109_rule(Parser *p) { D(p->level++); if (p->error_indicator) { D(p->level--); return NULL; } - void * _res = NULL; + void *_res = NULL; int _mark = p->mark; - { // ',' args + int _start_mark = p->mark; + void **_children = PyMem_Malloc(sizeof(void *)); + if (!_children) { + p->error_indicator = 1; + PyErr_NoMemory(); + D(p->level--); + return NULL; + } + ssize_t _children_capacity = 1; + ssize_t _n = 0; + { // ',' (starred_expression | named_expression !'=') if (p->error_indicator) { D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_108[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' args")); + D(fprintf(stderr, "%*c> _loop0_109[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (starred_expression | named_expression !'=')")); Token * _literal; - expr_ty c; - if ( + void *elem; + while ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (c = args_rule(p)) // args + (elem = _tmp_149_rule(p)) // starred_expression | named_expression !'=' ) { - D(fprintf(stderr, "%*c+ _tmp_108[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' args")); - _res = c; + _res = elem; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; + PyMem_Free(_children); D(p->level--); return NULL; } + if (_n == _children_capacity) { + _children_capacity *= 2; + void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); + if (!_new_children) { + p->error_indicator = 1; + PyErr_NoMemory(); + D(p->level--); + return NULL; + } + _children = _new_children; + } + _children[_n++] = _res; + _mark = p->mark; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _loop0_109[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (starred_expression | named_expression !'=')")); + } + asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + if (!_seq) { + PyMem_Free(_children); + p->error_indicator = 1; + PyErr_NoMemory(); + D(p->level--); + return NULL; + } + for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + PyMem_Free(_children); + _PyPegen_insert_memo(p, _start_mark, _loop0_109_type, _seq); + D(p->level--); + return _seq; +} + +// _gather_108: (starred_expression | named_expression !'=') _loop0_109 +static asdl_seq * +_gather_108_rule(Parser *p) +{ + D(p->level++); + if (p->error_indicator) { + D(p->level--); + return NULL; + } + asdl_seq * _res = NULL; + int _mark = p->mark; + { // (starred_expression | named_expression !'=') _loop0_109 + if (p->error_indicator) { + D(p->level--); + return NULL; + } + D(fprintf(stderr, "%*c> _gather_108[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(starred_expression | named_expression !'=') _loop0_109")); + void *elem; + asdl_seq * seq; + if ( + (elem = _tmp_149_rule(p)) // starred_expression | named_expression !'=' + && + (seq = _loop0_109_rule(p)) // _loop0_109 + ) + { + D(fprintf(stderr, "%*c+ _gather_108[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(starred_expression | named_expression !'=') _loop0_109")); + _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_108[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' args")); + D(fprintf(stderr, "%*c%s _gather_108[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(starred_expression | named_expression !'=') _loop0_109")); } _res = NULL; done: @@ -22167,9 +22196,9 @@ _tmp_108_rule(Parser *p) return _res; } -// _tmp_109: ',' args +// _tmp_110: ',' kwargs static void * -_tmp_109_rule(Parser *p) +_tmp_110_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22178,22 +22207,22 @@ _tmp_109_rule(Parser *p) } void * _res = NULL; int _mark = p->mark; - { // ',' args + { // ',' kwargs if (p->error_indicator) { D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_109[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' args")); + D(fprintf(stderr, "%*c> _tmp_110[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' kwargs")); Token * _literal; - expr_ty c; + asdl_seq* k; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (c = args_rule(p)) // args + (k = kwargs_rule(p)) // kwargs ) { - D(fprintf(stderr, "%*c+ _tmp_109[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' args")); - _res = c; + D(fprintf(stderr, "%*c+ _tmp_110[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' kwargs")); + _res = k; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -22202,8 +22231,8 @@ _tmp_109_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_109[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' args")); + D(fprintf(stderr, "%*c%s _tmp_110[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' kwargs")); } _res = NULL; done: @@ -22211,9 +22240,9 @@ _tmp_109_rule(Parser *p) return _res; } -// _loop0_111: ',' kwarg_or_starred +// _loop0_112: ',' kwarg_or_starred static asdl_seq * -_loop0_111_rule(Parser *p) +_loop0_112_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22237,7 +22266,7 @@ _loop0_111_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_111[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' kwarg_or_starred")); + D(fprintf(stderr, "%*c> _loop0_112[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' kwarg_or_starred")); Token * _literal; KeywordOrStarred* elem; while ( @@ -22268,7 +22297,7 @@ _loop0_111_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_111[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_112[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' kwarg_or_starred")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -22281,14 +22310,14 @@ _loop0_111_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_111_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_112_type, _seq); D(p->level--); return _seq; } -// _gather_110: kwarg_or_starred _loop0_111 +// _gather_111: kwarg_or_starred _loop0_112 static asdl_seq * -_gather_110_rule(Parser *p) +_gather_111_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22297,27 +22326,27 @@ _gather_110_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // kwarg_or_starred _loop0_111 + { // kwarg_or_starred _loop0_112 if (p->error_indicator) { D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _gather_110[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "kwarg_or_starred _loop0_111")); + D(fprintf(stderr, "%*c> _gather_111[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "kwarg_or_starred _loop0_112")); KeywordOrStarred* elem; asdl_seq * seq; if ( (elem = kwarg_or_starred_rule(p)) // kwarg_or_starred && - (seq = _loop0_111_rule(p)) // _loop0_111 + (seq = _loop0_112_rule(p)) // _loop0_112 ) { - D(fprintf(stderr, "%*c+ _gather_110[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "kwarg_or_starred _loop0_111")); + D(fprintf(stderr, "%*c+ _gather_111[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "kwarg_or_starred _loop0_112")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_110[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "kwarg_or_starred _loop0_111")); + D(fprintf(stderr, "%*c%s _gather_111[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "kwarg_or_starred _loop0_112")); } _res = NULL; done: @@ -22325,9 +22354,9 @@ _gather_110_rule(Parser *p) return _res; } -// _loop0_113: ',' kwarg_or_double_starred +// _loop0_114: ',' kwarg_or_double_starred static asdl_seq * -_loop0_113_rule(Parser *p) +_loop0_114_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22351,7 +22380,7 @@ _loop0_113_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_113[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' kwarg_or_double_starred")); + D(fprintf(stderr, "%*c> _loop0_114[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' kwarg_or_double_starred")); Token * _literal; KeywordOrStarred* elem; while ( @@ -22382,7 +22411,7 @@ _loop0_113_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_113[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_114[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' kwarg_or_double_starred")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -22395,14 +22424,14 @@ _loop0_113_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_113_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_114_type, _seq); D(p->level--); return _seq; } -// _gather_112: kwarg_or_double_starred _loop0_113 +// _gather_113: kwarg_or_double_starred _loop0_114 static asdl_seq * -_gather_112_rule(Parser *p) +_gather_113_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22411,27 +22440,27 @@ _gather_112_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // kwarg_or_double_starred _loop0_113 + { // kwarg_or_double_starred _loop0_114 if (p->error_indicator) { D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _gather_112[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "kwarg_or_double_starred _loop0_113")); + D(fprintf(stderr, "%*c> _gather_113[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "kwarg_or_double_starred _loop0_114")); KeywordOrStarred* elem; asdl_seq * seq; if ( (elem = kwarg_or_double_starred_rule(p)) // kwarg_or_double_starred && - (seq = _loop0_113_rule(p)) // _loop0_113 + (seq = _loop0_114_rule(p)) // _loop0_114 ) { - D(fprintf(stderr, "%*c+ _gather_112[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "kwarg_or_double_starred _loop0_113")); + D(fprintf(stderr, "%*c+ _gather_113[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "kwarg_or_double_starred _loop0_114")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_112[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "kwarg_or_double_starred _loop0_113")); + D(fprintf(stderr, "%*c%s _gather_113[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "kwarg_or_double_starred _loop0_114")); } _res = NULL; done: @@ -22439,9 +22468,9 @@ _gather_112_rule(Parser *p) return _res; } -// _loop0_115: ',' kwarg_or_starred +// _loop0_116: ',' kwarg_or_starred static asdl_seq * -_loop0_115_rule(Parser *p) +_loop0_116_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22465,7 +22494,7 @@ _loop0_115_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_115[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' kwarg_or_starred")); + D(fprintf(stderr, "%*c> _loop0_116[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' kwarg_or_starred")); Token * _literal; KeywordOrStarred* elem; while ( @@ -22496,7 +22525,7 @@ _loop0_115_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_115[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_116[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' kwarg_or_starred")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -22509,14 +22538,14 @@ _loop0_115_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_115_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_116_type, _seq); D(p->level--); return _seq; } -// _gather_114: kwarg_or_starred _loop0_115 +// _gather_115: kwarg_or_starred _loop0_116 static asdl_seq * -_gather_114_rule(Parser *p) +_gather_115_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22525,27 +22554,27 @@ _gather_114_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // kwarg_or_starred _loop0_115 + { // kwarg_or_starred _loop0_116 if (p->error_indicator) { D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _gather_114[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "kwarg_or_starred _loop0_115")); + D(fprintf(stderr, "%*c> _gather_115[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "kwarg_or_starred _loop0_116")); KeywordOrStarred* elem; asdl_seq * seq; if ( (elem = kwarg_or_starred_rule(p)) // kwarg_or_starred && - (seq = _loop0_115_rule(p)) // _loop0_115 + (seq = _loop0_116_rule(p)) // _loop0_116 ) { - D(fprintf(stderr, "%*c+ _gather_114[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "kwarg_or_starred _loop0_115")); + D(fprintf(stderr, "%*c+ _gather_115[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "kwarg_or_starred _loop0_116")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_114[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "kwarg_or_starred _loop0_115")); + D(fprintf(stderr, "%*c%s _gather_115[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "kwarg_or_starred _loop0_116")); } _res = NULL; done: @@ -22553,9 +22582,9 @@ _gather_114_rule(Parser *p) return _res; } -// _loop0_117: ',' kwarg_or_double_starred +// _loop0_118: ',' kwarg_or_double_starred static asdl_seq * -_loop0_117_rule(Parser *p) +_loop0_118_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22579,7 +22608,7 @@ _loop0_117_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_117[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' kwarg_or_double_starred")); + D(fprintf(stderr, "%*c> _loop0_118[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' kwarg_or_double_starred")); Token * _literal; KeywordOrStarred* elem; while ( @@ -22610,7 +22639,7 @@ _loop0_117_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_117[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_118[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' kwarg_or_double_starred")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -22623,14 +22652,14 @@ _loop0_117_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_117_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_118_type, _seq); D(p->level--); return _seq; } -// _gather_116: kwarg_or_double_starred _loop0_117 +// _gather_117: kwarg_or_double_starred _loop0_118 static asdl_seq * -_gather_116_rule(Parser *p) +_gather_117_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22639,27 +22668,27 @@ _gather_116_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // kwarg_or_double_starred _loop0_117 + { // kwarg_or_double_starred _loop0_118 if (p->error_indicator) { D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _gather_116[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "kwarg_or_double_starred _loop0_117")); + D(fprintf(stderr, "%*c> _gather_117[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "kwarg_or_double_starred _loop0_118")); KeywordOrStarred* elem; asdl_seq * seq; if ( (elem = kwarg_or_double_starred_rule(p)) // kwarg_or_double_starred && - (seq = _loop0_117_rule(p)) // _loop0_117 + (seq = _loop0_118_rule(p)) // _loop0_118 ) { - D(fprintf(stderr, "%*c+ _gather_116[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "kwarg_or_double_starred _loop0_117")); + D(fprintf(stderr, "%*c+ _gather_117[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "kwarg_or_double_starred _loop0_118")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_116[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "kwarg_or_double_starred _loop0_117")); + D(fprintf(stderr, "%*c%s _gather_117[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "kwarg_or_double_starred _loop0_118")); } _res = NULL; done: @@ -22667,9 +22696,9 @@ _gather_116_rule(Parser *p) return _res; } -// _loop0_118: (',' star_target) +// _loop0_119: (',' star_target) static asdl_seq * -_loop0_118_rule(Parser *p) +_loop0_119_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22693,13 +22722,13 @@ _loop0_118_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_118[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_target)")); - void *_tmp_148_var; + D(fprintf(stderr, "%*c> _loop0_119[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(',' star_target)")); + void *_tmp_150_var; while ( - (_tmp_148_var = _tmp_148_rule(p)) // ',' star_target + (_tmp_150_var = _tmp_150_rule(p)) // ',' star_target ) { - _res = _tmp_148_var; + _res = _tmp_150_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -22715,7 +22744,7 @@ _loop0_118_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_118[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_119[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(',' star_target)")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -22728,14 +22757,14 @@ _loop0_118_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_118_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_119_type, _seq); D(p->level--); return _seq; } -// _loop0_120: ',' star_target +// _loop0_121: ',' star_target static asdl_seq * -_loop0_120_rule(Parser *p) +_loop0_121_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22759,7 +22788,7 @@ _loop0_120_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_120[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c> _loop0_121[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); Token * _literal; expr_ty elem; while ( @@ -22790,7 +22819,7 @@ _loop0_120_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_120[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_121[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_target")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -22803,14 +22832,14 @@ _loop0_120_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_120_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_121_type, _seq); D(p->level--); return _seq; } -// _gather_119: star_target _loop0_120 +// _gather_120: star_target _loop0_121 static asdl_seq * -_gather_119_rule(Parser *p) +_gather_120_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22819,27 +22848,27 @@ _gather_119_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // star_target _loop0_120 + { // star_target _loop0_121 if (p->error_indicator) { D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _gather_119[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_target _loop0_120")); + D(fprintf(stderr, "%*c> _gather_120[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_target _loop0_121")); expr_ty elem; asdl_seq * seq; if ( (elem = star_target_rule(p)) // star_target && - (seq = _loop0_120_rule(p)) // _loop0_120 + (seq = _loop0_121_rule(p)) // _loop0_121 ) { - D(fprintf(stderr, "%*c+ _gather_119[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_target _loop0_120")); + D(fprintf(stderr, "%*c+ _gather_120[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_target _loop0_121")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_119[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_target _loop0_120")); + D(fprintf(stderr, "%*c%s _gather_120[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_target _loop0_121")); } _res = NULL; done: @@ -22847,9 +22876,9 @@ _gather_119_rule(Parser *p) return _res; } -// _tmp_121: !'*' star_target +// _tmp_122: !'*' star_target static void * -_tmp_121_rule(Parser *p) +_tmp_122_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22863,7 +22892,7 @@ _tmp_121_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_121[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "!'*' star_target")); + D(fprintf(stderr, "%*c> _tmp_122[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "!'*' star_target")); expr_ty star_target_var; if ( _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, 16) // token='*' @@ -22871,12 +22900,12 @@ _tmp_121_rule(Parser *p) (star_target_var = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_121[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "!'*' star_target")); + D(fprintf(stderr, "%*c+ _tmp_122[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "!'*' star_target")); _res = star_target_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_121[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_122[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "!'*' star_target")); } _res = NULL; @@ -22885,9 +22914,9 @@ _tmp_121_rule(Parser *p) return _res; } -// _loop0_123: ',' del_target +// _loop0_124: ',' del_target static asdl_seq * -_loop0_123_rule(Parser *p) +_loop0_124_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22911,7 +22940,7 @@ _loop0_123_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_123[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' del_target")); + D(fprintf(stderr, "%*c> _loop0_124[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' del_target")); Token * _literal; expr_ty elem; while ( @@ -22942,7 +22971,7 @@ _loop0_123_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_123[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_124[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' del_target")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -22955,14 +22984,14 @@ _loop0_123_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_123_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_124_type, _seq); D(p->level--); return _seq; } -// _gather_122: del_target _loop0_123 +// _gather_123: del_target _loop0_124 static asdl_seq * -_gather_122_rule(Parser *p) +_gather_123_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -22971,27 +23000,27 @@ _gather_122_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // del_target _loop0_123 + { // del_target _loop0_124 if (p->error_indicator) { D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _gather_122[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "del_target _loop0_123")); + D(fprintf(stderr, "%*c> _gather_123[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "del_target _loop0_124")); expr_ty elem; asdl_seq * seq; if ( (elem = del_target_rule(p)) // del_target && - (seq = _loop0_123_rule(p)) // _loop0_123 + (seq = _loop0_124_rule(p)) // _loop0_124 ) { - D(fprintf(stderr, "%*c+ _gather_122[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "del_target _loop0_123")); + D(fprintf(stderr, "%*c+ _gather_123[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "del_target _loop0_124")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_122[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "del_target _loop0_123")); + D(fprintf(stderr, "%*c%s _gather_123[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "del_target _loop0_124")); } _res = NULL; done: @@ -22999,9 +23028,9 @@ _gather_122_rule(Parser *p) return _res; } -// _loop0_125: ',' target +// _loop0_126: ',' target static asdl_seq * -_loop0_125_rule(Parser *p) +_loop0_126_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23025,7 +23054,7 @@ _loop0_125_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_125[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' target")); + D(fprintf(stderr, "%*c> _loop0_126[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' target")); Token * _literal; expr_ty elem; while ( @@ -23056,7 +23085,7 @@ _loop0_125_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_125[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_126[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' target")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -23069,14 +23098,14 @@ _loop0_125_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_125_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_126_type, _seq); D(p->level--); return _seq; } -// _gather_124: target _loop0_125 +// _gather_125: target _loop0_126 static asdl_seq * -_gather_124_rule(Parser *p) +_gather_125_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23085,27 +23114,27 @@ _gather_124_rule(Parser *p) } asdl_seq * _res = NULL; int _mark = p->mark; - { // target _loop0_125 + { // target _loop0_126 if (p->error_indicator) { D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _gather_124[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "target _loop0_125")); + D(fprintf(stderr, "%*c> _gather_125[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "target _loop0_126")); expr_ty elem; asdl_seq * seq; if ( (elem = target_rule(p)) // target && - (seq = _loop0_125_rule(p)) // _loop0_125 + (seq = _loop0_126_rule(p)) // _loop0_126 ) { - D(fprintf(stderr, "%*c+ _gather_124[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "target _loop0_125")); + D(fprintf(stderr, "%*c+ _gather_125[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "target _loop0_126")); _res = _PyPegen_seq_insert_in_front(p, elem, seq); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _gather_124[%d-%d]: %s failed!\n", p->level, ' ', - p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "target _loop0_125")); + D(fprintf(stderr, "%*c%s _gather_125[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "target _loop0_126")); } _res = NULL; done: @@ -23113,9 +23142,9 @@ _gather_124_rule(Parser *p) return _res; } -// _tmp_126: args | expression for_if_clauses +// _tmp_127: args | expression for_if_clauses static void * -_tmp_126_rule(Parser *p) +_tmp_127_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23129,18 +23158,18 @@ _tmp_126_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_126[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "args")); + D(fprintf(stderr, "%*c> _tmp_127[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "args")); expr_ty args_var; if ( (args_var = args_rule(p)) // args ) { - D(fprintf(stderr, "%*c+ _tmp_126[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "args")); + D(fprintf(stderr, "%*c+ _tmp_127[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "args")); _res = args_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_126[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_127[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "args")); } { // expression for_if_clauses @@ -23148,7 +23177,7 @@ _tmp_126_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_126[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression for_if_clauses")); + D(fprintf(stderr, "%*c> _tmp_127[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression for_if_clauses")); expr_ty expression_var; asdl_seq* for_if_clauses_var; if ( @@ -23157,12 +23186,12 @@ _tmp_126_rule(Parser *p) (for_if_clauses_var = for_if_clauses_rule(p)) // for_if_clauses ) { - D(fprintf(stderr, "%*c+ _tmp_126[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression for_if_clauses")); + D(fprintf(stderr, "%*c+ _tmp_127[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "expression for_if_clauses")); _res = _PyPegen_dummy_name(p, expression_var, for_if_clauses_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_126[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_127[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "expression for_if_clauses")); } _res = NULL; @@ -23171,9 +23200,9 @@ _tmp_126_rule(Parser *p) return _res; } -// _loop0_127: star_named_expressions +// _loop0_128: star_named_expressions static asdl_seq * -_loop0_127_rule(Parser *p) +_loop0_128_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23197,7 +23226,7 @@ _loop0_127_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_127[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expressions")); + D(fprintf(stderr, "%*c> _loop0_128[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expressions")); asdl_seq* star_named_expressions_var; while ( (star_named_expressions_var = star_named_expressions_rule(p)) // star_named_expressions @@ -23219,7 +23248,7 @@ _loop0_127_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_127[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_128[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_named_expressions")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -23232,14 +23261,14 @@ _loop0_127_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_127_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_128_type, _seq); D(p->level--); return _seq; } -// _loop0_128: (star_targets '=') +// _loop0_129: (star_targets '=') static asdl_seq * -_loop0_128_rule(Parser *p) +_loop0_129_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23263,13 +23292,13 @@ _loop0_128_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_128[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_149_var; + D(fprintf(stderr, "%*c> _loop0_129[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); + void *_tmp_151_var; while ( - (_tmp_149_var = _tmp_149_rule(p)) // star_targets '=' + (_tmp_151_var = _tmp_151_rule(p)) // star_targets '=' ) { - _res = _tmp_149_var; + _res = _tmp_151_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -23285,7 +23314,7 @@ _loop0_128_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_128[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_129[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -23298,14 +23327,14 @@ _loop0_128_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_128_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_129_type, _seq); D(p->level--); return _seq; } -// _loop0_129: (star_targets '=') +// _loop0_130: (star_targets '=') static asdl_seq * -_loop0_129_rule(Parser *p) +_loop0_130_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23329,13 +23358,13 @@ _loop0_129_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_129[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); - void *_tmp_150_var; + D(fprintf(stderr, "%*c> _loop0_130[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(star_targets '=')")); + void *_tmp_152_var; while ( - (_tmp_150_var = _tmp_150_rule(p)) // star_targets '=' + (_tmp_152_var = _tmp_152_rule(p)) // star_targets '=' ) { - _res = _tmp_150_var; + _res = _tmp_152_var; if (_n == _children_capacity) { _children_capacity *= 2; void **_new_children = PyMem_Realloc(_children, _children_capacity*sizeof(void *)); @@ -23351,7 +23380,7 @@ _loop0_129_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_129[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_130[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -23364,14 +23393,14 @@ _loop0_129_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_129_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_130_type, _seq); D(p->level--); return _seq; } -// _tmp_130: yield_expr | star_expressions +// _tmp_131: yield_expr | star_expressions static void * -_tmp_130_rule(Parser *p) +_tmp_131_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23385,18 +23414,18 @@ _tmp_130_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_130[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "yield_expr")); + D(fprintf(stderr, "%*c> _tmp_131[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "yield_expr")); expr_ty yield_expr_var; if ( (yield_expr_var = yield_expr_rule(p)) // yield_expr ) { - D(fprintf(stderr, "%*c+ _tmp_130[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "yield_expr")); + D(fprintf(stderr, "%*c+ _tmp_131[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "yield_expr")); _res = yield_expr_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_130[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_131[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "yield_expr")); } { // star_expressions @@ -23404,18 +23433,18 @@ _tmp_130_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_130[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions")); + D(fprintf(stderr, "%*c> _tmp_131[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_expressions")); expr_ty star_expressions_var; if ( (star_expressions_var = star_expressions_rule(p)) // star_expressions ) { - D(fprintf(stderr, "%*c+ _tmp_130[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions")); + D(fprintf(stderr, "%*c+ _tmp_131[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_expressions")); _res = star_expressions_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_130[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_131[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_expressions")); } _res = NULL; @@ -23424,9 +23453,9 @@ _tmp_130_rule(Parser *p) return _res; } -// _tmp_131: '[' | '(' | '{' +// _tmp_132: '[' | '(' | '{' static void * -_tmp_131_rule(Parser *p) +_tmp_132_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23440,18 +23469,18 @@ _tmp_131_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_131[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c> _tmp_132[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'['")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' ) { - D(fprintf(stderr, "%*c+ _tmp_131[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); + D(fprintf(stderr, "%*c+ _tmp_132[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'['")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_131[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_132[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'['")); } { // '(' @@ -23459,18 +23488,18 @@ _tmp_131_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_131[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'('")); + D(fprintf(stderr, "%*c> _tmp_132[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'('")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 7)) // token='(' ) { - D(fprintf(stderr, "%*c+ _tmp_131[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'('")); + D(fprintf(stderr, "%*c+ _tmp_132[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'('")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_131[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_132[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'('")); } { // '{' @@ -23478,18 +23507,18 @@ _tmp_131_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_131[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c> _tmp_132[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' ) { - D(fprintf(stderr, "%*c+ _tmp_131[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); + D(fprintf(stderr, "%*c+ _tmp_132[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'{'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_131[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_132[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'{'")); } _res = NULL; @@ -23498,9 +23527,9 @@ _tmp_131_rule(Parser *p) return _res; } -// _loop0_132: param_no_default +// _loop0_133: param_no_default static asdl_seq * -_loop0_132_rule(Parser *p) +_loop0_133_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23524,7 +23553,7 @@ _loop0_132_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_132[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); + D(fprintf(stderr, "%*c> _loop0_133[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default")); arg_ty param_no_default_var; while ( (param_no_default_var = param_no_default_rule(p)) // param_no_default @@ -23546,7 +23575,7 @@ _loop0_132_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_132[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_133[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -23559,14 +23588,14 @@ _loop0_132_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_132_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_133_type, _seq); D(p->level--); return _seq; } -// _tmp_133: slash_with_default | param_with_default+ +// _tmp_134: slash_with_default | param_with_default+ static void * -_tmp_133_rule(Parser *p) +_tmp_134_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23580,18 +23609,18 @@ _tmp_133_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_133[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "slash_with_default")); + D(fprintf(stderr, "%*c> _tmp_134[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "slash_with_default")); SlashWithDefault* slash_with_default_var; if ( (slash_with_default_var = slash_with_default_rule(p)) // slash_with_default ) { - D(fprintf(stderr, "%*c+ _tmp_133[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "slash_with_default")); + D(fprintf(stderr, "%*c+ _tmp_134[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "slash_with_default")); _res = slash_with_default_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_133[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_134[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "slash_with_default")); } { // param_with_default+ @@ -23599,18 +23628,18 @@ _tmp_133_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_133[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default+")); - asdl_seq * _loop1_151_var; + D(fprintf(stderr, "%*c> _tmp_134[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default+")); + asdl_seq * _loop1_153_var; if ( - (_loop1_151_var = _loop1_151_rule(p)) // param_with_default+ + (_loop1_153_var = _loop1_153_rule(p)) // param_with_default+ ) { - D(fprintf(stderr, "%*c+ _tmp_133[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "param_with_default+")); - _res = _loop1_151_var; + D(fprintf(stderr, "%*c+ _tmp_134[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "param_with_default+")); + _res = _loop1_153_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_133[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_134[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_with_default+")); } _res = NULL; @@ -23619,9 +23648,9 @@ _tmp_133_rule(Parser *p) return _res; } -// _loop0_134: lambda_param_no_default +// _loop0_135: lambda_param_no_default static asdl_seq * -_loop0_134_rule(Parser *p) +_loop0_135_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23645,7 +23674,7 @@ _loop0_134_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop0_134[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); + D(fprintf(stderr, "%*c> _loop0_135[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default")); arg_ty lambda_param_no_default_var; while ( (lambda_param_no_default_var = lambda_param_no_default_rule(p)) // lambda_param_no_default @@ -23667,7 +23696,7 @@ _loop0_134_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop0_134[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop0_135[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_no_default")); } asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); @@ -23680,14 +23709,14 @@ _loop0_134_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop0_134_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop0_135_type, _seq); D(p->level--); return _seq; } -// _tmp_135: lambda_slash_with_default | lambda_param_with_default+ +// _tmp_136: lambda_slash_with_default | lambda_param_with_default+ static void * -_tmp_135_rule(Parser *p) +_tmp_136_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23701,18 +23730,18 @@ _tmp_135_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_135[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_slash_with_default")); + D(fprintf(stderr, "%*c> _tmp_136[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_slash_with_default")); SlashWithDefault* lambda_slash_with_default_var; if ( (lambda_slash_with_default_var = lambda_slash_with_default_rule(p)) // lambda_slash_with_default ) { - D(fprintf(stderr, "%*c+ _tmp_135[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_slash_with_default")); + D(fprintf(stderr, "%*c+ _tmp_136[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_slash_with_default")); _res = lambda_slash_with_default_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_135[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_136[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_slash_with_default")); } { // lambda_param_with_default+ @@ -23720,18 +23749,18 @@ _tmp_135_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_135[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); - asdl_seq * _loop1_152_var; + D(fprintf(stderr, "%*c> _tmp_136[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); + asdl_seq * _loop1_154_var; if ( - (_loop1_152_var = _loop1_152_rule(p)) // lambda_param_with_default+ + (_loop1_154_var = _loop1_154_rule(p)) // lambda_param_with_default+ ) { - D(fprintf(stderr, "%*c+ _tmp_135[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); - _res = _loop1_152_var; + D(fprintf(stderr, "%*c+ _tmp_136[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default+")); + _res = _loop1_154_var; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_135[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_136[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_with_default+")); } _res = NULL; @@ -23740,9 +23769,9 @@ _tmp_135_rule(Parser *p) return _res; } -// _tmp_136: ')' | ',' (')' | '**') +// _tmp_137: ')' | ',' (')' | '**') static void * -_tmp_136_rule(Parser *p) +_tmp_137_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23756,18 +23785,18 @@ _tmp_136_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_136[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_137[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_136[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_137[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_136[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_137[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // ',' (')' | '**') @@ -23775,21 +23804,21 @@ _tmp_136_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_136[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); + D(fprintf(stderr, "%*c> _tmp_137[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); Token * _literal; - void *_tmp_153_var; + void *_tmp_155_var; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_tmp_153_var = _tmp_153_rule(p)) // ')' | '**' + (_tmp_155_var = _tmp_155_rule(p)) // ')' | '**' ) { - D(fprintf(stderr, "%*c+ _tmp_136[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); - _res = _PyPegen_dummy_name(p, _literal, _tmp_153_var); + D(fprintf(stderr, "%*c+ _tmp_137[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (')' | '**')")); + _res = _PyPegen_dummy_name(p, _literal, _tmp_155_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_136[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_137[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (')' | '**')")); } _res = NULL; @@ -23798,9 +23827,9 @@ _tmp_136_rule(Parser *p) return _res; } -// _tmp_137: ':' | ',' (':' | '**') +// _tmp_138: ':' | ',' (':' | '**') static void * -_tmp_137_rule(Parser *p) +_tmp_138_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23814,18 +23843,18 @@ _tmp_137_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_137[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_138[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_137[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_138[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_137[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_138[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } { // ',' (':' | '**') @@ -23833,21 +23862,21 @@ _tmp_137_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_137[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); + D(fprintf(stderr, "%*c> _tmp_138[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); Token * _literal; - void *_tmp_154_var; + void *_tmp_156_var; if ( (_literal = _PyPegen_expect_token(p, 12)) // token=',' && - (_tmp_154_var = _tmp_154_rule(p)) // ':' | '**' + (_tmp_156_var = _tmp_156_rule(p)) // ':' | '**' ) { - D(fprintf(stderr, "%*c+ _tmp_137[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); - _res = _PyPegen_dummy_name(p, _literal, _tmp_154_var); + D(fprintf(stderr, "%*c+ _tmp_138[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' (':' | '**')")); + _res = _PyPegen_dummy_name(p, _literal, _tmp_156_var); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_137[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_138[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (':' | '**')")); } _res = NULL; @@ -23856,9 +23885,9 @@ _tmp_137_rule(Parser *p) return _res; } -// _tmp_138: star_targets '=' +// _tmp_139: star_targets '=' static void * -_tmp_138_rule(Parser *p) +_tmp_139_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23872,7 +23901,7 @@ _tmp_138_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_138[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_139[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty z; if ( @@ -23881,7 +23910,7 @@ _tmp_138_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_138[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_139[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -23891,7 +23920,7 @@ _tmp_138_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_138[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_139[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -23900,9 +23929,9 @@ _tmp_138_rule(Parser *p) return _res; } -// _tmp_139: '.' | '...' +// _tmp_140: '.' | '...' static void * -_tmp_139_rule(Parser *p) +_tmp_140_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23916,18 +23945,18 @@ _tmp_139_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_139[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c> _tmp_140[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 23)) // token='.' ) { - D(fprintf(stderr, "%*c+ _tmp_139[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c+ _tmp_140[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_139[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_140[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'.'")); } { // '...' @@ -23935,18 +23964,18 @@ _tmp_139_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_139[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c> _tmp_140[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 52)) // token='...' ) { - D(fprintf(stderr, "%*c+ _tmp_139[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c+ _tmp_140[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_139[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_140[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'...'")); } _res = NULL; @@ -23955,9 +23984,9 @@ _tmp_139_rule(Parser *p) return _res; } -// _tmp_140: '.' | '...' +// _tmp_141: '.' | '...' static void * -_tmp_140_rule(Parser *p) +_tmp_141_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -23971,18 +24000,18 @@ _tmp_140_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_140[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c> _tmp_141[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'.'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 23)) // token='.' ) { - D(fprintf(stderr, "%*c+ _tmp_140[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); + D(fprintf(stderr, "%*c+ _tmp_141[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'.'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_140[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_141[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'.'")); } { // '...' @@ -23990,18 +24019,18 @@ _tmp_140_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_140[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c> _tmp_141[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'...'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 52)) // token='...' ) { - D(fprintf(stderr, "%*c+ _tmp_140[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); + D(fprintf(stderr, "%*c+ _tmp_141[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'...'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_140[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_141[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'...'")); } _res = NULL; @@ -24010,9 +24039,9 @@ _tmp_140_rule(Parser *p) return _res; } -// _tmp_141: '@' named_expression NEWLINE +// _tmp_142: '@' named_expression NEWLINE static void * -_tmp_141_rule(Parser *p) +_tmp_142_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24026,7 +24055,7 @@ _tmp_141_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_141[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); + D(fprintf(stderr, "%*c> _tmp_142[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); Token * _literal; expr_ty f; Token * newline_var; @@ -24038,7 +24067,7 @@ _tmp_141_rule(Parser *p) (newline_var = _PyPegen_expect_token(p, NEWLINE)) // token='NEWLINE' ) { - D(fprintf(stderr, "%*c+ _tmp_141[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); + D(fprintf(stderr, "%*c+ _tmp_142[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'@' named_expression NEWLINE")); _res = f; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -24048,7 +24077,7 @@ _tmp_141_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_141[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_142[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'@' named_expression NEWLINE")); } _res = NULL; @@ -24057,9 +24086,9 @@ _tmp_141_rule(Parser *p) return _res; } -// _tmp_142: ',' star_expression +// _tmp_143: ',' star_expression static void * -_tmp_142_rule(Parser *p) +_tmp_143_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24073,7 +24102,7 @@ _tmp_142_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_142[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_expression")); + D(fprintf(stderr, "%*c> _tmp_143[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_expression")); Token * _literal; expr_ty c; if ( @@ -24082,7 +24111,7 @@ _tmp_142_rule(Parser *p) (c = star_expression_rule(p)) // star_expression ) { - D(fprintf(stderr, "%*c+ _tmp_142[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_expression")); + D(fprintf(stderr, "%*c+ _tmp_143[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_expression")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -24092,7 +24121,7 @@ _tmp_142_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_142[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_143[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_expression")); } _res = NULL; @@ -24101,9 +24130,9 @@ _tmp_142_rule(Parser *p) return _res; } -// _tmp_143: ',' expression +// _tmp_144: ',' expression static void * -_tmp_143_rule(Parser *p) +_tmp_144_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24117,7 +24146,7 @@ _tmp_143_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_143[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' expression")); + D(fprintf(stderr, "%*c> _tmp_144[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' expression")); Token * _literal; expr_ty c; if ( @@ -24126,7 +24155,7 @@ _tmp_143_rule(Parser *p) (c = expression_rule(p)) // expression ) { - D(fprintf(stderr, "%*c+ _tmp_143[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' expression")); + D(fprintf(stderr, "%*c+ _tmp_144[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' expression")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -24136,7 +24165,7 @@ _tmp_143_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_143[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_144[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' expression")); } _res = NULL; @@ -24145,9 +24174,9 @@ _tmp_143_rule(Parser *p) return _res; } -// _tmp_144: 'or' conjunction +// _tmp_145: 'or' conjunction static void * -_tmp_144_rule(Parser *p) +_tmp_145_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24161,7 +24190,7 @@ _tmp_144_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_144[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); + D(fprintf(stderr, "%*c> _tmp_145[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); Token * _keyword; expr_ty c; if ( @@ -24170,7 +24199,7 @@ _tmp_144_rule(Parser *p) (c = conjunction_rule(p)) // conjunction ) { - D(fprintf(stderr, "%*c+ _tmp_144[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); + D(fprintf(stderr, "%*c+ _tmp_145[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'or' conjunction")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -24180,7 +24209,7 @@ _tmp_144_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_144[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_145[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'or' conjunction")); } _res = NULL; @@ -24189,9 +24218,9 @@ _tmp_144_rule(Parser *p) return _res; } -// _tmp_145: 'and' inversion +// _tmp_146: 'and' inversion static void * -_tmp_145_rule(Parser *p) +_tmp_146_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24205,7 +24234,7 @@ _tmp_145_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_145[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'and' inversion")); + D(fprintf(stderr, "%*c> _tmp_146[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'and' inversion")); Token * _keyword; expr_ty c; if ( @@ -24214,7 +24243,7 @@ _tmp_145_rule(Parser *p) (c = inversion_rule(p)) // inversion ) { - D(fprintf(stderr, "%*c+ _tmp_145[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'and' inversion")); + D(fprintf(stderr, "%*c+ _tmp_146[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'and' inversion")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -24224,7 +24253,7 @@ _tmp_145_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_145[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_146[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'and' inversion")); } _res = NULL; @@ -24233,9 +24262,9 @@ _tmp_145_rule(Parser *p) return _res; } -// _tmp_146: 'if' disjunction +// _tmp_147: 'if' disjunction static void * -_tmp_146_rule(Parser *p) +_tmp_147_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24249,7 +24278,7 @@ _tmp_146_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_146[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); Token * _keyword; expr_ty z; if ( @@ -24258,7 +24287,7 @@ _tmp_146_rule(Parser *p) (z = disjunction_rule(p)) // disjunction ) { - D(fprintf(stderr, "%*c+ _tmp_146[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -24268,7 +24297,7 @@ _tmp_146_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_146[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'if' disjunction")); } _res = NULL; @@ -24277,9 +24306,9 @@ _tmp_146_rule(Parser *p) return _res; } -// _tmp_147: 'if' disjunction +// _tmp_148: 'if' disjunction static void * -_tmp_147_rule(Parser *p) +_tmp_148_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24293,7 +24322,7 @@ _tmp_147_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_147[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c> _tmp_148[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); Token * _keyword; expr_ty z; if ( @@ -24302,7 +24331,7 @@ _tmp_147_rule(Parser *p) (z = disjunction_rule(p)) // disjunction ) { - D(fprintf(stderr, "%*c+ _tmp_147[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); + D(fprintf(stderr, "%*c+ _tmp_148[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'if' disjunction")); _res = z; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -24312,7 +24341,7 @@ _tmp_147_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_147[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_148[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'if' disjunction")); } _res = NULL; @@ -24321,9 +24350,66 @@ _tmp_147_rule(Parser *p) return _res; } -// _tmp_148: ',' star_target +// _tmp_149: starred_expression | named_expression !'=' static void * -_tmp_148_rule(Parser *p) +_tmp_149_rule(Parser *p) +{ + D(p->level++); + if (p->error_indicator) { + D(p->level--); + return NULL; + } + void * _res = NULL; + int _mark = p->mark; + { // starred_expression + if (p->error_indicator) { + D(p->level--); + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "starred_expression")); + expr_ty starred_expression_var; + if ( + (starred_expression_var = starred_expression_rule(p)) // starred_expression + ) + { + D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "starred_expression")); + _res = starred_expression_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "starred_expression")); + } + { // named_expression !'=' + if (p->error_indicator) { + D(p->level--); + return NULL; + } + D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "named_expression !'='")); + expr_ty named_expression_var; + if ( + (named_expression_var = named_expression_rule(p)) // named_expression + && + _PyPegen_lookahead_with_int(0, _PyPegen_expect_token, p, 22) // token='=' + ) + { + D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "named_expression !'='")); + _res = named_expression_var; + goto done; + } + p->mark = _mark; + D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', + p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "named_expression !'='")); + } + _res = NULL; + done: + D(p->level--); + return _res; +} + +// _tmp_150: ',' star_target +static void * +_tmp_150_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24337,7 +24423,7 @@ _tmp_148_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_148[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c> _tmp_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "',' star_target")); Token * _literal; expr_ty c; if ( @@ -24346,7 +24432,7 @@ _tmp_148_rule(Parser *p) (c = star_target_rule(p)) // star_target ) { - D(fprintf(stderr, "%*c+ _tmp_148[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); + D(fprintf(stderr, "%*c+ _tmp_150[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "',' star_target")); _res = c; if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; @@ -24356,7 +24442,7 @@ _tmp_148_rule(Parser *p) goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_148[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_150[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_target")); } _res = NULL; @@ -24365,9 +24451,9 @@ _tmp_148_rule(Parser *p) return _res; } -// _tmp_149: star_targets '=' +// _tmp_151: star_targets '=' static void * -_tmp_149_rule(Parser *p) +_tmp_151_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24381,7 +24467,7 @@ _tmp_149_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_149[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty star_targets_var; if ( @@ -24390,12 +24476,12 @@ _tmp_149_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_149[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_151[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = _PyPegen_dummy_name(p, star_targets_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_149[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_151[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -24404,9 +24490,9 @@ _tmp_149_rule(Parser *p) return _res; } -// _tmp_150: star_targets '=' +// _tmp_152: star_targets '=' static void * -_tmp_150_rule(Parser *p) +_tmp_152_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24420,7 +24506,7 @@ _tmp_150_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_150[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c> _tmp_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_targets '='")); Token * _literal; expr_ty star_targets_var; if ( @@ -24429,12 +24515,12 @@ _tmp_150_rule(Parser *p) (_literal = _PyPegen_expect_token(p, 22)) // token='=' ) { - D(fprintf(stderr, "%*c+ _tmp_150[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); + D(fprintf(stderr, "%*c+ _tmp_152[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "star_targets '='")); _res = _PyPegen_dummy_name(p, star_targets_var, _literal); goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_150[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_152[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_targets '='")); } _res = NULL; @@ -24443,9 +24529,9 @@ _tmp_150_rule(Parser *p) return _res; } -// _loop1_151: param_with_default +// _loop1_153: param_with_default static asdl_seq * -_loop1_151_rule(Parser *p) +_loop1_153_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24469,7 +24555,7 @@ _loop1_151_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop1_151[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default")); + D(fprintf(stderr, "%*c> _loop1_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_with_default")); NameDefaultPair* param_with_default_var; while ( (param_with_default_var = param_with_default_rule(p)) // param_with_default @@ -24491,7 +24577,7 @@ _loop1_151_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_151[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_153[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_with_default")); } if (_n == 0 || p->error_indicator) { @@ -24509,14 +24595,14 @@ _loop1_151_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_151_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_153_type, _seq); D(p->level--); return _seq; } -// _loop1_152: lambda_param_with_default +// _loop1_154: lambda_param_with_default static asdl_seq * -_loop1_152_rule(Parser *p) +_loop1_154_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24540,7 +24626,7 @@ _loop1_152_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _loop1_152[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default")); + D(fprintf(stderr, "%*c> _loop1_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_with_default")); NameDefaultPair* lambda_param_with_default_var; while ( (lambda_param_with_default_var = lambda_param_with_default_rule(p)) // lambda_param_with_default @@ -24562,7 +24648,7 @@ _loop1_152_rule(Parser *p) _mark = p->mark; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _loop1_152[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _loop1_154[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_with_default")); } if (_n == 0 || p->error_indicator) { @@ -24580,14 +24666,14 @@ _loop1_152_rule(Parser *p) } for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); PyMem_Free(_children); - _PyPegen_insert_memo(p, _start_mark, _loop1_152_type, _seq); + _PyPegen_insert_memo(p, _start_mark, _loop1_154_type, _seq); D(p->level--); return _seq; } -// _tmp_153: ')' | '**' +// _tmp_155: ')' | '**' static void * -_tmp_153_rule(Parser *p) +_tmp_155_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24601,18 +24687,18 @@ _tmp_153_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "')'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 8)) // token=')' ) { - D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); + D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "')'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "')'")); } { // '**' @@ -24620,18 +24706,18 @@ _tmp_153_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_153[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c> _tmp_155[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 35)) // token='**' ) { - D(fprintf(stderr, "%*c+ _tmp_153[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c+ _tmp_155[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_153[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_155[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'**'")); } _res = NULL; @@ -24640,9 +24726,9 @@ _tmp_153_rule(Parser *p) return _res; } -// _tmp_154: ':' | '**' +// _tmp_156: ':' | '**' static void * -_tmp_154_rule(Parser *p) +_tmp_156_rule(Parser *p) { D(p->level++); if (p->error_indicator) { @@ -24656,18 +24742,18 @@ _tmp_154_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c> _tmp_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "':'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 11)) // token=':' ) { - D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); + D(fprintf(stderr, "%*c+ _tmp_156[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "':'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_156[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "':'")); } { // '**' @@ -24675,18 +24761,18 @@ _tmp_154_rule(Parser *p) D(p->level--); return NULL; } - D(fprintf(stderr, "%*c> _tmp_154[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c> _tmp_156[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'**'")); Token * _literal; if ( (_literal = _PyPegen_expect_token(p, 35)) // token='**' ) { - D(fprintf(stderr, "%*c+ _tmp_154[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); + D(fprintf(stderr, "%*c+ _tmp_156[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**'")); _res = _literal; goto done; } p->mark = _mark; - D(fprintf(stderr, "%*c%s _tmp_154[%d-%d]: %s failed!\n", p->level, ' ', + D(fprintf(stderr, "%*c%s _tmp_156[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'**'")); } _res = NULL; diff --git a/Parser/pegen.c b/Parser/pegen.c index f615907f5f571..2507bc4b38280 100644 --- a/Parser/pegen.c +++ b/Parser/pegen.c @@ -2217,3 +2217,38 @@ _PyPegen_nonparen_genexp_in_call(Parser *p, expr_ty args) "Generator expression must be parenthesized" ); } + + +expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_seq *a, asdl_seq *b) { + Py_ssize_t args_len = asdl_seq_LEN(a); + Py_ssize_t total_len = args_len; + + if (b == NULL) { + expr_ty first = asdl_seq_GET(a, 0); + expr_ty last = asdl_seq_GET(a, args_len - 1); + return _Py_Call(_PyPegen_dummy_name(p), a, NULL, EXTRA_EXPR(first, last)); + + } + + asdl_seq *starreds = _PyPegen_seq_extract_starred_exprs(p, b); + asdl_seq *keywords = _PyPegen_seq_delete_starred_exprs(p, b); + + if (starreds) { + total_len += asdl_seq_LEN(starreds); + } + + asdl_seq *args = _Py_asdl_seq_new(total_len, p->arena); + + Py_ssize_t i = 0; + for (i = 0; i < args_len; i++) { + asdl_seq_SET(args, i, asdl_seq_GET(a, i)); + } + for (; i < total_len; i++) { + asdl_seq_SET(args, i, asdl_seq_GET(starreds, i - args_len)); + } + + expr_ty first = asdl_seq_GET(args, 0); + expr_ty last = asdl_seq_GET(b, asdl_seq_LEN(b)-1); + + return _Py_Call(_PyPegen_dummy_name(p), args, keywords, EXTRA_EXPR(first, last)); +} diff --git a/Parser/pegen.h b/Parser/pegen.h index f407709863c69..3e74e3ac73e0a 100644 --- a/Parser/pegen.h +++ b/Parser/pegen.h @@ -257,6 +257,7 @@ stmt_ty _PyPegen_class_def_decorators(Parser *, asdl_seq *, stmt_ty); KeywordOrStarred *_PyPegen_keyword_or_starred(Parser *, void *, int); asdl_seq *_PyPegen_seq_extract_starred_exprs(Parser *, asdl_seq *); asdl_seq *_PyPegen_seq_delete_starred_exprs(Parser *, asdl_seq *); +expr_ty _PyPegen_collect_call_seqs(Parser *, asdl_seq *, asdl_seq *); expr_ty _PyPegen_concatenate_strings(Parser *p, asdl_seq *); asdl_seq *_PyPegen_join_sequences(Parser *, asdl_seq *, asdl_seq *); int _PyPegen_check_barry_as_flufl(Parser *); From webhook-mailer at python.org Thu Sep 3 00:54:55 2020 From: webhook-mailer at python.org (Shantanu) Date: Thu, 03 Sep 2020 04:54:55 -0000 Subject: [Python-checkins] bpo-41696: Fix handling of debug mode in asyncio.run (#22069) Message-ID: https://github.com/python/cpython/commit/0770ad948cb6d9f7f6c4002efd83e27c27069808 commit: 0770ad948cb6d9f7f6c4002efd83e27c27069808 branch: master author: Shantanu <12621235+hauntsaninja at users.noreply.github.com> committer: GitHub date: 2020-09-02T21:54:46-07:00 summary: bpo-41696: Fix handling of debug mode in asyncio.run (#22069) * bpo-41696: Fix handling of debug mode in asyncio.run This allows PYTHONASYNCIODEBUG or -X dev to enable asyncio debug mode when using asyncio.run * ?? Added by blurb_it. Co-authored-by: hauntsaninja <> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst M Lib/asyncio/runners.py M Lib/test/test_asyncio/test_runners.py diff --git a/Lib/asyncio/runners.py b/Lib/asyncio/runners.py index 03ce33300eba8..268635d68fb0c 100644 --- a/Lib/asyncio/runners.py +++ b/Lib/asyncio/runners.py @@ -5,7 +5,7 @@ from . import tasks -def run(main, *, debug=False): +def run(main, *, debug=None): """Execute the coroutine and return the result. This function runs the passed coroutine, taking care of @@ -39,7 +39,8 @@ async def main(): loop = events.new_event_loop() try: events.set_event_loop(loop) - loop.set_debug(debug) + if debug is not None: + loop.set_debug(debug) return loop.run_until_complete(main) finally: try: diff --git a/Lib/test/test_asyncio/test_runners.py b/Lib/test/test_asyncio/test_runners.py index 3b58ddee443ad..b9ae02dc3c04e 100644 --- a/Lib/test/test_asyncio/test_runners.py +++ b/Lib/test/test_asyncio/test_runners.py @@ -87,6 +87,9 @@ async def main(expected): asyncio.run(main(False)) asyncio.run(main(True), debug=True) + with mock.patch('asyncio.coroutines._is_debug_mode', lambda: True): + asyncio.run(main(True)) + asyncio.run(main(False), debug=False) def test_asyncio_run_from_running_loop(self): async def main(): diff --git a/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst b/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst new file mode 100644 index 0000000000000..67bbbb857f18c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst @@ -0,0 +1 @@ +Fix handling of debug mode in :func:`asyncio.run`. This allows setting ``PYTHONASYNCIODEBUG`` or ``-X dev`` to enable asyncio debug mode when using :func:`asyncio.run`. \ No newline at end of file From webhook-mailer at python.org Thu Sep 3 00:58:54 2020 From: webhook-mailer at python.org (Ben Darnell) Date: Thu, 03 Sep 2020 04:58:54 -0000 Subject: [Python-checkins] bpo-39010: Improve test shutdown (#22066) Message-ID: https://github.com/python/cpython/commit/be435ae2b064dc64f04475bec632862e1dbf605f commit: be435ae2b064dc64f04475bec632862e1dbf605f branch: master author: Ben Darnell committer: GitHub date: 2020-09-02T21:58:50-07:00 summary: bpo-39010: Improve test shutdown (#22066) Simply closing the event loop isn't enough to avoid warnings. If we don't also shut down the event loop's default executor, it sometimes logs a "dangling thread" warning. Follow-up to GH-22017 files: M Lib/test/test_asyncio/test_windows_events.py diff --git a/Lib/test/test_asyncio/test_windows_events.py b/Lib/test/test_asyncio/test_windows_events.py index 33388a87d48f3..f276cd205a2f8 100644 --- a/Lib/test/test_asyncio/test_windows_events.py +++ b/Lib/test/test_asyncio/test_windows_events.py @@ -225,10 +225,18 @@ def test_read_self_pipe_restart(self): self.loop.run_forever() self.loop.stop() self.loop.run_forever() - # If we don't wait for f to complete here, we may get another - # warning logged about a thread that didn't shut down cleanly. + + # Shut everything down cleanly. This is an important part of the + # test - in issue 39010, the error occurred during loop.close(), + # so we want to close the loop during the test instead of leaving + # it for tearDown. + # + # First wait for f to complete to avoid a "future's result was never + # retrieved" error. self.loop.run_until_complete(f) - self.loop.close() + # Now shut down the loop itself (self.close_loop also shuts down the + # loop's default executor). + self.close_loop(self.loop) self.assertFalse(self.loop.call_exception_handler.called) From webhook-mailer at python.org Thu Sep 3 01:22:45 2020 From: webhook-mailer at python.org (Todd) Date: Thu, 03 Sep 2020 05:22:45 -0000 Subject: [Python-checkins] bpo-39883: Use BSD0 license for code in docs (GH-17635) Message-ID: https://github.com/python/cpython/commit/e223d06a8b2067c68e9f97c4628c2b0f056dcae6 commit: e223d06a8b2067c68e9f97c4628c2b0f056dcae6 branch: master author: Todd committer: GitHub date: 2020-09-02T22:22:36-07:00 summary: bpo-39883: Use BSD0 license for code in docs (GH-17635) The PSF board approved this use. files: A Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst M Doc/license.rst M LICENSE diff --git a/Doc/license.rst b/Doc/license.rst index fa6d71a78042d..4030825bbd28e 100644 --- a/Doc/license.rst +++ b/Doc/license.rst @@ -72,6 +72,19 @@ make these releases possible. Terms and conditions for accessing or otherwise using Python ============================================================ +Python software and documentation are licensed under the +:ref:`PSF License Agreement `. + +Starting with Python 3.8.6, examples, recipes, and other code in +the documentation are dual licensed under the PSF License Agreement +and the :ref:`Zero-Clause BSD license `. + +Some software incorporated into Python is under different licenses. +The licenses are listed with code falling under that license. +See :ref:`OtherLicenses` for an incomplete list of these licenses. + + +.. _PSF-license: PSF LICENSE AGREEMENT FOR PYTHON |release| ------------------------------------------ @@ -258,6 +271,27 @@ CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2 SOFTWARE. +.. _BSD0: + +ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON |release| DOCUMENTATION +---------------------------------------------------------------------- + +.. parsed-literal:: + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH + REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY + AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, + INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM + LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR + OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR + PERFORMANCE OF THIS SOFTWARE. + + +.. _OtherLicenses: + Licenses and Acknowledgements for Incorporated Software ======================================================= diff --git a/LICENSE b/LICENSE index 66a3ac80d729a..f42f8adbed845 100644 --- a/LICENSE +++ b/LICENSE @@ -59,6 +59,17 @@ direction to make these releases possible. B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON =============================================================== +Python software and documentation are licensed under the +Python Software Foundation License Version 2. + +Starting with Python 3.8.6, examples, recipes, and other code in +the documentation are dual licensed under the PSF License Version 2 +and the Zero-Clause BSD license. + +Some software incorporated into Python is under different licenses. +The licenses are listed with code falling under that license. + + PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2 -------------------------------------------- @@ -252,3 +263,17 @@ FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + +ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON DOCUMENTATION +---------------------------------------------------------------------- + +Permission to use, copy, modify, and/or distribute this software for any +purpose with or without fee is hereby granted. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH +REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, +INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR +OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +PERFORMANCE OF THIS SOFTWARE. diff --git a/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst b/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst new file mode 100644 index 0000000000000..4941d50a560e2 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst @@ -0,0 +1 @@ +Make code, examples, and recipes in the Python documentation be licensed under the more permissive BSD0 license in addition to the existing Python 2.0 license. \ No newline at end of file From webhook-mailer at python.org Thu Sep 3 04:21:26 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Thu, 03 Sep 2020 08:21:26 -0000 Subject: [Python-checkins] bpo-1635741: Port _signal module to multi-phase init (PEP 489) (GH-22049) Message-ID: https://github.com/python/cpython/commit/71d1bd9569c8a497e279f2fea6fe47cd70a87ea3 commit: 71d1bd9569c8a497e279f2fea6fe47cd70a87ea3 branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-03T10:21:06+02:00 summary: bpo-1635741: Port _signal module to multi-phase init (PEP 489) (GH-22049) files: A Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-07-20.bpo-1635741.7wSuCc.rst M Modules/signalmodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-07-20.bpo-1635741.7wSuCc.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-07-20.bpo-1635741.7wSuCc.rst new file mode 100644 index 0000000000000..ff7cb352869a2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-07-20.bpo-1635741.7wSuCc.rst @@ -0,0 +1 @@ +Port the :mod:`_signal` extension module to multi-phase initialization (:pep:`489`). diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c index c49a3ea52e71d..262f2b66a57e2 100644 --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -1377,77 +1377,63 @@ ITIMER_PROF -- decrements both when the process is executing and\n\ A signal handler function is called with two arguments:\n\ the first is the signal number, the second is the interrupted stack frame."); -static struct PyModuleDef signalmodule = { - PyModuleDef_HEAD_INIT, - "_signal", - module_doc, - -1, - signal_methods, - NULL, - NULL, - NULL, - NULL -}; - -PyMODINIT_FUNC -PyInit__signal(void) -{ - PyObject *m, *d; - int i; - /* Create the module and add the functions */ - m = PyModule_Create(&signalmodule); - if (m == NULL) - return NULL; +static int +signal_exec(PyObject *m) +{ + /* add the functions */ #if defined(HAVE_SIGWAITINFO) || defined(HAVE_SIGTIMEDWAIT) if (!initialized) { - if (PyStructSequence_InitType2(&SiginfoType, &struct_siginfo_desc) < 0) - return NULL; + if (PyStructSequence_InitType2(&SiginfoType, &struct_siginfo_desc) < 0) { + return -1; + } + } + + if (PyModule_AddType(m, &SiginfoType) < 0) { + return -1; } - Py_INCREF((PyObject*) &SiginfoType); - PyModule_AddObject(m, "struct_siginfo", (PyObject*) &SiginfoType); initialized = 1; #endif /* Add some symbolic constants to the module */ - d = PyModule_GetDict(m); + PyObject *d = PyModule_GetDict(m); DefaultHandler = PyLong_FromVoidPtr((void *)SIG_DFL); if (!DefaultHandler || PyDict_SetItemString(d, "SIG_DFL", DefaultHandler) < 0) { - goto finally; + return -1; } IgnoreHandler = PyLong_FromVoidPtr((void *)SIG_IGN); if (!IgnoreHandler || PyDict_SetItemString(d, "SIG_IGN", IgnoreHandler) < 0) { - goto finally; + return -1; } if (PyModule_AddIntMacro(m, NSIG)) - goto finally; + return -1; #ifdef SIG_BLOCK if (PyModule_AddIntMacro(m, SIG_BLOCK)) - goto finally; + return -1; #endif #ifdef SIG_UNBLOCK if (PyModule_AddIntMacro(m, SIG_UNBLOCK)) - goto finally; + return -1; #endif #ifdef SIG_SETMASK if (PyModule_AddIntMacro(m, SIG_SETMASK)) - goto finally; + return -1; #endif IntHandler = PyDict_GetItemString(d, "default_int_handler"); if (!IntHandler) - goto finally; + return -1; Py_INCREF(IntHandler); _Py_atomic_store_relaxed(&Handlers[0].tripped, 0); - for (i = 1; i < NSIG; i++) { + for (int i = 1; i < NSIG; i++) { void (*t)(int); t = PyOS_getsig(i); _Py_atomic_store_relaxed(&Handlers[i].tripped, 0); @@ -1468,168 +1454,168 @@ PyInit__signal(void) #ifdef SIGHUP if (PyModule_AddIntMacro(m, SIGHUP)) - goto finally; + return -1; #endif #ifdef SIGINT if (PyModule_AddIntMacro(m, SIGINT)) - goto finally; + return -1; #endif #ifdef SIGBREAK if (PyModule_AddIntMacro(m, SIGBREAK)) - goto finally; + return -1; #endif #ifdef SIGQUIT if (PyModule_AddIntMacro(m, SIGQUIT)) - goto finally; + return -1; #endif #ifdef SIGILL if (PyModule_AddIntMacro(m, SIGILL)) - goto finally; + return -1; #endif #ifdef SIGTRAP if (PyModule_AddIntMacro(m, SIGTRAP)) - goto finally; + return -1; #endif #ifdef SIGIOT if (PyModule_AddIntMacro(m, SIGIOT)) - goto finally; + return -1; #endif #ifdef SIGABRT if (PyModule_AddIntMacro(m, SIGABRT)) - goto finally; + return -1; #endif #ifdef SIGEMT if (PyModule_AddIntMacro(m, SIGEMT)) - goto finally; + return -1; #endif #ifdef SIGFPE if (PyModule_AddIntMacro(m, SIGFPE)) - goto finally; + return -1; #endif #ifdef SIGKILL if (PyModule_AddIntMacro(m, SIGKILL)) - goto finally; + return -1; #endif #ifdef SIGBUS if (PyModule_AddIntMacro(m, SIGBUS)) - goto finally; + return -1; #endif #ifdef SIGSEGV if (PyModule_AddIntMacro(m, SIGSEGV)) - goto finally; + return -1; #endif #ifdef SIGSYS if (PyModule_AddIntMacro(m, SIGSYS)) - goto finally; + return -1; #endif #ifdef SIGPIPE if (PyModule_AddIntMacro(m, SIGPIPE)) - goto finally; + return -1; #endif #ifdef SIGALRM if (PyModule_AddIntMacro(m, SIGALRM)) - goto finally; + return -1; #endif #ifdef SIGTERM if (PyModule_AddIntMacro(m, SIGTERM)) - goto finally; + return -1; #endif #ifdef SIGUSR1 if (PyModule_AddIntMacro(m, SIGUSR1)) - goto finally; + return -1; #endif #ifdef SIGUSR2 if (PyModule_AddIntMacro(m, SIGUSR2)) - goto finally; + return -1; #endif #ifdef SIGCLD if (PyModule_AddIntMacro(m, SIGCLD)) - goto finally; + return -1; #endif #ifdef SIGCHLD if (PyModule_AddIntMacro(m, SIGCHLD)) - goto finally; + return -1; #endif #ifdef SIGPWR if (PyModule_AddIntMacro(m, SIGPWR)) - goto finally; + return -1; #endif #ifdef SIGIO if (PyModule_AddIntMacro(m, SIGIO)) - goto finally; + return -1; #endif #ifdef SIGURG if (PyModule_AddIntMacro(m, SIGURG)) - goto finally; + return -1; #endif #ifdef SIGWINCH if (PyModule_AddIntMacro(m, SIGWINCH)) - goto finally; + return -1; #endif #ifdef SIGPOLL if (PyModule_AddIntMacro(m, SIGPOLL)) - goto finally; + return -1; #endif #ifdef SIGSTOP if (PyModule_AddIntMacro(m, SIGSTOP)) - goto finally; + return -1; #endif #ifdef SIGTSTP if (PyModule_AddIntMacro(m, SIGTSTP)) - goto finally; + return -1; #endif #ifdef SIGCONT if (PyModule_AddIntMacro(m, SIGCONT)) - goto finally; + return -1; #endif #ifdef SIGTTIN if (PyModule_AddIntMacro(m, SIGTTIN)) - goto finally; + return -1; #endif #ifdef SIGTTOU if (PyModule_AddIntMacro(m, SIGTTOU)) - goto finally; + return -1; #endif #ifdef SIGVTALRM if (PyModule_AddIntMacro(m, SIGVTALRM)) - goto finally; + return -1; #endif #ifdef SIGPROF if (PyModule_AddIntMacro(m, SIGPROF)) - goto finally; + return -1; #endif #ifdef SIGXCPU if (PyModule_AddIntMacro(m, SIGXCPU)) - goto finally; + return -1; #endif #ifdef SIGXFSZ if (PyModule_AddIntMacro(m, SIGXFSZ)) - goto finally; + return -1; #endif #ifdef SIGRTMIN if (PyModule_AddIntMacro(m, SIGRTMIN)) - goto finally; + return -1; #endif #ifdef SIGRTMAX if (PyModule_AddIntMacro(m, SIGRTMAX)) - goto finally; + return -1; #endif #ifdef SIGINFO if (PyModule_AddIntMacro(m, SIGINFO)) - goto finally; + return -1; #endif #ifdef ITIMER_REAL if (PyModule_AddIntMacro(m, ITIMER_REAL)) - goto finally; + return -1; #endif #ifdef ITIMER_VIRTUAL if (PyModule_AddIntMacro(m, ITIMER_VIRTUAL)) - goto finally; + return -1; #endif #ifdef ITIMER_PROF if (PyModule_AddIntMacro(m, ITIMER_PROF)) - goto finally; + return -1; #endif #if defined (HAVE_SETITIMER) || defined (HAVE_GETITIMER) @@ -1637,18 +1623,18 @@ PyInit__signal(void) PyExc_OSError, NULL); if (!ItimerError || PyDict_SetItemString(d, "ItimerError", ItimerError) < 0) { - goto finally; + return -1; } #endif #ifdef CTRL_C_EVENT if (PyModule_AddIntMacro(m, CTRL_C_EVENT)) - goto finally; + return -1; #endif #ifdef CTRL_BREAK_EVENT if (PyModule_AddIntMacro(m, CTRL_BREAK_EVENT)) - goto finally; + return -1; #endif #ifdef MS_WINDOWS @@ -1657,12 +1643,30 @@ PyInit__signal(void) #endif if (PyErr_Occurred()) { - Py_DECREF(m); - m = NULL; + return -1; } - finally: - return m; + return 0; +} + +static PyModuleDef_Slot signal_slots[] = { + {Py_mod_exec, signal_exec}, + {0, NULL} +}; + +static struct PyModuleDef signalmodule = { + PyModuleDef_HEAD_INIT, + "_signal", + .m_doc = module_doc, + .m_size = 0, + .m_methods = signal_methods, + .m_slots = signal_slots +}; + +PyMODINIT_FUNC +PyInit__signal(void) +{ + return PyModuleDef_Init(&signalmodule); } static void From webhook-mailer at python.org Thu Sep 3 10:29:44 2020 From: webhook-mailer at python.org (Pablo Galindo) Date: Thu, 03 Sep 2020 14:29:44 -0000 Subject: [Python-checkins] bpo-41697: Correctly handle KeywordOrStarred when parsing arguments in the parser (GH-22077) Message-ID: https://github.com/python/cpython/commit/315a61f7a9418d904e0eea14b1f054fac3a90e9f commit: 315a61f7a9418d904e0eea14b1f054fac3a90e9f branch: master author: Pablo Galindo committer: GitHub date: 2020-09-03T15:29:32+01:00 summary: bpo-41697: Correctly handle KeywordOrStarred when parsing arguments in the parser (GH-22077) files: M Grammar/python.gram M Parser/parser.c M Parser/pegen.c M Parser/pegen.h diff --git a/Grammar/python.gram b/Grammar/python.gram index 84835b731c540..524e88eb38996 100644 --- a/Grammar/python.gram +++ b/Grammar/python.gram @@ -535,7 +535,7 @@ arguments[expr_ty] (memo): | a=args [','] &')' { a } | incorrect_arguments args[expr_ty]: - | a=','.(starred_expression | named_expression !'=')+ b=[',' k=kwargs {k}] { _PyPegen_collect_call_seqs(p, a, b) } + | a=','.(starred_expression | named_expression !'=')+ b=[',' k=kwargs {k}] { _PyPegen_collect_call_seqs(p, a, b, EXTRA) } | a=kwargs { _Py_Call(_PyPegen_dummy_name(p), CHECK_NULL_ALLOWED(_PyPegen_seq_extract_starred_exprs(p, a)), CHECK_NULL_ALLOWED(_PyPegen_seq_delete_starred_exprs(p, a)), diff --git a/Parser/parser.c b/Parser/parser.c index 3e724a260d90a..8a7cb62fd7cf8 100644 --- a/Parser/parser.c +++ b/Parser/parser.c @@ -12237,7 +12237,16 @@ args_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ args[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.(starred_expression | named_expression !'=')+ [',' kwargs]")); - _res = _PyPegen_collect_call_seqs ( p , a , b ); + Token *_token = _PyPegen_get_last_nonnwhitespace_token(p); + if (_token == NULL) { + D(p->level--); + return NULL; + } + int _end_lineno = _token->end_lineno; + UNUSED(_end_lineno); // Only used by EXTRA macro + int _end_col_offset = _token->end_col_offset; + UNUSED(_end_col_offset); // Only used by EXTRA macro + _res = _PyPegen_collect_call_seqs ( p , a , b , EXTRA ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); diff --git a/Parser/pegen.c b/Parser/pegen.c index 2507bc4b38280..4beb2abdd296a 100644 --- a/Parser/pegen.c +++ b/Parser/pegen.c @@ -2219,14 +2219,15 @@ _PyPegen_nonparen_genexp_in_call(Parser *p, expr_ty args) } -expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_seq *a, asdl_seq *b) { +expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_seq *a, asdl_seq *b, + int lineno, int col_offset, int end_lineno, + int end_col_offset, PyArena *arena) { Py_ssize_t args_len = asdl_seq_LEN(a); Py_ssize_t total_len = args_len; if (b == NULL) { - expr_ty first = asdl_seq_GET(a, 0); - expr_ty last = asdl_seq_GET(a, args_len - 1); - return _Py_Call(_PyPegen_dummy_name(p), a, NULL, EXTRA_EXPR(first, last)); + return _Py_Call(_PyPegen_dummy_name(p), a, NULL, lineno, col_offset, + end_lineno, end_col_offset, arena); } @@ -2237,7 +2238,7 @@ expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_seq *a, asdl_seq *b) { total_len += asdl_seq_LEN(starreds); } - asdl_seq *args = _Py_asdl_seq_new(total_len, p->arena); + asdl_seq *args = _Py_asdl_seq_new(total_len, arena); Py_ssize_t i = 0; for (i = 0; i < args_len; i++) { @@ -2247,8 +2248,8 @@ expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_seq *a, asdl_seq *b) { asdl_seq_SET(args, i, asdl_seq_GET(starreds, i - args_len)); } - expr_ty first = asdl_seq_GET(args, 0); - expr_ty last = asdl_seq_GET(b, asdl_seq_LEN(b)-1); + return _Py_Call(_PyPegen_dummy_name(p), args, keywords, lineno, + col_offset, end_lineno, end_col_offset, arena); + - return _Py_Call(_PyPegen_dummy_name(p), args, keywords, EXTRA_EXPR(first, last)); } diff --git a/Parser/pegen.h b/Parser/pegen.h index 3e74e3ac73e0a..c81681efad208 100644 --- a/Parser/pegen.h +++ b/Parser/pegen.h @@ -257,7 +257,9 @@ stmt_ty _PyPegen_class_def_decorators(Parser *, asdl_seq *, stmt_ty); KeywordOrStarred *_PyPegen_keyword_or_starred(Parser *, void *, int); asdl_seq *_PyPegen_seq_extract_starred_exprs(Parser *, asdl_seq *); asdl_seq *_PyPegen_seq_delete_starred_exprs(Parser *, asdl_seq *); -expr_ty _PyPegen_collect_call_seqs(Parser *, asdl_seq *, asdl_seq *); +expr_ty _PyPegen_collect_call_seqs(Parser *, asdl_seq *, asdl_seq *, + int lineno, int col_offset, int end_lineno, + int end_col_offset, PyArena *arena); expr_ty _PyPegen_concatenate_strings(Parser *p, asdl_seq *); asdl_seq *_PyPegen_join_sequences(Parser *, asdl_seq *, asdl_seq *); int _PyPegen_check_barry_as_flufl(Parser *); From webhook-mailer at python.org Thu Sep 3 10:29:59 2020 From: webhook-mailer at python.org (Pablo Galindo) Date: Thu, 03 Sep 2020 14:29:59 -0000 Subject: [Python-checkins] Fix 'gather' rules in the python parser generator (GH-22021) Message-ID: https://github.com/python/cpython/commit/e55a0e971b4bf9e6473bf0ca6bebdff76c075ef6 commit: e55a0e971b4bf9e6473bf0ca6bebdff76c075ef6 branch: master author: Pablo Galindo committer: GitHub date: 2020-09-03T15:29:55+01:00 summary: Fix 'gather' rules in the python parser generator (GH-22021) Currently, empty sequences in gather rules make the conditional for gather rules fail as empty sequences evaluate as "False". We need to explicitly check for "None" (the failure condition) to avoid false negatives. files: M Lib/test/test_peg_generator/test_pegen.py M Tools/peg_generator/pegen/python_generator.py diff --git a/Lib/test/test_peg_generator/test_pegen.py b/Lib/test/test_peg_generator/test_pegen.py index 5b4e964d698ad..bcfee3f2c5f8c 100644 --- a/Lib/test/test_peg_generator/test_pegen.py +++ b/Lib/test/test_peg_generator/test_pegen.py @@ -74,7 +74,7 @@ def test_typed_rules(self) -> None: "Rule('term', 'int', Rhs([Alt([NamedItem(None, NameLeaf('NUMBER'))])]))" ) - def test_repeat_with_separator_rules(self) -> None: + def test_gather(self) -> None: grammar = """ start: ','.thing+ NEWLINE thing: NUMBER @@ -85,6 +85,20 @@ def test_repeat_with_separator_rules(self) -> None: "Rule('start', None, Rhs([Alt([NamedItem(None, Gather(StringLeaf(\"','\"), NameLeaf('thing'" )) self.assertEqual(str(rules["thing"]), "thing: NUMBER") + parser_class = make_parser(grammar) + node = parse_string("42\n", parser_class) + assert node == [ + [[TokenInfo(NUMBER, string="42", start=(1, 0), end=(1, 2), line="42\n")]], + TokenInfo(NEWLINE, string="\n", start=(1, 2), end=(1, 3), line="42\n"), + ] + node = parse_string("1, 2\n", parser_class) + assert node == [ + [ + [TokenInfo(NUMBER, string="1", start=(1, 0), end=(1, 1), line="1, 2\n")], + [TokenInfo(NUMBER, string="2", start=(1, 3), end=(1, 4), line="1, 2\n")], + ], + TokenInfo(NEWLINE, string="\n", start=(1, 4), end=(1, 5), line="1, 2\n"), + ] def test_expr_grammar(self) -> None: grammar = """ diff --git a/Tools/peg_generator/pegen/python_generator.py b/Tools/peg_generator/pegen/python_generator.py index 45a75975dbf5e..b786de7fee5b4 100644 --- a/Tools/peg_generator/pegen/python_generator.py +++ b/Tools/peg_generator/pegen/python_generator.py @@ -217,6 +217,9 @@ def visit_Alt(self, node: Alt, is_loop: bool, is_gather: bool) -> None: else: self.print("and") self.visit(item) + if is_gather: + self.print("is not None") + self.print("):") with self.indent(): action = node.action From webhook-mailer at python.org Thu Sep 3 10:41:19 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 03 Sep 2020 14:41:19 -0000 Subject: [Python-checkins] bpo-39883: Use BSD0 license for code in docs (GH-17635) (GH-22074) Message-ID: https://github.com/python/cpython/commit/9fef7c54a0adfade5ec94259d97f22e05fe9e3e3 commit: 9fef7c54a0adfade5ec94259d97f22e05fe9e3e3 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-03T07:41:09-07:00 summary: bpo-39883: Use BSD0 license for code in docs (GH-17635) (GH-22074) The PSF board approved this use. (cherry picked from commit e223d06a8b2067c68e9f97c4628c2b0f056dcae6) Co-authored-by: Todd files: A Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst M Doc/license.rst M LICENSE diff --git a/Doc/license.rst b/Doc/license.rst index 472a5cf3d88b3..f70c34d3fde60 100644 --- a/Doc/license.rst +++ b/Doc/license.rst @@ -72,6 +72,19 @@ make these releases possible. Terms and conditions for accessing or otherwise using Python ============================================================ +Python software and documentation are licensed under the +:ref:`PSF License Agreement `. + +Starting with Python 3.8.6, examples, recipes, and other code in +the documentation are dual licensed under the PSF License Agreement +and the :ref:`Zero-Clause BSD license `. + +Some software incorporated into Python is under different licenses. +The licenses are listed with code falling under that license. +See :ref:`OtherLicenses` for an incomplete list of these licenses. + + +.. _PSF-license: PSF LICENSE AGREEMENT FOR PYTHON |release| ------------------------------------------ @@ -258,6 +271,27 @@ CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2 SOFTWARE. +.. _BSD0: + +ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON |release| DOCUMENTATION +---------------------------------------------------------------------- + +.. parsed-literal:: + + Permission to use, copy, modify, and/or distribute this software for any + purpose with or without fee is hereby granted. + + THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH + REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY + AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, + INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM + LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR + OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR + PERFORMANCE OF THIS SOFTWARE. + + +.. _OtherLicenses: + Licenses and Acknowledgements for Incorporated Software ======================================================= diff --git a/LICENSE b/LICENSE index 66a3ac80d729a..f42f8adbed845 100644 --- a/LICENSE +++ b/LICENSE @@ -59,6 +59,17 @@ direction to make these releases possible. B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON =============================================================== +Python software and documentation are licensed under the +Python Software Foundation License Version 2. + +Starting with Python 3.8.6, examples, recipes, and other code in +the documentation are dual licensed under the PSF License Version 2 +and the Zero-Clause BSD license. + +Some software incorporated into Python is under different licenses. +The licenses are listed with code falling under that license. + + PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2 -------------------------------------------- @@ -252,3 +263,17 @@ FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + +ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON DOCUMENTATION +---------------------------------------------------------------------- + +Permission to use, copy, modify, and/or distribute this software for any +purpose with or without fee is hereby granted. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH +REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, +INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR +OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +PERFORMANCE OF THIS SOFTWARE. diff --git a/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst b/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst new file mode 100644 index 0000000000000..4941d50a560e2 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst @@ -0,0 +1 @@ +Make code, examples, and recipes in the Python documentation be licensed under the more permissive BSD0 license in addition to the existing Python 2.0 license. \ No newline at end of file From webhook-mailer at python.org Thu Sep 3 14:44:27 2020 From: webhook-mailer at python.org (Vinay Sajip) Date: Thu, 03 Sep 2020 18:44:27 -0000 Subject: [Python-checkins] [doc] Update documentation on logging optimization. (GH-22075) Message-ID: https://github.com/python/cpython/commit/cdbff3527c5b10e79761ad25316b46cd079304bc commit: cdbff3527c5b10e79761ad25316b46cd079304bc branch: master author: Vinay Sajip committer: GitHub date: 2020-09-03T19:44:12+01:00 summary: [doc] Update documentation on logging optimization. (GH-22075) files: M Doc/howto/logging.rst diff --git a/Doc/howto/logging.rst b/Doc/howto/logging.rst index 6316e086ef43b..b4dd9206c9d52 100644 --- a/Doc/howto/logging.rst +++ b/Doc/howto/logging.rst @@ -1078,20 +1078,22 @@ need more precise control over what logging information is collected. Here's a list of things you can do to avoid processing during logging which you don't need: -+-----------------------------------------------+----------------------------------------+ -| What you don't want to collect | How to avoid collecting it | -+===============================================+========================================+ -| Information about where calls were made from. | Set ``logging._srcfile`` to ``None``. | -| | This avoids calling | -| | :func:`sys._getframe`, which may help | -| | to speed up your code in environments | -| | like PyPy (which can't speed up code | -| | that uses :func:`sys._getframe`). | -+-----------------------------------------------+----------------------------------------+ -| Threading information. | Set ``logging.logThreads`` to ``0``. | -+-----------------------------------------------+----------------------------------------+ -| Process information. | Set ``logging.logProcesses`` to ``0``. | -+-----------------------------------------------+----------------------------------------+ ++-----------------------------------------------------+---------------------------------------------------+ +| What you don't want to collect | How to avoid collecting it | ++=====================================================+===================================================+ +| Information about where calls were made from. | Set ``logging._srcfile`` to ``None``. | +| | This avoids calling :func:`sys._getframe`, which | +| | may help to speed up your code in environments | +| | like PyPy (which can't speed up code that uses | +| | :func:`sys._getframe`). | ++-----------------------------------------------------+---------------------------------------------------+ +| Threading information. | Set ``logging.logThreads`` to ``False``. | ++-----------------------------------------------------+---------------------------------------------------+ +| Current process ID (:func:`os.getpid`) | Set ``logging.logProcesses`` to ``False``. | ++-----------------------------------------------------+---------------------------------------------------+ +| Current process name when using ``multiprocessing`` | Set ``logging.logMultiprocessing`` to ``False``. | +| to manage multiple processes. | | ++-----------------------------------------------------+---------------------------------------------------+ Also note that the core logging module only includes the basic handlers. If you don't import :mod:`logging.handlers` and :mod:`logging.config`, they won't From webhook-mailer at python.org Thu Sep 3 15:38:39 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 03 Sep 2020 19:38:39 -0000 Subject: [Python-checkins] bpo-39010: Fix errors logged on proactor loop restart (GH-22017) (#22035) Message-ID: https://github.com/python/cpython/commit/a986b061a3cd4661eb9f857e2936291f1847bd15 commit: a986b061a3cd4661eb9f857e2936291f1847bd15 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-03T12:38:34-07:00 summary: bpo-39010: Fix errors logged on proactor loop restart (GH-22017) (#22035) Stopping and restarting a proactor event loop on windows can lead to spurious errors logged (ConnectionResetError while reading from the self pipe). This fixes the issue by ensuring that we don't attempt to start multiple copies of the self-pipe reading loop. (cherry picked from commit ea5a6363c3f8cc90b7c0cc573922b10f296073b6) Co-authored-by: Ben Darnell Co-authored-by: Ben Darnell files: A Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rst M Lib/asyncio/proactor_events.py M Lib/asyncio/windows_events.py M Lib/test/test_asyncio/test_proactor_events.py M Lib/test/test_asyncio/test_windows_events.py diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py index 830d8edc32f90..54e4deb2c349c 100644 --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -766,6 +766,14 @@ def _loop_self_reading(self, f=None): try: if f is not None: f.result() # may raise + if self._self_reading_future is not f: + # When we scheduled this Future, we assigned it to + # _self_reading_future. If it's not there now, something has + # tried to cancel the loop while this callback was still in the + # queue (see windows_events.ProactorEventLoop.run_forever). In + # that case stop here instead of continuing to schedule a new + # iteration. + return f = self._proactor.recv(self._ssock, 4096) except exceptions.CancelledError: # _close_self_pipe() has been called, stop waiting for data diff --git a/Lib/asyncio/windows_events.py b/Lib/asyncio/windows_events.py index 12e87abfbf2ea..a5ad6fc423fbe 100644 --- a/Lib/asyncio/windows_events.py +++ b/Lib/asyncio/windows_events.py @@ -318,8 +318,12 @@ def run_forever(self): if self._self_reading_future is not None: ov = self._self_reading_future._ov self._self_reading_future.cancel() - # self_reading_future was just cancelled so it will never be signalled - # Unregister it otherwise IocpProactor.close will wait for it forever + # self_reading_future was just cancelled so if it hasn't been + # finished yet, it never will be (it's possible that it has + # already finished and its callback is waiting in the queue, + # where it could still happen if the event loop is restarted). + # Unregister it otherwise IocpProactor.close will wait for it + # forever if ov is not None: self._proactor._unregister(ov) self._self_reading_future = None diff --git a/Lib/test/test_asyncio/test_proactor_events.py b/Lib/test/test_asyncio/test_proactor_events.py index 007039a7cdf5d..e4809c303a71d 100644 --- a/Lib/test/test_asyncio/test_proactor_events.py +++ b/Lib/test/test_asyncio/test_proactor_events.py @@ -736,6 +736,7 @@ def test_loop_self_reading(self): def test_loop_self_reading_fut(self): fut = mock.Mock() + self.loop._self_reading_future = fut self.loop._loop_self_reading(fut) self.assertTrue(fut.result.called) self.proactor.recv.assert_called_with(self.ssock, 4096) diff --git a/Lib/test/test_asyncio/test_windows_events.py b/Lib/test/test_asyncio/test_windows_events.py index 6b005702c9be7..33388a87d48f3 100644 --- a/Lib/test/test_asyncio/test_windows_events.py +++ b/Lib/test/test_asyncio/test_windows_events.py @@ -211,6 +211,26 @@ def test_wait_for_handle_cancel(self): fut.cancel() fut.cancel() + def test_read_self_pipe_restart(self): + # Regression test for https://bugs.python.org/issue39010 + # Previously, restarting a proactor event loop in certain states + # would lead to spurious ConnectionResetErrors being logged. + self.loop.call_exception_handler = mock.Mock() + # Start an operation in another thread so that the self-pipe is used. + # This is theoretically timing-dependent (the task in the executor + # must complete before our start/stop cycles), but in practice it + # seems to work every time. + f = self.loop.run_in_executor(None, lambda: None) + self.loop.stop() + self.loop.run_forever() + self.loop.stop() + self.loop.run_forever() + # If we don't wait for f to complete here, we may get another + # warning logged about a thread that didn't shut down cleanly. + self.loop.run_until_complete(f) + self.loop.close() + self.assertFalse(self.loop.call_exception_handler.called) + class WinPolicyTests(test_utils.TestCase): diff --git a/Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rst b/Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rst new file mode 100644 index 0000000000000..0d9015b490e4c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rst @@ -0,0 +1,2 @@ +Restarting a ``ProactorEventLoop`` on Windows no longer logs spurious +``ConnectionResetErrors``. From webhook-mailer at python.org Thu Sep 3 16:54:16 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 03 Sep 2020 20:54:16 -0000 Subject: [Python-checkins] bpo-41696: Fix handling of debug mode in asyncio.run (GH-22069) (#22072) Message-ID: https://github.com/python/cpython/commit/1f5f12737755f246fee83a54c600779ad76934a4 commit: 1f5f12737755f246fee83a54c600779ad76934a4 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-03T13:54:06-07:00 summary: bpo-41696: Fix handling of debug mode in asyncio.run (GH-22069) (#22072) * bpo-41696: Fix handling of debug mode in asyncio.run This allows PYTHONASYNCIODEBUG or -X dev to enable asyncio debug mode when using asyncio.run * ?? Added by blurb_it. Co-authored-by: hauntsaninja <> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> (cherry picked from commit 0770ad948cb6d9f7f6c4002efd83e27c27069808) Co-authored-by: Shantanu <12621235+hauntsaninja at users.noreply.github.com> Co-authored-by: Shantanu <12621235+hauntsaninja at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst M Lib/asyncio/runners.py M Lib/test/test_asyncio/test_runners.py diff --git a/Lib/asyncio/runners.py b/Lib/asyncio/runners.py index 2e37e18b45fc1..d2fa8b7f13861 100644 --- a/Lib/asyncio/runners.py +++ b/Lib/asyncio/runners.py @@ -5,7 +5,7 @@ from . import tasks -def run(main, *, debug=False): +def run(main, *, debug=None): """Execute the coroutine and return the result. This function runs the passed coroutine, taking care of @@ -39,7 +39,8 @@ async def main(): loop = events.new_event_loop() try: events.set_event_loop(loop) - loop.set_debug(debug) + if debug is not None: + loop.set_debug(debug) return loop.run_until_complete(main) finally: try: diff --git a/Lib/test/test_asyncio/test_runners.py b/Lib/test/test_asyncio/test_runners.py index 3b58ddee443ad..b9ae02dc3c04e 100644 --- a/Lib/test/test_asyncio/test_runners.py +++ b/Lib/test/test_asyncio/test_runners.py @@ -87,6 +87,9 @@ async def main(expected): asyncio.run(main(False)) asyncio.run(main(True), debug=True) + with mock.patch('asyncio.coroutines._is_debug_mode', lambda: True): + asyncio.run(main(True)) + asyncio.run(main(False), debug=False) def test_asyncio_run_from_running_loop(self): async def main(): diff --git a/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst b/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst new file mode 100644 index 0000000000000..67bbbb857f18c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst @@ -0,0 +1 @@ +Fix handling of debug mode in :func:`asyncio.run`. This allows setting ``PYTHONASYNCIODEBUG`` or ``-X dev`` to enable asyncio debug mode when using :func:`asyncio.run`. \ No newline at end of file From webhook-mailer at python.org Thu Sep 3 16:54:41 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 03 Sep 2020 20:54:41 -0000 Subject: [Python-checkins] bpo-39010: Improve test shutdown (GH-22066) (#22083) Message-ID: https://github.com/python/cpython/commit/40e2444c364eede59f193979df5a02ed958f56e9 commit: 40e2444c364eede59f193979df5a02ed958f56e9 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-03T13:54:36-07:00 summary: bpo-39010: Improve test shutdown (GH-22066) (#22083) Simply closing the event loop isn't enough to avoid warnings. If we don't also shut down the event loop's default executor, it sometimes logs a "dangling thread" warning. Follow-up to GH-22017 (cherry picked from commit be435ae2b064dc64f04475bec632862e1dbf605f) Co-authored-by: Ben Darnell Co-authored-by: Ben Darnell files: M Lib/test/test_asyncio/test_windows_events.py diff --git a/Lib/test/test_asyncio/test_windows_events.py b/Lib/test/test_asyncio/test_windows_events.py index 33388a87d48f3..f276cd205a2f8 100644 --- a/Lib/test/test_asyncio/test_windows_events.py +++ b/Lib/test/test_asyncio/test_windows_events.py @@ -225,10 +225,18 @@ def test_read_self_pipe_restart(self): self.loop.run_forever() self.loop.stop() self.loop.run_forever() - # If we don't wait for f to complete here, we may get another - # warning logged about a thread that didn't shut down cleanly. + + # Shut everything down cleanly. This is an important part of the + # test - in issue 39010, the error occurred during loop.close(), + # so we want to close the loop during the test instead of leaving + # it for tearDown. + # + # First wait for f to complete to avoid a "future's result was never + # retrieved" error. self.loop.run_until_complete(f) - self.loop.close() + # Now shut down the loop itself (self.close_loop also shuts down the + # loop's default executor). + self.close_loop(self.loop) self.assertFalse(self.loop.call_exception_handler.called) From webhook-mailer at python.org Thu Sep 3 20:54:25 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 04 Sep 2020 00:54:25 -0000 Subject: [Python-checkins] bpo-39603: Prevent header injection in http methods (GH-18485) (#21946) Message-ID: https://github.com/python/cpython/commit/524b8de630036a29ca340bc2ae6fd6dc7dda8f40 commit: 524b8de630036a29ca340bc2ae6fd6dc7dda8f40 branch: 3.5 author: Victor Stinner committer: GitHub date: 2020-09-03T17:54:15-07:00 summary: bpo-39603: Prevent header injection in http methods (GH-18485) (#21946) reject control chars in http method in http.client.putrequest to prevent http header injection (cherry picked from commit 8ca8a2e8fb068863c1138f07e3098478ef8be12e) Co-authored-by: AMIR <31338382+amiremohamadi at users.noreply.github.com> files: A Misc/NEWS.d/next/Security/2020-02-12-14-17-39.bpo-39603.Gt3RSg.rst M Lib/http/client.py M Lib/test/test_httplib.py diff --git a/Lib/http/client.py b/Lib/http/client.py index 5d5d7a7f707b4..f7efe151f5b04 100644 --- a/Lib/http/client.py +++ b/Lib/http/client.py @@ -151,6 +151,10 @@ # _is_allowed_url_pchars_re = re.compile(r"^[/!$&'()*+,;=:@%a-zA-Z0-9._~-]+$") # We are more lenient for assumed real world compatibility purposes. +# These characters are not allowed within HTTP method names +# to prevent http header injection. +_contains_disallowed_method_pchar_re = re.compile('[\x00-\x1f]') + # We always set the Content-Length header for these methods because some # servers will otherwise respond with a 411 _METHODS_EXPECTING_BODY = {'PATCH', 'POST', 'PUT'} @@ -985,6 +989,8 @@ def putrequest(self, method, url, skip_host=False, else: raise CannotSendRequest(self.__state) + self._validate_method(method) + # Save the method for use later in the response phase self._method = method @@ -1075,6 +1081,16 @@ def _encode_request(self, request): # ASCII also helps prevent CVE-2019-9740. return request.encode('ascii') + def _validate_method(self, method): + """Validate a method name for putrequest.""" + # prevent http header injection + match = _contains_disallowed_method_pchar_re.search(method) + if match: + raise ValueError( + "method can't contain control characters. %r " + "(found at least %r)" + % (method, match.group())) + def _validate_path(self, url): """Validate a url for putrequest.""" # Prevent CVE-2019-9740. diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py index a8e7a30af1a9f..b1cd08b23a4b1 100644 --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -344,6 +344,28 @@ def test_invalid_headers(self): conn.putheader(name, value) +class HttpMethodTests(TestCase): + def test_invalid_method_names(self): + methods = ( + 'GET\r', + 'POST\n', + 'PUT\n\r', + 'POST\nValue', + 'POST\nHOST:abc', + 'GET\nrHost:abc\n', + 'POST\rRemainder:\r', + 'GET\rHOST:\n', + '\nPUT' + ) + + for method in methods: + with self.assertRaisesRegex( + ValueError, "method can't contain control characters"): + conn = client.HTTPConnection('example.com') + conn.sock = FakeSocket(None) + conn.request(method=method, url="/") + + class BasicTest(TestCase): def test_status_lines(self): # Test HTTP status lines @@ -1783,8 +1805,8 @@ def test_tunnel_debuglog(self): @support.reap_threads def test_main(verbose=None): - support.run_unittest(HeaderTests, OfflineTest, BasicTest, TimeoutTest, - PersistenceTest, + support.run_unittest(HeaderTests, OfflineTest, HttpMethodTests, + BasicTest, TimeoutTest, PersistenceTest, HTTPSTest, RequestBodyTest, SourceAddressTest, HTTPResponseTest, ExtendedReadTest, ExtendedReadTestChunked, TunnelTests) diff --git a/Misc/NEWS.d/next/Security/2020-02-12-14-17-39.bpo-39603.Gt3RSg.rst b/Misc/NEWS.d/next/Security/2020-02-12-14-17-39.bpo-39603.Gt3RSg.rst new file mode 100644 index 0000000000000..990affc3edd9d --- /dev/null +++ b/Misc/NEWS.d/next/Security/2020-02-12-14-17-39.bpo-39603.Gt3RSg.rst @@ -0,0 +1,2 @@ +Prevent http header injection by rejecting control characters in +http.client.putrequest(...). From webhook-mailer at python.org Fri Sep 4 04:48:04 2020 From: webhook-mailer at python.org (Dong-hee Na) Date: Fri, 04 Sep 2020 08:48:04 -0000 Subject: [Python-checkins] bpo-41700: Skip test if the locale is not supported (GH-22081) Message-ID: https://github.com/python/cpython/commit/54a66ade2067c373d31003ad260e1b7d14c81564 commit: 54a66ade2067c373d31003ad260e1b7d14c81564 branch: master author: Dong-hee Na committer: GitHub date: 2020-09-04T17:47:40+09:00 summary: bpo-41700: Skip test if the locale is not supported (GH-22081) files: M Lib/test/test_c_locale_coercion.py diff --git a/Lib/test/test_c_locale_coercion.py b/Lib/test/test_c_locale_coercion.py index 8340a9eb2ea3a..fcc85992345db 100644 --- a/Lib/test/test_c_locale_coercion.py +++ b/Lib/test/test_c_locale_coercion.py @@ -407,7 +407,10 @@ def test_PYTHONCOERCECLOCALE_set_to_one(self): # skip the test if the LC_CTYPE locale is C or coerced old_loc = locale.setlocale(locale.LC_CTYPE, None) self.addCleanup(locale.setlocale, locale.LC_CTYPE, old_loc) - loc = locale.setlocale(locale.LC_CTYPE, "") + try: + loc = locale.setlocale(locale.LC_CTYPE, "") + except locale.Error as e: + self.skipTest(str(e)) if loc == "C": self.skipTest("test requires LC_CTYPE locale different than C") if loc in TARGET_LOCALES : From webhook-mailer at python.org Fri Sep 4 05:55:50 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 04 Sep 2020 09:55:50 -0000 Subject: [Python-checkins] bpo-41700: Skip test if the locale is not supported (GH-22081) (GH-22086) Message-ID: https://github.com/python/cpython/commit/4bcff52447b472bc5c8698d4ab29c09df9e122b4 commit: 4bcff52447b472bc5c8698d4ab29c09df9e122b4 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-04T18:55:44+09:00 summary: bpo-41700: Skip test if the locale is not supported (GH-22081) (GH-22086) (cherry picked from commit 54a66ade2067c373d31003ad260e1b7d14c81564) Co-authored-by: Dong-hee Na Co-authored-by: Dong-hee Na files: M Lib/test/test_c_locale_coercion.py diff --git a/Lib/test/test_c_locale_coercion.py b/Lib/test/test_c_locale_coercion.py index fb599b0fcb7a5..418f9b6154b53 100644 --- a/Lib/test/test_c_locale_coercion.py +++ b/Lib/test/test_c_locale_coercion.py @@ -403,7 +403,10 @@ def test_PYTHONCOERCECLOCALE_set_to_one(self): # skip the test if the LC_CTYPE locale is C or coerced old_loc = locale.setlocale(locale.LC_CTYPE, None) self.addCleanup(locale.setlocale, locale.LC_CTYPE, old_loc) - loc = locale.setlocale(locale.LC_CTYPE, "") + try: + loc = locale.setlocale(locale.LC_CTYPE, "") + except locale.Error as e: + self.skipTest(str(e)) if loc == "C": self.skipTest("test requires LC_CTYPE locale different than C") if loc in TARGET_LOCALES : From webhook-mailer at python.org Fri Sep 4 08:51:20 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 04 Sep 2020 12:51:20 -0000 Subject: [Python-checkins] bpo-41713: _signal doesn't use multi-phase init (GH-22087) Message-ID: https://github.com/python/cpython/commit/4b8032e5a4994a7902076efa72fca1e2c85d8b7f commit: 4b8032e5a4994a7902076efa72fca1e2c85d8b7f branch: master author: Victor Stinner committer: GitHub date: 2020-09-04T14:51:05+02:00 summary: bpo-41713: _signal doesn't use multi-phase init (GH-22087) Partially revert commit 71d1bd9569c8a497e279f2fea6fe47cd70a87ea3: don't use multi-phase initialization (PEP 489) for the _signal extension module. files: D Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-07-20.bpo-1635741.7wSuCc.rst M Modules/signalmodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-07-20.bpo-1635741.7wSuCc.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-07-20.bpo-1635741.7wSuCc.rst deleted file mode 100644 index ff7cb352869a2..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-07-20.bpo-1635741.7wSuCc.rst +++ /dev/null @@ -1 +0,0 @@ -Port the :mod:`_signal` extension module to multi-phase initialization (:pep:`489`). diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c index 262f2b66a57e2..3440894b2159d 100644 --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -1649,26 +1649,32 @@ signal_exec(PyObject *m) return 0; } -static PyModuleDef_Slot signal_slots[] = { - {Py_mod_exec, signal_exec}, - {0, NULL} -}; static struct PyModuleDef signalmodule = { PyModuleDef_HEAD_INIT, "_signal", .m_doc = module_doc, - .m_size = 0, + .m_size = -1, .m_methods = signal_methods, - .m_slots = signal_slots }; + PyMODINIT_FUNC PyInit__signal(void) { - return PyModuleDef_Init(&signalmodule); + PyObject *mod = PyModule_Create(&signalmodule); + if (mod == NULL) { + return NULL; + } + + if (signal_exec(mod) < 0) { + Py_DECREF(mod); + return NULL; + } + return mod; } + static void finisignal(void) { From webhook-mailer at python.org Fri Sep 4 13:55:50 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Fri, 04 Sep 2020 17:55:50 -0000 Subject: [Python-checkins] bpo-41638: Improve ProgrammingError message for absent parameter. (GH-21999) Message-ID: https://github.com/python/cpython/commit/81715808716198471fbca0a3db42ac408468dbc5 commit: 81715808716198471fbca0a3db42ac408468dbc5 branch: master author: Serhiy Storchaka committer: GitHub date: 2020-09-04T20:55:41+03:00 summary: bpo-41638: Improve ProgrammingError message for absent parameter. (GH-21999) It contains now the name of the parameter instead of its index when parameters are supplied as a dict. files: A Misc/NEWS.d/next/Library/2020-08-29-16-45-12.bpo-41638.iZfW5N.rst M Modules/_sqlite/statement.c diff --git a/Misc/NEWS.d/next/Library/2020-08-29-16-45-12.bpo-41638.iZfW5N.rst b/Misc/NEWS.d/next/Library/2020-08-29-16-45-12.bpo-41638.iZfW5N.rst new file mode 100644 index 0000000000000..8ab7b5e9903dc --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-08-29-16-45-12.bpo-41638.iZfW5N.rst @@ -0,0 +1,3 @@ +:exc:`~sqlite3.ProgrammingError` message for absent parameter in :mod:`sqlite3` +contains now the name of the parameter instead of its index when parameters +are supplied as a dict. diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c index 9de8f9b67228f..26599b423eb8b 100644 --- a/Modules/_sqlite/statement.c +++ b/Modules/_sqlite/statement.c @@ -295,7 +295,7 @@ void pysqlite_statement_bind_parameters(pysqlite_Statement* self, PyObject* para Py_DECREF(binding_name_obj); if (!current_param) { if (!PyErr_Occurred() || PyErr_ExceptionMatches(PyExc_LookupError)) { - PyErr_Format(pysqlite_ProgrammingError, "You did not supply a value for binding %d.", i); + PyErr_Format(pysqlite_ProgrammingError, "You did not supply a value for binding parameter :%s.", binding_name); } return; } From webhook-mailer at python.org Fri Sep 4 14:19:38 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Fri, 04 Sep 2020 18:19:38 -0000 Subject: [Python-checkins] bpo-40486: Specify what happens if directory content change diring iteration (GH-22025) Message-ID: https://github.com/python/cpython/commit/306cfb3a37e1438f6ba9f0a9f3af3c00aae4ec64 commit: 306cfb3a37e1438f6ba9f0a9f3af3c00aae4ec64 branch: master author: Serhiy Storchaka committer: GitHub date: 2020-09-04T21:19:30+03:00 summary: bpo-40486: Specify what happens if directory content change diring iteration (GH-22025) files: M Doc/library/glob.rst M Doc/library/os.rst M Doc/library/pathlib.rst diff --git a/Doc/library/glob.rst b/Doc/library/glob.rst index 280e9f0826602..3fdba6937c1de 100644 --- a/Doc/library/glob.rst +++ b/Doc/library/glob.rst @@ -43,7 +43,9 @@ For example, ``'[?]'`` matches the character ``'?'``. (like :file:`/usr/src/Python-1.5/Makefile`) or relative (like :file:`../../Tools/\*/\*.gif`), and can contain shell-style wildcards. Broken symlinks are included in the results (as in the shell). Whether or not the - results are sorted depends on the file system. + results are sorted depends on the file system. If a file that satisfies + conditions is removed or added during the call of this function, whether + a path name for that file be included is unspecified. If *root_dir* is not ``None``, it should be a :term:`path-like object` specifying the root directory for searching. It has the same effect on diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 275b2d390e7cf..8c3bc5fb87d61 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1852,6 +1852,8 @@ features: Return a list containing the names of the entries in the directory given by *path*. The list is in arbitrary order, and does not include the special entries ``'.'`` and ``'..'`` even if they are present in the directory. + If a file is removed from or added to the directory during the call of + this function, whether a name for that file be included is unspecified. *path* may be a :term:`path-like object`. If *path* is of type ``bytes`` (directly or indirectly through the :class:`PathLike` interface), @@ -2257,7 +2259,9 @@ features: Return an iterator of :class:`os.DirEntry` objects corresponding to the entries in the directory given by *path*. The entries are yielded in arbitrary order, and the special entries ``'.'`` and ``'..'`` are not - included. + included. If a file is removed from or added to the directory after + creating the iterator, whether an entry for that file be included is + unspecified. Using :func:`scandir` instead of :func:`listdir` can significantly increase the performance of code that also needs file type or file @@ -3007,7 +3011,10 @@ features: *filenames* is a list of the names of the non-directory files in *dirpath*. Note that the names in the lists contain no path components. To get a full path (which begins with *top*) to a file or directory in *dirpath*, do - ``os.path.join(dirpath, name)``. + ``os.path.join(dirpath, name)``. Whether or not the lists are sorted + depends on the file system. If a file is removed from or added to the + *dirpath* directory during generating the lists, whether a name for that + file be included is unspecified. If optional argument *topdown* is ``True`` or not specified, the triple for a directory is generated before the triples for any of its subdirectories diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst index 04810f5204397..23486b625072f 100644 --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -890,6 +890,11 @@ call fails (for example because the path doesn't exist). PosixPath('docs/_static') PosixPath('docs/Makefile') + The children are yielded in arbitrary order, and the special entries + ``'.'`` and ``'..'`` are not included. If a file is removed from or added + to the directory after creating the iterator, whether an path object for + that file be included is unspecified. + .. method:: Path.lchmod(mode) Like :meth:`Path.chmod` but, if the path points to a symbolic link, the From webhook-mailer at python.org Fri Sep 4 16:33:25 2020 From: webhook-mailer at python.org (Stefan Krah) Date: Fri, 04 Sep 2020 20:33:25 -0000 Subject: [Python-checkins] bpo-41721: Add xlc options (GH-22096) Message-ID: https://github.com/python/cpython/commit/84a7917b4c9afec07575065cffa143b91fe98c14 commit: 84a7917b4c9afec07575065cffa143b91fe98c14 branch: master author: Stefan Krah committer: GitHub date: 2020-09-04T22:33:17+02:00 summary: bpo-41721: Add xlc options (GH-22096) files: M configure M configure.ac diff --git a/configure b/configure index 4c18ae7e364ca..ad74754e9a721 100755 --- a/configure +++ b/configure @@ -7592,11 +7592,14 @@ $as_echo "$MACOSX_DEPLOYMENT_TARGET" >&6; } ;; esac -# ICC needs -fp-model strict or floats behave badly case "$CC" in *icc*) + # ICC needs -fp-model strict or floats behave badly CFLAGS_NODIST="$CFLAGS_NODIST -fp-model strict" ;; +*xlc*) + CFLAGS_NODIST="$CFLAGS_NODIST -qalias=noansi -qmaxmem=-1" + ;; esac if test "$assertions" = 'true'; then diff --git a/configure.ac b/configure.ac index 3b40f39124dad..f0bc8c625844b 100644 --- a/configure.ac +++ b/configure.ac @@ -1993,11 +1993,14 @@ yes) ;; esac -# ICC needs -fp-model strict or floats behave badly case "$CC" in *icc*) + # ICC needs -fp-model strict or floats behave badly CFLAGS_NODIST="$CFLAGS_NODIST -fp-model strict" ;; +*xlc*) + CFLAGS_NODIST="$CFLAGS_NODIST -qalias=noansi -qmaxmem=-1" + ;; esac if test "$assertions" = 'true'; then From webhook-mailer at python.org Fri Sep 4 16:57:57 2020 From: webhook-mailer at python.org (Zackery Spytz) Date: Fri, 04 Sep 2020 20:57:57 -0000 Subject: [Python-checkins] bpo-38585: Remove references to defusedexpat (GH-22095) Message-ID: https://github.com/python/cpython/commit/51b84f8e96a441c498210f827c1297ee4973525f commit: 51b84f8e96a441c498210f827c1297ee4973525f branch: master author: Zackery Spytz committer: GitHub date: 2020-09-04T13:57:48-07:00 summary: bpo-38585: Remove references to defusedexpat (GH-22095) defusedexpat is not maintained. files: M Doc/library/xml.rst diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst index fb86b6f5564d7..1981cab7cd438 100644 --- a/Doc/library/xml.rst +++ b/Doc/library/xml.rst @@ -20,7 +20,7 @@ Python's interfaces for processing XML are grouped in the ``xml`` package. The XML modules are not secure against erroneous or maliciously constructed data. If you need to parse untrusted or unauthenticated data see the :ref:`xml-vulnerabilities` and - :ref:`defused-packages` sections. + :ref:`defusedxml-package` sections. It is important to note that modules in the :mod:`xml` package require that there be at least one SAX-compliant XML parser available. The Expat parser is @@ -113,9 +113,9 @@ decompression bomb The documentation for `defusedxml`_ on PyPI has further information about all known attack vectors with examples and references. -.. _defused-packages: +.. _defusedxml-package: -The :mod:`defusedxml` and :mod:`defusedexpat` Packages +The :mod:`defusedxml` Package ------------------------------------------------------ `defusedxml`_ is a pure Python package with modified subclasses of all stdlib @@ -124,16 +124,8 @@ package is recommended for any server code that parses untrusted XML data. The package also ships with example exploits and extended documentation on more XML exploits such as XPath injection. -`defusedexpat`_ provides a modified libexpat and a patched -:mod:`pyexpat` module that have countermeasures against entity expansion -DoS attacks. The :mod:`defusedexpat` module still allows a sane and configurable amount of entity -expansions. The modifications may be included in some future release of Python, -but will not be included in any bugfix releases of -Python because they break backward compatibility. - .. _defusedxml: https://pypi.org/project/defusedxml/ -.. _defusedexpat: https://pypi.org/project/defusedexpat/ .. _Billion Laughs: https://en.wikipedia.org/wiki/Billion_laughs .. _ZIP bomb: https://en.wikipedia.org/wiki/Zip_bomb .. _DTD: https://en.wikipedia.org/wiki/Document_type_definition From webhook-mailer at python.org Fri Sep 4 18:25:13 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 04 Sep 2020 22:25:13 -0000 Subject: [Python-checkins] bpo-38585: Remove references to defusedexpat (GH-22095) Message-ID: https://github.com/python/cpython/commit/1470c9189c8b099e0be94b4d89842f5345b776ec commit: 1470c9189c8b099e0be94b4d89842f5345b776ec branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-04T15:25:04-07:00 summary: bpo-38585: Remove references to defusedexpat (GH-22095) defusedexpat is not maintained. (cherry picked from commit 51b84f8e96a441c498210f827c1297ee4973525f) Co-authored-by: Zackery Spytz files: M Doc/library/xml.rst diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst index fb86b6f5564d7..1981cab7cd438 100644 --- a/Doc/library/xml.rst +++ b/Doc/library/xml.rst @@ -20,7 +20,7 @@ Python's interfaces for processing XML are grouped in the ``xml`` package. The XML modules are not secure against erroneous or maliciously constructed data. If you need to parse untrusted or unauthenticated data see the :ref:`xml-vulnerabilities` and - :ref:`defused-packages` sections. + :ref:`defusedxml-package` sections. It is important to note that modules in the :mod:`xml` package require that there be at least one SAX-compliant XML parser available. The Expat parser is @@ -113,9 +113,9 @@ decompression bomb The documentation for `defusedxml`_ on PyPI has further information about all known attack vectors with examples and references. -.. _defused-packages: +.. _defusedxml-package: -The :mod:`defusedxml` and :mod:`defusedexpat` Packages +The :mod:`defusedxml` Package ------------------------------------------------------ `defusedxml`_ is a pure Python package with modified subclasses of all stdlib @@ -124,16 +124,8 @@ package is recommended for any server code that parses untrusted XML data. The package also ships with example exploits and extended documentation on more XML exploits such as XPath injection. -`defusedexpat`_ provides a modified libexpat and a patched -:mod:`pyexpat` module that have countermeasures against entity expansion -DoS attacks. The :mod:`defusedexpat` module still allows a sane and configurable amount of entity -expansions. The modifications may be included in some future release of Python, -but will not be included in any bugfix releases of -Python because they break backward compatibility. - .. _defusedxml: https://pypi.org/project/defusedxml/ -.. _defusedexpat: https://pypi.org/project/defusedexpat/ .. _Billion Laughs: https://en.wikipedia.org/wiki/Billion_laughs .. _ZIP bomb: https://en.wikipedia.org/wiki/Zip_bomb .. _DTD: https://en.wikipedia.org/wiki/Document_type_definition From webhook-mailer at python.org Fri Sep 4 18:25:58 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 04 Sep 2020 22:25:58 -0000 Subject: [Python-checkins] [3.8] bpo-40486: Specify what happens if directory content change diring iteration (GH-22025) (GH-22094) Message-ID: https://github.com/python/cpython/commit/e52f5bc898c9a11fb1d57a42a1f9ec60b424d576 commit: e52f5bc898c9a11fb1d57a42a1f9ec60b424d576 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-04T15:25:51-07:00 summary: [3.8] bpo-40486: Specify what happens if directory content change diring iteration (GH-22025) (GH-22094) (cherry picked from commit 306cfb3a37e1438f6ba9f0a9f3af3c00aae4ec64) Co-authored-by: Serhiy Storchaka files: M Doc/library/glob.rst M Doc/library/os.rst M Doc/library/pathlib.rst diff --git a/Doc/library/glob.rst b/Doc/library/glob.rst index 92a8c4d1eb871..3c468ebf73769 100644 --- a/Doc/library/glob.rst +++ b/Doc/library/glob.rst @@ -43,7 +43,9 @@ For example, ``'[?]'`` matches the character ``'?'``. (like :file:`/usr/src/Python-1.5/Makefile`) or relative (like :file:`../../Tools/\*/\*.gif`), and can contain shell-style wildcards. Broken symlinks are included in the results (as in the shell). Whether or not the - results are sorted depends on the file system. + results are sorted depends on the file system. If a file that satisfies + conditions is removed or added during the call of this function, whether + a path name for that file be included is unspecified. .. index:: single: **; in glob-style wildcards diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 77bbf99886345..c8e316a7403c1 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1828,6 +1828,8 @@ features: Return a list containing the names of the entries in the directory given by *path*. The list is in arbitrary order, and does not include the special entries ``'.'`` and ``'..'`` even if they are present in the directory. + If a file is removed from or added to the directory during the call of + this function, whether a name for that file be included is unspecified. *path* may be a :term:`path-like object`. If *path* is of type ``bytes`` (directly or indirectly through the :class:`PathLike` interface), @@ -2233,7 +2235,9 @@ features: Return an iterator of :class:`os.DirEntry` objects corresponding to the entries in the directory given by *path*. The entries are yielded in arbitrary order, and the special entries ``'.'`` and ``'..'`` are not - included. + included. If a file is removed from or added to the directory after + creating the iterator, whether an entry for that file be included is + unspecified. Using :func:`scandir` instead of :func:`listdir` can significantly increase the performance of code that also needs file type or file @@ -2983,7 +2987,10 @@ features: *filenames* is a list of the names of the non-directory files in *dirpath*. Note that the names in the lists contain no path components. To get a full path (which begins with *top*) to a file or directory in *dirpath*, do - ``os.path.join(dirpath, name)``. + ``os.path.join(dirpath, name)``. Whether or not the lists are sorted + depends on the file system. If a file is removed from or added to the + *dirpath* directory during generating the lists, whether a name for that + file be included is unspecified. If optional argument *topdown* is ``True`` or not specified, the triple for a directory is generated before the triples for any of its subdirectories diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst index ebaedd835425c..c855a6e11dac1 100644 --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -849,6 +849,11 @@ call fails (for example because the path doesn't exist). PosixPath('docs/_static') PosixPath('docs/Makefile') + The children are yielded in arbitrary order, and the special entries + ``'.'`` and ``'..'`` are not included. If a file is removed from or added + to the directory after creating the iterator, whether an path object for + that file be included is unspecified. + .. method:: Path.lchmod(mode) Like :meth:`Path.chmod` but, if the path points to a symbolic link, the From webhook-mailer at python.org Fri Sep 4 18:26:14 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 04 Sep 2020 22:26:14 -0000 Subject: [Python-checkins] [3.8] closes bpo-41533: Fix a potential memory leak when allocating a stack (GH-21847) (GH-22015) Message-ID: https://github.com/python/cpython/commit/66e9c2aee4af846ab1b77faa8a46fe3a9373d943 commit: 66e9c2aee4af846ab1b77faa8a46fe3a9373d943 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-04T15:26:05-07:00 summary: [3.8] closes bpo-41533: Fix a potential memory leak when allocating a stack (GH-21847) (GH-22015) Free the stack allocated in va_build_stack if do_mkstack fails and the stack is not a small_stack (cherry picked from commit 75c80b0bda89debf312f075716b8c467d411f90e) Co-authored-by: Tony Solomonik files: A Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-41533.4pcVAc.rst M Python/modsupport.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-41533.4pcVAc.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-41533.4pcVAc.rst new file mode 100644 index 0000000000000..e166f0c0b621a --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-41533.4pcVAc.rst @@ -0,0 +1,2 @@ +Free the stack allocated in ``va_build_stack`` if ``do_mkstack`` fails and +the stack is not a ``small_stack``. diff --git a/Python/modsupport.c b/Python/modsupport.c index 7271af3ac5332..506281191c522 100644 --- a/Python/modsupport.c +++ b/Python/modsupport.c @@ -618,6 +618,9 @@ va_build_stack(PyObject **small_stack, Py_ssize_t small_stack_len, va_end(lva); if (res < 0) { + if (stack != small_stack) { + PyMem_Free(stack); + } return NULL; } From webhook-mailer at python.org Fri Sep 4 18:26:30 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 04 Sep 2020 22:26:30 -0000 Subject: [Python-checkins] [3.8] Fix error in argparse documentation example (GH-17399) (GH-21990) Message-ID: https://github.com/python/cpython/commit/e77547b90cb3b819113c722579a7fc070de0c2ca commit: e77547b90cb3b819113c722579a7fc070de0c2ca branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-04T15:26:21-07:00 summary: [3.8] Fix error in argparse documentation example (GH-17399) (GH-21990) Automerge-Triggered-By: @rhettinger (cherry picked from commit 8784d3300ec4ffc58bc0e9ab3cff9a24187dbe4c) Co-authored-by: SarahPythonista <4283226+SarahPythonista at users.noreply.github.com> files: M Doc/library/argparse.rst diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst index e1b28e32ed185..2ec7ea3ecb4e3 100644 --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -1123,8 +1123,8 @@ keyword argument to :meth:`~ArgumentParser.add_argument`:: >>> parser.parse_args(['--foo', 'BAR']) Namespace(foo='BAR') >>> parser.parse_args([]) - usage: argparse.py [-h] [--foo FOO] - argparse.py: error: option --foo is required + usage: [-h] --foo FOO + : error: the following arguments are required: --foo As the example shows, if an option is marked as ``required``, :meth:`~ArgumentParser.parse_args` will report an error if that option is not From webhook-mailer at python.org Fri Sep 4 18:30:25 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 04 Sep 2020 22:30:25 -0000 Subject: [Python-checkins] [3.8] Improve asyncio-dev 'Concurrency and Multithreading' docs (GH-20882) (GH-22010) Message-ID: https://github.com/python/cpython/commit/4ddb2d73ac661af516a252fe697ec05e726b0f20 commit: 4ddb2d73ac661af516a252fe697ec05e726b0f20 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-04T15:30:21-07:00 summary: [3.8] Improve asyncio-dev 'Concurrency and Multithreading' docs (GH-20882) (GH-22010) I added some information to the `Concurrency and Multithreading` section of the `Developing with asyncio` guide. This is all information that would have helped me when I started using asyncio. I incorrectly assumed that `loop.call_soon_threadsafe()` and `run_coroutine_threadsafe()` could be called from a thread in a process separate from the one that the event loop is running in. Explicitly stating that this will not work will probably help some people starting out with asyncio in the future. I also added references to some other functions that can be used for inter-process communication without blocking the event loop. The section already mentions running blocking code in a ThreadPoolExecutor, but I think listing these other options in this section will also be helpful. (cherry picked from commit c68c5af2dc5ada8875a662f2beaac6234eae2a5a) Co-authored-by: Roger Iyengar files: M Doc/library/asyncio-dev.rst diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst index 0d302ea49b0de..d7ed96282fee1 100644 --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -107,6 +107,16 @@ The :meth:`loop.run_in_executor` method can be used with a blocking code in a different OS thread without blocking the OS thread that the event loop runs in. +There is currently no way to schedule coroutines or callbacks directly +from a different process (such as one started with +:mod:`multiprocessing`). The :ref:`Event Loop Methods ` +section lists APIs that can read from pipes and watch file descriptors +without blocking the event loop. In addition, asyncio's +:ref:`Subprocess ` APIs provide a way to start a +process and communicate with it from the event loop. Lastly, the +aforementioned :meth:`loop.run_in_executor` method can also be used +with a :class:`concurrent.futures.ProcessPoolExecutor` to execute +code in a different process. .. _asyncio-handle-blocking: From webhook-mailer at python.org Fri Sep 4 19:46:01 2020 From: webhook-mailer at python.org (Steve Dower) Date: Fri, 04 Sep 2020 23:46:01 -0000 Subject: [Python-checkins] bpo-41627: Distinguish 32 and 64-bit user site packages on Windows (GH-22098) Message-ID: https://github.com/python/cpython/commit/dd18001c308fb3bb65006c91d95f6639583a3420 commit: dd18001c308fb3bb65006c91d95f6639583a3420 branch: master author: Steve Dower committer: GitHub date: 2020-09-05T00:45:54+01:00 summary: bpo-41627: Distinguish 32 and 64-bit user site packages on Windows (GH-22098) Also fixes the error message returned when sysconfig fails to interpolate a variable correctly. files: A Misc/NEWS.d/next/Windows/2020-09-04-21-35-28.bpo-41627.sx2KN1.rst M Lib/site.py M Lib/sysconfig.py diff --git a/Lib/site.py b/Lib/site.py index 8979365cafc37..4c095774729c5 100644 --- a/Lib/site.py +++ b/Lib/site.py @@ -274,7 +274,8 @@ def _get_path(userbase): version = sys.version_info if os.name == 'nt': - return f'{userbase}\\Python{version[0]}{version[1]}\\site-packages' + ver_nodot = sys.winver.replace('.', '') + return f'{userbase}\\Python{ver_nodot}\\site-packages' if sys.platform == 'darwin' and sys._framework: return f'{userbase}/lib/python/site-packages' diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py index bf04ac541e6b0..6c87b06634c45 100644 --- a/Lib/sysconfig.py +++ b/Lib/sysconfig.py @@ -53,12 +53,12 @@ }, # NOTE: When modifying "purelib" scheme, update site._get_path() too. 'nt_user': { - 'stdlib': '{userbase}/Python{py_version_nodot}', - 'platstdlib': '{userbase}/Python{py_version_nodot}', - 'purelib': '{userbase}/Python{py_version_nodot}/site-packages', - 'platlib': '{userbase}/Python{py_version_nodot}/site-packages', - 'include': '{userbase}/Python{py_version_nodot}/Include', - 'scripts': '{userbase}/Python{py_version_nodot}/Scripts', + 'stdlib': '{userbase}/Python{py_version_nodot_plat}', + 'platstdlib': '{userbase}/Python{py_version_nodot_plat}', + 'purelib': '{userbase}/Python{py_version_nodot_plat}/site-packages', + 'platlib': '{userbase}/Python{py_version_nodot_plat}/site-packages', + 'include': '{userbase}/Python{py_version_nodot_plat}/Include', + 'scripts': '{userbase}/Python{py_version_nodot_plat}/Scripts', 'data': '{userbase}', }, 'posix_user': { @@ -149,10 +149,10 @@ def is_python_build(check_home=False): def _subst_vars(s, local_vars): try: return s.format(**local_vars) - except KeyError: + except KeyError as var: try: return s.format(**os.environ) - except KeyError as var: + except KeyError: raise AttributeError('{%s}' % var) from None def _extend_dict(target_dict, other_dict): @@ -431,6 +431,7 @@ def _init_non_posix(vars): vars['EXE'] = '.exe' vars['VERSION'] = _PY_VERSION_SHORT_NO_DOT vars['BINDIR'] = os.path.dirname(_safe_realpath(sys.executable)) + vars['TZPATH'] = '' # # public APIs @@ -543,10 +544,13 @@ def get_config_vars(*args): except AttributeError: # sys.abiflags may not be defined on all platforms. _CONFIG_VARS['abiflags'] = '' + try: + _CONFIG_VARS['py_version_nodot_plat'] = sys.winver.replace('.', '') + except AttributeError: + _CONFIG_VARS['py_version_nodot_plat'] = '' if os.name == 'nt': _init_non_posix(_CONFIG_VARS) - _CONFIG_VARS['TZPATH'] = '' if os.name == 'posix': _init_posix(_CONFIG_VARS) # For backward compatibility, see issue19555 diff --git a/Misc/NEWS.d/next/Windows/2020-09-04-21-35-28.bpo-41627.sx2KN1.rst b/Misc/NEWS.d/next/Windows/2020-09-04-21-35-28.bpo-41627.sx2KN1.rst new file mode 100644 index 0000000000000..043bd5e9341c3 --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2020-09-04-21-35-28.bpo-41627.sx2KN1.rst @@ -0,0 +1,2 @@ +The user site directory for 32-bit now includes a ``-32`` suffix to +distinguish it from the 64-bit interpreter's directory. From webhook-mailer at python.org Sat Sep 5 05:05:51 2020 From: webhook-mailer at python.org (Larry Hastings) Date: Sat, 05 Sep 2020 09:05:51 -0000 Subject: [Python-checkins] Blurb release and pydoc topics for 3.5.10. Message-ID: https://github.com/python/cpython/commit/f2f17bd54c0121d4a6f6fd88c64c963e9b3b0ba1 commit: f2f17bd54c0121d4a6f6fd88c64c963e9b3b0ba1 branch: 3.5 author: Larry Hastings committer: Larry Hastings date: 2020-09-05T00:21:11-07:00 summary: Blurb release and pydoc topics for 3.5.10. files: A Misc/NEWS.d/3.5.10.rst D Misc/NEWS.d/next/Security/2020-02-12-14-17-39.bpo-39603.Gt3RSg.rst M Lib/pydoc_data/topics.py diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py index 3ddb0cf98767d..40a52ed1c1ae4 100644 --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Wed Aug 19 12:50:32 2020 +# Autogenerated by Sphinx on Sat Sep 5 00:19:52 2020 topics = {'assert': 'The "assert" statement\n' '**********************\n' '\n' diff --git a/Misc/NEWS.d/3.5.10.rst b/Misc/NEWS.d/3.5.10.rst new file mode 100644 index 0000000000000..fda8bb44137fa --- /dev/null +++ b/Misc/NEWS.d/3.5.10.rst @@ -0,0 +1,8 @@ +.. bpo: 39603 +.. date: 2020-02-12-14-17-39 +.. nonce: Gt3RSg +.. release date: 2020-09-05 +.. section: Security + +Prevent http header injection by rejecting control characters in +http.client.putrequest(...). diff --git a/Misc/NEWS.d/next/Security/2020-02-12-14-17-39.bpo-39603.Gt3RSg.rst b/Misc/NEWS.d/next/Security/2020-02-12-14-17-39.bpo-39603.Gt3RSg.rst deleted file mode 100644 index 990affc3edd9d..0000000000000 --- a/Misc/NEWS.d/next/Security/2020-02-12-14-17-39.bpo-39603.Gt3RSg.rst +++ /dev/null @@ -1,2 +0,0 @@ -Prevent http header injection by rejecting control characters in -http.client.putrequest(...). From webhook-mailer at python.org Sat Sep 5 12:08:02 2020 From: webhook-mailer at python.org (Pablo Galindo) Date: Sat, 05 Sep 2020 16:08:02 -0000 Subject: [Python-checkins] bpo-41654: Explicitly cast PyExc_MemoryError to PyTypeObject to avoid warning (GH-22102) Message-ID: https://github.com/python/cpython/commit/6ae61959ec51a6b3dddc8e665ce6a7b8aeb26c04 commit: 6ae61959ec51a6b3dddc8e665ce6a7b8aeb26c04 branch: 3.8 author: Pablo Galindo committer: GitHub date: 2020-09-05T17:07:54+01:00 summary: bpo-41654: Explicitly cast PyExc_MemoryError to PyTypeObject to avoid warning (GH-22102) files: M Objects/exceptions.c diff --git a/Objects/exceptions.c b/Objects/exceptions.c index 966983810cd1e..8bcf76ff860a2 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -2295,8 +2295,9 @@ MemoryError_dealloc(PyBaseExceptionObject *self) { BaseException_clear(self); - if (Py_TYPE(self) != PyExc_MemoryError) { - return Py_TYPE(self)->tp_free((PyObject *)self); + if (Py_TYPE(self) != (PyTypeObject *)PyExc_MemoryError) { + Py_TYPE(self)->tp_free((PyObject *)self); + return; } _PyObject_GC_UNTRACK(self); From webhook-mailer at python.org Sat Sep 5 16:43:36 2020 From: webhook-mailer at python.org (Erlend Egeberg Aasland) Date: Sat, 05 Sep 2020 20:43:36 -0000 Subject: [Python-checkins] bpo-40318: Migrate to SQLite3 trace v2 API (GH-19581) Message-ID: https://github.com/python/cpython/commit/7f331c898a36d937c66ba93f86d2a445c96d382e commit: 7f331c898a36d937c66ba93f86d2a445c96d382e branch: master author: Erlend Egeberg Aasland committer: GitHub date: 2020-09-05T21:43:31+01:00 summary: bpo-40318: Migrate to SQLite3 trace v2 API (GH-19581) Ref. https://sqlite.org/c3ref/trace_v2.html Co-authored-by: Pablo Galindo files: A Misc/NEWS.d/next/Library/2020-04-18-14-16-02.bpo-40318.K2UdRx.rst M Modules/_sqlite/connection.c diff --git a/Misc/NEWS.d/next/Library/2020-04-18-14-16-02.bpo-40318.K2UdRx.rst b/Misc/NEWS.d/next/Library/2020-04-18-14-16-02.bpo-40318.K2UdRx.rst new file mode 100644 index 0000000000000..3d5fcfb74a0fe --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-04-18-14-16-02.bpo-40318.K2UdRx.rst @@ -0,0 +1 @@ +Use SQLite3 trace v2 API, if it is available. diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index 958be7d869794..1bf9710763a5a 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -43,6 +43,10 @@ #define HAVE_BACKUP_API #endif +#if SQLITE_VERSION_NUMBER >= 3014000 +#define HAVE_TRACE_V2 +#endif + _Py_IDENTIFIER(cursor); static const char * const begin_statements[] = { @@ -962,13 +966,29 @@ static int _progress_handler(void* user_arg) return rc; } +#ifdef HAVE_TRACE_V2 +/* + * From https://sqlite.org/c3ref/trace_v2.html: + * The integer return value from the callback is currently ignored, though this + * may change in future releases. Callback implementations should return zero + * to ensure future compatibility. + */ +static int _trace_callback(unsigned int type, void* user_arg, void* prepared_statement, void* statement_string) +#else static void _trace_callback(void* user_arg, const char* statement_string) +#endif { PyObject *py_statement = NULL; PyObject *ret = NULL; PyGILState_STATE gilstate; +#ifdef HAVE_TRACE_V2 + if (type != SQLITE_TRACE_STMT) { + return 0; + } +#endif + gilstate = PyGILState_Ensure(); py_statement = PyUnicode_DecodeUTF8(statement_string, strlen(statement_string), "replace"); @@ -988,6 +1008,9 @@ static void _trace_callback(void* user_arg, const char* statement_string) } PyGILState_Release(gilstate); +#ifdef HAVE_TRACE_V2 + return 0; +#endif } static PyObject* pysqlite_connection_set_authorizer(pysqlite_Connection* self, PyObject* args, PyObject* kwargs) @@ -1046,6 +1069,11 @@ static PyObject* pysqlite_connection_set_progress_handler(pysqlite_Connection* s Py_RETURN_NONE; } +/* + * Ref. + * - https://sqlite.org/c3ref/c_trace.html + * - https://sqlite.org/c3ref/trace_v2.html + */ static PyObject* pysqlite_connection_set_trace_callback(pysqlite_Connection* self, PyObject* args, PyObject* kwargs) { PyObject* trace_callback; @@ -1063,10 +1091,18 @@ static PyObject* pysqlite_connection_set_trace_callback(pysqlite_Connection* sel if (trace_callback == Py_None) { /* None clears the trace callback previously set */ +#ifdef HAVE_TRACE_V2 + sqlite3_trace_v2(self->db, SQLITE_TRACE_STMT, 0, 0); +#else sqlite3_trace(self->db, 0, (void*)0); +#endif Py_XSETREF(self->function_pinboard_trace_callback, NULL); } else { +#ifdef HAVE_TRACE_V2 + sqlite3_trace_v2(self->db, SQLITE_TRACE_STMT, _trace_callback, trace_callback); +#else sqlite3_trace(self->db, _trace_callback, trace_callback); +#endif Py_INCREF(trace_callback); Py_XSETREF(self->function_pinboard_trace_callback, trace_callback); } From webhook-mailer at python.org Sat Sep 5 16:53:56 2020 From: webhook-mailer at python.org (johnthagen) Date: Sat, 05 Sep 2020 20:53:56 -0000 Subject: [Python-checkins] Fix documented Python version for venv --upgrade-deps (GH-22113) Message-ID: https://github.com/python/cpython/commit/1264d0465ad77019a43ff8c1f8d329b17f049f80 commit: 1264d0465ad77019a43ff8c1f8d329b17f049f80 branch: master author: johnthagen committer: GitHub date: 2020-09-05T13:53:47-07:00 summary: Fix documented Python version for venv --upgrade-deps (GH-22113) Fixes incorrect Python version added for `venv` `--upgrade-deps` in #13100. This feature was added in Python 3.9 not 3.8. Relates to: - - https://github.com/python/cpython/commit/1cba1c9abadf76f458ecf883a48515aa3b534dbd Automerge-Triggered-By: @vsajip files: M Doc/using/venv-create.inc diff --git a/Doc/using/venv-create.inc b/Doc/using/venv-create.inc index 8f850a74d413c..ddb36f94667d9 100644 --- a/Doc/using/venv-create.inc +++ b/Doc/using/venv-create.inc @@ -67,7 +67,7 @@ The command, if run with ``-h``, will show the available options:: Once an environment has been created, you may wish to activate it, e.g. by sourcing an activate script in its bin directory. -.. versionchanged:: 3.8 +.. versionchanged:: 3.9 Add ``--upgrade-deps`` option to upgrade pip + setuptools to the latest on PyPI .. versionchanged:: 3.4 From webhook-mailer at python.org Sat Sep 5 19:40:35 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Sat, 05 Sep 2020 23:40:35 -0000 Subject: [Python-checkins] [doc] Fix padding in some typing definitions (GH-22114) Message-ID: https://github.com/python/cpython/commit/2623868ede4ef3c848fc83a9b1e19e0d031dee1d commit: 2623868ede4ef3c848fc83a9b1e19e0d031dee1d branch: master author: Andre Delfino committer: GitHub date: 2020-09-05T16:40:25-07:00 summary: [doc] Fix padding in some typing definitions (GH-22114) Automerge-Triggered-By: @gvanrossum files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 9f98f8ce3f642..6d6b76c1d0895 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1195,7 +1195,7 @@ Corresponding to collections in :mod:`collections.abc` .. class:: AbstractSet(Sized, Collection[T_co]) - A generic version of :class:`collections.abc.Set`. + A generic version of :class:`collections.abc.Set`. .. deprecated:: 3.9 :class:`collections.abc.Set` now supports ``[]``. See :pep:`585`. @@ -1224,7 +1224,7 @@ Corresponding to collections in :mod:`collections.abc` .. class:: Container(Generic[T_co]) - A generic version of :class:`collections.abc.Container`. + A generic version of :class:`collections.abc.Container`. .. deprecated:: 3.9 :class:`collections.abc.Container` now supports ``[]``. See :pep:`585`. @@ -1245,11 +1245,11 @@ Corresponding to collections in :mod:`collections.abc` .. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) - A generic version of :class:`collections.abc.Mapping`. - This type can be used as follows:: + A generic version of :class:`collections.abc.Mapping`. + This type can be used as follows:: - def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: - return word_list[word] + def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: + return word_list[word] .. deprecated:: 3.9 :class:`collections.abc.Mapping` now supports ``[]``. See :pep:`585`. @@ -1263,7 +1263,7 @@ Corresponding to collections in :mod:`collections.abc` .. class:: MutableMapping(Mapping[KT, VT]) - A generic version of :class:`collections.abc.MutableMapping`. + A generic version of :class:`collections.abc.MutableMapping`. .. deprecated:: 3.9 :class:`collections.abc.MutableMapping` now supports ``[]``. See :pep:`585`. @@ -1277,14 +1277,14 @@ Corresponding to collections in :mod:`collections.abc` .. class:: MutableSet(AbstractSet[T]) - A generic version of :class:`collections.abc.MutableSet`. + A generic version of :class:`collections.abc.MutableSet`. .. deprecated:: 3.9 :class:`collections.abc.MutableSet` now supports ``[]``. See :pep:`585`. .. class:: Sequence(Reversible[T_co], Collection[T_co]) - A generic version of :class:`collections.abc.Sequence`. + A generic version of :class:`collections.abc.Sequence`. .. deprecated:: 3.9 :class:`collections.abc.Sequence` now supports ``[]``. See :pep:`585`. @@ -1301,14 +1301,14 @@ Corresponding to other types in :mod:`collections.abc` .. class:: Iterable(Generic[T_co]) - A generic version of :class:`collections.abc.Iterable`. + A generic version of :class:`collections.abc.Iterable`. .. deprecated:: 3.9 :class:`collections.abc.Iterable` now supports ``[]``. See :pep:`585`. .. class:: Iterator(Iterable[T_co]) - A generic version of :class:`collections.abc.Iterator`. + A generic version of :class:`collections.abc.Iterator`. .. deprecated:: 3.9 :class:`collections.abc.Iterator` now supports ``[]``. See :pep:`585`. @@ -1353,7 +1353,7 @@ Corresponding to other types in :mod:`collections.abc` .. class:: Reversible(Iterable[T_co]) - A generic version of :class:`collections.abc.Reversible`. + A generic version of :class:`collections.abc.Reversible`. .. deprecated:: 3.9 :class:`collections.abc.Reversible` now supports ``[]``. See :pep:`585`. From webhook-mailer at python.org Sat Sep 5 23:39:37 2020 From: webhook-mailer at python.org (Zackery Spytz) Date: Sun, 06 Sep 2020 03:39:37 -0000 Subject: [Python-checkins] closes bpo-41723: Fix an error in the py_compile documentation. (GH-22110) Message-ID: https://github.com/python/cpython/commit/5371a464ce88ffc88f3bb95cfd86f355b7d02953 commit: 5371a464ce88ffc88f3bb95cfd86f355b7d02953 branch: master author: Zackery Spytz committer: GitHub date: 2020-09-05T22:39:23-05:00 summary: closes bpo-41723: Fix an error in the py_compile documentation. (GH-22110) files: M Doc/library/py_compile.rst diff --git a/Doc/library/py_compile.rst b/Doc/library/py_compile.rst index 9b5c8ee645a38..4fba4cba4d356 100644 --- a/Doc/library/py_compile.rst +++ b/Doc/library/py_compile.rst @@ -35,7 +35,7 @@ byte-code cache files in the directory containing the source code. in ``.pyc``. For example, if *file* is ``/foo/bar/baz.py`` *cfile* will default to ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2. If *dfile* is - specified, it is used as the name of the source file in error messages when + specified, it is used as the name of the source file in error messages instead of *file*. If *doraise* is true, a :exc:`PyCompileError` is raised when an error is encountered while compiling *file*. If *doraise* is false (the default), an error string is written to ``sys.stderr``, but no exception From webhook-mailer at python.org Sat Sep 5 23:47:22 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 06 Sep 2020 03:47:22 -0000 Subject: [Python-checkins] closes bpo-41723: Fix an error in the py_compile documentation. (GH-22110) Message-ID: https://github.com/python/cpython/commit/4e581d64b8aff3e2eda99b12f080c877bb78dfca commit: 4e581d64b8aff3e2eda99b12f080c877bb78dfca branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-05T20:47:17-07:00 summary: closes bpo-41723: Fix an error in the py_compile documentation. (GH-22110) (cherry picked from commit 5371a464ce88ffc88f3bb95cfd86f355b7d02953) Co-authored-by: Zackery Spytz files: M Doc/library/py_compile.rst diff --git a/Doc/library/py_compile.rst b/Doc/library/py_compile.rst index a12a5bb0b1aa2..cac6dcb63e7f3 100644 --- a/Doc/library/py_compile.rst +++ b/Doc/library/py_compile.rst @@ -35,7 +35,7 @@ byte-code cache files in the directory containing the source code. in ``.pyc``. For example, if *file* is ``/foo/bar/baz.py`` *cfile* will default to ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2. If *dfile* is - specified, it is used as the name of the source file in error messages when + specified, it is used as the name of the source file in error messages instead of *file*. If *doraise* is true, a :exc:`PyCompileError` is raised when an error is encountered while compiling *file*. If *doraise* is false (the default), an error string is written to ``sys.stderr``, but no exception From webhook-mailer at python.org Thu Sep 3 06:00:22 2020 From: webhook-mailer at python.org (Mario =?utf-8?q?=C5=A0a=C5=A1ko?=) Date: Thu, 03 Sep 2020 10:00:22 -0000 Subject: [Python-checkins] (no subject) Message-ID: To: python-checkins at python.org Subject: [doc] Fix a typo in the graphlib docs (#22030) Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 https://github.com/python/cpython/commit/851606007665dc30089e5e1953208c5428e4= 55b1 commit: 851606007665dc30089e5e1953208c5428e455b1 branch: master author: Mario =C5=A0a=C5=A1ko committer: GitHub date: 2020-09-03T11:00:10+01:00 summary: [doc] Fix a typo in the graphlib docs (#22030) files: M Doc/library/graphlib.rst diff --git a/Doc/library/graphlib.rst b/Doc/library/graphlib.rst index 820615e723015..0faca2186b268 100644 --- a/Doc/library/graphlib.rst +++ b/Doc/library/graphlib.rst @@ -121,7 +121,7 @@ if ts.is_active(): ... =20 - if possible to simply do:: + it is possible to simply do:: =20 if ts: ... From webhook-mailer at python.org Sat Sep 5 15:10:10 2020 From: webhook-mailer at python.org (Jakub =?utf-8?q?Kul=C3=ADk?=) Date: Sat, 05 Sep 2020 19:10:10 -0000 Subject: [Python-checkins] (no subject) Message-ID: To: python-checkins at python.org Subject: bpo-41687: Fix sendfile implementation to work with Solaris (#22040) Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 https://github.com/python/cpython/commit/8c0be6fd9101746235b63ddfb84106d1e9ca= 286b commit: 8c0be6fd9101746235b63ddfb84106d1e9ca286b branch: master author: Jakub Kul=C3=ADk committer: GitHub date: 2020-09-05T12:10:01-07:00 summary: bpo-41687: Fix sendfile implementation to work with Solaris (#22040) files: A Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rst M Lib/test/test_asyncio/test_sendfile.py M Modules/posixmodule.c diff --git a/Lib/test/test_asyncio/test_sendfile.py b/Lib/test/test_asyncio/t= est_sendfile.py index a30d9b9b4d9a0..01c698653ec67 100644 --- a/Lib/test/test_asyncio/test_sendfile.py +++ b/Lib/test/test_asyncio/test_sendfile.py @@ -446,6 +446,12 @@ def test_sendfile_ssl_close_peer_after_receiving(self): self.assertEqual(srv_proto.data, self.DATA) self.assertEqual(self.file.tell(), len(self.DATA)) =20 + # On Solaris, lowering SO_RCVBUF on a TCP connection after it has been + # established has no effect. Due to its age, this bug affects both Oracle + # Solaris as well as all other OpenSolaris forks (unless they fixed it + # themselves). + @unittest.skipIf(sys.platform.startswith('sunos'), + "Doesn't work on Solaris") def test_sendfile_close_peer_in_the_middle_of_receiving(self): srv_proto, cli_proto =3D self.prepare_sendfile(close_after=3D1024) with self.assertRaises(ConnectionError): diff --git a/Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rs= t b/Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rst new file mode 100644 index 0000000000000..284f500735701 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rst @@ -0,0 +1 @@ +Fix implementation of sendfile to be compatible with Solaris. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index a6a4b9f012f00..00ba7580302bb 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -9518,6 +9518,25 @@ os_sendfile_impl(PyObject *module, int out_fd, int in_= fd, PyObject *offobj, if (!Py_off_t_converter(offobj, &offset)) return NULL; =20 +#if defined(__sun) && defined(__SVR4) + // On Solaris, sendfile raises EINVAL rather than returning 0 + // when the offset is equal or bigger than the in_fd size. + int res; + struct stat st; + + do { + Py_BEGIN_ALLOW_THREADS + res =3D fstat(in_fd, &st); + Py_END_ALLOW_THREADS + } while (res !=3D 0 && errno =3D=3D EINTR && !(async_err =3D PyErr_Check= Signals())); + if (ret < 0) + return (!async_err) ? posix_error() : NULL; + + if (offset >=3D st.st_size) { + return Py_BuildValue("i", 0); + } +#endif + do { Py_BEGIN_ALLOW_THREADS ret =3D sendfile(out_fd, in_fd, &offset, count); From webhook-mailer at python.org Sun Sep 6 06:10:22 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Sun, 06 Sep 2020 10:10:22 -0000 Subject: [Python-checkins] bpo-1635741: Port _sha1, _sha512, _md5 to multiphase init (GH-21818) Message-ID: https://github.com/python/cpython/commit/63f102fe079ecb5cb7b921a1cf8bce4077a9d7e2 commit: 63f102fe079ecb5cb7b921a1cf8bce4077a9d7e2 branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-06T12:09:51+02:00 summary: bpo-1635741: Port _sha1, _sha512, _md5 to multiphase init (GH-21818) Port the _sha1, _sha512, and _md5 extension modules to multi-phase initialization API (PEP 489). files: A Misc/NEWS.d/next/Core and Builtins/2020-08-10-16-11-32.bpo-1635741.O0d3ym.rst M Modules/clinic/md5module.c.h M Modules/clinic/sha1module.c.h M Modules/clinic/sha512module.c.h M Modules/md5module.c M Modules/sha1module.c M Modules/sha512module.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-10-16-11-32.bpo-1635741.O0d3ym.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-10-16-11-32.bpo-1635741.O0d3ym.rst new file mode 100644 index 0000000000000..12af3d01ed8ef --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-10-16-11-32.bpo-1635741.O0d3ym.rst @@ -0,0 +1,2 @@ +Port the :mod:`_sha1`, :mod:`_sha512`, and :mod:`_md5` extension modules +to multi-phase initialization API (:pep:`489`). diff --git a/Modules/clinic/md5module.c.h b/Modules/clinic/md5module.c.h index c109f9efec6b6..4762f2800d4b8 100644 --- a/Modules/clinic/md5module.c.h +++ b/Modules/clinic/md5module.c.h @@ -9,15 +9,26 @@ PyDoc_STRVAR(MD5Type_copy__doc__, "Return a copy of the hash object."); #define MD5TYPE_COPY_METHODDEF \ - {"copy", (PyCFunction)MD5Type_copy, METH_NOARGS, MD5Type_copy__doc__}, + {"copy", (PyCFunction)(void(*)(void))MD5Type_copy, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, MD5Type_copy__doc__}, static PyObject * -MD5Type_copy_impl(MD5object *self); +MD5Type_copy_impl(MD5object *self, PyTypeObject *cls); static PyObject * -MD5Type_copy(MD5object *self, PyObject *Py_UNUSED(ignored)) +MD5Type_copy(MD5object *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return MD5Type_copy_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":copy", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = MD5Type_copy_impl(self, cls); + +exit: + return return_value; } PyDoc_STRVAR(MD5Type_digest__doc__, @@ -115,4 +126,4 @@ _md5_md5(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kw exit: return return_value; } -/*[clinic end generated code: output=dbe3abc60086f3ef input=a9049054013a1b77]*/ +/*[clinic end generated code: output=53ff7f22dbaaea36 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/sha1module.c.h b/Modules/clinic/sha1module.c.h index fc37b1ab880ff..3a3ab58c1233c 100644 --- a/Modules/clinic/sha1module.c.h +++ b/Modules/clinic/sha1module.c.h @@ -9,15 +9,26 @@ PyDoc_STRVAR(SHA1Type_copy__doc__, "Return a copy of the hash object."); #define SHA1TYPE_COPY_METHODDEF \ - {"copy", (PyCFunction)SHA1Type_copy, METH_NOARGS, SHA1Type_copy__doc__}, + {"copy", (PyCFunction)(void(*)(void))SHA1Type_copy, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, SHA1Type_copy__doc__}, static PyObject * -SHA1Type_copy_impl(SHA1object *self); +SHA1Type_copy_impl(SHA1object *self, PyTypeObject *cls); static PyObject * -SHA1Type_copy(SHA1object *self, PyObject *Py_UNUSED(ignored)) +SHA1Type_copy(SHA1object *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return SHA1Type_copy_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":copy", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = SHA1Type_copy_impl(self, cls); + +exit: + return return_value; } PyDoc_STRVAR(SHA1Type_digest__doc__, @@ -115,4 +126,4 @@ _sha1_sha1(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject * exit: return return_value; } -/*[clinic end generated code: output=3ddd637ae17e14b3 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=abf1ab2545cea5a2 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/sha512module.c.h b/Modules/clinic/sha512module.c.h index b8185b62bb669..f1192d74f9a1a 100644 --- a/Modules/clinic/sha512module.c.h +++ b/Modules/clinic/sha512module.c.h @@ -9,15 +9,26 @@ PyDoc_STRVAR(SHA512Type_copy__doc__, "Return a copy of the hash object."); #define SHA512TYPE_COPY_METHODDEF \ - {"copy", (PyCFunction)SHA512Type_copy, METH_NOARGS, SHA512Type_copy__doc__}, + {"copy", (PyCFunction)(void(*)(void))SHA512Type_copy, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, SHA512Type_copy__doc__}, static PyObject * -SHA512Type_copy_impl(SHAobject *self); +SHA512Type_copy_impl(SHAobject *self, PyTypeObject *cls); static PyObject * -SHA512Type_copy(SHAobject *self, PyObject *Py_UNUSED(ignored)) +SHA512Type_copy(SHAobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return SHA512Type_copy_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":copy", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = SHA512Type_copy_impl(self, cls); + +exit: + return return_value; } PyDoc_STRVAR(SHA512Type_digest__doc__, @@ -166,4 +177,4 @@ _sha512_sha384(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObje exit: return return_value; } -/*[clinic end generated code: output=bbfa72d8703c82b5 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=9ff9f11937fabf35 input=a9049054013a1b77]*/ diff --git a/Modules/md5module.c b/Modules/md5module.c index e4d9db40f22df..5cd4e94510132 100644 --- a/Modules/md5module.c +++ b/Modules/md5module.c @@ -318,22 +318,32 @@ md5_done(struct md5_state *md5, unsigned char *out) * ------------------------------------------------------------------------ */ -static PyTypeObject MD5type; +typedef struct { + PyTypeObject* md5_type; +} MD5State; +static inline MD5State* +md5_get_state(PyObject *module) +{ + void *state = PyModule_GetState(module); + assert(state != NULL); + return (MD5State *)state; +} static MD5object * -newMD5object(void) +newMD5object(MD5State * st) { - return (MD5object *)PyObject_New(MD5object, &MD5type); + return (MD5object *)PyObject_New(MD5object, st->md5_type); } - /* Internal methods for a hash object */ static void MD5_dealloc(PyObject *ptr) { + PyTypeObject *tp = Py_TYPE(ptr); PyObject_Del(ptr); + Py_DECREF(tp); } @@ -342,16 +352,19 @@ MD5_dealloc(PyObject *ptr) /*[clinic input] MD5Type.copy + cls: defining_class + Return a copy of the hash object. [clinic start generated code]*/ static PyObject * -MD5Type_copy_impl(MD5object *self) -/*[clinic end generated code: output=596eb36852f02071 input=2c09e6d2493f3079]*/ +MD5Type_copy_impl(MD5object *self, PyTypeObject *cls) +/*[clinic end generated code: output=bf055e08244bf5ee input=d89087dcfb2a8620]*/ { - MD5object *newobj; + MD5State *st = PyType_GetModuleState(cls); - if ((newobj = newMD5object())==NULL) + MD5object *newobj; + if ((newobj = newMD5object(st))==NULL) return NULL; newobj->hash_state = self->hash_state; @@ -445,7 +458,6 @@ md5_get_digest_size(PyObject *self, void *closure) return PyLong_FromLong(MD5_DIGESTSIZE); } - static PyGetSetDef MD5_getseters[] = { {"block_size", (getter)MD5_get_block_size, NULL, @@ -462,40 +474,19 @@ static PyGetSetDef MD5_getseters[] = { {NULL} /* Sentinel */ }; -static PyTypeObject MD5type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_md5.md5", /*tp_name*/ - sizeof(MD5object), /*tp_basicsize*/ - 0, /*tp_itemsize*/ - /* methods */ - MD5_dealloc, /*tp_dealloc*/ - 0, /*tp_vectorcall_offset*/ - 0, /*tp_getattr*/ - 0, /*tp_setattr*/ - 0, /*tp_as_async*/ - 0, /*tp_repr*/ - 0, /*tp_as_number*/ - 0, /*tp_as_sequence*/ - 0, /*tp_as_mapping*/ - 0, /*tp_hash*/ - 0, /*tp_call*/ - 0, /*tp_str*/ - 0, /*tp_getattro*/ - 0, /*tp_setattro*/ - 0, /*tp_as_buffer*/ - Py_TPFLAGS_DEFAULT, /*tp_flags*/ - 0, /*tp_doc*/ - 0, /*tp_traverse*/ - 0, /*tp_clear*/ - 0, /*tp_richcompare*/ - 0, /*tp_weaklistoffset*/ - 0, /*tp_iter*/ - 0, /*tp_iternext*/ - MD5_methods, /* tp_methods */ - NULL, /* tp_members */ - MD5_getseters, /* tp_getset */ +static PyType_Slot md5_type_slots[] = { + {Py_tp_dealloc, MD5_dealloc}, + {Py_tp_methods, MD5_methods}, + {Py_tp_getset, MD5_getseters}, + {0,0} }; +static PyType_Spec md5_type_spec = { + .name = "_md5.md5", + .basicsize = sizeof(MD5object), + .flags = Py_TPFLAGS_DEFAULT, + .slots = md5_type_slots +}; /* The single module-level function: new() */ @@ -519,7 +510,8 @@ _md5_md5_impl(PyObject *module, PyObject *string, int usedforsecurity) if (string) GET_BUFFER_VIEW_OR_ERROUT(string, &buf); - if ((new = newMD5object()) == NULL) { + MD5State *st = md5_get_state(module); + if ((new = newMD5object(st)) == NULL) { if (string) PyBuffer_Release(&buf); return NULL; @@ -549,37 +541,69 @@ static struct PyMethodDef MD5_functions[] = { {NULL, NULL} /* Sentinel */ }; +static int +_md5_traverse(PyObject *module, visitproc visit, void *arg) +{ + MD5State *state = md5_get_state(module); + Py_VISIT(state->md5_type); + return 0; +} + +static int +_md5_clear(PyObject *module) +{ + MD5State *state = md5_get_state(module); + Py_CLEAR(state->md5_type); + return 0; +} + +static void +_md5_free(void *module) +{ + _md5_clear((PyObject *)module); +} /* Initialize this module. */ +static int +md5_exec(PyObject *m) +{ + MD5State *st = md5_get_state(m); + + st->md5_type = (PyTypeObject *)PyType_FromModuleAndSpec( + m, &md5_type_spec, NULL); + + if (st->md5_type == NULL) { + return -1; + } + + Py_INCREF((PyObject *)st->md5_type); + if (PyModule_AddObject(m, "MD5Type", (PyObject *)st->md5_type) < 0) { + Py_DECREF(st->md5_type); + return -1; + } + + return 0; +} + +static PyModuleDef_Slot _md5_slots[] = { + {Py_mod_exec, md5_exec}, + {0, NULL} +}; + static struct PyModuleDef _md5module = { PyModuleDef_HEAD_INIT, - "_md5", - NULL, - -1, - MD5_functions, - NULL, - NULL, - NULL, - NULL + .m_name = "_md5", + .m_size = sizeof(MD5State), + .m_methods = MD5_functions, + .m_slots = _md5_slots, + .m_traverse = _md5_traverse, + .m_clear = _md5_clear, + .m_free = _md5_free, }; PyMODINIT_FUNC PyInit__md5(void) { - PyObject *m; - - Py_SET_TYPE(&MD5type, &PyType_Type); - if (PyType_Ready(&MD5type) < 0) { - return NULL; - } - - m = PyModule_Create(&_md5module); - if (m == NULL) { - return NULL; - } - - Py_INCREF((PyObject *)&MD5type); - PyModule_AddObject(m, "MD5Type", (PyObject *)&MD5type); - return m; + return PyModuleDef_Init(&_md5module); } diff --git a/Modules/sha1module.c b/Modules/sha1module.c index b0656d83b3ae8..c22437de256b6 100644 --- a/Modules/sha1module.c +++ b/Modules/sha1module.c @@ -295,13 +295,22 @@ sha1_done(struct sha1_state *sha1, unsigned char *out) * ------------------------------------------------------------------------ */ -static PyTypeObject SHA1type; +typedef struct { + PyTypeObject* sha1_type; +} SHA1State; +static inline SHA1State* +sha1_get_state(PyObject *module) +{ + void *state = PyModule_GetState(module); + assert(state != NULL); + return (SHA1State *)state; +} static SHA1object * -newSHA1object(void) +newSHA1object(SHA1State *st) { - return (SHA1object *)PyObject_New(SHA1object, &SHA1type); + return (SHA1object *)PyObject_New(SHA1object, st->sha1_type); } @@ -310,7 +319,9 @@ newSHA1object(void) static void SHA1_dealloc(PyObject *ptr) { + PyTypeObject *tp = Py_TYPE(ptr); PyObject_Del(ptr); + Py_DECREF(tp); } @@ -319,16 +330,19 @@ SHA1_dealloc(PyObject *ptr) /*[clinic input] SHA1Type.copy + cls: defining_class + Return a copy of the hash object. [clinic start generated code]*/ static PyObject * -SHA1Type_copy_impl(SHA1object *self) -/*[clinic end generated code: output=b4e001264620f02a input=b7eae10df6f89b36]*/ +SHA1Type_copy_impl(SHA1object *self, PyTypeObject *cls) +/*[clinic end generated code: output=b32d4461ce8bc7a7 input=6c22e66fcc34c58e]*/ { - SHA1object *newobj; + SHA1State *st = PyType_GetModuleState(cls); - if ((newobj = newSHA1object()) == NULL) + SHA1object *newobj; + if ((newobj = newSHA1object(st)) == NULL) return NULL; newobj->hash_state = self->hash_state; @@ -422,7 +436,6 @@ sha1_get_digest_size(PyObject *self, void *closure) return PyLong_FromLong(SHA1_DIGESTSIZE); } - static PyGetSetDef SHA1_getseters[] = { {"block_size", (getter)SHA1_get_block_size, NULL, @@ -439,40 +452,19 @@ static PyGetSetDef SHA1_getseters[] = { {NULL} /* Sentinel */ }; -static PyTypeObject SHA1type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_sha1.sha1", /*tp_name*/ - sizeof(SHA1object), /*tp_basicsize*/ - 0, /*tp_itemsize*/ - /* methods */ - SHA1_dealloc, /*tp_dealloc*/ - 0, /*tp_vectorcall_offset*/ - 0, /*tp_getattr*/ - 0, /*tp_setattr*/ - 0, /*tp_as_async*/ - 0, /*tp_repr*/ - 0, /*tp_as_number*/ - 0, /*tp_as_sequence*/ - 0, /*tp_as_mapping*/ - 0, /*tp_hash*/ - 0, /*tp_call*/ - 0, /*tp_str*/ - 0, /*tp_getattro*/ - 0, /*tp_setattro*/ - 0, /*tp_as_buffer*/ - Py_TPFLAGS_DEFAULT, /*tp_flags*/ - 0, /*tp_doc*/ - 0, /*tp_traverse*/ - 0, /*tp_clear*/ - 0, /*tp_richcompare*/ - 0, /*tp_weaklistoffset*/ - 0, /*tp_iter*/ - 0, /*tp_iternext*/ - SHA1_methods, /* tp_methods */ - NULL, /* tp_members */ - SHA1_getseters, /* tp_getset */ +static PyType_Slot sha1_type_slots[] = { + {Py_tp_dealloc, SHA1_dealloc}, + {Py_tp_methods, SHA1_methods}, + {Py_tp_getset, SHA1_getseters}, + {0,0} }; +static PyType_Spec sha1_type_spec = { + .name = "_sha1.sha1", + .basicsize = sizeof(SHA1object), + .flags = Py_TPFLAGS_DEFAULT, + .slots = sha1_type_slots +}; /* The single module-level function: new() */ @@ -496,7 +488,8 @@ _sha1_sha1_impl(PyObject *module, PyObject *string, int usedforsecurity) if (string) GET_BUFFER_VIEW_OR_ERROUT(string, &buf); - if ((new = newSHA1object()) == NULL) { + SHA1State *st = sha1_get_state(module); + if ((new = newSHA1object(st)) == NULL) { if (string) PyBuffer_Release(&buf); return NULL; @@ -526,37 +519,72 @@ static struct PyMethodDef SHA1_functions[] = { {NULL, NULL} /* Sentinel */ }; +static int +_sha1_traverse(PyObject *module, visitproc visit, void *arg) +{ + SHA1State *state = sha1_get_state(module); + Py_VISIT(state->sha1_type); + return 0; +} + +static int +_sha1_clear(PyObject *module) +{ + SHA1State *state = sha1_get_state(module); + Py_CLEAR(state->sha1_type); + return 0; +} + +static void +_sha1_free(void *module) +{ + _sha1_clear((PyObject *)module); +} + +static int +_sha1_exec(PyObject *module) +{ + SHA1State* st = sha1_get_state(module); + + st->sha1_type = (PyTypeObject *)PyType_FromModuleAndSpec( + module, &sha1_type_spec, NULL); + + if (st->sha1_type == NULL) { + return -1; + } + + Py_INCREF(st->sha1_type); + if (PyModule_AddObject(module, + "SHA1Type", + (PyObject *)st->sha1_type) < 0) { + Py_DECREF(st->sha1_type); + return -1; + } + + return 0; +} + /* Initialize this module. */ +static PyModuleDef_Slot _sha1_slots[] = { + {Py_mod_exec, _sha1_exec}, + {0, NULL} +}; + static struct PyModuleDef _sha1module = { PyModuleDef_HEAD_INIT, - "_sha1", - NULL, - -1, - SHA1_functions, - NULL, - NULL, - NULL, - NULL + .m_name = "_sha1", + .m_size = sizeof(SHA1State), + .m_methods = SHA1_functions, + .m_slots = _sha1_slots, + .m_traverse = _sha1_traverse, + .m_clear = _sha1_clear, + .m_free = _sha1_free }; PyMODINIT_FUNC PyInit__sha1(void) { - PyObject *m; - - Py_SET_TYPE(&SHA1type, &PyType_Type); - if (PyType_Ready(&SHA1type) < 0) { - return NULL; - } - - m = PyModule_Create(&_sha1module); - if (m == NULL) { - return NULL; - } - - Py_INCREF((PyObject *)&SHA1type); - PyModule_AddObject(m, "SHA1Type", (PyObject *)&SHA1type); - return m; + return PyModuleDef_Init(&_sha1module); } diff --git a/Modules/sha512module.c b/Modules/sha512module.c index aa2aeedcc6c64..725098def4d06 100644 --- a/Modules/sha512module.c +++ b/Modules/sha512module.c @@ -422,20 +422,29 @@ sha512_final(unsigned char digest[SHA_DIGESTSIZE], SHAobject *sha_info) * ------------------------------------------------------------------------ */ -static PyTypeObject SHA384type; -static PyTypeObject SHA512type; +typedef struct { + PyTypeObject* sha384_type; + PyTypeObject* sha512_type; +} SHA512State; +static inline SHA512State* +sha512_get_state(PyObject *module) +{ + void *state = PyModule_GetState(module); + assert(state != NULL); + return (SHA512State *)state; +} static SHAobject * -newSHA384object(void) +newSHA384object(SHA512State *st) { - return (SHAobject *)PyObject_New(SHAobject, &SHA384type); + return (SHAobject *)PyObject_New(SHAobject, st->sha384_type); } static SHAobject * -newSHA512object(void) +newSHA512object(SHA512State *st) { - return (SHAobject *)PyObject_New(SHAobject, &SHA512type); + return (SHAobject *)PyObject_New(SHAobject, st->sha512_type); } /* Internal methods for a hash object */ @@ -443,7 +452,9 @@ newSHA512object(void) static void SHA512_dealloc(PyObject *ptr) { + PyTypeObject *tp = Py_TYPE(ptr); PyObject_Del(ptr); + Py_DECREF(tp); } @@ -452,21 +463,27 @@ SHA512_dealloc(PyObject *ptr) /*[clinic input] SHA512Type.copy + cls: defining_class + Return a copy of the hash object. [clinic start generated code]*/ static PyObject * -SHA512Type_copy_impl(SHAobject *self) -/*[clinic end generated code: output=adea896ed3164821 input=9f5f31e6c457776a]*/ +SHA512Type_copy_impl(SHAobject *self, PyTypeObject *cls) +/*[clinic end generated code: output=85ea5b47837a08e6 input=f673a18f66527c90]*/ { SHAobject *newobj; + SHA512State *st = PyType_GetModuleState(cls); - if (Py_IS_TYPE((PyObject*)self, &SHA512type)) { - if ( (newobj = newSHA512object())==NULL) + if (Py_IS_TYPE((PyObject*)self, st->sha512_type)) { + if ( (newobj = newSHA512object(st))==NULL) { return NULL; - } else { - if ( (newobj = newSHA384object())==NULL) + } + } + else { + if ( (newobj = newSHA384object(st))==NULL) { return NULL; + } } SHAcopy(self, newobj); @@ -574,74 +591,37 @@ static PyMemberDef SHA_members[] = { {NULL} /* Sentinel */ }; -static PyTypeObject SHA384type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_sha512.sha384", /*tp_name*/ - sizeof(SHAobject), /*tp_basicsize*/ - 0, /*tp_itemsize*/ - /* methods */ - SHA512_dealloc, /*tp_dealloc*/ - 0, /*tp_vectorcall_offset*/ - 0, /*tp_getattr*/ - 0, /*tp_setattr*/ - 0, /*tp_as_async*/ - 0, /*tp_repr*/ - 0, /*tp_as_number*/ - 0, /*tp_as_sequence*/ - 0, /*tp_as_mapping*/ - 0, /*tp_hash*/ - 0, /*tp_call*/ - 0, /*tp_str*/ - 0, /*tp_getattro*/ - 0, /*tp_setattro*/ - 0, /*tp_as_buffer*/ - Py_TPFLAGS_DEFAULT, /*tp_flags*/ - 0, /*tp_doc*/ - 0, /*tp_traverse*/ - 0, /*tp_clear*/ - 0, /*tp_richcompare*/ - 0, /*tp_weaklistoffset*/ - 0, /*tp_iter*/ - 0, /*tp_iternext*/ - SHA_methods, /* tp_methods */ - SHA_members, /* tp_members */ - SHA_getseters, /* tp_getset */ +static PyType_Slot sha512_sha384_type_slots[] = { + {Py_tp_dealloc, SHA512_dealloc}, + {Py_tp_methods, SHA_methods}, + {Py_tp_members, SHA_members}, + {Py_tp_getset, SHA_getseters}, + {0,0} +}; + +static PyType_Spec sha512_sha384_type_spec = { + .name = "_sha512.sha384", + .basicsize = sizeof(SHAobject), + .flags = Py_TPFLAGS_DEFAULT, + .slots = sha512_sha384_type_slots }; -static PyTypeObject SHA512type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_sha512.sha512", /*tp_name*/ - sizeof(SHAobject), /*tp_basicsize*/ - 0, /*tp_itemsize*/ - /* methods */ - SHA512_dealloc, /*tp_dealloc*/ - 0, /*tp_vectorcall_offset*/ - 0, /*tp_getattr*/ - 0, /*tp_setattr*/ - 0, /*tp_as_async*/ - 0, /*tp_repr*/ - 0, /*tp_as_number*/ - 0, /*tp_as_sequence*/ - 0, /*tp_as_mapping*/ - 0, /*tp_hash*/ - 0, /*tp_call*/ - 0, /*tp_str*/ - 0, /*tp_getattro*/ - 0, /*tp_setattro*/ - 0, /*tp_as_buffer*/ - Py_TPFLAGS_DEFAULT, /*tp_flags*/ - 0, /*tp_doc*/ - 0, /*tp_traverse*/ - 0, /*tp_clear*/ - 0, /*tp_richcompare*/ - 0, /*tp_weaklistoffset*/ - 0, /*tp_iter*/ - 0, /*tp_iternext*/ - SHA_methods, /* tp_methods */ - SHA_members, /* tp_members */ - SHA_getseters, /* tp_getset */ +static PyType_Slot sha512_sha512_type_slots[] = { + {Py_tp_dealloc, SHA512_dealloc}, + {Py_tp_methods, SHA_methods}, + {Py_tp_members, SHA_members}, + {Py_tp_getset, SHA_getseters}, + {0,0} }; +// Using PyType_GetModuleState() on this type is safe since +// it cannot be subclassed: it does not have the Py_TPFLAGS_BASETYPE flag. +static PyType_Spec sha512_sha512_type_spec = { + .name = "_sha512.sha512", + .basicsize = sizeof(SHAobject), + .flags = Py_TPFLAGS_DEFAULT, + .slots = sha512_sha512_type_slots +}; /* The single module-level function: new() */ @@ -662,10 +642,12 @@ _sha512_sha512_impl(PyObject *module, PyObject *string, int usedforsecurity) SHAobject *new; Py_buffer buf; + SHA512State *st = sha512_get_state(module); + if (string) GET_BUFFER_VIEW_OR_ERROUT(string, &buf); - if ((new = newSHA512object()) == NULL) { + if ((new = newSHA512object(st)) == NULL) { if (string) PyBuffer_Release(&buf); return NULL; @@ -704,10 +686,12 @@ _sha512_sha384_impl(PyObject *module, PyObject *string, int usedforsecurity) SHAobject *new; Py_buffer buf; + SHA512State *st = sha512_get_state(module); + if (string) GET_BUFFER_VIEW_OR_ERROUT(string, &buf); - if ((new = newSHA384object()) == NULL) { + if ((new = newSHA384object(st)) == NULL) { if (string) PyBuffer_Release(&buf); return NULL; @@ -738,43 +722,80 @@ static struct PyMethodDef SHA_functions[] = { {NULL, NULL} /* Sentinel */ }; +static int +_sha512_traverse(PyObject *module, visitproc visit, void *arg) +{ + SHA512State *state = sha512_get_state(module); + Py_VISIT(state->sha384_type); + Py_VISIT(state->sha512_type); + return 0; +} -/* Initialize this module. */ +static int +_sha512_clear(PyObject *module) +{ + SHA512State *state = sha512_get_state(module); + Py_CLEAR(state->sha384_type); + Py_CLEAR(state->sha512_type); + return 0; +} -static struct PyModuleDef _sha512module = { - PyModuleDef_HEAD_INIT, - "_sha512", - NULL, - -1, - SHA_functions, - NULL, - NULL, - NULL, - NULL -}; +static void +_sha512_free(void *module) +{ + _sha512_clear((PyObject *)module); +} -PyMODINIT_FUNC -PyInit__sha512(void) + +/* Initialize this module. */ +static int +_sha512_exec(PyObject *m) { - PyObject *m; + SHA512State* st = sha512_get_state(m); - Py_SET_TYPE(&SHA384type, &PyType_Type); - if (PyType_Ready(&SHA384type) < 0) { - return NULL; + st->sha384_type = (PyTypeObject *)PyType_FromModuleAndSpec( + m, &sha512_sha384_type_spec, NULL); + + st->sha512_type = (PyTypeObject *)PyType_FromModuleAndSpec( + m, &sha512_sha512_type_spec, NULL); + + if (st->sha384_type == NULL || st->sha512_type == NULL) { + return -1; } - Py_SET_TYPE(&SHA512type, &PyType_Type); - if (PyType_Ready(&SHA512type) < 0) { - return NULL; + + Py_INCREF(st->sha384_type); + if (PyModule_AddObject(m, "SHA384Type", (PyObject *)st->sha384_type) < 0) { + Py_DECREF(st->sha384_type); + return -1; } - m = PyModule_Create(&_sha512module); - if (m == NULL) { - return NULL; + Py_INCREF(st->sha512_type); + if (PyModule_AddObject(m, "SHA384Type", (PyObject *)st->sha512_type) < 0) { + Py_DECREF(st->sha512_type); + return -1; } - Py_INCREF((PyObject *)&SHA384type); - PyModule_AddObject(m, "SHA384Type", (PyObject *)&SHA384type); - Py_INCREF((PyObject *)&SHA512type); - PyModule_AddObject(m, "SHA512Type", (PyObject *)&SHA512type); - return m; + return 0; +} + +static PyModuleDef_Slot _sha512_slots[] = { + {Py_mod_exec, _sha512_exec}, + {0, NULL} +}; + +static struct PyModuleDef _sha512module = { + PyModuleDef_HEAD_INIT, + .m_name = "_sha512", + .m_size = sizeof(SHA512State), + .m_methods = SHA_functions, + .m_slots = _sha512_slots, + .m_traverse = _sha512_traverse, + .m_clear = _sha512_clear, + .m_free = _sha512_free +}; + +PyMODINIT_FUNC +PyInit__sha512(void) +{ + return PyModuleDef_Init(&_sha512module); } From webhook-mailer at python.org Sun Sep 6 18:10:17 2020 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sun, 06 Sep 2020 22:10:17 -0000 Subject: [Python-checkins] bpo-41513: Expand comments and add references for a better understanding (GH-22123) Message-ID: https://github.com/python/cpython/commit/67c998de24985b36498b0fdac68d1beceb43aba7 commit: 67c998de24985b36498b0fdac68d1beceb43aba7 branch: master author: Raymond Hettinger committer: GitHub date: 2020-09-06T15:10:07-07:00 summary: bpo-41513: Expand comments and add references for a better understanding (GH-22123) files: M Modules/mathmodule.c diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index d227a5d15dca2..29137ae91a22f 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -2419,9 +2419,9 @@ To avoid overflow/underflow and to achieve high accuracy giving results that are almost always correctly rounded, four techniques are used: * lossless scaling using a power-of-two scaling factor -* accurate squaring using Veltkamp-Dekker splitting -* compensated summation using a variant of the Neumaier algorithm -* differential correction of the square root +* accurate squaring using Veltkamp-Dekker splitting [1] +* compensated summation using a variant of the Neumaier algorithm [2] +* differential correction of the square root [3] The usual presentation of the Neumaier summation algorithm has an expensive branch depending on which operand has the larger @@ -2456,7 +2456,11 @@ Given that csum >= 1.0, we have: Since lo**2 is less than 1/2 ulp(csum), we have csum+lo*lo == csum. To minimize loss of information during the accumulation of fractional -values, each term has a separate accumulator. +values, each term has a separate accumulator. This also breaks up +sequential dependencies in the inner loop so the CPU can maximize +floating point throughput. [5] On a 2.6 GHz Haswell, adding one +dimension has an incremental cost of only 5ns -- for example when +moving from hypot(x,y) to hypot(x,y,z). The square root differential correction is needed because a correctly rounded square root of a correctly rounded sum of @@ -2466,7 +2470,7 @@ The differential correction starts with a value *x* that is the difference between the square of *h*, the possibly inaccurately rounded square root, and the accurately computed sum of squares. The correction is the first order term of the Maclaurin series -expansion of sqrt(h**2 + x) == h + x/(2*h) + O(x**2). +expansion of sqrt(h**2 + x) == h + x/(2*h) + O(x**2). [4] Essentially, this differential correction is equivalent to one refinement step in Newton's divide-and-average square root @@ -2474,12 +2478,24 @@ algorithm, effectively doubling the number of accurate bits. This technique is used in Dekker's SQRT2 algorithm and again in Borges' ALGORITHM 4 and 5. +Without proof for all cases, hypot() cannot claim to be always +correctly rounded. However for n <= 1000, prior to the final addition +that rounds the overall result, the internal accuracy of "h" together +with its correction of "x / (2.0 * h)" is at least 100 bits. [6] +Also, hypot() was tested against a Decimal implementation with +prec=300. After 100 million trials, no incorrectly rounded examples +were found. In addition, perfect commutativity (all permutations are +exactly equal) was verified for 1 billion random inputs with n=5. [7] + References: 1. Veltkamp-Dekker splitting: http://csclub.uwaterloo.ca/~pbarfuss/dekker1971.pdf 2. Compensated summation: http://www.ti3.tu-harburg.de/paper/rump/Ru08b.pdf 3. Square root differential correction: https://arxiv.org/pdf/1904.09481.pdf 4. https://www.wolframalpha.com/input/?i=Maclaurin+series+sqrt%28h**2+%2B+x%29+at+x%3D0 +5. https://bugs.python.org/file49439/hypot.png +6. https://bugs.python.org/file49435/best_frac.py +7. https://bugs.python.org/file49448/test_hypot_commutativity.py */ From webhook-mailer at python.org Mon Sep 7 01:29:51 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Mon, 07 Sep 2020 05:29:51 -0000 Subject: [Python-checkins] [doc] Add link to Generic in typing (GH-22125) Message-ID: https://github.com/python/cpython/commit/5bfd60fc2bf26bd6fa23a3a50c7990b7f68b3ea3 commit: 5bfd60fc2bf26bd6fa23a3a50c7990b7f68b3ea3 branch: master author: Andre Delfino committer: GitHub date: 2020-09-06T22:29:38-07:00 summary: [doc] Add link to Generic in typing (GH-22125) files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 6d6b76c1d0895..3125ae97808a9 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -818,7 +818,7 @@ These are not used in annotations. They are building blocks for creating generic Type variables exist primarily for the benefit of static type checkers. They serve as the parameters for generic types as well - as for generic function definitions. See class Generic for more + as for generic function definitions. See :class:`Generic` for more information on generic types. Generic functions work as follows:: def repeat(x: T, n: int) -> Sequence[T]: From webhook-mailer at python.org Mon Sep 7 04:28:04 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Mon, 07 Sep 2020 08:28:04 -0000 Subject: [Python-checkins] bpo-1635741 port zlib module to multi-phase init (GH-21995) Message-ID: https://github.com/python/cpython/commit/1aaa21ff818b08af2a68862b552b7ba0857492eb commit: 1aaa21ff818b08af2a68862b552b7ba0857492eb branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-07T10:27:55+02:00 summary: bpo-1635741 port zlib module to multi-phase init (GH-21995) Port the zlib extension module to multi-phase initialization (PEP 489). files: A Misc/NEWS.d/next/Core and Builtins/2020-08-28-20-54-04.bpo-1635741.7ijlcI.rst M Modules/clinic/zlibmodule.c.h M Modules/zlibmodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-28-20-54-04.bpo-1635741.7ijlcI.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-28-20-54-04.bpo-1635741.7ijlcI.rst new file mode 100644 index 0000000000000..4d6ce1185ed93 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-28-20-54-04.bpo-1635741.7ijlcI.rst @@ -0,0 +1 @@ +Port the :mod:`zlib` extension module to multi-phase initialization (:pep:`489`). diff --git a/Modules/clinic/zlibmodule.c.h b/Modules/clinic/zlibmodule.c.h index 61dfa9a87b5fb..14e955db64e72 100644 --- a/Modules/clinic/zlibmodule.c.h +++ b/Modules/clinic/zlibmodule.c.h @@ -329,25 +329,25 @@ PyDoc_STRVAR(zlib_Compress_compress__doc__, "Call the flush() method to clear these buffers."); #define ZLIB_COMPRESS_COMPRESS_METHODDEF \ - {"compress", (PyCFunction)zlib_Compress_compress, METH_O, zlib_Compress_compress__doc__}, + {"compress", (PyCFunction)(void(*)(void))zlib_Compress_compress, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, zlib_Compress_compress__doc__}, static PyObject * -zlib_Compress_compress_impl(compobject *self, Py_buffer *data); +zlib_Compress_compress_impl(compobject *self, PyTypeObject *cls, + Py_buffer *data); static PyObject * -zlib_Compress_compress(compobject *self, PyObject *arg) +zlib_Compress_compress(compobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"", NULL}; + static _PyArg_Parser _parser = {"y*:compress", _keywords, 0}; Py_buffer data = {NULL, NULL}; - if (PyObject_GetBuffer(arg, &data, PyBUF_SIMPLE) != 0) { + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &data)) { goto exit; } - if (!PyBuffer_IsContiguous(&data, 'C')) { - _PyArg_BadArgument("compress", "argument", "contiguous buffer", arg); - goto exit; - } - return_value = zlib_Compress_compress_impl(self, &data); + return_value = zlib_Compress_compress_impl(self, cls, &data); exit: /* Cleanup for data */ @@ -376,51 +376,26 @@ PyDoc_STRVAR(zlib_Decompress_decompress__doc__, "Call the flush() method to clear these buffers."); #define ZLIB_DECOMPRESS_DECOMPRESS_METHODDEF \ - {"decompress", (PyCFunction)(void(*)(void))zlib_Decompress_decompress, METH_FASTCALL|METH_KEYWORDS, zlib_Decompress_decompress__doc__}, + {"decompress", (PyCFunction)(void(*)(void))zlib_Decompress_decompress, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, zlib_Decompress_decompress__doc__}, static PyObject * -zlib_Decompress_decompress_impl(compobject *self, Py_buffer *data, - Py_ssize_t max_length); +zlib_Decompress_decompress_impl(compobject *self, PyTypeObject *cls, + Py_buffer *data, Py_ssize_t max_length); static PyObject * -zlib_Decompress_decompress(compobject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +zlib_Decompress_decompress(compobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; static const char * const _keywords[] = {"", "max_length", NULL}; - static _PyArg_Parser _parser = {NULL, _keywords, "decompress", 0}; - PyObject *argsbuf[2]; - Py_ssize_t noptargs = nargs + (kwnames ? PyTuple_GET_SIZE(kwnames) : 0) - 1; + static _PyArg_Parser _parser = {"y*|n:decompress", _keywords, 0}; Py_buffer data = {NULL, NULL}; Py_ssize_t max_length = 0; - args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 1, 2, 0, argsbuf); - if (!args) { - goto exit; - } - if (PyObject_GetBuffer(args[0], &data, PyBUF_SIMPLE) != 0) { - goto exit; - } - if (!PyBuffer_IsContiguous(&data, 'C')) { - _PyArg_BadArgument("decompress", "argument 1", "contiguous buffer", args[0]); + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &data, &max_length)) { goto exit; } - if (!noptargs) { - goto skip_optional_pos; - } - { - Py_ssize_t ival = -1; - PyObject *iobj = _PyNumber_Index(args[1]); - if (iobj != NULL) { - ival = PyLong_AsSsize_t(iobj); - Py_DECREF(iobj); - } - if (ival == -1 && PyErr_Occurred()) { - goto exit; - } - max_length = ival; - } -skip_optional_pos: - return_value = zlib_Decompress_decompress_impl(self, &data, max_length); + return_value = zlib_Decompress_decompress_impl(self, cls, &data, max_length); exit: /* Cleanup for data */ @@ -444,29 +419,24 @@ PyDoc_STRVAR(zlib_Compress_flush__doc__, " can still be compressed."); #define ZLIB_COMPRESS_FLUSH_METHODDEF \ - {"flush", (PyCFunction)(void(*)(void))zlib_Compress_flush, METH_FASTCALL, zlib_Compress_flush__doc__}, + {"flush", (PyCFunction)(void(*)(void))zlib_Compress_flush, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, zlib_Compress_flush__doc__}, static PyObject * -zlib_Compress_flush_impl(compobject *self, int mode); +zlib_Compress_flush_impl(compobject *self, PyTypeObject *cls, int mode); static PyObject * -zlib_Compress_flush(compobject *self, PyObject *const *args, Py_ssize_t nargs) +zlib_Compress_flush(compobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"", NULL}; + static _PyArg_Parser _parser = {"|i:flush", _keywords, 0}; int mode = Z_FINISH; - if (!_PyArg_CheckPositional("flush", nargs, 0, 1)) { - goto exit; - } - if (nargs < 1) { - goto skip_optional; - } - mode = _PyLong_AsInt(args[0]); - if (mode == -1 && PyErr_Occurred()) { + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &mode)) { goto exit; } -skip_optional: - return_value = zlib_Compress_flush_impl(self, mode); + return_value = zlib_Compress_flush_impl(self, cls, mode); exit: return return_value; @@ -481,15 +451,26 @@ PyDoc_STRVAR(zlib_Compress_copy__doc__, "Return a copy of the compression object."); #define ZLIB_COMPRESS_COPY_METHODDEF \ - {"copy", (PyCFunction)zlib_Compress_copy, METH_NOARGS, zlib_Compress_copy__doc__}, + {"copy", (PyCFunction)(void(*)(void))zlib_Compress_copy, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, zlib_Compress_copy__doc__}, static PyObject * -zlib_Compress_copy_impl(compobject *self); +zlib_Compress_copy_impl(compobject *self, PyTypeObject *cls); static PyObject * -zlib_Compress_copy(compobject *self, PyObject *Py_UNUSED(ignored)) +zlib_Compress_copy(compobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return zlib_Compress_copy_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":copy", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = zlib_Compress_copy_impl(self, cls); + +exit: + return return_value; } #endif /* defined(HAVE_ZLIB_COPY) */ @@ -502,15 +483,26 @@ PyDoc_STRVAR(zlib_Compress___copy____doc__, "\n"); #define ZLIB_COMPRESS___COPY___METHODDEF \ - {"__copy__", (PyCFunction)zlib_Compress___copy__, METH_NOARGS, zlib_Compress___copy____doc__}, + {"__copy__", (PyCFunction)(void(*)(void))zlib_Compress___copy__, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, zlib_Compress___copy____doc__}, static PyObject * -zlib_Compress___copy___impl(compobject *self); +zlib_Compress___copy___impl(compobject *self, PyTypeObject *cls); static PyObject * -zlib_Compress___copy__(compobject *self, PyObject *Py_UNUSED(ignored)) +zlib_Compress___copy__(compobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return zlib_Compress___copy___impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":__copy__", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = zlib_Compress___copy___impl(self, cls); + +exit: + return return_value; } #endif /* defined(HAVE_ZLIB_COPY) */ @@ -523,7 +515,29 @@ PyDoc_STRVAR(zlib_Compress___deepcopy____doc__, "\n"); #define ZLIB_COMPRESS___DEEPCOPY___METHODDEF \ - {"__deepcopy__", (PyCFunction)zlib_Compress___deepcopy__, METH_O, zlib_Compress___deepcopy____doc__}, + {"__deepcopy__", (PyCFunction)(void(*)(void))zlib_Compress___deepcopy__, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, zlib_Compress___deepcopy____doc__}, + +static PyObject * +zlib_Compress___deepcopy___impl(compobject *self, PyTypeObject *cls, + PyObject *memo); + +static PyObject * +zlib_Compress___deepcopy__(compobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"", NULL}; + static _PyArg_Parser _parser = {"O:__deepcopy__", _keywords, 0}; + PyObject *memo; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &memo)) { + goto exit; + } + return_value = zlib_Compress___deepcopy___impl(self, cls, memo); + +exit: + return return_value; +} #endif /* defined(HAVE_ZLIB_COPY) */ @@ -536,15 +550,26 @@ PyDoc_STRVAR(zlib_Decompress_copy__doc__, "Return a copy of the decompression object."); #define ZLIB_DECOMPRESS_COPY_METHODDEF \ - {"copy", (PyCFunction)zlib_Decompress_copy, METH_NOARGS, zlib_Decompress_copy__doc__}, + {"copy", (PyCFunction)(void(*)(void))zlib_Decompress_copy, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, zlib_Decompress_copy__doc__}, static PyObject * -zlib_Decompress_copy_impl(compobject *self); +zlib_Decompress_copy_impl(compobject *self, PyTypeObject *cls); static PyObject * -zlib_Decompress_copy(compobject *self, PyObject *Py_UNUSED(ignored)) +zlib_Decompress_copy(compobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return zlib_Decompress_copy_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":copy", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = zlib_Decompress_copy_impl(self, cls); + +exit: + return return_value; } #endif /* defined(HAVE_ZLIB_COPY) */ @@ -557,15 +582,26 @@ PyDoc_STRVAR(zlib_Decompress___copy____doc__, "\n"); #define ZLIB_DECOMPRESS___COPY___METHODDEF \ - {"__copy__", (PyCFunction)zlib_Decompress___copy__, METH_NOARGS, zlib_Decompress___copy____doc__}, + {"__copy__", (PyCFunction)(void(*)(void))zlib_Decompress___copy__, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, zlib_Decompress___copy____doc__}, static PyObject * -zlib_Decompress___copy___impl(compobject *self); +zlib_Decompress___copy___impl(compobject *self, PyTypeObject *cls); static PyObject * -zlib_Decompress___copy__(compobject *self, PyObject *Py_UNUSED(ignored)) +zlib_Decompress___copy__(compobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return zlib_Decompress___copy___impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":__copy__", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = zlib_Decompress___copy___impl(self, cls); + +exit: + return return_value; } #endif /* defined(HAVE_ZLIB_COPY) */ @@ -578,7 +614,29 @@ PyDoc_STRVAR(zlib_Decompress___deepcopy____doc__, "\n"); #define ZLIB_DECOMPRESS___DEEPCOPY___METHODDEF \ - {"__deepcopy__", (PyCFunction)zlib_Decompress___deepcopy__, METH_O, zlib_Decompress___deepcopy____doc__}, + {"__deepcopy__", (PyCFunction)(void(*)(void))zlib_Decompress___deepcopy__, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, zlib_Decompress___deepcopy____doc__}, + +static PyObject * +zlib_Decompress___deepcopy___impl(compobject *self, PyTypeObject *cls, + PyObject *memo); + +static PyObject * +zlib_Decompress___deepcopy__(compobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"", NULL}; + static _PyArg_Parser _parser = {"O:__deepcopy__", _keywords, 0}; + PyObject *memo; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &memo)) { + goto exit; + } + return_value = zlib_Decompress___deepcopy___impl(self, cls, memo); + +exit: + return return_value; +} #endif /* defined(HAVE_ZLIB_COPY) */ @@ -592,37 +650,25 @@ PyDoc_STRVAR(zlib_Decompress_flush__doc__, " the initial size of the output buffer."); #define ZLIB_DECOMPRESS_FLUSH_METHODDEF \ - {"flush", (PyCFunction)(void(*)(void))zlib_Decompress_flush, METH_FASTCALL, zlib_Decompress_flush__doc__}, + {"flush", (PyCFunction)(void(*)(void))zlib_Decompress_flush, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, zlib_Decompress_flush__doc__}, static PyObject * -zlib_Decompress_flush_impl(compobject *self, Py_ssize_t length); +zlib_Decompress_flush_impl(compobject *self, PyTypeObject *cls, + Py_ssize_t length); static PyObject * -zlib_Decompress_flush(compobject *self, PyObject *const *args, Py_ssize_t nargs) +zlib_Decompress_flush(compobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"", NULL}; + static _PyArg_Parser _parser = {"|n:flush", _keywords, 0}; Py_ssize_t length = DEF_BUF_SIZE; - if (!_PyArg_CheckPositional("flush", nargs, 0, 1)) { + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &length)) { goto exit; } - if (nargs < 1) { - goto skip_optional; - } - { - Py_ssize_t ival = -1; - PyObject *iobj = _PyNumber_Index(args[0]); - if (iobj != NULL) { - ival = PyLong_AsSsize_t(iobj); - Py_DECREF(iobj); - } - if (ival == -1 && PyErr_Occurred()) { - goto exit; - } - length = ival; - } -skip_optional: - return_value = zlib_Decompress_flush_impl(self, length); + return_value = zlib_Decompress_flush_impl(self, cls, length); exit: return return_value; @@ -757,4 +803,4 @@ zlib_crc32(PyObject *module, PyObject *const *args, Py_ssize_t nargs) #ifndef ZLIB_DECOMPRESS___DEEPCOPY___METHODDEF #define ZLIB_DECOMPRESS___DEEPCOPY___METHODDEF #endif /* !defined(ZLIB_DECOMPRESS___DEEPCOPY___METHODDEF) */ -/*[clinic end generated code: output=be34f273564e39a8 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=6736bae59fab268b input=a9049054013a1b77]*/ diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c index fd3064952869b..def617671f18f 100644 --- a/Modules/zlibmodule.c +++ b/Modules/zlibmodule.c @@ -37,18 +37,16 @@ typedef struct { PyTypeObject *Comptype; PyTypeObject *Decomptype; PyObject *ZlibError; -} _zlibstate; +} zlibstate; -static inline _zlibstate* +static inline zlibstate* get_zlib_state(PyObject *module) { void *state = PyModule_GetState(module); assert(state != NULL); - return (_zlibstate *)state; + return (zlibstate *)state; } -#define _zlibstate_global ((_zlibstate *)PyModule_GetState(PyState_FindModule(&zlibmodule))) - typedef struct { PyObject_HEAD @@ -62,7 +60,7 @@ typedef struct } compobject; static void -zlib_error(z_stream zst, int err, const char *msg) +zlib_error(zlibstate *state, z_stream zst, int err, const char *msg) { const char *zmsg = Z_NULL; /* In case of a version mismatch, zst.msg won't be initialized. @@ -85,9 +83,9 @@ zlib_error(z_stream zst, int err, const char *msg) } } if (zmsg == Z_NULL) - PyErr_Format(_zlibstate_global->ZlibError, "Error %d %s", err, msg); + PyErr_Format(state->ZlibError, "Error %d %s", err, msg); else - PyErr_Format(_zlibstate_global->ZlibError, "Error %d %s: %.200s", err, msg, zmsg); + PyErr_Format(state->ZlibError, "Error %d %s: %.200s", err, msg, zmsg); } /*[clinic input] @@ -216,19 +214,20 @@ zlib_compress_impl(PyObject *module, Py_buffer *data, int level) /*[clinic end generated code: output=d80906d73f6294c8 input=638d54b6315dbed3]*/ { PyObject *RetVal = NULL; - Byte *ibuf; - Py_ssize_t ibuflen, obuflen = DEF_BUF_SIZE; - int err, flush; + Py_ssize_t obuflen = DEF_BUF_SIZE; + int flush; z_stream zst; - ibuf = data->buf; - ibuflen = data->len; + zlibstate *state = get_zlib_state(module); + + Byte *ibuf = data->buf; + Py_ssize_t ibuflen = data->len; zst.opaque = NULL; zst.zalloc = PyZlib_Malloc; zst.zfree = PyZlib_Free; zst.next_in = ibuf; - err = deflateInit(&zst, level); + int err = deflateInit(&zst, level); switch (err) { case Z_OK: @@ -238,11 +237,11 @@ zlib_compress_impl(PyObject *module, Py_buffer *data, int level) "Out of memory while compressing data"); goto error; case Z_STREAM_ERROR: - PyErr_SetString(_zlibstate_global->ZlibError, "Bad compression level"); + PyErr_SetString(state->ZlibError, "Bad compression level"); goto error; default: deflateEnd(&zst); - zlib_error(zst, err, "while compressing data"); + zlib_error(state, zst, err, "while compressing data"); goto error; } @@ -263,7 +262,7 @@ zlib_compress_impl(PyObject *module, Py_buffer *data, int level) if (err == Z_STREAM_ERROR) { deflateEnd(&zst); - zlib_error(zst, err, "while compressing data"); + zlib_error(state, zst, err, "while compressing data"); goto error; } @@ -281,7 +280,7 @@ zlib_compress_impl(PyObject *module, Py_buffer *data, int level) return RetVal; } else - zlib_error(zst, err, "while finishing compression"); + zlib_error(state, zst, err, "while finishing compression"); error: Py_XDECREF(RetVal); return NULL; @@ -312,6 +311,8 @@ zlib_decompress_impl(PyObject *module, Py_buffer *data, int wbits, int err, flush; z_stream zst; + zlibstate *state = get_zlib_state(module); + if (bufsize < 0) { PyErr_SetString(PyExc_ValueError, "bufsize must be non-negative"); return NULL; @@ -338,7 +339,7 @@ zlib_decompress_impl(PyObject *module, Py_buffer *data, int wbits, goto error; default: inflateEnd(&zst); - zlib_error(zst, err, "while preparing to decompress data"); + zlib_error(state, zst, err, "while preparing to decompress data"); goto error; } @@ -369,7 +370,7 @@ zlib_decompress_impl(PyObject *module, Py_buffer *data, int wbits, goto error; default: inflateEnd(&zst); - zlib_error(zst, err, "while decompressing data"); + zlib_error(state, zst, err, "while decompressing data"); goto error; } @@ -380,13 +381,13 @@ zlib_decompress_impl(PyObject *module, Py_buffer *data, int wbits, if (err != Z_STREAM_END) { inflateEnd(&zst); - zlib_error(zst, err, "while decompressing data"); + zlib_error(state, zst, err, "while decompressing data"); goto error; } err = inflateEnd(&zst); if (err != Z_OK) { - zlib_error(zst, err, "while finishing decompression"); + zlib_error(state, zst, err, "while finishing decompression"); goto error; } @@ -434,16 +435,14 @@ zlib_compressobj_impl(PyObject *module, int level, int method, int wbits, int memLevel, int strategy, Py_buffer *zdict) /*[clinic end generated code: output=8b5bed9c8fc3814d input=2fa3d026f90ab8d5]*/ { - compobject *self = NULL; - int err; - + zlibstate *state = get_zlib_state(module); if (zdict->buf != NULL && (size_t)zdict->len > UINT_MAX) { PyErr_SetString(PyExc_OverflowError, "zdict length does not fit in an unsigned int"); - goto error; + return NULL; } - self = newcompobject(_zlibstate_global->Comptype); + compobject *self = newcompobject(state->Comptype); if (self == NULL) goto error; self->zst.opaque = NULL; @@ -451,7 +450,7 @@ zlib_compressobj_impl(PyObject *module, int level, int method, int wbits, self->zst.zfree = PyZlib_Free; self->zst.next_in = NULL; self->zst.avail_in = 0; - err = deflateInit2(&self->zst, level, method, wbits, memLevel, strategy); + int err = deflateInit2(&self->zst, level, method, wbits, memLevel, strategy); switch (err) { case Z_OK: self->is_initialised = 1; @@ -479,7 +478,7 @@ zlib_compressobj_impl(PyObject *module, int level, int method, int wbits, PyErr_SetString(PyExc_ValueError, "Invalid initialization option"); goto error; default: - zlib_error(self->zst, err, "while creating compression object"); + zlib_error(state, self->zst, err, "while creating compression object"); goto error; } @@ -490,11 +489,9 @@ zlib_compressobj_impl(PyObject *module, int level, int method, int wbits, } static int -set_inflate_zdict(compobject *self) +set_inflate_zdict(zlibstate *state, compobject *self) { Py_buffer zdict_buf; - int err; - if (PyObject_GetBuffer(self->zdict, &zdict_buf, PyBUF_SIMPLE) == -1) { return -1; } @@ -504,11 +501,12 @@ set_inflate_zdict(compobject *self) PyBuffer_Release(&zdict_buf); return -1; } + int err; err = inflateSetDictionary(&self->zst, zdict_buf.buf, (unsigned int)zdict_buf.len); PyBuffer_Release(&zdict_buf); if (err != Z_OK) { - zlib_error(self->zst, err, "while setting zdict"); + zlib_error(state, self->zst, err, "while setting zdict"); return -1; } return 0; @@ -530,8 +528,7 @@ static PyObject * zlib_decompressobj_impl(PyObject *module, int wbits, PyObject *zdict) /*[clinic end generated code: output=3069b99994f36906 input=d3832b8511fc977b]*/ { - int err; - compobject *self; + zlibstate *state = get_zlib_state(module); if (zdict != NULL && !PyObject_CheckBuffer(zdict)) { PyErr_SetString(PyExc_TypeError, @@ -539,7 +536,7 @@ zlib_decompressobj_impl(PyObject *module, int wbits, PyObject *zdict) return NULL; } - self = newcompobject(_zlibstate_global->Decomptype); + compobject *self = newcompobject(state->Decomptype); if (self == NULL) return NULL; self->zst.opaque = NULL; @@ -551,18 +548,18 @@ zlib_decompressobj_impl(PyObject *module, int wbits, PyObject *zdict) Py_INCREF(zdict); self->zdict = zdict; } - err = inflateInit2(&self->zst, wbits); + int err = inflateInit2(&self->zst, wbits); switch (err) { case Z_OK: self->is_initialised = 1; if (self->zdict != NULL && wbits < 0) { #ifdef AT_LEAST_ZLIB_1_2_2_1 - if (set_inflate_zdict(self) < 0) { + if (set_inflate_zdict(state, self) < 0) { Py_DECREF(self); return NULL; } #else - PyErr_Format(_zlibstate_global->ZlibError, + PyErr_Format(state->ZlibError, "zlib version %s does not allow raw inflate with dictionary", ZLIB_VERSION); Py_DECREF(self); @@ -580,7 +577,7 @@ zlib_decompressobj_impl(PyObject *module, int wbits, PyObject *zdict) "Can't allocate memory for decompression object"); return NULL; default: - zlib_error(self->zst, err, "while creating decompression object"); + zlib_error(state, self->zst, err, "while creating decompression object"); Py_DECREF(self); return NULL; } @@ -617,6 +614,7 @@ Decomp_dealloc(compobject *self) /*[clinic input] zlib.Compress.compress + cls: defining_class data: Py_buffer Binary data to be compressed. / @@ -629,15 +627,18 @@ Call the flush() method to clear these buffers. [clinic start generated code]*/ static PyObject * -zlib_Compress_compress_impl(compobject *self, Py_buffer *data) -/*[clinic end generated code: output=5d5cd791cbc6a7f4 input=0d95908d6e64fab8]*/ +zlib_Compress_compress_impl(compobject *self, PyTypeObject *cls, + Py_buffer *data) +/*[clinic end generated code: output=6731b3f0ff357ca6 input=04d00f65ab01d260]*/ { PyObject *RetVal = NULL; - Py_ssize_t ibuflen, obuflen = DEF_BUF_SIZE; + Py_ssize_t obuflen = DEF_BUF_SIZE; int err; + zlibstate *state = PyType_GetModuleState(cls); + self->zst.next_in = data->buf; - ibuflen = data->len; + Py_ssize_t ibuflen = data->len; ENTER_ZLIB(self); @@ -654,7 +655,7 @@ zlib_Compress_compress_impl(compobject *self, Py_buffer *data) Py_END_ALLOW_THREADS if (err == Z_STREAM_ERROR) { - zlib_error(self->zst, err, "while compressing data"); + zlib_error(state, self->zst, err, "while compressing data"); goto error; } @@ -722,6 +723,7 @@ save_unconsumed_input(compobject *self, Py_buffer *data, int err) /*[clinic input] zlib.Decompress.decompress + cls: defining_class data: Py_buffer The binary data to decompress. / @@ -738,14 +740,19 @@ Call the flush() method to clear these buffers. [clinic start generated code]*/ static PyObject * -zlib_Decompress_decompress_impl(compobject *self, Py_buffer *data, - Py_ssize_t max_length) -/*[clinic end generated code: output=6e5173c74e710352 input=0a95d05a3bceaeaa]*/ +zlib_Decompress_decompress_impl(compobject *self, PyTypeObject *cls, + Py_buffer *data, Py_ssize_t max_length) +/*[clinic end generated code: output=b024a93c2c922d57 input=bfb37b3864cfb606]*/ { int err = Z_OK; Py_ssize_t ibuflen, obuflen = DEF_BUF_SIZE, hard_limit; PyObject *RetVal = NULL; + PyObject *module = PyType_GetModule(cls); + if (module == NULL) + return NULL; + + zlibstate *state = get_zlib_state(module); if (max_length < 0) { PyErr_SetString(PyExc_ValueError, "max_length must be non-negative"); return NULL; @@ -790,8 +797,9 @@ zlib_Decompress_decompress_impl(compobject *self, Py_buffer *data, break; default: if (err == Z_NEED_DICT && self->zdict != NULL) { - if (set_inflate_zdict(self) < 0) + if (set_inflate_zdict(state, self) < 0) { goto abort; + } else break; } @@ -815,7 +823,7 @@ zlib_Decompress_decompress_impl(compobject *self, Py_buffer *data, but there wasn't more output when we tried again, so it is not an error condition. */ - zlib_error(self->zst, err, "while decompressing data"); + zlib_error(state, self->zst, err, "while decompressing data"); goto abort; } @@ -833,6 +841,7 @@ zlib_Decompress_decompress_impl(compobject *self, Py_buffer *data, /*[clinic input] zlib.Compress.flush + cls: defining_class mode: int(c_default="Z_FINISH") = zlib.Z_FINISH One of the constants Z_SYNC_FLUSH, Z_FULL_FLUSH, Z_FINISH. If mode == Z_FINISH, the compressor object can no longer be @@ -844,13 +853,14 @@ Return a bytes object containing any remaining compressed data. [clinic start generated code]*/ static PyObject * -zlib_Compress_flush_impl(compobject *self, int mode) -/*[clinic end generated code: output=a203f4cefc9de727 input=73ed066794bd15bc]*/ +zlib_Compress_flush_impl(compobject *self, PyTypeObject *cls, int mode) +/*[clinic end generated code: output=c7efd13efd62add2 input=286146e29442eb6c]*/ { int err; Py_ssize_t length = DEF_BUF_SIZE; PyObject *RetVal = NULL; + zlibstate *state = PyType_GetModuleState(cls); /* Flushing with Z_NO_FLUSH is a no-op, so there's no point in doing any work at all; just return an empty string. */ if (mode == Z_NO_FLUSH) { @@ -873,7 +883,7 @@ zlib_Compress_flush_impl(compobject *self, int mode) Py_END_ALLOW_THREADS if (err == Z_STREAM_ERROR) { - zlib_error(self->zst, err, "while flushing"); + zlib_error(state, self->zst, err, "while flushing"); Py_CLEAR(RetVal); goto error; } @@ -886,7 +896,7 @@ zlib_Compress_flush_impl(compobject *self, int mode) if (err == Z_STREAM_END && mode == Z_FINISH) { err = deflateEnd(&self->zst); if (err != Z_OK) { - zlib_error(self->zst, err, "while finishing compression"); + zlib_error(state, self->zst, err, "while finishing compression"); Py_CLEAR(RetVal); goto error; } @@ -898,7 +908,7 @@ zlib_Compress_flush_impl(compobject *self, int mode) not an error condition. */ } else if (err != Z_OK && err != Z_BUF_ERROR) { - zlib_error(self->zst, err, "while flushing"); + zlib_error(state, self->zst, err, "while flushing"); Py_CLEAR(RetVal); goto error; } @@ -917,24 +927,25 @@ zlib_Compress_flush_impl(compobject *self, int mode) /*[clinic input] zlib.Compress.copy + cls: defining_class + Return a copy of the compression object. [clinic start generated code]*/ static PyObject * -zlib_Compress_copy_impl(compobject *self) -/*[clinic end generated code: output=5144aa153c21e805 input=c656351f94b82718]*/ +zlib_Compress_copy_impl(compobject *self, PyTypeObject *cls) +/*[clinic end generated code: output=c4d2cfb4b0d7350b input=235497e482d40986]*/ { - compobject *retval = NULL; - int err; + zlibstate *state = PyType_GetModuleState(cls); - retval = newcompobject(_zlibstate_global->Comptype); + compobject *retval = newcompobject(state->Comptype); if (!retval) return NULL; /* Copy the zstream state * We use ENTER_ZLIB / LEAVE_ZLIB to make this thread-safe */ ENTER_ZLIB(self); - err = deflateCopy(&retval->zst, &self->zst); + int err = deflateCopy(&retval->zst, &self->zst); switch (err) { case Z_OK: break; @@ -946,7 +957,7 @@ zlib_Compress_copy_impl(compobject *self) "Can't allocate memory for compression object"); goto error; default: - zlib_error(self->zst, err, "while copying compression object"); + zlib_error(state, self->zst, err, "while copying compression object"); goto error; } Py_INCREF(self->unused_data); @@ -971,51 +982,57 @@ zlib_Compress_copy_impl(compobject *self) /*[clinic input] zlib.Compress.__copy__ + + cls: defining_class + [clinic start generated code]*/ static PyObject * -zlib_Compress___copy___impl(compobject *self) -/*[clinic end generated code: output=1875e6791975442e input=be97a05a788dfd83]*/ +zlib_Compress___copy___impl(compobject *self, PyTypeObject *cls) +/*[clinic end generated code: output=074613db332cb668 input=5c0188367ab0fe64]*/ { - return zlib_Compress_copy_impl(self); + return zlib_Compress_copy_impl(self, cls); } /*[clinic input] zlib.Compress.__deepcopy__ + cls: defining_class memo: object / [clinic start generated code]*/ static PyObject * -zlib_Compress___deepcopy__(compobject *self, PyObject *memo) -/*[clinic end generated code: output=f47a2213282c9eb0 input=a9a8b0b40d83388e]*/ +zlib_Compress___deepcopy___impl(compobject *self, PyTypeObject *cls, + PyObject *memo) +/*[clinic end generated code: output=24b3aed785f54033 input=c90347319a514430]*/ { - return zlib_Compress_copy_impl(self); + return zlib_Compress_copy_impl(self, cls); } /*[clinic input] zlib.Decompress.copy + cls: defining_class + Return a copy of the decompression object. [clinic start generated code]*/ static PyObject * -zlib_Decompress_copy_impl(compobject *self) -/*[clinic end generated code: output=02a883a2a510c8cc input=ba6c3e96712a596b]*/ +zlib_Decompress_copy_impl(compobject *self, PyTypeObject *cls) +/*[clinic end generated code: output=a7ddc016e1d0a781 input=20ef3aa208282ff2]*/ { - compobject *retval = NULL; - int err; + zlibstate *state = PyType_GetModuleState(cls); - retval = newcompobject(_zlibstate_global->Decomptype); + compobject *retval = newcompobject(state->Decomptype); if (!retval) return NULL; /* Copy the zstream state * We use ENTER_ZLIB / LEAVE_ZLIB to make this thread-safe */ ENTER_ZLIB(self); - err = inflateCopy(&retval->zst, &self->zst); + int err = inflateCopy(&retval->zst, &self->zst); switch (err) { case Z_OK: break; @@ -1027,7 +1044,7 @@ zlib_Decompress_copy_impl(compobject *self) "Can't allocate memory for decompression object"); goto error; default: - zlib_error(self->zst, err, "while copying decompression object"); + zlib_error(state, self->zst, err, "while copying decompression object"); goto error; } @@ -1053,28 +1070,33 @@ zlib_Decompress_copy_impl(compobject *self) /*[clinic input] zlib.Decompress.__copy__ + + cls: defining_class + [clinic start generated code]*/ static PyObject * -zlib_Decompress___copy___impl(compobject *self) -/*[clinic end generated code: output=80bae8bc43498ad4 input=efcb98b5472c13d2]*/ +zlib_Decompress___copy___impl(compobject *self, PyTypeObject *cls) +/*[clinic end generated code: output=cf1e6473744f53fa input=cc3143067b622bdf]*/ { - return zlib_Decompress_copy_impl(self); + return zlib_Decompress_copy_impl(self, cls); } /*[clinic input] zlib.Decompress.__deepcopy__ + cls: defining_class memo: object / [clinic start generated code]*/ static PyObject * -zlib_Decompress___deepcopy__(compobject *self, PyObject *memo) -/*[clinic end generated code: output=1f77286ab490124b input=6e99bd0ac4b9cd8b]*/ +zlib_Decompress___deepcopy___impl(compobject *self, PyTypeObject *cls, + PyObject *memo) +/*[clinic end generated code: output=34f7b719a0c0d51b input=fc13b9c58622544e]*/ { - return zlib_Decompress_copy_impl(self); + return zlib_Decompress_copy_impl(self, cls); } #endif @@ -1082,6 +1104,7 @@ zlib_Decompress___deepcopy__(compobject *self, PyObject *memo) /*[clinic input] zlib.Decompress.flush + cls: defining_class length: Py_ssize_t(c_default="DEF_BUF_SIZE") = zlib.DEF_BUF_SIZE the initial size of the output buffer. / @@ -1090,21 +1113,30 @@ Return a bytes object containing any remaining decompressed data. [clinic start generated code]*/ static PyObject * -zlib_Decompress_flush_impl(compobject *self, Py_ssize_t length) -/*[clinic end generated code: output=68c75ea127cbe654 input=427f2a05a8c2113a]*/ +zlib_Decompress_flush_impl(compobject *self, PyTypeObject *cls, + Py_ssize_t length) +/*[clinic end generated code: output=4532fc280bd0f8f2 input=42f1f4b75230e2cd]*/ { int err, flush; Py_buffer data; PyObject *RetVal = NULL; Py_ssize_t ibuflen; + PyObject *module = PyType_GetModule(cls); + if (module == NULL) { + return NULL; + } + + zlibstate *state = get_zlib_state(module); + if (length <= 0) { PyErr_SetString(PyExc_ValueError, "length must be greater than zero"); return NULL; } - if (PyObject_GetBuffer(self->unconsumed_tail, &data, PyBUF_SIMPLE) == -1) + if (PyObject_GetBuffer(self->unconsumed_tail, &data, PyBUF_SIMPLE) == -1) { return NULL; + } ENTER_ZLIB(self); @@ -1131,8 +1163,9 @@ zlib_Decompress_flush_impl(compobject *self, Py_ssize_t length) break; default: if (err == Z_NEED_DICT && self->zdict != NULL) { - if (set_inflate_zdict(self) < 0) + if (set_inflate_zdict(state, self) < 0) { goto abort; + } else break; } @@ -1144,8 +1177,9 @@ zlib_Decompress_flush_impl(compobject *self, Py_ssize_t length) } while (err != Z_STREAM_END && ibuflen != 0); save: - if (save_unconsumed_input(self, &data, err) < 0) + if (save_unconsumed_input(self, &data, err) < 0) { goto abort; + } /* If at end of stream, clean up any memory allocated by zlib. */ if (err == Z_STREAM_END) { @@ -1153,14 +1187,15 @@ zlib_Decompress_flush_impl(compobject *self, Py_ssize_t length) self->is_initialised = 0; err = inflateEnd(&self->zst); if (err != Z_OK) { - zlib_error(self->zst, err, "while finishing decompression"); + zlib_error(state, self->zst, err, "while finishing decompression"); goto abort; } } if (_PyBytes_Resize(&RetVal, self->zst.next_out - - (Byte *)PyBytes_AS_STRING(RetVal)) == 0) + (Byte *)PyBytes_AS_STRING(RetVal)) == 0) { goto success; + } abort: Py_CLEAR(RetVal); @@ -1337,9 +1372,9 @@ PyDoc_STRVAR(zlib_module_documentation, "objects support decompress() and flush()."); static int -zlib_clear(PyObject *m) +zlib_clear(PyObject *mod) { - _zlibstate *state = get_zlib_state(m); + zlibstate *state = get_zlib_state(mod); Py_CLEAR(state->Comptype); Py_CLEAR(state->Decomptype); Py_CLEAR(state->ZlibError); @@ -1347,9 +1382,9 @@ zlib_clear(PyObject *m) } static int -zlib_traverse(PyObject *m, visitproc visit, void *arg) +zlib_traverse(PyObject *mod, visitproc visit, void *arg) { - _zlibstate *state = get_zlib_state(m); + zlibstate *state = get_zlib_state(mod); Py_VISIT(state->Comptype); Py_VISIT(state->Decomptype); Py_VISIT(state->ZlibError); @@ -1357,93 +1392,122 @@ zlib_traverse(PyObject *m, visitproc visit, void *arg) } static void -zlib_free(void *m) +zlib_free(void *mod) { - zlib_clear((PyObject *)m); + zlib_clear((PyObject *)mod); } -static struct PyModuleDef zlibmodule = { - PyModuleDef_HEAD_INIT, - "zlib", - zlib_module_documentation, - sizeof(_zlibstate), - zlib_methods, - NULL, - zlib_traverse, - zlib_clear, - zlib_free, -}; - -PyMODINIT_FUNC -PyInit_zlib(void) +static int +zlib_exec(PyObject *mod) { - PyObject *m, *ver; - m = PyState_FindModule(&zlibmodule); - if (m != NULL) { - Py_INCREF(m); - return m; + zlibstate *state = get_zlib_state(mod); + + state->Comptype = (PyTypeObject *)PyType_FromModuleAndSpec( + mod, &Comptype_spec, NULL); + if (state->Comptype == NULL) { + return -1; } - m = PyModule_Create(&zlibmodule); - if (m == NULL) - return NULL; - PyTypeObject *Comptype = (PyTypeObject *)PyType_FromSpec(&Comptype_spec); - if (Comptype == NULL) - return NULL; - get_zlib_state(m)->Comptype = Comptype; + state->Decomptype = (PyTypeObject *)PyType_FromModuleAndSpec( + mod, &Decomptype_spec, NULL); + if (state->Decomptype == NULL) { + return -1; + } - PyTypeObject *Decomptype = (PyTypeObject *)PyType_FromSpec(&Decomptype_spec); - if (Decomptype == NULL) - return NULL; - get_zlib_state(m)->Decomptype = Decomptype; + state->ZlibError = PyErr_NewException("zlib.error", NULL, NULL); + if (state->ZlibError == NULL) { + return -1; + } - PyObject *ZlibError = PyErr_NewException("zlib.error", NULL, NULL); - if (ZlibError != NULL) { - Py_INCREF(ZlibError); - PyModule_AddObject(m, "error", ZlibError); - get_zlib_state(m)->ZlibError = ZlibError; + Py_INCREF(state->ZlibError); + if (PyModule_AddObject(mod, "error", state->ZlibError) < 0) { + Py_DECREF(state->ZlibError); + return -1; } - PyModule_AddIntMacro(m, MAX_WBITS); - PyModule_AddIntMacro(m, DEFLATED); - PyModule_AddIntMacro(m, DEF_MEM_LEVEL); - PyModule_AddIntMacro(m, DEF_BUF_SIZE); + +#define ZLIB_ADD_INT_MACRO(c) \ + do { \ + if ((PyModule_AddIntConstant(mod, #c, c)) < 0) { \ + return -1; \ + } \ + } while(0) + + ZLIB_ADD_INT_MACRO(MAX_WBITS); + ZLIB_ADD_INT_MACRO(DEFLATED); + ZLIB_ADD_INT_MACRO(DEF_MEM_LEVEL); + ZLIB_ADD_INT_MACRO(DEF_BUF_SIZE); // compression levels - PyModule_AddIntMacro(m, Z_NO_COMPRESSION); - PyModule_AddIntMacro(m, Z_BEST_SPEED); - PyModule_AddIntMacro(m, Z_BEST_COMPRESSION); - PyModule_AddIntMacro(m, Z_DEFAULT_COMPRESSION); + ZLIB_ADD_INT_MACRO(Z_NO_COMPRESSION); + ZLIB_ADD_INT_MACRO(Z_BEST_SPEED); + ZLIB_ADD_INT_MACRO(Z_BEST_COMPRESSION); + ZLIB_ADD_INT_MACRO(Z_DEFAULT_COMPRESSION); // compression strategies - PyModule_AddIntMacro(m, Z_FILTERED); - PyModule_AddIntMacro(m, Z_HUFFMAN_ONLY); + ZLIB_ADD_INT_MACRO(Z_FILTERED); + ZLIB_ADD_INT_MACRO(Z_HUFFMAN_ONLY); #ifdef Z_RLE // 1.2.0.1 - PyModule_AddIntMacro(m, Z_RLE); + ZLIB_ADD_INT_MACRO(Z_RLE); #endif #ifdef Z_FIXED // 1.2.2.2 - PyModule_AddIntMacro(m, Z_FIXED); + ZLIB_ADD_INT_MACRO(Z_FIXED); #endif - PyModule_AddIntMacro(m, Z_DEFAULT_STRATEGY); + ZLIB_ADD_INT_MACRO(Z_DEFAULT_STRATEGY); // allowed flush values - PyModule_AddIntMacro(m, Z_NO_FLUSH); - PyModule_AddIntMacro(m, Z_PARTIAL_FLUSH); - PyModule_AddIntMacro(m, Z_SYNC_FLUSH); - PyModule_AddIntMacro(m, Z_FULL_FLUSH); - PyModule_AddIntMacro(m, Z_FINISH); + ZLIB_ADD_INT_MACRO(Z_NO_FLUSH); + ZLIB_ADD_INT_MACRO(Z_PARTIAL_FLUSH); + ZLIB_ADD_INT_MACRO(Z_SYNC_FLUSH); + ZLIB_ADD_INT_MACRO(Z_FULL_FLUSH); + ZLIB_ADD_INT_MACRO(Z_FINISH); #ifdef Z_BLOCK // 1.2.0.5 for inflate, 1.2.3.4 for deflate - PyModule_AddIntMacro(m, Z_BLOCK); + ZLIB_ADD_INT_MACRO(Z_BLOCK); #endif #ifdef Z_TREES // 1.2.3.4, only for inflate - PyModule_AddIntMacro(m, Z_TREES); + ZLIB_ADD_INT_MACRO(Z_TREES); #endif - ver = PyUnicode_FromString(ZLIB_VERSION); - if (ver != NULL) - PyModule_AddObject(m, "ZLIB_VERSION", ver); + PyObject *ver = PyUnicode_FromString(ZLIB_VERSION); + if (ver == NULL) { + return -1; + } + + if (PyModule_AddObject(mod, "ZLIB_VERSION", ver) < 0) { + Py_DECREF(ver); + return -1; + } ver = PyUnicode_FromString(zlibVersion()); - if (ver != NULL) - PyModule_AddObject(m, "ZLIB_RUNTIME_VERSION", ver); + if (ver == NULL) { + return -1; + } + + if (PyModule_AddObject(mod, "ZLIB_RUNTIME_VERSION", ver) < 0) { + Py_DECREF(ver); + return -1; + } + + if (PyModule_AddStringConstant(mod, "__version__", "1.0") < 0) { + return -1; + } + return 0; +} - PyModule_AddStringConstant(m, "__version__", "1.0"); +static PyModuleDef_Slot zlib_slots[] = { + {Py_mod_exec, zlib_exec}, + {0, NULL} +}; - PyState_AddModule(m, &zlibmodule); - return m; +static struct PyModuleDef zlibmodule = { + PyModuleDef_HEAD_INIT, + .m_name = "zlib", + .m_doc = zlib_module_documentation, + .m_size = sizeof(zlibstate), + .m_methods = zlib_methods, + .m_slots = zlib_slots, + .m_traverse = zlib_traverse, + .m_clear = zlib_clear, + .m_free = zlib_free, +}; + +PyMODINIT_FUNC +PyInit_zlib(void) +{ + return PyModuleDef_Init(&zlibmodule); } From webhook-mailer at python.org Mon Sep 7 04:49:07 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Mon, 07 Sep 2020 08:49:07 -0000 Subject: [Python-checkins] bpo-1635741: Port _opcode module to multi-phase init (PEP 489) (GH-22050) Message-ID: https://github.com/python/cpython/commit/426f2b4f13f392875e7861dbd7f34735731eff17 commit: 426f2b4f13f392875e7861dbd7f34735731eff17 branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-07T10:48:44+02:00 summary: bpo-1635741: Port _opcode module to multi-phase init (PEP 489) (GH-22050) files: A Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-06-02.bpo-1635741.5jZymK.rst M Modules/_opcode.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-06-02.bpo-1635741.5jZymK.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-06-02.bpo-1635741.5jZymK.rst new file mode 100644 index 0000000000000..c3bc9a78a2e05 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-06-02.bpo-1635741.5jZymK.rst @@ -0,0 +1,2 @@ +Port the :mod:`_opcode` extension module to multi-phase initialization +(:pep:`489`). diff --git a/Modules/_opcode.c b/Modules/_opcode.c index 42a8732694afe..d8de0762e765a 100644 --- a/Modules/_opcode.c +++ b/Modules/_opcode.c @@ -36,8 +36,9 @@ _opcode_stack_effect_impl(PyObject *module, int opcode, PyObject *oparg, return -1; } oparg_int = (int)PyLong_AsLong(oparg); - if ((oparg_int == -1) && PyErr_Occurred()) + if ((oparg_int == -1) && PyErr_Occurred()) { return -1; + } } else if (oparg != Py_None) { PyErr_SetString(PyExc_ValueError, @@ -67,30 +68,22 @@ _opcode_stack_effect_impl(PyObject *module, int opcode, PyObject *oparg, return effect; } - - - static PyMethodDef opcode_functions[] = { _OPCODE_STACK_EFFECT_METHODDEF {NULL, NULL, 0, NULL} }; - static struct PyModuleDef opcodemodule = { PyModuleDef_HEAD_INIT, - "_opcode", - "Opcode support module.", - -1, - opcode_functions, - NULL, - NULL, - NULL, - NULL + .m_name = "_opcode", + .m_doc = "Opcode support module.", + .m_size = 0, + .m_methods = opcode_functions }; PyMODINIT_FUNC PyInit__opcode(void) { - return PyModule_Create(&opcodemodule); + return PyModuleDef_Init(&opcodemodule); } From webhook-mailer at python.org Mon Sep 7 09:12:45 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Mon, 07 Sep 2020 13:12:45 -0000 Subject: [Python-checkins] bpo-1635741: Port _overlapped module to multi-phase init (GH-22051) Message-ID: https://github.com/python/cpython/commit/2aabc3200bf03d2ec1aa987e1e20db704948111e commit: 2aabc3200bf03d2ec1aa987e1e20db704948111e branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-07T15:12:40+02:00 summary: bpo-1635741: Port _overlapped module to multi-phase init (GH-22051) Port the _overlapped extension module to multi-phase initialization (PEP 489). files: A Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-22-35.bpo-1635741.CnRME3.rst M Modules/overlapped.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-22-35.bpo-1635741.CnRME3.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-22-35.bpo-1635741.CnRME3.rst new file mode 100644 index 0000000000000..76f985bb87b4e --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-22-35.bpo-1635741.CnRME3.rst @@ -0,0 +1,2 @@ +Port the :mod:`_overlapped` extension module to multi-phase initialization +(:pep:`489`). diff --git a/Modules/overlapped.c b/Modules/overlapped.c index 5e7a1bbba7678..3829932070a96 100644 --- a/Modules/overlapped.c +++ b/Modules/overlapped.c @@ -100,6 +100,19 @@ typedef struct { }; } OverlappedObject; +typedef struct { + PyTypeObject *overlapped_type; +} OverlappedState; + +static inline OverlappedState* +overlapped_get_state(PyObject *module) +{ + void *state = PyModule_GetState(module); + assert(state != NULL); + return (OverlappedState *)state; +} + + /* * Map Windows error codes to subclasses of OSError */ @@ -706,8 +719,11 @@ Overlapped_dealloc(OverlappedObject *self) } Overlapped_clear(self); - PyObject_Del(self); SetLastError(olderr); + + PyTypeObject *tp = Py_TYPE(self); + PyObject_Del(self); + Py_DECREF(tp); } @@ -1846,45 +1862,22 @@ static PyGetSetDef Overlapped_getsets[] = { {NULL}, }; -PyTypeObject OverlappedType = { - PyVarObject_HEAD_INIT(NULL, 0) - /* tp_name */ "_overlapped.Overlapped", - /* tp_basicsize */ sizeof(OverlappedObject), - /* tp_itemsize */ 0, - /* tp_dealloc */ (destructor) Overlapped_dealloc, - /* tp_vectorcall_offset */ 0, - /* tp_getattr */ 0, - /* tp_setattr */ 0, - /* tp_as_async */ 0, - /* tp_repr */ 0, - /* tp_as_number */ 0, - /* tp_as_sequence */ 0, - /* tp_as_mapping */ 0, - /* tp_hash */ 0, - /* tp_call */ 0, - /* tp_str */ 0, - /* tp_getattro */ 0, - /* tp_setattro */ 0, - /* tp_as_buffer */ 0, - /* tp_flags */ Py_TPFLAGS_DEFAULT, - /* tp_doc */ _overlapped_Overlapped__doc__, - /* tp_traverse */ (traverseproc)Overlapped_traverse, - /* tp_clear */ 0, - /* tp_richcompare */ 0, - /* tp_weaklistoffset */ 0, - /* tp_iter */ 0, - /* tp_iternext */ 0, - /* tp_methods */ Overlapped_methods, - /* tp_members */ Overlapped_members, - /* tp_getset */ Overlapped_getsets, - /* tp_base */ 0, - /* tp_dict */ 0, - /* tp_descr_get */ 0, - /* tp_descr_set */ 0, - /* tp_dictoffset */ 0, - /* tp_init */ 0, - /* tp_alloc */ 0, - /* tp_new */ _overlapped_Overlapped, +static PyType_Slot overlapped_type_slots[] = { + {Py_tp_dealloc, Overlapped_dealloc}, + {Py_tp_doc, (char *)_overlapped_Overlapped__doc__}, + {Py_tp_traverse, Overlapped_traverse}, + {Py_tp_methods, Overlapped_methods}, + {Py_tp_members, Overlapped_members}, + {Py_tp_getset, Overlapped_getsets}, + {Py_tp_new, _overlapped_Overlapped}, + {0,0} +}; + +static PyType_Spec overlapped_type_spec = { + .name = "_overlapped.Overlapped", + .basicsize = sizeof(OverlappedObject), + .flags = Py_TPFLAGS_DEFAULT, + .slots = overlapped_type_slots }; static PyMethodDef overlapped_functions[] = { @@ -1904,41 +1897,65 @@ static PyMethodDef overlapped_functions[] = { {NULL} }; -static struct PyModuleDef overlapped_module = { - PyModuleDef_HEAD_INIT, - "_overlapped", - NULL, - -1, - overlapped_functions, - NULL, - NULL, - NULL, - NULL -}; +static int +overlapped_traverse(PyObject *module, visitproc visit, void *arg) +{ + OverlappedState *state = overlapped_get_state(module); + Py_VISIT(state->overlapped_type); + return 0; +} -#define WINAPI_CONSTANT(fmt, con) \ - PyDict_SetItemString(d, #con, Py_BuildValue(fmt, con)) +static int +overlapped_clear(PyObject *module) +{ + OverlappedState *state = overlapped_get_state(module); + Py_CLEAR(state->overlapped_type); + return 0; +} -PyMODINIT_FUNC -PyInit__overlapped(void) +static void +overlapped_free(void *module) { - PyObject *m, *d; + overlapped_clear((PyObject *)module); +} +#define WINAPI_CONSTANT(fmt, con) \ + do { \ + PyObject *value = Py_BuildValue(fmt, con); \ + if (value == NULL) { \ + return -1; \ + } \ + if (PyModule_AddObject(module, #con, value) < 0 ) { \ + Py_DECREF(value); \ + return -1; \ + } \ + } while (0) + +static int +overlapped_exec(PyObject *module) +{ /* Ensure WSAStartup() called before initializing function pointers */ - m = PyImport_ImportModule("_socket"); - if (!m) - return NULL; - Py_DECREF(m); + PyObject *socket_module = PyImport_ImportModule("_socket"); + if (!socket_module) { + return -1; + } - if (initialize_function_pointers() < 0) - return NULL; + Py_DECREF(socket_module); - m = PyModule_Create(&overlapped_module); - if (PyModule_AddType(m, &OverlappedType) < 0) { - return NULL; + if (initialize_function_pointers() < 0) { + return -1; } - d = PyModule_GetDict(m); + OverlappedState *st = overlapped_get_state(module); + st->overlapped_type = (PyTypeObject *)PyType_FromModuleAndSpec( + module, &overlapped_type_spec, NULL); + if (st->overlapped_type == NULL) { + return -1; + } + + if (PyModule_AddType(module, st->overlapped_type) < 0) { + return -1; + } WINAPI_CONSTANT(F_DWORD, ERROR_IO_PENDING); WINAPI_CONSTANT(F_DWORD, ERROR_NETNAME_DELETED); @@ -1952,5 +1969,27 @@ PyInit__overlapped(void) WINAPI_CONSTANT(F_DWORD, SO_UPDATE_CONNECT_CONTEXT); WINAPI_CONSTANT(F_DWORD, TF_REUSE_SOCKET); - return m; + return 0; +} + +static PyModuleDef_Slot overlapped_slots[] = { + {Py_mod_exec, overlapped_exec}, + {0, NULL} +}; + +static struct PyModuleDef overlapped_module = { + PyModuleDef_HEAD_INIT, + .m_name = "_overlapped", + .m_size = sizeof(OverlappedState), + .m_methods = overlapped_functions, + .m_slots = overlapped_slots, + .m_traverse = overlapped_traverse, + .m_clear = overlapped_clear, + .m_free = overlapped_free +}; + +PyMODINIT_FUNC +PyInit__overlapped(void) +{ + return PyModuleDef_Init(&overlapped_module); } From webhook-mailer at python.org Mon Sep 7 11:14:36 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Mon, 07 Sep 2020 15:14:36 -0000 Subject: [Python-checkins] bpo-1635741 port _curses_panel to multi-phase init (PEP 489) (GH-21986) Message-ID: https://github.com/python/cpython/commit/1baf030a902392fe92d934ed0fb6a385cf7d8869 commit: 1baf030a902392fe92d934ed0fb6a385cf7d8869 branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-07T17:14:25+02:00 summary: bpo-1635741 port _curses_panel to multi-phase init (PEP 489) (GH-21986) files: A Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-08-07.bpo-1635741.X9CZgo.rst M Modules/_curses_panel.c M Modules/clinic/_curses_panel.c.h diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-08-07.bpo-1635741.X9CZgo.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-08-07.bpo-1635741.X9CZgo.rst new file mode 100644 index 0000000000000..a39673a26307a --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-01-17-08-07.bpo-1635741.X9CZgo.rst @@ -0,0 +1,2 @@ +Port the :mod:`_curses_panel` extension module to multi-phase initialization +(:pep:`489`). diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c index f124803493d88..1a8f0b636821f 100644 --- a/Modules/_curses_panel.c +++ b/Modules/_curses_panel.c @@ -18,43 +18,42 @@ static const char PyCursesVersion[] = "2.1"; typedef struct { PyObject *PyCursesError; - PyObject *PyCursesPanel_Type; -} _curses_panelstate; + PyTypeObject *PyCursesPanel_Type; +} _curses_panel_state; -static inline _curses_panelstate* -get_curses_panelstate(PyObject *module) +static inline _curses_panel_state * +get_curses_panel_state(PyObject *module) { void *state = PyModule_GetState(module); assert(state != NULL); - return (_curses_panelstate *)state; + return (_curses_panel_state *)state; } static int -_curses_panel_clear(PyObject *m) +_curses_panel_clear(PyObject *mod) { - Py_CLEAR(get_curses_panelstate(m)->PyCursesError); + _curses_panel_state *state = get_curses_panel_state(mod); + Py_CLEAR(state->PyCursesError); + Py_CLEAR(state->PyCursesPanel_Type); return 0; } static int -_curses_panel_traverse(PyObject *m, visitproc visit, void *arg) +_curses_panel_traverse(PyObject *mod, visitproc visit, void *arg) { - Py_VISIT(Py_TYPE(m)); - Py_VISIT(get_curses_panelstate(m)->PyCursesError); + Py_VISIT(Py_TYPE(mod)); + _curses_panel_state *state = get_curses_panel_state(mod); + Py_VISIT(state->PyCursesError); + Py_VISIT(state->PyCursesPanel_Type); return 0; } static void -_curses_panel_free(void *m) +_curses_panel_free(void *mod) { - _curses_panel_clear((PyObject *) m); + _curses_panel_clear((PyObject *) mod); } -static struct PyModuleDef _curses_panelmodule; - -#define _curses_panelstate_global \ -((_curses_panelstate *) PyModule_GetState(PyState_FindModule(&_curses_panelmodule))) - /* Utility Functions */ /* @@ -63,15 +62,17 @@ static struct PyModuleDef _curses_panelmodule; */ static PyObject * -PyCursesCheckERR(int code, const char *fname) +PyCursesCheckERR(_curses_panel_state *state, int code, const char *fname) { if (code != ERR) { Py_RETURN_NONE; - } else { + } + else { if (fname == NULL) { - PyErr_SetString(_curses_panelstate_global->PyCursesError, catchall_ERR); - } else { - PyErr_Format(_curses_panelstate_global->PyCursesError, "%s() returned ERR", fname); + PyErr_SetString(state->PyCursesError, catchall_ERR); + } + else { + PyErr_Format(state->PyCursesError, "%s() returned ERR", fname); } return NULL; } @@ -89,9 +90,6 @@ typedef struct { PyCursesWindowObject *wo; /* for reference counts */ } PyCursesPanelObject; -#define PyCursesPanel_Check(v) \ - Py_IS_TYPE(v, _curses_panelstate_global->PyCursesPanel_Type) - /* Some helper functions. The problem is that there's always a window associated with a panel. To ensure that Python's GC doesn't pull this window from under our feet we need to keep track of references @@ -182,67 +180,81 @@ class _curses_panel.panel "PyCursesPanelObject *" "&PyCursesPanel_Type" /*[clinic input] _curses_panel.panel.bottom + cls: defining_class + Push the panel to the bottom of the stack. [clinic start generated code]*/ static PyObject * -_curses_panel_panel_bottom_impl(PyCursesPanelObject *self) -/*[clinic end generated code: output=7aa7d14d7e1d1ce6 input=b6c920c071b61e2e]*/ +_curses_panel_panel_bottom_impl(PyCursesPanelObject *self, PyTypeObject *cls) +/*[clinic end generated code: output=8ec7fbbc08554021 input=6b7d2c0578b5a1c4]*/ { - return PyCursesCheckERR(bottom_panel(self->pan), "bottom"); + _curses_panel_state *state = PyType_GetModuleState(cls); + return PyCursesCheckERR(state, bottom_panel(self->pan), "bottom"); } /*[clinic input] _curses_panel.panel.hide + cls: defining_class + Hide the panel. This does not delete the object, it just makes the window on screen invisible. [clinic start generated code]*/ static PyObject * -_curses_panel_panel_hide_impl(PyCursesPanelObject *self) -/*[clinic end generated code: output=a7bbbd523e1eab49 input=f6ab884e99386118]*/ +_curses_panel_panel_hide_impl(PyCursesPanelObject *self, PyTypeObject *cls) +/*[clinic end generated code: output=cc6ab7203cdc1450 input=1bfc741f473e6055]*/ { - return PyCursesCheckERR(hide_panel(self->pan), "hide"); + _curses_panel_state *state = PyType_GetModuleState(cls); + return PyCursesCheckERR(state, hide_panel(self->pan), "hide"); } /*[clinic input] _curses_panel.panel.show + cls: defining_class + Display the panel (which might have been hidden). [clinic start generated code]*/ static PyObject * -_curses_panel_panel_show_impl(PyCursesPanelObject *self) -/*[clinic end generated code: output=6b4553ab45c97769 input=57b167bbefaa3755]*/ +_curses_panel_panel_show_impl(PyCursesPanelObject *self, PyTypeObject *cls) +/*[clinic end generated code: output=dc3421de375f0409 input=8122e80151cb4379]*/ { - return PyCursesCheckERR(show_panel(self->pan), "show"); + _curses_panel_state *state = PyType_GetModuleState(cls); + return PyCursesCheckERR(state, show_panel(self->pan), "show"); } /*[clinic input] _curses_panel.panel.top + cls: defining_class + Push panel to the top of the stack. [clinic start generated code]*/ static PyObject * -_curses_panel_panel_top_impl(PyCursesPanelObject *self) -/*[clinic end generated code: output=0f5f2f8cdd2d1777 input=be33975ec3ca0e9a]*/ +_curses_panel_panel_top_impl(PyCursesPanelObject *self, PyTypeObject *cls) +/*[clinic end generated code: output=10a072e511e873f7 input=1f372d597dda3379]*/ { - return PyCursesCheckERR(top_panel(self->pan), "top"); + _curses_panel_state *state = PyType_GetModuleState(cls); + return PyCursesCheckERR(state, top_panel(self->pan), "top"); } /* Allocation and deallocation of Panel Objects */ static PyObject * -PyCursesPanel_New(PANEL *pan, PyCursesWindowObject *wo) +PyCursesPanel_New(_curses_panel_state *state, PANEL *pan, + PyCursesWindowObject *wo) { - PyCursesPanelObject *po; + PyCursesPanelObject *po = PyObject_New(PyCursesPanelObject, + state->PyCursesPanel_Type); + if (po == NULL) { + return NULL; + } - po = PyObject_New(PyCursesPanelObject, - (PyTypeObject *)(_curses_panelstate_global)->PyCursesPanel_Type); - if (po == NULL) return NULL; po->pan = pan; if (insert_lop(po) < 0) { po->wo = NULL; @@ -355,6 +367,7 @@ _curses_panel_panel_hidden_impl(PyCursesPanelObject *self) /*[clinic input] _curses_panel.panel.move + cls: defining_class y: int x: int / @@ -363,10 +376,12 @@ Move the panel to the screen coordinates (y, x). [clinic start generated code]*/ static PyObject * -_curses_panel_panel_move_impl(PyCursesPanelObject *self, int y, int x) -/*[clinic end generated code: output=d867535a89777415 input=e0b36b78acc03fba]*/ +_curses_panel_panel_move_impl(PyCursesPanelObject *self, PyTypeObject *cls, + int y, int x) +/*[clinic end generated code: output=ce546c93e56867da input=60a0e7912ff99849]*/ { - return PyCursesCheckERR(move_panel(self->pan, y, x), "move_panel"); + _curses_panel_state *state = PyType_GetModuleState(cls); + return PyCursesCheckERR(state, move_panel(self->pan, y, x), "move_panel"); } /*[clinic input] @@ -386,6 +401,7 @@ _curses_panel_panel_window_impl(PyCursesPanelObject *self) /*[clinic input] _curses_panel.panel.replace + cls: defining_class win: object(type="PyCursesWindowObject *", subclass_of="&PyCursesWindow_Type") / @@ -394,22 +410,22 @@ Change the window associated with the panel to the window win. static PyObject * _curses_panel_panel_replace_impl(PyCursesPanelObject *self, + PyTypeObject *cls, PyCursesWindowObject *win) -/*[clinic end generated code: output=2253a95f7b287255 input=4b1c4283987d9dfa]*/ +/*[clinic end generated code: output=c71f95c212d58ae7 input=dbec7180ece41ff5]*/ { - PyCursesPanelObject *po; - int rtn; + _curses_panel_state *state = PyType_GetModuleState(cls); - po = find_po(self->pan); + PyCursesPanelObject *po = find_po(self->pan); if (po == NULL) { PyErr_SetString(PyExc_RuntimeError, "replace_panel: can't find Panel Object"); return NULL; } - rtn = replace_panel(self->pan, win->win); + int rtn = replace_panel(self->pan, win->win); if (rtn == ERR) { - PyErr_SetString(_curses_panelstate_global->PyCursesError, "replace_panel() returned ERR"); + PyErr_SetString(state->PyCursesError, "replace_panel() returned ERR"); return NULL; } Py_INCREF(win); @@ -420,6 +436,7 @@ _curses_panel_panel_replace_impl(PyCursesPanelObject *self, /*[clinic input] _curses_panel.panel.set_userptr + cls: defining_class obj: object / @@ -427,38 +444,43 @@ Set the panel's user pointer to obj. [clinic start generated code]*/ static PyObject * -_curses_panel_panel_set_userptr(PyCursesPanelObject *self, PyObject *obj) -/*[clinic end generated code: output=6fb145b3af88cf4a input=d2c6a9dbefabbf39]*/ +_curses_panel_panel_set_userptr_impl(PyCursesPanelObject *self, + PyTypeObject *cls, PyObject *obj) +/*[clinic end generated code: output=db74f3db07b28080 input=e3fee2ff7b1b8e48]*/ { - PyObject *oldobj; - int rc; PyCursesInitialised; Py_INCREF(obj); - oldobj = (PyObject *) panel_userptr(self->pan); - rc = set_panel_userptr(self->pan, (void*)obj); + PyObject *oldobj = (PyObject *) panel_userptr(self->pan); + int rc = set_panel_userptr(self->pan, (void*)obj); if (rc == ERR) { /* In case of an ncurses error, decref the new object again */ Py_DECREF(obj); } Py_XDECREF(oldobj); - return PyCursesCheckERR(rc, "set_panel_userptr"); + + _curses_panel_state *state = PyType_GetModuleState(cls); + return PyCursesCheckERR(state, rc, "set_panel_userptr"); } /*[clinic input] _curses_panel.panel.userptr + cls: defining_class + Return the user pointer for the panel. [clinic start generated code]*/ static PyObject * -_curses_panel_panel_userptr_impl(PyCursesPanelObject *self) -/*[clinic end generated code: output=e849c307b5dc9237 input=f78b7a47aef0fd50]*/ +_curses_panel_panel_userptr_impl(PyCursesPanelObject *self, + PyTypeObject *cls) +/*[clinic end generated code: output=eea6e6f39ffc0179 input=f22ca4f115e30a80]*/ { - PyObject *obj; + _curses_panel_state *state = PyType_GetModuleState(cls); + PyCursesInitialised; - obj = (PyObject *) panel_userptr(self->pan); + PyObject *obj = (PyObject *) panel_userptr(self->pan); if (obj == NULL) { - PyErr_SetString(_curses_panelstate_global->PyCursesError, "no userptr set"); + PyErr_SetString(state->PyCursesError, "no userptr set"); return NULL; } @@ -494,11 +516,10 @@ static PyType_Slot PyCursesPanel_Type_slots[] = { }; static PyType_Spec PyCursesPanel_Type_spec = { - "_curses_panel.panel", - sizeof(PyCursesPanelObject), - 0, - Py_TPFLAGS_DEFAULT, - PyCursesPanel_Type_slots + .name = "_curses_panel.panel", + .basicsize = sizeof(PyCursesPanelObject), + .flags = Py_TPFLAGS_DEFAULT, + .slots = PyCursesPanel_Type_slots }; /* Wrapper for panel_above(NULL). This function returns the bottom @@ -549,12 +570,14 @@ static PyObject * _curses_panel_new_panel_impl(PyObject *module, PyCursesWindowObject *win) /*[clinic end generated code: output=45e948e0176a9bd2 input=74d4754e0ebe4800]*/ { + _curses_panel_state *state = get_curses_panel_state(module); + PANEL *pan = new_panel(win->win); if (pan == NULL) { - PyErr_SetString(_curses_panelstate_global->PyCursesError, catchall_NULL); + PyErr_SetString(state->PyCursesError, catchall_NULL); return NULL; } - return (PyObject *)PyCursesPanel_New(pan, win); + return (PyObject *)PyCursesPanel_New(state, pan, win); } @@ -610,7 +633,6 @@ _curses_panel_update_panels_impl(PyObject *module) Py_RETURN_NONE; } - /* List of functions defined in the module */ static PyMethodDef PyCurses_methods[] = { @@ -622,57 +644,75 @@ static PyMethodDef PyCurses_methods[] = { }; /* Initialization function for the module */ - - -static struct PyModuleDef _curses_panelmodule = { - PyModuleDef_HEAD_INIT, - "_curses_panel", - NULL, - sizeof(_curses_panelstate), - PyCurses_methods, - NULL, - _curses_panel_traverse, - _curses_panel_clear, - _curses_panel_free -}; - -PyMODINIT_FUNC -PyInit__curses_panel(void) +static int +_curses_panel_exec(PyObject *mod) { - PyObject *m, *d, *v; - - /* Create the module and add the functions */ - m = PyModule_Create(&_curses_panelmodule); - if (m == NULL) - goto fail; - d = PyModule_GetDict(m); - + _curses_panel_state *state = get_curses_panel_state(mod); /* Initialize object type */ - v = PyType_FromSpec(&PyCursesPanel_Type_spec); - if (v == NULL) - goto fail; - ((PyTypeObject *)v)->tp_new = NULL; - get_curses_panelstate(m)->PyCursesPanel_Type = v; + state->PyCursesPanel_Type = (PyTypeObject *)PyType_FromModuleAndSpec( + mod, &PyCursesPanel_Type_spec, NULL); + if (state->PyCursesPanel_Type == NULL) { + return -1; + } + + if (PyModule_AddType(mod, state->PyCursesPanel_Type) < 0) { + return -1; + } import_curses(); - if (PyErr_Occurred()) - goto fail; + if (PyErr_Occurred()) { + return -1; + } /* For exception _curses_panel.error */ - get_curses_panelstate(m)->PyCursesError = PyErr_NewException("_curses_panel.error", NULL, NULL); - PyDict_SetItemString(d, "error", get_curses_panelstate(m)->PyCursesError); + state->PyCursesError = PyErr_NewException( + "_curses_panel.error", NULL, NULL); + + Py_INCREF(state->PyCursesError); + if (PyModule_AddObject(mod, "error", state->PyCursesError) < 0) { + Py_DECREF(state->PyCursesError); + return -1; + } /* Make the version available */ - v = PyUnicode_FromString(PyCursesVersion); - PyDict_SetItemString(d, "version", v); - PyDict_SetItemString(d, "__version__", v); + PyObject *v = PyUnicode_FromString(PyCursesVersion); + if (v == NULL) { + return -1; + } + + PyObject *d = PyModule_GetDict(mod); + if (PyDict_SetItemString(d, "version", v) < 0) { + Py_DECREF(v); + return -1; + } + if (PyDict_SetItemString(d, "__version__", v) < 0) { + Py_DECREF(v); + return -1; + } + Py_DECREF(v); - Py_INCREF(get_curses_panelstate(m)->PyCursesPanel_Type); - PyModule_AddObject(m, "panel", - (PyObject *)get_curses_panelstate(m)->PyCursesPanel_Type); - return m; - fail: - Py_XDECREF(m); - return NULL; + return 0; } + +static PyModuleDef_Slot _curses_slots[] = { + {Py_mod_exec, _curses_panel_exec}, + {0, NULL} +}; + +static struct PyModuleDef _curses_panelmodule = { + PyModuleDef_HEAD_INIT, + .m_name = "_curses_panel", + .m_size = sizeof(_curses_panel_state), + .m_methods = PyCurses_methods, + .m_slots = _curses_slots, + .m_traverse = _curses_panel_traverse, + .m_clear = _curses_panel_clear, + .m_free = _curses_panel_free +}; + +PyMODINIT_FUNC +PyInit__curses_panel(void) +{ + return PyModuleDef_Init(&_curses_panelmodule); +} \ No newline at end of file diff --git a/Modules/clinic/_curses_panel.c.h b/Modules/clinic/_curses_panel.c.h index cff274657658a..45898070b1f54 100644 --- a/Modules/clinic/_curses_panel.c.h +++ b/Modules/clinic/_curses_panel.c.h @@ -9,15 +9,26 @@ PyDoc_STRVAR(_curses_panel_panel_bottom__doc__, "Push the panel to the bottom of the stack."); #define _CURSES_PANEL_PANEL_BOTTOM_METHODDEF \ - {"bottom", (PyCFunction)_curses_panel_panel_bottom, METH_NOARGS, _curses_panel_panel_bottom__doc__}, + {"bottom", (PyCFunction)(void(*)(void))_curses_panel_panel_bottom, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _curses_panel_panel_bottom__doc__}, static PyObject * -_curses_panel_panel_bottom_impl(PyCursesPanelObject *self); +_curses_panel_panel_bottom_impl(PyCursesPanelObject *self, PyTypeObject *cls); static PyObject * -_curses_panel_panel_bottom(PyCursesPanelObject *self, PyObject *Py_UNUSED(ignored)) +_curses_panel_panel_bottom(PyCursesPanelObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return _curses_panel_panel_bottom_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":bottom", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = _curses_panel_panel_bottom_impl(self, cls); + +exit: + return return_value; } PyDoc_STRVAR(_curses_panel_panel_hide__doc__, @@ -29,15 +40,26 @@ PyDoc_STRVAR(_curses_panel_panel_hide__doc__, "This does not delete the object, it just makes the window on screen invisible."); #define _CURSES_PANEL_PANEL_HIDE_METHODDEF \ - {"hide", (PyCFunction)_curses_panel_panel_hide, METH_NOARGS, _curses_panel_panel_hide__doc__}, + {"hide", (PyCFunction)(void(*)(void))_curses_panel_panel_hide, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _curses_panel_panel_hide__doc__}, static PyObject * -_curses_panel_panel_hide_impl(PyCursesPanelObject *self); +_curses_panel_panel_hide_impl(PyCursesPanelObject *self, PyTypeObject *cls); static PyObject * -_curses_panel_panel_hide(PyCursesPanelObject *self, PyObject *Py_UNUSED(ignored)) +_curses_panel_panel_hide(PyCursesPanelObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return _curses_panel_panel_hide_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":hide", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = _curses_panel_panel_hide_impl(self, cls); + +exit: + return return_value; } PyDoc_STRVAR(_curses_panel_panel_show__doc__, @@ -47,15 +69,26 @@ PyDoc_STRVAR(_curses_panel_panel_show__doc__, "Display the panel (which might have been hidden)."); #define _CURSES_PANEL_PANEL_SHOW_METHODDEF \ - {"show", (PyCFunction)_curses_panel_panel_show, METH_NOARGS, _curses_panel_panel_show__doc__}, + {"show", (PyCFunction)(void(*)(void))_curses_panel_panel_show, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _curses_panel_panel_show__doc__}, static PyObject * -_curses_panel_panel_show_impl(PyCursesPanelObject *self); +_curses_panel_panel_show_impl(PyCursesPanelObject *self, PyTypeObject *cls); static PyObject * -_curses_panel_panel_show(PyCursesPanelObject *self, PyObject *Py_UNUSED(ignored)) +_curses_panel_panel_show(PyCursesPanelObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return _curses_panel_panel_show_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":show", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = _curses_panel_panel_show_impl(self, cls); + +exit: + return return_value; } PyDoc_STRVAR(_curses_panel_panel_top__doc__, @@ -65,15 +98,26 @@ PyDoc_STRVAR(_curses_panel_panel_top__doc__, "Push panel to the top of the stack."); #define _CURSES_PANEL_PANEL_TOP_METHODDEF \ - {"top", (PyCFunction)_curses_panel_panel_top, METH_NOARGS, _curses_panel_panel_top__doc__}, + {"top", (PyCFunction)(void(*)(void))_curses_panel_panel_top, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _curses_panel_panel_top__doc__}, static PyObject * -_curses_panel_panel_top_impl(PyCursesPanelObject *self); +_curses_panel_panel_top_impl(PyCursesPanelObject *self, PyTypeObject *cls); static PyObject * -_curses_panel_panel_top(PyCursesPanelObject *self, PyObject *Py_UNUSED(ignored)) +_curses_panel_panel_top(PyCursesPanelObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return _curses_panel_panel_top_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":top", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = _curses_panel_panel_top_impl(self, cls); + +exit: + return return_value; } PyDoc_STRVAR(_curses_panel_panel_above__doc__, @@ -137,30 +181,26 @@ PyDoc_STRVAR(_curses_panel_panel_move__doc__, "Move the panel to the screen coordinates (y, x)."); #define _CURSES_PANEL_PANEL_MOVE_METHODDEF \ - {"move", (PyCFunction)(void(*)(void))_curses_panel_panel_move, METH_FASTCALL, _curses_panel_panel_move__doc__}, + {"move", (PyCFunction)(void(*)(void))_curses_panel_panel_move, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _curses_panel_panel_move__doc__}, static PyObject * -_curses_panel_panel_move_impl(PyCursesPanelObject *self, int y, int x); +_curses_panel_panel_move_impl(PyCursesPanelObject *self, PyTypeObject *cls, + int y, int x); static PyObject * -_curses_panel_panel_move(PyCursesPanelObject *self, PyObject *const *args, Py_ssize_t nargs) +_curses_panel_panel_move(PyCursesPanelObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"", "", NULL}; + static _PyArg_Parser _parser = {"ii:move", _keywords, 0}; int y; int x; - if (!_PyArg_CheckPositional("move", nargs, 2, 2)) { - goto exit; - } - y = _PyLong_AsInt(args[0]); - if (y == -1 && PyErr_Occurred()) { + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &y, &x)) { goto exit; } - x = _PyLong_AsInt(args[1]); - if (x == -1 && PyErr_Occurred()) { - goto exit; - } - return_value = _curses_panel_panel_move_impl(self, y, x); + return_value = _curses_panel_panel_move_impl(self, cls, y, x); exit: return return_value; @@ -191,24 +231,26 @@ PyDoc_STRVAR(_curses_panel_panel_replace__doc__, "Change the window associated with the panel to the window win."); #define _CURSES_PANEL_PANEL_REPLACE_METHODDEF \ - {"replace", (PyCFunction)_curses_panel_panel_replace, METH_O, _curses_panel_panel_replace__doc__}, + {"replace", (PyCFunction)(void(*)(void))_curses_panel_panel_replace, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _curses_panel_panel_replace__doc__}, static PyObject * _curses_panel_panel_replace_impl(PyCursesPanelObject *self, + PyTypeObject *cls, PyCursesWindowObject *win); static PyObject * -_curses_panel_panel_replace(PyCursesPanelObject *self, PyObject *arg) +_curses_panel_panel_replace(PyCursesPanelObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { PyObject *return_value = NULL; + static const char * const _keywords[] = {"", NULL}; + static _PyArg_Parser _parser = {"O!:replace", _keywords, 0}; PyCursesWindowObject *win; - if (!PyObject_TypeCheck(arg, &PyCursesWindow_Type)) { - _PyArg_BadArgument("replace", "argument", (&PyCursesWindow_Type)->tp_name, arg); + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &PyCursesWindow_Type, &win)) { goto exit; } - win = (PyCursesWindowObject *)arg; - return_value = _curses_panel_panel_replace_impl(self, win); + return_value = _curses_panel_panel_replace_impl(self, cls, win); exit: return return_value; @@ -221,7 +263,29 @@ PyDoc_STRVAR(_curses_panel_panel_set_userptr__doc__, "Set the panel\'s user pointer to obj."); #define _CURSES_PANEL_PANEL_SET_USERPTR_METHODDEF \ - {"set_userptr", (PyCFunction)_curses_panel_panel_set_userptr, METH_O, _curses_panel_panel_set_userptr__doc__}, + {"set_userptr", (PyCFunction)(void(*)(void))_curses_panel_panel_set_userptr, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _curses_panel_panel_set_userptr__doc__}, + +static PyObject * +_curses_panel_panel_set_userptr_impl(PyCursesPanelObject *self, + PyTypeObject *cls, PyObject *obj); + +static PyObject * +_curses_panel_panel_set_userptr(PyCursesPanelObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) +{ + PyObject *return_value = NULL; + static const char * const _keywords[] = {"", NULL}; + static _PyArg_Parser _parser = {"O:set_userptr", _keywords, 0}; + PyObject *obj; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser, + &obj)) { + goto exit; + } + return_value = _curses_panel_panel_set_userptr_impl(self, cls, obj); + +exit: + return return_value; +} PyDoc_STRVAR(_curses_panel_panel_userptr__doc__, "userptr($self, /)\n" @@ -230,15 +294,27 @@ PyDoc_STRVAR(_curses_panel_panel_userptr__doc__, "Return the user pointer for the panel."); #define _CURSES_PANEL_PANEL_USERPTR_METHODDEF \ - {"userptr", (PyCFunction)_curses_panel_panel_userptr, METH_NOARGS, _curses_panel_panel_userptr__doc__}, + {"userptr", (PyCFunction)(void(*)(void))_curses_panel_panel_userptr, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _curses_panel_panel_userptr__doc__}, static PyObject * -_curses_panel_panel_userptr_impl(PyCursesPanelObject *self); +_curses_panel_panel_userptr_impl(PyCursesPanelObject *self, + PyTypeObject *cls); static PyObject * -_curses_panel_panel_userptr(PyCursesPanelObject *self, PyObject *Py_UNUSED(ignored)) +_curses_panel_panel_userptr(PyCursesPanelObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return _curses_panel_panel_userptr_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":userptr", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = _curses_panel_panel_userptr_impl(self, cls); + +exit: + return return_value; } PyDoc_STRVAR(_curses_panel_bottom_panel__doc__, @@ -325,4 +401,4 @@ _curses_panel_update_panels(PyObject *module, PyObject *Py_UNUSED(ignored)) { return _curses_panel_update_panels_impl(module); } -/*[clinic end generated code: output=1226d5f94361ebfb input=a9049054013a1b77]*/ +/*[clinic end generated code: output=3081ef24e5560cb0 input=a9049054013a1b77]*/ From webhook-mailer at python.org Mon Sep 7 11:27:26 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Mon, 07 Sep 2020 15:27:26 -0000 Subject: [Python-checkins] bpo-41401: Fix test_fspath_support in test_io. (GH-21640) (GH-22133) (GH-22135) Message-ID: https://github.com/python/cpython/commit/84105cbaa395923e3584a87d67ccce72c8420bb4 commit: 84105cbaa395923e3584a87d67ccce72c8420bb4 branch: 3.8 author: Victor Stinner committer: GitHub date: 2020-09-07T17:27:03+02:00 summary: bpo-41401: Fix test_fspath_support in test_io. (GH-21640) (GH-22133) (GH-22135) The error is exposed on non-UTF-8 locales. (cherry picked from commit 67987acd5dc9776f55f4e139e2b3d9e7a6434d9f) Co-authored-by: Serhiy Storchaka (cherry picked from commit c73ee5acc96b4bbd6885156883b224b8cc3e470c) files: M Lib/test/test_io.py diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py index bea4342e7f0f9..8eef332b8e4b5 100644 --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -922,7 +922,7 @@ def check_path_succeeds(path): self.assertEqual(f.read(), "egg\n") check_path_succeeds(FakePath(support.TESTFN)) - check_path_succeeds(FakePath(support.TESTFN.encode('utf-8'))) + check_path_succeeds(FakePath(os.fsencode(support.TESTFN))) with self.open(support.TESTFN, "w") as f: bad_path = FakePath(f.fileno()) From webhook-mailer at python.org Mon Sep 7 11:55:30 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Mon, 07 Sep 2020 15:55:30 -0000 Subject: [Python-checkins] bpo-41720: Add "return NotImplemented" in turtle.Vec2D.__rmul__(). (GH-22092) Message-ID: https://github.com/python/cpython/commit/fd4cafd4700dc03cb05fc2e5263c2666d785d6e3 commit: fd4cafd4700dc03cb05fc2e5263c2666d785d6e3 branch: master author: Serhiy Storchaka committer: GitHub date: 2020-09-07T18:55:22+03:00 summary: bpo-41720: Add "return NotImplemented" in turtle.Vec2D.__rmul__(). (GH-22092) files: A Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst M Lib/test/test_turtle.py M Lib/turtle.py diff --git a/Lib/test/test_turtle.py b/Lib/test/test_turtle.py index 46ff4a3aac709..86d6507514475 100644 --- a/Lib/test/test_turtle.py +++ b/Lib/test/test_turtle.py @@ -130,6 +130,14 @@ def assertVectorsAlmostEqual(self, vec1, vec2): self.assertAlmostEqual( i, j, msg='values at index {} do not match'.format(idx)) +class Multiplier: + + def __mul__(self, other): + return f'M*{other}' + + def __rmul__(self, other): + return f'{other}*M' + class TestVec2D(VectorComparisonMixin, unittest.TestCase): @@ -211,9 +219,15 @@ def test_vector_multiply(self): self.assertAlmostEqual(answer, expected) vec = Vec2D(0.5, 3) - answer = vec * 10 expected = Vec2D(5, 30) - self.assertVectorsAlmostEqual(answer, expected) + self.assertVectorsAlmostEqual(vec * 10, expected) + self.assertVectorsAlmostEqual(10 * vec, expected) + self.assertVectorsAlmostEqual(vec * 10.0, expected) + self.assertVectorsAlmostEqual(10.0 * vec, expected) + + M = Multiplier() + self.assertEqual(vec * M, Vec2D(f"{vec[0]}*M", f"{vec[1]}*M")) + self.assertEqual(M * vec, f'M*{vec}') def test_vector_negative(self): vec = Vec2D(10, -10) diff --git a/Lib/turtle.py b/Lib/turtle.py index 92d4e5dda9c2d..81cfcfe8a7014 100644 --- a/Lib/turtle.py +++ b/Lib/turtle.py @@ -258,6 +258,7 @@ def __mul__(self, other): def __rmul__(self, other): if isinstance(other, int) or isinstance(other, float): return Vec2D(self[0]*other, self[1]*other) + return NotImplemented def __sub__(self, other): return Vec2D(self[0]-other[0], self[1]-other[1]) def __neg__(self): diff --git a/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst b/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst new file mode 100644 index 0000000000000..5d2a5094ddeaa --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst @@ -0,0 +1,2 @@ +Fixed :meth:`turtle.Vec2D.__rmul__` for arguments which are not int or +float. From webhook-mailer at python.org Mon Sep 7 12:28:09 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 07 Sep 2020 16:28:09 -0000 Subject: [Python-checkins] bpo-41720: Add "return NotImplemented" in turtle.Vec2D.__rmul__(). (GH-22092) Message-ID: https://github.com/python/cpython/commit/ebef6c0b50d94ca7daef6fdb38d043e0183ab7d1 commit: ebef6c0b50d94ca7daef6fdb38d043e0183ab7d1 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-07T09:28:04-07:00 summary: bpo-41720: Add "return NotImplemented" in turtle.Vec2D.__rmul__(). (GH-22092) (cherry picked from commit fd4cafd4700dc03cb05fc2e5263c2666d785d6e3) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst M Lib/test/test_turtle.py M Lib/turtle.py diff --git a/Lib/test/test_turtle.py b/Lib/test/test_turtle.py index 2fd10ccd505fe..571981fc6b6e5 100644 --- a/Lib/test/test_turtle.py +++ b/Lib/test/test_turtle.py @@ -127,6 +127,14 @@ def assertVectorsAlmostEqual(self, vec1, vec2): self.assertAlmostEqual( i, j, msg='values at index {} do not match'.format(idx)) +class Multiplier: + + def __mul__(self, other): + return f'M*{other}' + + def __rmul__(self, other): + return f'{other}*M' + class TestVec2D(VectorComparisonMixin, unittest.TestCase): @@ -208,9 +216,15 @@ def test_vector_multiply(self): self.assertAlmostEqual(answer, expected) vec = Vec2D(0.5, 3) - answer = vec * 10 expected = Vec2D(5, 30) - self.assertVectorsAlmostEqual(answer, expected) + self.assertVectorsAlmostEqual(vec * 10, expected) + self.assertVectorsAlmostEqual(10 * vec, expected) + self.assertVectorsAlmostEqual(vec * 10.0, expected) + self.assertVectorsAlmostEqual(10.0 * vec, expected) + + M = Multiplier() + self.assertEqual(vec * M, Vec2D(f"{vec[0]}*M", f"{vec[1]}*M")) + self.assertEqual(M * vec, f'M*{vec}') def test_vector_negative(self): vec = Vec2D(10, -10) diff --git a/Lib/turtle.py b/Lib/turtle.py index ee67a351b54f1..ba8288daf83f2 100644 --- a/Lib/turtle.py +++ b/Lib/turtle.py @@ -258,6 +258,7 @@ def __mul__(self, other): def __rmul__(self, other): if isinstance(other, int) or isinstance(other, float): return Vec2D(self[0]*other, self[1]*other) + return NotImplemented def __sub__(self, other): return Vec2D(self[0]-other[0], self[1]-other[1]) def __neg__(self): diff --git a/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst b/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst new file mode 100644 index 0000000000000..5d2a5094ddeaa --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst @@ -0,0 +1,2 @@ +Fixed :meth:`turtle.Vec2D.__rmul__` for arguments which are not int or +float. From webhook-mailer at python.org Mon Sep 7 12:46:41 2020 From: webhook-mailer at python.org (Artem Bulgakov) Date: Mon, 07 Sep 2020 16:46:41 -0000 Subject: [Python-checkins] bpo-41316: Make tarfile follow specs for FNAME (GH-21511) Message-ID: https://github.com/python/cpython/commit/22748a83d927d3da1beaed771be30887c42b2500 commit: 22748a83d927d3da1beaed771be30887c42b2500 branch: master author: Artem Bulgakov committer: GitHub date: 2020-09-07T09:46:33-07:00 summary: bpo-41316: Make tarfile follow specs for FNAME (GH-21511) tarfile writes full path to FNAME field of GZIP format instead of just basename if user specified absolute path. Some archive viewers may process file incorrectly. Also it creates security issue because anyone can know structure of directories on system and know username or other personal information. RFC1952 says about FNAME: This is the original name of the file being compressed, with any directory components removed. So tarfile must remove directory names from FNAME and write only basename of file. Automerge-Triggered-By: @jaraco files: A Misc/NEWS.d/next/Library/2020-07-28-12-08-58.bpo-41316.bSCbK4.rst M Lib/tarfile.py M Lib/test/test_tarfile.py M Misc/ACKS diff --git a/Lib/tarfile.py b/Lib/tarfile.py index 6769066cabd6f..1fae29430feff 100755 --- a/Lib/tarfile.py +++ b/Lib/tarfile.py @@ -420,6 +420,8 @@ def _init_write_gz(self): self.__write(b"\037\213\010\010" + timestamp + b"\002\377") if self.name.endswith(".gz"): self.name = self.name[:-3] + # Honor "directory components removed" from RFC1952 + self.name = os.path.basename(self.name) # RFC1952 says we must use ISO-8859-1 for the FNAME field. self.__write(self.name.encode("iso-8859-1", "replace") + NUL) diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py index 4ef20db097163..7b34d53d21601 100644 --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -1417,12 +1417,15 @@ def write(self, data): pax_headers={'non': 'empty'}) self.assertFalse(f.closed) + class GzipWriteTest(GzipTest, WriteTest): pass + class Bz2WriteTest(Bz2Test, WriteTest): pass + class LzmaWriteTest(LzmaTest, WriteTest): pass @@ -1465,8 +1468,17 @@ def test_file_mode(self): finally: os.umask(original_umask) + class GzipStreamWriteTest(GzipTest, StreamWriteTest): - pass + def test_source_directory_not_leaked(self): + """ + Ensure the source directory is not included in the tar header + per bpo-41316. + """ + tarfile.open(tmpname, self.mode).close() + payload = pathlib.Path(tmpname).read_text(encoding='latin-1') + assert os.path.dirname(tmpname) not in payload + class Bz2StreamWriteTest(Bz2Test, StreamWriteTest): decompressor = bz2.BZ2Decompressor if bz2 else None diff --git a/Misc/ACKS b/Misc/ACKS index a2cdeb8504059..8b0d7a45da169 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -242,6 +242,7 @@ Colm Buckley Erik de Bueger Jan-Hein B?hrman Lars Buitinck +Artem Bulgakov Dick Bulterman Bill Bumgarner Jimmy Burgett diff --git a/Misc/NEWS.d/next/Library/2020-07-28-12-08-58.bpo-41316.bSCbK4.rst b/Misc/NEWS.d/next/Library/2020-07-28-12-08-58.bpo-41316.bSCbK4.rst new file mode 100644 index 0000000000000..139a170866ed4 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-07-28-12-08-58.bpo-41316.bSCbK4.rst @@ -0,0 +1 @@ +Fix the :mod:`tarfile` module to write only basename of TAR file to GZIP compression header. \ No newline at end of file From webhook-mailer at python.org Mon Sep 7 17:27:04 2020 From: webhook-mailer at python.org (Erlend Egeberg Aasland) Date: Mon, 07 Sep 2020 21:27:04 -0000 Subject: [Python-checkins] bpo-40744: Drop support for SQLite pre 3.7.3 (GH-20909) Message-ID: https://github.com/python/cpython/commit/207c321f13cea3fee7f378057864e8c6453f5adf commit: 207c321f13cea3fee7f378057864e8c6453f5adf branch: master author: Erlend Egeberg Aasland committer: GitHub date: 2020-09-07T22:26:54+01:00 summary: bpo-40744: Drop support for SQLite pre 3.7.3 (GH-20909) Remove code required to support SQLite pre 3.7.3. Co-written-by: Berker Peksag Co-written-by: Sergey Fedoseev files: A Misc/NEWS.d/next/Library/2020-05-30-08-10-23.bpo-40744.jKURVV.rst M Doc/library/sqlite3.rst M Doc/whatsnew/3.10.rst M Lib/sqlite3/test/backup.py M Lib/sqlite3/test/dbapi.py M Lib/sqlite3/test/hooks.py M Lib/sqlite3/test/regression.py M Lib/sqlite3/test/transactions.py M Lib/sqlite3/test/types.py M Modules/_sqlite/connection.c M Modules/_sqlite/module.c M setup.py diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst index ccb82278bdaa1..13aa8c512d031 100644 --- a/Doc/library/sqlite3.rst +++ b/Doc/library/sqlite3.rst @@ -18,7 +18,8 @@ application using SQLite and then port the code to a larger database such as PostgreSQL or Oracle. The sqlite3 module was written by Gerhard H?ring. It provides a SQL interface -compliant with the DB-API 2.0 specification described by :pep:`249`. +compliant with the DB-API 2.0 specification described by :pep:`249`, and +requires SQLite 3.7.3 or newer. To use the module, you must first create a :class:`Connection` object that represents the database. Here the data will be stored in the @@ -591,8 +592,6 @@ Connection Objects dest = sqlite3.connect(':memory:') source.backup(dest) - Availability: SQLite 3.6.11 or higher - .. versionadded:: 3.7 @@ -701,9 +700,6 @@ Cursor Objects statements because we cannot determine the number of rows a query produced until all rows were fetched. - With SQLite versions before 3.6.5, :attr:`rowcount` is set to 0 if - you make a ``DELETE FROM table`` without any condition. - .. attribute:: lastrowid This read-only attribute provides the rowid of the last modified row. It is diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index eb5ae01a7c04d..f6f276a8bfa49 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -191,10 +191,15 @@ that may require changes to your code. Build Changes ============= + * The C99 functions :c:func:`snprintf` and :c:func:`vsnprintf` are now required to build Python. (Contributed by Victor Stinner in :issue:`36020`.) +* :mod:`sqlite3` requires SQLite 3.7.3 or higher. + (Contributed by Sergey Fedoseev and Erlend E. Aasland :issue:`40744`.) + + C API Changes ============= diff --git a/Lib/sqlite3/test/backup.py b/Lib/sqlite3/test/backup.py index 903bacf490301..2752a4db337dd 100644 --- a/Lib/sqlite3/test/backup.py +++ b/Lib/sqlite3/test/backup.py @@ -2,7 +2,6 @@ import unittest - at unittest.skipIf(sqlite.sqlite_version_info < (3, 6, 11), "Backup API not supported") class BackupTests(unittest.TestCase): def setUp(self): cx = self.cx = sqlite.connect(":memory:") diff --git a/Lib/sqlite3/test/dbapi.py b/Lib/sqlite3/test/dbapi.py index 119da12170331..a8dfeb9b2d693 100644 --- a/Lib/sqlite3/test/dbapi.py +++ b/Lib/sqlite3/test/dbapi.py @@ -185,12 +185,6 @@ def CheckOpenUri(self): with self.assertRaises(sqlite.OperationalError): cx.execute('insert into test(id) values(1)') - @unittest.skipIf(sqlite.sqlite_version_info >= (3, 3, 1), - 'needs sqlite versions older than 3.3.1') - def CheckSameThreadErrorOnOldVersion(self): - with self.assertRaises(sqlite.NotSupportedError) as cm: - sqlite.connect(':memory:', check_same_thread=False) - self.assertEqual(str(cm.exception), 'shared connections not available') class CursorTests(unittest.TestCase): def setUp(self): diff --git a/Lib/sqlite3/test/hooks.py b/Lib/sqlite3/test/hooks.py index b08adf1d8097b..2e620ecdf864c 100644 --- a/Lib/sqlite3/test/hooks.py +++ b/Lib/sqlite3/test/hooks.py @@ -61,8 +61,6 @@ def upper(self): self.assertEqual(result[0][0], 'b') self.assertEqual(result[1][0], 'a') - @unittest.skipIf(sqlite.sqlite_version_info < (3, 2, 1), - 'old SQLite versions crash on this test') def CheckCollationIsUsed(self): def mycoll(x, y): # reverse order @@ -240,16 +238,12 @@ def trace(statement): traced_statements.append(statement) con.set_trace_callback(trace) con.execute("create table foo(x)") - # Can't execute bound parameters as their values don't appear - # in traced statements before SQLite 3.6.21 - # (cf. http://www.sqlite.org/draft/releaselog/3_6_21.html) con.execute('insert into foo(x) values ("%s")' % unicode_value) con.commit() self.assertTrue(any(unicode_value in stmt for stmt in traced_statements), "Unicode data %s garbled in trace callback: %s" % (ascii(unicode_value), ', '.join(map(ascii, traced_statements)))) - @unittest.skipIf(sqlite.sqlite_version_info < (3, 3, 9), "sqlite3_prepare_v2 is not available") def CheckTraceCallbackContent(self): # set_trace_callback() shouldn't produce duplicate content (bpo-26187) traced_statements = [] diff --git a/Lib/sqlite3/test/regression.py b/Lib/sqlite3/test/regression.py index cbd46d4978afb..0735a5c129226 100644 --- a/Lib/sqlite3/test/regression.py +++ b/Lib/sqlite3/test/regression.py @@ -87,7 +87,6 @@ def CheckStatementFinalizationOnCloseDb(self): cur.execute("select 1 x union select " + str(i)) con.close() - @unittest.skipIf(sqlite.sqlite_version_info < (3, 2, 2), 'needs sqlite 3.2.2 or newer') def CheckOnConflictRollback(self): con = sqlite.connect(":memory:") con.execute("create table foo(x, unique(x) on conflict rollback)") diff --git a/Lib/sqlite3/test/transactions.py b/Lib/sqlite3/test/transactions.py index b8a13de55bc72..c463f7490da57 100644 --- a/Lib/sqlite3/test/transactions.py +++ b/Lib/sqlite3/test/transactions.py @@ -111,16 +111,12 @@ def CheckToggleAutoCommit(self): res = self.cur2.fetchall() self.assertEqual(len(res), 1) - @unittest.skipIf(sqlite.sqlite_version_info < (3, 2, 2), - 'test hangs on sqlite versions older than 3.2.2') def CheckRaiseTimeout(self): self.cur1.execute("create table test(i)") self.cur1.execute("insert into test(i) values (5)") with self.assertRaises(sqlite.OperationalError): self.cur2.execute("insert into test(i) values (5)") - @unittest.skipIf(sqlite.sqlite_version_info < (3, 2, 2), - 'test hangs on sqlite versions older than 3.2.2') def CheckLocking(self): """ This tests the improved concurrency with pysqlite 2.3.4. You needed diff --git a/Lib/sqlite3/test/types.py b/Lib/sqlite3/test/types.py index d26a9cb93f088..75a9d5601d580 100644 --- a/Lib/sqlite3/test/types.py +++ b/Lib/sqlite3/test/types.py @@ -401,8 +401,6 @@ def CheckSqliteTimestamp(self): ts2 = self.cur.fetchone()[0] self.assertEqual(ts, ts2) - @unittest.skipIf(sqlite.sqlite_version_info < (3, 1), - 'the date functions are available on 3.1 or later') def CheckSqlTimestamp(self): now = datetime.datetime.utcnow() self.cur.execute("insert into test(ts) values (current_timestamp)") diff --git a/Misc/NEWS.d/next/Library/2020-05-30-08-10-23.bpo-40744.jKURVV.rst b/Misc/NEWS.d/next/Library/2020-05-30-08-10-23.bpo-40744.jKURVV.rst new file mode 100644 index 0000000000000..2d1d1f9a20e32 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-05-30-08-10-23.bpo-40744.jKURVV.rst @@ -0,0 +1,4 @@ +The :mod:`sqlite3` module uses SQLite API functions that require SQLite +v3.7.3 or higher. This patch removes support for older SQLite versions, and +explicitly requires SQLite 3.7.3 both at build, compile and runtime. Patch by +Sergey Fedoseev and Erlend E. Aasland. diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index 1bf9710763a5a..f765ba1df2466 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -33,16 +33,6 @@ #define ACTION_FINALIZE 1 #define ACTION_RESET 2 -#if SQLITE_VERSION_NUMBER >= 3003008 -#ifndef SQLITE_OMIT_LOAD_EXTENSION -#define HAVE_LOAD_EXTENSION -#endif -#endif - -#if SQLITE_VERSION_NUMBER >= 3006011 -#define HAVE_BACKUP_API -#endif - #if SQLITE_VERSION_NUMBER >= 3014000 #define HAVE_TRACE_V2 #endif @@ -61,18 +51,6 @@ static int pysqlite_connection_set_isolation_level(pysqlite_Connection* self, Py static void _pysqlite_drop_unused_cursor_references(pysqlite_Connection* self); -static void _sqlite3_result_error(sqlite3_context* ctx, const char* errmsg, int len) -{ - /* in older SQLite versions, calling sqlite3_result_error in callbacks - * triggers a bug in SQLite that leads either to irritating results or - * segfaults, depending on the SQLite version */ -#if SQLITE_VERSION_NUMBER >= 3003003 - sqlite3_result_error(ctx, errmsg, len); -#else - PyErr_SetString(pysqlite_OperationalError, errmsg); -#endif -} - int pysqlite_connection_init(pysqlite_Connection* self, PyObject* args, PyObject* kwargs) { static char *kwlist[] = { @@ -182,10 +160,6 @@ int pysqlite_connection_init(pysqlite_Connection* self, PyObject* args, PyObject self->timeout = timeout; (void)sqlite3_busy_timeout(self->db, (int)(timeout*1000)); self->thread_ident = PyThread_get_thread_ident(); - if (!check_same_thread && sqlite3_libversion_number() < 3003001) { - PyErr_SetString(pysqlite_NotSupportedError, "shared connections not available"); - return -1; - } self->check_same_thread = check_same_thread; self->function_pinboard_trace_callback = NULL; @@ -620,7 +594,7 @@ void _pysqlite_func_callback(sqlite3_context* context, int argc, sqlite3_value** } else { PyErr_Clear(); } - _sqlite3_result_error(context, "user-defined function raised exception", -1); + sqlite3_result_error(context, "user-defined function raised exception", -1); } PyGILState_Release(threadstate); @@ -652,7 +626,7 @@ static void _pysqlite_step_callback(sqlite3_context *context, int argc, sqlite3_ } else { PyErr_Clear(); } - _sqlite3_result_error(context, "user-defined aggregate's '__init__' method raised error", -1); + sqlite3_result_error(context, "user-defined aggregate's '__init__' method raised error", -1); goto error; } } @@ -676,7 +650,7 @@ static void _pysqlite_step_callback(sqlite3_context *context, int argc, sqlite3_ } else { PyErr_Clear(); } - _sqlite3_result_error(context, "user-defined aggregate's 'step' method raised error", -1); + sqlite3_result_error(context, "user-defined aggregate's 'step' method raised error", -1); } error: @@ -693,7 +667,6 @@ void _pysqlite_final_callback(sqlite3_context* context) _Py_IDENTIFIER(finalize); int ok; PyObject *exception, *value, *tb; - int restore; PyGILState_STATE threadstate; @@ -709,7 +682,6 @@ void _pysqlite_final_callback(sqlite3_context* context) /* Keep the exception (if any) of the last call to step() */ PyErr_Fetch(&exception, &value, &tb); - restore = 1; function_result = _PyObject_CallMethodIdNoArgs(*aggregate_instance, &PyId_finalize); @@ -726,19 +698,12 @@ void _pysqlite_final_callback(sqlite3_context* context) } else { PyErr_Clear(); } - _sqlite3_result_error(context, "user-defined aggregate's 'finalize' method raised error", -1); -#if SQLITE_VERSION_NUMBER < 3003003 - /* with old SQLite versions, _sqlite3_result_error() sets a new Python - exception, so don't restore the previous exception */ - restore = 0; -#endif + sqlite3_result_error(context, "user-defined aggregate's 'finalize' method raised error", -1); } - if (restore) { - /* Restore the exception (if any) of the last call to step(), - but clear also the current exception if finalize() failed */ - PyErr_Restore(exception, value, tb); - } + /* Restore the exception (if any) of the last call to step(), + but clear also the current exception if finalize() failed */ + PyErr_Restore(exception, value, tb); error: PyGILState_Release(threadstate); @@ -1110,7 +1075,7 @@ static PyObject* pysqlite_connection_set_trace_callback(pysqlite_Connection* sel Py_RETURN_NONE; } -#ifdef HAVE_LOAD_EXTENSION +#ifndef SQLITE_OMIT_LOAD_EXTENSION static PyObject* pysqlite_enable_load_extension(pysqlite_Connection* self, PyObject* args) { int rc; @@ -1513,7 +1478,6 @@ pysqlite_connection_iterdump(pysqlite_Connection* self, PyObject* args) return retval; } -#ifdef HAVE_BACKUP_API static PyObject * pysqlite_connection_backup(pysqlite_Connection *self, PyObject *args, PyObject *kwds) { @@ -1664,7 +1628,6 @@ pysqlite_connection_backup(pysqlite_Connection *self, PyObject *args, PyObject * return NULL; } } -#endif static PyObject * pysqlite_connection_create_collation(pysqlite_Connection* self, PyObject* args) @@ -1816,7 +1779,7 @@ static PyMethodDef connection_methods[] = { PyDoc_STR("Creates a new aggregate. Non-standard.")}, {"set_authorizer", (PyCFunction)(void(*)(void))pysqlite_connection_set_authorizer, METH_VARARGS|METH_KEYWORDS, PyDoc_STR("Sets authorizer callback. Non-standard.")}, - #ifdef HAVE_LOAD_EXTENSION + #ifndef SQLITE_OMIT_LOAD_EXTENSION {"enable_load_extension", (PyCFunction)pysqlite_enable_load_extension, METH_VARARGS, PyDoc_STR("Enable dynamic loading of SQLite extension modules. Non-standard.")}, {"load_extension", (PyCFunction)pysqlite_load_extension, METH_VARARGS, @@ -1838,10 +1801,8 @@ static PyMethodDef connection_methods[] = { PyDoc_STR("Abort any pending database operation. Non-standard.")}, {"iterdump", (PyCFunction)pysqlite_connection_iterdump, METH_NOARGS, PyDoc_STR("Returns iterator to the dump of the database in an SQL text format. Non-standard.")}, - #ifdef HAVE_BACKUP_API {"backup", (PyCFunction)(void(*)(void))pysqlite_connection_backup, METH_VARARGS | METH_KEYWORDS, PyDoc_STR("Makes a backup of the database. Non-standard.")}, - #endif {"__enter__", (PyCFunction)pysqlite_connection_enter, METH_NOARGS, PyDoc_STR("For context manager. Non-standard.")}, {"__exit__", (PyCFunction)pysqlite_connection_exit, METH_VARARGS, diff --git a/Modules/_sqlite/module.c b/Modules/_sqlite/module.c index 71d951ee887e4..82f58eb248026 100644 --- a/Modules/_sqlite/module.c +++ b/Modules/_sqlite/module.c @@ -29,8 +29,8 @@ #include "microprotocols.h" #include "row.h" -#if SQLITE_VERSION_NUMBER >= 3003003 -#define HAVE_SHARED_CACHE +#if SQLITE_VERSION_NUMBER < 3007003 +#error "SQLite 3.7.3 or higher required" #endif /* static objects at module-level */ @@ -131,7 +131,6 @@ PyDoc_STRVAR(module_complete_doc, \n\ Checks if a string contains a complete SQL statement. Non-standard."); -#ifdef HAVE_SHARED_CACHE static PyObject* module_enable_shared_cache(PyObject* self, PyObject* args, PyObject* kwargs) { @@ -159,7 +158,6 @@ PyDoc_STRVAR(module_enable_shared_cache_doc, \n\ Enable or disable shared cache mode for the calling thread.\n\ Experimental/Non-standard."); -#endif /* HAVE_SHARED_CACHE */ static PyObject* module_register_adapter(PyObject* self, PyObject* args) { @@ -253,10 +251,8 @@ static PyMethodDef module_methods[] = { METH_VARARGS | METH_KEYWORDS, module_connect_doc}, {"complete_statement", (PyCFunction)(void(*)(void))module_complete, METH_VARARGS | METH_KEYWORDS, module_complete_doc}, -#ifdef HAVE_SHARED_CACHE {"enable_shared_cache", (PyCFunction)(void(*)(void))module_enable_shared_cache, METH_VARARGS | METH_KEYWORDS, module_enable_shared_cache_doc}, -#endif {"register_adapter", (PyCFunction)module_register_adapter, METH_VARARGS, module_register_adapter_doc}, {"register_converter", (PyCFunction)module_register_converter, @@ -307,29 +303,17 @@ static const IntConstantPair _int_constants[] = { {"SQLITE_UPDATE", SQLITE_UPDATE}, {"SQLITE_ATTACH", SQLITE_ATTACH}, {"SQLITE_DETACH", SQLITE_DETACH}, -#if SQLITE_VERSION_NUMBER >= 3002001 {"SQLITE_ALTER_TABLE", SQLITE_ALTER_TABLE}, {"SQLITE_REINDEX", SQLITE_REINDEX}, -#endif -#if SQLITE_VERSION_NUMBER >= 3003000 {"SQLITE_ANALYZE", SQLITE_ANALYZE}, -#endif -#if SQLITE_VERSION_NUMBER >= 3003007 {"SQLITE_CREATE_VTABLE", SQLITE_CREATE_VTABLE}, {"SQLITE_DROP_VTABLE", SQLITE_DROP_VTABLE}, -#endif -#if SQLITE_VERSION_NUMBER >= 3003008 {"SQLITE_FUNCTION", SQLITE_FUNCTION}, -#endif -#if SQLITE_VERSION_NUMBER >= 3006008 {"SQLITE_SAVEPOINT", SQLITE_SAVEPOINT}, -#endif #if SQLITE_VERSION_NUMBER >= 3008003 {"SQLITE_RECURSIVE", SQLITE_RECURSIVE}, #endif -#if SQLITE_VERSION_NUMBER >= 3006011 {"SQLITE_DONE", SQLITE_DONE}, -#endif {(char*)NULL, 0} }; @@ -360,6 +344,11 @@ PyMODINIT_FUNC PyInit__sqlite3(void) PyObject *tmp_obj; int i; + if (sqlite3_libversion_number() < 3007003) { + PyErr_SetString(PyExc_ImportError, MODULE_NAME ": SQLite 3.7.3 or higher required"); + return NULL; + } + module = PyModule_Create(&_sqlite3module); if (!module || diff --git a/setup.py b/setup.py index 21a5a58981fc1..04b1358bc916e 100644 --- a/setup.py +++ b/setup.py @@ -1452,7 +1452,6 @@ def detect_sqlite(self): sqlite_setup_debug = False # verbose debug prints from this script? # We hunt for #define SQLITE_VERSION "n.n.n" - # We need to find >= sqlite version 3.3.9, for sqlite3_prepare_v2 sqlite_incdir = sqlite_libdir = None sqlite_inc_paths = [ '/usr/include', '/usr/include/sqlite', @@ -1463,7 +1462,8 @@ def detect_sqlite(self): ] if CROSS_COMPILING: sqlite_inc_paths = [] - MIN_SQLITE_VERSION_NUMBER = (3, 7, 2) + # We need to find >= sqlite version 3.7.3, for sqlite3_create_function_v2() + MIN_SQLITE_VERSION_NUMBER = (3, 7, 3) MIN_SQLITE_VERSION = ".".join([str(x) for x in MIN_SQLITE_VERSION_NUMBER]) From webhook-mailer at python.org Tue Sep 8 03:29:01 2020 From: webhook-mailer at python.org (dxflores) Date: Tue, 08 Sep 2020 07:29:01 -0000 Subject: [Python-checkins] bpo-41732: add iterator to memoryview (GH-22119) Message-ID: https://github.com/python/cpython/commit/b0ac5d75a59c356c44cfc9b25077da3049dd16e9 commit: b0ac5d75a59c356c44cfc9b25077da3049dd16e9 branch: master author: dxflores committer: GitHub date: 2020-09-08T16:28:45+09:00 summary: bpo-41732: add iterator to memoryview (GH-22119) files: A Misc/NEWS.d/next/Library/2020-09-06-20-27-10.bpo-41732.1SKv26.rst M Objects/memoryobject.c diff --git a/Misc/NEWS.d/next/Library/2020-09-06-20-27-10.bpo-41732.1SKv26.rst b/Misc/NEWS.d/next/Library/2020-09-06-20-27-10.bpo-41732.1SKv26.rst new file mode 100644 index 0000000000000..caf237f37f4de --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-06-20-27-10.bpo-41732.1SKv26.rst @@ -0,0 +1 @@ +Added an :term:`iterator` to :class:`memoryview`. diff --git a/Objects/memoryobject.c b/Objects/memoryobject.c index 13d883ae4d35d..d328f4d40b7a4 100644 --- a/Objects/memoryobject.c +++ b/Objects/memoryobject.c @@ -3160,6 +3160,112 @@ static PyMethodDef memory_methods[] = { {NULL, NULL} }; +/**************************************************************************/ +/* Memoryview Iterator */ +/**************************************************************************/ + +static PyTypeObject PyMemoryIter_Type; + +typedef struct { + PyObject_HEAD + Py_ssize_t it_index; + PyMemoryViewObject *it_seq; // Set to NULL when iterator is exhausted + Py_ssize_t it_length; + const char *it_fmt; +} memoryiterobject; + +static void +memoryiter_dealloc(memoryiterobject *it) +{ + _PyObject_GC_UNTRACK(it); + Py_XDECREF(it->it_seq); + PyObject_GC_Del(it); +} + +static int +memoryiter_traverse(memoryiterobject *it, visitproc visit, void *arg) +{ + Py_VISIT(it->it_seq); + return 0; +} + +static PyObject * +memoryiter_next(memoryiterobject *it) +{ + PyMemoryViewObject *seq; + seq = it->it_seq; + if (seq == NULL) { + return NULL; + } + + if (it->it_index < it->it_length) { + CHECK_RELEASED(seq); + Py_buffer *view = &(seq->view); + char *ptr = (char *)seq->view.buf; + + ptr += view->strides[0] * it->it_index++; + ptr = ADJUST_PTR(ptr, view->suboffsets, 0); + if (ptr == NULL) { + return NULL; + } + return unpack_single(ptr, it->it_fmt); + } + + it->it_seq = NULL; + Py_DECREF(seq); + return NULL; +} + +static PyObject * +memory_iter(PyObject *seq) +{ + if (!PyMemoryView_Check(seq)) { + PyErr_BadInternalCall(); + return NULL; + } + PyMemoryViewObject *obj = (PyMemoryViewObject *)seq; + int ndims = obj->view.ndim; + if (ndims == 0) { + PyErr_SetString(PyExc_TypeError, "invalid indexing of 0-dim memory"); + return NULL; + } + if (ndims != 1) { + PyErr_SetString(PyExc_NotImplementedError, + "multi-dimensional sub-views are not implemented"); + return NULL; + } + + const char *fmt = adjust_fmt(&obj->view); + if (fmt == NULL) { + return NULL; + } + + memoryiterobject *it; + it = PyObject_GC_New(memoryiterobject, &PyMemoryIter_Type); + if (it == NULL) { + return NULL; + } + it->it_fmt = fmt; + it->it_length = memory_length(obj); + it->it_index = 0; + Py_INCREF(seq); + it->it_seq = obj; + _PyObject_GC_TRACK(it); + return (PyObject *)it; +} + +static PyTypeObject PyMemoryIter_Type = { + PyVarObject_HEAD_INIT(&PyType_Type, 0) + .tp_name = "memory_iterator", + .tp_basicsize = sizeof(memoryiterobject), + // methods + .tp_dealloc = (destructor)memoryiter_dealloc, + .tp_getattro = PyObject_GenericGetAttr, + .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, + .tp_traverse = (traverseproc)memoryiter_traverse, + .tp_iter = PyObject_SelfIter, + .tp_iternext = (iternextfunc)memoryiter_next, +}; PyTypeObject PyMemoryView_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) @@ -3187,7 +3293,7 @@ PyTypeObject PyMemoryView_Type = { (inquiry)memory_clear, /* tp_clear */ memory_richcompare, /* tp_richcompare */ offsetof(PyMemoryViewObject, weakreflist),/* tp_weaklistoffset */ - 0, /* tp_iter */ + memory_iter, /* tp_iter */ 0, /* tp_iternext */ memory_methods, /* tp_methods */ 0, /* tp_members */ From webhook-mailer at python.org Tue Sep 8 04:59:20 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Tue, 08 Sep 2020 08:59:20 -0000 Subject: [Python-checkins] bpo-1635741: Port the termios to multi-phase init (PEP 489) (GH-22139) Message-ID: https://github.com/python/cpython/commit/15dcdb211366e0788e831fc2a1f785e6a5ca2749 commit: 15dcdb211366e0788e831fc2a1f785e6a5ca2749 branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-08T10:59:15+02:00 summary: bpo-1635741: Port the termios to multi-phase init (PEP 489) (GH-22139) files: A Misc/NEWS.d/next/Core and Builtins/2020-09-07-11-35-02.bpo-1635741.rvIexb.rst M Modules/termios.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-07-11-35-02.bpo-1635741.rvIexb.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-07-11-35-02.bpo-1635741.rvIexb.rst new file mode 100644 index 0000000000000..1e19b34b372d8 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-07-11-35-02.bpo-1635741.rvIexb.rst @@ -0,0 +1,2 @@ +Port the :mod:`termios` extension module to multi-phase initialization +(:pep:`489`). diff --git a/Modules/termios.c b/Modules/termios.c index 178ae4ee6e41d..cc0d5853f85e3 100644 --- a/Modules/termios.c +++ b/Modules/termios.c @@ -51,8 +51,6 @@ get_termios_state(PyObject *module) return (termiosmodulestate *)state; } -#define modulestate_global get_termios_state(PyState_FindModule(&termiosmodule)) - static int fdconv(PyObject* obj, void* p) { int fd; @@ -79,31 +77,32 @@ indexing in the cc array must be done using the symbolic constants defined\n\ in this module."); static PyObject * -termios_tcgetattr(PyObject *self, PyObject *args) +termios_tcgetattr(PyObject *module, PyObject *args) { int fd; - struct termios mode; - PyObject *cc; - speed_t ispeed, ospeed; - PyObject *v; - int i; - char ch; - if (!PyArg_ParseTuple(args, "O&:tcgetattr", - fdconv, (void*)&fd)) + fdconv, (void*)&fd)) { return NULL; + } - if (tcgetattr(fd, &mode) == -1) - return PyErr_SetFromErrno(modulestate_global->TermiosError); + termiosmodulestate *state = PyModule_GetState(module); + struct termios mode; + if (tcgetattr(fd, &mode) == -1) { + return PyErr_SetFromErrno(state->TermiosError); + } - ispeed = cfgetispeed(&mode); - ospeed = cfgetospeed(&mode); + speed_t ispeed = cfgetispeed(&mode); + speed_t ospeed = cfgetospeed(&mode); - cc = PyList_New(NCCS); - if (cc == NULL) + PyObject *cc = PyList_New(NCCS); + if (cc == NULL) { return NULL; + } + + PyObject *v; + int i; for (i = 0; i < NCCS; i++) { - ch = (char)mode.c_cc[i]; + char ch = (char)mode.c_cc[i]; v = PyBytes_FromStringAndSize(&ch, 1); if (v == NULL) goto err; @@ -156,17 +155,15 @@ queued output, or termios.TCSAFLUSH to change after transmitting all\n\ queued output and discarding all queued input. "); static PyObject * -termios_tcsetattr(PyObject *self, PyObject *args) +termios_tcsetattr(PyObject *module, PyObject *args) { int fd, when; - struct termios mode; - speed_t ispeed, ospeed; - PyObject *term, *cc, *v; - int i; - + PyObject *term; if (!PyArg_ParseTuple(args, "O&iO:tcsetattr", - fdconv, &fd, &when, &term)) + fdconv, &fd, &when, &term)) { return NULL; + } + if (!PyList_Check(term) || PyList_Size(term) != 7) { PyErr_SetString(PyExc_TypeError, "tcsetattr, arg 3: must be 7 element list"); @@ -174,18 +171,22 @@ termios_tcsetattr(PyObject *self, PyObject *args) } /* Get the old mode, in case there are any hidden fields... */ - termiosmodulestate *state = modulestate_global; - if (tcgetattr(fd, &mode) == -1) + termiosmodulestate *state = PyModule_GetState(module); + struct termios mode; + if (tcgetattr(fd, &mode) == -1) { return PyErr_SetFromErrno(state->TermiosError); + } + mode.c_iflag = (tcflag_t) PyLong_AsLong(PyList_GetItem(term, 0)); mode.c_oflag = (tcflag_t) PyLong_AsLong(PyList_GetItem(term, 1)); mode.c_cflag = (tcflag_t) PyLong_AsLong(PyList_GetItem(term, 2)); mode.c_lflag = (tcflag_t) PyLong_AsLong(PyList_GetItem(term, 3)); - ispeed = (speed_t) PyLong_AsLong(PyList_GetItem(term, 4)); - ospeed = (speed_t) PyLong_AsLong(PyList_GetItem(term, 5)); - cc = PyList_GetItem(term, 6); - if (PyErr_Occurred()) + speed_t ispeed = (speed_t) PyLong_AsLong(PyList_GetItem(term, 4)); + speed_t ospeed = (speed_t) PyLong_AsLong(PyList_GetItem(term, 5)); + PyObject *cc = PyList_GetItem(term, 6); + if (PyErr_Occurred()) { return NULL; + } if (!PyList_Check(cc) || PyList_Size(cc) != NCCS) { PyErr_Format(PyExc_TypeError, @@ -194,6 +195,8 @@ termios_tcsetattr(PyObject *self, PyObject *args) return NULL; } + int i; + PyObject *v; for (i = 0; i < NCCS; i++) { v = PyList_GetItem(cc, i); @@ -226,15 +229,18 @@ A zero duration sends a break for 0.25-0.5 seconds; a nonzero duration\n\ has a system dependent meaning."); static PyObject * -termios_tcsendbreak(PyObject *self, PyObject *args) +termios_tcsendbreak(PyObject *module, PyObject *args) { int fd, duration; - if (!PyArg_ParseTuple(args, "O&i:tcsendbreak", - fdconv, &fd, &duration)) + fdconv, &fd, &duration)) { return NULL; - if (tcsendbreak(fd, duration) == -1) - return PyErr_SetFromErrno(modulestate_global->TermiosError); + } + + termiosmodulestate *state = PyModule_GetState(module); + if (tcsendbreak(fd, duration) == -1) { + return PyErr_SetFromErrno(state->TermiosError); + } Py_RETURN_NONE; } @@ -245,15 +251,18 @@ PyDoc_STRVAR(termios_tcdrain__doc__, Wait until all output written to file descriptor fd has been transmitted."); static PyObject * -termios_tcdrain(PyObject *self, PyObject *args) +termios_tcdrain(PyObject *module, PyObject *args) { int fd; - if (!PyArg_ParseTuple(args, "O&:tcdrain", - fdconv, &fd)) + fdconv, &fd)) { return NULL; - if (tcdrain(fd) == -1) - return PyErr_SetFromErrno(modulestate_global->TermiosError); + } + + termiosmodulestate *state = PyModule_GetState(module); + if (tcdrain(fd) == -1) { + return PyErr_SetFromErrno(state->TermiosError); + } Py_RETURN_NONE; } @@ -267,15 +276,18 @@ queue, termios.TCOFLUSH for the output queue, or termios.TCIOFLUSH for\n\ both queues. "); static PyObject * -termios_tcflush(PyObject *self, PyObject *args) +termios_tcflush(PyObject *module, PyObject *args) { int fd, queue; - if (!PyArg_ParseTuple(args, "O&i:tcflush", - fdconv, &fd, &queue)) + fdconv, &fd, &queue)) { return NULL; - if (tcflush(fd, queue) == -1) - return PyErr_SetFromErrno(modulestate_global->TermiosError); + } + + termiosmodulestate *state = PyModule_GetState(module); + if (tcflush(fd, queue) == -1) { + return PyErr_SetFromErrno(state->TermiosError); + } Py_RETURN_NONE; } @@ -289,15 +301,18 @@ termios.TCOON to restart output, termios.TCIOFF to suspend input,\n\ or termios.TCION to restart input."); static PyObject * -termios_tcflow(PyObject *self, PyObject *args) +termios_tcflow(PyObject *module, PyObject *args) { int fd, action; - if (!PyArg_ParseTuple(args, "O&i:tcflow", - fdconv, &fd, &action)) + fdconv, &fd, &action)) { return NULL; - if (tcflow(fd, action) == -1) - return PyErr_SetFromErrno(modulestate_global->TermiosError); + } + + termiosmodulestate *state = PyModule_GetState(module); + if (tcflow(fd, action) == -1) { + return PyErr_SetFromErrno(state->TermiosError); + } Py_RETURN_NONE; } @@ -997,44 +1012,49 @@ static void termiosmodule_free(void *m) { termiosmodule_clear((PyObject *)m); } -static struct PyModuleDef termiosmodule = { - PyModuleDef_HEAD_INIT, - "termios", - termios__doc__, - sizeof(termiosmodulestate), - termios_methods, - NULL, - termiosmodule_traverse, - termiosmodule_clear, - termiosmodule_free, -}; - -PyMODINIT_FUNC -PyInit_termios(void) +static int +termios_exec(PyObject *mod) { - PyObject *m; struct constant *constant = termios_constants; - - if ((m = PyState_FindModule(&termiosmodule)) != NULL) { - Py_INCREF(m); - return m; - } - - if ((m = PyModule_Create(&termiosmodule)) == NULL) { - return NULL; - } - - termiosmodulestate *state = get_termios_state(m); + termiosmodulestate *state = get_termios_state(mod); state->TermiosError = PyErr_NewException("termios.error", NULL, NULL); if (state->TermiosError == NULL) { - return NULL; + return -1; } Py_INCREF(state->TermiosError); - PyModule_AddObject(m, "error", state->TermiosError); + if (PyModule_AddObject(mod, "error", state->TermiosError) < 0) { + Py_DECREF(state->TermiosError); + return -1; + } while (constant->name != NULL) { - PyModule_AddIntConstant(m, constant->name, constant->value); + if (PyModule_AddIntConstant( + mod, constant->name, constant->value) < 0) { + return -1; + } ++constant; } - return m; + return 0; +} + +static PyModuleDef_Slot termios_slots[] = { + {Py_mod_exec, termios_exec}, + {0, NULL} +}; + +static struct PyModuleDef termiosmodule = { + PyModuleDef_HEAD_INIT, + .m_name = "termios", + .m_doc = termios__doc__, + .m_size = sizeof(termiosmodulestate), + .m_methods = termios_methods, + .m_slots = termios_slots, + .m_traverse = termiosmodule_traverse, + .m_clear = termiosmodule_clear, + .m_free = termiosmodule_free, +}; + +PyMODINIT_FUNC PyInit_termios(void) +{ + return PyModuleDef_Init(&termiosmodule); } From webhook-mailer at python.org Tue Sep 8 05:16:19 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Tue, 08 Sep 2020 09:16:19 -0000 Subject: [Python-checkins] bpo-1635741: Convert _sha256 types to heap types (GH-22134) Message-ID: https://github.com/python/cpython/commit/52a2df135c0470b1dbf889edc51b7c556ae4bc80 commit: 52a2df135c0470b1dbf889edc51b7c556ae4bc80 branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-08T11:16:14+02:00 summary: bpo-1635741: Convert _sha256 types to heap types (GH-22134) Convert the _sha256 extension module types to heap types. files: A Misc/NEWS.d/next/Core and Builtins/2020-09-07-09-45-47.bpo-1635741.QuDIut.rst M Modules/clinic/sha256module.c.h M Modules/sha256module.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-07-09-45-47.bpo-1635741.QuDIut.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-07-09-45-47.bpo-1635741.QuDIut.rst new file mode 100644 index 0000000000000..90e56542d1e97 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-07-09-45-47.bpo-1635741.QuDIut.rst @@ -0,0 +1 @@ +Convert the :mod:`_sha256` extension module types to heap types. diff --git a/Modules/clinic/sha256module.c.h b/Modules/clinic/sha256module.c.h index 2a788ea98499f..89205c4f14f4e 100644 --- a/Modules/clinic/sha256module.c.h +++ b/Modules/clinic/sha256module.c.h @@ -9,15 +9,26 @@ PyDoc_STRVAR(SHA256Type_copy__doc__, "Return a copy of the hash object."); #define SHA256TYPE_COPY_METHODDEF \ - {"copy", (PyCFunction)SHA256Type_copy, METH_NOARGS, SHA256Type_copy__doc__}, + {"copy", (PyCFunction)(void(*)(void))SHA256Type_copy, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, SHA256Type_copy__doc__}, static PyObject * -SHA256Type_copy_impl(SHAobject *self); +SHA256Type_copy_impl(SHAobject *self, PyTypeObject *cls); static PyObject * -SHA256Type_copy(SHAobject *self, PyObject *Py_UNUSED(ignored)) +SHA256Type_copy(SHAobject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return SHA256Type_copy_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":copy", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = SHA256Type_copy_impl(self, cls); + +exit: + return return_value; } PyDoc_STRVAR(SHA256Type_digest__doc__, @@ -166,4 +177,4 @@ _sha256_sha224(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObje exit: return return_value; } -/*[clinic end generated code: output=c8cca8adbe72ec9a input=a9049054013a1b77]*/ +/*[clinic end generated code: output=b7283f75c9d08f30 input=a9049054013a1b77]*/ diff --git a/Modules/sha256module.c b/Modules/sha256module.c index 06e4430bd7c33..edd4d010928f3 100644 --- a/Modules/sha256module.c +++ b/Modules/sha256module.c @@ -51,6 +51,19 @@ typedef struct { #include "clinic/sha256module.c.h" +typedef struct { + PyTypeObject* sha224_type; + PyTypeObject* sha256_type; +} _sha256_state; + +static inline _sha256_state* +_sha256_get_state(PyObject *module) +{ + void *state = PyModule_GetState(module); + assert(state != NULL); + return (_sha256_state *)state; +} + /* When run on a little-endian CPU we need to perform byte reversal on an array of longwords. */ @@ -365,20 +378,17 @@ sha_final(unsigned char digest[SHA_DIGESTSIZE], SHAobject *sha_info) * ------------------------------------------------------------------------ */ -static PyTypeObject SHA224type; -static PyTypeObject SHA256type; - static SHAobject * -newSHA224object(void) +newSHA224object(_sha256_state *state) { - return (SHAobject *)PyObject_New(SHAobject, &SHA224type); + return (SHAobject *)PyObject_New(SHAobject, state->sha224_type); } static SHAobject * -newSHA256object(void) +newSHA256object(_sha256_state *state) { - return (SHAobject *)PyObject_New(SHAobject, &SHA256type); + return (SHAobject *)PyObject_New(SHAobject, state->sha256_type); } /* Internal methods for a hash object */ @@ -386,7 +396,9 @@ newSHA256object(void) static void SHA_dealloc(PyObject *ptr) { + PyTypeObject *tp = Py_TYPE(ptr); PyObject_Del(ptr); + Py_DECREF(tp); } @@ -395,21 +407,25 @@ SHA_dealloc(PyObject *ptr) /*[clinic input] SHA256Type.copy + cls:defining_class + Return a copy of the hash object. [clinic start generated code]*/ static PyObject * -SHA256Type_copy_impl(SHAobject *self) -/*[clinic end generated code: output=1a8bbd66a0c9c168 input=f58840a618d4f2a7]*/ +SHA256Type_copy_impl(SHAobject *self, PyTypeObject *cls) +/*[clinic end generated code: output=9273f92c382be12f input=3137146fcb88e212]*/ { SHAobject *newobj; - - if (Py_IS_TYPE(self, &SHA256type)) { - if ( (newobj = newSHA256object())==NULL) + _sha256_state *state = PyType_GetModuleState(cls); + if (Py_IS_TYPE(self, state->sha256_type)) { + if ( (newobj = newSHA256object(state)) == NULL) { return NULL; + } } else { - if ( (newobj = newSHA224object())==NULL) + if ( (newobj = newSHA224object(state))==NULL) { return NULL; + } } SHAcopy(self, newobj); @@ -517,74 +533,27 @@ static PyMemberDef SHA_members[] = { {NULL} /* Sentinel */ }; -static PyTypeObject SHA224type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_sha256.sha224", /*tp_name*/ - sizeof(SHAobject), /*tp_basicsize*/ - 0, /*tp_itemsize*/ - /* methods */ - SHA_dealloc, /*tp_dealloc*/ - 0, /*tp_vectorcall_offset*/ - 0, /*tp_getattr*/ - 0, /*tp_setattr*/ - 0, /*tp_as_async*/ - 0, /*tp_repr*/ - 0, /*tp_as_number*/ - 0, /*tp_as_sequence*/ - 0, /*tp_as_mapping*/ - 0, /*tp_hash*/ - 0, /*tp_call*/ - 0, /*tp_str*/ - 0, /*tp_getattro*/ - 0, /*tp_setattro*/ - 0, /*tp_as_buffer*/ - Py_TPFLAGS_DEFAULT, /*tp_flags*/ - 0, /*tp_doc*/ - 0, /*tp_traverse*/ - 0, /*tp_clear*/ - 0, /*tp_richcompare*/ - 0, /*tp_weaklistoffset*/ - 0, /*tp_iter*/ - 0, /*tp_iternext*/ - SHA_methods, /* tp_methods */ - SHA_members, /* tp_members */ - SHA_getseters, /* tp_getset */ +static PyType_Slot sha256_types_slots[] = { + {Py_tp_dealloc, SHA_dealloc}, + {Py_tp_methods, SHA_methods}, + {Py_tp_members, SHA_members}, + {Py_tp_getset, SHA_getseters}, + {0,0} }; -static PyTypeObject SHA256type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_sha256.sha256", /*tp_name*/ - sizeof(SHAobject), /*tp_basicsize*/ - 0, /*tp_itemsize*/ - /* methods */ - SHA_dealloc, /*tp_dealloc*/ - 0, /*tp_vectorcall_offset*/ - 0, /*tp_getattr*/ - 0, /*tp_setattr*/ - 0, /*tp_as_async*/ - 0, /*tp_repr*/ - 0, /*tp_as_number*/ - 0, /*tp_as_sequence*/ - 0, /*tp_as_mapping*/ - 0, /*tp_hash*/ - 0, /*tp_call*/ - 0, /*tp_str*/ - 0, /*tp_getattro*/ - 0, /*tp_setattro*/ - 0, /*tp_as_buffer*/ - Py_TPFLAGS_DEFAULT, /*tp_flags*/ - 0, /*tp_doc*/ - 0, /*tp_traverse*/ - 0, /*tp_clear*/ - 0, /*tp_richcompare*/ - 0, /*tp_weaklistoffset*/ - 0, /*tp_iter*/ - 0, /*tp_iternext*/ - SHA_methods, /* tp_methods */ - SHA_members, /* tp_members */ - SHA_getseters, /* tp_getset */ +static PyType_Spec sha224_type_spec = { + .name = "_sha256.sha224", + .basicsize = sizeof(SHAobject), + .flags = Py_TPFLAGS_DEFAULT, + .slots = sha256_types_slots }; +static PyType_Spec sha256_type_spec = { + .name = "_sha256.sha256", + .basicsize = sizeof(SHAobject), + .flags = Py_TPFLAGS_DEFAULT, + .slots = sha256_types_slots +}; /* The single module-level function: new() */ @@ -602,15 +571,19 @@ static PyObject * _sha256_sha256_impl(PyObject *module, PyObject *string, int usedforsecurity) /*[clinic end generated code: output=a1de327e8e1185cf input=9be86301aeb14ea5]*/ { - SHAobject *new; Py_buffer buf; - if (string) + if (string) { GET_BUFFER_VIEW_OR_ERROUT(string, &buf); + } - if ((new = newSHA256object()) == NULL) { - if (string) + _sha256_state *state = PyModule_GetState(module); + + SHAobject *new; + if ((new = newSHA256object(state)) == NULL) { + if (string) { PyBuffer_Release(&buf); + } return NULL; } @@ -618,8 +591,9 @@ _sha256_sha256_impl(PyObject *module, PyObject *string, int usedforsecurity) if (PyErr_Occurred()) { Py_DECREF(new); - if (string) + if (string) { PyBuffer_Release(&buf); + } return NULL; } if (string) { @@ -644,15 +618,17 @@ static PyObject * _sha256_sha224_impl(PyObject *module, PyObject *string, int usedforsecurity) /*[clinic end generated code: output=08be6b36569bc69c input=9fcfb46e460860ac]*/ { - SHAobject *new; Py_buffer buf; - - if (string) + if (string) { GET_BUFFER_VIEW_OR_ERROUT(string, &buf); + } - if ((new = newSHA224object()) == NULL) { - if (string) + _sha256_state *state = PyModule_GetState(module); + SHAobject *new; + if ((new = newSHA224object(state)) == NULL) { + if (string) { PyBuffer_Release(&buf); + } return NULL; } @@ -660,8 +636,9 @@ _sha256_sha224_impl(PyObject *module, PyObject *string, int usedforsecurity) if (PyErr_Occurred()) { Py_DECREF(new); - if (string) + if (string) { PyBuffer_Release(&buf); + } return NULL; } if (string) { @@ -681,25 +658,56 @@ static struct PyMethodDef SHA_functions[] = { {NULL, NULL} /* Sentinel */ }; +static int +_sha256_traverse(PyObject *module, visitproc visit, void *arg) +{ + _sha256_state *state = _sha256_get_state(module); + Py_VISIT(state->sha224_type); + Py_VISIT(state->sha256_type); + return 0; +} + +static int +_sha256_clear(PyObject *module) +{ + _sha256_state *state = _sha256_get_state(module); + Py_CLEAR(state->sha224_type); + Py_CLEAR(state->sha256_type); + return 0; +} + +static void +_sha256_free(void *module) +{ + _sha256_clear((PyObject *)module); +} + static int sha256_exec(PyObject *module) { - Py_SET_TYPE(&SHA224type, &PyType_Type); - if (PyType_Ready(&SHA224type) < 0) { + _sha256_state *state = _sha256_get_state(module); + + state->sha224_type = (PyTypeObject *)PyType_FromModuleAndSpec( + module, &sha224_type_spec, NULL); + + if (state->sha224_type == NULL) { return -1; } - Py_SET_TYPE(&SHA256type, &PyType_Type); - if (PyType_Ready(&SHA256type) < 0) { + + state->sha256_type = (PyTypeObject *)PyType_FromModuleAndSpec( + module, &sha256_type_spec, NULL); + + if (state->sha256_type == NULL) { return -1; } - Py_INCREF((PyObject *)&SHA224type); - if (PyModule_AddObject(module, "SHA224Type", (PyObject *)&SHA224type) < 0) { - Py_DECREF((PyObject *)&SHA224type); + Py_INCREF((PyObject *)state->sha224_type); + if (PyModule_AddObject(module, "SHA224Type", (PyObject *)state->sha224_type) < 0) { + Py_DECREF((PyObject *)state->sha224_type); return -1; } - Py_INCREF((PyObject *)&SHA256type); - if (PyModule_AddObject(module, "SHA256Type", (PyObject *)&SHA256type) < 0) { - Py_DECREF((PyObject *)&SHA256type); + Py_INCREF((PyObject *)state->sha256_type); + if (PyModule_AddObject(module, "SHA256Type", (PyObject *)state->sha256_type) < 0) { + Py_DECREF((PyObject *)state->sha256_type); return -1; } return 0; @@ -713,8 +721,12 @@ static PyModuleDef_Slot _sha256_slots[] = { static struct PyModuleDef _sha256module = { PyModuleDef_HEAD_INIT, .m_name = "_sha256", + .m_size = sizeof(_sha256_state), .m_methods = SHA_functions, .m_slots = _sha256_slots, + .m_traverse = _sha256_traverse, + .m_clear = _sha256_clear, + .m_free = _sha256_free }; /* Initialize this module. */ From webhook-mailer at python.org Tue Sep 8 09:33:36 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 08 Sep 2020 13:33:36 -0000 Subject: [Python-checkins] bpo-1635741: Port _string module to multi-phase init (GH-22148) Message-ID: https://github.com/python/cpython/commit/bb083d33f7ffe93cee9e1f63d1e526dc81a6e34f commit: bb083d33f7ffe93cee9e1f63d1e526dc81a6e34f branch: master author: Victor Stinner committer: GitHub date: 2020-09-08T15:33:08+02:00 summary: bpo-1635741: Port _string module to multi-phase init (GH-22148) Port the _string extension module to the multi-phase initialization API (PEP 489). files: A Misc/NEWS.d/next/Library/2020-09-08-13-51-16.bpo-1635741.wkPeoT.rst M Objects/unicodeobject.c diff --git a/Misc/NEWS.d/next/Library/2020-09-08-13-51-16.bpo-1635741.wkPeoT.rst b/Misc/NEWS.d/next/Library/2020-09-08-13-51-16.bpo-1635741.wkPeoT.rst new file mode 100644 index 0000000000000..972d69b94b6ba --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-08-13-51-16.bpo-1635741.wkPeoT.rst @@ -0,0 +1,2 @@ +Port the ``_string`` extension module to the multi-phase initialization API +(:pep:`489`). diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 82e09ad05fcd1..fd0e8e008adae 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -16243,20 +16243,16 @@ static PyMethodDef _string_methods[] = { static struct PyModuleDef _string_module = { PyModuleDef_HEAD_INIT, - "_string", - PyDoc_STR("string helper module"), - 0, - _string_methods, - NULL, - NULL, - NULL, - NULL + .m_name = "_string", + .m_doc = PyDoc_STR("string helper module"), + .m_size = 0, + .m_methods = _string_methods, }; PyMODINIT_FUNC PyInit__string(void) { - return PyModule_Create(&_string_module); + return PyModuleDef_Init(&_string_module); } From webhook-mailer at python.org Tue Sep 8 09:34:00 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 08 Sep 2020 13:34:00 -0000 Subject: [Python-checkins] bpo-1635741: Port mashal module to multi-phase init (#22149) Message-ID: https://github.com/python/cpython/commit/f315142ddc61e54a59028db562aec5f62db783e1 commit: f315142ddc61e54a59028db562aec5f62db783e1 branch: master author: Victor Stinner committer: GitHub date: 2020-09-08T15:33:52+02:00 summary: bpo-1635741: Port mashal module to multi-phase init (#22149) Port the 'mashal' extension module to the multi-phase initialization API (PEP 489). files: A Misc/NEWS.d/next/Library/2020-09-08-13-55-34.bpo-1635741.56MLP-.rst M Python/marshal.c diff --git a/Misc/NEWS.d/next/Library/2020-09-08-13-55-34.bpo-1635741.56MLP-.rst b/Misc/NEWS.d/next/Library/2020-09-08-13-55-34.bpo-1635741.56MLP-.rst new file mode 100644 index 0000000000000..8b5bd5efdc2c0 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-08-13-55-34.bpo-1635741.56MLP-.rst @@ -0,0 +1,2 @@ +Port the ``mashal`` extension module to the multi-phase initialization API +(:pep:`489`). diff --git a/Python/marshal.c b/Python/marshal.c index c4538bd373a82..91a0f8acb1248 100644 --- a/Python/marshal.c +++ b/Python/marshal.c @@ -1785,28 +1785,30 @@ dumps() -- marshal value as a bytes object\n\ loads() -- read value from a bytes-like object"); +static int +marshal_module_exec(PyObject *mod) +{ + if (PyModule_AddIntConstant(mod, "version", Py_MARSHAL_VERSION) < 0) { + return -1; + } + return 0; +} + +static PyModuleDef_Slot marshalmodule_slots[] = { + {Py_mod_exec, marshal_module_exec}, + {0, NULL} +}; static struct PyModuleDef marshalmodule = { PyModuleDef_HEAD_INIT, - "marshal", - module_doc, - 0, - marshal_methods, - NULL, - NULL, - NULL, - NULL + .m_name = "marshal", + .m_doc = module_doc, + .m_methods = marshal_methods, + .m_slots = marshalmodule_slots, }; PyMODINIT_FUNC PyMarshal_Init(void) { - PyObject *mod = PyModule_Create(&marshalmodule); - if (mod == NULL) - return NULL; - if (PyModule_AddIntConstant(mod, "version", Py_MARSHAL_VERSION) < 0) { - Py_DECREF(mod); - return NULL; - } - return mod; + return PyModuleDef_Init(&marshalmodule); } From webhook-mailer at python.org Tue Sep 8 12:47:23 2020 From: webhook-mailer at python.org (Mark Shannon) Date: Tue, 08 Sep 2020 16:47:23 -0000 Subject: [Python-checkins] Fix incorrect bpo number in change notes. (GH-22151) Message-ID: https://github.com/python/cpython/commit/2de50f26008deb29ba04f7ad7c79e451d5aebba8 commit: 2de50f26008deb29ba04f7ad7c79e451d5aebba8 branch: master author: Mark Shannon committer: GitHub date: 2020-09-08T17:47:14+01:00 summary: Fix incorrect bpo number in change notes. (GH-22151) files: M Misc/NEWS.d/3.9.0a2.rst diff --git a/Misc/NEWS.d/3.9.0a2.rst b/Misc/NEWS.d/3.9.0a2.rst index 50478c08e9018..1fd23b763e2a1 100644 --- a/Misc/NEWS.d/3.9.0a2.rst +++ b/Misc/NEWS.d/3.9.0a2.rst @@ -229,7 +229,7 @@ coroutine of an asynchronous generator. .. -.. bpo: 32949 +.. bpo: 33387 .. date: 2018-03-13-14-46-03 .. nonce: v821M7 .. section: Core and Builtins From webhook-mailer at python.org Tue Sep 8 05:39:38 2020 From: webhook-mailer at python.org (=?utf-8?q?=C5=81ukasz?= Langa) Date: Tue, 08 Sep 2020 09:39:38 -0000 Subject: [Python-checkins] (no subject) Message-ID: To: python-checkins at python.org Subject: Python 3.8.6rc1 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 https://github.com/python/cpython/commit/08bd63da6e64c49130d244ecf8f246089b9c= 5d9f commit: 08bd63da6e64c49130d244ecf8f246089b9c5d9f branch: 3.8 author: =C5=81ukasz Langa committer: =C5=81ukasz Langa date: 2020-09-07T17:52:19+02:00 summary: Python 3.8.6rc1 files: A Misc/NEWS.d/3.8.6rc1.rst D Misc/NEWS.d/next/C API/2020-08-12-17-09-06.bpo-41524.u6Xfr2.rst D Misc/NEWS.d/next/Core and Builtins/2020-07-20-17-01-17.bpo-38156.ptcdRy.rst D Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-41533.4pcVAc.rst D Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst D Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst D Misc/NEWS.d/next/Documentation/2020-07-25-14-20-00.bpo-41314.yrjko0.rst D Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF6Ul.rst D Misc/NEWS.d/next/Documentation/2020-08-12-18-35-40.bpo-40204.C8A_pe.rst D Misc/NEWS.d/next/Documentation/2020-08-25-15-11-23.bpo-41624.ddjJlN.rst D Misc/NEWS.d/next/IDLE/2020-07-24-17-49-58.bpo-41373.YQIPu_.rst D Misc/NEWS.d/next/IDLE/2020-08-09-13-42-55.bpo-41468.zkP0_Y.rst D Misc/NEWS.d/next/Library/2018-06-12-23-30-41.bpo-33660.AdDn5Z.rst D Misc/NEWS.d/next/Library/2020-03-11-07-44-06.bpo-31122.zIQ80l.rst D Misc/NEWS.d/next/Library/2020-05-22-12-45-58.bpo-40726.7oBdMw.rst D Misc/NEWS.d/next/Library/2020-07-20-13-27-48.bpo-41344.iKipNd.rst D Misc/NEWS.d/next/Library/2020-07-21-21-45-55.bpo-41364.5O-k7A.rst D Misc/NEWS.d/next/Library/2020-07-25-23-18-51.bpo-38731.Am4wp2.rst D Misc/NEWS.d/next/Library/2020-07-26-21-18-43.bpo-41384.MlzIgV.rst D Misc/NEWS.d/next/Library/2020-08-03-01-59-48.bpo-41425.KJo6zF.rst D Misc/NEWS.d/next/Library/2020-08-04-00-20-30.bpo-41467.Z8DgTL.rst D Misc/NEWS.d/next/Library/2020-08-05-23-16-39.bpo-41490.6z47A_.rst D Misc/NEWS.d/next/Library/2020-08-07-06-06-29.bpo-41497.aBtsWz.rst D Misc/NEWS.d/next/Library/2020-08-07-15-18-16.bpo-41503.IYftcu.rst D Misc/NEWS.d/next/Library/2020-08-12-13-25-16.bpo-41520.BEUWa4.rst D Misc/NEWS.d/next/Library/2020-08-13-08-07-25.bpo-40782.aGZqmB.rst D Misc/NEWS.d/next/Library/2020-08-15-15-21-40.bpo-37658.f9nivB.rst D Misc/NEWS.d/next/Library/2020-08-15-15-50-12.bpo-32751.85je5X.rst D Misc/NEWS.d/next/Library/2020-08-21-15-51-15.bpo-41609.JmiUKG.rst D Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rst D Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst D Misc/NEWS.d/next/Windows/2020-07-20-23-26-26.bpo-40741.C9sc_d.rst D Misc/NEWS.d/next/Windows/2020-08-06-16-59-10.bpo-41492.2FQ9cM.rst M Include/patchlevel.h M Lib/pydoc_data/topics.py M README.rst diff --git a/Include/patchlevel.h b/Include/patchlevel.h index 278f33fcf9b72..d6d0908576cc9 100644 --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -18,12 +18,12 @@ /*--start constants--*/ #define PY_MAJOR_VERSION 3 #define PY_MINOR_VERSION 8 -#define PY_MICRO_VERSION 5 -#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL -#define PY_RELEASE_SERIAL 0 +#define PY_MICRO_VERSION 6 +#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA +#define PY_RELEASE_SERIAL 1 =20 /* Version as a string */ -#define PY_VERSION "3.8.5+" +#define PY_VERSION "3.8.6rc1" /*--end constants--*/ =20 /* Version as a single 4-byte hex number, e.g. 0x010502B2 =3D=3D 1.5.2b2. diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py index 68346572bc3ee..ab53a563a43bd 100644 --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Mon Jul 20 14:14:54 2020 +# Autogenerated by Sphinx on Mon Sep 7 17:49:24 2020 topics =3D {'assert': 'The "assert" statement\n' '**********************\n' '\n' diff --git a/Misc/NEWS.d/3.8.6rc1.rst b/Misc/NEWS.d/3.8.6rc1.rst new file mode 100644 index 0000000000000..7340e9deca014 --- /dev/null +++ b/Misc/NEWS.d/3.8.6rc1.rst @@ -0,0 +1,312 @@ +.. bpo: 41654 +.. date: 2020-08-30-20-38-33 +.. nonce: HtnhAM +.. release date: 2020-09-07 +.. section: Core and Builtins + +Fix a crash that occurred when destroying subclasses of +:class:`MemoryError`. Patch by Pablo Galindo. + +.. + +.. bpo: 41533 +.. date: 2020-08-12-20-29-57 +.. nonce: 4pcVAc +.. section: Core and Builtins + +Free the stack allocated in ``va_build_stack`` if ``do_mkstack`` fails and +the stack is not a ``small_stack``. + +.. + +.. bpo: 38156 +.. date: 2020-07-20-17-01-17 +.. nonce: ptcdRy +.. section: Core and Builtins + +Handle interrupts that come after EOF correctly in ``PyOS_StdioReadline``. + +.. + +.. bpo: 41696 +.. date: 2020-09-03-01-35-32 +.. nonce: zkYGre +.. section: Library + +Fix handling of debug mode in :func:`asyncio.run`. This allows setting +``PYTHONASYNCIODEBUG`` or ``-X dev`` to enable asyncio debug mode when using +:func:`asyncio.run`. + +.. + +.. bpo: 39010 +.. date: 2020-08-30-10-24-26 +.. nonce: _mzXJW +.. section: Library + +Restarting a ``ProactorEventLoop`` on Windows no longer logs spurious +``ConnectionResetErrors``. + +.. + +.. bpo: 41609 +.. date: 2020-08-21-15-51-15 +.. nonce: JmiUKG +.. section: Library + +The pdb whatis command correctly reports instance methods as 'Method' rather +than 'Function'. + +.. + +.. bpo: 32751 +.. date: 2020-08-15-15-50-12 +.. nonce: 85je5X +.. section: Library + +When cancelling the task due to a timeout, :meth:`asyncio.wait_for` will now +wait until the cancellation is complete also in the case when *timeout* is +<=3D 0, like it does with positive timeouts. + +.. + +.. bpo: 37658 +.. date: 2020-08-15-15-21-40 +.. nonce: f9nivB +.. section: Library + +:meth:`asyncio.wait_for` now properly handles races between cancellation of +itself and the completion of the wrapped awaitable. + +.. + +.. bpo: 40782 +.. date: 2020-08-13-08-07-25 +.. nonce: aGZqmB +.. section: Library + +Change the method asyncio.AbstractEventLoop.run_in_executor to not be a +coroutine. + +.. + +.. bpo: 41520 +.. date: 2020-08-12-13-25-16 +.. nonce: BEUWa4 +.. section: Library + +Fix :mod:`codeop` regression that prevented turning compile warnings into +errors. + +.. + +.. bpo: 41503 +.. date: 2020-08-07-15-18-16 +.. nonce: IYftcu +.. section: Library + +Fixed a race between setTarget and flush in logging.handlers.MemoryHandler. + +.. + +.. bpo: 41497 +.. date: 2020-08-07-06-06-29 +.. nonce: aBtsWz +.. section: Library + +Fix potential UnicodeDecodeError in dis module. + +.. + +.. bpo: 41490 +.. date: 2020-08-05-23-16-39 +.. nonce: 6z47A_ +.. section: Library + +Update :mod:`ensurepip` to install pip 20.2.1 and setuptools 49.2.1. + +.. + +.. bpo: 41467 +.. date: 2020-08-04-00-20-30 +.. nonce: Z8DgTL +.. section: Library + +On Windows, fix asyncio ``recv_into()`` return value when the socket/pipe is +closed (:exc:`BrokenPipeError`): return ``0`` rather than an empty byte +string (``b''``). + +.. + +.. bpo: 41425 +.. date: 2020-08-03-01-59-48 +.. nonce: KJo6zF +.. section: Library + +Make tkinter doc example runnable. + +.. + +.. bpo: 41384 +.. date: 2020-07-26-21-18-43 +.. nonce: MlzIgV +.. section: Library + +Raise TclError instead of TypeError when an unknown option is passed to +tkinter.OptionMenu. + +.. + +.. bpo: 38731 +.. date: 2020-07-25-23-18-51 +.. nonce: Am4wp2 +.. section: Library + +Fix :exc:`NameError` in command-line interface of :mod:`py_compile`. + +.. + +.. bpo: 41364 +.. date: 2020-07-21-21-45-55 +.. nonce: 5O-k7A +.. section: Library + +Reduce import overhead of :mod:`uuid`. + +.. + +.. bpo: 41344 +.. date: 2020-07-20-13-27-48 +.. nonce: iKipNd +.. section: Library + +Prevent creating :class:`shared_memory.SharedMemory` objects with +:code:`size=3D0`. + +.. + +.. bpo: 40726 +.. date: 2020-05-22-12-45-58 +.. nonce: 7oBdMw +.. section: Library + +Handle cases where the ``end_lineno`` is ``None`` on +:func:`ast.increment_lineno`. + +.. + +.. bpo: 31122 +.. date: 2020-03-11-07-44-06 +.. nonce: zIQ80l +.. section: Library + +ssl.wrap_socket() now raises ssl.SSLEOFError rather than OSError when peer +closes connection during TLS negotiation + +.. + +.. bpo: 33660 +.. date: 2018-06-12-23-30-41 +.. nonce: AdDn5Z +.. section: Library + +Fix pathlib.PosixPath to resolve a relative path located on the root +directory properly. + +.. + +.. bpo: 41624 +.. date: 2020-08-25-15-11-23 +.. nonce: ddjJlN +.. section: Documentation + +Fix the signature of :class:`typing.Coroutine`. + +.. + +.. bpo: 40204 +.. date: 2020-08-12-18-35-40 +.. nonce: C8A_pe +.. section: Documentation + +Enable Sphinx 3.2 ``c_allow_pre_v3`` option and disable +``c_warn_on_allowed_pre_v3`` option to make the documentation compatible +with Sphinx 2 and Sphinx 3. + +.. + +.. bpo: 41045 +.. date: 2020-07-27-20-46-17 +.. nonce: GFF6Ul +.. section: Documentation + +Add documentation for debug feature of f-strings. + +.. + +.. bpo: 41314 +.. date: 2020-07-25-14-20-00 +.. nonce: yrjko0 +.. section: Documentation + +Changed the release when ``from __future__ import annotations`` becomes the +default from ``4.0`` to ``3.10`` (following a change in PEP 563). + +.. + +.. bpo: 39883 +.. date: 2020-03-07-03-53-39 +.. nonce: 1tnb4- +.. section: Documentation + +Make code, examples, and recipes in the Python documentation be licensed +under the more permissive BSD0 license in addition to the existing Python +2.0 license. + +.. + +.. bpo: 41492 +.. date: 2020-08-06-16-59-10 +.. nonce: 2FQ9cM +.. section: Windows + +Fixes the description that appears in UAC prompts. + +.. + +.. bpo: 40741 +.. date: 2020-07-20-23-26-26 +.. nonce: C9sc_d +.. section: Windows + +Update Windows release to include SQLite 3.32.3. + +.. + +.. bpo: 41468 +.. date: 2020-08-09-13-42-55 +.. nonce: zkP0_Y +.. section: IDLE + +Improve IDLE run crash error message (which users should never see). + +.. + +.. bpo: 41373 +.. date: 2020-07-24-17-49-58 +.. nonce: YQIPu_ +.. section: IDLE + +Save files loaded with no line ending, as when blank, or different line +endings, by setting its line ending to the system default. Fix regression in +3.8.4 and 3.9.0b4. + +.. + +.. bpo: 41524 +.. date: 2020-08-12-17-09-06 +.. nonce: u6Xfr2 +.. section: C API + +Fix bug in PyOS_mystrnicmp and PyOS_mystricmp that incremented pointers +beyond the end of a string. diff --git a/Misc/NEWS.d/next/C API/2020-08-12-17-09-06.bpo-41524.u6Xfr2.rst = b/Misc/NEWS.d/next/C API/2020-08-12-17-09-06.bpo-41524.u6Xfr2.rst deleted file mode 100644 index 4704e29be29bb..0000000000000 --- a/Misc/NEWS.d/next/C API/2020-08-12-17-09-06.bpo-41524.u6Xfr2.rst=09 +++ /dev/null @@ -1,2 +0,0 @@ -Fix bug in PyOS_mystrnicmp and PyOS_mystricmp that incremented -pointers beyond the end of a string. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-07-20-17-01-17.bpo-38156= .ptcdRy.rst b/Misc/NEWS.d/next/Core and Builtins/2020-07-20-17-01-17.bpo-3815= 6.ptcdRy.rst deleted file mode 100644 index 254d13cf3ed3a..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-07-20-17-01-17.bpo-38156.ptcdRy= .rst=09 +++ /dev/null @@ -1 +0,0 @@ -Handle interrupts that come after EOF correctly in ``PyOS_StdioReadline``. diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-41533= .4pcVAc.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-4153= 3.4pcVAc.rst deleted file mode 100644 index e166f0c0b621a..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-41533.4pcVAc= .rst=09 +++ /dev/null @@ -1,2 +0,0 @@ -Free the stack allocated in ``va_build_stack`` if ``do_mkstack`` fails and -the stack is not a ``small_stack``. diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654= .HtnhAM.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-4165= 4.HtnhAM.rst deleted file mode 100644 index e05c3133e1262..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM= .rst=09 +++ /dev/null @@ -1,2 +0,0 @@ -Fix a crash that occurred when destroying subclasses of -:class:`MemoryError`. Patch by Pablo Galindo. diff --git a/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tn= b4-.rst b/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-= .rst deleted file mode 100644 index 4941d50a560e2..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst +++ /dev/null @@ -1 +0,0 @@ -Make code, examples, and recipes in the Python documentation be licensed und= er the more permissive BSD0 license in addition to the existing Python 2.0 li= cense. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Documentation/2020-07-25-14-20-00.bpo-41314.yrj= ko0.rst b/Misc/NEWS.d/next/Documentation/2020-07-25-14-20-00.bpo-41314.yrjko0= .rst deleted file mode 100644 index 48f9c933828b0..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-07-25-14-20-00.bpo-41314.yrjko0.rst +++ /dev/null @@ -1 +0,0 @@ -Changed the release when ``from __future__ import annotations`` becomes the = default from ``4.0`` to ``3.10`` (following a change in PEP 563). diff --git a/Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF= 6Ul.rst b/Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF6Ul= .rst deleted file mode 100644 index dfc9891bb89f2..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-07-27-20-46-17.bpo-41045.GFF6Ul.rst +++ /dev/null @@ -1 +0,0 @@ -Add documentation for debug feature of f-strings. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Documentation/2020-08-12-18-35-40.bpo-40204.C8A= _pe.rst b/Misc/NEWS.d/next/Documentation/2020-08-12-18-35-40.bpo-40204.C8A_pe= .rst deleted file mode 100644 index 152f6c98b9004..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-08-12-18-35-40.bpo-40204.C8A_pe.rst +++ /dev/null @@ -1,3 +0,0 @@ -Enable Sphinx 3.2 ``c_allow_pre_v3`` option and disable -``c_warn_on_allowed_pre_v3`` option to make the documentation compatible -with Sphinx 2 and Sphinx 3. diff --git a/Misc/NEWS.d/next/Documentation/2020-08-25-15-11-23.bpo-41624.ddj= JlN.rst b/Misc/NEWS.d/next/Documentation/2020-08-25-15-11-23.bpo-41624.ddjJlN= .rst deleted file mode 100644 index bdbc5a445f339..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-08-25-15-11-23.bpo-41624.ddjJlN.rst +++ /dev/null @@ -1 +0,0 @@ -Fix the signature of :class:`typing.Coroutine`. diff --git a/Misc/NEWS.d/next/IDLE/2020-07-24-17-49-58.bpo-41373.YQIPu_.rst b= /Misc/NEWS.d/next/IDLE/2020-07-24-17-49-58.bpo-41373.YQIPu_.rst deleted file mode 100644 index b50a72fe676a8..0000000000000 --- a/Misc/NEWS.d/next/IDLE/2020-07-24-17-49-58.bpo-41373.YQIPu_.rst +++ /dev/null @@ -1,3 +0,0 @@ -Save files loaded with no line ending, as when blank, or different line -endings, by setting its line ending to the system default. Fix regression in -3.8.4 and 3.9.0b4. diff --git a/Misc/NEWS.d/next/IDLE/2020-08-09-13-42-55.bpo-41468.zkP0_Y.rst b= /Misc/NEWS.d/next/IDLE/2020-08-09-13-42-55.bpo-41468.zkP0_Y.rst deleted file mode 100644 index e41c7d574905c..0000000000000 --- a/Misc/NEWS.d/next/IDLE/2020-08-09-13-42-55.bpo-41468.zkP0_Y.rst +++ /dev/null @@ -1 +0,0 @@ -Improve IDLE run crash error message (which users should never see). diff --git a/Misc/NEWS.d/next/Library/2018-06-12-23-30-41.bpo-33660.AdDn5Z.rs= t b/Misc/NEWS.d/next/Library/2018-06-12-23-30-41.bpo-33660.AdDn5Z.rst deleted file mode 100644 index cce3dbb1c6ea5..0000000000000 --- a/Misc/NEWS.d/next/Library/2018-06-12-23-30-41.bpo-33660.AdDn5Z.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix pathlib.PosixPath to resolve a relative path located on the root -directory properly. diff --git a/Misc/NEWS.d/next/Library/2020-03-11-07-44-06.bpo-31122.zIQ80l.rs= t b/Misc/NEWS.d/next/Library/2020-03-11-07-44-06.bpo-31122.zIQ80l.rst deleted file mode 100644 index 2e70f7aee65c8..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-03-11-07-44-06.bpo-31122.zIQ80l.rst +++ /dev/null @@ -1 +0,0 @@ -ssl.wrap_socket() now raises ssl.SSLEOFError rather than OSError when peer c= loses connection during TLS negotiation \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2020-05-22-12-45-58.bpo-40726.7oBdMw.rs= t b/Misc/NEWS.d/next/Library/2020-05-22-12-45-58.bpo-40726.7oBdMw.rst deleted file mode 100644 index 7409eb3d80df6..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-05-22-12-45-58.bpo-40726.7oBdMw.rst +++ /dev/null @@ -1,2 +0,0 @@ -Handle cases where the ``end_lineno`` is ``None`` on -:func:`ast.increment_lineno`. diff --git a/Misc/NEWS.d/next/Library/2020-07-20-13-27-48.bpo-41344.iKipNd.rs= t b/Misc/NEWS.d/next/Library/2020-07-20-13-27-48.bpo-41344.iKipNd.rst deleted file mode 100644 index 475bc9bddb0d5..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-07-20-13-27-48.bpo-41344.iKipNd.rst +++ /dev/null @@ -1 +0,0 @@ -Prevent creating :class:`shared_memory.SharedMemory` objects with :code:`siz= e=3D0`. diff --git a/Misc/NEWS.d/next/Library/2020-07-21-21-45-55.bpo-41364.5O-k7A.rs= t b/Misc/NEWS.d/next/Library/2020-07-21-21-45-55.bpo-41364.5O-k7A.rst deleted file mode 100644 index f136e892ae5fe..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-07-21-21-45-55.bpo-41364.5O-k7A.rst +++ /dev/null @@ -1 +0,0 @@ -Reduce import overhead of :mod:`uuid`. diff --git a/Misc/NEWS.d/next/Library/2020-07-25-23-18-51.bpo-38731.Am4wp2.rs= t b/Misc/NEWS.d/next/Library/2020-07-25-23-18-51.bpo-38731.Am4wp2.rst deleted file mode 100644 index 61c583dd1bdab..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-07-25-23-18-51.bpo-38731.Am4wp2.rst +++ /dev/null @@ -1 +0,0 @@ -Fix :exc:`NameError` in command-line interface of :mod:`py_compile`. diff --git a/Misc/NEWS.d/next/Library/2020-07-26-21-18-43.bpo-41384.MlzIgV.rs= t b/Misc/NEWS.d/next/Library/2020-07-26-21-18-43.bpo-41384.MlzIgV.rst deleted file mode 100644 index d797374a09e6f..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-07-26-21-18-43.bpo-41384.MlzIgV.rst +++ /dev/null @@ -1,2 +0,0 @@ -Raise TclError instead of TypeError when an unknown option is passed to -tkinter.OptionMenu. diff --git a/Misc/NEWS.d/next/Library/2020-08-03-01-59-48.bpo-41425.KJo6zF.rs= t b/Misc/NEWS.d/next/Library/2020-08-03-01-59-48.bpo-41425.KJo6zF.rst deleted file mode 100644 index 617df72faeb37..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-03-01-59-48.bpo-41425.KJo6zF.rst +++ /dev/null @@ -1 +0,0 @@ -Make tkinter doc example runnable. diff --git a/Misc/NEWS.d/next/Library/2020-08-04-00-20-30.bpo-41467.Z8DgTL.rs= t b/Misc/NEWS.d/next/Library/2020-08-04-00-20-30.bpo-41467.Z8DgTL.rst deleted file mode 100644 index f12693e117631..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-04-00-20-30.bpo-41467.Z8DgTL.rst +++ /dev/null @@ -1,3 +0,0 @@ -On Windows, fix asyncio ``recv_into()`` return value when the socket/pipe is -closed (:exc:`BrokenPipeError`): return ``0`` rather than an empty byte -string (``b''``). diff --git a/Misc/NEWS.d/next/Library/2020-08-05-23-16-39.bpo-41490.6z47A_.rs= t b/Misc/NEWS.d/next/Library/2020-08-05-23-16-39.bpo-41490.6z47A_.rst deleted file mode 100644 index e89180db00429..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-05-23-16-39.bpo-41490.6z47A_.rst +++ /dev/null @@ -1 +0,0 @@ -Update :mod:`ensurepip` to install pip 20.2.1 and setuptools 49.2.1. diff --git a/Misc/NEWS.d/next/Library/2020-08-07-06-06-29.bpo-41497.aBtsWz.rs= t b/Misc/NEWS.d/next/Library/2020-08-07-06-06-29.bpo-41497.aBtsWz.rst deleted file mode 100644 index 2c863ed7ffa3f..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-07-06-06-29.bpo-41497.aBtsWz.rst +++ /dev/null @@ -1 +0,0 @@ -Fix potential UnicodeDecodeError in dis module. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2020-08-07-15-18-16.bpo-41503.IYftcu.rs= t b/Misc/NEWS.d/next/Library/2020-08-07-15-18-16.bpo-41503.IYftcu.rst deleted file mode 100644 index c34996d881937..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-07-15-18-16.bpo-41503.IYftcu.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed a race between setTarget and flush in logging.handlers.MemoryHandler. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2020-08-12-13-25-16.bpo-41520.BEUWa4.rs= t b/Misc/NEWS.d/next/Library/2020-08-12-13-25-16.bpo-41520.BEUWa4.rst deleted file mode 100644 index 0e140d91bb4b6..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-12-13-25-16.bpo-41520.BEUWa4.rst +++ /dev/null @@ -1 +0,0 @@ -Fix :mod:`codeop` regression that prevented turning compile warnings into er= rors. diff --git a/Misc/NEWS.d/next/Library/2020-08-13-08-07-25.bpo-40782.aGZqmB.rs= t b/Misc/NEWS.d/next/Library/2020-08-13-08-07-25.bpo-40782.aGZqmB.rst deleted file mode 100644 index d4c7e0e2419df..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-13-08-07-25.bpo-40782.aGZqmB.rst +++ /dev/null @@ -1 +0,0 @@ -Change the method asyncio.AbstractEventLoop.run_in_executor to not be a coro= utine. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2020-08-15-15-21-40.bpo-37658.f9nivB.rs= t b/Misc/NEWS.d/next/Library/2020-08-15-15-21-40.bpo-37658.f9nivB.rst deleted file mode 100644 index 694fbbbf346dc..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-15-15-21-40.bpo-37658.f9nivB.rst +++ /dev/null @@ -1,2 +0,0 @@ -:meth:`asyncio.wait_for` now properly handles races between cancellation of -itself and the completion of the wrapped awaitable. diff --git a/Misc/NEWS.d/next/Library/2020-08-15-15-50-12.bpo-32751.85je5X.rs= t b/Misc/NEWS.d/next/Library/2020-08-15-15-50-12.bpo-32751.85je5X.rst deleted file mode 100644 index c172ce5d9e948..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-15-15-50-12.bpo-32751.85je5X.rst +++ /dev/null @@ -1,3 +0,0 @@ -When cancelling the task due to a timeout, :meth:`asyncio.wait_for` will now -wait until the cancellation is complete also in the case when *timeout* is -<=3D 0, like it does with positive timeouts. diff --git a/Misc/NEWS.d/next/Library/2020-08-21-15-51-15.bpo-41609.JmiUKG.rs= t b/Misc/NEWS.d/next/Library/2020-08-21-15-51-15.bpo-41609.JmiUKG.rst deleted file mode 100644 index ecaf40eee7bab..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-21-15-51-15.bpo-41609.JmiUKG.rst +++ /dev/null @@ -1 +0,0 @@ -The pdb whatis command correctly reports instance methods as 'Method' rather= than 'Function'. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rs= t b/Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rst deleted file mode 100644 index 0d9015b490e4c..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rst +++ /dev/null @@ -1,2 +0,0 @@ -Restarting a ``ProactorEventLoop`` on Windows no longer logs spurious -``ConnectionResetErrors``. diff --git a/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rs= t b/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst deleted file mode 100644 index 67bbbb857f18c..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst +++ /dev/null @@ -1 +0,0 @@ -Fix handling of debug mode in :func:`asyncio.run`. This allows setting ``PYT= HONASYNCIODEBUG`` or ``-X dev`` to enable asyncio debug mode when using :func= :`asyncio.run`. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Windows/2020-07-20-23-26-26.bpo-40741.C9sc_d.rs= t b/Misc/NEWS.d/next/Windows/2020-07-20-23-26-26.bpo-40741.C9sc_d.rst deleted file mode 100644 index 69b7cce43803c..0000000000000 --- a/Misc/NEWS.d/next/Windows/2020-07-20-23-26-26.bpo-40741.C9sc_d.rst +++ /dev/null @@ -1 +0,0 @@ -Update Windows release to include SQLite 3.32.3. diff --git a/Misc/NEWS.d/next/Windows/2020-08-06-16-59-10.bpo-41492.2FQ9cM.rs= t b/Misc/NEWS.d/next/Windows/2020-08-06-16-59-10.bpo-41492.2FQ9cM.rst deleted file mode 100644 index 065803e2c2075..0000000000000 --- a/Misc/NEWS.d/next/Windows/2020-08-06-16-59-10.bpo-41492.2FQ9cM.rst +++ /dev/null @@ -1 +0,0 @@ -Fixes the description that appears in UAC prompts. diff --git a/README.rst b/README.rst index 862a24d5efc03..f2b1e36aef1f6 100644 --- a/README.rst +++ b/README.rst @@ -1,5 +1,5 @@ -This is Python version 3.8.5 -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D +This is Python version 3.8.6rc1 +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D =20 .. image:: https://travis-ci.org/python/cpython.svg?branch=3D3.8 :alt: CPython build status on Travis CI From webhook-mailer at python.org Tue Sep 15 02:19:39 2020 From: webhook-mailer at python.org (Raymond Hettinger) Date: Tue, 15 Sep 2020 06:19:39 -0000 Subject: [Python-checkins] bpo-41513: Remove broken tests that fail on Gentoo (GH-22249) Message-ID: https://github.com/python/cpython/commit/95a8a0e01d1f4f49ad6b4d4f2ae3bbf93c462013 commit: 95a8a0e01d1f4f49ad6b4d4f2ae3bbf93c462013 branch: master author: Raymond Hettinger committer: GitHub date: 2020-09-14T17:13:49-07:00 summary: bpo-41513: Remove broken tests that fail on Gentoo (GH-22249) files: M Lib/test/test_math.py diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py index bbaa533c8489e..4d62eb1b11993 100644 --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -803,57 +803,6 @@ def testHypot(self): scale = FLOAT_MIN / 2.0 ** exp self.assertEqual(math.hypot(4*scale, 3*scale), 5*scale) - def testHypotAccuracy(self): - # Verify improved accuracy in cases that were known to be inaccurate. - - hypot = math.hypot - Decimal = decimal.Decimal - high_precision = decimal.Context(prec=500) - - for hx, hy in [ - # Cases with a 1 ulp error in Python 3.7 compiled with Clang - ('0x1.10e89518dca48p+29', '0x1.1970f7565b7efp+30'), - ('0x1.10106eb4b44a2p+29', '0x1.ef0596cdc97f8p+29'), - ('0x1.459c058e20bb7p+30', '0x1.993ca009b9178p+29'), - ('0x1.378371ae67c0cp+30', '0x1.fbe6619854b4cp+29'), - ('0x1.f4cd0574fb97ap+29', '0x1.50fe31669340ep+30'), - ('0x1.494b2cdd3d446p+29', '0x1.212a5367b4c7cp+29'), - ('0x1.f84e649f1e46dp+29', '0x1.1fa56bef8eec4p+30'), - ('0x1.2e817edd3d6fap+30', '0x1.eb0814f1e9602p+29'), - ('0x1.0d3a6e3d04245p+29', '0x1.32a62fea52352p+30'), - ('0x1.888e19611bfc5p+29', '0x1.52b8e70b24353p+29'), - - # Cases with 2 ulp error in Python 3.8 - ('0x1.538816d48a13fp+29', '0x1.7967c5ca43e16p+29'), - ('0x1.57b47b7234530p+29', '0x1.74e2c7040e772p+29'), - ('0x1.821b685e9b168p+30', '0x1.677dc1c1e3dc6p+29'), - ('0x1.9e8247f67097bp+29', '0x1.24bd2dc4f4baep+29'), - ('0x1.b73b59e0cb5f9p+29', '0x1.da899ab784a97p+28'), - ('0x1.94a8d2842a7cfp+30', '0x1.326a51d4d8d8ap+30'), - ('0x1.e930b9cd99035p+29', '0x1.5a1030e18dff9p+30'), - ('0x1.1592bbb0e4690p+29', '0x1.a9c337b33fb9ap+29'), - ('0x1.1243a50751fd4p+29', '0x1.a5a10175622d9p+29'), - ('0x1.57a8596e74722p+30', '0x1.42d1af9d04da9p+30'), - - # Cases with 1 ulp error in version fff3c28052e6b0750d6218e00acacd2fded4991a - ('0x1.ee7dbd9565899p+29', '0x1.7ab4d6fc6e4b4p+29'), - ('0x1.5c6bfbec5c4dcp+30', '0x1.02511184b4970p+30'), - ('0x1.59dcebba995cap+30', '0x1.50ca7e7c38854p+29'), - ('0x1.768cdd94cf5aap+29', '0x1.9cfdc5571d38ep+29'), - ('0x1.dcf137d60262ep+29', '0x1.1101621990b3ep+30'), - ('0x1.3a2d006e288b0p+30', '0x1.e9a240914326cp+29'), - ('0x1.62a32f7f53c61p+29', '0x1.47eb6cd72684fp+29'), - ('0x1.d3bcb60748ef2p+29', '0x1.3f13c4056312cp+30'), - ('0x1.282bdb82f17f3p+30', '0x1.640ba4c4eed3ap+30'), - ('0x1.89d8c423ea0c6p+29', '0x1.d35dcfe902bc3p+29'), - ]: - with self.subTest(hx=hx, hy=hy): - x = float.fromhex(hx) - y = float.fromhex(hy) - with decimal.localcontext(high_precision): - z = float((Decimal(x)**2 + Decimal(y)**2).sqrt()) - self.assertEqual(hypot(x, y), z) - def testDist(self): from decimal import Decimal as D from fractions import Fraction as F From webhook-mailer at python.org Tue Sep 15 02:20:25 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Tue, 15 Sep 2020 06:20:25 -0000 Subject: [Python-checkins] bpo-1635741: Port cmath to multi-phase init (PEP 489) (GH-22165) Message-ID: https://github.com/python/cpython/commit/f76d894dc5d5facce1a6c1b71637f6a2b3f9fd2b commit: f76d894dc5d5facce1a6c1b71637f6a2b3f9fd2b branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-10T16:09:04+02:00 summary: bpo-1635741: Port cmath to multi-phase init (PEP 489) (GH-22165) files: A Misc/NEWS.d/next/Core and Builtins/2020-09-08-21-58-47.bpo-1635741.vdjSLH.rst M Modules/cmathmodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-08-21-58-47.bpo-1635741.vdjSLH.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-08-21-58-47.bpo-1635741.vdjSLH.rst new file mode 100644 index 0000000000000..bc1a6c888e33b --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-08-21-58-47.bpo-1635741.vdjSLH.rst @@ -0,0 +1,2 @@ +Port the :mod:`cmath` extension module to multi-phase initialization +(:pep:`489`). diff --git a/Modules/cmathmodule.c b/Modules/cmathmodule.c index 5eac4b4940bea..0f22049a17084 100644 --- a/Modules/cmathmodule.c +++ b/Modules/cmathmodule.c @@ -1254,37 +1254,35 @@ static PyMethodDef cmath_methods[] = { {NULL, NULL} /* sentinel */ }; - -static struct PyModuleDef cmathmodule = { - PyModuleDef_HEAD_INIT, - "cmath", - module_doc, - -1, - cmath_methods, - NULL, - NULL, - NULL, - NULL -}; - -PyMODINIT_FUNC -PyInit_cmath(void) +static int +cmath_exec(PyObject *mod) { - PyObject *m; - - m = PyModule_Create(&cmathmodule); - if (m == NULL) - return NULL; - - PyModule_AddObject(m, "pi", - PyFloat_FromDouble(Py_MATH_PI)); - PyModule_AddObject(m, "e", PyFloat_FromDouble(Py_MATH_E)); - PyModule_AddObject(m, "tau", PyFloat_FromDouble(Py_MATH_TAU)); /* 2pi */ - PyModule_AddObject(m, "inf", PyFloat_FromDouble(m_inf())); - PyModule_AddObject(m, "infj", PyComplex_FromCComplex(c_infj())); + if (PyModule_AddObject(mod, "pi", PyFloat_FromDouble(Py_MATH_PI)) < 0) { + return -1; + } + if (PyModule_AddObject(mod, "e", PyFloat_FromDouble(Py_MATH_E)) < 0) { + return -1; + } + // 2pi + if (PyModule_AddObject(mod, "tau", PyFloat_FromDouble(Py_MATH_TAU)) < 0) { + return -1; + } + if (PyModule_AddObject(mod, "inf", PyFloat_FromDouble(m_inf())) < 0) { + return -1; + } + + if (PyModule_AddObject(mod, "infj", + PyComplex_FromCComplex(c_infj())) < 0) { + return -1; + } #if !defined(PY_NO_SHORT_FLOAT_REPR) || defined(Py_NAN) - PyModule_AddObject(m, "nan", PyFloat_FromDouble(m_nan())); - PyModule_AddObject(m, "nanj", PyComplex_FromCComplex(c_nanj())); + if (PyModule_AddObject(mod, "nan", PyFloat_FromDouble(m_nan())) < 0) { + return -1; + } + if (PyModule_AddObject(mod, "nanj", + PyComplex_FromCComplex(c_nanj())) < 0) { + return -1; + } #endif /* initialize special value tables */ @@ -1401,5 +1399,25 @@ PyInit_cmath(void) C(INF,N) C(U,U) C(INF,-0.) C(INF,0.) C(U,U) C(INF,N) C(INF,N) C(N,N) C(N,N) C(N,0.) C(N,0.) C(N,N) C(N,N) C(N,N) }) - return m; + return 0; } + +static PyModuleDef_Slot cmath_slots[] = { + {Py_mod_exec, cmath_exec}, + {0, NULL} +}; + +static struct PyModuleDef cmathmodule = { + PyModuleDef_HEAD_INIT, + .m_name = "cmath", + .m_doc = module_doc, + .m_size = 0, + .m_methods = cmath_methods, + .m_slots = cmath_slots +}; + +PyMODINIT_FUNC +PyInit_cmath(void) +{ + return PyModuleDef_Init(&cmathmodule); +} \ No newline at end of file From webhook-mailer at python.org Tue Sep 15 02:22:14 2020 From: webhook-mailer at python.org (Irit Katriel) Date: Tue, 15 Sep 2020 06:22:14 -0000 Subject: [Python-checkins] bpo-38762: Extend logging.test_multiprocessing to cover missing cases. (GH-22142) Message-ID: https://github.com/python/cpython/commit/3fd69991f47a6672c510fafd76bf183f17ac52ec commit: 3fd69991f47a6672c510fafd76bf183f17ac52ec branch: master author: Irit Katriel committer: GitHub date: 2020-09-08T20:40:04+01:00 summary: bpo-38762: Extend logging.test_multiprocessing to cover missing cases. (GH-22142) files: M Lib/test/test_logging.py diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index 00a4825d6da88..d23fbfb4fe281 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -4354,15 +4354,65 @@ def test_dict_arg(self): r.removeHandler(h) h.close() - def test_multiprocessing(self): - r = logging.makeLogRecord({}) - self.assertEqual(r.processName, 'MainProcess') + @staticmethod # pickled as target of child process in the following test + def _extract_logrecord_process_name(key, logMultiprocessing, conn=None): + prev_logMultiprocessing = logging.logMultiprocessing + logging.logMultiprocessing = logMultiprocessing try: import multiprocessing as mp + name = mp.current_process().name + + r1 = logging.makeLogRecord({'msg': f'msg1_{key}'}) + del sys.modules['multiprocessing'] + r2 = logging.makeLogRecord({'msg': f'msg2_{key}'}) + + results = {'processName' : name, + 'r1.processName': r1.processName, + 'r2.processName': r2.processName, + } + finally: + logging.logMultiprocessing = prev_logMultiprocessing + if conn: + conn.send(results) + else: + return results + + def test_multiprocessing(self): + multiprocessing_imported = 'multiprocessing' in sys.modules + try: + # logMultiprocessing is True by default + self.assertEqual(logging.logMultiprocessing, True) + + LOG_MULTI_PROCESSING = True + # When logMultiprocessing == True: + # In the main process processName = 'MainProcess' r = logging.makeLogRecord({}) - self.assertEqual(r.processName, mp.current_process().name) - except ImportError: - pass + self.assertEqual(r.processName, 'MainProcess') + + results = self._extract_logrecord_process_name(1, LOG_MULTI_PROCESSING) + self.assertEqual('MainProcess', results['processName']) + self.assertEqual('MainProcess', results['r1.processName']) + self.assertEqual('MainProcess', results['r2.processName']) + + # In other processes, processName is correct when multiprocessing in imported, + # but it is (incorrectly) defaulted to 'MainProcess' otherwise (bpo-38762). + import multiprocessing + parent_conn, child_conn = multiprocessing.Pipe() + p = multiprocessing.Process( + target=self._extract_logrecord_process_name, + args=(2, LOG_MULTI_PROCESSING, child_conn,) + ) + p.start() + results = parent_conn.recv() + self.assertNotEqual('MainProcess', results['processName']) + self.assertEqual(results['processName'], results['r1.processName']) + self.assertEqual('MainProcess', results['r2.processName']) + p.join() + + finally: + if multiprocessing_imported: + import multiprocessing + def test_optional(self): r = logging.makeLogRecord({}) From webhook-mailer at python.org Tue Sep 15 02:25:07 2020 From: webhook-mailer at python.org (Graham Bleaney) Date: Tue, 15 Sep 2020 06:25:07 -0000 Subject: [Python-checkins] Fix typo in typing.py (GH-22121) Message-ID: https://github.com/python/cpython/commit/84ef33c5117acd9867781135a9aeb62052432e8a commit: 84ef33c5117acd9867781135a9aeb62052432e8a branch: master author: Graham Bleaney committer: GitHub date: 2020-09-08T15:41:10-07:00 summary: Fix typo in typing.py (GH-22121) This is a trivial PR to fix a typo in a docstring in typing.py. From reverences -> references files: M Lib/typing.py diff --git a/Lib/typing.py b/Lib/typing.py index fce8da4fe3cf0..2899a0213d434 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -245,7 +245,7 @@ def inner(*args, **kwds): def _eval_type(t, globalns, localns, recursive_guard=frozenset()): - """Evaluate all forward reverences in the given type t. + """Evaluate all forward references in the given type t. For use of globalns and localns see the docstring for get_type_hints(). recursive_guard is used to prevent prevent infinite recursion with recursive ForwardRef. From webhook-mailer at python.org Tue Sep 15 02:26:57 2020 From: webhook-mailer at python.org (Mariatta) Date: Tue, 15 Sep 2020 06:26:57 -0000 Subject: [Python-checkins] [3.8] Fix typo in typing.py (GH-22121) (GH-22158) Message-ID: https://github.com/python/cpython/commit/7799dd38e78c800537569ef501a5b8daa03500cd commit: 7799dd38e78c800537569ef501a5b8daa03500cd branch: 3.8 author: Mariatta committer: GitHub date: 2020-09-08T16:28:02-07:00 summary: [3.8] Fix typo in typing.py (GH-22121) (GH-22158) This is a trivial PR to fix a typo in a docstring in typing.py. From reverences -> references. (cherry picked from commit 84ef33c5117acd9867781135a9aeb62052432e8a) Co-authored-by: Graham Bleaney Automerge-Triggered-By: @Mariatta files: M Lib/typing.py diff --git a/Lib/typing.py b/Lib/typing.py index 589eea98ad31c..4ec538da00a60 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -263,7 +263,7 @@ def inner(*args, **kwds): def _eval_type(t, globalns, localns): - """Evaluate all forward reverences in the given type t. + """Evaluate all forward references in the given type t. For use of globalns and localns see the docstring for get_type_hints(). """ if isinstance(t, ForwardRef): From webhook-mailer at python.org Tue Sep 15 02:27:55 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 06:27:55 -0000 Subject: [Python-checkins] [3.9] Fix typo in typing.py (GH-22121) (GH-22156) Message-ID: https://github.com/python/cpython/commit/6e770fc4fe511a3b40169307c91e9093c90f7eef commit: 6e770fc4fe511a3b40169307c91e9093c90f7eef branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-08T16:36:07-07:00 summary: [3.9] Fix typo in typing.py (GH-22121) (GH-22156) This is a trivial PR to fix a typo in a docstring in typing.py. From reverences -> references (cherry picked from commit 84ef33c5117acd9867781135a9aeb62052432e8a) Co-authored-by: Graham Bleaney files: M Lib/typing.py diff --git a/Lib/typing.py b/Lib/typing.py index 5da032bbee8f1..39c956dd1834d 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -245,7 +245,7 @@ def inner(*args, **kwds): def _eval_type(t, globalns, localns, recursive_guard=frozenset()): - """Evaluate all forward reverences in the given type t. + """Evaluate all forward references in the given type t. For use of globalns and localns see the docstring for get_type_hints(). recursive_guard is used to prevent prevent infinite recursion with recursive ForwardRef. From webhook-mailer at python.org Tue Sep 15 07:13:43 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Tue, 15 Sep 2020 11:13:43 -0000 Subject: [Python-checkins] [doc] Fix padding in timeit (GH-22152) Message-ID: https://github.com/python/cpython/commit/594f0ce73be807c0fecf958dc0644fa37983f5fe commit: 594f0ce73be807c0fecf958dc0644fa37983f5fe branch: master author: Andre Delfino committer: GitHub date: 2020-09-08T16:39:19-07:00 summary: [doc] Fix padding in timeit (GH-22152) Compare -p and -u options help in rendered output to see the difference. files: M Doc/library/timeit.rst diff --git a/Doc/library/timeit.rst b/Doc/library/timeit.rst index 46fa62c15fc2e..668fcb860cea8 100644 --- a/Doc/library/timeit.rst +++ b/Doc/library/timeit.rst @@ -233,7 +233,7 @@ Where the following options are understood: .. cmdoption:: -u, --unit=U - specify a time unit for timer output; can select nsec, usec, msec, or sec + specify a time unit for timer output; can select nsec, usec, msec, or sec .. versionadded:: 3.5 From webhook-mailer at python.org Tue Sep 15 07:14:31 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 11:14:31 -0000 Subject: [Python-checkins] [doc] Fix padding in timeit (GH-22152) Message-ID: https://github.com/python/cpython/commit/fd1c7258673704278d7377a6c9ef6e3db649aca4 commit: fd1c7258673704278d7377a6c9ef6e3db649aca4 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-08T16:56:02-07:00 summary: [doc] Fix padding in timeit (GH-22152) Compare -p and -u options help in rendered output to see the difference. (cherry picked from commit 594f0ce73be807c0fecf958dc0644fa37983f5fe) Co-authored-by: Andre Delfino files: M Doc/library/timeit.rst diff --git a/Doc/library/timeit.rst b/Doc/library/timeit.rst index 46fa62c15fc2e..668fcb860cea8 100644 --- a/Doc/library/timeit.rst +++ b/Doc/library/timeit.rst @@ -233,7 +233,7 @@ Where the following options are understood: .. cmdoption:: -u, --unit=U - specify a time unit for timer output; can select nsec, usec, msec, or sec + specify a time unit for timer output; can select nsec, usec, msec, or sec .. versionadded:: 3.5 From webhook-mailer at python.org Tue Sep 15 07:14:59 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 11:14:59 -0000 Subject: [Python-checkins] [3.8] [doc] Fix padding in timeit (GH-22152) (GH-22159) Message-ID: https://github.com/python/cpython/commit/68bc6792b4725939499b8b4eb7fa8bb6ccc190a2 commit: 68bc6792b4725939499b8b4eb7fa8bb6ccc190a2 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-08T16:57:11-07:00 summary: [3.8] [doc] Fix padding in timeit (GH-22152) (GH-22159) Compare -p and -u options help in rendered output to see the difference. (cherry picked from commit 594f0ce73be807c0fecf958dc0644fa37983f5fe) Co-authored-by: Andre Delfino files: M Doc/library/timeit.rst diff --git a/Doc/library/timeit.rst b/Doc/library/timeit.rst index 46fa62c15fc2e..668fcb860cea8 100644 --- a/Doc/library/timeit.rst +++ b/Doc/library/timeit.rst @@ -233,7 +233,7 @@ Where the following options are understood: .. cmdoption:: -u, --unit=U - specify a time unit for timer output; can select nsec, usec, msec, or sec + specify a time unit for timer output; can select nsec, usec, msec, or sec .. versionadded:: 3.5 From webhook-mailer at python.org Tue Sep 15 07:16:01 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Tue, 15 Sep 2020 11:16:01 -0000 Subject: [Python-checkins] bpo-41525: Make the Python program help ASCII-only (GH-21836) Message-ID: https://github.com/python/cpython/commit/58de1dd6a8677bd213802c19204b827cb7134695 commit: 58de1dd6a8677bd213802c19204b827cb7134695 branch: master author: Serhiy Storchaka committer: GitHub date: 2020-09-09T01:28:02+01:00 summary: bpo-41525: Make the Python program help ASCII-only (GH-21836) files: A Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst M Lib/test/test_cmd_line.py M Misc/python.man M Python/initconfig.c diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py index 4794d446f08c7..fa3329efa28b8 100644 --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -46,7 +46,11 @@ def test_site_flag(self): def test_usage(self): rc, out, err = assert_python_ok('-h') - self.assertIn(b'usage', out) + lines = out.splitlines() + self.assertIn(b'usage', lines[0]) + # The first line contains the program name, + # but the rest should be ASCII-only + b''.join(lines[1:]).decode('ascii') def test_version(self): version = ('Python %d.%d' % sys.version_info[:2]).encode("ascii") diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst new file mode 100644 index 0000000000000..acc00f8b992c9 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst @@ -0,0 +1 @@ +The output of ``python --help`` contains now only ASCII characters. diff --git a/Misc/python.man b/Misc/python.man index 74b2d72939eeb..225376574a26a 100644 --- a/Misc/python.man +++ b/Misc/python.man @@ -291,7 +291,7 @@ Set implementation specific option. The following options are available: nested imports). Note that its output may be broken in multi-threaded application. Typical usage is python3 -X importtime -c 'import asyncio' - -X dev: enable CPython?s ?development mode?, introducing additional runtime + -X dev: enable CPython's "development mode", introducing additional runtime checks which are too expensive to be enabled by default. It will not be more verbose than the default if the code is correct: new warnings are only emitted when an issue is detected. Effect of the developer mode: diff --git a/Python/initconfig.c b/Python/initconfig.c index 64286763b621e..38d64b63afcc9 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -83,7 +83,7 @@ static const char usage_3[] = "\ cumulative time (including nested imports) and self time (excluding\n\ nested imports). Note that its output may be broken in multi-threaded\n\ application. Typical usage is python3 -X importtime -c 'import asyncio'\n\ - -X dev: enable CPython?s ?development mode?, introducing additional runtime\n\ + -X dev: enable CPython's \"development mode\", introducing additional runtime\n\ checks which are too expensive to be enabled by default. Effect of the\n\ developer mode:\n\ * Add default warning filter, as -W default\n\ From webhook-mailer at python.org Tue Sep 15 07:16:24 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 11:16:24 -0000 Subject: [Python-checkins] bpo-41525: Make the Python program help ASCII-only (GH-21836) Message-ID: https://github.com/python/cpython/commit/9171dc2827650a1a3c20dee994145e335befa00e commit: 9171dc2827650a1a3c20dee994145e335befa00e branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-08T17:46:41-07:00 summary: bpo-41525: Make the Python program help ASCII-only (GH-21836) (cherry picked from commit 58de1dd6a8677bd213802c19204b827cb7134695) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst M Lib/test/test_cmd_line.py M Misc/python.man M Python/initconfig.c diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py index 5fc5bff6603e2..a5ece9bd611f4 100644 --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -44,7 +44,11 @@ def test_site_flag(self): def test_usage(self): rc, out, err = assert_python_ok('-h') - self.assertIn(b'usage', out) + lines = out.splitlines() + self.assertIn(b'usage', lines[0]) + # The first line contains the program name, + # but the rest should be ASCII-only + b''.join(lines[1:]).decode('ascii') def test_version(self): version = ('Python %d.%d' % sys.version_info[:2]).encode("ascii") diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst new file mode 100644 index 0000000000000..acc00f8b992c9 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst @@ -0,0 +1 @@ +The output of ``python --help`` contains now only ASCII characters. diff --git a/Misc/python.man b/Misc/python.man index 3645b0206eb2b..fa5d79996e2d7 100644 --- a/Misc/python.man +++ b/Misc/python.man @@ -295,7 +295,7 @@ Set implementation specific option. The following options are available: nested imports). Note that its output may be broken in multi-threaded application. Typical usage is python3 -X importtime -c 'import asyncio' - -X dev: enable CPython?s ?development mode?, introducing additional runtime + -X dev: enable CPython's "development mode", introducing additional runtime checks which are too expensive to be enabled by default. It will not be more verbose than the default if the code is correct: new warnings are only emitted when an issue is detected. Effect of the developer mode: diff --git a/Python/initconfig.c b/Python/initconfig.c index a930b5d38fa26..42f21b34a7599 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -80,7 +80,7 @@ static const char usage_3[] = "\ cumulative time (including nested imports) and self time (excluding\n\ nested imports). Note that its output may be broken in multi-threaded\n\ application. Typical usage is python3 -X importtime -c 'import asyncio'\n\ - -X dev: enable CPython?s ?development mode?, introducing additional runtime\n\ + -X dev: enable CPython's \"development mode\", introducing additional runtime\n\ checks which are too expensive to be enabled by default. Effect of the\n\ developer mode:\n\ * Add default warning filter, as -W default\n\ From webhook-mailer at python.org Tue Sep 15 07:16:53 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 11:16:53 -0000 Subject: [Python-checkins] bpo-41525: Make the Python program help ASCII-only (GH-21836) Message-ID: https://github.com/python/cpython/commit/11a82c7220a29aa1cabd6a1b042742b44c9c9b91 commit: 11a82c7220a29aa1cabd6a1b042742b44c9c9b91 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-08T17:47:53-07:00 summary: bpo-41525: Make the Python program help ASCII-only (GH-21836) (cherry picked from commit 58de1dd6a8677bd213802c19204b827cb7134695) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst M Lib/test/test_cmd_line.py M Misc/python.man M Python/initconfig.c diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py index 724402533038d..051c092eb6c39 100644 --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -45,7 +45,11 @@ def test_site_flag(self): def test_usage(self): rc, out, err = assert_python_ok('-h') - self.assertIn(b'usage', out) + lines = out.splitlines() + self.assertIn(b'usage', lines[0]) + # The first line contains the program name, + # but the rest should be ASCII-only + b''.join(lines[1:]).decode('ascii') def test_version(self): version = ('Python %d.%d' % sys.version_info[:2]).encode("ascii") diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst new file mode 100644 index 0000000000000..acc00f8b992c9 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst @@ -0,0 +1 @@ +The output of ``python --help`` contains now only ASCII characters. diff --git a/Misc/python.man b/Misc/python.man index 74b2d72939eeb..225376574a26a 100644 --- a/Misc/python.man +++ b/Misc/python.man @@ -291,7 +291,7 @@ Set implementation specific option. The following options are available: nested imports). Note that its output may be broken in multi-threaded application. Typical usage is python3 -X importtime -c 'import asyncio' - -X dev: enable CPython?s ?development mode?, introducing additional runtime + -X dev: enable CPython's "development mode", introducing additional runtime checks which are too expensive to be enabled by default. It will not be more verbose than the default if the code is correct: new warnings are only emitted when an issue is detected. Effect of the developer mode: diff --git a/Python/initconfig.c b/Python/initconfig.c index 885525b41848d..9349fdd82504c 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -84,7 +84,7 @@ static const char usage_3[] = "\ cumulative time (including nested imports) and self time (excluding\n\ nested imports). Note that its output may be broken in multi-threaded\n\ application. Typical usage is python3 -X importtime -c 'import asyncio'\n\ - -X dev: enable CPython?s ?development mode?, introducing additional runtime\n\ + -X dev: enable CPython's \"development mode\", introducing additional runtime\n\ checks which are too expensive to be enabled by default. Effect of the\n\ developer mode:\n\ * Add default warning filter, as -W default\n\ From webhook-mailer at python.org Tue Sep 15 07:17:26 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Tue, 15 Sep 2020 11:17:26 -0000 Subject: [Python-checkins] bpo-1635741: port scproxy to multi-phase init (GH-22164) Message-ID: https://github.com/python/cpython/commit/3ff6975e2c0af0399467f234b2e307cc76efcfa9 commit: 3ff6975e2c0af0399467f234b2e307cc76efcfa9 branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-09T12:28:48+09:00 summary: bpo-1635741: port scproxy to multi-phase init (GH-22164) files: A Misc/NEWS.d/next/Core and Builtins/2020-09-08-20-39-43.bpo-1635741.jiXmyT.rst M Modules/_scproxy.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-08-20-39-43.bpo-1635741.jiXmyT.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-08-20-39-43.bpo-1635741.jiXmyT.rst new file mode 100644 index 0000000000000..17752b2ccd3fa --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-08-20-39-43.bpo-1635741.jiXmyT.rst @@ -0,0 +1,2 @@ +Port the :mod:`_scproxy` extension module to multi-phase initialization +(:pep:`489`). diff --git a/Modules/_scproxy.c b/Modules/_scproxy.c index dbee3f7367ede..4c1f1aa300c71 100644 --- a/Modules/_scproxy.c +++ b/Modules/_scproxy.c @@ -231,21 +231,18 @@ static PyMethodDef mod_methods[] = { { 0, 0, 0, 0 } }; +static PyModuleDef_Slot _scproxy_slots[] = { + {0, NULL} +}; - -static struct PyModuleDef mod_module = { +static struct PyModuleDef _scproxy_module = { PyModuleDef_HEAD_INIT, - "_scproxy", - NULL, - -1, - mod_methods, - NULL, - NULL, - NULL, - NULL + .m_name = "_scproxy", + .m_size = 0, + .m_methods = mod_methods, + .m_slots = _scproxy_slots, }; - #ifdef __cplusplus extern "C" { #endif @@ -253,10 +250,9 @@ extern "C" { PyMODINIT_FUNC PyInit__scproxy(void) { - return PyModule_Create(&mod_module); + return PyModuleDef_Init(&_scproxy_module); } #ifdef __cplusplus } #endif - From webhook-mailer at python.org Tue Sep 15 09:39:49 2020 From: webhook-mailer at python.org (Neeraj Samtani) Date: Tue, 15 Sep 2020 13:39:49 -0000 Subject: [Python-checkins] bpo-41776: Revise example of "continue" in the tutorial documentation (GH-22234) Message-ID: https://github.com/python/cpython/commit/7bcc6456ad4704da9b287c8045768fa53961adc5 commit: 7bcc6456ad4704da9b287c8045768fa53961adc5 branch: master author: Neeraj Samtani committer: GitHub date: 2020-09-15T09:39:29-04:00 summary: bpo-41776: Revise example of "continue" in the tutorial documentation (GH-22234) Revise example of "continue" in the tutorial documentation files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index 5d24a19cfc079..b8aec2b04f13f 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -210,15 +210,15 @@ iteration of the loop:: ... if num % 2 == 0: ... print("Found an even number", num) ... continue - ... print("Found a number", num) + ... print("Found an odd number", num) Found an even number 2 - Found a number 3 + Found an odd number 3 Found an even number 4 - Found a number 5 + Found an odd number 5 Found an even number 6 - Found a number 7 + Found an odd number 7 Found an even number 8 - Found a number 9 + Found an odd number 9 .. _tut-pass: From webhook-mailer at python.org Tue Sep 15 09:40:56 2020 From: webhook-mailer at python.org (Hai Shi) Date: Tue, 15 Sep 2020 13:40:56 -0000 Subject: [Python-checkins] bpo-41726: Update the refcounts info of PyType_FromModuleAndSpec in refcounts.dat (GH-22112) Message-ID: https://github.com/python/cpython/commit/1e2f051a6127904cfee5f9e4021dd6e4a4d51c0f commit: 1e2f051a6127904cfee5f9e4021dd6e4a4d51c0f branch: master author: Hai Shi committer: GitHub date: 2020-09-09T11:48:44+02:00 summary: bpo-41726: Update the refcounts info of PyType_FromModuleAndSpec in refcounts.dat (GH-22112) Update refcounts info of PyType_FromModuleAndSpec in refcounts.dat files: A Misc/NEWS.d/next/Documentation/2020-09-08-16-57-09.bpo-41726.g0UXrn.rst M Doc/data/refcounts.dat diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat index 882d7d6d62fc3..355a4d6d3fa7b 100644 --- a/Doc/data/refcounts.dat +++ b/Doc/data/refcounts.dat @@ -2283,6 +2283,11 @@ PyType_CheckExact:PyObject*:o:0: PyType_FromSpec:PyObject*::+1: PyType_FromSpec:PyType_Spec*:spec:: +PyType_FromModuleAndSpec:PyObject*::+1: +PyType_FromModuleAndSpec:PyObject*:module:+1: +PyType_FromModuleAndSpec:PyType_Spec*:spec:: +PyType_FromModuleAndSpec:PyObject*:bases:0: + PyType_FromSpecWithBases:PyObject*::+1: PyType_FromSpecWithBases:PyType_Spec*:spec:: PyType_FromSpecWithBases:PyObject*:bases:0: diff --git a/Misc/NEWS.d/next/Documentation/2020-09-08-16-57-09.bpo-41726.g0UXrn.rst b/Misc/NEWS.d/next/Documentation/2020-09-08-16-57-09.bpo-41726.g0UXrn.rst new file mode 100644 index 0000000000000..1079a757c054a --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-09-08-16-57-09.bpo-41726.g0UXrn.rst @@ -0,0 +1 @@ +Update the refcounts info of ``PyType_FromModuleAndSpec``. From webhook-mailer at python.org Tue Sep 15 09:41:23 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 15 Sep 2020 13:41:23 -0000 Subject: [Python-checkins] Fix compiler warnings in init_dump_ascii_wstr() (GH-22150) Message-ID: https://github.com/python/cpython/commit/640e8e1d5f61d5868453d992da04bf4741327748 commit: 640e8e1d5f61d5868453d992da04bf4741327748 branch: master author: Victor Stinner committer: GitHub date: 2020-09-09T12:07:17+02:00 summary: Fix compiler warnings in init_dump_ascii_wstr() (GH-22150) Fix GCC 9.3 (using -O3) warnings on x86: initconfig.c: In function ?init_dump_ascii_wstr?: initconfig.c:2679:34: warning: format ?%lc? expects argument of type ?wint_t?, but argument 2 has type ?wchar_t? {aka ?long int?} 2679 | PySys_WriteStderr("%lc", ch); initconfig.c:2682:38: warning: format ?%x? expects argument of type ?unsigned int?, but argument 2 has type ?wchar_t? {aka ?long int?} 2682 | PySys_WriteStderr("\\x%02x", ch); initconfig.c:2686:38: warning: format ?%x? expects argument of type ?unsigned int?, but argument 2 has type ?wchar_t? {aka ?long int?} 2686 | PySys_WriteStderr("\\U%08x", ch); initconfig.c:2690:38: warning: format ?%x? expects argument of type ?unsigned int?, but argument 2 has type ?wchar_t? {aka ?long int?} 2690 | PySys_WriteStderr("\\u%04x", ch); files: M Python/initconfig.c diff --git a/Python/initconfig.c b/Python/initconfig.c index 38d64b63afcc9..880e145ec031c 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -2670,7 +2670,7 @@ init_dump_ascii_wstr(const wchar_t *str) PySys_WriteStderr("'"); for (; *str != L'\0'; str++) { - wchar_t ch = *str; + unsigned int ch = (unsigned int)*str; if (ch == L'\'') { PySys_WriteStderr("\\'"); } else if (0x20 <= ch && ch < 0x7f) { From webhook-mailer at python.org Tue Sep 15 09:41:56 2020 From: webhook-mailer at python.org (Vinay Sajip) Date: Tue, 15 Sep 2020 13:41:56 -0000 Subject: [Python-checkins] Add minor clarification in logging documentation. (GH-22167) Message-ID: https://github.com/python/cpython/commit/76553e5d2eae3e8a47406a6de4f354fe33ff370b commit: 76553e5d2eae3e8a47406a6de4f354fe33ff370b branch: master author: Vinay Sajip committer: GitHub date: 2020-09-09T11:21:22+01:00 summary: Add minor clarification in logging documentation. (GH-22167) files: M Doc/library/logging.rst diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst index 19691d50937a7..989016e649d65 100644 --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -575,9 +575,9 @@ The useful mapping keys in a :class:`LogRecord` are given in the section on pickled and sent across the wire, but you should be careful if you have more than one :class:`Formatter` subclass which customizes the formatting of exception information. In this case, you will have to clear the cached - value after a formatter has done its formatting, so that the next - formatter to handle the event doesn't use the cached value but - recalculates it afresh. + value (by setting the *exc_text* attribute to ``None``) after a formatter + has done its formatting, so that the next formatter to handle the event + doesn't use the cached value, but recalculates it afresh. If stack information is available, it's appended after the exception information, using :meth:`formatStack` to transform it if necessary. From webhook-mailer at python.org Tue Sep 15 09:42:17 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:42:17 -0000 Subject: [Python-checkins] Fix compiler warnings in init_dump_ascii_wstr() (GH-22150) Message-ID: https://github.com/python/cpython/commit/23d3681671bb538ef69c65875b68d59f6b756e61 commit: 23d3681671bb538ef69c65875b68d59f6b756e61 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T03:26:32-07:00 summary: Fix compiler warnings in init_dump_ascii_wstr() (GH-22150) Fix GCC 9.3 (using -O3) warnings on x86: initconfig.c: In function ?init_dump_ascii_wstr?: initconfig.c:2679:34: warning: format ?%lc? expects argument of type ?wint_t?, but argument 2 has type ?wchar_t? {aka ?long int?} 2679 | PySys_WriteStderr("%lc", ch); initconfig.c:2682:38: warning: format ?%x? expects argument of type ?unsigned int?, but argument 2 has type ?wchar_t? {aka ?long int?} 2682 | PySys_WriteStderr("\\x%02x", ch); initconfig.c:2686:38: warning: format ?%x? expects argument of type ?unsigned int?, but argument 2 has type ?wchar_t? {aka ?long int?} 2686 | PySys_WriteStderr("\\U%08x", ch); initconfig.c:2690:38: warning: format ?%x? expects argument of type ?unsigned int?, but argument 2 has type ?wchar_t? {aka ?long int?} 2690 | PySys_WriteStderr("\\u%04x", ch); (cherry picked from commit 640e8e1d5f61d5868453d992da04bf4741327748) Co-authored-by: Victor Stinner files: M Python/initconfig.c diff --git a/Python/initconfig.c b/Python/initconfig.c index 42f21b34a7599..b28e0a076468e 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -2589,7 +2589,7 @@ init_dump_ascii_wstr(const wchar_t *str) PySys_WriteStderr("'"); for (; *str != L'\0'; str++) { - wchar_t ch = *str; + unsigned int ch = (unsigned int)*str; if (ch == L'\'') { PySys_WriteStderr("\\'"); } else if (0x20 <= ch && ch < 0x7f) { From webhook-mailer at python.org Tue Sep 15 09:42:38 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:42:38 -0000 Subject: [Python-checkins] Fix compiler warnings in init_dump_ascii_wstr() (GH-22150) Message-ID: https://github.com/python/cpython/commit/b2376f9141625dfa3844ef8a568c91e0f0fecd24 commit: b2376f9141625dfa3844ef8a568c91e0f0fecd24 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T03:31:25-07:00 summary: Fix compiler warnings in init_dump_ascii_wstr() (GH-22150) Fix GCC 9.3 (using -O3) warnings on x86: initconfig.c: In function ?init_dump_ascii_wstr?: initconfig.c:2679:34: warning: format ?%lc? expects argument of type ?wint_t?, but argument 2 has type ?wchar_t? {aka ?long int?} 2679 | PySys_WriteStderr("%lc", ch); initconfig.c:2682:38: warning: format ?%x? expects argument of type ?unsigned int?, but argument 2 has type ?wchar_t? {aka ?long int?} 2682 | PySys_WriteStderr("\\x%02x", ch); initconfig.c:2686:38: warning: format ?%x? expects argument of type ?unsigned int?, but argument 2 has type ?wchar_t? {aka ?long int?} 2686 | PySys_WriteStderr("\\U%08x", ch); initconfig.c:2690:38: warning: format ?%x? expects argument of type ?unsigned int?, but argument 2 has type ?wchar_t? {aka ?long int?} 2690 | PySys_WriteStderr("\\u%04x", ch); (cherry picked from commit 640e8e1d5f61d5868453d992da04bf4741327748) Co-authored-by: Victor Stinner files: M Python/initconfig.c diff --git a/Python/initconfig.c b/Python/initconfig.c index 9349fdd82504c..9887079a86fd1 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -2672,7 +2672,7 @@ init_dump_ascii_wstr(const wchar_t *str) PySys_WriteStderr("'"); for (; *str != L'\0'; str++) { - wchar_t ch = *str; + unsigned int ch = (unsigned int)*str; if (ch == L'\'') { PySys_WriteStderr("\\'"); } else if (0x20 <= ch && ch < 0x7f) { From webhook-mailer at python.org Tue Sep 15 09:43:06 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:43:06 -0000 Subject: [Python-checkins] Fixes dead links to Django's logging config docs (GH-20823) (GH-22171) Message-ID: https://github.com/python/cpython/commit/778a9b2c5520f1612e4b537c4b34f08c2a3e8486 commit: 778a9b2c5520f1612e4b537c4b34f08c2a3e8486 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T14:40:41+01:00 summary: Fixes dead links to Django's logging config docs (GH-20823) (GH-22171) (cherry picked from commit 714217f9561507bbc7218a02089d0e1da0239372) files: M Doc/howto/logging-cookbook.rst diff --git a/Doc/howto/logging-cookbook.rst b/Doc/howto/logging-cookbook.rst index 17f4ff6e474c2..de0f834551f5d 100644 --- a/Doc/howto/logging-cookbook.rst +++ b/Doc/howto/logging-cookbook.rst @@ -1368,7 +1368,7 @@ An example dictionary-based configuration ----------------------------------------- Below is an example of a logging configuration dictionary - it's taken from -the `documentation on the Django project `_. +the `documentation on the Django project `_. This dictionary is passed to :func:`~config.dictConfig` to put the configuration into effect:: LOGGING = { @@ -1424,7 +1424,7 @@ This dictionary is passed to :func:`~config.dictConfig` to put the configuration } For more information about this configuration, you can see the `relevant -section `_ +section `_ of the Django documentation. .. _cookbook-rotator-namer: From webhook-mailer at python.org Tue Sep 15 09:43:31 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:43:31 -0000 Subject: [Python-checkins] Fixes dead links to Django's logging config docs (GH-20823) (GH-22170) Message-ID: https://github.com/python/cpython/commit/014b4ef0a175eb26759a3daa363cb230267d6775 commit: 014b4ef0a175eb26759a3daa363cb230267d6775 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T14:45:08+01:00 summary: Fixes dead links to Django's logging config docs (GH-20823) (GH-22170) (cherry picked from commit 714217f9561507bbc7218a02089d0e1da0239372) files: M Doc/howto/logging-cookbook.rst diff --git a/Doc/howto/logging-cookbook.rst b/Doc/howto/logging-cookbook.rst index 17f4ff6e474c2..de0f834551f5d 100644 --- a/Doc/howto/logging-cookbook.rst +++ b/Doc/howto/logging-cookbook.rst @@ -1368,7 +1368,7 @@ An example dictionary-based configuration ----------------------------------------- Below is an example of a logging configuration dictionary - it's taken from -the `documentation on the Django project `_. +the `documentation on the Django project `_. This dictionary is passed to :func:`~config.dictConfig` to put the configuration into effect:: LOGGING = { @@ -1424,7 +1424,7 @@ This dictionary is passed to :func:`~config.dictConfig` to put the configuration } For more information about this configuration, you can see the `relevant -section `_ +section `_ of the Django documentation. .. _cookbook-rotator-namer: From webhook-mailer at python.org Tue Sep 15 09:43:52 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:43:52 -0000 Subject: [Python-checkins] Fix incorrect bpo number in change notes. (GH-22151) (#22174) Message-ID: https://github.com/python/cpython/commit/5dd8b3d27a2c28c175ea42ecb5ea16ca1412084e commit: 5dd8b3d27a2c28c175ea42ecb5ea16ca1412084e branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T17:33:21+01:00 summary: Fix incorrect bpo number in change notes. (GH-22151) (#22174) files: M Misc/NEWS.d/3.9.0a2.rst diff --git a/Misc/NEWS.d/3.9.0a2.rst b/Misc/NEWS.d/3.9.0a2.rst index 50478c08e9018..1fd23b763e2a1 100644 --- a/Misc/NEWS.d/3.9.0a2.rst +++ b/Misc/NEWS.d/3.9.0a2.rst @@ -229,7 +229,7 @@ coroutine of an asynchronous generator. .. -.. bpo: 32949 +.. bpo: 33387 .. date: 2018-03-13-14-46-03 .. nonce: v821M7 .. section: Core and Builtins From webhook-mailer at python.org Tue Sep 15 09:44:11 2020 From: webhook-mailer at python.org (Ammar Askar) Date: Tue, 15 Sep 2020 13:44:11 -0000 Subject: [Python-checkins] =?utf-8?q?=5B3=2E9=5D_bpo-41688=3A_Document_bu?= =?utf-8?q?g_in_**=3D_dispatching_in_the_language_data=E2=80=A6_=28GH-2217?= =?utf-8?q?2=29?= Message-ID: https://github.com/python/cpython/commit/46bc21e1780016aaacd34e472f838dc792fb674c commit: 46bc21e1780016aaacd34e472f838dc792fb674c branch: 3.9 author: Ammar Askar committer: GitHub date: 2020-09-09T11:01:38-07:00 summary: [3.9] bpo-41688: Document bug in **= dispatching in the language data? (GH-22172) I tossed in a "This bug is fixed in Python 3.10" sentence but I'm not sure if that's appropriate to have here. files: M Doc/reference/datamodel.rst diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index a817408c3b1ef..fc304a191a314 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2409,6 +2409,13 @@ left undefined. :ref:`faq-augmented-assignment-tuple-error`), but this behavior is in fact part of the data model. + .. note:: + + Due to a bug in the dispatching mechanism for ``**=``, a class that + defines :meth:`__ipow__` but returns ``NotImplemented`` would fail to + fall back to ``x.__pow__(y)`` and ``y.__rpow__(x)``. This bug is fixed + in Python 3.10. + .. method:: object.__neg__(self) object.__pos__(self) From webhook-mailer at python.org Tue Sep 15 09:44:32 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:44:32 -0000 Subject: [Python-checkins] =?utf-8?b?WzMuOF0gWzMuOV0gYnBvLTQxNjg4OiBEb2N1?= =?utf-8?q?ment_bug_in_**=3D_dispatching_in_the_language_data=E2=80=A6_=28?= =?utf-8?q?GH-22172=29_=28GH-22175=29?= Message-ID: https://github.com/python/cpython/commit/4c4c354de2c9a290afbd4e5fcba07e3696ca0396 commit: 4c4c354de2c9a290afbd4e5fcba07e3696ca0396 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T11:15:49-07:00 summary: [3.8] [3.9] bpo-41688: Document bug in **= dispatching in the language data? (GH-22172) (GH-22175) (cherry picked from commit 46bc21e1780016aaacd34e472f838dc792fb674c) Co-authored-by: Ammar Askar Automerge-Triggered-By: @brettcannon files: M Doc/reference/datamodel.rst diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index c1262cdb9c106..3f05385862416 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2404,6 +2404,13 @@ left undefined. :ref:`faq-augmented-assignment-tuple-error`), but this behavior is in fact part of the data model. + .. note:: + + Due to a bug in the dispatching mechanism for ``**=``, a class that + defines :meth:`__ipow__` but returns ``NotImplemented`` would fail to + fall back to ``x.__pow__(y)`` and ``y.__rpow__(x)``. This bug is fixed + in Python 3.10. + .. method:: object.__neg__(self) object.__pos__(self) From webhook-mailer at python.org Tue Sep 15 09:44:53 2020 From: webhook-mailer at python.org (Jakub =?utf-8?q?Kul=C3=ADk?=) Date: Tue, 15 Sep 2020 13:44:53 -0000 Subject: [Python-checkins] bpo-41687: Fix error handling in Solaris sendfile implementation (GH-22128) Message-ID: https://github.com/python/cpython/commit/fa8c9e70104b0aef966a518eb3a80a4881906ae0 commit: fa8c9e70104b0aef966a518eb3a80a4881906ae0 branch: master author: Jakub Kul?k committer: GitHub date: 2020-09-09T12:29:42-07:00 summary: bpo-41687: Fix error handling in Solaris sendfile implementation (GH-22128) I just realized that my recent PR with sendfile on Solaris ([PR 22040](https://github.com/python/cpython/pull/22040)) has broken error handling. Sorry for that, this simple followup fixes that. Automerge-Triggered-By: @1st1 files: M Modules/posixmodule.c diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 00ba7580302bb..7c496938ed4c5 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -9521,14 +9521,13 @@ os_sendfile_impl(PyObject *module, int out_fd, int in_fd, PyObject *offobj, #if defined(__sun) && defined(__SVR4) // On Solaris, sendfile raises EINVAL rather than returning 0 // when the offset is equal or bigger than the in_fd size. - int res; struct stat st; do { Py_BEGIN_ALLOW_THREADS - res = fstat(in_fd, &st); + ret = fstat(in_fd, &st); Py_END_ALLOW_THREADS - } while (res != 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); + } while (ret != 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); if (ret < 0) return (!async_err) ? posix_error() : NULL; From webhook-mailer at python.org Tue Sep 15 09:45:18 2020 From: webhook-mailer at python.org (Maggie Moss) Date: Tue, 15 Sep 2020 13:45:18 -0000 Subject: [Python-checkins] bpo-41428: Implementation for PEP 604 (GH-21515) Message-ID: https://github.com/python/cpython/commit/1b4552c5e8e925f24c15f707050f22c977171125 commit: 1b4552c5e8e925f24c15f707050f22c977171125 branch: master author: Maggie Moss committer: GitHub date: 2020-09-09T21:23:24+01:00 summary: bpo-41428: Implementation for PEP 604 (GH-21515) See https://www.python.org/dev/peps/pep-0604/ for more information. Co-authored-by: Pablo Galindo files: A Include/internal/pycore_unionobject.h A Misc/NEWS.d/next/Core and Builtins/2020-07-28-22-43-27.bpo-41428.FM6xsI.rst A Objects/unionobject.c M Lib/test/test_isinstance.py M Lib/test/test_types.py M Lib/test/test_typing.py M Lib/types.py M Lib/typing.py M Makefile.pre.in M Objects/abstract.c M Objects/typeobject.c M PCbuild/pythoncore.vcxproj M PCbuild/pythoncore.vcxproj.filters diff --git a/Include/internal/pycore_unionobject.h b/Include/internal/pycore_unionobject.h new file mode 100644 index 0000000000000..fa8ba6ed944c1 --- /dev/null +++ b/Include/internal/pycore_unionobject.h @@ -0,0 +1,17 @@ +#ifndef Py_INTERNAL_UNIONOBJECT_H +#define Py_INTERNAL_UNIONOBJECT_H +#ifdef __cplusplus +extern "C" { +#endif + +#ifndef Py_BUILD_CORE +# error "this header requires Py_BUILD_CORE define" +#endif + +PyAPI_FUNC(PyObject *) _Py_Union(PyObject *args); +PyAPI_DATA(PyTypeObject) _Py_UnionType; + +#ifdef __cplusplus +} +#endif +#endif /* !Py_INTERNAL_UNIONOBJECT_H */ diff --git a/Lib/test/test_isinstance.py b/Lib/test/test_isinstance.py index 53639e984e48a..91e79c295481d 100644 --- a/Lib/test/test_isinstance.py +++ b/Lib/test/test_isinstance.py @@ -4,6 +4,7 @@ import unittest import sys +import typing @@ -208,6 +209,25 @@ def test_isinstance_abstract(self): self.assertEqual(False, isinstance(AbstractChild(), Super)) self.assertEqual(False, isinstance(AbstractChild(), Child)) + def test_isinstance_with_or_union(self): + self.assertTrue(isinstance(Super(), Super | int)) + self.assertFalse(isinstance(None, str | int)) + self.assertTrue(isinstance(3, str | int)) + self.assertTrue(isinstance("", str | int)) + self.assertTrue(isinstance([], typing.List | typing.Tuple)) + self.assertTrue(isinstance(2, typing.List | int)) + self.assertFalse(isinstance(2, typing.List | typing.Tuple)) + self.assertTrue(isinstance(None, int | None)) + self.assertFalse(isinstance(3.14, int | str)) + with self.assertRaises(TypeError): + isinstance(2, list[int]) + with self.assertRaises(TypeError): + isinstance(2, list[int] | int) + with self.assertRaises(TypeError): + isinstance(2, int | str | list[int] | float) + + + def test_subclass_normal(self): # normal classes self.assertEqual(True, issubclass(Super, Super)) @@ -217,6 +237,8 @@ def test_subclass_normal(self): self.assertEqual(True, issubclass(Child, Child)) self.assertEqual(True, issubclass(Child, Super)) self.assertEqual(False, issubclass(Child, AbstractSuper)) + self.assertTrue(issubclass(typing.List, typing.List|typing.Tuple)) + self.assertFalse(issubclass(int, typing.List|typing.Tuple)) def test_subclass_abstract(self): # abstract classes @@ -251,6 +273,16 @@ def test_isinstance_recursion_limit(self): # blown self.assertRaises(RecursionError, blowstack, isinstance, '', str) + def test_subclass_with_union(self): + self.assertTrue(issubclass(int, int | float | int)) + self.assertTrue(issubclass(str, str | Child | str)) + self.assertFalse(issubclass(dict, float|str)) + self.assertFalse(issubclass(object, float|str)) + with self.assertRaises(TypeError): + issubclass(2, Child | Super) + with self.assertRaises(TypeError): + issubclass(int, list[int] | Child) + def test_issubclass_refcount_handling(self): # bpo-39382: abstract_issubclass() didn't hold item reference while # peeking in the bases tuple, in the single inheritance case. diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py index 49dc5bf40e3ed..f499fb9c8c51a 100644 --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -2,6 +2,7 @@ from test.support import run_with_locale import collections.abc +from collections import namedtuple import inspect import pickle import locale @@ -9,6 +10,12 @@ import types import unittest.mock import weakref +import typing + +class Example: + pass + +class Forward: ... class TypesTests(unittest.TestCase): @@ -598,6 +605,113 @@ def test_method_descriptor_types(self): self.assertIsInstance(int.from_bytes, types.BuiltinMethodType) self.assertIsInstance(int.__new__, types.BuiltinMethodType) + def test_or_types_operator(self): + self.assertEqual(int | str, typing.Union[int, str]) + self.assertNotEqual(int | list, typing.Union[int, str]) + self.assertEqual(str | int, typing.Union[int, str]) + self.assertEqual(int | None, typing.Union[int, None]) + self.assertEqual(None | int, typing.Union[int, None]) + self.assertEqual(int | str | list, typing.Union[int, str, list]) + self.assertEqual(int | (str | list), typing.Union[int, str, list]) + self.assertEqual(str | (int | list), typing.Union[int, str, list]) + self.assertEqual(typing.List | typing.Tuple, typing.Union[typing.List, typing.Tuple]) + self.assertEqual(typing.List[int] | typing.Tuple[int], typing.Union[typing.List[int], typing.Tuple[int]]) + self.assertEqual(typing.List[int] | None, typing.Union[typing.List[int], None]) + self.assertEqual(None | typing.List[int], typing.Union[None, typing.List[int]]) + self.assertEqual(str | float | int | complex | int, (int | str) | (float | complex)) + self.assertEqual(typing.Union[str, int, typing.List[int]], str | int | typing.List[int]) + self.assertEqual(int | int, int) + self.assertEqual( + BaseException | + bool | + bytes | + complex | + float | + int | + list | + map | + set, + typing.Union[ + BaseException, + bool, + bytes, + complex, + float, + int, + list, + map, + set, + ]) + with self.assertRaises(TypeError): + int | 3 + with self.assertRaises(TypeError): + 3 | int + with self.assertRaises(TypeError): + Example() | int + with self.assertRaises(TypeError): + (int | str) < typing.Union[str, int] + with self.assertRaises(TypeError): + (int | str) < (int | bool) + with self.assertRaises(TypeError): + (int | str) <= (int | str) + with self.assertRaises(TypeError): + # Check that we don't crash if typing.Union does not have a tuple in __args__ + x = typing.Union[str, int] + x.__args__ = [str, int] + (int | str ) == x + + def test_or_type_operator_with_TypeVar(self): + TV = typing.TypeVar('T') + assert TV | str == typing.Union[TV, str] + assert str | TV == typing.Union[str, TV] + + def test_or_type_operator_with_forward(self): + T = typing.TypeVar('T') + ForwardAfter = T | 'Forward' + ForwardBefore = 'Forward' | T + def forward_after(x: ForwardAfter[int]) -> None: ... + def forward_before(x: ForwardBefore[int]) -> None: ... + assert typing.get_args(typing.get_type_hints(forward_after)['x']) == (int, Forward) + assert typing.get_args(typing.get_type_hints(forward_before)['x']) == (int, Forward) + + def test_or_type_operator_with_Protocol(self): + class Proto(typing.Protocol): + def meth(self) -> int: + ... + assert Proto | str == typing.Union[Proto, str] + + def test_or_type_operator_with_Alias(self): + assert list | str == typing.Union[list, str] + assert typing.List | str == typing.Union[typing.List, str] + + def test_or_type_operator_with_NamedTuple(self): + NT=namedtuple('A', ['B', 'C', 'D']) + assert NT | str == typing.Union[NT,str] + + def test_or_type_operator_with_TypedDict(self): + class Point2D(typing.TypedDict): + x: int + y: int + label: str + assert Point2D | str == typing.Union[Point2D, str] + + def test_or_type_operator_with_NewType(self): + UserId = typing.NewType('UserId', int) + assert UserId | str == typing.Union[UserId, str] + + def test_or_type_operator_with_IO(self): + assert typing.IO | str == typing.Union[typing.IO, str] + + def test_or_type_operator_with_SpecialForm(self): + assert typing.Any | str == typing.Union[typing.Any, str] + assert typing.NoReturn | str == typing.Union[typing.NoReturn, str] + assert typing.Optional[int] | str == typing.Union[typing.Optional[int], str] + assert typing.Optional[int] | str == typing.Union[int, str, None] + assert typing.Union[int, bool] | str == typing.Union[int, bool, str] + + def test_or_type_repr(self): + assert repr(int | None) == "int | None" + assert repr(int | typing.GenericAlias(list, int)) == "int | list[int]" class MappingProxyTests(unittest.TestCase): mappingproxy = types.MappingProxyType diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index b3be99141afca..05140fc61b9bf 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -244,8 +244,6 @@ def test_subclass_error(self): issubclass(int, Union) with self.assertRaises(TypeError): issubclass(Union, int) - with self.assertRaises(TypeError): - issubclass(int, Union[int, str]) with self.assertRaises(TypeError): issubclass(Union[int, str], int) @@ -347,10 +345,6 @@ def test_empty(self): with self.assertRaises(TypeError): Union[()] - def test_union_instance_type_error(self): - with self.assertRaises(TypeError): - isinstance(42, Union[int, str]) - def test_no_eval_union(self): u = Union[int, str] def f(x: u): ... diff --git a/Lib/types.py b/Lib/types.py index ad2020ec69b63..9642e7212caac 100644 --- a/Lib/types.py +++ b/Lib/types.py @@ -294,6 +294,7 @@ def wrapped(*args, **kwargs): GenericAlias = type(list[int]) +Union = type(int | str) __all__ = [n for n in globals() if n[:1] != '_'] diff --git a/Lib/typing.py b/Lib/typing.py index 2899a0213d434..2aedbeb852a71 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -117,7 +117,6 @@ # namespace, but excluded from __all__ because they might stomp on # legitimate imports of those modules. - def _type_check(arg, msg, is_argument=True): """Check that the argument is a type, and return it (internal helper). @@ -145,7 +144,7 @@ def _type_check(arg, msg, is_argument=True): return arg if isinstance(arg, _SpecialForm) or arg in (Generic, Protocol): raise TypeError(f"Plain {arg} is not valid as type argument") - if isinstance(arg, (type, TypeVar, ForwardRef)): + if isinstance(arg, (type, TypeVar, ForwardRef, types.Union)): return arg if not callable(arg): raise TypeError(f"{msg} Got {arg!r:.100}.") @@ -205,7 +204,7 @@ def _remove_dups_flatten(parameters): # Flatten out Union[Union[...], ...]. params = [] for p in parameters: - if isinstance(p, _UnionGenericAlias): + if isinstance(p, (_UnionGenericAlias, types.Union)): params.extend(p.__args__) elif isinstance(p, tuple) and len(p) > 0 and p[0] is Union: params.extend(p[1:]) @@ -586,6 +585,12 @@ def __init__(self, name, *constraints, bound=None, if def_mod != 'typing': self.__module__ = def_mod + def __or__(self, right): + return Union[self, right] + + def __ror__(self, right): + return Union[self, right] + def __repr__(self): if self.__covariant__: prefix = '+' @@ -693,6 +698,12 @@ def __eq__(self, other): def __hash__(self): return hash((self.__origin__, self.__args__)) + def __or__(self, right): + return Union[self, right] + + def __ror__(self, right): + return Union[self, right] + @_tp_cache def __getitem__(self, params): if self.__origin__ in (Generic, Protocol): @@ -792,6 +803,11 @@ def __subclasscheck__(self, cls): def __reduce__(self): return self._name + def __or__(self, right): + return Union[self, right] + + def __ror__(self, right): + return Union[self, right] class _CallableGenericAlias(_GenericAlias, _root=True): def __repr__(self): @@ -878,6 +894,15 @@ def __repr__(self): return f'typing.Optional[{_type_repr(args[0])}]' return super().__repr__() + def __instancecheck__(self, obj): + return self.__subclasscheck__(type(obj)) + + def __subclasscheck__(self, cls): + for arg in self.__args__: + if issubclass(cls, arg): + return True + + class Generic: """Abstract base class for generic types. diff --git a/Makefile.pre.in b/Makefile.pre.in index 5d3ac705a3625..921bd08ea505d 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -432,6 +432,7 @@ OBJECT_OBJS= \ Objects/typeobject.o \ Objects/unicodeobject.o \ Objects/unicodectype.o \ + Objects/unionobject.o \ Objects/weakrefobject.o ########################################################################## @@ -1128,6 +1129,7 @@ PYTHON_HEADERS= \ $(srcdir)/Include/internal/pycore_sysmodule.h \ $(srcdir)/Include/internal/pycore_traceback.h \ $(srcdir)/Include/internal/pycore_tuple.h \ + $(srcdir)/Include/internal/pycore_unionobject.h \ $(srcdir)/Include/internal/pycore_warnings.h \ $(DTRACE_HEADERS) diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-07-28-22-43-27.bpo-41428.FM6xsI.rst b/Misc/NEWS.d/next/Core and Builtins/2020-07-28-22-43-27.bpo-41428.FM6xsI.rst new file mode 100644 index 0000000000000..a6652de927511 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-07-28-22-43-27.bpo-41428.FM6xsI.rst @@ -0,0 +1 @@ +Implement PEP 604. This supports (int | str) etc. in place of Union[str, int]. \ No newline at end of file diff --git a/Objects/abstract.c b/Objects/abstract.c index 7bd72c9b5dcc2..c471f184f6c84 100644 --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -1,6 +1,7 @@ /* Abstract Object Interface (many thanks to Jim Fulton) */ #include "Python.h" +#include "pycore_unionobject.h" // _Py_UnionType && _Py_Union() #include "pycore_abstract.h" // _PyIndex_Check() #include "pycore_ceval.h" // _Py_EnterRecursiveCall() #include "pycore_pyerrors.h" // _PyErr_Occurred() @@ -839,7 +840,6 @@ binary_op(PyObject *v, PyObject *w, const int op_slot, const char *op_name) Py_TYPE(w)->tp_name); return NULL; } - return binop_type_error(v, w, op_name); } return result; @@ -2412,7 +2412,6 @@ object_isinstance(PyObject *inst, PyObject *cls) PyObject *icls; int retval; _Py_IDENTIFIER(__class__); - if (PyType_Check(cls)) { retval = PyObject_TypeCheck(inst, (PyTypeObject *)cls); if (retval == 0) { @@ -2432,7 +2431,7 @@ object_isinstance(PyObject *inst, PyObject *cls) } else { if (!check_class(cls, - "isinstance() arg 2 must be a type or tuple of types")) + "isinstance() arg 2 must be a type, a tuple of types or a union")) return -1; retval = _PyObject_LookupAttrId(inst, &PyId___class__, &icls); if (icls != NULL) { @@ -2525,10 +2524,14 @@ recursive_issubclass(PyObject *derived, PyObject *cls) if (!check_class(derived, "issubclass() arg 1 must be a class")) return -1; - if (!check_class(cls, - "issubclass() arg 2 must be a class" - " or tuple of classes")) + + PyTypeObject *type = Py_TYPE(cls); + int is_union = (PyType_Check(type) && type == &_Py_UnionType); + if (!is_union && !check_class(cls, + "issubclass() arg 2 must be a class," + " a tuple of classes, or a union.")) { return -1; + } return abstract_issubclass(derived, cls); } diff --git a/Objects/typeobject.c b/Objects/typeobject.c index 74040757a0702..3bb2c338fe0b5 100644 --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -6,6 +6,7 @@ #include "pycore_object.h" #include "pycore_pyerrors.h" #include "pycore_pystate.h" // _PyThreadState_GET() +#include "pycore_unionobject.h" // _Py_Union() #include "frameobject.h" #include "structmember.h" // PyMemberDef @@ -3753,6 +3754,21 @@ type_is_gc(PyTypeObject *type) return type->tp_flags & Py_TPFLAGS_HEAPTYPE; } +static PyObject * +type_or(PyTypeObject* self, PyObject* param) { + PyObject *tuple = PyTuple_Pack(2, self, param); + if (tuple == NULL) { + return NULL; + } + PyObject *new_union = _Py_Union(tuple); + Py_DECREF(tuple); + return new_union; +} + +static PyNumberMethods type_as_number = { + .nb_or = (binaryfunc)type_or, // Add __or__ function +}; + PyTypeObject PyType_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "type", /* tp_name */ @@ -3764,7 +3780,7 @@ PyTypeObject PyType_Type = { 0, /* tp_setattr */ 0, /* tp_as_async */ (reprfunc)type_repr, /* tp_repr */ - 0, /* tp_as_number */ + &type_as_number, /* tp_as_number */ 0, /* tp_as_sequence */ 0, /* tp_as_mapping */ 0, /* tp_hash */ @@ -5598,7 +5614,6 @@ PyType_Ready(PyTypeObject *type) add_subclass((PyTypeObject *)b, type) < 0) goto error; } - /* All done -- set the ready flag */ type->tp_flags = (type->tp_flags & ~Py_TPFLAGS_READYING) | Py_TPFLAGS_READY; diff --git a/Objects/unionobject.c b/Objects/unionobject.c new file mode 100644 index 0000000000000..0ef7abb4c55c2 --- /dev/null +++ b/Objects/unionobject.c @@ -0,0 +1,464 @@ +// types.Union -- used to represent e.g. Union[int, str], int | str +#include "Python.h" +#include "pycore_unionobject.h" +#include "structmember.h" + + +typedef struct { + PyObject_HEAD + PyObject *args; +} unionobject; + +static void +unionobject_dealloc(PyObject *self) +{ + unionobject *alias = (unionobject *)self; + + Py_XDECREF(alias->args); + self->ob_type->tp_free(self); +} + +static Py_hash_t +union_hash(PyObject *self) +{ + unionobject *alias = (unionobject *)self; + Py_hash_t h1 = PyObject_Hash(alias->args); + if (h1 == -1) { + return -1; + } + return h1; +} + +static int +is_generic_alias_in_args(PyObject *args) { + Py_ssize_t nargs = PyTuple_GET_SIZE(args); + for (Py_ssize_t iarg = 0; iarg < nargs; iarg++) { + PyObject *arg = PyTuple_GET_ITEM(args, iarg); + if (Py_TYPE(arg) == &Py_GenericAliasType) { + return 0; + } + } + return 1; +} + +static PyObject * +union_instancecheck(PyObject *self, PyObject *instance) +{ + unionobject *alias = (unionobject *) self; + Py_ssize_t nargs = PyTuple_GET_SIZE(alias->args); + if (!is_generic_alias_in_args(alias->args)) { + PyErr_SetString(PyExc_TypeError, + "isinstance() argument 2 cannot contain a parameterized generic"); + return NULL; + } + for (Py_ssize_t iarg = 0; iarg < nargs; iarg++) { + PyObject *arg = PyTuple_GET_ITEM(alias->args, iarg); + if (arg == Py_None) { + arg = (PyObject *)&_PyNone_Type; + } + if (PyType_Check(arg) && PyObject_IsInstance(instance, arg) != 0) { + Py_RETURN_TRUE; + } + } + Py_RETURN_FALSE; +} + +static PyObject * +union_subclasscheck(PyObject *self, PyObject *instance) +{ + if (!PyType_Check(instance)) { + PyErr_SetString(PyExc_TypeError, "issubclass() arg 1 must be a class"); + return NULL; + } + unionobject *alias = (unionobject *)self; + if (!is_generic_alias_in_args(alias->args)) { + PyErr_SetString(PyExc_TypeError, + "issubclass() argument 2 cannot contain a parameterized generic"); + return NULL; + } + Py_ssize_t nargs = PyTuple_GET_SIZE(alias->args); + for (Py_ssize_t iarg = 0; iarg < nargs; iarg++) { + PyObject *arg = PyTuple_GET_ITEM(alias->args, iarg); + if (PyType_Check(arg) && (PyType_IsSubtype((PyTypeObject *)instance, (PyTypeObject *)arg) != 0)) { + Py_RETURN_TRUE; + } + } + Py_RETURN_FALSE; +} + +static int +is_typing_module(PyObject *obj) { + PyObject *module = PyObject_GetAttrString(obj, "__module__"); + if (module == NULL) { + return -1; + } + int is_typing = PyUnicode_Check(module) && _PyUnicode_EqualToASCIIString(module, "typing"); + Py_DECREF(module); + return is_typing; +} + +static int +is_typing_name(PyObject *obj, char *name) +{ + PyTypeObject *type = Py_TYPE(obj); + if (strcmp(type->tp_name, name) != 0) { + return 0; + } + return is_typing_module(obj); +} + +static PyObject * +union_richcompare(PyObject *a, PyObject *b, int op) +{ + PyObject *result = NULL; + if (op != Py_EQ && op != Py_NE) { + result = Py_NotImplemented; + Py_INCREF(result); + return result; + } + + PyTypeObject *type = Py_TYPE(b); + + PyObject* a_set = PySet_New(((unionobject*)a)->args); + if (a_set == NULL) { + return NULL; + } + PyObject* b_set = PySet_New(NULL); + if (b_set == NULL) { + goto exit; + } + + // Populate b_set with the data from the right object + int is_typing_union = is_typing_name(b, "_UnionGenericAlias"); + if (is_typing_union < 0) { + goto exit; + } + if (is_typing_union) { + PyObject *b_args = PyObject_GetAttrString(b, "__args__"); + if (b_args == NULL) { + goto exit; + } + if (!PyTuple_CheckExact(b_args)) { + Py_DECREF(b_args); + PyErr_SetString(PyExc_TypeError, "__args__ argument of typing.Union object is not a tuple"); + goto exit; + } + Py_ssize_t b_arg_length = PyTuple_GET_SIZE(b_args); + for (Py_ssize_t i = 0; i < b_arg_length; i++) { + PyObject* arg = PyTuple_GET_ITEM(b_args, i); + if (arg == (PyObject *)&_PyNone_Type) { + arg = Py_None; + } + if (PySet_Add(b_set, arg) == -1) { + Py_DECREF(b_args); + goto exit; + } + } + Py_DECREF(b_args); + } else if (type == &_Py_UnionType) { + PyObject* args = ((unionobject*) b)->args; + Py_ssize_t arg_length = PyTuple_GET_SIZE(args); + for (Py_ssize_t i = 0; i < arg_length; i++) { + PyObject* arg = PyTuple_GET_ITEM(args, i); + if (PySet_Add(b_set, arg) == -1) { + goto exit; + } + } + } else { + if (PySet_Add(b_set, b) == -1) { + goto exit; + } + } + result = PyObject_RichCompare(a_set, b_set, op); +exit: + Py_XDECREF(a_set); + Py_XDECREF(b_set); + return result; +} + +static PyObject* +flatten_args(PyObject* args) +{ + int arg_length = PyTuple_GET_SIZE(args); + int total_args = 0; + // Get number of total args once it's flattened. + for (Py_ssize_t i = 0; i < arg_length; i++) { + PyObject *arg = PyTuple_GET_ITEM(args, i); + PyTypeObject* arg_type = Py_TYPE(arg); + if (arg_type == &_Py_UnionType) { + total_args += PyTuple_GET_SIZE(((unionobject*) arg)->args); + } else { + total_args++; + } + } + // Create new tuple of flattened args. + PyObject *flattened_args = PyTuple_New(total_args); + if (flattened_args == NULL) { + return NULL; + } + Py_ssize_t pos = 0; + for (Py_ssize_t i = 0; i < arg_length; i++) { + PyObject *arg = PyTuple_GET_ITEM(args, i); + PyTypeObject* arg_type = Py_TYPE(arg); + if (arg_type == &_Py_UnionType) { + PyObject* nested_args = ((unionobject*)arg)->args; + int nested_arg_length = PyTuple_GET_SIZE(nested_args); + for (int j = 0; j < nested_arg_length; j++) { + PyObject* nested_arg = PyTuple_GET_ITEM(nested_args, j); + Py_INCREF(nested_arg); + PyTuple_SET_ITEM(flattened_args, pos, nested_arg); + pos++; + } + } else { + Py_INCREF(arg); + PyTuple_SET_ITEM(flattened_args, pos, arg); + pos++; + } + } + return flattened_args; +} + +static PyObject* +dedup_and_flatten_args(PyObject* args) +{ + args = flatten_args(args); + if (args == NULL) { + return NULL; + } + Py_ssize_t arg_length = PyTuple_GET_SIZE(args); + PyObject *new_args = PyTuple_New(arg_length); + if (new_args == NULL) { + return NULL; + } + // Add unique elements to an array. + int added_items = 0; + for (Py_ssize_t i = 0; i < arg_length; i++) { + int is_duplicate = 0; + PyObject* i_element = PyTuple_GET_ITEM(args, i); + for (Py_ssize_t j = i + 1; j < arg_length; j++) { + PyObject* j_element = PyTuple_GET_ITEM(args, j); + if (i_element == j_element) { + is_duplicate = 1; + } + } + if (!is_duplicate) { + Py_INCREF(i_element); + PyTuple_SET_ITEM(new_args, added_items, i_element); + added_items++; + } + } + Py_DECREF(args); + _PyTuple_Resize(&new_args, added_items); + return new_args; +} + +static int +is_typevar(PyObject *obj) +{ + return is_typing_name(obj, "TypeVar"); +} + +static int +is_special_form(PyObject *obj) +{ + return is_typing_name(obj, "_SpecialForm"); +} + +static int +is_new_type(PyObject *obj) +{ + PyTypeObject *type = Py_TYPE(obj); + if (type != &PyFunction_Type) { + return 0; + } + return is_typing_module(obj); +} + +static int +is_unionable(PyObject *obj) +{ + if (obj == Py_None) { + return 1; + } + PyTypeObject *type = Py_TYPE(obj); + return ( + is_typevar(obj) || + is_new_type(obj) || + is_special_form(obj) || + PyType_Check(obj) || + type == &Py_GenericAliasType || + type == &_Py_UnionType); +} + +static PyObject * +type_or(PyTypeObject* self, PyObject* param) +{ + PyObject *tuple = PyTuple_Pack(2, self, param); + if (tuple == NULL) { + return NULL; + } + PyObject *new_union = _Py_Union(tuple); + Py_DECREF(tuple); + return new_union; +} + +static int +union_repr_item(_PyUnicodeWriter *writer, PyObject *p) +{ + _Py_IDENTIFIER(__module__); + _Py_IDENTIFIER(__qualname__); + _Py_IDENTIFIER(__origin__); + _Py_IDENTIFIER(__args__); + PyObject *qualname = NULL; + PyObject *module = NULL; + PyObject *r = NULL; + int err; + + int has_origin = _PyObject_HasAttrId(p, &PyId___origin__); + if (has_origin < 0) { + goto exit; + } + + if (has_origin) { + int has_args = _PyObject_HasAttrId(p, &PyId___args__); + if (has_args < 0) { + goto exit; + } + if (has_args) { + // It looks like a GenericAlias + goto use_repr; + } + } + + if (_PyObject_LookupAttrId(p, &PyId___qualname__, &qualname) < 0) { + goto exit; + } + if (qualname == NULL) { + goto use_repr; + } + if (_PyObject_LookupAttrId(p, &PyId___module__, &module) < 0) { + goto exit; + } + if (module == NULL || module == Py_None) { + goto use_repr; + } + + // Looks like a class + if (PyUnicode_Check(module) && + _PyUnicode_EqualToASCIIString(module, "builtins")) + { + // builtins don't need a module name + r = PyObject_Str(qualname); + goto exit; + } + else { + r = PyUnicode_FromFormat("%S.%S", module, qualname); + goto exit; + } + +use_repr: + r = PyObject_Repr(p); +exit: + Py_XDECREF(qualname); + Py_XDECREF(module); + if (r == NULL) { + return -1; + } + err = _PyUnicodeWriter_WriteStr(writer, r); + Py_DECREF(r); + return err; +} + +static PyObject * +union_repr(PyObject *self) +{ + unionobject *alias = (unionobject *)self; + Py_ssize_t len = PyTuple_GET_SIZE(alias->args); + + _PyUnicodeWriter writer; + _PyUnicodeWriter_Init(&writer); + for (Py_ssize_t i = 0; i < len; i++) { + if (i > 0 && _PyUnicodeWriter_WriteASCIIString(&writer, " | ", 3) < 0) { + goto error; + } + PyObject *p = PyTuple_GET_ITEM(alias->args, i); + if (union_repr_item(&writer, p) < 0) { + goto error; + } + } + return _PyUnicodeWriter_Finish(&writer); +error: + _PyUnicodeWriter_Dealloc(&writer); + return NULL; +} + +static PyMemberDef union_members[] = { + {"__args__", T_OBJECT, offsetof(unionobject, args), READONLY}, + {0} +}; + +static PyMethodDef union_methods[] = { + {"__instancecheck__", union_instancecheck, METH_O}, + {"__subclasscheck__", union_subclasscheck, METH_O}, + {0}}; + +static PyNumberMethods union_as_number = { + .nb_or = (binaryfunc)type_or, // Add __or__ function +}; + +PyTypeObject _Py_UnionType = { + PyVarObject_HEAD_INIT(&PyType_Type, 0) + .tp_name = "types.Union", + .tp_doc = "Represent a PEP 604 union type\n" + "\n" + "E.g. for int | str", + .tp_basicsize = sizeof(unionobject), + .tp_dealloc = unionobject_dealloc, + .tp_alloc = PyType_GenericAlloc, + .tp_free = PyObject_Del, + .tp_flags = Py_TPFLAGS_DEFAULT, + .tp_hash = union_hash, + .tp_getattro = PyObject_GenericGetAttr, + .tp_members = union_members, + .tp_methods = union_methods, + .tp_richcompare = union_richcompare, + .tp_as_number = &union_as_number, + .tp_repr = union_repr, +}; + +PyObject * +_Py_Union(PyObject *args) +{ + assert(PyTuple_CheckExact(args)); + + unionobject* result = NULL; + + // Check arguments are unionable. + int nargs = PyTuple_GET_SIZE(args); + for (Py_ssize_t iarg = 0; iarg < nargs; iarg++) { + PyObject *arg = PyTuple_GET_ITEM(args, iarg); + if (arg == NULL) { + return NULL; + } + int is_arg_unionable = is_unionable(arg); + if (is_arg_unionable < 0) { + return NULL; + } + if (!is_arg_unionable) { + Py_INCREF(Py_NotImplemented); + return Py_NotImplemented; + } + } + + result = PyObject_New(unionobject, &_Py_UnionType); + if (result == NULL) { + return NULL; + } + + result->args = dedup_and_flatten_args(args); + if (result->args == NULL) { + Py_DECREF(result); + return NULL; + } + return (PyObject*)result; +} diff --git a/PCbuild/pythoncore.vcxproj b/PCbuild/pythoncore.vcxproj index 295b5e21eba5c..266a193c1e86a 100644 --- a/PCbuild/pythoncore.vcxproj +++ b/PCbuild/pythoncore.vcxproj @@ -196,6 +196,7 @@ + @@ -412,6 +413,7 @@ + diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index ec82e42f61e8b..22d9b79157698 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -573,6 +573,9 @@ Include\internal + + Include\internal + Modules\zlib @@ -1175,6 +1178,9 @@ Objects + + Objects + From webhook-mailer at python.org Tue Sep 15 09:45:37 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Tue, 15 Sep 2020 13:45:37 -0000 Subject: [Python-checkins] Add missing colon to IDLE doc markup (GH-22007) Message-ID: https://github.com/python/cpython/commit/7a797a3967fdfedbaf49fd1f140ee670d7db40ad commit: 7a797a3967fdfedbaf49fd1f140ee670d7db40ad branch: master author: Andre Delfino committer: GitHub date: 2020-09-09T18:17:14-04:00 summary: Add missing colon to IDLE doc markup (GH-22007) files: M Doc/library/idle.rst diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 75b6fa3861b23..43096b014fed3 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -491,7 +491,7 @@ in the settings dialog. (To prevent auto popups, set the delay to a large number of milliseconds, such as 100000000.) For imported module names or class or function attributes, type '.'. For filenames in the root directory, type :data:`os.sep` or -data:`os.altsep` immediately after an opening quote. (On Windows, +:data:`os.altsep` immediately after an opening quote. (On Windows, one can specify a drive first.) Move into subdirectories by typing a directory name and a separator. From webhook-mailer at python.org Tue Sep 15 09:45:55 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:45:55 -0000 Subject: [Python-checkins] Add missing colon to IDLE doc markup (GH-22007) Message-ID: https://github.com/python/cpython/commit/41dbd6e484d773d8c7181775fc578ff5a3b0a62a commit: 41dbd6e484d773d8c7181775fc578ff5a3b0a62a branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T15:29:51-07:00 summary: Add missing colon to IDLE doc markup (GH-22007) (cherry picked from commit 7a797a3967fdfedbaf49fd1f140ee670d7db40ad) Co-authored-by: Andre Delfino files: M Doc/library/idle.rst diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 75b6fa3861b23..43096b014fed3 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -491,7 +491,7 @@ in the settings dialog. (To prevent auto popups, set the delay to a large number of milliseconds, such as 100000000.) For imported module names or class or function attributes, type '.'. For filenames in the root directory, type :data:`os.sep` or -data:`os.altsep` immediately after an opening quote. (On Windows, +:data:`os.altsep` immediately after an opening quote. (On Windows, one can specify a drive first.) Move into subdirectories by typing a directory name and a separator. From webhook-mailer at python.org Tue Sep 15 09:46:37 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:46:37 -0000 Subject: [Python-checkins] Add missing colon to IDLE doc markup (GH-22007) Message-ID: https://github.com/python/cpython/commit/d423aa8e951ab6b9bcbf6cb3266646e04cf30b8f commit: d423aa8e951ab6b9bcbf6cb3266646e04cf30b8f branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T18:34:20-04:00 summary: Add missing colon to IDLE doc markup (GH-22007) (cherry picked from commit 7a797a3967fdfedbaf49fd1f140ee670d7db40ad) Co-authored-by: Andre Delfino Co-authored-by: Andre Delfino files: M Doc/library/idle.rst diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 75b6fa3861b23..43096b014fed3 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -491,7 +491,7 @@ in the settings dialog. (To prevent auto popups, set the delay to a large number of milliseconds, such as 100000000.) For imported module names or class or function attributes, type '.'. For filenames in the root directory, type :data:`os.sep` or -data:`os.altsep` immediately after an opening quote. (On Windows, +:data:`os.altsep` immediately after an opening quote. (On Windows, one can specify a drive first.) Move into subdirectories by typing a directory name and a separator. From webhook-mailer at python.org Tue Sep 15 09:46:56 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Tue, 15 Sep 2020 13:46:56 -0000 Subject: [Python-checkins] Update idlelib/help.html to current IDLE doc (GH-22181) Message-ID: https://github.com/python/cpython/commit/471247150e9707e583297ac6b4edff978efd8941 commit: 471247150e9707e583297ac6b4edff978efd8941 branch: master author: Terry Jan Reedy committer: GitHub date: 2020-09-09T18:53:18-04:00 summary: Update idlelib/help.html to current IDLE doc (GH-22181) files: M Lib/idlelib/help.html diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html index 81ce5100bb8ad..b2853cffe0c26 100644 --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -481,13 +481,13 @@

Automatic indentationos.sep or -data:os.altsep immediately after an opening quote. (On Windows, +os.altsep immediately after an opening quote. (On Windows, one can specify a drive first.) Move into subdirectories by typing a directory name and a separator.

-

Instead of waiting, or after a box is closed. open a completion box +

Instead of waiting, or after a box is closed, open a completion box immediately with Show Completions on the Edit menu. The default hot key is C-space. If one types a prefix for the desired name -before opening the box, the first match is displayed. +before opening the box, the first match or near miss is made visible. The result is the same as if one enters a prefix after the box is displayed. Show Completions after a quote completes filenames in the current directory instead of a root directory.

@@ -975,7 +975,7 @@

Navigation



- Last updated on Jul 08, 2020. + Last updated on Sep 09, 2020. Found a bug?
From webhook-mailer at python.org Tue Sep 15 09:47:26 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:47:26 -0000 Subject: [Python-checkins] Update idlelib/help.html to current IDLE doc (GH-22181) Message-ID: https://github.com/python/cpython/commit/9133b01b58ee1af821611cc436adb20f3e8b879d commit: 9133b01b58ee1af821611cc436adb20f3e8b879d branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T16:10:09-07:00 summary: Update idlelib/help.html to current IDLE doc (GH-22181) (cherry picked from commit 471247150e9707e583297ac6b4edff978efd8941) Co-authored-by: Terry Jan Reedy files: M Lib/idlelib/help.html diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html index 81ce5100bb8ad..b2853cffe0c26 100644 --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -481,13 +481,13 @@

Automatic indentationos.sep or -data:os.altsep immediately after an opening quote. (On Windows, +os.altsep immediately after an opening quote. (On Windows, one can specify a drive first.) Move into subdirectories by typing a directory name and a separator.

-

Instead of waiting, or after a box is closed. open a completion box +

Instead of waiting, or after a box is closed, open a completion box immediately with Show Completions on the Edit menu. The default hot key is C-space. If one types a prefix for the desired name -before opening the box, the first match is displayed. +before opening the box, the first match or near miss is made visible. The result is the same as if one enters a prefix after the box is displayed. Show Completions after a quote completes filenames in the current directory instead of a root directory.

@@ -975,7 +975,7 @@

Navigation



- Last updated on Jul 08, 2020. + Last updated on Sep 09, 2020. Found a bug?
From webhook-mailer at python.org Tue Sep 15 09:47:45 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:47:45 -0000 Subject: [Python-checkins] Update idlelib/help.html to current IDLE doc (GH-22181) Message-ID: https://github.com/python/cpython/commit/5f76d720928a868033444bdf73cac6f3515c0307 commit: 5f76d720928a868033444bdf73cac6f3515c0307 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T16:16:05-07:00 summary: Update idlelib/help.html to current IDLE doc (GH-22181) (cherry picked from commit 471247150e9707e583297ac6b4edff978efd8941) Co-authored-by: Terry Jan Reedy files: M Lib/idlelib/help.html diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html index 81ce5100bb8ad..b2853cffe0c26 100644 --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -481,13 +481,13 @@

Automatic indentationos.sep or -data:os.altsep immediately after an opening quote. (On Windows, +os.altsep immediately after an opening quote. (On Windows, one can specify a drive first.) Move into subdirectories by typing a directory name and a separator.

-

Instead of waiting, or after a box is closed. open a completion box +

Instead of waiting, or after a box is closed, open a completion box immediately with Show Completions on the Edit menu. The default hot key is C-space. If one types a prefix for the desired name -before opening the box, the first match is displayed. +before opening the box, the first match or near miss is made visible. The result is the same as if one enters a prefix after the box is displayed. Show Completions after a quote completes filenames in the current directory instead of a root directory.

@@ -975,7 +975,7 @@

Navigation



- Last updated on Jul 08, 2020. + Last updated on Sep 09, 2020. Found a bug?
From webhook-mailer at python.org Tue Sep 15 09:48:50 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Tue, 15 Sep 2020 13:48:50 -0000 Subject: [Python-checkins] [doc] Remove superfluous comment about equal in f-strings (GH-22006) Message-ID: https://github.com/python/cpython/commit/788b79fa7b6184221e68d4f1a3fbe0b3270693f6 commit: 788b79fa7b6184221e68d4f1a3fbe0b3270693f6 branch: master author: Andre Delfino committer: GitHub date: 2020-09-09T23:33:13-07:00 summary: [doc] Remove superfluous comment about equal in f-strings (GH-22006) Automerge-Triggered-By: @kushaldas files: M Doc/reference/lexical_analysis.rst diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 4c0f5688da4d7..19ba83a5513d8 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -702,7 +702,7 @@ defaults to the :func:`str` of the expression unless a conversion ``'!r'`` is declared. .. versionadded:: 3.8 - The equal sign ``'='`` was added in Python 3.8. + The equal sign ``'='``. If a conversion is specified, the result of evaluating the expression is converted before formatting. Conversion ``'!s'`` calls :func:`str` on From webhook-mailer at python.org Tue Sep 15 09:49:10 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:49:10 -0000 Subject: [Python-checkins] [doc] Remove superfluous comment about equal in f-strings (GH-22006) Message-ID: https://github.com/python/cpython/commit/e3d0e9bab9640bfe062cea972da374618472b056 commit: e3d0e9bab9640bfe062cea972da374618472b056 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T23:41:13-07:00 summary: [doc] Remove superfluous comment about equal in f-strings (GH-22006) Automerge-Triggered-By: @kushaldas (cherry picked from commit 788b79fa7b6184221e68d4f1a3fbe0b3270693f6) Co-authored-by: Andre Delfino files: M Doc/reference/lexical_analysis.rst diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 677b7e20f3776..ebacd7f471ee1 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -702,7 +702,7 @@ defaults to the :func:`str` of the expression unless a conversion ``'!r'`` is declared. .. versionadded:: 3.8 - The equal sign ``'='`` was added in Python 3.8. + The equal sign ``'='``. If a conversion is specified, the result of evaluating the expression is converted before formatting. Conversion ``'!s'`` calls :func:`str` on From webhook-mailer at python.org Tue Sep 15 09:49:28 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:49:28 -0000 Subject: [Python-checkins] [doc] Remove superfluous comment about equal in f-strings (GH-22006) Message-ID: https://github.com/python/cpython/commit/b86e62644fb394402850fbc16ca0b8b3638b8378 commit: b86e62644fb394402850fbc16ca0b8b3638b8378 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-09T23:48:54-07:00 summary: [doc] Remove superfluous comment about equal in f-strings (GH-22006) Automerge-Triggered-By: @kushaldas (cherry picked from commit 788b79fa7b6184221e68d4f1a3fbe0b3270693f6) Co-authored-by: Andre Delfino files: M Doc/reference/lexical_analysis.rst diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 4c0f5688da4d7..19ba83a5513d8 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -702,7 +702,7 @@ defaults to the :func:`str` of the expression unless a conversion ``'!r'`` is declared. .. versionadded:: 3.8 - The equal sign ``'='`` was added in Python 3.8. + The equal sign ``'='``. If a conversion is specified, the result of evaluating the expression is converted before formatting. Conversion ``'!s'`` calls :func:`str` on From webhook-mailer at python.org Tue Sep 15 09:49:44 2020 From: webhook-mailer at python.org (Bar Harel) Date: Tue, 15 Sep 2020 13:49:44 -0000 Subject: [Python-checkins] Update logging documentation to tidy up formatting (GH-22173) Message-ID: https://github.com/python/cpython/commit/1e874d5bec652198b012debeec309efc49ce1cdd commit: 1e874d5bec652198b012debeec309efc49ce1cdd branch: master author: Bar Harel committer: GitHub date: 2020-09-10T11:50:23+01:00 summary: Update logging documentation to tidy up formatting (GH-22173) files: M Doc/library/logging.rst diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst index 989016e649d65..fb8ea705b0469 100644 --- a/Doc/library/logging.rst +++ b/Doc/library/logging.rst @@ -529,8 +529,7 @@ The useful mapping keys in a :class:`LogRecord` are given in the section on :ref:`logrecord-attributes`. -.. class:: Formatter(fmt=None, datefmt=None, style='%', validate=True, *, - defaults=None) +.. class:: Formatter(fmt=None, datefmt=None, style='%', validate=True, *, defaults=None) Returns a new instance of the :class:`Formatter` class. The instance is initialized with a format string for the message as a whole, as well as a From webhook-mailer at python.org Tue Sep 15 09:50:22 2020 From: webhook-mailer at python.org (Mark Roseman) Date: Tue, 15 Sep 2020 13:50:22 -0000 Subject: [Python-checkins] bpo-37149: Change Shipman tkinter link from archive.org to TkDocs (GH-22188) Message-ID: https://github.com/python/cpython/commit/06d0b8b67e8aebd8fe4c34e97d6915c11f4afa30 commit: 06d0b8b67e8aebd8fe4c34e97d6915c11f4afa30 branch: master author: Mark Roseman committer: GitHub date: 2020-09-10T16:04:20-04:00 summary: bpo-37149: Change Shipman tkinter link from archive.org to TkDocs (GH-22188) The new link responds much faster and begins with a short explanation of the status of the doc. files: A Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst M Doc/library/tkinter.rst diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index 9f954255c8b30..7739f2f60a798 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -31,7 +31,7 @@ installed, so you can read the Tcl/Tk documentation specific to that version. `TKDocs `_ Extensive tutorial plus friendlier widget pages for some of the widgets. - `Tkinter 8.5 reference: a GUI for Python `_ + `Tkinter 8.5 reference: a GUI for Python `_ On-line reference material. `Tkinter docs from effbot `_ diff --git a/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst b/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst new file mode 100644 index 0000000000000..aeca652b4ed97 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst @@ -0,0 +1 @@ +Change Shipman tkinter doc link from archive.org to TkDocs. (The doc has been removed from the NMT server.) The new link responds much faster and includes a short explanatory note. From webhook-mailer at python.org Tue Sep 15 09:50:43 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:50:43 -0000 Subject: [Python-checkins] bpo-37149: Change Shipman tkinter link from archive.org to TkDocs (GH-22188) Message-ID: https://github.com/python/cpython/commit/1b4bdb4cd71df6339da3f247516e0c642f40c37e commit: 1b4bdb4cd71df6339da3f247516e0c642f40c37e branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-10T13:19:29-07:00 summary: bpo-37149: Change Shipman tkinter link from archive.org to TkDocs (GH-22188) The new link responds much faster and begins with a short explanation of the status of the doc. (cherry picked from commit 06d0b8b67e8aebd8fe4c34e97d6915c11f4afa30) Co-authored-by: Mark Roseman files: A Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst M Doc/library/tkinter.rst diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index 9f954255c8b30..7739f2f60a798 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -31,7 +31,7 @@ installed, so you can read the Tcl/Tk documentation specific to that version. `TKDocs `_ Extensive tutorial plus friendlier widget pages for some of the widgets. - `Tkinter 8.5 reference: a GUI for Python `_ + `Tkinter 8.5 reference: a GUI for Python `_ On-line reference material. `Tkinter docs from effbot `_ diff --git a/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst b/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst new file mode 100644 index 0000000000000..aeca652b4ed97 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst @@ -0,0 +1 @@ +Change Shipman tkinter doc link from archive.org to TkDocs. (The doc has been removed from the NMT server.) The new link responds much faster and includes a short explanatory note. From webhook-mailer at python.org Tue Sep 15 09:51:02 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:51:02 -0000 Subject: [Python-checkins] bpo-37149: Change Shipman tkinter link from archive.org to TkDocs (GH-22188) (#22193) Message-ID: https://github.com/python/cpython/commit/8a30bdd21dff6d1957df135c9d0b9983a0f61228 commit: 8a30bdd21dff6d1957df135c9d0b9983a0f61228 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-10T16:21:35-04:00 summary: bpo-37149: Change Shipman tkinter link from archive.org to TkDocs (GH-22188) (#22193) The new link responds much faster and begins with a short explanation of the status of the doc. (cherry picked from commit 06d0b8b67e8aebd8fe4c34e97d6915c11f4afa30) Co-authored-by: Mark Roseman files: A Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst M Doc/library/tkinter.rst diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index e28664d680a04..79633b772a04d 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -31,7 +31,7 @@ installed, so you can read the Tcl/Tk documentation specific to that version. `TKDocs `_ Extensive tutorial plus friendlier widget pages for some of the widgets. - `Tkinter 8.5 reference: a GUI for Python `_ + `Tkinter 8.5 reference: a GUI for Python `_ On-line reference material. `Tkinter docs from effbot `_ diff --git a/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst b/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst new file mode 100644 index 0000000000000..aeca652b4ed97 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst @@ -0,0 +1 @@ +Change Shipman tkinter doc link from archive.org to TkDocs. (The doc has been removed from the NMT server.) The new link responds much faster and includes a short explanatory note. From webhook-mailer at python.org Tue Sep 15 09:51:21 2020 From: webhook-mailer at python.org (Benjamin Peterson) Date: Tue, 15 Sep 2020 13:51:21 -0000 Subject: [Python-checkins] Doc: Fix alphabetical ordering of removeprefix/suffix. (GH-22194) Message-ID: https://github.com/python/cpython/commit/3304cbd990f06b72480a6dc8f6f4fe1a1376dde7 commit: 3304cbd990f06b72480a6dc8f6f4fe1a1376dde7 branch: master author: Benjamin Peterson committer: GitHub date: 2020-09-11T08:59:02+09:00 summary: Doc: Fix alphabetical ordering of removeprefix/suffix. (GH-22194) files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 5a10faa7bbd29..0ffe7b7526fa7 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1568,33 +1568,6 @@ expression support in the :mod:`re` module). interpreted as in slice notation. -.. method:: str.removeprefix(prefix, /) - - If the string starts with the *prefix* string, return - ``string[len(prefix):]``. Otherwise, return a copy of the original - string:: - - >>> 'TestHook'.removeprefix('Test') - 'Hook' - >>> 'BaseTestCase'.removeprefix('Test') - 'BaseTestCase' - - .. versionadded:: 3.9 - -.. method:: str.removesuffix(suffix, /) - - If the string ends with the *suffix* string and that *suffix* is not empty, - return ``string[:-len(suffix)]``. Otherwise, return a copy of the - original string:: - - >>> 'MiscTests'.removesuffix('Tests') - 'Misc' - >>> 'TmpDirMixin'.removesuffix('Tests') - 'TmpDirMixin' - - .. versionadded:: 3.9 - - .. method:: str.encode(encoding="utf-8", errors="strict") Return an encoded version of the string as a bytes object. Default encoding @@ -1909,6 +1882,34 @@ expression support in the :mod:`re` module). the string itself, followed by two empty strings. +.. method:: str.removeprefix(prefix, /) + + If the string starts with the *prefix* string, return + ``string[len(prefix):]``. Otherwise, return a copy of the original + string:: + + >>> 'TestHook'.removeprefix('Test') + 'Hook' + >>> 'BaseTestCase'.removeprefix('Test') + 'BaseTestCase' + + .. versionadded:: 3.9 + + +.. method:: str.removesuffix(suffix, /) + + If the string ends with the *suffix* string and that *suffix* is not empty, + return ``string[:-len(suffix)]``. Otherwise, return a copy of the + original string:: + + >>> 'MiscTests'.removesuffix('Tests') + 'Misc' + >>> 'TmpDirMixin'.removesuffix('Tests') + 'TmpDirMixin' + + .. versionadded:: 3.9 + + .. method:: str.replace(old, new[, count]) Return a copy of the string with all occurrences of substring *old* replaced by From webhook-mailer at python.org Tue Sep 15 09:51:38 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:51:38 -0000 Subject: [Python-checkins] Doc: Fix alphabetical ordering of removeprefix/suffix. (GH-22194) Message-ID: https://github.com/python/cpython/commit/7af6bf4dad3762eeb3fb17e955e69729e35e9a97 commit: 7af6bf4dad3762eeb3fb17e955e69729e35e9a97 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-10T17:13:56-07:00 summary: Doc: Fix alphabetical ordering of removeprefix/suffix. (GH-22194) (cherry picked from commit 3304cbd990f06b72480a6dc8f6f4fe1a1376dde7) Co-authored-by: Benjamin Peterson files: M Doc/library/stdtypes.rst diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 6a9fdcb38d24b..c3f5c04aac7d9 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1547,33 +1547,6 @@ expression support in the :mod:`re` module). interpreted as in slice notation. -.. method:: str.removeprefix(prefix, /) - - If the string starts with the *prefix* string, return - ``string[len(prefix):]``. Otherwise, return a copy of the original - string:: - - >>> 'TestHook'.removeprefix('Test') - 'Hook' - >>> 'BaseTestCase'.removeprefix('Test') - 'BaseTestCase' - - .. versionadded:: 3.9 - -.. method:: str.removesuffix(suffix, /) - - If the string ends with the *suffix* string and that *suffix* is not empty, - return ``string[:-len(suffix)]``. Otherwise, return a copy of the - original string:: - - >>> 'MiscTests'.removesuffix('Tests') - 'Misc' - >>> 'TmpDirMixin'.removesuffix('Tests') - 'TmpDirMixin' - - .. versionadded:: 3.9 - - .. method:: str.encode(encoding="utf-8", errors="strict") Return an encoded version of the string as a bytes object. Default encoding @@ -1888,6 +1861,34 @@ expression support in the :mod:`re` module). the string itself, followed by two empty strings. +.. method:: str.removeprefix(prefix, /) + + If the string starts with the *prefix* string, return + ``string[len(prefix):]``. Otherwise, return a copy of the original + string:: + + >>> 'TestHook'.removeprefix('Test') + 'Hook' + >>> 'BaseTestCase'.removeprefix('Test') + 'BaseTestCase' + + .. versionadded:: 3.9 + + +.. method:: str.removesuffix(suffix, /) + + If the string ends with the *suffix* string and that *suffix* is not empty, + return ``string[:-len(suffix)]``. Otherwise, return a copy of the + original string:: + + >>> 'MiscTests'.removesuffix('Tests') + 'Misc' + >>> 'TmpDirMixin'.removesuffix('Tests') + 'TmpDirMixin' + + .. versionadded:: 3.9 + + .. method:: str.replace(old, new[, count]) Return a copy of the string with all occurrences of substring *old* replaced by From webhook-mailer at python.org Tue Sep 15 09:51:55 2020 From: webhook-mailer at python.org (Stargirl Flowers) Date: Tue, 15 Sep 2020 13:51:55 -0000 Subject: [Python-checkins] [doc] struct: update note about network byte order form to be more helpful (GH-22201) Message-ID: https://github.com/python/cpython/commit/fb2718720346c8c7a0ad2d7477f20e9a5524ea0c commit: fb2718720346c8c7a0ad2d7477f20e9a5524ea0c branch: master author: Stargirl Flowers committer: GitHub date: 2020-09-11T08:20:12-07:00 summary: [doc] struct: update note about network byte order form to be more helpful (GH-22201) Update the sentence to provide some context on why network byte order is defined as big endian. files: M Doc/library/struct.rst diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index 856b6da8bb255..eccba20fb8fe7 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -159,8 +159,8 @@ the :ref:`format-characters` section. Note the difference between ``'@'`` and ``'='``: both use native byte order, but the size and alignment of the latter is standardized. -The form ``'!'`` is available for those poor souls who claim they can't remember -whether network byte order is big-endian or little-endian. +The form ``'!'`` represents the network byte order which is always big-endian +as defined in `IETF RFC 1700 `_. There is no way to indicate non-native byte order (force byte-swapping); use the appropriate choice of ``'<'`` or ``'>'``. @@ -467,3 +467,5 @@ The :mod:`struct` module also defines the following type: .. _half precision format: https://en.wikipedia.org/wiki/Half-precision_floating-point_format .. _ieee 754 standard: https://en.wikipedia.org/wiki/IEEE_floating_point#IEEE_754-2008 + +.. _IETF RFC 1700: https://tools.ietf.org/html/rfc1700 From webhook-mailer at python.org Tue Sep 15 09:52:16 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:52:16 -0000 Subject: [Python-checkins] [doc] struct: update note about network byte order form to be more helpful (GH-22201) Message-ID: https://github.com/python/cpython/commit/e4c6947024d5ad3d90c0a35f762bb85ccc2b0092 commit: e4c6947024d5ad3d90c0a35f762bb85ccc2b0092 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-11T08:29:04-07:00 summary: [doc] struct: update note about network byte order form to be more helpful (GH-22201) Update the sentence to provide some context on why network byte order is defined as big endian. (cherry picked from commit fb2718720346c8c7a0ad2d7477f20e9a5524ea0c) Co-authored-by: Stargirl Flowers files: M Doc/library/struct.rst diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index 56b0a3614df53..278cb19f5b472 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -159,8 +159,8 @@ the :ref:`format-characters` section. Note the difference between ``'@'`` and ``'='``: both use native byte order, but the size and alignment of the latter is standardized. -The form ``'!'`` is available for those poor souls who claim they can't remember -whether network byte order is big-endian or little-endian. +The form ``'!'`` represents the network byte order which is always big-endian +as defined in `IETF RFC 1700 `_. There is no way to indicate non-native byte order (force byte-swapping); use the appropriate choice of ``'<'`` or ``'>'``. @@ -467,3 +467,5 @@ The :mod:`struct` module also defines the following type: .. _half precision format: https://en.wikipedia.org/wiki/Half-precision_floating-point_format .. _ieee 754 standard: https://en.wikipedia.org/wiki/IEEE_floating_point#IEEE_754-2008 + +.. _IETF RFC 1700: https://tools.ietf.org/html/rfc1700 From webhook-mailer at python.org Tue Sep 15 09:52:34 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:52:34 -0000 Subject: [Python-checkins] [doc] struct: update note about network byte order form to be more helpful (GH-22201) Message-ID: https://github.com/python/cpython/commit/c97e402a4c1e6d4012068f3d070cd676fa9bb48a commit: c97e402a4c1e6d4012068f3d070cd676fa9bb48a branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-11T08:35:23-07:00 summary: [doc] struct: update note about network byte order form to be more helpful (GH-22201) Update the sentence to provide some context on why network byte order is defined as big endian. (cherry picked from commit fb2718720346c8c7a0ad2d7477f20e9a5524ea0c) Co-authored-by: Stargirl Flowers files: M Doc/library/struct.rst diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index 856b6da8bb255..eccba20fb8fe7 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -159,8 +159,8 @@ the :ref:`format-characters` section. Note the difference between ``'@'`` and ``'='``: both use native byte order, but the size and alignment of the latter is standardized. -The form ``'!'`` is available for those poor souls who claim they can't remember -whether network byte order is big-endian or little-endian. +The form ``'!'`` represents the network byte order which is always big-endian +as defined in `IETF RFC 1700 `_. There is no way to indicate non-native byte order (force byte-swapping); use the appropriate choice of ``'<'`` or ``'>'``. @@ -467,3 +467,5 @@ The :mod:`struct` module also defines the following type: .. _half precision format: https://en.wikipedia.org/wiki/Half-precision_floating-point_format .. _ieee 754 standard: https://en.wikipedia.org/wiki/IEEE_floating_point#IEEE_754-2008 + +.. _IETF RFC 1700: https://tools.ietf.org/html/rfc1700 From webhook-mailer at python.org Tue Sep 15 09:52:48 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Tue, 15 Sep 2020 13:52:48 -0000 Subject: [Python-checkins] bpo-41729: Fix test_winconsole failures (3) and hang (GH-22146) Message-ID: https://github.com/python/cpython/commit/31c9828ec026e5d9b9122d55bf0aa7cb45bfecc5 commit: 31c9828ec026e5d9b9122d55bf0aa7cb45bfecc5 branch: master author: Terry Jan Reedy committer: GitHub date: 2020-09-12T01:51:52-04:00 summary: bpo-41729: Fix test_winconsole failures (3) and hang (GH-22146) The problems occured with a repository build on machine with freshly updated Windows 10 Pro. files: M PC/_testconsole.c diff --git a/PC/_testconsole.c b/PC/_testconsole.c index f6fcfcf1964b8..b62f21c339aa4 100644 --- a/PC/_testconsole.c +++ b/PC/_testconsole.c @@ -63,7 +63,7 @@ _testconsole_write_input_impl(PyObject *module, PyObject *file, for (DWORD i = 0; i < size; ++i, ++p, ++prec) { prec->EventType = KEY_EVENT; prec->Event.KeyEvent.bKeyDown = TRUE; - prec->Event.KeyEvent.wRepeatCount = 10; + prec->Event.KeyEvent.wRepeatCount = 1; prec->Event.KeyEvent.uChar.UnicodeChar = *p; } From webhook-mailer at python.org Tue Sep 15 09:53:06 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:53:06 -0000 Subject: [Python-checkins] bpo-41729: Fix test_winconsole failures (3) and hang (GH-22146) Message-ID: https://github.com/python/cpython/commit/bbab34084e876291d96a65741f075db3adfa7733 commit: bbab34084e876291d96a65741f075db3adfa7733 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-11T23:09:08-07:00 summary: bpo-41729: Fix test_winconsole failures (3) and hang (GH-22146) The problems occured with a repository build on machine with freshly updated Windows 10 Pro. (cherry picked from commit 31c9828ec026e5d9b9122d55bf0aa7cb45bfecc5) Co-authored-by: Terry Jan Reedy files: M PC/_testconsole.c diff --git a/PC/_testconsole.c b/PC/_testconsole.c index 23d1286ed4f49..1823ff2cf1cc0 100644 --- a/PC/_testconsole.c +++ b/PC/_testconsole.c @@ -64,7 +64,7 @@ _testconsole_write_input_impl(PyObject *module, PyObject *file, for (DWORD i = 0; i < size; ++i, ++p, ++prec) { prec->EventType = KEY_EVENT; prec->Event.KeyEvent.bKeyDown = TRUE; - prec->Event.KeyEvent.wRepeatCount = 10; + prec->Event.KeyEvent.wRepeatCount = 1; prec->Event.KeyEvent.uChar.UnicodeChar = *p; } From webhook-mailer at python.org Tue Sep 15 09:53:26 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:53:26 -0000 Subject: [Python-checkins] bpo-41729: Fix test_winconsole failures (3) and hang (GH-22146) Message-ID: https://github.com/python/cpython/commit/c2577b9e0e2f8652f079b4ac473347c63f15e25c commit: c2577b9e0e2f8652f079b4ac473347c63f15e25c branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-12T02:18:59-04:00 summary: bpo-41729: Fix test_winconsole failures (3) and hang (GH-22146) The problems occured with a repository build on machine with freshly updated Windows 10 Pro. (cherry picked from commit 31c9828ec026e5d9b9122d55bf0aa7cb45bfecc5) Co-authored-by: Terry Jan Reedy files: M PC/_testconsole.c diff --git a/PC/_testconsole.c b/PC/_testconsole.c index f6fcfcf1964b8..b62f21c339aa4 100644 --- a/PC/_testconsole.c +++ b/PC/_testconsole.c @@ -63,7 +63,7 @@ _testconsole_write_input_impl(PyObject *module, PyObject *file, for (DWORD i = 0; i < size; ++i, ++p, ++prec) { prec->EventType = KEY_EVENT; prec->Event.KeyEvent.bKeyDown = TRUE; - prec->Event.KeyEvent.wRepeatCount = 10; + prec->Event.KeyEvent.wRepeatCount = 1; prec->Event.KeyEvent.uChar.UnicodeChar = *p; } From webhook-mailer at python.org Tue Sep 15 09:53:53 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Tue, 15 Sep 2020 13:53:53 -0000 Subject: [Python-checkins] bpo-41731: Make test_cmd_line_script pass with -vv (GH-22206) Message-ID: https://github.com/python/cpython/commit/7e711ead26fea6465e0ef2e3b8880b57ba8fc129 commit: 7e711ead26fea6465e0ef2e3b8880b57ba8fc129 branch: master author: Terry Jan Reedy committer: GitHub date: 2020-09-12T02:25:36-04:00 summary: bpo-41731: Make test_cmd_line_script pass with -vv (GH-22206) Argument script_exec_args is usually an absolute file name, but twice has form ['-m', 'module_name']. files: A Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst M Lib/test/test_cmd_line_script.py diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index 6c8c28f40b564..f10ab40017a33 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -145,7 +145,7 @@ def _check_import_error(self, script_exec_args, expected_msg, *run_args, __isolated=False, __cwd=cwd, **env_vars ) if verbose > 1: - print('Output from test script %r:' % script_exec_args) + print(f'Output from test script {script_exec_args!r:}') print(repr(err)) print('Expected output: %r' % expected_msg) self.assertIn(expected_msg.encode('utf-8'), err) diff --git a/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst b/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst new file mode 100644 index 0000000000000..e368a60f77b1e --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst @@ -0,0 +1 @@ +Make test_cmd_line_script pass with option '-vv'. From webhook-mailer at python.org Tue Sep 15 09:54:48 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:54:48 -0000 Subject: [Python-checkins] bpo-41731: Make test_cmd_line_script pass with -vv (GH-22206) Message-ID: https://github.com/python/cpython/commit/77901dc6c3d06dd31f6c84b2d3cb21dc26b1e351 commit: 77901dc6c3d06dd31f6c84b2d3cb21dc26b1e351 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-11T23:43:27-07:00 summary: bpo-41731: Make test_cmd_line_script pass with -vv (GH-22206) Argument script_exec_args is usually an absolute file name, but twice has form ['-m', 'module_name']. (cherry picked from commit 7e711ead26fea6465e0ef2e3b8880b57ba8fc129) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst M Lib/test/test_cmd_line_script.py diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index f0bd013e55367..a4a868adb08b7 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -143,7 +143,7 @@ def _check_import_error(self, script_exec_args, expected_msg, *run_args, __isolated=False, __cwd=cwd, **env_vars ) if verbose > 1: - print('Output from test script %r:' % script_exec_args) + print(f'Output from test script {script_exec_args!r:}') print(repr(err)) print('Expected output: %r' % expected_msg) self.assertIn(expected_msg.encode('utf-8'), err) diff --git a/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst b/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst new file mode 100644 index 0000000000000..e368a60f77b1e --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst @@ -0,0 +1 @@ +Make test_cmd_line_script pass with option '-vv'. From webhook-mailer at python.org Tue Sep 15 09:55:17 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:55:17 -0000 Subject: [Python-checkins] bpo-41731: Make test_cmd_line_script pass with -vv (GH-22206) Message-ID: https://github.com/python/cpython/commit/34e3c7592be70f652c293eb179593bf2928adeb4 commit: 34e3c7592be70f652c293eb179593bf2928adeb4 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-11T23:45:34-07:00 summary: bpo-41731: Make test_cmd_line_script pass with -vv (GH-22206) Argument script_exec_args is usually an absolute file name, but twice has form ['-m', 'module_name']. (cherry picked from commit 7e711ead26fea6465e0ef2e3b8880b57ba8fc129) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst M Lib/test/test_cmd_line_script.py diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index 15fca7b8a5191..7a3707d0f112f 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -143,7 +143,7 @@ def _check_import_error(self, script_exec_args, expected_msg, *run_args, __isolated=False, __cwd=cwd, **env_vars ) if verbose > 1: - print('Output from test script %r:' % script_exec_args) + print(f'Output from test script {script_exec_args!r:}') print(repr(err)) print('Expected output: %r' % expected_msg) self.assertIn(expected_msg.encode('utf-8'), err) diff --git a/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst b/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst new file mode 100644 index 0000000000000..e368a60f77b1e --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst @@ -0,0 +1 @@ +Make test_cmd_line_script pass with option '-vv'. From webhook-mailer at python.org Tue Sep 15 09:56:16 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 15 Sep 2020 13:56:16 -0000 Subject: [Python-checkins] bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197) Message-ID: https://github.com/python/cpython/commit/1b0f0e3d7d03155da1cf9769a847874d559e57e3 commit: 1b0f0e3d7d03155da1cf9769a847874d559e57e3 branch: master author: Victor Stinner committer: GitHub date: 2020-09-12T08:50:18+02:00 summary: bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197) Fix a race condition in the call_soon_threadsafe() method of asyncio.ProactorEventLoop: do nothing if the self-pipe socket has been closed. files: A Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst M Lib/asyncio/proactor_events.py M Lib/asyncio/selector_events.py diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py index 4670bd683ab32..45c11ee4b487e 100644 --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -793,8 +793,17 @@ def _loop_self_reading(self, f=None): f.add_done_callback(self._loop_self_reading) def _write_to_self(self): + # This may be called from a different thread, possibly after + # _close_self_pipe() has been called or even while it is + # running. Guard for self._csock being None or closed. When + # a socket is closed, send() raises OSError (with errno set to + # EBADF, but let's not rely on the exact error code). + csock = self._csock + if csock is None: + return + try: - self._csock.send(b'\0') + csock.send(b'\0') except OSError: if self._debug: logger.debug("Fail to write a null byte into the " diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index 8495a7901cd34..59cb6b1babec5 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -133,14 +133,16 @@ def _write_to_self(self): # a socket is closed, send() raises OSError (with errno set to # EBADF, but let's not rely on the exact error code). csock = self._csock - if csock is not None: - try: - csock.send(b'\0') - except OSError: - if self._debug: - logger.debug("Fail to write a null byte into the " - "self-pipe socket", - exc_info=True) + if csock is None: + return + + try: + csock.send(b'\0') + except OSError: + if self._debug: + logger.debug("Fail to write a null byte into the " + "self-pipe socket", + exc_info=True) def _start_serving(self, protocol_factory, sock, sslcontext=None, server=None, backlog=100, diff --git a/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst b/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst new file mode 100644 index 0000000000000..78dcff1370029 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst @@ -0,0 +1,3 @@ +Fix a race condition in the ``call_soon_threadsafe()`` method of +``asyncio.ProactorEventLoop``: do nothing if the self-pipe socket has been +closed. From webhook-mailer at python.org Tue Sep 15 09:56:36 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:56:36 -0000 Subject: [Python-checkins] bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197) Message-ID: https://github.com/python/cpython/commit/530d1105ed7b0aa5ef76e3116918fe39bc6a4823 commit: 530d1105ed7b0aa5ef76e3116918fe39bc6a4823 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-12T00:09:54-07:00 summary: bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197) Fix a race condition in the call_soon_threadsafe() method of asyncio.ProactorEventLoop: do nothing if the self-pipe socket has been closed. (cherry picked from commit 1b0f0e3d7d03155da1cf9769a847874d559e57e3) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst M Lib/asyncio/proactor_events.py M Lib/asyncio/selector_events.py diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py index 54e4deb2c349c..3e0a14f6f4224 100644 --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -791,8 +791,17 @@ def _loop_self_reading(self, f=None): f.add_done_callback(self._loop_self_reading) def _write_to_self(self): + # This may be called from a different thread, possibly after + # _close_self_pipe() has been called or even while it is + # running. Guard for self._csock being None or closed. When + # a socket is closed, send() raises OSError (with errno set to + # EBADF, but let's not rely on the exact error code). + csock = self._csock + if csock is None: + return + try: - self._csock.send(b'\0') + csock.send(b'\0') except OSError: if self._debug: logger.debug("Fail to write a null byte into the " diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index a05cbb6bdd69d..6ba307163e1f7 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -133,14 +133,16 @@ def _write_to_self(self): # a socket is closed, send() raises OSError (with errno set to # EBADF, but let's not rely on the exact error code). csock = self._csock - if csock is not None: - try: - csock.send(b'\0') - except OSError: - if self._debug: - logger.debug("Fail to write a null byte into the " - "self-pipe socket", - exc_info=True) + if csock is None: + return + + try: + csock.send(b'\0') + except OSError: + if self._debug: + logger.debug("Fail to write a null byte into the " + "self-pipe socket", + exc_info=True) def _start_serving(self, protocol_factory, sock, sslcontext=None, server=None, backlog=100, diff --git a/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst b/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst new file mode 100644 index 0000000000000..78dcff1370029 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst @@ -0,0 +1,3 @@ +Fix a race condition in the ``call_soon_threadsafe()`` method of +``asyncio.ProactorEventLoop``: do nothing if the self-pipe socket has been +closed. From webhook-mailer at python.org Tue Sep 15 09:56:39 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:56:39 -0000 Subject: [Python-checkins] bpo-41776: Revise example of "continue" in the tutorial documentation (GH-22234) (GH-22255) Message-ID: https://github.com/python/cpython/commit/0cc037f8a72c283bf64d1968e34cbdc22b0e3010 commit: 0cc037f8a72c283bf64d1968e34cbdc22b0e3010 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-15T09:56:28-04:00 summary: bpo-41776: Revise example of "continue" in the tutorial documentation (GH-22234) (GH-22255) Revise example of "continue" in the tutorial documentation (cherry picked from commit 7bcc6456ad4704da9b287c8045768fa53961adc5) Co-authored-by: Neeraj Samtani Co-authored-by: Neeraj Samtani files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index 547f4aec47b68..3af288a17b270 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -207,15 +207,15 @@ iteration of the loop:: ... if num % 2 == 0: ... print("Found an even number", num) ... continue - ... print("Found a number", num) + ... print("Found an odd number", num) Found an even number 2 - Found a number 3 + Found an odd number 3 Found an even number 4 - Found a number 5 + Found an odd number 5 Found an even number 6 - Found a number 7 + Found an odd number 7 Found an even number 8 - Found a number 9 + Found an odd number 9 .. _tut-pass: From webhook-mailer at python.org Tue Sep 15 09:56:47 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:56:47 -0000 Subject: [Python-checkins] bpo-41776: Revise example of "continue" in the tutorial documentation (GH-22234) (GH-22256) Message-ID: https://github.com/python/cpython/commit/a0da90720d5330c53b8083272374ede1c7a1e33a commit: a0da90720d5330c53b8083272374ede1c7a1e33a branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-15T09:56:43-04:00 summary: bpo-41776: Revise example of "continue" in the tutorial documentation (GH-22234) (GH-22256) Revise example of "continue" in the tutorial documentation (cherry picked from commit 7bcc6456ad4704da9b287c8045768fa53961adc5) Co-authored-by: Neeraj Samtani Co-authored-by: Neeraj Samtani files: M Doc/tutorial/controlflow.rst diff --git a/Doc/tutorial/controlflow.rst b/Doc/tutorial/controlflow.rst index 547f4aec47b68..3af288a17b270 100644 --- a/Doc/tutorial/controlflow.rst +++ b/Doc/tutorial/controlflow.rst @@ -207,15 +207,15 @@ iteration of the loop:: ... if num % 2 == 0: ... print("Found an even number", num) ... continue - ... print("Found a number", num) + ... print("Found an odd number", num) Found an even number 2 - Found a number 3 + Found an odd number 3 Found an even number 4 - Found a number 5 + Found an odd number 5 Found an even number 6 - Found a number 7 + Found an odd number 7 Found an even number 8 - Found a number 9 + Found an odd number 9 .. _tut-pass: From webhook-mailer at python.org Tue Sep 15 09:56:56 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:56:56 -0000 Subject: [Python-checkins] bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197) Message-ID: https://github.com/python/cpython/commit/7dfcc8e0795cce6a8cb42ae2a7f159a38da6b38a commit: 7dfcc8e0795cce6a8cb42ae2a7f159a38da6b38a branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-12T00:11:38-07:00 summary: bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197) Fix a race condition in the call_soon_threadsafe() method of asyncio.ProactorEventLoop: do nothing if the self-pipe socket has been closed. (cherry picked from commit 1b0f0e3d7d03155da1cf9769a847874d559e57e3) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst M Lib/asyncio/proactor_events.py M Lib/asyncio/selector_events.py diff --git a/Lib/asyncio/proactor_events.py b/Lib/asyncio/proactor_events.py index 6b6bf8945e588..b4cd414b82d72 100644 --- a/Lib/asyncio/proactor_events.py +++ b/Lib/asyncio/proactor_events.py @@ -791,8 +791,17 @@ def _loop_self_reading(self, f=None): f.add_done_callback(self._loop_self_reading) def _write_to_self(self): + # This may be called from a different thread, possibly after + # _close_self_pipe() has been called or even while it is + # running. Guard for self._csock being None or closed. When + # a socket is closed, send() raises OSError (with errno set to + # EBADF, but let's not rely on the exact error code). + csock = self._csock + if csock is None: + return + try: - self._csock.send(b'\0') + csock.send(b'\0') except OSError: if self._debug: logger.debug("Fail to write a null byte into the " diff --git a/Lib/asyncio/selector_events.py b/Lib/asyncio/selector_events.py index 8495a7901cd34..59cb6b1babec5 100644 --- a/Lib/asyncio/selector_events.py +++ b/Lib/asyncio/selector_events.py @@ -133,14 +133,16 @@ def _write_to_self(self): # a socket is closed, send() raises OSError (with errno set to # EBADF, but let's not rely on the exact error code). csock = self._csock - if csock is not None: - try: - csock.send(b'\0') - except OSError: - if self._debug: - logger.debug("Fail to write a null byte into the " - "self-pipe socket", - exc_info=True) + if csock is None: + return + + try: + csock.send(b'\0') + except OSError: + if self._debug: + logger.debug("Fail to write a null byte into the " + "self-pipe socket", + exc_info=True) def _start_serving(self, protocol_factory, sock, sslcontext=None, server=None, backlog=100, diff --git a/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst b/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst new file mode 100644 index 0000000000000..78dcff1370029 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst @@ -0,0 +1,3 @@ +Fix a race condition in the ``call_soon_threadsafe()`` method of +``asyncio.ProactorEventLoop``: do nothing if the self-pipe socket has been +closed. From webhook-mailer at python.org Tue Sep 15 09:57:15 2020 From: webhook-mailer at python.org (Norbert Cyran) Date: Tue, 15 Sep 2020 13:57:15 -0000 Subject: [Python-checkins] bpo-41672: Fix type mismatches in imaplib docs (GH-22207) Message-ID: https://github.com/python/cpython/commit/c75330605d4795850ec74fdc4d69aa5d92f76c00 commit: c75330605d4795850ec74fdc4d69aa5d92f76c00 branch: master author: Norbert Cyran committer: GitHub date: 2020-09-12T16:58:56+09:00 summary: bpo-41672: Fix type mismatches in imaplib docs (GH-22207) files: M Doc/library/imaplib.rst diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst index 7c5b075016159..02ecfd95d4376 100644 --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -143,7 +143,7 @@ The following utility functions are defined: .. function:: Int2AP(num) - Converts an integer into a string representation using characters from the set + Converts an integer into a bytes representation using characters from the set [``A`` .. ``P``]. @@ -197,7 +197,7 @@ you want to avoid having an argument string quoted (eg: the *flags* argument to Each command returns a tuple: ``(type, [data, ...])`` where *type* is usually ``'OK'`` or ``'NO'``, and *data* is either the text from the command response, -or mandated results from the command. Each *data* is either a string, or a +or mandated results from the command. Each *data* is either a ``bytes``, or a tuple. If a tuple, then the first part is the header of the response, and the second part contains the data (ie: 'literal' value). From webhook-mailer at python.org Tue Sep 15 09:57:33 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:57:33 -0000 Subject: [Python-checkins] bpo-41672: Fix type mismatches in imaplib docs (GH-22207) Message-ID: https://github.com/python/cpython/commit/98f6e67a3d6e57727177ec2fb5cb68ef9a468aca commit: 98f6e67a3d6e57727177ec2fb5cb68ef9a468aca branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-12T01:14:37-07:00 summary: bpo-41672: Fix type mismatches in imaplib docs (GH-22207) (cherry picked from commit c75330605d4795850ec74fdc4d69aa5d92f76c00) Co-authored-by: Norbert Cyran files: M Doc/library/imaplib.rst diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst index 7c5b075016159..02ecfd95d4376 100644 --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -143,7 +143,7 @@ The following utility functions are defined: .. function:: Int2AP(num) - Converts an integer into a string representation using characters from the set + Converts an integer into a bytes representation using characters from the set [``A`` .. ``P``]. @@ -197,7 +197,7 @@ you want to avoid having an argument string quoted (eg: the *flags* argument to Each command returns a tuple: ``(type, [data, ...])`` where *type* is usually ``'OK'`` or ``'NO'``, and *data* is either the text from the command response, -or mandated results from the command. Each *data* is either a string, or a +or mandated results from the command. Each *data* is either a ``bytes``, or a tuple. If a tuple, then the first part is the header of the response, and the second part contains the data (ie: 'literal' value). From webhook-mailer at python.org Tue Sep 15 09:57:50 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:57:50 -0000 Subject: [Python-checkins] bpo-41672: Fix type mismatches in imaplib docs (GH-22207) (#22218) Message-ID: https://github.com/python/cpython/commit/3a150c77c31da240f1b92cab9ec74f1fa701b869 commit: 3a150c77c31da240f1b92cab9ec74f1fa701b869 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-12T20:04:50+09:00 summary: bpo-41672: Fix type mismatches in imaplib docs (GH-22207) (#22218) (cherry picked from commit c75330605d4795850ec74fdc4d69aa5d92f76c00) Co-authored-by: Norbert Cyran Co-authored-by: Norbert Cyran files: M Doc/library/imaplib.rst diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst index df63d820cfe04..b3d533728df6d 100644 --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -132,7 +132,7 @@ The following utility functions are defined: .. function:: Int2AP(num) - Converts an integer into a string representation using characters from the set + Converts an integer into a bytes representation using characters from the set [``A`` .. ``P``]. @@ -186,7 +186,7 @@ you want to avoid having an argument string quoted (eg: the *flags* argument to Each command returns a tuple: ``(type, [data, ...])`` where *type* is usually ``'OK'`` or ``'NO'``, and *data* is either the text from the command response, -or mandated results from the command. Each *data* is either a string, or a +or mandated results from the command. Each *data* is either a ``bytes``, or a tuple. If a tuple, then the first part is the header of the response, and the second part contains the data (ie: 'literal' value). From webhook-mailer at python.org Tue Sep 15 09:58:09 2020 From: webhook-mailer at python.org (Sergey Fedoseev) Date: Tue, 15 Sep 2020 13:58:09 -0000 Subject: [Python-checkins] bpo-33239: Fix default value of 'buffering' parameter in docs of tempfile.* functions (GH-21763) Message-ID: https://github.com/python/cpython/commit/b48389d95093c3f912549add8da339edc164bf0d commit: b48389d95093c3f912549add8da339edc164bf0d branch: master author: Sergey Fedoseev committer: GitHub date: 2020-09-13T10:59:01-07:00 summary: bpo-33239: Fix default value of 'buffering' parameter in docs of tempfile.* functions (GH-21763) `None` doesn't work: ```python >>> import tempfile >>> tempfile.TemporaryFile(buffering=None) Traceback (most recent call last): File "", line 1, in File "/home/sergey/tmp/cpython-dev/Lib/tempfile.py", line 607, in TemporaryFile return _io.open(fd, mode, buffering=buffering, TypeError: 'NoneType' object cannot be interpreted as an integer ``` Automerge-Triggered-By: @vsajip files: M Doc/library/tempfile.rst diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index 3a2b88c0cb6a2..f9421da5fe7df 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -31,7 +31,7 @@ is recommended to use keyword arguments for clarity. The module defines the following user-callable items: -.. function:: TemporaryFile(mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) +.. function:: TemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) Return a :term:`file-like object` that can be used as a temporary storage area. The file is created securely, using the same rules as :func:`mkstemp`. It will be destroyed as soon @@ -72,7 +72,7 @@ The module defines the following user-callable items: Added *errors* parameter. -.. function:: NamedTemporaryFile(mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None) +.. function:: NamedTemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None) This function operates exactly as :func:`TemporaryFile` does, except that the file is guaranteed to have a visible name in the file system (on @@ -93,7 +93,7 @@ The module defines the following user-callable items: Added *errors* parameter. -.. function:: SpooledTemporaryFile(max_size=0, mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) +.. function:: SpooledTemporaryFile(max_size=0, mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) This function operates exactly as :func:`TemporaryFile` does, except that data is spooled in memory until the file size exceeds *max_size*, or From webhook-mailer at python.org Tue Sep 15 09:58:26 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:58:26 -0000 Subject: [Python-checkins] bpo-33239: Fix default value of 'buffering' parameter in docs of tempfile.* functions (GH-21763) (GH-22226) Message-ID: https://github.com/python/cpython/commit/bf50b0e80a8a0d651af2f953b662eeadd27c7c93 commit: bf50b0e80a8a0d651af2f953b662eeadd27c7c93 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-13T19:16:39+01:00 summary: bpo-33239: Fix default value of 'buffering' parameter in docs of tempfile.* functions (GH-21763) (GH-22226) (cherry picked from commit b48389d95093c3f912549add8da339edc164bf0d) files: M Doc/library/tempfile.rst diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index 3a2b88c0cb6a2..f9421da5fe7df 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -31,7 +31,7 @@ is recommended to use keyword arguments for clarity. The module defines the following user-callable items: -.. function:: TemporaryFile(mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) +.. function:: TemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) Return a :term:`file-like object` that can be used as a temporary storage area. The file is created securely, using the same rules as :func:`mkstemp`. It will be destroyed as soon @@ -72,7 +72,7 @@ The module defines the following user-callable items: Added *errors* parameter. -.. function:: NamedTemporaryFile(mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None) +.. function:: NamedTemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None) This function operates exactly as :func:`TemporaryFile` does, except that the file is guaranteed to have a visible name in the file system (on @@ -93,7 +93,7 @@ The module defines the following user-callable items: Added *errors* parameter. -.. function:: SpooledTemporaryFile(max_size=0, mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) +.. function:: SpooledTemporaryFile(max_size=0, mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) This function operates exactly as :func:`TemporaryFile` does, except that data is spooled in memory until the file size exceeds *max_size*, or From webhook-mailer at python.org Tue Sep 15 09:58:46 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 13:58:46 -0000 Subject: [Python-checkins] bpo-33239: Fix default value of 'buffering' parameter in docs of tempfile.* functions (GH-21763) (GH-22225) Message-ID: https://github.com/python/cpython/commit/c978be283db768743d995f04414c65439a632bdd commit: c978be283db768743d995f04414c65439a632bdd branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-13T19:20:38+01:00 summary: bpo-33239: Fix default value of 'buffering' parameter in docs of tempfile.* functions (GH-21763) (GH-22225) (cherry picked from commit b48389d95093c3f912549add8da339edc164bf0d) files: M Doc/library/tempfile.rst diff --git a/Doc/library/tempfile.rst b/Doc/library/tempfile.rst index 3a2b88c0cb6a2..f9421da5fe7df 100644 --- a/Doc/library/tempfile.rst +++ b/Doc/library/tempfile.rst @@ -31,7 +31,7 @@ is recommended to use keyword arguments for clarity. The module defines the following user-callable items: -.. function:: TemporaryFile(mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) +.. function:: TemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) Return a :term:`file-like object` that can be used as a temporary storage area. The file is created securely, using the same rules as :func:`mkstemp`. It will be destroyed as soon @@ -72,7 +72,7 @@ The module defines the following user-callable items: Added *errors* parameter. -.. function:: NamedTemporaryFile(mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None) +.. function:: NamedTemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None) This function operates exactly as :func:`TemporaryFile` does, except that the file is guaranteed to have a visible name in the file system (on @@ -93,7 +93,7 @@ The module defines the following user-callable items: Added *errors* parameter. -.. function:: SpooledTemporaryFile(max_size=0, mode='w+b', buffering=None, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) +.. function:: SpooledTemporaryFile(max_size=0, mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None) This function operates exactly as :func:`TemporaryFile` does, except that data is spooled in memory until the file size exceeds *max_size*, or From webhook-mailer at python.org Tue Sep 15 10:01:12 2020 From: webhook-mailer at python.org (Zackery Spytz) Date: Tue, 15 Sep 2020 14:01:12 -0000 Subject: [Python-checkins] bpo-38967: Improve the error msg for reserved _sunder_ names in enum (GH-18370) Message-ID: https://github.com/python/cpython/commit/2ec67526a6ed3312f4fac22b527ca0ff161531b8 commit: 2ec67526a6ed3312f4fac22b527ca0ff161531b8 branch: master author: Zackery Spytz committer: GitHub date: 2020-09-13T13:27:51-07:00 summary: bpo-38967: Improve the error msg for reserved _sunder_ names in enum (GH-18370) files: M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 49b552ba0ecf6..bc24f2ae2dfc0 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -76,7 +76,8 @@ def __setitem__(self, key, value): '_order_', '_create_pseudo_member_', '_generate_next_value_', '_missing_', '_ignore_', ): - raise ValueError('_names_ are reserved for future Enum use') + raise ValueError(f'_sunder_ names, such as "{key}", are ' + 'reserved for future Enum use') if key == '_generate_next_value_': # check if members already defined as auto() if self._auto_called: diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index e7bad62406773..b18f3b38a6619 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -419,6 +419,11 @@ def red(self): green = 2 blue = 3 + def test_reserved__sunder_(self): + with self.assertRaisesRegex(ValueError, '_sunder_ names, such as ' + '"_bad_", are reserved'): + class Bad(Enum): + _bad_ = 1 def test_enum_with_value_name(self): class Huh(Enum): From webhook-mailer at python.org Tue Sep 15 10:01:27 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Tue, 15 Sep 2020 14:01:27 -0000 Subject: [Python-checkins] [3.8] bpo-37479: Enum - use correct __format__ (GH-14545) Message-ID: https://github.com/python/cpython/commit/38c8d3930eb872258a82339bcba3bce1d0e3ac2c commit: 38c8d3930eb872258a82339bcba3bce1d0e3ac2c branch: 3.8 author: Ethan Furman committer: GitHub date: 2020-09-13T13:47:43-07:00 summary: [3.8] bpo-37479: Enum - use correct __format__ (GH-14545) * bpo-37479: on Enum subclasses with mixins, __format__ uses overridden __str__. (cherry picked from commit 2f19e82fbe98ce86bcd98a176328af2808b678e8) Co-authored-by: thatneat files: A Misc/NEWS.d/next/Library/2019-07-02-12-43-57.bpo-37479.O53a5S.rst M Doc/library/enum.rst M Lib/enum.py M Lib/test/test_enum.py M Misc/ACKS diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index ff1510cdd0e41..03aa2f1b21655 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -744,9 +744,11 @@ Some rules: :meth:`__str__` and :meth:`__repr__` respectively; other codes (such as `%i` or `%h` for IntEnum) treat the enum member as its mixed-in type. 5. :ref:`Formatted string literals `, :meth:`str.format`, - and :func:`format` will use the mixed-in - type's :meth:`__format__`. If the :class:`Enum` class's :func:`str` or - :func:`repr` is desired, use the `!s` or `!r` format codes. + and :func:`format` will use the mixed-in type's :meth:`__format__` + unless :meth:`__str__` or :meth:`__format__` is overridden in the subclass, + in which case the overridden methods or :class:`Enum` methods will be used. + Use the !s and !r format codes to force usage of the :class:`Enum` class's + :meth:`__str__` and :meth:`__repr__` methods. When to use :meth:`__new__` vs. :meth:`__init__` ------------------------------------------------ diff --git a/Lib/enum.py b/Lib/enum.py index 14cc00e783915..16033b1c6fac5 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -633,8 +633,9 @@ def __format__(self, format_spec): # we can get strange results with the Enum name showing up instead of # the value - # pure Enum branch - if self._member_type_ is object: + # pure Enum branch, or branch with __str__ explicitly overridden + str_overridden = type(self).__str__ != Enum.__str__ + if self._member_type_ is object or str_overridden: cls = str val = str(self) # mix-in branch diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index a2a3c56734901..0f91f00d92dd9 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -445,12 +445,63 @@ def test_format_enum(self): self.assertEqual('{:<20}'.format(Season.SPRING), '{:<20}'.format(str(Season.SPRING))) - def test_format_enum_custom(self): + def test_str_override_enum(self): + class EnumWithStrOverrides(Enum): + one = auto() + two = auto() + + def __str__(self): + return 'Str!' + self.assertEqual(str(EnumWithStrOverrides.one), 'Str!') + self.assertEqual('{}'.format(EnumWithStrOverrides.one), 'Str!') + + def test_format_override_enum(self): + class EnumWithFormatOverride(Enum): + one = 1.0 + two = 2.0 + def __format__(self, spec): + return 'Format!!' + self.assertEqual(str(EnumWithFormatOverride.one), 'EnumWithFormatOverride.one') + self.assertEqual('{}'.format(EnumWithFormatOverride.one), 'Format!!') + + def test_str_and_format_override_enum(self): + class EnumWithStrFormatOverrides(Enum): + one = auto() + two = auto() + def __str__(self): + return 'Str!' + def __format__(self, spec): + return 'Format!' + self.assertEqual(str(EnumWithStrFormatOverrides.one), 'Str!') + self.assertEqual('{}'.format(EnumWithStrFormatOverrides.one), 'Format!') + + def test_str_override_mixin(self): + class MixinEnumWithStrOverride(float, Enum): + one = 1.0 + two = 2.0 + def __str__(self): + return 'Overridden!' + self.assertEqual(str(MixinEnumWithStrOverride.one), 'Overridden!') + self.assertEqual('{}'.format(MixinEnumWithStrOverride.one), 'Overridden!') + + def test_str_and_format_override_mixin(self): + class MixinWithStrFormatOverrides(float, Enum): + one = 1.0 + two = 2.0 + def __str__(self): + return 'Str!' + def __format__(self, spec): + return 'Format!' + self.assertEqual(str(MixinWithStrFormatOverrides.one), 'Str!') + self.assertEqual('{}'.format(MixinWithStrFormatOverrides.one), 'Format!') + + def test_format_override_mixin(self): class TestFloat(float, Enum): one = 1.0 two = 2.0 def __format__(self, spec): return 'TestFloat success!' + self.assertEqual(str(TestFloat.one), 'TestFloat.one') self.assertEqual('{}'.format(TestFloat.one), 'TestFloat success!') def assertFormatIsValue(self, spec, member): diff --git a/Misc/ACKS b/Misc/ACKS index a08e917b30765..e6d0c3ba12708 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -364,6 +364,7 @@ Ra?l Cumplido Antonio Cuni Brian Curtin Hakan Celik +Jason Curtis Paul Dagnelie Lisandro Dalcin Darren Dale diff --git a/Misc/NEWS.d/next/Library/2019-07-02-12-43-57.bpo-37479.O53a5S.rst b/Misc/NEWS.d/next/Library/2019-07-02-12-43-57.bpo-37479.O53a5S.rst new file mode 100644 index 0000000000000..cf23155c33958 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-07-02-12-43-57.bpo-37479.O53a5S.rst @@ -0,0 +1,2 @@ +When `Enum.__str__` is overridden in a derived class, the override will be +used by `Enum.__format__` regardless of whether mixin classes are present. \ No newline at end of file From webhook-mailer at python.org Tue Sep 15 10:01:42 2020 From: webhook-mailer at python.org (Emmanuel Arias) Date: Tue, 15 Sep 2020 14:01:42 -0000 Subject: [Python-checkins] bpo-41778: Change a punctuation on documentation. (GH-22229) Message-ID: https://github.com/python/cpython/commit/94bfdee25db31941b187591ae5ae9bf3ed431090 commit: 94bfdee25db31941b187591ae5ae9bf3ed431090 branch: master author: Emmanuel Arias committer: GitHub date: 2020-09-13T14:05:44-07:00 summary: bpo-41778: Change a punctuation on documentation. (GH-22229) On this paragrapah the clarification about IIS7 seems there's not connection beacuase is in other sentence. Move the punctuation to connect both the last sentence with the information in the parenthesis. I think the NEWS is not necessary here. Automerge-Triggered-By: @ericvsmith files: M Doc/library/wsgiref.rst diff --git a/Doc/library/wsgiref.rst b/Doc/library/wsgiref.rst index 1e30aa4a898c1..e92a689de0b9b 100644 --- a/Doc/library/wsgiref.rst +++ b/Doc/library/wsgiref.rst @@ -480,8 +480,8 @@ input, output, and error streams. rarely used and is not guaranteed by WSGI. On IIS<7, though, the setting can only be made on a vhost level, affecting all other script mappings, many of which break when exposed to the ``PATH_TRANSLATED`` bug. - For this reason IIS<7 is almost never deployed with the fix. (Even IIS7 - rarely uses it because there is still no UI for it.) + For this reason IIS<7 is almost never deployed with the fix (Even IIS7 + rarely uses it because there is still no UI for it.). There is no way for CGI code to tell whether the option was set, so a separate handler class is provided. It is used in the same way as From webhook-mailer at python.org Tue Sep 15 10:02:01 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 14:02:01 -0000 Subject: [Python-checkins] bpo-41778: Change a punctuation on documentation. (GH-22229) (GH-22230) Message-ID: https://github.com/python/cpython/commit/d09bead93990eed26ecb8fcd02a8a3f6e8fa73b7 commit: d09bead93990eed26ecb8fcd02a8a3f6e8fa73b7 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-13T17:21:49-04:00 summary: bpo-41778: Change a punctuation on documentation. (GH-22229) (GH-22230) On this paragrapah the clarification about IIS7 seems there's not connection beacuase is in other sentence. Move the punctuation to connect both the last sentence with the information in the parenthesis. I think the NEWS is not necessary here. Automerge-Triggered-By: @ericvsmith (cherry picked from commit 94bfdee25db31941b187591ae5ae9bf3ed431090) Co-authored-by: Emmanuel Arias files: M Doc/library/wsgiref.rst diff --git a/Doc/library/wsgiref.rst b/Doc/library/wsgiref.rst index 1e30aa4a898c1..e92a689de0b9b 100644 --- a/Doc/library/wsgiref.rst +++ b/Doc/library/wsgiref.rst @@ -480,8 +480,8 @@ input, output, and error streams. rarely used and is not guaranteed by WSGI. On IIS<7, though, the setting can only be made on a vhost level, affecting all other script mappings, many of which break when exposed to the ``PATH_TRANSLATED`` bug. - For this reason IIS<7 is almost never deployed with the fix. (Even IIS7 - rarely uses it because there is still no UI for it.) + For this reason IIS<7 is almost never deployed with the fix (Even IIS7 + rarely uses it because there is still no UI for it.). There is no way for CGI code to tell whether the option was set, so a separate handler class is provided. It is used in the same way as From webhook-mailer at python.org Tue Sep 15 10:02:19 2020 From: webhook-mailer at python.org (Emmanuel Arias) Date: Tue, 15 Sep 2020 14:02:19 -0000 Subject: [Python-checkins] [3.9] bpo-41778: Change a punctuation on documentation. (GH-22229) (GH-22232) Message-ID: https://github.com/python/cpython/commit/b3d11abcbc0fcf794f9be29aa78bb3d100a54960 commit: b3d11abcbc0fcf794f9be29aa78bb3d100a54960 branch: 3.9 author: Emmanuel Arias committer: GitHub date: 2020-09-13T17:38:01-04:00 summary: [3.9] bpo-41778: Change a punctuation on documentation. (GH-22229) (GH-22232) On this paragrapah the clarification about IIS7 seems there's not connection beacuase is in other sentence. Move the punctuation to connect both the last sentence with the information in the parenthesis. I think the NEWS is not necessary here. Automerge-Triggered-By: @ericvsmith (cherry picked from commit 94bfdee25db31941b187591ae5ae9bf3ed431090) Co-authored-by: Emmanuel Arias files: M Doc/library/wsgiref.rst diff --git a/Doc/library/wsgiref.rst b/Doc/library/wsgiref.rst index 1e30aa4a898c1..e92a689de0b9b 100644 --- a/Doc/library/wsgiref.rst +++ b/Doc/library/wsgiref.rst @@ -480,8 +480,8 @@ input, output, and error streams. rarely used and is not guaranteed by WSGI. On IIS<7, though, the setting can only be made on a vhost level, affecting all other script mappings, many of which break when exposed to the ``PATH_TRANSLATED`` bug. - For this reason IIS<7 is almost never deployed with the fix. (Even IIS7 - rarely uses it because there is still no UI for it.) + For this reason IIS<7 is almost never deployed with the fix (Even IIS7 + rarely uses it because there is still no UI for it.). There is no way for CGI code to tell whether the option was set, so a separate handler class is provided. It is used in the same way as From webhook-mailer at python.org Tue Sep 15 10:02:32 2020 From: webhook-mailer at python.org (Ned Deily) Date: Tue, 15 Sep 2020 14:02:32 -0000 Subject: [Python-checkins] bpo-39883: Update macOS installer copy of LICENSE. (GH-22235) Message-ID: https://github.com/python/cpython/commit/7dbbea75cec27a48b68cc07c23f3f317cacf4a16 commit: 7dbbea75cec27a48b68cc07c23f3f317cacf4a16 branch: master author: Ned Deily committer: GitHub date: 2020-09-14T01:18:01-04:00 summary: bpo-39883: Update macOS installer copy of LICENSE. (GH-22235) files: M Mac/BuildScript/resources/License.rtf diff --git a/Mac/BuildScript/resources/License.rtf b/Mac/BuildScript/resources/License.rtf index 25d53386da01a..1dfdc1edc17fa 100644 --- a/Mac/BuildScript/resources/License.rtf +++ b/Mac/BuildScript/resources/License.rtf @@ -1,9 +1,9 @@ -{\rtf1\ansi\ansicpg1252\cocoartf2511 +{\rtf1\ansi\ansicpg1252\cocoartf2513 \cocoatextscaling0\cocoaplatform0{\fonttbl\f0\fswiss\fcharset0 Helvetica-Bold;\f1\fswiss\fcharset0 Helvetica;\f2\fmodern\fcharset0 CourierNewPS-BoldMT; \f3\fmodern\fcharset0 CourierNewPSMT;} {\colortbl;\red255\green255\blue255;} {\*\expandedcolortbl;;} -\margl1440\margr1440\vieww14620\viewh13380\viewkind0 +\margl1440\margr1440\vieww18500\viewh13520\viewkind0 \pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 \f0\b\fs36 \cf0 \ul \ulc0 HISTORY AND LICENSE\ @@ -47,8 +47,17 @@ Thanks to the many outside volunteers who have worked under Guido's direction to \ \f0\b \ul TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON\ +\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 -\f1\b0 \ulnone \ +\f1\b0 \cf0 \ulnone Python software and documentation are licensed under the Python Software Foundation License Version 2.\ +\ +Starting with Python 3.8.6, examples, recipes, and other code in the documentation are dual licensed under the PSF License Version 2 and the Zero-Clause BSD license.\ +\ +Some software incorporated into Python is under different licenses. The licenses are listed with code falling under that license.\ +\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 +\cf0 \ \f0\b PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2\ @@ -125,6 +134,18 @@ Permission to use, copy, modify, and distribute this software and its documentat STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\ \ \ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 + +\f0\b \cf0 ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON DOCUMENTATION\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 + +\f1\b0 \cf0 \ +Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted.\ +\ +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 +\cf0 \ +\ \f0\b \ul LICENSES AND ACKNOWLEDGEMENTS FOR INCORPORATED SOFTWARE\ From webhook-mailer at python.org Tue Sep 15 10:02:52 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 14:02:52 -0000 Subject: [Python-checkins] bpo-39883: Update macOS installer copy of LICENSE. (GH-22235) Message-ID: https://github.com/python/cpython/commit/3c618d0a073ea6863d1b4e2616e284308fbc12fc commit: 3c618d0a073ea6863d1b4e2616e284308fbc12fc branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-13T22:38:20-07:00 summary: bpo-39883: Update macOS installer copy of LICENSE. (GH-22235) (cherry picked from commit 7dbbea75cec27a48b68cc07c23f3f317cacf4a16) Co-authored-by: Ned Deily files: M Mac/BuildScript/resources/License.rtf diff --git a/Mac/BuildScript/resources/License.rtf b/Mac/BuildScript/resources/License.rtf index 25d53386da01a..1dfdc1edc17fa 100644 --- a/Mac/BuildScript/resources/License.rtf +++ b/Mac/BuildScript/resources/License.rtf @@ -1,9 +1,9 @@ -{\rtf1\ansi\ansicpg1252\cocoartf2511 +{\rtf1\ansi\ansicpg1252\cocoartf2513 \cocoatextscaling0\cocoaplatform0{\fonttbl\f0\fswiss\fcharset0 Helvetica-Bold;\f1\fswiss\fcharset0 Helvetica;\f2\fmodern\fcharset0 CourierNewPS-BoldMT; \f3\fmodern\fcharset0 CourierNewPSMT;} {\colortbl;\red255\green255\blue255;} {\*\expandedcolortbl;;} -\margl1440\margr1440\vieww14620\viewh13380\viewkind0 +\margl1440\margr1440\vieww18500\viewh13520\viewkind0 \pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 \f0\b\fs36 \cf0 \ul \ulc0 HISTORY AND LICENSE\ @@ -47,8 +47,17 @@ Thanks to the many outside volunteers who have worked under Guido's direction to \ \f0\b \ul TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON\ +\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 -\f1\b0 \ulnone \ +\f1\b0 \cf0 \ulnone Python software and documentation are licensed under the Python Software Foundation License Version 2.\ +\ +Starting with Python 3.8.6, examples, recipes, and other code in the documentation are dual licensed under the PSF License Version 2 and the Zero-Clause BSD license.\ +\ +Some software incorporated into Python is under different licenses. The licenses are listed with code falling under that license.\ +\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 +\cf0 \ \f0\b PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2\ @@ -125,6 +134,18 @@ Permission to use, copy, modify, and distribute this software and its documentat STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\ \ \ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 + +\f0\b \cf0 ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON DOCUMENTATION\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 + +\f1\b0 \cf0 \ +Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted.\ +\ +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 +\cf0 \ +\ \f0\b \ul LICENSES AND ACKNOWLEDGEMENTS FOR INCORPORATED SOFTWARE\ From webhook-mailer at python.org Tue Sep 15 10:03:13 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 14:03:13 -0000 Subject: [Python-checkins] bpo-39883: Update macOS installer copy of LICENSE. (GH-22235) (GH-22236) Message-ID: https://github.com/python/cpython/commit/bf7d4d039c46232262a0f736f12761b085a6e7a8 commit: bf7d4d039c46232262a0f736f12761b085a6e7a8 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-14T01:59:58-04:00 summary: bpo-39883: Update macOS installer copy of LICENSE. (GH-22235) (GH-22236) (cherry picked from commit 7dbbea75cec27a48b68cc07c23f3f317cacf4a16) Co-authored-by: Ned Deily files: M Mac/BuildScript/resources/License.rtf diff --git a/Mac/BuildScript/resources/License.rtf b/Mac/BuildScript/resources/License.rtf index 25d53386da01a..1dfdc1edc17fa 100644 --- a/Mac/BuildScript/resources/License.rtf +++ b/Mac/BuildScript/resources/License.rtf @@ -1,9 +1,9 @@ -{\rtf1\ansi\ansicpg1252\cocoartf2511 +{\rtf1\ansi\ansicpg1252\cocoartf2513 \cocoatextscaling0\cocoaplatform0{\fonttbl\f0\fswiss\fcharset0 Helvetica-Bold;\f1\fswiss\fcharset0 Helvetica;\f2\fmodern\fcharset0 CourierNewPS-BoldMT; \f3\fmodern\fcharset0 CourierNewPSMT;} {\colortbl;\red255\green255\blue255;} {\*\expandedcolortbl;;} -\margl1440\margr1440\vieww14620\viewh13380\viewkind0 +\margl1440\margr1440\vieww18500\viewh13520\viewkind0 \pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 \f0\b\fs36 \cf0 \ul \ulc0 HISTORY AND LICENSE\ @@ -47,8 +47,17 @@ Thanks to the many outside volunteers who have worked under Guido's direction to \ \f0\b \ul TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON\ +\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 -\f1\b0 \ulnone \ +\f1\b0 \cf0 \ulnone Python software and documentation are licensed under the Python Software Foundation License Version 2.\ +\ +Starting with Python 3.8.6, examples, recipes, and other code in the documentation are dual licensed under the PSF License Version 2 and the Zero-Clause BSD license.\ +\ +Some software incorporated into Python is under different licenses. The licenses are listed with code falling under that license.\ +\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 +\cf0 \ \f0\b PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2\ @@ -125,6 +134,18 @@ Permission to use, copy, modify, and distribute this software and its documentat STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\ \ \ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 + +\f0\b \cf0 ZERO-CLAUSE BSD LICENSE FOR CODE IN THE PYTHON DOCUMENTATION\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 + +\f1\b0 \cf0 \ +Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted.\ +\ +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\ +\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0 +\cf0 \ +\ \f0\b \ul LICENSES AND ACKNOWLEDGEMENTS FOR INCORPORATED SOFTWARE\ From webhook-mailer at python.org Tue Sep 15 10:03:30 2020 From: webhook-mailer at python.org (Raymond Hettinger) Date: Tue, 15 Sep 2020 14:03:30 -0000 Subject: [Python-checkins] bpo-41513: Add docs and tests for hypot() (GH-22238) Message-ID: https://github.com/python/cpython/commit/457d4e97de0369bc786e363cb53c7ef3276fdfcd commit: 457d4e97de0369bc786e363cb53c7ef3276fdfcd branch: master author: Raymond Hettinger committer: GitHub date: 2020-09-13T23:33:41-07:00 summary: bpo-41513: Add docs and tests for hypot() (GH-22238) files: M Doc/library/math.rst M Lib/test/test_math.py M Modules/mathmodule.c diff --git a/Doc/library/math.rst b/Doc/library/math.rst index 6ec1feee35a6d..bbf64643ff59f 100644 --- a/Doc/library/math.rst +++ b/Doc/library/math.rst @@ -481,6 +481,11 @@ Trigonometric functions Added support for n-dimensional points. Formerly, only the two dimensional case was supported. + .. versionchanged:: 3.10 + Improved the algorithm's accuracy so that the maximum error is + under 1 ulp (unit in the last place). More typically, the result + is almost always correctly rounded to within 1/2 ulp. + .. function:: sin(x) diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py index 4d62eb1b11993..bbaa533c8489e 100644 --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -803,6 +803,57 @@ def testHypot(self): scale = FLOAT_MIN / 2.0 ** exp self.assertEqual(math.hypot(4*scale, 3*scale), 5*scale) + def testHypotAccuracy(self): + # Verify improved accuracy in cases that were known to be inaccurate. + + hypot = math.hypot + Decimal = decimal.Decimal + high_precision = decimal.Context(prec=500) + + for hx, hy in [ + # Cases with a 1 ulp error in Python 3.7 compiled with Clang + ('0x1.10e89518dca48p+29', '0x1.1970f7565b7efp+30'), + ('0x1.10106eb4b44a2p+29', '0x1.ef0596cdc97f8p+29'), + ('0x1.459c058e20bb7p+30', '0x1.993ca009b9178p+29'), + ('0x1.378371ae67c0cp+30', '0x1.fbe6619854b4cp+29'), + ('0x1.f4cd0574fb97ap+29', '0x1.50fe31669340ep+30'), + ('0x1.494b2cdd3d446p+29', '0x1.212a5367b4c7cp+29'), + ('0x1.f84e649f1e46dp+29', '0x1.1fa56bef8eec4p+30'), + ('0x1.2e817edd3d6fap+30', '0x1.eb0814f1e9602p+29'), + ('0x1.0d3a6e3d04245p+29', '0x1.32a62fea52352p+30'), + ('0x1.888e19611bfc5p+29', '0x1.52b8e70b24353p+29'), + + # Cases with 2 ulp error in Python 3.8 + ('0x1.538816d48a13fp+29', '0x1.7967c5ca43e16p+29'), + ('0x1.57b47b7234530p+29', '0x1.74e2c7040e772p+29'), + ('0x1.821b685e9b168p+30', '0x1.677dc1c1e3dc6p+29'), + ('0x1.9e8247f67097bp+29', '0x1.24bd2dc4f4baep+29'), + ('0x1.b73b59e0cb5f9p+29', '0x1.da899ab784a97p+28'), + ('0x1.94a8d2842a7cfp+30', '0x1.326a51d4d8d8ap+30'), + ('0x1.e930b9cd99035p+29', '0x1.5a1030e18dff9p+30'), + ('0x1.1592bbb0e4690p+29', '0x1.a9c337b33fb9ap+29'), + ('0x1.1243a50751fd4p+29', '0x1.a5a10175622d9p+29'), + ('0x1.57a8596e74722p+30', '0x1.42d1af9d04da9p+30'), + + # Cases with 1 ulp error in version fff3c28052e6b0750d6218e00acacd2fded4991a + ('0x1.ee7dbd9565899p+29', '0x1.7ab4d6fc6e4b4p+29'), + ('0x1.5c6bfbec5c4dcp+30', '0x1.02511184b4970p+30'), + ('0x1.59dcebba995cap+30', '0x1.50ca7e7c38854p+29'), + ('0x1.768cdd94cf5aap+29', '0x1.9cfdc5571d38ep+29'), + ('0x1.dcf137d60262ep+29', '0x1.1101621990b3ep+30'), + ('0x1.3a2d006e288b0p+30', '0x1.e9a240914326cp+29'), + ('0x1.62a32f7f53c61p+29', '0x1.47eb6cd72684fp+29'), + ('0x1.d3bcb60748ef2p+29', '0x1.3f13c4056312cp+30'), + ('0x1.282bdb82f17f3p+30', '0x1.640ba4c4eed3ap+30'), + ('0x1.89d8c423ea0c6p+29', '0x1.d35dcfe902bc3p+29'), + ]: + with self.subTest(hx=hx, hy=hy): + x = float.fromhex(hx) + y = float.fromhex(hy) + with decimal.localcontext(high_precision): + z = float((Decimal(x)**2 + Decimal(y)**2).sqrt()) + self.assertEqual(hypot(x, y), z) + def testDist(self): from decimal import Decimal as D from fractions import Fraction as F diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index 29137ae91a22f..ecd291ecd1b4b 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -2429,7 +2429,7 @@ magnitude. We avoid this cost by arranging the calculation so that fabs(csum) is always as large as fabs(x). To establish the invariant, *csum* is initialized to 1.0 which is -always larger than x**2 after scaling or division by *max*. +always larger than x**2 after scaling or after division by *max*. After the loop is finished, the initial 1.0 is subtracted out for a net zero effect on the final sum. Since *csum* will be greater than 1.0, the subtraction of 1.0 will not cause fractional digits to be @@ -2458,7 +2458,7 @@ Since lo**2 is less than 1/2 ulp(csum), we have csum+lo*lo == csum. To minimize loss of information during the accumulation of fractional values, each term has a separate accumulator. This also breaks up sequential dependencies in the inner loop so the CPU can maximize -floating point throughput. [5] On a 2.6 GHz Haswell, adding one +floating point throughput. [4] On a 2.6 GHz Haswell, adding one dimension has an incremental cost of only 5ns -- for example when moving from hypot(x,y) to hypot(x,y,z). @@ -2470,7 +2470,7 @@ The differential correction starts with a value *x* that is the difference between the square of *h*, the possibly inaccurately rounded square root, and the accurately computed sum of squares. The correction is the first order term of the Maclaurin series -expansion of sqrt(h**2 + x) == h + x/(2*h) + O(x**2). [4] +expansion of sqrt(h**2 + x) == h + x/(2*h) + O(x**2). [5] Essentially, this differential correction is equivalent to one refinement step in Newton's divide-and-average square root @@ -2492,10 +2492,10 @@ exactly equal) was verified for 1 billion random inputs with n=5. [7] 1. Veltkamp-Dekker splitting: http://csclub.uwaterloo.ca/~pbarfuss/dekker1971.pdf 2. Compensated summation: http://www.ti3.tu-harburg.de/paper/rump/Ru08b.pdf 3. Square root differential correction: https://arxiv.org/pdf/1904.09481.pdf -4. https://www.wolframalpha.com/input/?i=Maclaurin+series+sqrt%28h**2+%2B+x%29+at+x%3D0 -5. https://bugs.python.org/file49439/hypot.png -6. https://bugs.python.org/file49435/best_frac.py -7. https://bugs.python.org/file49448/test_hypot_commutativity.py +4. Data dependency graph: https://bugs.python.org/file49439/hypot.png +5. https://www.wolframalpha.com/input/?i=Maclaurin+series+sqrt%28h**2+%2B+x%29+at+x%3D0 +6. Analysis of internal accuracy: https://bugs.python.org/file49435/best_frac.py +7. Commutativity test: https://bugs.python.org/file49448/test_hypot_commutativity.py */ From webhook-mailer at python.org Tue Sep 15 10:03:48 2020 From: webhook-mailer at python.org (abdo) Date: Tue, 15 Sep 2020 14:03:48 -0000 Subject: [Python-checkins] Fix a typo in locale Docs (#22233) Message-ID: https://github.com/python/cpython/commit/efcbe786cf9e6f99448f364f1b5e19e8fc10444f commit: efcbe786cf9e6f99448f364f1b5e19e8fc10444f branch: master author: abdo committer: GitHub date: 2020-09-14T20:36:34+03:00 summary: Fix a typo in locale Docs (#22233) files: M Doc/library/locale.rst diff --git a/Doc/library/locale.rst b/Doc/library/locale.rst index bf57a08355916..678148a0dda29 100644 --- a/Doc/library/locale.rst +++ b/Doc/library/locale.rst @@ -508,7 +508,7 @@ Background, details, hints, tips and caveats -------------------------------------------- The C standard defines the locale as a program-wide property that may be -relatively expensive to change. On top of that, some implementation are broken +relatively expensive to change. On top of that, some implementations are broken in such a way that frequent locale changes may cause core dumps. This makes the locale somewhat painful to use correctly. From webhook-mailer at python.org Tue Sep 15 10:04:04 2020 From: webhook-mailer at python.org (Zackery Spytz) Date: Tue, 15 Sep 2020 14:04:04 -0000 Subject: [Python-checkins] bpo-41646: Mention path-like objects support in the docs for shutil.copy() (GH-22208) Message-ID: https://github.com/python/cpython/commit/8f2b991eef062c22c429a96983c78be007180b66 commit: 8f2b991eef062c22c429a96983c78be007180b66 branch: master author: Zackery Spytz committer: GitHub date: 2020-09-14T20:28:46+01:00 summary: bpo-41646: Mention path-like objects support in the docs for shutil.copy() (GH-22208) files: M Doc/library/shutil.rst diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst index 1b094aeb9ca3d..ecc3309ed5cc0 100644 --- a/Doc/library/shutil.rst +++ b/Doc/library/shutil.rst @@ -158,9 +158,9 @@ Directory and files operations .. function:: copy(src, dst, *, follow_symlinks=True) Copies the file *src* to the file or directory *dst*. *src* and *dst* - should be strings. If *dst* specifies a directory, the file will be - copied into *dst* using the base filename from *src*. Returns the - path to the newly created file. + should be :term:`path-like objects ` or strings. If + *dst* specifies a directory, the file will be copied into *dst* using the + base filename from *src*. Returns the path to the newly created file. If *follow_symlinks* is false, and *src* is a symbolic link, *dst* will be created as a symbolic link. If *follow_symlinks* From webhook-mailer at python.org Tue Sep 15 10:04:23 2020 From: webhook-mailer at python.org (=?utf-8?q?V=C3=A1clav_Slav=C3=ADk?=) Date: Tue, 15 Sep 2020 14:04:23 -0000 Subject: [Python-checkins] bpo-41744: Package python.props with correct name in NuGet package (GH-22154) Message-ID: https://github.com/python/cpython/commit/7c11a9accabe3720940f334eb1226bb7bb9179c7 commit: 7c11a9accabe3720940f334eb1226bb7bb9179c7 branch: master author: V?clav Slav?k committer: GitHub date: 2020-09-14T20:30:15+01:00 summary: bpo-41744: Package python.props with correct name in NuGet package (GH-22154) NuGet automatically includes .props file from the build directory in the target using the package, but only if the .props file has the correct name: it must be $(id).props Rename python.props correspondingly in all the nuspec variants. Also keep python.props as it were for backward compatibility. files: A Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst M Tools/nuget/python.nuspec M Tools/nuget/pythonarm32.nuspec M Tools/nuget/pythondaily.nuspec M Tools/nuget/pythonx86.nuspec diff --git a/Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst b/Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst new file mode 100644 index 0000000000000..6106d6604c7dd --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst @@ -0,0 +1 @@ +Fixes automatic import of props file when using the Nuget package. \ No newline at end of file diff --git a/Tools/nuget/python.nuspec b/Tools/nuget/python.nuspec index 8f98e808916a3..2da5f2037eb23 100644 --- a/Tools/nuget/python.nuspec +++ b/Tools/nuget/python.nuspec @@ -13,6 +13,6 @@ - + diff --git a/Tools/nuget/pythonarm32.nuspec b/Tools/nuget/pythonarm32.nuspec index 273d79a0312bd..2d197931edb32 100644 --- a/Tools/nuget/pythonarm32.nuspec +++ b/Tools/nuget/pythonarm32.nuspec @@ -14,6 +14,7 @@ - + + diff --git a/Tools/nuget/pythondaily.nuspec b/Tools/nuget/pythondaily.nuspec index 5cf55806ddfb3..7df1983f42a6f 100644 --- a/Tools/nuget/pythondaily.nuspec +++ b/Tools/nuget/pythondaily.nuspec @@ -13,6 +13,7 @@ - + + diff --git a/Tools/nuget/pythonx86.nuspec b/Tools/nuget/pythonx86.nuspec index 27ef67e7f5d36..ea878ba0bf390 100644 --- a/Tools/nuget/pythonx86.nuspec +++ b/Tools/nuget/pythonx86.nuspec @@ -13,6 +13,7 @@ - + + From webhook-mailer at python.org Tue Sep 15 10:05:00 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 14:05:00 -0000 Subject: [Python-checkins] bpo-41744: Package python.props with correct name in NuGet package (GH-22154) Message-ID: https://github.com/python/cpython/commit/b007a9be6fab6d55deae070e5eaef9755794e169 commit: b007a9be6fab6d55deae070e5eaef9755794e169 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-14T12:47:40-07:00 summary: bpo-41744: Package python.props with correct name in NuGet package (GH-22154) NuGet automatically includes .props file from the build directory in the target using the package, but only if the .props file has the correct name: it must be $(id).props Rename python.props correspondingly in all the nuspec variants. Also keep python.props as it were for backward compatibility. (cherry picked from commit 7c11a9accabe3720940f334eb1226bb7bb9179c7) Co-authored-by: V?clav Slav?k files: A Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst M Tools/nuget/python.nuspec M Tools/nuget/pythonarm32.nuspec M Tools/nuget/pythondaily.nuspec M Tools/nuget/pythonx86.nuspec diff --git a/Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst b/Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst new file mode 100644 index 0000000000000..6106d6604c7dd --- /dev/null +++ b/Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst @@ -0,0 +1 @@ +Fixes automatic import of props file when using the Nuget package. \ No newline at end of file diff --git a/Tools/nuget/python.nuspec b/Tools/nuget/python.nuspec index 8f98e808916a3..2da5f2037eb23 100644 --- a/Tools/nuget/python.nuspec +++ b/Tools/nuget/python.nuspec @@ -13,6 +13,6 @@ - + diff --git a/Tools/nuget/pythonarm32.nuspec b/Tools/nuget/pythonarm32.nuspec index 273d79a0312bd..2d197931edb32 100644 --- a/Tools/nuget/pythonarm32.nuspec +++ b/Tools/nuget/pythonarm32.nuspec @@ -14,6 +14,7 @@ - + + diff --git a/Tools/nuget/pythondaily.nuspec b/Tools/nuget/pythondaily.nuspec index 5cf55806ddfb3..7df1983f42a6f 100644 --- a/Tools/nuget/pythondaily.nuspec +++ b/Tools/nuget/pythondaily.nuspec @@ -13,6 +13,7 @@ - + + diff --git a/Tools/nuget/pythonx86.nuspec b/Tools/nuget/pythonx86.nuspec index 27ef67e7f5d36..ea878ba0bf390 100644 --- a/Tools/nuget/pythonx86.nuspec +++ b/Tools/nuget/pythonx86.nuspec @@ -13,6 +13,7 @@ - + + From webhook-mailer at python.org Tue Sep 15 10:05:16 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Tue, 15 Sep 2020 14:05:16 -0000 Subject: [Python-checkins] bpo-40721: add note about enum member name case (GH-22231) Message-ID: https://github.com/python/cpython/commit/542e1df2b018ee7068dba8076f2d6e84efd6e144 commit: 542e1df2b018ee7068dba8076f2d6e84efd6e144 branch: master author: Ethan Furman committer: GitHub date: 2020-09-14T13:32:44-07:00 summary: bpo-40721: add note about enum member name case (GH-22231) * UPPER_CASE preferred as enum members are constants files: M Doc/library/enum.rst diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 00bfb260c0204..32e8bbf950927 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -19,6 +19,12 @@ An enumeration is a set of symbolic names (members) bound to unique, constant values. Within an enumeration, the members can be compared by identity, and the enumeration itself can be iterated over. +.. note:: Case of Enum Members + + Because Enums are used to represent constants we recommend using + UPPER_CASE names for enum members, and will be using that style + in our examples. + Module Contents --------------- From webhook-mailer at python.org Tue Sep 15 10:05:38 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 14:05:38 -0000 Subject: [Python-checkins] bpo-40721: add note about enum member name case (GH-22231) Message-ID: https://github.com/python/cpython/commit/624cc10ee4aae513be9f2812f6f5621b6f32f17c commit: 624cc10ee4aae513be9f2812f6f5621b6f32f17c branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-14T13:42:38-07:00 summary: bpo-40721: add note about enum member name case (GH-22231) * UPPER_CASE preferred as enum members are constants (cherry picked from commit 542e1df2b018ee7068dba8076f2d6e84efd6e144) Co-authored-by: Ethan Furman files: M Doc/library/enum.rst diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 03aa2f1b21655..8753416a27416 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -19,6 +19,12 @@ An enumeration is a set of symbolic names (members) bound to unique, constant values. Within an enumeration, the members can be compared by identity, and the enumeration itself can be iterated over. +.. note:: Case of Enum Members + + Because Enums are used to represent constants we recommend using + UPPER_CASE names for enum members, and will be using that style + in our examples. + Module Contents --------------- From webhook-mailer at python.org Tue Sep 15 10:06:32 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 14:06:32 -0000 Subject: [Python-checkins] bpo-40721: add note about enum member name case (GH-22231) Message-ID: https://github.com/python/cpython/commit/b502c7618d710d31517a7ee6a72890d0963e357a commit: b502c7618d710d31517a7ee6a72890d0963e357a branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-14T13:49:29-07:00 summary: bpo-40721: add note about enum member name case (GH-22231) * UPPER_CASE preferred as enum members are constants (cherry picked from commit 542e1df2b018ee7068dba8076f2d6e84efd6e144) Co-authored-by: Ethan Furman files: M Doc/library/enum.rst diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 382db11a72ba6..5f0762edc2590 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -19,6 +19,12 @@ An enumeration is a set of symbolic names (members) bound to unique, constant values. Within an enumeration, the members can be compared by identity, and the enumeration itself can be iterated over. +.. note:: Case of Enum Members + + Because Enums are used to represent constants we recommend using + UPPER_CASE names for enum members, and will be using that style + in our examples. + Module Contents --------------- From webhook-mailer at python.org Tue Sep 15 12:03:41 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 15 Sep 2020 16:03:41 -0000 Subject: [Python-checkins] bpo-41631: _ast module uses again a global state (#21961) Message-ID: https://github.com/python/cpython/commit/e5fbe0cbd4be99ced5f000ad382208ad2a561c90 commit: e5fbe0cbd4be99ced5f000ad382208ad2a561c90 branch: master author: Victor Stinner committer: GitHub date: 2020-09-15T18:03:34+02:00 summary: bpo-41631: _ast module uses again a global state (#21961) Partially revert commit ac46eb4ad6662cf6d771b20d8963658b2186c48c: "bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)". Using a module state per module instance is causing subtle practical problems. For example, the Mercurial project replaces the __import__() function to implement lazy import, whereas Python expected that "import _ast" always return a fully initialized _ast module. Add _PyAST_Fini() to clear the state at exit. The _ast module has no state (set _astmodule.m_size to 0). Remove astmodule_traverse(), astmodule_clear() and astmodule_free() functions. files: A Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst M Include/internal/pycore_pylifecycle.h M Lib/ast.py M Lib/test/test_ast.py M Parser/asdl_c.py M Python/Python-ast.c M Python/pylifecycle.c diff --git a/Include/internal/pycore_pylifecycle.h b/Include/internal/pycore_pylifecycle.h index 22def3dbc8b66..6d84e37232b30 100644 --- a/Include/internal/pycore_pylifecycle.h +++ b/Include/internal/pycore_pylifecycle.h @@ -84,6 +84,7 @@ extern void _PyFaulthandler_Fini(void); extern void _PyHash_Fini(void); extern void _PyTraceMalloc_Fini(void); extern void _PyWarnings_Fini(PyInterpreterState *interp); +extern void _PyAST_Fini(PyThreadState *tstate); extern PyStatus _PyGILState_Init(PyThreadState *tstate); extern void _PyGILState_Fini(PyThreadState *tstate); diff --git a/Lib/ast.py b/Lib/ast.py index 65ebd0100de00..d860917f4d03a 100644 --- a/Lib/ast.py +++ b/Lib/ast.py @@ -497,18 +497,20 @@ def generic_visit(self, node): return node -# The following code is for backward compatibility. -# It will be removed in future. +# If the ast module is loaded more than once, only add deprecated methods once +if not hasattr(Constant, 'n'): + # The following code is for backward compatibility. + # It will be removed in future. -def _getter(self): - """Deprecated. Use value instead.""" - return self.value + def _getter(self): + """Deprecated. Use value instead.""" + return self.value -def _setter(self, value): - self.value = value + def _setter(self, value): + self.value = value -Constant.n = property(_getter, _setter) -Constant.s = property(_getter, _setter) + Constant.n = property(_getter, _setter) + Constant.s = property(_getter, _setter) class _ABC(type): @@ -600,14 +602,19 @@ class ExtSlice(slice): def __new__(cls, dims=(), **kwargs): return Tuple(list(dims), Load(), **kwargs) -def _dims_getter(self): - """Deprecated. Use elts instead.""" - return self.elts +# If the ast module is loaded more than once, only add deprecated methods once +if not hasattr(Tuple, 'dims'): + # The following code is for backward compatibility. + # It will be removed in future. -def _dims_setter(self, value): - self.elts = value + def _dims_getter(self): + """Deprecated. Use elts instead.""" + return self.elts -Tuple.dims = property(_dims_getter, _dims_setter) + def _dims_setter(self, value): + self.elts = value + + Tuple.dims = property(_dims_getter, _dims_setter) class Suite(mod): """Deprecated AST node class. Unused in Python 3.""" diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py index f5aef61ec6f7c..5f57ce8724482 100644 --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -1,7 +1,9 @@ import ast +import builtins import dis import os import sys +import types import unittest import warnings import weakref @@ -1945,6 +1947,88 @@ def visit_Ellipsis(self, node): ]) + at support.cpython_only +class ModuleStateTests(unittest.TestCase): + # bpo-41194, bpo-41261, bpo-41631: The _ast module uses a global state. + + def check_ast_module(self): + # Check that the _ast module still works as expected + code = 'x + 1' + filename = '' + mode = 'eval' + + # Create _ast.AST subclasses instances + ast_tree = compile(code, filename, mode, flags=ast.PyCF_ONLY_AST) + + # Call PyAST_Check() + code = compile(ast_tree, filename, mode) + self.assertIsInstance(code, types.CodeType) + + def test_reload_module(self): + # bpo-41194: Importing the _ast module twice must not crash. + with support.swap_item(sys.modules, '_ast', None): + del sys.modules['_ast'] + import _ast as ast1 + + del sys.modules['_ast'] + import _ast as ast2 + + self.check_ast_module() + + # Unloading the two _ast module instances must not crash. + del ast1 + del ast2 + support.gc_collect() + + self.check_ast_module() + + def test_sys_modules(self): + # bpo-41631: Test reproducing a Mercurial crash when PyAST_Check() + # imported the _ast module internally. + lazy_mod = object() + + def my_import(name, *args, **kw): + sys.modules[name] = lazy_mod + return lazy_mod + + with support.swap_item(sys.modules, '_ast', None): + del sys.modules['_ast'] + + with support.swap_attr(builtins, '__import__', my_import): + # Test that compile() does not import the _ast module + self.check_ast_module() + self.assertNotIn('_ast', sys.modules) + + # Sanity check of the test itself + import _ast + self.assertIs(_ast, lazy_mod) + + def test_subinterpreter(self): + # bpo-41631: Importing and using the _ast module in a subinterpreter + # must not crash. + code = dedent(''' + import _ast + import ast + import gc + import sys + import types + + # Create _ast.AST subclasses instances and call PyAST_Check() + ast_tree = compile('x+1', '', 'eval', + flags=ast.PyCF_ONLY_AST) + code = compile(ast_tree, 'string', 'eval') + if not isinstance(code, types.CodeType): + raise AssertionError + + # Unloading the _ast module must not crash. + del ast, _ast + del sys.modules['ast'], sys.modules['_ast'] + gc.collect() + ''') + res = support.run_in_subinterp(code) + self.assertEqual(res, 0) + + def main(): if __name__ != '__main__': return diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst new file mode 100644 index 0000000000000..68bb51024d9e7 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst @@ -0,0 +1,5 @@ +The ``_ast`` module uses again a global state. Using a module state per module +instance is causing subtle practical problems. For example, the Mercurial +project replaces the ``__import__()`` function to implement lazy import, +whereas Python expected that ``import _ast`` always return a fully initialized +``_ast`` module. diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py index 6fe44b99f793b..0c053393d688b 100755 --- a/Parser/asdl_c.py +++ b/Parser/asdl_c.py @@ -1089,11 +1089,9 @@ def visitModule(self, mod): static struct PyModuleDef _astmodule = { PyModuleDef_HEAD_INIT, .m_name = "_ast", - .m_size = sizeof(astmodulestate), + // The _ast module uses a global state (global_ast_state). + .m_size = 0, .m_slots = astmodule_slots, - .m_traverse = astmodule_traverse, - .m_clear = astmodule_clear, - .m_free = astmodule_free, }; PyMODINIT_FUNC @@ -1374,59 +1372,40 @@ def generate_module_def(f, mod): f.write(' PyObject *' + s + ';\n') f.write('} astmodulestate;\n\n') f.write(""" -static astmodulestate* -get_ast_state(PyObject *module) -{ - void *state = PyModule_GetState(module); - assert(state != NULL); - return (astmodulestate*)state; -} +// Forward declaration +static int init_types(astmodulestate *state); + +// bpo-41194, bpo-41261, bpo-41631: The _ast module uses a global state. +static astmodulestate global_ast_state = {0}; static astmodulestate* get_global_ast_state(void) { - _Py_IDENTIFIER(_ast); - PyObject *name = _PyUnicode_FromId(&PyId__ast); // borrowed reference - if (name == NULL) { + astmodulestate* state = &global_ast_state; + if (!init_types(state)) { return NULL; } - PyObject *module = PyImport_GetModule(name); - if (module == NULL) { - if (PyErr_Occurred()) { - return NULL; - } - module = PyImport_Import(name); - if (module == NULL) { - return NULL; - } - } - astmodulestate *state = get_ast_state(module); - Py_DECREF(module); return state; } -static int astmodule_clear(PyObject *module) +static astmodulestate* +get_ast_state(PyObject* Py_UNUSED(module)) { - astmodulestate *state = get_ast_state(module); -""") - for s in module_state: - f.write(" Py_CLEAR(state->" + s + ');\n') - f.write(""" - return 0; + astmodulestate* state = get_global_ast_state(); + // get_ast_state() must only be called after _ast module is imported, + // and astmodule_exec() calls init_types() + assert(state != NULL); + return state; } -static int astmodule_traverse(PyObject *module, visitproc visit, void* arg) +void _PyAST_Fini(PyThreadState *tstate) { - astmodulestate *state = get_ast_state(module); + astmodulestate* state = &global_ast_state; """) for s in module_state: - f.write(" Py_VISIT(state->" + s + ');\n') + f.write(" Py_CLEAR(state->" + s + ');\n') f.write(""" - return 0; -} - -static void astmodule_free(void* module) { - astmodule_clear((PyObject*)module); + state->initialized = 0; } """) diff --git a/Python/Python-ast.c b/Python/Python-ast.c index 396a6832702b3..094010e6c9ddc 100644 --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -224,40 +224,35 @@ typedef struct { } astmodulestate; -static astmodulestate* -get_ast_state(PyObject *module) -{ - void *state = PyModule_GetState(module); - assert(state != NULL); - return (astmodulestate*)state; -} +// Forward declaration +static int init_types(astmodulestate *state); + +// bpo-41194, bpo-41261, bpo-41631: The _ast module uses a global state. +static astmodulestate global_ast_state = {0}; static astmodulestate* get_global_ast_state(void) { - _Py_IDENTIFIER(_ast); - PyObject *name = _PyUnicode_FromId(&PyId__ast); // borrowed reference - if (name == NULL) { + astmodulestate* state = &global_ast_state; + if (!init_types(state)) { return NULL; } - PyObject *module = PyImport_GetModule(name); - if (module == NULL) { - if (PyErr_Occurred()) { - return NULL; - } - module = PyImport_Import(name); - if (module == NULL) { - return NULL; - } - } - astmodulestate *state = get_ast_state(module); - Py_DECREF(module); return state; } -static int astmodule_clear(PyObject *module) +static astmodulestate* +get_ast_state(PyObject* Py_UNUSED(module)) { - astmodulestate *state = get_ast_state(module); + astmodulestate* state = get_global_ast_state(); + // get_ast_state() must only be called after _ast module is imported, + // and astmodule_exec() calls init_types() + assert(state != NULL); + return state; +} + +void _PyAST_Fini(PyThreadState *tstate) +{ + astmodulestate* state = &global_ast_state; Py_CLEAR(state->AST_type); Py_CLEAR(state->Add_singleton); Py_CLEAR(state->Add_type); @@ -472,231 +467,7 @@ static int astmodule_clear(PyObject *module) Py_CLEAR(state->vararg); Py_CLEAR(state->withitem_type); - return 0; -} - -static int astmodule_traverse(PyObject *module, visitproc visit, void* arg) -{ - astmodulestate *state = get_ast_state(module); - Py_VISIT(state->AST_type); - Py_VISIT(state->Add_singleton); - Py_VISIT(state->Add_type); - Py_VISIT(state->And_singleton); - Py_VISIT(state->And_type); - Py_VISIT(state->AnnAssign_type); - Py_VISIT(state->Assert_type); - Py_VISIT(state->Assign_type); - Py_VISIT(state->AsyncFor_type); - Py_VISIT(state->AsyncFunctionDef_type); - Py_VISIT(state->AsyncWith_type); - Py_VISIT(state->Attribute_type); - Py_VISIT(state->AugAssign_type); - Py_VISIT(state->Await_type); - Py_VISIT(state->BinOp_type); - Py_VISIT(state->BitAnd_singleton); - Py_VISIT(state->BitAnd_type); - Py_VISIT(state->BitOr_singleton); - Py_VISIT(state->BitOr_type); - Py_VISIT(state->BitXor_singleton); - Py_VISIT(state->BitXor_type); - Py_VISIT(state->BoolOp_type); - Py_VISIT(state->Break_type); - Py_VISIT(state->Call_type); - Py_VISIT(state->ClassDef_type); - Py_VISIT(state->Compare_type); - Py_VISIT(state->Constant_type); - Py_VISIT(state->Continue_type); - Py_VISIT(state->Del_singleton); - Py_VISIT(state->Del_type); - Py_VISIT(state->Delete_type); - Py_VISIT(state->DictComp_type); - Py_VISIT(state->Dict_type); - Py_VISIT(state->Div_singleton); - Py_VISIT(state->Div_type); - Py_VISIT(state->Eq_singleton); - Py_VISIT(state->Eq_type); - Py_VISIT(state->ExceptHandler_type); - Py_VISIT(state->Expr_type); - Py_VISIT(state->Expression_type); - Py_VISIT(state->FloorDiv_singleton); - Py_VISIT(state->FloorDiv_type); - Py_VISIT(state->For_type); - Py_VISIT(state->FormattedValue_type); - Py_VISIT(state->FunctionDef_type); - Py_VISIT(state->FunctionType_type); - Py_VISIT(state->GeneratorExp_type); - Py_VISIT(state->Global_type); - Py_VISIT(state->GtE_singleton); - Py_VISIT(state->GtE_type); - Py_VISIT(state->Gt_singleton); - Py_VISIT(state->Gt_type); - Py_VISIT(state->IfExp_type); - Py_VISIT(state->If_type); - Py_VISIT(state->ImportFrom_type); - Py_VISIT(state->Import_type); - Py_VISIT(state->In_singleton); - Py_VISIT(state->In_type); - Py_VISIT(state->Interactive_type); - Py_VISIT(state->Invert_singleton); - Py_VISIT(state->Invert_type); - Py_VISIT(state->IsNot_singleton); - Py_VISIT(state->IsNot_type); - Py_VISIT(state->Is_singleton); - Py_VISIT(state->Is_type); - Py_VISIT(state->JoinedStr_type); - Py_VISIT(state->LShift_singleton); - Py_VISIT(state->LShift_type); - Py_VISIT(state->Lambda_type); - Py_VISIT(state->ListComp_type); - Py_VISIT(state->List_type); - Py_VISIT(state->Load_singleton); - Py_VISIT(state->Load_type); - Py_VISIT(state->LtE_singleton); - Py_VISIT(state->LtE_type); - Py_VISIT(state->Lt_singleton); - Py_VISIT(state->Lt_type); - Py_VISIT(state->MatMult_singleton); - Py_VISIT(state->MatMult_type); - Py_VISIT(state->Mod_singleton); - Py_VISIT(state->Mod_type); - Py_VISIT(state->Module_type); - Py_VISIT(state->Mult_singleton); - Py_VISIT(state->Mult_type); - Py_VISIT(state->Name_type); - Py_VISIT(state->NamedExpr_type); - Py_VISIT(state->Nonlocal_type); - Py_VISIT(state->NotEq_singleton); - Py_VISIT(state->NotEq_type); - Py_VISIT(state->NotIn_singleton); - Py_VISIT(state->NotIn_type); - Py_VISIT(state->Not_singleton); - Py_VISIT(state->Not_type); - Py_VISIT(state->Or_singleton); - Py_VISIT(state->Or_type); - Py_VISIT(state->Pass_type); - Py_VISIT(state->Pow_singleton); - Py_VISIT(state->Pow_type); - Py_VISIT(state->RShift_singleton); - Py_VISIT(state->RShift_type); - Py_VISIT(state->Raise_type); - Py_VISIT(state->Return_type); - Py_VISIT(state->SetComp_type); - Py_VISIT(state->Set_type); - Py_VISIT(state->Slice_type); - Py_VISIT(state->Starred_type); - Py_VISIT(state->Store_singleton); - Py_VISIT(state->Store_type); - Py_VISIT(state->Sub_singleton); - Py_VISIT(state->Sub_type); - Py_VISIT(state->Subscript_type); - Py_VISIT(state->Try_type); - Py_VISIT(state->Tuple_type); - Py_VISIT(state->TypeIgnore_type); - Py_VISIT(state->UAdd_singleton); - Py_VISIT(state->UAdd_type); - Py_VISIT(state->USub_singleton); - Py_VISIT(state->USub_type); - Py_VISIT(state->UnaryOp_type); - Py_VISIT(state->While_type); - Py_VISIT(state->With_type); - Py_VISIT(state->YieldFrom_type); - Py_VISIT(state->Yield_type); - Py_VISIT(state->__dict__); - Py_VISIT(state->__doc__); - Py_VISIT(state->__module__); - Py_VISIT(state->_attributes); - Py_VISIT(state->_fields); - Py_VISIT(state->alias_type); - Py_VISIT(state->annotation); - Py_VISIT(state->arg); - Py_VISIT(state->arg_type); - Py_VISIT(state->args); - Py_VISIT(state->argtypes); - Py_VISIT(state->arguments_type); - Py_VISIT(state->asname); - Py_VISIT(state->ast); - Py_VISIT(state->attr); - Py_VISIT(state->bases); - Py_VISIT(state->body); - Py_VISIT(state->boolop_type); - Py_VISIT(state->cause); - Py_VISIT(state->cmpop_type); - Py_VISIT(state->col_offset); - Py_VISIT(state->comparators); - Py_VISIT(state->comprehension_type); - Py_VISIT(state->context_expr); - Py_VISIT(state->conversion); - Py_VISIT(state->ctx); - Py_VISIT(state->decorator_list); - Py_VISIT(state->defaults); - Py_VISIT(state->elt); - Py_VISIT(state->elts); - Py_VISIT(state->end_col_offset); - Py_VISIT(state->end_lineno); - Py_VISIT(state->exc); - Py_VISIT(state->excepthandler_type); - Py_VISIT(state->expr_context_type); - Py_VISIT(state->expr_type); - Py_VISIT(state->finalbody); - Py_VISIT(state->format_spec); - Py_VISIT(state->func); - Py_VISIT(state->generators); - Py_VISIT(state->handlers); - Py_VISIT(state->id); - Py_VISIT(state->ifs); - Py_VISIT(state->is_async); - Py_VISIT(state->items); - Py_VISIT(state->iter); - Py_VISIT(state->key); - Py_VISIT(state->keys); - Py_VISIT(state->keyword_type); - Py_VISIT(state->keywords); - Py_VISIT(state->kind); - Py_VISIT(state->kw_defaults); - Py_VISIT(state->kwarg); - Py_VISIT(state->kwonlyargs); - Py_VISIT(state->left); - Py_VISIT(state->level); - Py_VISIT(state->lineno); - Py_VISIT(state->lower); - Py_VISIT(state->mod_type); - Py_VISIT(state->module); - Py_VISIT(state->msg); - Py_VISIT(state->name); - Py_VISIT(state->names); - Py_VISIT(state->op); - Py_VISIT(state->operand); - Py_VISIT(state->operator_type); - Py_VISIT(state->ops); - Py_VISIT(state->optional_vars); - Py_VISIT(state->orelse); - Py_VISIT(state->posonlyargs); - Py_VISIT(state->returns); - Py_VISIT(state->right); - Py_VISIT(state->simple); - Py_VISIT(state->slice); - Py_VISIT(state->step); - Py_VISIT(state->stmt_type); - Py_VISIT(state->tag); - Py_VISIT(state->target); - Py_VISIT(state->targets); - Py_VISIT(state->test); - Py_VISIT(state->type); - Py_VISIT(state->type_comment); - Py_VISIT(state->type_ignore_type); - Py_VISIT(state->type_ignores); - Py_VISIT(state->unaryop_type); - Py_VISIT(state->upper); - Py_VISIT(state->value); - Py_VISIT(state->values); - Py_VISIT(state->vararg); - Py_VISIT(state->withitem_type); - - return 0; -} - -static void astmodule_free(void* module) { - astmodule_clear((PyObject*)module); + state->initialized = 0; } static int init_identifiers(astmodulestate *state) @@ -10316,11 +10087,9 @@ static PyModuleDef_Slot astmodule_slots[] = { static struct PyModuleDef _astmodule = { PyModuleDef_HEAD_INIT, .m_name = "_ast", - .m_size = sizeof(astmodulestate), + // The _ast module uses a global state (global_ast_state). + .m_size = 0, .m_slots = astmodule_slots, - .m_traverse = astmodule_traverse, - .m_clear = astmodule_clear, - .m_free = astmodule_free, }; PyMODINIT_FUNC diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index ab5a6767864dc..75d57805c07b6 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -1259,6 +1259,12 @@ flush_std_files(void) static void finalize_interp_types(PyThreadState *tstate) { + // The _ast module state is shared by all interpreters. + // The state must only be cleared by the main interpreter. + if (_Py_IsMainInterpreter(tstate)) { + _PyAST_Fini(tstate); + } + _PyExc_Fini(tstate); _PyFrame_Fini(tstate); _PyAsyncGen_Fini(tstate); From webhook-mailer at python.org Tue Sep 15 14:33:07 2020 From: webhook-mailer at python.org (Pablo Galindo) Date: Tue, 15 Sep 2020 18:33:07 -0000 Subject: [Python-checkins] [3.9] bpo-41631: _ast module uses again a global state (GH-21961) (GH-22258) Message-ID: https://github.com/python/cpython/commit/55e0836849c14fb474e1ba7f37851e07660eea3c commit: 55e0836849c14fb474e1ba7f37851e07660eea3c branch: 3.9 author: Pablo Galindo committer: GitHub date: 2020-09-15T20:32:56+02:00 summary: [3.9] bpo-41631: _ast module uses again a global state (GH-21961) (GH-22258) Partially revert commit ac46eb4ad6662cf6d771b20d8963658b2186c48c: "bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)". Using a module state per module instance is causing subtle practical problems. For example, the Mercurial project replaces the __import__() function to implement lazy import, whereas Python expected that "import _ast" always return a fully initialized _ast module. Add _PyAST_Fini() to clear the state at exit. The _ast module has no state (set _astmodule.m_size to 0). Remove astmodule_traverse(), astmodule_clear() and astmodule_free() functions.. (cherry picked from commit e5fbe0cbd4be99ced5f000ad382208ad2a561c90) Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst M Include/internal/pycore_pylifecycle.h M Lib/ast.py M Lib/test/test_ast.py M Parser/asdl_c.py M Python/Python-ast.c M Python/pylifecycle.c diff --git a/Include/internal/pycore_pylifecycle.h b/Include/internal/pycore_pylifecycle.h index 77ea3f27454da..b0adc4562c447 100644 --- a/Include/internal/pycore_pylifecycle.h +++ b/Include/internal/pycore_pylifecycle.h @@ -82,6 +82,7 @@ extern void _PyFaulthandler_Fini(void); extern void _PyHash_Fini(void); extern void _PyTraceMalloc_Fini(void); extern void _PyWarnings_Fini(PyInterpreterState *interp); +extern void _PyAST_Fini(); extern PyStatus _PyGILState_Init(PyThreadState *tstate); extern void _PyGILState_Fini(PyThreadState *tstate); diff --git a/Lib/ast.py b/Lib/ast.py index 65ebd0100de00..d860917f4d03a 100644 --- a/Lib/ast.py +++ b/Lib/ast.py @@ -497,18 +497,20 @@ def generic_visit(self, node): return node -# The following code is for backward compatibility. -# It will be removed in future. +# If the ast module is loaded more than once, only add deprecated methods once +if not hasattr(Constant, 'n'): + # The following code is for backward compatibility. + # It will be removed in future. -def _getter(self): - """Deprecated. Use value instead.""" - return self.value + def _getter(self): + """Deprecated. Use value instead.""" + return self.value -def _setter(self, value): - self.value = value + def _setter(self, value): + self.value = value -Constant.n = property(_getter, _setter) -Constant.s = property(_getter, _setter) + Constant.n = property(_getter, _setter) + Constant.s = property(_getter, _setter) class _ABC(type): @@ -600,14 +602,19 @@ class ExtSlice(slice): def __new__(cls, dims=(), **kwargs): return Tuple(list(dims), Load(), **kwargs) -def _dims_getter(self): - """Deprecated. Use elts instead.""" - return self.elts +# If the ast module is loaded more than once, only add deprecated methods once +if not hasattr(Tuple, 'dims'): + # The following code is for backward compatibility. + # It will be removed in future. -def _dims_setter(self, value): - self.elts = value + def _dims_getter(self): + """Deprecated. Use elts instead.""" + return self.elts -Tuple.dims = property(_dims_getter, _dims_setter) + def _dims_setter(self, value): + self.elts = value + + Tuple.dims = property(_dims_getter, _dims_setter) class Suite(mod): """Deprecated AST node class. Unused in Python 3.""" diff --git a/Lib/test/test_ast.py b/Lib/test/test_ast.py index f5aef61ec6f7c..5f57ce8724482 100644 --- a/Lib/test/test_ast.py +++ b/Lib/test/test_ast.py @@ -1,7 +1,9 @@ import ast +import builtins import dis import os import sys +import types import unittest import warnings import weakref @@ -1945,6 +1947,88 @@ def visit_Ellipsis(self, node): ]) + at support.cpython_only +class ModuleStateTests(unittest.TestCase): + # bpo-41194, bpo-41261, bpo-41631: The _ast module uses a global state. + + def check_ast_module(self): + # Check that the _ast module still works as expected + code = 'x + 1' + filename = '' + mode = 'eval' + + # Create _ast.AST subclasses instances + ast_tree = compile(code, filename, mode, flags=ast.PyCF_ONLY_AST) + + # Call PyAST_Check() + code = compile(ast_tree, filename, mode) + self.assertIsInstance(code, types.CodeType) + + def test_reload_module(self): + # bpo-41194: Importing the _ast module twice must not crash. + with support.swap_item(sys.modules, '_ast', None): + del sys.modules['_ast'] + import _ast as ast1 + + del sys.modules['_ast'] + import _ast as ast2 + + self.check_ast_module() + + # Unloading the two _ast module instances must not crash. + del ast1 + del ast2 + support.gc_collect() + + self.check_ast_module() + + def test_sys_modules(self): + # bpo-41631: Test reproducing a Mercurial crash when PyAST_Check() + # imported the _ast module internally. + lazy_mod = object() + + def my_import(name, *args, **kw): + sys.modules[name] = lazy_mod + return lazy_mod + + with support.swap_item(sys.modules, '_ast', None): + del sys.modules['_ast'] + + with support.swap_attr(builtins, '__import__', my_import): + # Test that compile() does not import the _ast module + self.check_ast_module() + self.assertNotIn('_ast', sys.modules) + + # Sanity check of the test itself + import _ast + self.assertIs(_ast, lazy_mod) + + def test_subinterpreter(self): + # bpo-41631: Importing and using the _ast module in a subinterpreter + # must not crash. + code = dedent(''' + import _ast + import ast + import gc + import sys + import types + + # Create _ast.AST subclasses instances and call PyAST_Check() + ast_tree = compile('x+1', '', 'eval', + flags=ast.PyCF_ONLY_AST) + code = compile(ast_tree, 'string', 'eval') + if not isinstance(code, types.CodeType): + raise AssertionError + + # Unloading the _ast module must not crash. + del ast, _ast + del sys.modules['ast'], sys.modules['_ast'] + gc.collect() + ''') + res = support.run_in_subinterp(code) + self.assertEqual(res, 0) + + def main(): if __name__ != '__main__': return diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst new file mode 100644 index 0000000000000..68bb51024d9e7 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst @@ -0,0 +1,5 @@ +The ``_ast`` module uses again a global state. Using a module state per module +instance is causing subtle practical problems. For example, the Mercurial +project replaces the ``__import__()`` function to implement lazy import, +whereas Python expected that ``import _ast`` always return a fully initialized +``_ast`` module. diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py index 6fe44b99f793b..6f3154aeb6fc2 100755 --- a/Parser/asdl_c.py +++ b/Parser/asdl_c.py @@ -1089,11 +1089,9 @@ def visitModule(self, mod): static struct PyModuleDef _astmodule = { PyModuleDef_HEAD_INIT, .m_name = "_ast", - .m_size = sizeof(astmodulestate), + // The _ast module uses a global state (global_ast_state). + .m_size = 0, .m_slots = astmodule_slots, - .m_traverse = astmodule_traverse, - .m_clear = astmodule_clear, - .m_free = astmodule_free, }; PyMODINIT_FUNC @@ -1374,59 +1372,40 @@ def generate_module_def(f, mod): f.write(' PyObject *' + s + ';\n') f.write('} astmodulestate;\n\n') f.write(""" -static astmodulestate* -get_ast_state(PyObject *module) -{ - void *state = PyModule_GetState(module); - assert(state != NULL); - return (astmodulestate*)state; -} +// Forward declaration +static int init_types(astmodulestate *state); + +// bpo-41194, bpo-41261, bpo-41631: The _ast module uses a global state. +static astmodulestate global_ast_state = {0}; static astmodulestate* get_global_ast_state(void) { - _Py_IDENTIFIER(_ast); - PyObject *name = _PyUnicode_FromId(&PyId__ast); // borrowed reference - if (name == NULL) { + astmodulestate* state = &global_ast_state; + if (!init_types(state)) { return NULL; } - PyObject *module = PyImport_GetModule(name); - if (module == NULL) { - if (PyErr_Occurred()) { - return NULL; - } - module = PyImport_Import(name); - if (module == NULL) { - return NULL; - } - } - astmodulestate *state = get_ast_state(module); - Py_DECREF(module); return state; } -static int astmodule_clear(PyObject *module) +static astmodulestate* +get_ast_state(PyObject* Py_UNUSED(module)) { - astmodulestate *state = get_ast_state(module); -""") - for s in module_state: - f.write(" Py_CLEAR(state->" + s + ');\n') - f.write(""" - return 0; + astmodulestate* state = get_global_ast_state(); + // get_ast_state() must only be called after _ast module is imported, + // and astmodule_exec() calls init_types() + assert(state != NULL); + return state; } -static int astmodule_traverse(PyObject *module, visitproc visit, void* arg) +void _PyAST_Fini() { - astmodulestate *state = get_ast_state(module); + astmodulestate* state = &global_ast_state; """) for s in module_state: - f.write(" Py_VISIT(state->" + s + ');\n') + f.write(" Py_CLEAR(state->" + s + ');\n') f.write(""" - return 0; -} - -static void astmodule_free(void* module) { - astmodule_clear((PyObject*)module); + state->initialized = 0; } """) diff --git a/Python/Python-ast.c b/Python/Python-ast.c index 396a6832702b3..9e86f43178957 100644 --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -224,40 +224,35 @@ typedef struct { } astmodulestate; -static astmodulestate* -get_ast_state(PyObject *module) -{ - void *state = PyModule_GetState(module); - assert(state != NULL); - return (astmodulestate*)state; -} +// Forward declaration +static int init_types(astmodulestate *state); + +// bpo-41194, bpo-41261, bpo-41631: The _ast module uses a global state. +static astmodulestate global_ast_state = {0}; static astmodulestate* get_global_ast_state(void) { - _Py_IDENTIFIER(_ast); - PyObject *name = _PyUnicode_FromId(&PyId__ast); // borrowed reference - if (name == NULL) { + astmodulestate* state = &global_ast_state; + if (!init_types(state)) { return NULL; } - PyObject *module = PyImport_GetModule(name); - if (module == NULL) { - if (PyErr_Occurred()) { - return NULL; - } - module = PyImport_Import(name); - if (module == NULL) { - return NULL; - } - } - astmodulestate *state = get_ast_state(module); - Py_DECREF(module); return state; } -static int astmodule_clear(PyObject *module) +static astmodulestate* +get_ast_state(PyObject* Py_UNUSED(module)) { - astmodulestate *state = get_ast_state(module); + astmodulestate* state = get_global_ast_state(); + // get_ast_state() must only be called after _ast module is imported, + // and astmodule_exec() calls init_types() + assert(state != NULL); + return state; +} + +void _PyAST_Fini() +{ + astmodulestate* state = &global_ast_state; Py_CLEAR(state->AST_type); Py_CLEAR(state->Add_singleton); Py_CLEAR(state->Add_type); @@ -472,231 +467,7 @@ static int astmodule_clear(PyObject *module) Py_CLEAR(state->vararg); Py_CLEAR(state->withitem_type); - return 0; -} - -static int astmodule_traverse(PyObject *module, visitproc visit, void* arg) -{ - astmodulestate *state = get_ast_state(module); - Py_VISIT(state->AST_type); - Py_VISIT(state->Add_singleton); - Py_VISIT(state->Add_type); - Py_VISIT(state->And_singleton); - Py_VISIT(state->And_type); - Py_VISIT(state->AnnAssign_type); - Py_VISIT(state->Assert_type); - Py_VISIT(state->Assign_type); - Py_VISIT(state->AsyncFor_type); - Py_VISIT(state->AsyncFunctionDef_type); - Py_VISIT(state->AsyncWith_type); - Py_VISIT(state->Attribute_type); - Py_VISIT(state->AugAssign_type); - Py_VISIT(state->Await_type); - Py_VISIT(state->BinOp_type); - Py_VISIT(state->BitAnd_singleton); - Py_VISIT(state->BitAnd_type); - Py_VISIT(state->BitOr_singleton); - Py_VISIT(state->BitOr_type); - Py_VISIT(state->BitXor_singleton); - Py_VISIT(state->BitXor_type); - Py_VISIT(state->BoolOp_type); - Py_VISIT(state->Break_type); - Py_VISIT(state->Call_type); - Py_VISIT(state->ClassDef_type); - Py_VISIT(state->Compare_type); - Py_VISIT(state->Constant_type); - Py_VISIT(state->Continue_type); - Py_VISIT(state->Del_singleton); - Py_VISIT(state->Del_type); - Py_VISIT(state->Delete_type); - Py_VISIT(state->DictComp_type); - Py_VISIT(state->Dict_type); - Py_VISIT(state->Div_singleton); - Py_VISIT(state->Div_type); - Py_VISIT(state->Eq_singleton); - Py_VISIT(state->Eq_type); - Py_VISIT(state->ExceptHandler_type); - Py_VISIT(state->Expr_type); - Py_VISIT(state->Expression_type); - Py_VISIT(state->FloorDiv_singleton); - Py_VISIT(state->FloorDiv_type); - Py_VISIT(state->For_type); - Py_VISIT(state->FormattedValue_type); - Py_VISIT(state->FunctionDef_type); - Py_VISIT(state->FunctionType_type); - Py_VISIT(state->GeneratorExp_type); - Py_VISIT(state->Global_type); - Py_VISIT(state->GtE_singleton); - Py_VISIT(state->GtE_type); - Py_VISIT(state->Gt_singleton); - Py_VISIT(state->Gt_type); - Py_VISIT(state->IfExp_type); - Py_VISIT(state->If_type); - Py_VISIT(state->ImportFrom_type); - Py_VISIT(state->Import_type); - Py_VISIT(state->In_singleton); - Py_VISIT(state->In_type); - Py_VISIT(state->Interactive_type); - Py_VISIT(state->Invert_singleton); - Py_VISIT(state->Invert_type); - Py_VISIT(state->IsNot_singleton); - Py_VISIT(state->IsNot_type); - Py_VISIT(state->Is_singleton); - Py_VISIT(state->Is_type); - Py_VISIT(state->JoinedStr_type); - Py_VISIT(state->LShift_singleton); - Py_VISIT(state->LShift_type); - Py_VISIT(state->Lambda_type); - Py_VISIT(state->ListComp_type); - Py_VISIT(state->List_type); - Py_VISIT(state->Load_singleton); - Py_VISIT(state->Load_type); - Py_VISIT(state->LtE_singleton); - Py_VISIT(state->LtE_type); - Py_VISIT(state->Lt_singleton); - Py_VISIT(state->Lt_type); - Py_VISIT(state->MatMult_singleton); - Py_VISIT(state->MatMult_type); - Py_VISIT(state->Mod_singleton); - Py_VISIT(state->Mod_type); - Py_VISIT(state->Module_type); - Py_VISIT(state->Mult_singleton); - Py_VISIT(state->Mult_type); - Py_VISIT(state->Name_type); - Py_VISIT(state->NamedExpr_type); - Py_VISIT(state->Nonlocal_type); - Py_VISIT(state->NotEq_singleton); - Py_VISIT(state->NotEq_type); - Py_VISIT(state->NotIn_singleton); - Py_VISIT(state->NotIn_type); - Py_VISIT(state->Not_singleton); - Py_VISIT(state->Not_type); - Py_VISIT(state->Or_singleton); - Py_VISIT(state->Or_type); - Py_VISIT(state->Pass_type); - Py_VISIT(state->Pow_singleton); - Py_VISIT(state->Pow_type); - Py_VISIT(state->RShift_singleton); - Py_VISIT(state->RShift_type); - Py_VISIT(state->Raise_type); - Py_VISIT(state->Return_type); - Py_VISIT(state->SetComp_type); - Py_VISIT(state->Set_type); - Py_VISIT(state->Slice_type); - Py_VISIT(state->Starred_type); - Py_VISIT(state->Store_singleton); - Py_VISIT(state->Store_type); - Py_VISIT(state->Sub_singleton); - Py_VISIT(state->Sub_type); - Py_VISIT(state->Subscript_type); - Py_VISIT(state->Try_type); - Py_VISIT(state->Tuple_type); - Py_VISIT(state->TypeIgnore_type); - Py_VISIT(state->UAdd_singleton); - Py_VISIT(state->UAdd_type); - Py_VISIT(state->USub_singleton); - Py_VISIT(state->USub_type); - Py_VISIT(state->UnaryOp_type); - Py_VISIT(state->While_type); - Py_VISIT(state->With_type); - Py_VISIT(state->YieldFrom_type); - Py_VISIT(state->Yield_type); - Py_VISIT(state->__dict__); - Py_VISIT(state->__doc__); - Py_VISIT(state->__module__); - Py_VISIT(state->_attributes); - Py_VISIT(state->_fields); - Py_VISIT(state->alias_type); - Py_VISIT(state->annotation); - Py_VISIT(state->arg); - Py_VISIT(state->arg_type); - Py_VISIT(state->args); - Py_VISIT(state->argtypes); - Py_VISIT(state->arguments_type); - Py_VISIT(state->asname); - Py_VISIT(state->ast); - Py_VISIT(state->attr); - Py_VISIT(state->bases); - Py_VISIT(state->body); - Py_VISIT(state->boolop_type); - Py_VISIT(state->cause); - Py_VISIT(state->cmpop_type); - Py_VISIT(state->col_offset); - Py_VISIT(state->comparators); - Py_VISIT(state->comprehension_type); - Py_VISIT(state->context_expr); - Py_VISIT(state->conversion); - Py_VISIT(state->ctx); - Py_VISIT(state->decorator_list); - Py_VISIT(state->defaults); - Py_VISIT(state->elt); - Py_VISIT(state->elts); - Py_VISIT(state->end_col_offset); - Py_VISIT(state->end_lineno); - Py_VISIT(state->exc); - Py_VISIT(state->excepthandler_type); - Py_VISIT(state->expr_context_type); - Py_VISIT(state->expr_type); - Py_VISIT(state->finalbody); - Py_VISIT(state->format_spec); - Py_VISIT(state->func); - Py_VISIT(state->generators); - Py_VISIT(state->handlers); - Py_VISIT(state->id); - Py_VISIT(state->ifs); - Py_VISIT(state->is_async); - Py_VISIT(state->items); - Py_VISIT(state->iter); - Py_VISIT(state->key); - Py_VISIT(state->keys); - Py_VISIT(state->keyword_type); - Py_VISIT(state->keywords); - Py_VISIT(state->kind); - Py_VISIT(state->kw_defaults); - Py_VISIT(state->kwarg); - Py_VISIT(state->kwonlyargs); - Py_VISIT(state->left); - Py_VISIT(state->level); - Py_VISIT(state->lineno); - Py_VISIT(state->lower); - Py_VISIT(state->mod_type); - Py_VISIT(state->module); - Py_VISIT(state->msg); - Py_VISIT(state->name); - Py_VISIT(state->names); - Py_VISIT(state->op); - Py_VISIT(state->operand); - Py_VISIT(state->operator_type); - Py_VISIT(state->ops); - Py_VISIT(state->optional_vars); - Py_VISIT(state->orelse); - Py_VISIT(state->posonlyargs); - Py_VISIT(state->returns); - Py_VISIT(state->right); - Py_VISIT(state->simple); - Py_VISIT(state->slice); - Py_VISIT(state->step); - Py_VISIT(state->stmt_type); - Py_VISIT(state->tag); - Py_VISIT(state->target); - Py_VISIT(state->targets); - Py_VISIT(state->test); - Py_VISIT(state->type); - Py_VISIT(state->type_comment); - Py_VISIT(state->type_ignore_type); - Py_VISIT(state->type_ignores); - Py_VISIT(state->unaryop_type); - Py_VISIT(state->upper); - Py_VISIT(state->value); - Py_VISIT(state->values); - Py_VISIT(state->vararg); - Py_VISIT(state->withitem_type); - - return 0; -} - -static void astmodule_free(void* module) { - astmodule_clear((PyObject*)module); + state->initialized = 0; } static int init_identifiers(astmodulestate *state) @@ -10316,11 +10087,9 @@ static PyModuleDef_Slot astmodule_slots[] = { static struct PyModuleDef _astmodule = { PyModuleDef_HEAD_INIT, .m_name = "_ast", - .m_size = sizeof(astmodulestate), + // The _ast module uses a global state (global_ast_state). + .m_size = 0, .m_slots = astmodule_slots, - .m_traverse = astmodule_traverse, - .m_clear = astmodule_clear, - .m_free = astmodule_free, }; PyMODINIT_FUNC diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c index 00a9b99be4aa3..cfb3a7d941710 100644 --- a/Python/pylifecycle.c +++ b/Python/pylifecycle.c @@ -1255,6 +1255,7 @@ finalize_interp_types(PyThreadState *tstate, int is_main_interp) { if (is_main_interp) { /* Sundry finalizers */ + _PyAST_Fini(); _PyFrame_Fini(); _PyTuple_Fini(); _PyList_Fini(); From webhook-mailer at python.org Tue Sep 15 15:20:59 2020 From: webhook-mailer at python.org (Mandeep) Date: Tue, 15 Sep 2020 19:20:59 -0000 Subject: [Python-checkins] Improve the description of difflib in the documentation (GH-22253) Message-ID: https://github.com/python/cpython/commit/5531269f698f789d1247123fd82681f7a455f66e commit: 5531269f698f789d1247123fd82681f7a455f66e branch: master author: Mandeep committer: GitHub date: 2020-09-15T12:20:49-07:00 summary: Improve the description of difflib in the documentation (GH-22253) >From "can produce difference information in various formats ..." to " can produce information about file differences in various formats ..." Automerge-Triggered-By: @Mariatta files: M Doc/library/difflib.rst diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst index 25e3511d01785..aa08988c8b36f 100644 --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -18,8 +18,8 @@ -------------- This module provides classes and functions for comparing sequences. It -can be used for example, for comparing files, and can produce difference -information in various formats, including HTML and context and unified +can be used for example, for comparing files, and can produce information +about file differences in various formats, including HTML and context and unified diffs. For comparing directories and files, see also, the :mod:`filecmp` module. From webhook-mailer at python.org Tue Sep 15 15:27:20 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Tue, 15 Sep 2020 19:27:20 -0000 Subject: [Python-checkins] minor reformat of enum tests (GH-22259) Message-ID: https://github.com/python/cpython/commit/5a565b3d7c31f9f402306a9bd58df36e4fe66ba4 commit: 5a565b3d7c31f9f402306a9bd58df36e4fe66ba4 branch: master author: Ethan Furman committer: GitHub date: 2020-09-15T12:27:06-07:00 summary: minor reformat of enum tests (GH-22259) Automerge-Triggered-By: @ethanfurman files: M Lib/test/test_enum.py diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index b18f3b38a6619..138f572906a47 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -420,8 +420,10 @@ def red(self): blue = 3 def test_reserved__sunder_(self): - with self.assertRaisesRegex(ValueError, '_sunder_ names, such as ' - '"_bad_", are reserved'): + with self.assertRaisesRegex( + ValueError, + '_sunder_ names, such as "_bad_", are reserved', + ): class Bad(Enum): _bad_ = 1 @@ -1136,9 +1138,11 @@ def __name__(self): return self._intname def __repr__(self): # repr() is updated to include the name and type info - return "{}({!r}, {})".format(type(self).__name__, - self.__name__, - int.__repr__(self)) + return "{}({!r}, {})".format( + type(self).__name__, + self.__name__, + int.__repr__(self), + ) def __str__(self): # str() is unchanged, even if it relies on the repr() fallback base = int @@ -1153,7 +1157,8 @@ def __add__(self, other): if isinstance(self, NamedInt) and isinstance(other, NamedInt): return NamedInt( '({0} + {1})'.format(self.__name__, other.__name__), - temp ) + temp, + ) else: return temp @@ -1193,9 +1198,11 @@ def __name__(self): return self._intname def __repr__(self): # repr() is updated to include the name and type info - return "{}({!r}, {})".format(type(self).__name__, - self.__name__, - int.__repr__(self)) + return "{}({!r}, {})".format( + type(self).__name__, + self.__name__, + int.__repr__(self), + ) def __str__(self): # str() is unchanged, even if it relies on the repr() fallback base = int @@ -1210,7 +1217,8 @@ def __add__(self, other): if isinstance(self, NamedInt) and isinstance(other, NamedInt): return NamedInt( '({0} + {1})'.format(self.__name__, other.__name__), - temp ) + temp, + ) else: return temp @@ -1250,9 +1258,11 @@ def __name__(self): return self._intname def __repr__(self): # repr() is updated to include the name and type info - return "{}({!r}, {})".format(type(self).__name__, - self.__name__, - int.__repr__(self)) + return "{}({!r}, {})".format( + type(self).__name__, + self.__name__, + int.__repr__(self), + ) def __str__(self): # str() is unchanged, even if it relies on the repr() fallback base = int @@ -1267,7 +1277,8 @@ def __add__(self, other): if isinstance(self, NamedInt) and isinstance(other, NamedInt): return NamedInt( '({0} + {1})'.format(self.__name__, other.__name__), - temp ) + temp, + ) else: return temp @@ -1307,9 +1318,11 @@ def __name__(self): return self._intname def __repr__(self): # repr() is updated to include the name and type info - return "{}({!r}, {})".format(type(self).__name__, - self.__name__, - int.__repr__(self)) + return "{}({!r}, {})".format( + type(self).__name__, + self.__name__, + int.__repr__(self), + ) def __str__(self): # str() is unchanged, even if it relies on the repr() fallback base = int @@ -1324,7 +1337,8 @@ def __add__(self, other): if isinstance(self, NamedInt) and isinstance(other, NamedInt): return NamedInt( '({0} + {1})'.format(self.__name__, other.__name__), - temp ) + temp, + ) else: return temp @@ -1333,7 +1347,6 @@ class NEI(NamedInt, Enum): x = ('the-x', 1) y = ('the-y', 2) - self.assertIs(NEI.__new__, Enum.__new__) self.assertEqual(repr(NEI.x + NEI.y), "NamedInt('(the-x + the-y)', 3)") globals()['NamedInt'] = NamedInt @@ -1362,9 +1375,11 @@ def __name__(self): return self._intname def __repr__(self): # repr() is updated to include the name and type info - return "{}({!r}, {})".format(type(self).__name__, - self.__name__, - int.__repr__(self)) + return "{}({!r}, {})".format( + type(self).__name__, + self.__name__, + int.__repr__(self), + ) def __str__(self): # str() is unchanged, even if it relies on the repr() fallback base = int @@ -1415,9 +1430,11 @@ def __name__(self): return self._intname def __repr__(self): # repr() is updated to include the name and type info - return "{}({!r}, {})".format(type(self).__name__, - self.__name__, - int.__repr__(self)) + return "{}({!r}, {})".format( + type(self).__name__, + self.__name__, + int.__repr__(self), + ) def __str__(self): # str() is unchanged, even if it relies on the repr() fallback base = int @@ -1432,7 +1449,8 @@ def __add__(self, other): if isinstance(self, NamedInt) and isinstance(other, NamedInt): return NamedInt( '({0} + {1})'.format(self.__name__, other.__name__), - temp ) + temp, + ) else: return temp From webhook-mailer at python.org Tue Sep 15 15:45:13 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 19:45:13 -0000 Subject: [Python-checkins] Improve the description of difflib in the documentation (GH-22253) (#22261) Message-ID: https://github.com/python/cpython/commit/03748837342d45126e71727b400e8fb3d96afc9f commit: 03748837342d45126e71727b400e8fb3d96afc9f branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-15T12:45:05-07:00 summary: Improve the description of difflib in the documentation (GH-22253) (#22261) >From "can produce difference information in various formats ..." to " can produce information about file differences in various formats ..." Automerge-Triggered-By: @Mariatta (cherry picked from commit 5531269f698f789d1247123fd82681f7a455f66e) Co-authored-by: Mandeep files: M Doc/library/difflib.rst diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst index 25e3511d01785..aa08988c8b36f 100644 --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -18,8 +18,8 @@ -------------- This module provides classes and functions for comparing sequences. It -can be used for example, for comparing files, and can produce difference -information in various formats, including HTML and context and unified +can be used for example, for comparing files, and can produce information +about file differences in various formats, including HTML and context and unified diffs. For comparing directories and files, see also, the :mod:`filecmp` module. From webhook-mailer at python.org Tue Sep 15 15:45:33 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 19:45:33 -0000 Subject: [Python-checkins] Improve the description of difflib in the documentation (GH-22253) (GH-22260) Message-ID: https://github.com/python/cpython/commit/5fdc1425974c794966da1d91d54b4717d1ceeeff commit: 5fdc1425974c794966da1d91d54b4717d1ceeeff branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-15T12:45:29-07:00 summary: Improve the description of difflib in the documentation (GH-22253) (GH-22260) >From "can produce difference information in various formats ..." to " can produce information about file differences in various formats ..." Automerge-Triggered-By: @Mariatta (cherry picked from commit 5531269f698f789d1247123fd82681f7a455f66e) Co-authored-by: Mandeep files: M Doc/library/difflib.rst diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst index 0837809df5902..a8543b38c197e 100644 --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -18,8 +18,8 @@ -------------- This module provides classes and functions for comparing sequences. It -can be used for example, for comparing files, and can produce difference -information in various formats, including HTML and context and unified +can be used for example, for comparing files, and can produce information +about file differences in various formats, including HTML and context and unified diffs. For comparing directories and files, see also, the :mod:`filecmp` module. From webhook-mailer at python.org Tue Sep 15 16:13:37 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Tue, 15 Sep 2020 20:13:37 -0000 Subject: [Python-checkins] Fix all Python Cookbook links (#22205) Message-ID: https://github.com/python/cpython/commit/ac0333e1e117b7f61ed7ef1dbcdb6e515ada603b commit: ac0333e1e117b7f61ed7ef1dbcdb6e515ada603b branch: master author: Andre Delfino committer: GitHub date: 2020-09-15T21:13:26+01:00 summary: Fix all Python Cookbook links (#22205) files: M Doc/faq/programming.rst M Doc/howto/urllib2.rst M Doc/library/bisect.rst M Doc/library/collections.abc.rst M Doc/library/collections.rst M Doc/library/difflib.rst M Doc/library/math.rst M Doc/library/random.rst M Doc/library/shelve.rst M Doc/library/stdtypes.rst M Doc/library/sys.rst M Doc/tutorial/whatnow.rst M Doc/whatsnew/3.2.rst M Lib/collections/__init__.py M Lib/heapq.py M Lib/test/test_math.py M Tools/peg_generator/pegen/sccutils.py diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 66d210a55fac7..eecbbf4a5c4ff 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1141,7 +1141,7 @@ How do you remove duplicates from a list? See the Python Cookbook for a long discussion of many ways to do this: - https://code.activestate.com/recipes/52560/ + https://github.com/ActiveState/code/tree/master/recipes/Python/52560_Remove_duplicates/recipe-52560.py If you don't mind reordering the list, sort it and then scan from the end of the list, deleting duplicates as you go:: diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst index 046a88af62f0b..38623371fbabf 100644 --- a/Doc/howto/urllib2.rst +++ b/Doc/howto/urllib2.rst @@ -601,5 +601,5 @@ This document was reviewed and revised by John Lee. scripts with a localhost server, I have to prevent urllib from using the proxy. .. [#] urllib opener for SSL proxy (CONNECT method): `ASPN Cookbook Recipe - `_. + `_. diff --git a/Doc/library/bisect.rst b/Doc/library/bisect.rst index 6bf7814b257f4..6666d55abe2e5 100644 --- a/Doc/library/bisect.rst +++ b/Doc/library/bisect.rst @@ -60,7 +60,7 @@ The following functions are provided: .. seealso:: `SortedCollection recipe - `_ that uses + `_ that uses bisect to build a full-featured collection class with straight-forward search methods and support for a key-function. The keys are precomputed to save unnecessary calls to the key function during searches. diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst index db0e25bb0772e..a6038098675da 100644 --- a/Doc/library/collections.abc.rst +++ b/Doc/library/collections.abc.rst @@ -308,7 +308,7 @@ Notes on using :class:`Set` and :class:`MutableSet` as a mixin: .. seealso:: - * `OrderedSet recipe `_ for an + * `OrderedSet recipe `_ for an example built on :class:`MutableSet`. * For more about ABCs, see the :mod:`abc` module and :pep:`3119`. diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index f538da5e1c9fa..a7d01b3f397a7 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -135,12 +135,12 @@ The class can be used to simulate nested scopes and is useful in templating. :attr:`~collections.ChainMap.parents` property. * The `Nested Contexts recipe - `_ has options to control + `_ has options to control whether writes and other mutations apply only to the first mapping or to any mapping in the chain. * A `greatly simplified read-only version of Chainmap - `_. + `_. :class:`ChainMap` Examples and Recipes diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst index aa08988c8b36f..009b7976dff15 100644 --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -633,7 +633,7 @@ If you want to know how to change the first sequence into the second, use work. * `Simple version control recipe - `_ for a small application + `_ for a small application built with :class:`SequenceMatcher`. diff --git a/Doc/library/math.rst b/Doc/library/math.rst index bbf64643ff59f..f152c45a87aa3 100644 --- a/Doc/library/math.rst +++ b/Doc/library/math.rst @@ -123,7 +123,7 @@ Number-theoretic and representation functions For further discussion and two alternative approaches, see the `ASPN cookbook recipes for accurate floating point summation - `_\. + `_\. .. function:: gcd(*integers) diff --git a/Doc/library/random.rst b/Doc/library/random.rst index 0cdf0a6ac4a47..4e97b1dbad85c 100644 --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -57,7 +57,7 @@ from sources provided by the operating system. `Complementary-Multiply-with-Carry recipe - `_ for a compatible alternative + `_ for a compatible alternative random number generator with a long period and comparatively simple update operations. diff --git a/Doc/library/shelve.rst b/Doc/library/shelve.rst index f08c58179a2f9..a94255bbf698e 100644 --- a/Doc/library/shelve.rst +++ b/Doc/library/shelve.rst @@ -75,7 +75,7 @@ Two additional methods are supported: .. seealso:: - `Persistent dictionary recipe `_ + `Persistent dictionary recipe `_ with widely supported storage formats and having the speed of native dictionaries. diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 0ffe7b7526fa7..2eee22c79af76 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1404,7 +1404,7 @@ objects that compare equal might have different :attr:`~range.start`, .. seealso:: - * The `linspace recipe `_ + * The `linspace recipe `_ shows how to implement a lazy version of range suitable for floating point applications. diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index d201d7061f980..aa417ede40228 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -679,7 +679,7 @@ always available. additional garbage collector overhead if the object is managed by the garbage collector. - See `recursive sizeof recipe `_ + See `recursive sizeof recipe `_ for an example of using :func:`getsizeof` recursively to find the size of containers and all their contents. diff --git a/Doc/tutorial/whatnow.rst b/Doc/tutorial/whatnow.rst index 3208201312b87..38ce9f0a900c2 100644 --- a/Doc/tutorial/whatnow.rst +++ b/Doc/tutorial/whatnow.rst @@ -43,7 +43,7 @@ More Python resources: for download. Once you begin releasing code, you can register it here so that others can find it. -* https://code.activestate.com/recipes/langs/python/: The Python Cookbook is a +* https://github.com/ActiveState/code/tree/master/recipes/Python: The Python Cookbook is a sizable collection of code examples, larger modules, and useful scripts. Particularly notable contributions are collected in a book also titled Python Cookbook (O'Reilly & Associates, ISBN 0-596-00797-3.) diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index 06bee9966c0be..37bae34ce74ad 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -781,8 +781,8 @@ functools (Contributed by Raymond Hettinger and incorporating design ideas from Jim Baker, Miki Tebeka, and Nick Coghlan; see `recipe 498245 - `_\, `recipe 577479 - `_\, :issue:`10586`, and + `_\, `recipe 577479 + `_\, :issue:`10586`, and :issue:`10593`.) * The :func:`functools.wraps` decorator now adds a :attr:`__wrapped__` attribute diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py index 5d75501645fc4..f4da9d0cefd6b 100644 --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -574,7 +574,7 @@ class Counter(dict): # http://en.wikipedia.org/wiki/Multiset # http://www.gnu.org/software/smalltalk/manual-base/html_node/Bag.html # http://www.demo2s.com/Tutorial/Cpp/0380__set-multiset/Catalog0380__set-multiset.htm - # http://code.activestate.com/recipes/259174/ + # https://github.com/ActiveState/code/tree/master/recipes/Python/259174_bag_collection_class/recipe-259174.py # Knuth, TAOCP Vol. II section 4.6.3 def __init__(self, iterable=None, /, **kwds): diff --git a/Lib/heapq.py b/Lib/heapq.py index fabefd87f8bf8..5895562db4142 100644 --- a/Lib/heapq.py +++ b/Lib/heapq.py @@ -456,7 +456,7 @@ def merge(*iterables, key=None, reverse=False): # 2) Made multiple passes over the data. # 3) Made more comparisons in common cases (small k, large n, semi-random input). # See the more detailed comparison of approach at: -# http://code.activestate.com/recipes/577573-compare-algorithms-for-heapqsmallest +# https://github.com/ActiveState/code/tree/master/recipes/Python/577573_Compare_algorithms/recipe-577573.py def nsmallest(n, iterable, key=None): """Find the n smallest elements in a dataset. diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py index 4d62eb1b11993..f5283c5e0dcb6 100644 --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -611,7 +611,7 @@ def testFsum(self): def msum(iterable): """Full precision summation. Compute sum(iterable) without any intermediate accumulation of error. Based on the 'lsum' function - at http://code.activestate.com/recipes/393090/ + at https://github.com/ActiveState/code/tree/master/recipes/Python/393090_Binary_floating_point_summatiaccurate_full/recipe-393090.py """ tmant, texp = 0, 0 diff --git a/Tools/peg_generator/pegen/sccutils.py b/Tools/peg_generator/pegen/sccutils.py index 1f0586bb2f7d6..0c295196607ec 100644 --- a/Tools/peg_generator/pegen/sccutils.py +++ b/Tools/peg_generator/pegen/sccutils.py @@ -18,7 +18,7 @@ def strongly_connected_components( exactly once; vertices not part of a SCC are returned as singleton sets. - From http://code.activestate.com/recipes/578507/. + From https://github.com/ActiveState/code/tree/master/recipes/Python/578507_Strongly_connected_components_directed/recipe-578507.py. """ identified: Set[str] = set() stack: List[str] = [] @@ -81,7 +81,7 @@ def topsort( {B, C} {A} - From http://code.activestate.com/recipes/577413/. + From https://github.com/ActiveState/code/tree/master/recipes/Python/577413_Topological_Sort/recipe-577413.py. """ # TODO: Use a faster algorithm? for k, v in data.items(): From webhook-mailer at python.org Tue Sep 15 17:58:41 2020 From: webhook-mailer at python.org (Batuhan Taskaya) Date: Tue, 15 Sep 2020 21:58:41 -0000 Subject: [Python-checkins] bpo-41780: Fix __dir__ of types.GenericAlias (GH-22262) Message-ID: https://github.com/python/cpython/commit/2e87774df1a0eaf2a1fe8cc4d958df60f7125b6e commit: 2e87774df1a0eaf2a1fe8cc4d958df60f7125b6e branch: master author: Batuhan Taskaya committer: GitHub date: 2020-09-15T14:58:32-07:00 summary: bpo-41780: Fix __dir__ of types.GenericAlias (GH-22262) Automerge-Triggered-By: @gvanrossum files: A Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst M Lib/test/test_genericalias.py M Objects/genericaliasobject.c diff --git a/Lib/test/test_genericalias.py b/Lib/test/test_genericalias.py index 1f24469471428..643fffc073e82 100644 --- a/Lib/test/test_genericalias.py +++ b/Lib/test/test_genericalias.py @@ -287,6 +287,11 @@ def test_union_generic(self): self.assertEqual(a.__args__, (list[T], tuple[T, ...])) self.assertEqual(a.__parameters__, (T,)) + def test_dir(self): + dir_of_gen_alias = set(dir(list[int])) + self.assertTrue(dir_of_gen_alias.issuperset(dir(list))) + for generic_alias_property in ("__origin__", "__args__", "__parameters__"): + self.assertIn(generic_alias_property, dir_of_gen_alias) if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst new file mode 100644 index 0000000000000..9a7594fc45338 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst @@ -0,0 +1,2 @@ +Fix :meth:`__dir__` of :class:`types.GenericAlias`. Patch by Batuhan +Taskaya. diff --git a/Objects/genericaliasobject.c b/Objects/genericaliasobject.c index 87bd1ae5c1430..ab56e1c4bf1a8 100644 --- a/Objects/genericaliasobject.c +++ b/Objects/genericaliasobject.c @@ -487,11 +487,50 @@ ga_reduce(PyObject *self, PyObject *Py_UNUSED(ignored)) alias->origin, alias->args); } +static PyObject * +ga_dir(PyObject *self, PyObject *Py_UNUSED(ignored)) +{ + gaobject *alias = (gaobject *)self; + PyObject *dir = PyObject_Dir(alias->origin); + if (dir == NULL) { + return NULL; + } + + PyObject *dir_entry = NULL; + for (const char * const *p = attr_exceptions; ; p++) { + if (*p == NULL) { + break; + } + else { + dir_entry = PyUnicode_FromString(*p); + if (dir_entry == NULL) { + goto error; + } + int contains = PySequence_Contains(dir, dir_entry); + if (contains < 0) { + goto error; + } + if (contains == 0 && PyList_Append(dir, dir_entry) < 0) { + goto error; + } + + Py_CLEAR(dir_entry); + } + } + return dir; + +error: + Py_DECREF(dir); + Py_XDECREF(dir_entry); + return NULL; +} + static PyMethodDef ga_methods[] = { {"__mro_entries__", ga_mro_entries, METH_O}, {"__instancecheck__", ga_instancecheck, METH_O}, {"__subclasscheck__", ga_subclasscheck, METH_O}, {"__reduce__", ga_reduce, METH_NOARGS}, + {"__dir__", ga_dir, METH_NOARGS}, {0} }; From webhook-mailer at python.org Tue Sep 15 18:56:46 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Tue, 15 Sep 2020 22:56:46 -0000 Subject: [Python-checkins] bpo-39587: Enum - use correct mixed-in data type (GH-22263) Message-ID: https://github.com/python/cpython/commit/bff01f3a3aac0c15fe8fbe8b2f561f7927d117a1 commit: bff01f3a3aac0c15fe8fbe8b2f561f7927d117a1 branch: master author: Ethan Furman committer: GitHub date: 2020-09-15T15:56:26-07:00 summary: bpo-39587: Enum - use correct mixed-in data type (GH-22263) files: A Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index bc24f2ae2dfc0..5e0088ee89fee 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -482,14 +482,25 @@ def _get_mixins_(bases): return object, Enum def _find_data_type(bases): + data_types = [] for chain in bases: + candidate = None for base in chain.__mro__: if base is object: continue elif '__new__' in base.__dict__: if issubclass(base, Enum): continue - return base + data_types.append(candidate or base) + break + elif not issubclass(base, Enum): + candidate = base + if len(data_types) > 1: + raise TypeError('too many data types: %r' % data_types) + elif data_types: + return data_types[0] + else: + return None # ensure final parent class is an Enum derivative, find any concrete # data type, and check that Enum has no members diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 138f572906a47..a909c108833fc 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -560,6 +560,49 @@ def test_format_enum_str(self): self.assertFormatIsValue('{:>20}', Directional.WEST) self.assertFormatIsValue('{:<20}', Directional.WEST) + def test_enum_str_override(self): + class MyStrEnum(Enum): + def __str__(self): + return 'MyStr' + class MyMethodEnum(Enum): + def hello(self): + return 'Hello! My name is %s' % self.name + class Test1Enum(MyMethodEnum, int, MyStrEnum): + One = 1 + Two = 2 + self.assertEqual(str(Test1Enum.One), 'MyStr') + # + class Test2Enum(MyStrEnum, MyMethodEnum): + One = 1 + Two = 2 + self.assertEqual(str(Test2Enum.One), 'MyStr') + + def test_inherited_data_type(self): + class HexInt(int): + def __repr__(self): + return hex(self) + class MyEnum(HexInt, enum.Enum): + A = 1 + B = 2 + C = 3 + self.assertEqual(repr(MyEnum.A), '') + + def test_too_many_data_types(self): + with self.assertRaisesRegex(TypeError, 'too many data types'): + class Huh(str, int, Enum): + One = 1 + + class MyStr(str): + def hello(self): + return 'hello, %s' % self + class MyInt(int): + def repr(self): + return hex(self) + with self.assertRaisesRegex(TypeError, 'too many data types'): + class Huh(MyStr, MyInt, Enum): + One = 1 + + def test_hash(self): Season = self.Season dates = {} diff --git a/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst b/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst new file mode 100644 index 0000000000000..e2f2b64867bed --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst @@ -0,0 +1 @@ +use the correct mix-in data type when constructing Enums From webhook-mailer at python.org Tue Sep 15 19:24:09 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 23:24:09 -0000 Subject: [Python-checkins] bpo-39587: Enum - use correct mixed-in data type (GH-22263) Message-ID: https://github.com/python/cpython/commit/8f8ebcca95d3b6ed0a522a9736ab53d6d4f0208c commit: 8f8ebcca95d3b6ed0a522a9736ab53d6d4f0208c branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-15T16:23:57-07:00 summary: bpo-39587: Enum - use correct mixed-in data type (GH-22263) (cherry picked from commit bff01f3a3aac0c15fe8fbe8b2f561f7927d117a1) Co-authored-by: Ethan Furman files: A Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 16033b1c6fac5..dfde75048b0f0 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -487,14 +487,25 @@ def _get_mixins_(bases): return object, Enum def _find_data_type(bases): + data_types = [] for chain in bases: + candidate = None for base in chain.__mro__: if base is object: continue elif '__new__' in base.__dict__: if issubclass(base, Enum): continue - return base + data_types.append(candidate or base) + break + elif not issubclass(base, Enum): + candidate = base + if len(data_types) > 1: + raise TypeError('too many data types: %r' % data_types) + elif data_types: + return data_types[0] + else: + return None # ensure final parent class is an Enum derivative, find any concrete # data type, and check that Enum has no members diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 0f91f00d92dd9..13f7ba4949fcc 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -551,6 +551,49 @@ def test_format_enum_str(self): self.assertFormatIsValue('{:>20}', Directional.WEST) self.assertFormatIsValue('{:<20}', Directional.WEST) + def test_enum_str_override(self): + class MyStrEnum(Enum): + def __str__(self): + return 'MyStr' + class MyMethodEnum(Enum): + def hello(self): + return 'Hello! My name is %s' % self.name + class Test1Enum(MyMethodEnum, int, MyStrEnum): + One = 1 + Two = 2 + self.assertEqual(str(Test1Enum.One), 'MyStr') + # + class Test2Enum(MyStrEnum, MyMethodEnum): + One = 1 + Two = 2 + self.assertEqual(str(Test2Enum.One), 'MyStr') + + def test_inherited_data_type(self): + class HexInt(int): + def __repr__(self): + return hex(self) + class MyEnum(HexInt, enum.Enum): + A = 1 + B = 2 + C = 3 + self.assertEqual(repr(MyEnum.A), '') + + def test_too_many_data_types(self): + with self.assertRaisesRegex(TypeError, 'too many data types'): + class Huh(str, int, Enum): + One = 1 + + class MyStr(str): + def hello(self): + return 'hello, %s' % self + class MyInt(int): + def repr(self): + return hex(self) + with self.assertRaisesRegex(TypeError, 'too many data types'): + class Huh(MyStr, MyInt, Enum): + One = 1 + + def test_hash(self): Season = self.Season dates = {} diff --git a/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst b/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst new file mode 100644 index 0000000000000..e2f2b64867bed --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst @@ -0,0 +1 @@ +use the correct mix-in data type when constructing Enums From webhook-mailer at python.org Tue Sep 15 19:26:15 2020 From: webhook-mailer at python.org (Tim Burke) Date: Tue, 15 Sep 2020 23:26:15 -0000 Subject: [Python-checkins] Doc: Fix broken manpage link (GH-21937) Message-ID: https://github.com/python/cpython/commit/47f6ec4c09a138e9049fd245ca312842ff50ce42 commit: 47f6ec4c09a138e9049fd245ca312842ff50ce42 branch: master author: Tim Burke committer: GitHub date: 2020-09-15T20:26:06-03:00 summary: Doc: Fix broken manpage link (GH-21937) sigprocmask is in section 2, not 3. files: M Doc/library/signal.rst diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst index 05b285ed110ec..00a730b6b9ca6 100644 --- a/Doc/library/signal.rst +++ b/Doc/library/signal.rst @@ -416,7 +416,7 @@ The :mod:`signal` module defines the following functions: :data:`SIGKILL` and :data:`SIGSTOP` cannot be blocked. - .. availability:: Unix. See the man page :manpage:`sigprocmask(3)` and + .. availability:: Unix. See the man page :manpage:`sigprocmask(2)` and :manpage:`pthread_sigmask(3)` for further information. See also :func:`pause`, :func:`sigpending` and :func:`sigwait`. From webhook-mailer at python.org Tue Sep 15 19:28:34 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Tue, 15 Sep 2020 23:28:34 -0000 Subject: [Python-checkins] bpo-41789: honor object overrides in Enum classes (GH-22250) Message-ID: https://github.com/python/cpython/commit/22415ad62555d79bd583b4a7d6a96006624a8277 commit: 22415ad62555d79bd583b4a7d6a96006624a8277 branch: master author: Ethan Furman committer: GitHub date: 2020-09-15T16:28:25-07:00 summary: bpo-41789: honor object overrides in Enum classes (GH-22250) EnumMeta double-checks that `__repr__`, `__str__`, `__format__`, and `__reduce_ex__` are not the same as `object`'s, and replaces them if they are -- even if that replacement was intentionally done in the Enum being constructed. This patch fixes that. Automerge-Triggered-By: @ethanfurman files: A Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 5e0088ee89fee..e72d306267461 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -250,7 +250,11 @@ def __new__(metacls, cls, bases, classdict): # double check that repr and friends are not the mixin's or various # things break (such as pickle) + # however, if the method is defined in the Enum itself, don't replace + # it for name in ('__repr__', '__str__', '__format__', '__reduce_ex__'): + if name in classdict: + continue class_method = getattr(enum_class, name) obj_method = getattr(member_type, name, None) enum_method = getattr(first_enum, name, None) diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index a909c108833fc..865edf1d9cfc6 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -560,6 +560,14 @@ def test_format_enum_str(self): self.assertFormatIsValue('{:>20}', Directional.WEST) self.assertFormatIsValue('{:<20}', Directional.WEST) + def test_object_str_override(self): + class Colors(Enum): + RED, GREEN, BLUE = 1, 2, 3 + def __repr__(self): + return "test.%s" % (self._name_, ) + __str__ = object.__str__ + self.assertEqual(str(Colors.RED), 'test.RED') + def test_enum_str_override(self): class MyStrEnum(Enum): def __str__(self): @@ -602,7 +610,6 @@ def repr(self): class Huh(MyStr, MyInt, Enum): One = 1 - def test_hash(self): Season = self.Season dates = {} diff --git a/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst b/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst new file mode 100644 index 0000000000000..5ce7a3ca67b72 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst @@ -0,0 +1,2 @@ +Honor `object` overrides in `Enum` class creation (specifically, `__str__`, +`__repr__`, `__format__`, and `__reduce_ex__`). From webhook-mailer at python.org Tue Sep 15 19:37:22 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 23:37:22 -0000 Subject: [Python-checkins] bpo-41780: Fix __dir__ of types.GenericAlias (GH-22262) Message-ID: https://github.com/python/cpython/commit/49917d576a578c8c29b8dbcbd5257c366a53d498 commit: 49917d576a578c8c29b8dbcbd5257c366a53d498 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-15T16:37:13-07:00 summary: bpo-41780: Fix __dir__ of types.GenericAlias (GH-22262) Automerge-Triggered-By: @gvanrossum (cherry picked from commit 2e87774df1a0eaf2a1fe8cc4d958df60f7125b6e) Co-authored-by: Batuhan Taskaya files: A Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst M Lib/test/test_genericalias.py M Objects/genericaliasobject.c diff --git a/Lib/test/test_genericalias.py b/Lib/test/test_genericalias.py index 4f3798e8f87d8..ec1acd475ce5a 100644 --- a/Lib/test/test_genericalias.py +++ b/Lib/test/test_genericalias.py @@ -283,6 +283,11 @@ def test_union_generic(self): self.assertEqual(a.__args__, (list[T], tuple[T, ...])) self.assertEqual(a.__parameters__, (T,)) + def test_dir(self): + dir_of_gen_alias = set(dir(list[int])) + self.assertTrue(dir_of_gen_alias.issuperset(dir(list))) + for generic_alias_property in ("__origin__", "__args__", "__parameters__"): + self.assertIn(generic_alias_property, dir_of_gen_alias) if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst new file mode 100644 index 0000000000000..9a7594fc45338 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst @@ -0,0 +1,2 @@ +Fix :meth:`__dir__` of :class:`types.GenericAlias`. Patch by Batuhan +Taskaya. diff --git a/Objects/genericaliasobject.c b/Objects/genericaliasobject.c index 4d511a239063c..4f952162ec930 100644 --- a/Objects/genericaliasobject.c +++ b/Objects/genericaliasobject.c @@ -487,11 +487,50 @@ ga_reduce(PyObject *self, PyObject *Py_UNUSED(ignored)) alias->origin, alias->args); } +static PyObject * +ga_dir(PyObject *self, PyObject *Py_UNUSED(ignored)) +{ + gaobject *alias = (gaobject *)self; + PyObject *dir = PyObject_Dir(alias->origin); + if (dir == NULL) { + return NULL; + } + + PyObject *dir_entry = NULL; + for (const char * const *p = attr_exceptions; ; p++) { + if (*p == NULL) { + break; + } + else { + dir_entry = PyUnicode_FromString(*p); + if (dir_entry == NULL) { + goto error; + } + int contains = PySequence_Contains(dir, dir_entry); + if (contains < 0) { + goto error; + } + if (contains == 0 && PyList_Append(dir, dir_entry) < 0) { + goto error; + } + + Py_CLEAR(dir_entry); + } + } + return dir; + +error: + Py_DECREF(dir); + Py_XDECREF(dir_entry); + return NULL; +} + static PyMethodDef ga_methods[] = { {"__mro_entries__", ga_mro_entries, METH_O}, {"__instancecheck__", ga_instancecheck, METH_O}, {"__subclasscheck__", ga_subclasscheck, METH_O}, {"__reduce__", ga_reduce, METH_NOARGS}, + {"__dir__", ga_dir, METH_NOARGS}, {0} }; From webhook-mailer at python.org Tue Sep 15 19:59:58 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 15 Sep 2020 23:59:58 -0000 Subject: [Python-checkins] bpo-39587: Enum - use correct mixed-in data type (GH-22263) (GH-22266) Message-ID: https://github.com/python/cpython/commit/95b81e2f8c955823dbf5632a817902b8a4916eaa commit: 95b81e2f8c955823dbf5632a817902b8a4916eaa branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-15T16:59:48-07:00 summary: bpo-39587: Enum - use correct mixed-in data type (GH-22263) (GH-22266) (cherry picked from commit bff01f3a3aac0c15fe8fbe8b2f561f7927d117a1) files: A Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 49b552ba0ecf6..9dfba3c37cd2c 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -481,14 +481,25 @@ def _get_mixins_(bases): return object, Enum def _find_data_type(bases): + data_types = [] for chain in bases: + candidate = None for base in chain.__mro__: if base is object: continue elif '__new__' in base.__dict__: if issubclass(base, Enum): continue - return base + data_types.append(candidate or base) + break + elif not issubclass(base, Enum): + candidate = base + if len(data_types) > 1: + raise TypeError('too many data types: %r' % data_types) + elif data_types: + return data_types[0] + else: + return None # ensure final parent class is an Enum derivative, find any concrete # data type, and check that Enum has no members diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 1df0313da0a7e..b505a6ed13423 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -552,6 +552,49 @@ def test_format_enum_str(self): self.assertFormatIsValue('{:>20}', Directional.WEST) self.assertFormatIsValue('{:<20}', Directional.WEST) + def test_enum_str_override(self): + class MyStrEnum(Enum): + def __str__(self): + return 'MyStr' + class MyMethodEnum(Enum): + def hello(self): + return 'Hello! My name is %s' % self.name + class Test1Enum(MyMethodEnum, int, MyStrEnum): + One = 1 + Two = 2 + self.assertEqual(str(Test1Enum.One), 'MyStr') + # + class Test2Enum(MyStrEnum, MyMethodEnum): + One = 1 + Two = 2 + self.assertEqual(str(Test2Enum.One), 'MyStr') + + def test_inherited_data_type(self): + class HexInt(int): + def __repr__(self): + return hex(self) + class MyEnum(HexInt, enum.Enum): + A = 1 + B = 2 + C = 3 + self.assertEqual(repr(MyEnum.A), '') + + def test_too_many_data_types(self): + with self.assertRaisesRegex(TypeError, 'too many data types'): + class Huh(str, int, Enum): + One = 1 + + class MyStr(str): + def hello(self): + return 'hello, %s' % self + class MyInt(int): + def repr(self): + return hex(self) + with self.assertRaisesRegex(TypeError, 'too many data types'): + class Huh(MyStr, MyInt, Enum): + One = 1 + + def test_hash(self): Season = self.Season dates = {} diff --git a/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst b/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst new file mode 100644 index 0000000000000..e2f2b64867bed --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst @@ -0,0 +1 @@ +use the correct mix-in data type when constructing Enums From webhook-mailer at python.org Tue Sep 15 20:16:47 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 16 Sep 2020 00:16:47 -0000 Subject: [Python-checkins] bpo-41789: honor object overrides in Enum classes (GH-22250) Message-ID: https://github.com/python/cpython/commit/929112ef81ccef20d3aef25c8a1142059ee941da commit: 929112ef81ccef20d3aef25c8a1142059ee941da branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-15T17:16:36-07:00 summary: bpo-41789: honor object overrides in Enum classes (GH-22250) EnumMeta double-checks that `__repr__`, `__str__`, `__format__`, and `__reduce_ex__` are not the same as `object`'s, and replaces them if they are -- even if that replacement was intentionally done in the Enum being constructed. This patch fixes that. Automerge-Triggered-By: @ethanfurman (cherry picked from commit 22415ad62555d79bd583b4a7d6a96006624a8277) Co-authored-by: Ethan Furman files: A Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index dfde75048b0f0..c892d738f8b38 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -249,7 +249,11 @@ def __new__(metacls, cls, bases, classdict): # double check that repr and friends are not the mixin's or various # things break (such as pickle) + # however, if the method is defined in the Enum itself, don't replace + # it for name in ('__repr__', '__str__', '__format__', '__reduce_ex__'): + if name in classdict: + continue class_method = getattr(enum_class, name) obj_method = getattr(member_type, name, None) enum_method = getattr(first_enum, name, None) diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 13f7ba4949fcc..32ff32cf17567 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -551,6 +551,14 @@ def test_format_enum_str(self): self.assertFormatIsValue('{:>20}', Directional.WEST) self.assertFormatIsValue('{:<20}', Directional.WEST) + def test_object_str_override(self): + class Colors(Enum): + RED, GREEN, BLUE = 1, 2, 3 + def __repr__(self): + return "test.%s" % (self._name_, ) + __str__ = object.__str__ + self.assertEqual(str(Colors.RED), 'test.RED') + def test_enum_str_override(self): class MyStrEnum(Enum): def __str__(self): @@ -593,7 +601,6 @@ def repr(self): class Huh(MyStr, MyInt, Enum): One = 1 - def test_hash(self): Season = self.Season dates = {} diff --git a/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst b/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst new file mode 100644 index 0000000000000..5ce7a3ca67b72 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst @@ -0,0 +1,2 @@ +Honor `object` overrides in `Enum` class creation (specifically, `__str__`, +`__repr__`, `__format__`, and `__reduce_ex__`). From webhook-mailer at python.org Wed Sep 16 00:58:40 2020 From: webhook-mailer at python.org (Patrick Reader) Date: Wed, 16 Sep 2020 04:58:40 -0000 Subject: [Python-checkins] bpo-41792: Add is_typeddict function to typing.py (GH-22254) Message-ID: https://github.com/python/cpython/commit/0705ec8a149e27023b0420ae0366d16255f6c9f7 commit: 0705ec8a149e27023b0420ae0366d16255f6c9f7 branch: master author: Patrick Reader committer: GitHub date: 2020-09-15T21:58:32-07:00 summary: bpo-41792: Add is_typeddict function to typing.py (GH-22254) Closes issue41792. Also closes https://github.com/python/typing/issues/751. files: A Misc/NEWS.d/next/Library/2020-09-15-07-55-35.bpo-41792.qMpSlU.rst M Doc/library/typing.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 3125ae97808a9..bfff81e26760f 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1658,6 +1658,20 @@ Introspection helpers .. versionadded:: 3.8 +.. function:: is_typeddict(tp) + + Check if an annotation is a TypedDict class. + + For example:: + class Film(TypedDict): + title: str + year: int + + is_typeddict(Film) # => True + is_typeddict(Union[list, str]) # => False + + .. versionadded:: 3.10 + .. class:: ForwardRef A class used for internal typing representation of string forward references. diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 05140fc61b9bf..42aa430c5e107 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -16,6 +16,7 @@ from typing import cast, runtime_checkable from typing import get_type_hints from typing import get_origin, get_args +from typing import is_typeddict from typing import no_type_check, no_type_check_decorator from typing import Type from typing import NewType @@ -3900,6 +3901,12 @@ class Cat(Animal): 'voice': str, } + def test_is_typeddict(self): + assert is_typeddict(Point2D) is True + assert is_typeddict(Union[str, int]) is False + # classes, not instances + assert is_typeddict(Point2D()) is False + class IOTests(BaseTestCase): diff --git a/Lib/typing.py b/Lib/typing.py index 2aedbeb852a71..8c61bd8e084a8 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -103,6 +103,7 @@ 'get_args', 'get_origin', 'get_type_hints', + 'is_typeddict', 'NewType', 'no_type_check', 'no_type_check_decorator', @@ -1479,6 +1480,20 @@ def get_args(tp): return () +def is_typeddict(tp): + """Check if an annotation is a TypedDict class + + For example:: + class Film(TypedDict): + title: str + year: int + + is_typeddict(Film) # => True + is_typeddict(Union[list, str]) # => False + """ + return isinstance(tp, _TypedDictMeta) + + def no_type_check(arg): """Decorator to indicate that annotations are not type hints. diff --git a/Misc/NEWS.d/next/Library/2020-09-15-07-55-35.bpo-41792.qMpSlU.rst b/Misc/NEWS.d/next/Library/2020-09-15-07-55-35.bpo-41792.qMpSlU.rst new file mode 100644 index 0000000000000..fbbc6724ba51e --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-15-07-55-35.bpo-41792.qMpSlU.rst @@ -0,0 +1,6 @@ +Add is_typeddict function to typing.py to check if a type is a TypedDict +class + +Previously there was no way to check that without using private API. See the +`relevant issue in python/typing +` From webhook-mailer at python.org Wed Sep 16 06:58:54 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Wed, 16 Sep 2020 10:58:54 -0000 Subject: [Python-checkins] [3.9] bpo-41789: honor object overrides in Enum classes (GH-22250) (GH-22272) Message-ID: https://github.com/python/cpython/commit/a4677068dd61662f5a56b184d5e3aa07db65b88e commit: a4677068dd61662f5a56b184d5e3aa07db65b88e branch: 3.9 author: Ethan Furman committer: GitHub date: 2020-09-16T03:58:33-07:00 summary: [3.9] bpo-41789: honor object overrides in Enum classes (GH-22250) (GH-22272) EnumMeta double-checks that `__repr__`, `__str__`, `__format__`, and `__reduce_ex__` are not the same as `object`'s, and replaces them if they are -- even if that replacement was intentionally done in the Enum being constructed. This patch fixes that. files: A Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 9dfba3c37cd2c..8711558448107 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -249,7 +249,11 @@ def __new__(metacls, cls, bases, classdict): # double check that repr and friends are not the mixin's or various # things break (such as pickle) + # however, if the method is defined in the Enum itself, don't replace + # it for name in ('__repr__', '__str__', '__format__', '__reduce_ex__'): + if name in classdict: + continue class_method = getattr(enum_class, name) obj_method = getattr(member_type, name, None) enum_method = getattr(first_enum, name, None) diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index b505a6ed13423..0983513d58147 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -552,6 +552,14 @@ def test_format_enum_str(self): self.assertFormatIsValue('{:>20}', Directional.WEST) self.assertFormatIsValue('{:<20}', Directional.WEST) + def test_object_str_override(self): + class Colors(Enum): + RED, GREEN, BLUE = 1, 2, 3 + def __repr__(self): + return "test.%s" % (self._name_, ) + __str__ = object.__str__ + self.assertEqual(str(Colors.RED), 'test.RED') + def test_enum_str_override(self): class MyStrEnum(Enum): def __str__(self): @@ -594,7 +602,6 @@ def repr(self): class Huh(MyStr, MyInt, Enum): One = 1 - def test_hash(self): Season = self.Season dates = {} diff --git a/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst b/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst new file mode 100644 index 0000000000000..5ce7a3ca67b72 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst @@ -0,0 +1,2 @@ +Honor `object` overrides in `Enum` class creation (specifically, `__str__`, +`__repr__`, `__format__`, and `__reduce_ex__`). From webhook-mailer at python.org Wed Sep 16 07:52:53 2020 From: webhook-mailer at python.org (=?utf-8?q?=C5=81ukasz?= Langa) Date: Wed, 16 Sep 2020 11:52:53 -0000 Subject: [Python-checkins] [3.9] bpo-41687: Fix sendfile implementation to work with Solaris (GH-22040) (GH-22273) Message-ID: https://github.com/python/cpython/commit/7e356f17e4c91392b6fa45a512efc95923388813 commit: 7e356f17e4c91392b6fa45a512efc95923388813 branch: 3.9 author: ?ukasz Langa committer: GitHub date: 2020-09-16T13:52:26+02:00 summary: [3.9] bpo-41687: Fix sendfile implementation to work with Solaris (GH-22040) (GH-22273) (cherry picked from commit 8c0be6fd9101746235b63ddfb84106d1e9ca286b) Co-authored-by: Jakub Kul?k files: A Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rst M Lib/test/test_asyncio/test_sendfile.py M Modules/posixmodule.c diff --git a/Lib/test/test_asyncio/test_sendfile.py b/Lib/test/test_asyncio/test_sendfile.py index dbce199a9b8e1..1b1af08db0a13 100644 --- a/Lib/test/test_asyncio/test_sendfile.py +++ b/Lib/test/test_asyncio/test_sendfile.py @@ -445,6 +445,12 @@ def test_sendfile_ssl_close_peer_after_receiving(self): self.assertEqual(srv_proto.data, self.DATA) self.assertEqual(self.file.tell(), len(self.DATA)) + # On Solaris, lowering SO_RCVBUF on a TCP connection after it has been + # established has no effect. Due to its age, this bug affects both Oracle + # Solaris as well as all other OpenSolaris forks (unless they fixed it + # themselves). + @unittest.skipIf(sys.platform.startswith('sunos'), + "Doesn't work on Solaris") def test_sendfile_close_peer_in_the_middle_of_receiving(self): srv_proto, cli_proto = self.prepare_sendfile(close_after=1024) with self.assertRaises(ConnectionError): diff --git a/Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rst b/Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rst new file mode 100644 index 0000000000000..284f500735701 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rst @@ -0,0 +1 @@ +Fix implementation of sendfile to be compatible with Solaris. diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index ddff28354a7c1..39f5f577a8286 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -9453,6 +9453,25 @@ os_sendfile_impl(PyObject *module, int out_fd, int in_fd, PyObject *offobj, if (!Py_off_t_converter(offobj, &offset)) return NULL; +#if defined(__sun) && defined(__SVR4) + // On Solaris, sendfile raises EINVAL rather than returning 0 + // when the offset is equal or bigger than the in_fd size. + int res; + struct stat st; + + do { + Py_BEGIN_ALLOW_THREADS + res = fstat(in_fd, &st); + Py_END_ALLOW_THREADS + } while (res != 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); + if (ret < 0) + return (!async_err) ? posix_error() : NULL; + + if (offset >= st.st_size) { + return Py_BuildValue("i", 0); + } +#endif + do { Py_BEGIN_ALLOW_THREADS ret = sendfile(out_fd, in_fd, &offset, count); From webhook-mailer at python.org Wed Sep 16 08:25:17 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 16 Sep 2020 12:25:17 -0000 Subject: [Python-checkins] bpo-41687: Fix error handling in Solaris sendfile implementation (GH-22128) Message-ID: https://github.com/python/cpython/commit/6a39888c2c4bcfbcdc61a1953911ad30c62da1ef commit: 6a39888c2c4bcfbcdc61a1953911ad30c62da1ef branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-16T05:25:09-07:00 summary: bpo-41687: Fix error handling in Solaris sendfile implementation (GH-22128) I just realized that my recent PR with sendfile on Solaris ([PR 22040](https://github.com/python/cpython/pull/22040)) has broken error handling. Sorry for that, this simple followup fixes that. Automerge-Triggered-By: @1st1 (cherry picked from commit fa8c9e70104b0aef966a518eb3a80a4881906ae0) Co-authored-by: Jakub Kul?k files: M Modules/posixmodule.c diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 39f5f577a8286..01e8bcbd2981a 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -9456,14 +9456,13 @@ os_sendfile_impl(PyObject *module, int out_fd, int in_fd, PyObject *offobj, #if defined(__sun) && defined(__SVR4) // On Solaris, sendfile raises EINVAL rather than returning 0 // when the offset is equal or bigger than the in_fd size. - int res; struct stat st; do { Py_BEGIN_ALLOW_THREADS - res = fstat(in_fd, &st); + ret = fstat(in_fd, &st); Py_END_ALLOW_THREADS - } while (res != 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); + } while (ret != 0 && errno == EINTR && !(async_err = PyErr_CheckSignals())); if (ret < 0) return (!async_err) ? posix_error() : NULL; From webhook-mailer at python.org Wed Sep 16 10:12:05 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Wed, 16 Sep 2020 14:12:05 -0000 Subject: [Python-checkins] bpo-41517: do not allow Enums to be extended (#22271) Message-ID: https://github.com/python/cpython/commit/3064dbf5df1021e85b507366a7ea448c8895efe7 commit: 3064dbf5df1021e85b507366a7ea448c8895efe7 branch: master author: Ethan Furman committer: GitHub date: 2020-09-16T07:11:57-07:00 summary: bpo-41517: do not allow Enums to be extended (#22271) fix bug that let Enums be extended via multiple inheritance files: A Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index e72d306267461..0c2cf569fac88 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -124,10 +124,12 @@ class EnumMeta(type): """Metaclass for Enum""" @classmethod def __prepare__(metacls, cls, bases): + # check that previous enum members do not exist + metacls._check_for_existing_members(cls, bases) # create the namespace dict enum_dict = _EnumDict() # inherit previous flags and _generate_next_value_ function - member_type, first_enum = metacls._get_mixins_(bases) + member_type, first_enum = metacls._get_mixins_(cls, bases) if first_enum is not None: enum_dict['_generate_next_value_'] = getattr(first_enum, '_generate_next_value_', None) return enum_dict @@ -143,7 +145,7 @@ def __new__(metacls, cls, bases, classdict): ignore = classdict['_ignore_'] for key in ignore: classdict.pop(key, None) - member_type, first_enum = metacls._get_mixins_(bases) + member_type, first_enum = metacls._get_mixins_(cls, bases) __new__, save_new, use_args = metacls._find_new_(classdict, member_type, first_enum) @@ -402,7 +404,7 @@ def _create_(cls, class_name, names, *, module=None, qualname=None, type=None, s """ metacls = cls.__class__ bases = (cls, ) if type is None else (type, cls) - _, first_enum = cls._get_mixins_(bases) + _, first_enum = cls._get_mixins_(cls, bases) classdict = metacls.__prepare__(class_name, bases) # special processing needed for names? @@ -475,7 +477,14 @@ def _convert_(cls, name, module, filter, source=None): return cls @staticmethod - def _get_mixins_(bases): + def _check_for_existing_members(class_name, bases): + for chain in bases: + for base in chain.__mro__: + if issubclass(base, Enum) and base._member_names_: + raise TypeError("%s: cannot extend enumeration %r" % (class_name, base.__name__)) + + @staticmethod + def _get_mixins_(class_name, bases): """Returns the type for creating enum members, and the first inherited enum class. @@ -500,7 +509,7 @@ def _find_data_type(bases): elif not issubclass(base, Enum): candidate = base if len(data_types) > 1: - raise TypeError('too many data types: %r' % data_types) + raise TypeError('%r: too many data types: %r' % (class_name, data_types)) elif data_types: return data_types[0] else: diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 865edf1d9cfc6..2fcd047989afb 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1009,6 +1009,9 @@ class MoreColor(Color): cyan = 4 magenta = 5 yellow = 6 + with self.assertRaisesRegex(TypeError, "EvenMoreColor: cannot extend enumeration 'Color'"): + class EvenMoreColor(Color, IntEnum): + chartruese = 7 def test_exclude_methods(self): class whatever(Enum): diff --git a/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst b/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst new file mode 100644 index 0000000000000..e7654711062ce --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst @@ -0,0 +1 @@ +fix bug allowing Enums to be extended via multiple inheritance From webhook-mailer at python.org Wed Sep 16 10:31:12 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 16 Sep 2020 14:31:12 -0000 Subject: [Python-checkins] bpo-41517: do not allow Enums to be extended (GH-22271) Message-ID: https://github.com/python/cpython/commit/3f4012117bf80aa7c005f8fa6fb8e1f8b1aef5d5 commit: 3f4012117bf80aa7c005f8fa6fb8e1f8b1aef5d5 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-16T07:30:54-07:00 summary: bpo-41517: do not allow Enums to be extended (GH-22271) fix bug that let Enums be extended via multiple inheritance (cherry picked from commit 3064dbf5df1021e85b507366a7ea448c8895efe7) Co-authored-by: Ethan Furman files: A Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index c892d738f8b38..4c1acc6e360bd 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -123,10 +123,12 @@ class EnumMeta(type): """Metaclass for Enum""" @classmethod def __prepare__(metacls, cls, bases): + # check that previous enum members do not exist + metacls._check_for_existing_members(cls, bases) # create the namespace dict enum_dict = _EnumDict() # inherit previous flags and _generate_next_value_ function - member_type, first_enum = metacls._get_mixins_(bases) + member_type, first_enum = metacls._get_mixins_(cls, bases) if first_enum is not None: enum_dict['_generate_next_value_'] = getattr(first_enum, '_generate_next_value_', None) return enum_dict @@ -142,7 +144,7 @@ def __new__(metacls, cls, bases, classdict): ignore = classdict['_ignore_'] for key in ignore: classdict.pop(key, None) - member_type, first_enum = metacls._get_mixins_(bases) + member_type, first_enum = metacls._get_mixins_(cls, bases) __new__, save_new, use_args = metacls._find_new_(classdict, member_type, first_enum) @@ -401,7 +403,7 @@ def _create_(cls, class_name, names, *, module=None, qualname=None, type=None, s """ metacls = cls.__class__ bases = (cls, ) if type is None else (type, cls) - _, first_enum = cls._get_mixins_(bases) + _, first_enum = cls._get_mixins_(cls, bases) classdict = metacls.__prepare__(class_name, bases) # special processing needed for names? @@ -480,7 +482,14 @@ def _convert(cls, *args, **kwargs): return cls._convert_(*args, **kwargs) @staticmethod - def _get_mixins_(bases): + def _check_for_existing_members(class_name, bases): + for chain in bases: + for base in chain.__mro__: + if issubclass(base, Enum) and base._member_names_: + raise TypeError("%s: cannot extend enumeration %r" % (class_name, base.__name__)) + + @staticmethod + def _get_mixins_(class_name, bases): """Returns the type for creating enum members, and the first inherited enum class. @@ -505,7 +514,7 @@ def _find_data_type(bases): elif not issubclass(base, Enum): candidate = base if len(data_types) > 1: - raise TypeError('too many data types: %r' % data_types) + raise TypeError('%r: too many data types: %r' % (class_name, data_types)) elif data_types: return data_types[0] else: diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 32ff32cf17567..11e88dba6b5b3 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1000,6 +1000,9 @@ class MoreColor(Color): cyan = 4 magenta = 5 yellow = 6 + with self.assertRaisesRegex(TypeError, "EvenMoreColor: cannot extend enumeration 'Color'"): + class EvenMoreColor(Color, IntEnum): + chartruese = 7 def test_exclude_methods(self): class whatever(Enum): diff --git a/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst b/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst new file mode 100644 index 0000000000000..e7654711062ce --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst @@ -0,0 +1 @@ +fix bug allowing Enums to be extended via multiple inheritance From webhook-mailer at python.org Wed Sep 16 10:35:23 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 16 Sep 2020 14:35:23 -0000 Subject: [Python-checkins] bpo-41517: do not allow Enums to be extended (GH-22271) Message-ID: https://github.com/python/cpython/commit/48f99250ff319e36b15b555128cd62e408d8165f commit: 48f99250ff319e36b15b555128cd62e408d8165f branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-16T07:35:14-07:00 summary: bpo-41517: do not allow Enums to be extended (GH-22271) fix bug that let Enums be extended via multiple inheritance (cherry picked from commit 3064dbf5df1021e85b507366a7ea448c8895efe7) Co-authored-by: Ethan Furman files: A Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 8711558448107..bdbfbb69bdea6 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -123,10 +123,12 @@ class EnumMeta(type): """Metaclass for Enum""" @classmethod def __prepare__(metacls, cls, bases): + # check that previous enum members do not exist + metacls._check_for_existing_members(cls, bases) # create the namespace dict enum_dict = _EnumDict() # inherit previous flags and _generate_next_value_ function - member_type, first_enum = metacls._get_mixins_(bases) + member_type, first_enum = metacls._get_mixins_(cls, bases) if first_enum is not None: enum_dict['_generate_next_value_'] = getattr(first_enum, '_generate_next_value_', None) return enum_dict @@ -142,7 +144,7 @@ def __new__(metacls, cls, bases, classdict): ignore = classdict['_ignore_'] for key in ignore: classdict.pop(key, None) - member_type, first_enum = metacls._get_mixins_(bases) + member_type, first_enum = metacls._get_mixins_(cls, bases) __new__, save_new, use_args = metacls._find_new_(classdict, member_type, first_enum) @@ -401,7 +403,7 @@ def _create_(cls, class_name, names, *, module=None, qualname=None, type=None, s """ metacls = cls.__class__ bases = (cls, ) if type is None else (type, cls) - _, first_enum = cls._get_mixins_(bases) + _, first_enum = cls._get_mixins_(cls, bases) classdict = metacls.__prepare__(class_name, bases) # special processing needed for names? @@ -474,7 +476,14 @@ def _convert_(cls, name, module, filter, source=None): return cls @staticmethod - def _get_mixins_(bases): + def _check_for_existing_members(class_name, bases): + for chain in bases: + for base in chain.__mro__: + if issubclass(base, Enum) and base._member_names_: + raise TypeError("%s: cannot extend enumeration %r" % (class_name, base.__name__)) + + @staticmethod + def _get_mixins_(class_name, bases): """Returns the type for creating enum members, and the first inherited enum class. @@ -499,7 +508,7 @@ def _find_data_type(bases): elif not issubclass(base, Enum): candidate = base if len(data_types) > 1: - raise TypeError('too many data types: %r' % data_types) + raise TypeError('%r: too many data types: %r' % (class_name, data_types)) elif data_types: return data_types[0] else: diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 0983513d58147..0ecf6413b1313 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1001,6 +1001,9 @@ class MoreColor(Color): cyan = 4 magenta = 5 yellow = 6 + with self.assertRaisesRegex(TypeError, "EvenMoreColor: cannot extend enumeration 'Color'"): + class EvenMoreColor(Color, IntEnum): + chartruese = 7 def test_exclude_methods(self): class whatever(Enum): diff --git a/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst b/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst new file mode 100644 index 0000000000000..e7654711062ce --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst @@ -0,0 +1 @@ +fix bug allowing Enums to be extended via multiple inheritance From webhook-mailer at python.org Wed Sep 16 11:06:32 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Wed, 16 Sep 2020 15:06:32 -0000 Subject: [Python-checkins] [doc] Minor improvements to is_typeddict (GH-22280) Message-ID: https://github.com/python/cpython/commit/83f6dcd2070a5dbd9702b32ec4ee150bb716c9a1 commit: 83f6dcd2070a5dbd9702b32ec4ee150bb716c9a1 branch: master author: Andre Delfino committer: GitHub date: 2020-09-16T08:06:23-07:00 summary: [doc] Minor improvements to is_typeddict (GH-22280) 1. The check is on the type 2. Add link to TypeDict files: M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index bfff81e26760f..d31c65d38e1ab 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1660,7 +1660,7 @@ Introspection helpers .. function:: is_typeddict(tp) - Check if an annotation is a TypedDict class. + Check if a type is a :class:`TypedDict`. For example:: class Film(TypedDict): From webhook-mailer at python.org Wed Sep 16 13:26:58 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Wed, 16 Sep 2020 17:26:58 -0000 Subject: [Python-checkins] bpo-39728: Enum: fix duplicate `ValueError` (GH-22277) Message-ID: https://github.com/python/cpython/commit/c95ad7a91fbd7636f33a098d3b39964ab083bf49 commit: c95ad7a91fbd7636f33a098d3b39964ab083bf49 branch: master author: Ethan Furman committer: GitHub date: 2020-09-16T10:26:50-07:00 summary: bpo-39728: Enum: fix duplicate `ValueError` (GH-22277) fix default `_missing_` to return `None` instead of raising a `ValueError` Co-authored-by: Andrey Darascheka files: A Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst M Lib/enum.py M Lib/test/test_enum.py M Misc/ACKS diff --git a/Lib/enum.py b/Lib/enum.py index 0c2cf569fac88..060b2a0dadf45 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -629,7 +629,7 @@ def _generate_next_value_(name, start, count, last_values): @classmethod def _missing_(cls, value): - raise ValueError("%r is not a valid %s" % (value, cls.__qualname__)) + return None def __repr__(self): return "<%s.%s: %r>" % ( diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 2fcd047989afb..5d72d82cec27f 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1845,6 +1845,18 @@ class Dupes(Enum): third = auto() self.assertEqual([Dupes.first, Dupes.second, Dupes.third], list(Dupes)) + def test_default_missing(self): + class Color(Enum): + RED = 1 + GREEN = 2 + BLUE = 3 + try: + Color(7) + except ValueError as exc: + self.assertTrue(exc.__context__ is None) + else: + raise Exception('Exception not raised.') + def test_missing(self): class Color(Enum): red = 1 @@ -1863,7 +1875,12 @@ def _missing_(cls, item): # trigger not found return None self.assertIs(Color('three'), Color.blue) - self.assertRaises(ValueError, Color, 7) + try: + Color(7) + except ValueError as exc: + self.assertTrue(exc.__context__ is None) + else: + raise Exception('Exception not raised.') try: Color('bad return') except TypeError as exc: diff --git a/Misc/ACKS b/Misc/ACKS index 8b0d7a45da169..2628d6b690d1c 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -433,6 +433,7 @@ Marcos Donolo Dima Dorfman Yves Dorfsman Michael Dorman +Andrey Doroschenko Steve Dower Allen Downey Cesar Douady diff --git a/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst b/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst new file mode 100644 index 0000000000000..beb2016a85ba6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst @@ -0,0 +1 @@ +fix default `_missing_` so a duplicate `ValueError` is not set as the `__context__` of the original `ValueError` From webhook-mailer at python.org Wed Sep 16 14:37:32 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Wed, 16 Sep 2020 18:37:32 -0000 Subject: [Python-checkins] acknowledge Weipeng Hong's contributions (GH-22284) Message-ID: https://github.com/python/cpython/commit/5c1b46d897d4c693e2f3ae049d54725dfb09f2dc commit: 5c1b46d897d4c693e2f3ae049d54725dfb09f2dc branch: master author: Ethan Furman committer: GitHub date: 2020-09-16T11:37:24-07:00 summary: acknowledge Weipeng Hong's contributions (GH-22284) files: M Misc/ACKS diff --git a/Misc/ACKS b/Misc/ACKS index 2628d6b690d1c..d5bdb084a1e1e 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -736,6 +736,7 @@ Thomas Holmes Craig Holmquist Philip Homburg Naofumi Honda +Weipeng Hong Jeffrey Honig Rob Hooft Michiel de Hoon From webhook-mailer at python.org Wed Sep 16 14:42:09 2020 From: webhook-mailer at python.org (Pablo Galindo) Date: Wed, 16 Sep 2020 18:42:09 -0000 Subject: [Python-checkins] bpo-41746: Add type information to asdl_seq objects (GH-22223) Message-ID: https://github.com/python/cpython/commit/a5634c406767ef694df49b624adf9cfa6c0d9064 commit: a5634c406767ef694df49b624adf9cfa6c0d9064 branch: master author: Pablo Galindo committer: GitHub date: 2020-09-16T19:42:00+01:00 summary: bpo-41746: Add type information to asdl_seq objects (GH-22223) * Add new capability to the PEG parser to type variable assignments. For instance: ``` | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } ``` * Add new sequence types from the asdl definition (automatically generated) * Make `asdl_seq` type a generic aliasing pointer type. * Create a new `asdl_generic_seq` for the generic case using `void*`. * The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed. * New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences. * Changes all possible `asdl_seq` types to use specific versions everywhere. files: M Doc/tools/extensions/peg_highlight.py M Grammar/python.gram M Include/Python-ast.h M Include/asdl.h M Include/ast.h M Parser/asdl_c.py M Parser/parser.c M Parser/pegen.c M Parser/pegen.h M Parser/string_parser.c M Python/Python-ast.c M Python/asdl.c M Python/ast.c M Python/ast_opt.c M Python/ast_unparse.c M Python/compile.c M Python/future.c M Python/symtable.c M Tools/peg_generator/pegen/c_generator.py M Tools/peg_generator/pegen/grammar.py M Tools/peg_generator/pegen/grammar_parser.py M Tools/peg_generator/pegen/metagrammar.gram diff --git a/Doc/tools/extensions/peg_highlight.py b/Doc/tools/extensions/peg_highlight.py index 8bc24670fbe0a..9a2acb7f320ba 100644 --- a/Doc/tools/extensions/peg_highlight.py +++ b/Doc/tools/extensions/peg_highlight.py @@ -43,7 +43,7 @@ class PEGLexer(RegexLexer): (r"'\W+?'", Text), (r'"\W+?"', Text), ], - "variables": [(_name + _text_ws + "(=)", bygroups(None, None, None),),], + "variables": [(_name + _text_ws + r"(\[.*\])?" + _text_ws + "(=)", bygroups(None, None, None, None, None),),], "invalids": [ (r"^(\s+\|\s+invalid_\w+\s*\n)", bygroups(None)), (r"^(\s+\|\s+incorrect_\w+\s*\n)", bygroups(None)), diff --git a/Grammar/python.gram b/Grammar/python.gram index 524e88eb38996..e4533b1a1b879 100644 --- a/Grammar/python.gram +++ b/Grammar/python.gram @@ -34,27 +34,27 @@ func_type[mod_ty]: '(' a=[type_expressions] ')' '->' b=expression NEWLINE* ENDMA fstring[expr_ty]: star_expressions # type_expressions allow */** but ignore them -type_expressions[asdl_seq*]: +type_expressions[asdl_expr_seq*]: | a=','.expression+ ',' '*' b=expression ',' '**' c=expression { - _PyPegen_seq_append_to_end(p, CHECK(_PyPegen_seq_append_to_end(p, a, b)), c) } - | a=','.expression+ ',' '*' b=expression { _PyPegen_seq_append_to_end(p, a, b) } - | a=','.expression+ ',' '**' b=expression { _PyPegen_seq_append_to_end(p, a, b) } + (asdl_expr_seq*)_PyPegen_seq_append_to_end(p, CHECK(_PyPegen_seq_append_to_end(p, a, b)), c) } + | a=','.expression+ ',' '*' b=expression { (asdl_expr_seq*)_PyPegen_seq_append_to_end(p, a, b) } + | a=','.expression+ ',' '**' b=expression { (asdl_expr_seq*)_PyPegen_seq_append_to_end(p, a, b) } | '*' a=expression ',' '**' b=expression { - _PyPegen_seq_append_to_end(p, CHECK(_PyPegen_singleton_seq(p, a)), b) } - | '*' a=expression { _PyPegen_singleton_seq(p, a) } - | '**' a=expression { _PyPegen_singleton_seq(p, a) } - | ','.expression+ - -statements[asdl_seq*]: a=statement+ { _PyPegen_seq_flatten(p, a) } -statement[asdl_seq*]: a=compound_stmt { _PyPegen_singleton_seq(p, a) } | simple_stmt -statement_newline[asdl_seq*]: - | a=compound_stmt NEWLINE { _PyPegen_singleton_seq(p, a) } + (asdl_expr_seq*)_PyPegen_seq_append_to_end(p, CHECK(_PyPegen_singleton_seq(p, a)), b) } + | '*' a=expression { (asdl_expr_seq*)_PyPegen_singleton_seq(p, a) } + | '**' a=expression { (asdl_expr_seq*)_PyPegen_singleton_seq(p, a) } + | a[asdl_expr_seq*]=','.expression+ {a} + +statements[asdl_stmt_seq*]: a=statement+ { (asdl_stmt_seq*)_PyPegen_seq_flatten(p, a) } +statement[asdl_stmt_seq*]: a=compound_stmt { (asdl_stmt_seq*)_PyPegen_singleton_seq(p, a) } | a[asdl_stmt_seq*]=simple_stmt { a } +statement_newline[asdl_stmt_seq*]: + | a=compound_stmt NEWLINE { (asdl_stmt_seq*)_PyPegen_singleton_seq(p, a) } | simple_stmt - | NEWLINE { _PyPegen_singleton_seq(p, CHECK(_Py_Pass(EXTRA))) } + | NEWLINE { (asdl_stmt_seq*)_PyPegen_singleton_seq(p, CHECK(_Py_Pass(EXTRA))) } | ENDMARKER { _PyPegen_interactive_exit(p) } -simple_stmt[asdl_seq*]: - | a=small_stmt !';' NEWLINE { _PyPegen_singleton_seq(p, a) } # Not needed, there for speedup - | a=';'.small_stmt+ [';'] NEWLINE { a } +simple_stmt[asdl_stmt_seq*]: + | a=small_stmt !';' NEWLINE { (asdl_stmt_seq*)_PyPegen_singleton_seq(p, a) } # Not needed, there for speedup + | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } # NOTE: assignment MUST precede expression, else parsing a simple assignment # will throw a SyntaxError. small_stmt[stmt_ty] (memo): @@ -91,7 +91,7 @@ assignment[stmt_ty]: | a=('(' b=single_target ')' { b } | single_subscript_attribute_target) ':' b=expression c=['=' d=annotated_rhs { d }] { CHECK_VERSION(6, "Variable annotations syntax is", _Py_AnnAssign(a, b, c, 0, EXTRA)) } - | a=(z=star_targets '=' { z })+ b=(yield_expr | star_expressions) !'=' tc=[TYPE_COMMENT] { + | a[asdl_expr_seq*]=(z=star_targets '=' { z })+ b=(yield_expr | star_expressions) !'=' tc=[TYPE_COMMENT] { _Py_Assign(a, b, NEW_TYPE_COMMENT(p, tc), EXTRA) } | a=single_target b=augassign ~ c=(yield_expr | star_expressions) { _Py_AugAssign(a, b->kind, c, EXTRA) } @@ -112,9 +112,9 @@ augassign[AugOperator*]: | '**=' { _PyPegen_augoperator(p, Pow) } | '//=' { _PyPegen_augoperator(p, FloorDiv) } -global_stmt[stmt_ty]: 'global' a=','.NAME+ { +global_stmt[stmt_ty]: 'global' a[asdl_expr_seq*]=','.NAME+ { _Py_Global(CHECK(_PyPegen_map_names_to_ids(p, a)), EXTRA) } -nonlocal_stmt[stmt_ty]: 'nonlocal' a=','.NAME+ { +nonlocal_stmt[stmt_ty]: 'nonlocal' a[asdl_expr_seq*]=','.NAME+ { _Py_Nonlocal(CHECK(_PyPegen_map_names_to_ids(p, a)), EXTRA) } yield_stmt[stmt_ty]: y=yield_expr { _Py_Expr(y, EXTRA) } @@ -133,19 +133,19 @@ import_from[stmt_ty]: _Py_ImportFrom(b->v.Name.id, c, _PyPegen_seq_count_dots(a), EXTRA) } | 'from' a=('.' | '...')+ 'import' b=import_from_targets { _Py_ImportFrom(NULL, b, _PyPegen_seq_count_dots(a), EXTRA) } -import_from_targets[asdl_seq*]: +import_from_targets[asdl_alias_seq*]: | '(' a=import_from_as_names [','] ')' { a } | import_from_as_names !',' - | '*' { _PyPegen_singleton_seq(p, CHECK(_PyPegen_alias_for_star(p))) } + | '*' { (asdl_alias_seq*)_PyPegen_singleton_seq(p, CHECK(_PyPegen_alias_for_star(p))) } | invalid_import_from_targets -import_from_as_names[asdl_seq*]: - | a=','.import_from_as_name+ { a } +import_from_as_names[asdl_alias_seq*]: + | a[asdl_alias_seq*]=','.import_from_as_name+ { a } import_from_as_name[alias_ty]: | a=NAME b=['as' z=NAME { z }] { _Py_alias(a->v.Name.id, (b) ? ((expr_ty) b)->v.Name.id : NULL, p->arena) } -dotted_as_names[asdl_seq*]: - | a=','.dotted_as_name+ { a } +dotted_as_names[asdl_alias_seq*]: + | a[asdl_alias_seq*]=','.dotted_as_name+ { a } dotted_as_name[alias_ty]: | a=dotted_name b=['as' z=NAME { z }] { _Py_alias(a->v.Name.id, (b) ? ((expr_ty) b)->v.Name.id : NULL, @@ -155,12 +155,12 @@ dotted_name[expr_ty]: | NAME if_stmt[stmt_ty]: - | 'if' a=named_expression ':' b=block c=elif_stmt { _Py_If(a, b, CHECK(_PyPegen_singleton_seq(p, c)), EXTRA) } + | 'if' a=named_expression ':' b=block c=elif_stmt { _Py_If(a, b, CHECK((asdl_stmt_seq*)_PyPegen_singleton_seq(p, c)), EXTRA) } | 'if' a=named_expression ':' b=block c=[else_block] { _Py_If(a, b, c, EXTRA) } elif_stmt[stmt_ty]: | 'elif' a=named_expression ':' b=block c=elif_stmt { _Py_If(a, b, CHECK(_PyPegen_singleton_seq(p, c)), EXTRA) } | 'elif' a=named_expression ':' b=block c=[else_block] { _Py_If(a, b, c, EXTRA) } -else_block[asdl_seq*]: 'else' ':' b=block { b } +else_block[asdl_stmt_seq*]: 'else' ':' b=block { b } while_stmt[stmt_ty]: | 'while' a=named_expression ':' b=block c=[else_block] { _Py_While(a, b, c, EXTRA) } @@ -173,13 +173,13 @@ for_stmt[stmt_ty]: | invalid_for_target with_stmt[stmt_ty]: - | 'with' '(' a=','.with_item+ ','? ')' ':' b=block { + | 'with' '(' a[asdl_withitem_seq*]=','.with_item+ ','? ')' ':' b=block { _Py_With(a, b, NULL, EXTRA) } - | 'with' a=','.with_item+ ':' tc=[TYPE_COMMENT] b=block { + | 'with' a[asdl_withitem_seq*]=','.with_item+ ':' tc=[TYPE_COMMENT] b=block { _Py_With(a, b, NEW_TYPE_COMMENT(p, tc), EXTRA) } - | ASYNC 'with' '(' a=','.with_item+ ','? ')' ':' b=block { + | ASYNC 'with' '(' a[asdl_withitem_seq*]=','.with_item+ ','? ')' ':' b=block { CHECK_VERSION(5, "Async with statements are", _Py_AsyncWith(a, b, NULL, EXTRA)) } - | ASYNC 'with' a=','.with_item+ ':' tc=[TYPE_COMMENT] b=block { + | ASYNC 'with' a[asdl_withitem_seq*]=','.with_item+ ':' tc=[TYPE_COMMENT] b=block { CHECK_VERSION(5, "Async with statements are", _Py_AsyncWith(a, b, NEW_TYPE_COMMENT(p, tc), EXTRA)) } with_item[withitem_ty]: | e=expression 'as' t=target &(',' | ')' | ':') { _Py_withitem(e, t, p->arena) } @@ -188,12 +188,12 @@ with_item[withitem_ty]: try_stmt[stmt_ty]: | 'try' ':' b=block f=finally_block { _Py_Try(b, NULL, NULL, f, EXTRA) } - | 'try' ':' b=block ex=except_block+ el=[else_block] f=[finally_block] { _Py_Try(b, ex, el, f, EXTRA) } + | 'try' ':' b=block ex[asdl_excepthandler_seq*]=except_block+ el=[else_block] f=[finally_block] { _Py_Try(b, ex, el, f, EXTRA) } except_block[excepthandler_ty]: | 'except' e=expression t=['as' z=NAME { z }] ':' b=block { _Py_ExceptHandler(e, (t) ? ((expr_ty) t)->v.Name.id : NULL, b, EXTRA) } | 'except' ':' b=block { _Py_ExceptHandler(NULL, NULL, b, EXTRA) } -finally_block[asdl_seq*]: 'finally' ':' a=block { a } +finally_block[asdl_stmt_seq*]: 'finally' ':' a=block { a } return_stmt[stmt_ty]: | 'return' a=[star_expressions] { _Py_Return(a, EXTRA) } @@ -229,11 +229,11 @@ params[arguments_ty]: | parameters parameters[arguments_ty]: - | a=slash_no_default b=param_no_default* c=param_with_default* d=[star_etc] { + | a=slash_no_default b[asdl_arg_seq*]=param_no_default* c=param_with_default* d=[star_etc] { _PyPegen_make_arguments(p, a, NULL, b, c, d) } | a=slash_with_default b=param_with_default* c=[star_etc] { _PyPegen_make_arguments(p, NULL, a, NULL, b, c) } - | a=param_no_default+ b=param_with_default* c=[star_etc] { + | a[asdl_arg_seq*]=param_no_default+ b=param_with_default* c=[star_etc] { _PyPegen_make_arguments(p, NULL, NULL, a, b, c) } | a=param_with_default+ b=[star_etc] { _PyPegen_make_arguments(p, NULL, NULL, NULL, a, b)} | a=star_etc { _PyPegen_make_arguments(p, NULL, NULL, NULL, NULL, a) } @@ -241,12 +241,12 @@ parameters[arguments_ty]: # Some duplication here because we can't write (',' | &')'), # which is because we don't support empty alternatives (yet). # -slash_no_default[asdl_seq*]: - | a=param_no_default+ '/' ',' { a } - | a=param_no_default+ '/' &')' { a } +slash_no_default[asdl_arg_seq*]: + | a[asdl_arg_seq*]=param_no_default+ '/' ',' { a } + | a[asdl_arg_seq*]=param_no_default+ '/' &')' { a } slash_with_default[SlashWithDefault*]: - | a=param_no_default* b=param_with_default+ '/' ',' { _PyPegen_slash_with_default(p, a, b) } - | a=param_no_default* b=param_with_default+ '/' &')' { _PyPegen_slash_with_default(p, a, b) } + | a=param_no_default* b=param_with_default+ '/' ',' { _PyPegen_slash_with_default(p, (asdl_arg_seq *)a, b) } + | a=param_no_default* b=param_with_default+ '/' &')' { _PyPegen_slash_with_default(p, (asdl_arg_seq *)a, b) } star_etc[StarEtc*]: | '*' a=param_no_default b=param_maybe_default* c=[kwds] { @@ -284,7 +284,7 @@ param[arg_ty]: a=NAME b=annotation? { _Py_arg(a->v.Name.id, b, NULL, EXTRA) } annotation[expr_ty]: ':' a=expression { a } default[expr_ty]: '=' a=expression { a } -decorators[asdl_seq*]: a=('@' f=named_expression NEWLINE { f })+ { a } +decorators[asdl_expr_seq*]: a[asdl_expr_seq*]=('@' f=named_expression NEWLINE { f })+ { a } class_def[stmt_ty]: | a=decorators b=class_def_raw { _PyPegen_class_def_decorators(p, a, b) } @@ -296,12 +296,12 @@ class_def_raw[stmt_ty]: (b) ? ((expr_ty) b)->v.Call.keywords : NULL, c, NULL, EXTRA) } -block[asdl_seq*] (memo): +block[asdl_stmt_seq*] (memo): | NEWLINE INDENT a=statements DEDENT { a } | simple_stmt | invalid_block -expressions_list[asdl_seq*]: a=','.star_expression+ [','] { a } +expressions_list[asdl_expr_seq*]: a[asdl_expr_seq*]=','.star_expression+ [','] { a } star_expressions[expr_ty]: | a=star_expression b=(',' c=star_expression { c })+ [','] { _Py_Tuple(CHECK(_PyPegen_seq_insert_in_front(p, a, b)), Load, EXTRA) } @@ -311,7 +311,7 @@ star_expression[expr_ty] (memo): | '*' a=bitwise_or { _Py_Starred(a, Load, EXTRA) } | expression -star_named_expressions[asdl_seq*]: a=','.star_named_expression+ [','] { a } +star_named_expressions[asdl_expr_seq*]: a[asdl_expr_seq*]=','.star_named_expression+ [','] { a } star_named_expression[expr_ty]: | '*' a=bitwise_or { _Py_Starred(a, Load, EXTRA) } | named_expression @@ -344,21 +344,21 @@ lambda_params[arguments_ty]: # a colon, not a close parenthesis. (For more, see parameters above.) # lambda_parameters[arguments_ty]: - | a=lambda_slash_no_default b=lambda_param_no_default* c=lambda_param_with_default* d=[lambda_star_etc] { + | a=lambda_slash_no_default b[asdl_arg_seq*]=lambda_param_no_default* c=lambda_param_with_default* d=[lambda_star_etc] { _PyPegen_make_arguments(p, a, NULL, b, c, d) } | a=lambda_slash_with_default b=lambda_param_with_default* c=[lambda_star_etc] { _PyPegen_make_arguments(p, NULL, a, NULL, b, c) } - | a=lambda_param_no_default+ b=lambda_param_with_default* c=[lambda_star_etc] { + | a[asdl_arg_seq*]=lambda_param_no_default+ b=lambda_param_with_default* c=[lambda_star_etc] { _PyPegen_make_arguments(p, NULL, NULL, a, b, c) } | a=lambda_param_with_default+ b=[lambda_star_etc] { _PyPegen_make_arguments(p, NULL, NULL, NULL, a, b)} | a=lambda_star_etc { _PyPegen_make_arguments(p, NULL, NULL, NULL, NULL, a) } -lambda_slash_no_default[asdl_seq*]: - | a=lambda_param_no_default+ '/' ',' { a } - | a=lambda_param_no_default+ '/' &':' { a } +lambda_slash_no_default[asdl_arg_seq*]: + | a[asdl_arg_seq*]=lambda_param_no_default+ '/' ',' { a } + | a[asdl_arg_seq*]=lambda_param_no_default+ '/' &':' { a } lambda_slash_with_default[SlashWithDefault*]: - | a=lambda_param_no_default* b=lambda_param_with_default+ '/' ',' { _PyPegen_slash_with_default(p, a, b) } - | a=lambda_param_no_default* b=lambda_param_with_default+ '/' &':' { _PyPegen_slash_with_default(p, a, b) } + | a=lambda_param_no_default* b=lambda_param_with_default+ '/' ',' { _PyPegen_slash_with_default(p, (asdl_arg_seq *)a, b) } + | a=lambda_param_no_default* b=lambda_param_with_default+ '/' &':' { _PyPegen_slash_with_default(p, (asdl_arg_seq *)a, b) } lambda_star_etc[StarEtc*]: | '*' a=lambda_param_no_default b=lambda_param_maybe_default* c=[lambda_kwds] { @@ -472,7 +472,7 @@ primary[expr_ty]: slices[expr_ty]: | a=slice !',' { a } - | a=','.slice+ [','] { _Py_Tuple(a, Load, EXTRA) } + | a[asdl_expr_seq*]=','.slice+ [','] { _Py_Tuple(a, Load, EXTRA) } slice[expr_ty]: | a=[expression] ':' b=[expression] c=[':' d=[expression] { d }] { _Py_Slice(a, b, c, EXTRA) } | a=expression { a } @@ -518,12 +518,12 @@ double_starred_kvpair[KeyValuePair*]: | '**' a=bitwise_or { _PyPegen_key_value_pair(p, NULL, a) } | kvpair kvpair[KeyValuePair*]: a=expression ':' b=expression { _PyPegen_key_value_pair(p, a, b) } -for_if_clauses[asdl_seq*]: - | for_if_clause+ +for_if_clauses[asdl_comprehension_seq*]: + | a[asdl_comprehension_seq*]=for_if_clause+ { a } for_if_clause[comprehension_ty]: - | ASYNC 'for' a=star_targets 'in' ~ b=disjunction c=('if' z=disjunction { z })* { + | ASYNC 'for' a=star_targets 'in' ~ b=disjunction c[asdl_expr_seq*]=('if' z=disjunction { z })* { CHECK_VERSION(6, "Async comprehensions are", _Py_comprehension(a, b, c, 1, p->arena)) } - | 'for' a=star_targets 'in' ~ b=disjunction c=('if' z=disjunction { z })* { + | 'for' a=star_targets 'in' ~ b=disjunction c[asdl_expr_seq*]=('if' z=disjunction { z })* { _Py_comprehension(a, b, c, 0, p->arena) } | invalid_for_target @@ -535,7 +535,7 @@ arguments[expr_ty] (memo): | a=args [','] &')' { a } | incorrect_arguments args[expr_ty]: - | a=','.(starred_expression | named_expression !'=')+ b=[',' k=kwargs {k}] { _PyPegen_collect_call_seqs(p, a, b, EXTRA) } + | a[asdl_expr_seq*]=','.(starred_expression | named_expression !'=')+ b=[',' k=kwargs {k}] { _PyPegen_collect_call_seqs(p, a, b, EXTRA) } | a=kwargs { _Py_Call(_PyPegen_dummy_name(p), CHECK_NULL_ALLOWED(_PyPegen_seq_extract_starred_exprs(p, a)), CHECK_NULL_ALLOWED(_PyPegen_seq_delete_starred_exprs(p, a)), @@ -562,7 +562,7 @@ star_targets[expr_ty]: | a=star_target !',' { a } | a=star_target b=(',' c=star_target { c })* [','] { _Py_Tuple(CHECK(_PyPegen_seq_insert_in_front(p, a, b)), Store, EXTRA) } -star_targets_seq[asdl_seq*]: a=','.star_target+ [','] { a } +star_targets_seq[asdl_expr_seq*]: a[asdl_expr_seq*]=','.star_target+ [','] { a } star_target[expr_ty] (memo): | '*' a=(!'*' star_target) { _Py_Starred(CHECK(_PyPegen_set_expr_context(p, a, Store)), Store, EXTRA) } @@ -583,7 +583,7 @@ single_subscript_attribute_target[expr_ty]: | a=t_primary '.' b=NAME !t_lookahead { _Py_Attribute(a, b->v.Name.id, Store, EXTRA) } | a=t_primary '[' b=slices ']' !t_lookahead { _Py_Subscript(a, b, Store, EXTRA) } -del_targets[asdl_seq*]: a=','.del_target+ [','] { a } +del_targets[asdl_expr_seq*]: a[asdl_expr_seq*]=','.del_target+ [','] { a } del_target[expr_ty] (memo): | a=t_primary '.' b=NAME !t_lookahead { _Py_Attribute(a, b->v.Name.id, Del, EXTRA) } | a=t_primary '[' b=slices ']' !t_lookahead { _Py_Subscript(a, b, Del, EXTRA) } @@ -594,7 +594,7 @@ del_t_atom[expr_ty]: | '(' a=[del_targets] ')' { _Py_Tuple(a, Del, EXTRA) } | '[' a=[del_targets] ']' { _Py_List(a, Del, EXTRA) } -targets[asdl_seq*]: a=','.target+ [','] { a } +targets[asdl_expr_seq*]: a[asdl_expr_seq*]=','.target+ [','] { a } target[expr_ty] (memo): | a=t_primary '.' b=NAME !t_lookahead { _Py_Attribute(a, b->v.Name.id, Store, EXTRA) } | a=t_primary '[' b=slices ']' !t_lookahead { _Py_Subscript(a, b, Store, EXTRA) } diff --git a/Include/Python-ast.h b/Include/Python-ast.h index e7afa1e6579e8..e14bab566fb5a 100644 --- a/Include/Python-ast.h +++ b/Include/Python-ast.h @@ -47,18 +47,99 @@ typedef struct _withitem *withitem_ty; typedef struct _type_ignore *type_ignore_ty; +typedef struct { + _ASDL_SEQ_HEAD + mod_ty typed_elements[1]; +} asdl_mod_seq; + +asdl_mod_seq *_Py_asdl_mod_seq_new(Py_ssize_t size, PyArena *arena); + +typedef struct { + _ASDL_SEQ_HEAD + stmt_ty typed_elements[1]; +} asdl_stmt_seq; + +asdl_stmt_seq *_Py_asdl_stmt_seq_new(Py_ssize_t size, PyArena *arena); + +typedef struct { + _ASDL_SEQ_HEAD + expr_ty typed_elements[1]; +} asdl_expr_seq; + +asdl_expr_seq *_Py_asdl_expr_seq_new(Py_ssize_t size, PyArena *arena); + +typedef struct { + _ASDL_SEQ_HEAD + comprehension_ty typed_elements[1]; +} asdl_comprehension_seq; + +asdl_comprehension_seq *_Py_asdl_comprehension_seq_new(Py_ssize_t size, PyArena + *arena); + +typedef struct { + _ASDL_SEQ_HEAD + excepthandler_ty typed_elements[1]; +} asdl_excepthandler_seq; + +asdl_excepthandler_seq *_Py_asdl_excepthandler_seq_new(Py_ssize_t size, PyArena + *arena); + +typedef struct { + _ASDL_SEQ_HEAD + arguments_ty typed_elements[1]; +} asdl_arguments_seq; + +asdl_arguments_seq *_Py_asdl_arguments_seq_new(Py_ssize_t size, PyArena *arena); + +typedef struct { + _ASDL_SEQ_HEAD + arg_ty typed_elements[1]; +} asdl_arg_seq; + +asdl_arg_seq *_Py_asdl_arg_seq_new(Py_ssize_t size, PyArena *arena); + +typedef struct { + _ASDL_SEQ_HEAD + keyword_ty typed_elements[1]; +} asdl_keyword_seq; + +asdl_keyword_seq *_Py_asdl_keyword_seq_new(Py_ssize_t size, PyArena *arena); + +typedef struct { + _ASDL_SEQ_HEAD + alias_ty typed_elements[1]; +} asdl_alias_seq; + +asdl_alias_seq *_Py_asdl_alias_seq_new(Py_ssize_t size, PyArena *arena); + +typedef struct { + _ASDL_SEQ_HEAD + withitem_ty typed_elements[1]; +} asdl_withitem_seq; + +asdl_withitem_seq *_Py_asdl_withitem_seq_new(Py_ssize_t size, PyArena *arena); + +typedef struct { + _ASDL_SEQ_HEAD + type_ignore_ty typed_elements[1]; +} asdl_type_ignore_seq; + +asdl_type_ignore_seq *_Py_asdl_type_ignore_seq_new(Py_ssize_t size, PyArena + *arena); + + enum _mod_kind {Module_kind=1, Interactive_kind=2, Expression_kind=3, FunctionType_kind=4}; struct _mod { enum _mod_kind kind; union { struct { - asdl_seq *body; - asdl_seq *type_ignores; + asdl_stmt_seq *body; + asdl_type_ignore_seq *type_ignores; } Module; struct { - asdl_seq *body; + asdl_stmt_seq *body; } Interactive; struct { @@ -66,7 +147,7 @@ struct _mod { } Expression; struct { - asdl_seq *argtypes; + asdl_expr_seq *argtypes; expr_ty returns; } FunctionType; @@ -87,8 +168,8 @@ struct _stmt { struct { identifier name; arguments_ty args; - asdl_seq *body; - asdl_seq *decorator_list; + asdl_stmt_seq *body; + asdl_expr_seq *decorator_list; expr_ty returns; string type_comment; } FunctionDef; @@ -96,18 +177,18 @@ struct _stmt { struct { identifier name; arguments_ty args; - asdl_seq *body; - asdl_seq *decorator_list; + asdl_stmt_seq *body; + asdl_expr_seq *decorator_list; expr_ty returns; string type_comment; } AsyncFunctionDef; struct { identifier name; - asdl_seq *bases; - asdl_seq *keywords; - asdl_seq *body; - asdl_seq *decorator_list; + asdl_expr_seq *bases; + asdl_keyword_seq *keywords; + asdl_stmt_seq *body; + asdl_expr_seq *decorator_list; } ClassDef; struct { @@ -115,11 +196,11 @@ struct _stmt { } Return; struct { - asdl_seq *targets; + asdl_expr_seq *targets; } Delete; struct { - asdl_seq *targets; + asdl_expr_seq *targets; expr_ty value; string type_comment; } Assign; @@ -140,40 +221,40 @@ struct _stmt { struct { expr_ty target; expr_ty iter; - asdl_seq *body; - asdl_seq *orelse; + asdl_stmt_seq *body; + asdl_stmt_seq *orelse; string type_comment; } For; struct { expr_ty target; expr_ty iter; - asdl_seq *body; - asdl_seq *orelse; + asdl_stmt_seq *body; + asdl_stmt_seq *orelse; string type_comment; } AsyncFor; struct { expr_ty test; - asdl_seq *body; - asdl_seq *orelse; + asdl_stmt_seq *body; + asdl_stmt_seq *orelse; } While; struct { expr_ty test; - asdl_seq *body; - asdl_seq *orelse; + asdl_stmt_seq *body; + asdl_stmt_seq *orelse; } If; struct { - asdl_seq *items; - asdl_seq *body; + asdl_withitem_seq *items; + asdl_stmt_seq *body; string type_comment; } With; struct { - asdl_seq *items; - asdl_seq *body; + asdl_withitem_seq *items; + asdl_stmt_seq *body; string type_comment; } AsyncWith; @@ -183,10 +264,10 @@ struct _stmt { } Raise; struct { - asdl_seq *body; - asdl_seq *handlers; - asdl_seq *orelse; - asdl_seq *finalbody; + asdl_stmt_seq *body; + asdl_excepthandler_seq *handlers; + asdl_stmt_seq *orelse; + asdl_stmt_seq *finalbody; } Try; struct { @@ -195,21 +276,21 @@ struct _stmt { } Assert; struct { - asdl_seq *names; + asdl_alias_seq *names; } Import; struct { identifier module; - asdl_seq *names; + asdl_alias_seq *names; int level; } ImportFrom; struct { - asdl_seq *names; + asdl_identifier_seq *names; } Global; struct { - asdl_seq *names; + asdl_identifier_seq *names; } Nonlocal; struct { @@ -236,7 +317,7 @@ struct _expr { union { struct { boolop_ty op; - asdl_seq *values; + asdl_expr_seq *values; } BoolOp; struct { @@ -267,33 +348,33 @@ struct _expr { } IfExp; struct { - asdl_seq *keys; - asdl_seq *values; + asdl_expr_seq *keys; + asdl_expr_seq *values; } Dict; struct { - asdl_seq *elts; + asdl_expr_seq *elts; } Set; struct { expr_ty elt; - asdl_seq *generators; + asdl_comprehension_seq *generators; } ListComp; struct { expr_ty elt; - asdl_seq *generators; + asdl_comprehension_seq *generators; } SetComp; struct { expr_ty key; expr_ty value; - asdl_seq *generators; + asdl_comprehension_seq *generators; } DictComp; struct { expr_ty elt; - asdl_seq *generators; + asdl_comprehension_seq *generators; } GeneratorExp; struct { @@ -311,13 +392,13 @@ struct _expr { struct { expr_ty left; asdl_int_seq *ops; - asdl_seq *comparators; + asdl_expr_seq *comparators; } Compare; struct { expr_ty func; - asdl_seq *args; - asdl_seq *keywords; + asdl_expr_seq *args; + asdl_keyword_seq *keywords; } Call; struct { @@ -327,7 +408,7 @@ struct _expr { } FormattedValue; struct { - asdl_seq *values; + asdl_expr_seq *values; } JoinedStr; struct { @@ -358,12 +439,12 @@ struct _expr { } Name; struct { - asdl_seq *elts; + asdl_expr_seq *elts; expr_context_ty ctx; } List; struct { - asdl_seq *elts; + asdl_expr_seq *elts; expr_context_ty ctx; } Tuple; @@ -383,7 +464,7 @@ struct _expr { struct _comprehension { expr_ty target; expr_ty iter; - asdl_seq *ifs; + asdl_expr_seq *ifs; int is_async; }; @@ -394,7 +475,7 @@ struct _excepthandler { struct { expr_ty type; identifier name; - asdl_seq *body; + asdl_stmt_seq *body; } ExceptHandler; } v; @@ -405,13 +486,13 @@ struct _excepthandler { }; struct _arguments { - asdl_seq *posonlyargs; - asdl_seq *args; + asdl_arg_seq *posonlyargs; + asdl_arg_seq *args; arg_ty vararg; - asdl_seq *kwonlyargs; - asdl_seq *kw_defaults; + asdl_arg_seq *kwonlyargs; + asdl_expr_seq *kw_defaults; arg_ty kwarg; - asdl_seq *defaults; + asdl_expr_seq *defaults; }; struct _arg { @@ -458,39 +539,41 @@ struct _type_ignore { // Note: these macros affect function definitions, not only call sites. #define Module(a0, a1, a2) _Py_Module(a0, a1, a2) -mod_ty _Py_Module(asdl_seq * body, asdl_seq * type_ignores, PyArena *arena); +mod_ty _Py_Module(asdl_stmt_seq * body, asdl_type_ignore_seq * type_ignores, + PyArena *arena); #define Interactive(a0, a1) _Py_Interactive(a0, a1) -mod_ty _Py_Interactive(asdl_seq * body, PyArena *arena); +mod_ty _Py_Interactive(asdl_stmt_seq * body, PyArena *arena); #define Expression(a0, a1) _Py_Expression(a0, a1) mod_ty _Py_Expression(expr_ty body, PyArena *arena); #define FunctionType(a0, a1, a2) _Py_FunctionType(a0, a1, a2) -mod_ty _Py_FunctionType(asdl_seq * argtypes, expr_ty returns, PyArena *arena); +mod_ty _Py_FunctionType(asdl_expr_seq * argtypes, expr_ty returns, PyArena + *arena); #define FunctionDef(a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10) _Py_FunctionDef(a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10) -stmt_ty _Py_FunctionDef(identifier name, arguments_ty args, asdl_seq * body, - asdl_seq * decorator_list, expr_ty returns, string - type_comment, int lineno, int col_offset, int +stmt_ty _Py_FunctionDef(identifier name, arguments_ty args, asdl_stmt_seq * + body, asdl_expr_seq * decorator_list, expr_ty returns, + string type_comment, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define AsyncFunctionDef(a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10) _Py_AsyncFunctionDef(a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10) -stmt_ty _Py_AsyncFunctionDef(identifier name, arguments_ty args, asdl_seq * - body, asdl_seq * decorator_list, expr_ty returns, - string type_comment, int lineno, int col_offset, - int end_lineno, int end_col_offset, PyArena - *arena); +stmt_ty _Py_AsyncFunctionDef(identifier name, arguments_ty args, asdl_stmt_seq + * body, asdl_expr_seq * decorator_list, expr_ty + returns, string type_comment, int lineno, int + col_offset, int end_lineno, int end_col_offset, + PyArena *arena); #define ClassDef(a0, a1, a2, a3, a4, a5, a6, a7, a8, a9) _Py_ClassDef(a0, a1, a2, a3, a4, a5, a6, a7, a8, a9) -stmt_ty _Py_ClassDef(identifier name, asdl_seq * bases, asdl_seq * keywords, - asdl_seq * body, asdl_seq * decorator_list, int lineno, - int col_offset, int end_lineno, int end_col_offset, - PyArena *arena); +stmt_ty _Py_ClassDef(identifier name, asdl_expr_seq * bases, asdl_keyword_seq * + keywords, asdl_stmt_seq * body, asdl_expr_seq * + decorator_list, int lineno, int col_offset, int + end_lineno, int end_col_offset, PyArena *arena); #define Return(a0, a1, a2, a3, a4, a5) _Py_Return(a0, a1, a2, a3, a4, a5) stmt_ty _Py_Return(expr_ty value, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Delete(a0, a1, a2, a3, a4, a5) _Py_Delete(a0, a1, a2, a3, a4, a5) -stmt_ty _Py_Delete(asdl_seq * targets, int lineno, int col_offset, int +stmt_ty _Py_Delete(asdl_expr_seq * targets, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Assign(a0, a1, a2, a3, a4, a5, a6, a7) _Py_Assign(a0, a1, a2, a3, a4, a5, a6, a7) -stmt_ty _Py_Assign(asdl_seq * targets, expr_ty value, string type_comment, int - lineno, int col_offset, int end_lineno, int end_col_offset, - PyArena *arena); +stmt_ty _Py_Assign(asdl_expr_seq * targets, expr_ty value, string type_comment, + int lineno, int col_offset, int end_lineno, int + end_col_offset, PyArena *arena); #define AugAssign(a0, a1, a2, a3, a4, a5, a6, a7) _Py_AugAssign(a0, a1, a2, a3, a4, a5, a6, a7) stmt_ty _Py_AugAssign(expr_ty target, operator_ty op, expr_ty value, int lineno, int col_offset, int end_lineno, int @@ -500,52 +583,54 @@ stmt_ty _Py_AnnAssign(expr_ty target, expr_ty annotation, expr_ty value, int simple, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define For(a0, a1, a2, a3, a4, a5, a6, a7, a8, a9) _Py_For(a0, a1, a2, a3, a4, a5, a6, a7, a8, a9) -stmt_ty _Py_For(expr_ty target, expr_ty iter, asdl_seq * body, asdl_seq * - orelse, string type_comment, int lineno, int col_offset, int - end_lineno, int end_col_offset, PyArena *arena); +stmt_ty _Py_For(expr_ty target, expr_ty iter, asdl_stmt_seq * body, + asdl_stmt_seq * orelse, string type_comment, int lineno, int + col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define AsyncFor(a0, a1, a2, a3, a4, a5, a6, a7, a8, a9) _Py_AsyncFor(a0, a1, a2, a3, a4, a5, a6, a7, a8, a9) -stmt_ty _Py_AsyncFor(expr_ty target, expr_ty iter, asdl_seq * body, asdl_seq * - orelse, string type_comment, int lineno, int col_offset, - int end_lineno, int end_col_offset, PyArena *arena); +stmt_ty _Py_AsyncFor(expr_ty target, expr_ty iter, asdl_stmt_seq * body, + asdl_stmt_seq * orelse, string type_comment, int lineno, + int col_offset, int end_lineno, int end_col_offset, + PyArena *arena); #define While(a0, a1, a2, a3, a4, a5, a6, a7) _Py_While(a0, a1, a2, a3, a4, a5, a6, a7) -stmt_ty _Py_While(expr_ty test, asdl_seq * body, asdl_seq * orelse, int lineno, - int col_offset, int end_lineno, int end_col_offset, PyArena - *arena); +stmt_ty _Py_While(expr_ty test, asdl_stmt_seq * body, asdl_stmt_seq * orelse, + int lineno, int col_offset, int end_lineno, int + end_col_offset, PyArena *arena); #define If(a0, a1, a2, a3, a4, a5, a6, a7) _Py_If(a0, a1, a2, a3, a4, a5, a6, a7) -stmt_ty _Py_If(expr_ty test, asdl_seq * body, asdl_seq * orelse, int lineno, - int col_offset, int end_lineno, int end_col_offset, PyArena - *arena); +stmt_ty _Py_If(expr_ty test, asdl_stmt_seq * body, asdl_stmt_seq * orelse, int + lineno, int col_offset, int end_lineno, int end_col_offset, + PyArena *arena); #define With(a0, a1, a2, a3, a4, a5, a6, a7) _Py_With(a0, a1, a2, a3, a4, a5, a6, a7) -stmt_ty _Py_With(asdl_seq * items, asdl_seq * body, string type_comment, int - lineno, int col_offset, int end_lineno, int end_col_offset, - PyArena *arena); +stmt_ty _Py_With(asdl_withitem_seq * items, asdl_stmt_seq * body, string + type_comment, int lineno, int col_offset, int end_lineno, int + end_col_offset, PyArena *arena); #define AsyncWith(a0, a1, a2, a3, a4, a5, a6, a7) _Py_AsyncWith(a0, a1, a2, a3, a4, a5, a6, a7) -stmt_ty _Py_AsyncWith(asdl_seq * items, asdl_seq * body, string type_comment, - int lineno, int col_offset, int end_lineno, int - end_col_offset, PyArena *arena); +stmt_ty _Py_AsyncWith(asdl_withitem_seq * items, asdl_stmt_seq * body, string + type_comment, int lineno, int col_offset, int end_lineno, + int end_col_offset, PyArena *arena); #define Raise(a0, a1, a2, a3, a4, a5, a6) _Py_Raise(a0, a1, a2, a3, a4, a5, a6) stmt_ty _Py_Raise(expr_ty exc, expr_ty cause, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Try(a0, a1, a2, a3, a4, a5, a6, a7, a8) _Py_Try(a0, a1, a2, a3, a4, a5, a6, a7, a8) -stmt_ty _Py_Try(asdl_seq * body, asdl_seq * handlers, asdl_seq * orelse, - asdl_seq * finalbody, int lineno, int col_offset, int - end_lineno, int end_col_offset, PyArena *arena); +stmt_ty _Py_Try(asdl_stmt_seq * body, asdl_excepthandler_seq * handlers, + asdl_stmt_seq * orelse, asdl_stmt_seq * finalbody, int lineno, + int col_offset, int end_lineno, int end_col_offset, PyArena + *arena); #define Assert(a0, a1, a2, a3, a4, a5, a6) _Py_Assert(a0, a1, a2, a3, a4, a5, a6) stmt_ty _Py_Assert(expr_ty test, expr_ty msg, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Import(a0, a1, a2, a3, a4, a5) _Py_Import(a0, a1, a2, a3, a4, a5) -stmt_ty _Py_Import(asdl_seq * names, int lineno, int col_offset, int +stmt_ty _Py_Import(asdl_alias_seq * names, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define ImportFrom(a0, a1, a2, a3, a4, a5, a6, a7) _Py_ImportFrom(a0, a1, a2, a3, a4, a5, a6, a7) -stmt_ty _Py_ImportFrom(identifier module, asdl_seq * names, int level, int - lineno, int col_offset, int end_lineno, int +stmt_ty _Py_ImportFrom(identifier module, asdl_alias_seq * names, int level, + int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Global(a0, a1, a2, a3, a4, a5) _Py_Global(a0, a1, a2, a3, a4, a5) -stmt_ty _Py_Global(asdl_seq * names, int lineno, int col_offset, int +stmt_ty _Py_Global(asdl_identifier_seq * names, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Nonlocal(a0, a1, a2, a3, a4, a5) _Py_Nonlocal(a0, a1, a2, a3, a4, a5) -stmt_ty _Py_Nonlocal(asdl_seq * names, int lineno, int col_offset, int - end_lineno, int end_col_offset, PyArena *arena); +stmt_ty _Py_Nonlocal(asdl_identifier_seq * names, int lineno, int col_offset, + int end_lineno, int end_col_offset, PyArena *arena); #define Expr(a0, a1, a2, a3, a4, a5) _Py_Expr(a0, a1, a2, a3, a4, a5) stmt_ty _Py_Expr(expr_ty value, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); @@ -559,8 +644,9 @@ stmt_ty _Py_Break(int lineno, int col_offset, int end_lineno, int stmt_ty _Py_Continue(int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define BoolOp(a0, a1, a2, a3, a4, a5, a6) _Py_BoolOp(a0, a1, a2, a3, a4, a5, a6) -expr_ty _Py_BoolOp(boolop_ty op, asdl_seq * values, int lineno, int col_offset, - int end_lineno, int end_col_offset, PyArena *arena); +expr_ty _Py_BoolOp(boolop_ty op, asdl_expr_seq * values, int lineno, int + col_offset, int end_lineno, int end_col_offset, PyArena + *arena); #define NamedExpr(a0, a1, a2, a3, a4, a5, a6) _Py_NamedExpr(a0, a1, a2, a3, a4, a5, a6) expr_ty _Py_NamedExpr(expr_ty target, expr_ty value, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena @@ -580,28 +666,28 @@ expr_ty _Py_IfExp(expr_ty test, expr_ty body, expr_ty orelse, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Dict(a0, a1, a2, a3, a4, a5, a6) _Py_Dict(a0, a1, a2, a3, a4, a5, a6) -expr_ty _Py_Dict(asdl_seq * keys, asdl_seq * values, int lineno, int +expr_ty _Py_Dict(asdl_expr_seq * keys, asdl_expr_seq * values, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Set(a0, a1, a2, a3, a4, a5) _Py_Set(a0, a1, a2, a3, a4, a5) -expr_ty _Py_Set(asdl_seq * elts, int lineno, int col_offset, int end_lineno, - int end_col_offset, PyArena *arena); +expr_ty _Py_Set(asdl_expr_seq * elts, int lineno, int col_offset, int + end_lineno, int end_col_offset, PyArena *arena); #define ListComp(a0, a1, a2, a3, a4, a5, a6) _Py_ListComp(a0, a1, a2, a3, a4, a5, a6) -expr_ty _Py_ListComp(expr_ty elt, asdl_seq * generators, int lineno, int - col_offset, int end_lineno, int end_col_offset, PyArena - *arena); -#define SetComp(a0, a1, a2, a3, a4, a5, a6) _Py_SetComp(a0, a1, a2, a3, a4, a5, a6) -expr_ty _Py_SetComp(expr_ty elt, asdl_seq * generators, int lineno, int - col_offset, int end_lineno, int end_col_offset, PyArena - *arena); -#define DictComp(a0, a1, a2, a3, a4, a5, a6, a7) _Py_DictComp(a0, a1, a2, a3, a4, a5, a6, a7) -expr_ty _Py_DictComp(expr_ty key, expr_ty value, asdl_seq * generators, int +expr_ty _Py_ListComp(expr_ty elt, asdl_comprehension_seq * generators, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); +#define SetComp(a0, a1, a2, a3, a4, a5, a6) _Py_SetComp(a0, a1, a2, a3, a4, a5, a6) +expr_ty _Py_SetComp(expr_ty elt, asdl_comprehension_seq * generators, int + lineno, int col_offset, int end_lineno, int end_col_offset, + PyArena *arena); +#define DictComp(a0, a1, a2, a3, a4, a5, a6, a7) _Py_DictComp(a0, a1, a2, a3, a4, a5, a6, a7) +expr_ty _Py_DictComp(expr_ty key, expr_ty value, asdl_comprehension_seq * + generators, int lineno, int col_offset, int end_lineno, + int end_col_offset, PyArena *arena); #define GeneratorExp(a0, a1, a2, a3, a4, a5, a6) _Py_GeneratorExp(a0, a1, a2, a3, a4, a5, a6) -expr_ty _Py_GeneratorExp(expr_ty elt, asdl_seq * generators, int lineno, int - col_offset, int end_lineno, int end_col_offset, - PyArena *arena); +expr_ty _Py_GeneratorExp(expr_ty elt, asdl_comprehension_seq * generators, int + lineno, int col_offset, int end_lineno, int + end_col_offset, PyArena *arena); #define Await(a0, a1, a2, a3, a4, a5) _Py_Await(a0, a1, a2, a3, a4, a5) expr_ty _Py_Await(expr_ty value, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); @@ -612,19 +698,19 @@ expr_ty _Py_Yield(expr_ty value, int lineno, int col_offset, int end_lineno, expr_ty _Py_YieldFrom(expr_ty value, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Compare(a0, a1, a2, a3, a4, a5, a6, a7) _Py_Compare(a0, a1, a2, a3, a4, a5, a6, a7) -expr_ty _Py_Compare(expr_ty left, asdl_int_seq * ops, asdl_seq * comparators, - int lineno, int col_offset, int end_lineno, int - end_col_offset, PyArena *arena); +expr_ty _Py_Compare(expr_ty left, asdl_int_seq * ops, asdl_expr_seq * + comparators, int lineno, int col_offset, int end_lineno, + int end_col_offset, PyArena *arena); #define Call(a0, a1, a2, a3, a4, a5, a6, a7) _Py_Call(a0, a1, a2, a3, a4, a5, a6, a7) -expr_ty _Py_Call(expr_ty func, asdl_seq * args, asdl_seq * keywords, int - lineno, int col_offset, int end_lineno, int end_col_offset, - PyArena *arena); +expr_ty _Py_Call(expr_ty func, asdl_expr_seq * args, asdl_keyword_seq * + keywords, int lineno, int col_offset, int end_lineno, int + end_col_offset, PyArena *arena); #define FormattedValue(a0, a1, a2, a3, a4, a5, a6, a7) _Py_FormattedValue(a0, a1, a2, a3, a4, a5, a6, a7) expr_ty _Py_FormattedValue(expr_ty value, int conversion, expr_ty format_spec, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define JoinedStr(a0, a1, a2, a3, a4, a5) _Py_JoinedStr(a0, a1, a2, a3, a4, a5) -expr_ty _Py_JoinedStr(asdl_seq * values, int lineno, int col_offset, int +expr_ty _Py_JoinedStr(asdl_expr_seq * values, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Constant(a0, a1, a2, a3, a4, a5, a6) _Py_Constant(a0, a1, a2, a3, a4, a5, a6) expr_ty _Py_Constant(constant value, string kind, int lineno, int col_offset, @@ -646,11 +732,11 @@ expr_ty _Py_Name(identifier id, expr_context_ty ctx, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define List(a0, a1, a2, a3, a4, a5, a6) _Py_List(a0, a1, a2, a3, a4, a5, a6) -expr_ty _Py_List(asdl_seq * elts, expr_context_ty ctx, int lineno, int +expr_ty _Py_List(asdl_expr_seq * elts, expr_context_ty ctx, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Tuple(a0, a1, a2, a3, a4, a5, a6) _Py_Tuple(a0, a1, a2, a3, a4, a5, a6) -expr_ty _Py_Tuple(asdl_seq * elts, expr_context_ty ctx, int lineno, int +expr_ty _Py_Tuple(asdl_expr_seq * elts, expr_context_ty ctx, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define Slice(a0, a1, a2, a3, a4, a5, a6, a7) _Py_Slice(a0, a1, a2, a3, a4, a5, a6, a7) @@ -658,18 +744,18 @@ expr_ty _Py_Slice(expr_ty lower, expr_ty upper, expr_ty step, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define comprehension(a0, a1, a2, a3, a4) _Py_comprehension(a0, a1, a2, a3, a4) -comprehension_ty _Py_comprehension(expr_ty target, expr_ty iter, asdl_seq * - ifs, int is_async, PyArena *arena); +comprehension_ty _Py_comprehension(expr_ty target, expr_ty iter, asdl_expr_seq + * ifs, int is_async, PyArena *arena); #define ExceptHandler(a0, a1, a2, a3, a4, a5, a6, a7) _Py_ExceptHandler(a0, a1, a2, a3, a4, a5, a6, a7) -excepthandler_ty _Py_ExceptHandler(expr_ty type, identifier name, asdl_seq * - body, int lineno, int col_offset, int +excepthandler_ty _Py_ExceptHandler(expr_ty type, identifier name, asdl_stmt_seq + * body, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); #define arguments(a0, a1, a2, a3, a4, a5, a6, a7) _Py_arguments(a0, a1, a2, a3, a4, a5, a6, a7) -arguments_ty _Py_arguments(asdl_seq * posonlyargs, asdl_seq * args, arg_ty - vararg, asdl_seq * kwonlyargs, asdl_seq * - kw_defaults, arg_ty kwarg, asdl_seq * defaults, - PyArena *arena); +arguments_ty _Py_arguments(asdl_arg_seq * posonlyargs, asdl_arg_seq * args, + arg_ty vararg, asdl_arg_seq * kwonlyargs, + asdl_expr_seq * kw_defaults, arg_ty kwarg, + asdl_expr_seq * defaults, PyArena *arena); #define arg(a0, a1, a2, a3, a4, a5, a6, a7) _Py_arg(a0, a1, a2, a3, a4, a5, a6, a7) arg_ty _Py_arg(identifier arg, expr_ty annotation, string type_comment, int lineno, int col_offset, int end_lineno, int end_col_offset, diff --git a/Include/asdl.h b/Include/asdl.h index e962560bcd4cb..8b61e16c329ea 100644 --- a/Include/asdl.h +++ b/Include/asdl.h @@ -13,25 +13,80 @@ typedef PyObject * constant; interned Python strings. */ -/* XXX A sequence should be typed so that its use can be typechecked. */ +#define _ASDL_SEQ_HEAD \ + Py_ssize_t size; \ + void **elements; typedef struct { - Py_ssize_t size; - void *elements[1]; + _ASDL_SEQ_HEAD } asdl_seq; typedef struct { - Py_ssize_t size; - int elements[1]; + _ASDL_SEQ_HEAD + void *typed_elements[1]; +} asdl_generic_seq; + +typedef struct { + _ASDL_SEQ_HEAD + PyObject *typed_elements[1]; +} asdl_identifier_seq; + +typedef struct { + _ASDL_SEQ_HEAD + int typed_elements[1]; } asdl_int_seq; -asdl_seq *_Py_asdl_seq_new(Py_ssize_t size, PyArena *arena); +asdl_generic_seq *_Py_asdl_generic_seq_new(Py_ssize_t size, PyArena *arena); +asdl_identifier_seq *_Py_asdl_identifier_seq_new(Py_ssize_t size, PyArena *arena); asdl_int_seq *_Py_asdl_int_seq_new(Py_ssize_t size, PyArena *arena); -#define asdl_seq_GET(S, I) (S)->elements[(I)] + +#define GENERATE_ASDL_SEQ_CONSTRUCTOR(NAME, TYPE) \ +asdl_ ## NAME ## _seq *_Py_asdl_ ## NAME ## _seq_new(Py_ssize_t size, PyArena *arena) \ +{ \ + asdl_ ## NAME ## _seq *seq = NULL; \ + size_t n; \ + /* check size is sane */ \ + if (size < 0 || \ + (size && (((size_t)size - 1) > (SIZE_MAX / sizeof(void *))))) { \ + PyErr_NoMemory(); \ + return NULL; \ + } \ + n = (size ? (sizeof(TYPE *) * (size - 1)) : 0); \ + /* check if size can be added safely */ \ + if (n > SIZE_MAX - sizeof(asdl_ ## NAME ## _seq)) { \ + PyErr_NoMemory(); \ + return NULL; \ + } \ + n += sizeof(asdl_ ## NAME ## _seq); \ + seq = (asdl_ ## NAME ## _seq *)PyArena_Malloc(arena, n); \ + if (!seq) { \ + PyErr_NoMemory(); \ + return NULL; \ + } \ + memset(seq, 0, n); \ + seq->size = size; \ + seq->elements = (void**)seq->typed_elements; \ + return seq; \ +} + +#define asdl_seq_GET_UNTYPED(S, I) (S)->elements[(I)] +#define asdl_seq_GET(S, I) (S)->typed_elements[(I)] #define asdl_seq_LEN(S) ((S) == NULL ? 0 : (S)->size) #ifdef Py_DEBUG #define asdl_seq_SET(S, I, V) \ + do { \ + Py_ssize_t _asdl_i = (I); \ + assert((S) != NULL); \ + assert(0 <= _asdl_i && _asdl_i < (S)->size); \ + (S)->typed_elements[_asdl_i] = (V); \ + } while (0) +#else +#define asdl_seq_SET(S, I, V) (S)->typed_elements[I] = (V) +#endif + +#ifdef Py_DEBUG +#define asdl_seq_SET_UNTYPED(S, I, V) \ do { \ Py_ssize_t _asdl_i = (I); \ assert((S) != NULL); \ @@ -39,7 +94,7 @@ asdl_int_seq *_Py_asdl_int_seq_new(Py_ssize_t size, PyArena *arena); (S)->elements[_asdl_i] = (V); \ } while (0) #else -#define asdl_seq_SET(S, I, V) (S)->elements[I] = (V) +#define asdl_seq_SET_UNTYPED(S, I, V) (S)->elements[I] = (V) #endif #endif /* !Py_ASDL_H */ diff --git a/Include/ast.h b/Include/ast.h index de42a3b5e6f91..434ee18dd91b2 100644 --- a/Include/ast.h +++ b/Include/ast.h @@ -15,7 +15,7 @@ PyAPI_FUNC(PyObject *) _PyAST_ExprAsUnicode(expr_ty); /* Return the borrowed reference to the first literal string in the sequence of statements or NULL if it doesn't start from a literal string. Doesn't set exception. */ -PyAPI_FUNC(PyObject *) _PyAST_GetDocString(asdl_seq *); +PyAPI_FUNC(PyObject *) _PyAST_GetDocString(asdl_stmt_seq *); #ifdef __cplusplus } diff --git a/Parser/asdl_c.py b/Parser/asdl_c.py index 0c053393d688b..242eccf3d37d7 100755 --- a/Parser/asdl_c.py +++ b/Parser/asdl_c.py @@ -163,6 +163,32 @@ def visitProduct(self, product, name, depth): self.emit(s, depth) self.emit("", depth) +class SequenceDefVisitor(EmitVisitor): + def visitModule(self, mod): + for dfn in mod.dfns: + self.visit(dfn) + + def visitType(self, type, depth=0): + self.visit(type.value, type.name, depth) + + def visitSum(self, sum, name, depth): + if is_simple(sum): + return + self.emit_sequence_constructor(name, depth) + + def emit_sequence_constructor(self, name,depth): + ctype = get_c_type(name) + self.emit("""\ +typedef struct { + _ASDL_SEQ_HEAD + %(ctype)s typed_elements[1]; +} asdl_%(name)s_seq;""" % locals(), reflow=False, depth=depth) + self.emit("", depth) + self.emit("asdl_%(name)s_seq *_Py_asdl_%(name)s_seq_new(Py_ssize_t size, PyArena *arena);" % locals(), depth) + self.emit("", depth) + + def visitProduct(self, product, name, depth): + self.emit_sequence_constructor(name, depth) class StructVisitor(EmitVisitor): """Visitor to generate typedefs for AST.""" @@ -219,7 +245,8 @@ def visitField(self, field, depth): if field.type == 'cmpop': self.emit("asdl_int_seq *%(name)s;" % locals(), depth) else: - self.emit("asdl_seq *%(name)s;" % locals(), depth) + _type = field.type + self.emit("asdl_%(_type)s_seq *%(name)s;" % locals(), depth) else: self.emit("%(ctype)s %(name)s;" % locals(), depth) @@ -274,7 +301,7 @@ def get_args(self, fields): if f.type == 'cmpop': ctype = "asdl_int_seq *" else: - ctype = "asdl_seq *" + ctype = f"asdl_{f.type}_seq *" else: ctype = get_c_type(f.type) args.append((ctype, name, f.opt or f.seq)) @@ -507,7 +534,8 @@ def visitFieldDeclaration(self, field, name, sum=None, prod=None, depth=0): if self.isSimpleType(field): self.emit("asdl_int_seq* %s;" % field.name, depth) else: - self.emit("asdl_seq* %s;" % field.name, depth) + _type = field.type + self.emit(f"asdl_{field.type}_seq* {field.name};", depth) else: ctype = get_c_type(field.type) self.emit("%s %s;" % (ctype, field.name), depth) @@ -562,7 +590,7 @@ def visitField(self, field, name, sum=None, prod=None, depth=0): if self.isSimpleType(field): self.emit("%s = _Py_asdl_int_seq_new(len, arena);" % field.name, depth+1) else: - self.emit("%s = _Py_asdl_seq_new(len, arena);" % field.name, depth+1) + self.emit("%s = _Py_asdl_%s_seq_new(len, arena);" % (field.name, field.type), depth+1) self.emit("if (%s == NULL) goto failed;" % field.name, depth+1) self.emit("for (i = 0; i < len; i++) {", depth+1) self.emit("%s val;" % ctype, depth+2) @@ -600,6 +628,24 @@ def prototype(self, sum, name): visitProduct = visitSum = prototype +class SequenceConstructorVisitor(EmitVisitor): + def visitModule(self, mod): + for dfn in mod.dfns: + self.visit(dfn) + + def visitType(self, type): + self.visit(type.value, type.name) + + def visitProduct(self, prod, name): + self.emit_sequence_constructor(name, get_c_type(name)) + + def visitSum(self, sum, name): + if not is_simple(sum): + self.emit_sequence_constructor(name, get_c_type(name)) + + def emit_sequence_constructor(self, name, type): + self.emit(f"GENERATE_ASDL_SEQ_CONSTRUCTOR({name}, {type})", depth=0) + class PyTypesDeclareVisitor(PickleVisitor): def visitProduct(self, prod, name): @@ -647,6 +693,7 @@ def visitConstructor(self, cons, name): self.emit('"%s",' % t.name, 1) self.emit("};",0) + class PyTypesVisitor(PickleVisitor): def visitModule(self, mod): @@ -874,7 +921,7 @@ def visitModule(self, mod): if (!result) return NULL; for (i = 0; i < n; i++) { - value = func(state, asdl_seq_GET(seq, i)); + value = func(state, asdl_seq_GET_UNTYPED(seq, i)); if (!value) { Py_DECREF(result); return NULL; @@ -1264,7 +1311,7 @@ def set(self, field, value, depth): depth+2, reflow=False) self.emit("}", depth) else: - self.emit("value = ast2obj_list(state, %s, ast2obj_%s);" % (value, field.type), depth) + self.emit("value = ast2obj_list(state, (asdl_seq*)%s, ast2obj_%s);" % (value, field.type), depth) else: ctype = get_c_type(field.type) self.emit("value = ast2obj_%s(state, %s);" % (field.type, value), depth, reflow=False) @@ -1431,6 +1478,7 @@ def write_header(f, mod): f.write('#undef Yield /* undefine macro conflicting with */\n') f.write('\n') c = ChainOfVisitors(TypeDefVisitor(f), + SequenceDefVisitor(f), StructVisitor(f)) c.visit(mod) f.write("// Note: these macros affect function definitions, not only call sites.\n") @@ -1457,6 +1505,7 @@ def write_source(f, mod): generate_module_def(f, mod) v = ChainOfVisitors( + SequenceConstructorVisitor(f), PyTypesDeclareVisitor(f), PyTypesVisitor(f), Obj2ModPrototypeVisitor(f), diff --git a/Parser/parser.c b/Parser/parser.c index 8a7cb62fd7cf8..1bd74a38fbc2b 100644 --- a/Parser/parser.c +++ b/Parser/parser.c @@ -389,11 +389,11 @@ static mod_ty interactive_rule(Parser *p); static mod_ty eval_rule(Parser *p); static mod_ty func_type_rule(Parser *p); static expr_ty fstring_rule(Parser *p); -static asdl_seq* type_expressions_rule(Parser *p); -static asdl_seq* statements_rule(Parser *p); -static asdl_seq* statement_rule(Parser *p); -static asdl_seq* statement_newline_rule(Parser *p); -static asdl_seq* simple_stmt_rule(Parser *p); +static asdl_expr_seq* type_expressions_rule(Parser *p); +static asdl_stmt_seq* statements_rule(Parser *p); +static asdl_stmt_seq* statement_rule(Parser *p); +static asdl_stmt_seq* statement_newline_rule(Parser *p); +static asdl_stmt_seq* simple_stmt_rule(Parser *p); static stmt_ty small_stmt_rule(Parser *p); static stmt_ty compound_stmt_rule(Parser *p); static stmt_ty assignment_rule(Parser *p); @@ -406,22 +406,22 @@ static stmt_ty del_stmt_rule(Parser *p); static stmt_ty import_stmt_rule(Parser *p); static stmt_ty import_name_rule(Parser *p); static stmt_ty import_from_rule(Parser *p); -static asdl_seq* import_from_targets_rule(Parser *p); -static asdl_seq* import_from_as_names_rule(Parser *p); +static asdl_alias_seq* import_from_targets_rule(Parser *p); +static asdl_alias_seq* import_from_as_names_rule(Parser *p); static alias_ty import_from_as_name_rule(Parser *p); -static asdl_seq* dotted_as_names_rule(Parser *p); +static asdl_alias_seq* dotted_as_names_rule(Parser *p); static alias_ty dotted_as_name_rule(Parser *p); static expr_ty dotted_name_rule(Parser *p); static stmt_ty if_stmt_rule(Parser *p); static stmt_ty elif_stmt_rule(Parser *p); -static asdl_seq* else_block_rule(Parser *p); +static asdl_stmt_seq* else_block_rule(Parser *p); static stmt_ty while_stmt_rule(Parser *p); static stmt_ty for_stmt_rule(Parser *p); static stmt_ty with_stmt_rule(Parser *p); static withitem_ty with_item_rule(Parser *p); static stmt_ty try_stmt_rule(Parser *p); static excepthandler_ty except_block_rule(Parser *p); -static asdl_seq* finally_block_rule(Parser *p); +static asdl_stmt_seq* finally_block_rule(Parser *p); static stmt_ty return_stmt_rule(Parser *p); static stmt_ty raise_stmt_rule(Parser *p); static stmt_ty function_def_rule(Parser *p); @@ -429,7 +429,7 @@ static stmt_ty function_def_raw_rule(Parser *p); static Token* func_type_comment_rule(Parser *p); static arguments_ty params_rule(Parser *p); static arguments_ty parameters_rule(Parser *p); -static asdl_seq* slash_no_default_rule(Parser *p); +static asdl_arg_seq* slash_no_default_rule(Parser *p); static SlashWithDefault* slash_with_default_rule(Parser *p); static StarEtc* star_etc_rule(Parser *p); static arg_ty kwds_rule(Parser *p); @@ -439,14 +439,14 @@ static NameDefaultPair* param_maybe_default_rule(Parser *p); static arg_ty param_rule(Parser *p); static expr_ty annotation_rule(Parser *p); static expr_ty default_rule(Parser *p); -static asdl_seq* decorators_rule(Parser *p); +static asdl_expr_seq* decorators_rule(Parser *p); static stmt_ty class_def_rule(Parser *p); static stmt_ty class_def_raw_rule(Parser *p); -static asdl_seq* block_rule(Parser *p); -static asdl_seq* expressions_list_rule(Parser *p); +static asdl_stmt_seq* block_rule(Parser *p); +static asdl_expr_seq* expressions_list_rule(Parser *p); static expr_ty star_expressions_rule(Parser *p); static expr_ty star_expression_rule(Parser *p); -static asdl_seq* star_named_expressions_rule(Parser *p); +static asdl_expr_seq* star_named_expressions_rule(Parser *p); static expr_ty star_named_expression_rule(Parser *p); static expr_ty named_expression_rule(Parser *p); static expr_ty annotated_rhs_rule(Parser *p); @@ -455,7 +455,7 @@ static expr_ty expression_rule(Parser *p); static expr_ty lambdef_rule(Parser *p); static arguments_ty lambda_params_rule(Parser *p); static arguments_ty lambda_parameters_rule(Parser *p); -static asdl_seq* lambda_slash_no_default_rule(Parser *p); +static asdl_arg_seq* lambda_slash_no_default_rule(Parser *p); static SlashWithDefault* lambda_slash_with_default_rule(Parser *p); static StarEtc* lambda_star_etc_rule(Parser *p); static arg_ty lambda_kwds_rule(Parser *p); @@ -504,7 +504,7 @@ static expr_ty dictcomp_rule(Parser *p); static asdl_seq* double_starred_kvpairs_rule(Parser *p); static KeyValuePair* double_starred_kvpair_rule(Parser *p); static KeyValuePair* kvpair_rule(Parser *p); -static asdl_seq* for_if_clauses_rule(Parser *p); +static asdl_comprehension_seq* for_if_clauses_rule(Parser *p); static comprehension_ty for_if_clause_rule(Parser *p); static expr_ty yield_expr_rule(Parser *p); static expr_ty arguments_rule(Parser *p); @@ -514,15 +514,15 @@ static expr_ty starred_expression_rule(Parser *p); static KeywordOrStarred* kwarg_or_starred_rule(Parser *p); static KeywordOrStarred* kwarg_or_double_starred_rule(Parser *p); static expr_ty star_targets_rule(Parser *p); -static asdl_seq* star_targets_seq_rule(Parser *p); +static asdl_expr_seq* star_targets_seq_rule(Parser *p); static expr_ty star_target_rule(Parser *p); static expr_ty star_atom_rule(Parser *p); static expr_ty single_target_rule(Parser *p); static expr_ty single_subscript_attribute_target_rule(Parser *p); -static asdl_seq* del_targets_rule(Parser *p); +static asdl_expr_seq* del_targets_rule(Parser *p); static expr_ty del_target_rule(Parser *p); static expr_ty del_t_atom_rule(Parser *p); -static asdl_seq* targets_rule(Parser *p); +static asdl_expr_seq* targets_rule(Parser *p); static expr_ty target_rule(Parser *p); static expr_ty t_primary_rule(Parser *p); static void *t_lookahead_rule(Parser *p); @@ -764,7 +764,7 @@ interactive_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> interactive[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "statement_newline")); - asdl_seq* a; + asdl_stmt_seq* a; if ( (a = statement_newline_rule(p)) // statement_newline ) @@ -938,7 +938,7 @@ fstring_rule(Parser *p) // | '*' expression // | '**' expression // | ','.expression+ -static asdl_seq* +static asdl_expr_seq* type_expressions_rule(Parser *p) { D(p->level++); @@ -946,7 +946,7 @@ type_expressions_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_expr_seq* _res = NULL; int _mark = p->mark; { // ','.expression+ ',' '*' expression ',' '**' expression if (p->error_indicator) { @@ -978,7 +978,7 @@ type_expressions_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ type_expressions[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.expression+ ',' '*' expression ',' '**' expression")); - _res = _PyPegen_seq_append_to_end ( p , CHECK ( _PyPegen_seq_append_to_end ( p , a , b ) ) , c ); + _res = ( asdl_expr_seq * ) _PyPegen_seq_append_to_end ( p , CHECK ( _PyPegen_seq_append_to_end ( p , a , b ) ) , c ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -1011,7 +1011,7 @@ type_expressions_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ type_expressions[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.expression+ ',' '*' expression")); - _res = _PyPegen_seq_append_to_end ( p , a , b ); + _res = ( asdl_expr_seq * ) _PyPegen_seq_append_to_end ( p , a , b ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -1044,7 +1044,7 @@ type_expressions_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ type_expressions[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.expression+ ',' '**' expression")); - _res = _PyPegen_seq_append_to_end ( p , a , b ); + _res = ( asdl_expr_seq * ) _PyPegen_seq_append_to_end ( p , a , b ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -1080,7 +1080,7 @@ type_expressions_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ type_expressions[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' expression ',' '**' expression")); - _res = _PyPegen_seq_append_to_end ( p , CHECK ( _PyPegen_singleton_seq ( p , a ) ) , b ); + _res = ( asdl_expr_seq * ) _PyPegen_seq_append_to_end ( p , CHECK ( _PyPegen_singleton_seq ( p , a ) ) , b ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -1107,7 +1107,7 @@ type_expressions_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ type_expressions[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*' expression")); - _res = _PyPegen_singleton_seq ( p , a ); + _res = ( asdl_expr_seq * ) _PyPegen_singleton_seq ( p , a ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -1134,7 +1134,7 @@ type_expressions_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ type_expressions[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'**' expression")); - _res = _PyPegen_singleton_seq ( p , a ); + _res = ( asdl_expr_seq * ) _PyPegen_singleton_seq ( p , a ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -1152,13 +1152,18 @@ type_expressions_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> type_expressions[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.expression+")); - asdl_seq * _gather_9_var; + asdl_expr_seq* a; if ( - (_gather_9_var = _gather_9_rule(p)) // ','.expression+ + (a = (asdl_expr_seq*)_gather_9_rule(p)) // ','.expression+ ) { D(fprintf(stderr, "%*c+ type_expressions[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.expression+")); - _res = _gather_9_var; + _res = a; + if (_res == NULL && PyErr_Occurred()) { + p->error_indicator = 1; + D(p->level--); + return NULL; + } goto done; } p->mark = _mark; @@ -1172,7 +1177,7 @@ type_expressions_rule(Parser *p) } // statements: statement+ -static asdl_seq* +static asdl_stmt_seq* statements_rule(Parser *p) { D(p->level++); @@ -1180,7 +1185,7 @@ statements_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_stmt_seq* _res = NULL; int _mark = p->mark; { // statement+ if (p->error_indicator) { @@ -1194,7 +1199,7 @@ statements_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ statements[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "statement+")); - _res = _PyPegen_seq_flatten ( p , a ); + _res = ( asdl_stmt_seq * ) _PyPegen_seq_flatten ( p , a ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -1213,7 +1218,7 @@ statements_rule(Parser *p) } // statement: compound_stmt | simple_stmt -static asdl_seq* +static asdl_stmt_seq* statement_rule(Parser *p) { D(p->level++); @@ -1221,7 +1226,7 @@ statement_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_stmt_seq* _res = NULL; int _mark = p->mark; { // compound_stmt if (p->error_indicator) { @@ -1235,7 +1240,7 @@ statement_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ statement[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "compound_stmt")); - _res = _PyPegen_singleton_seq ( p , a ); + _res = ( asdl_stmt_seq * ) _PyPegen_singleton_seq ( p , a ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -1253,13 +1258,18 @@ statement_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> statement[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "simple_stmt")); - asdl_seq* simple_stmt_var; + asdl_stmt_seq* a; if ( - (simple_stmt_var = simple_stmt_rule(p)) // simple_stmt + (a = (asdl_stmt_seq*)simple_stmt_rule(p)) // simple_stmt ) { D(fprintf(stderr, "%*c+ statement[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "simple_stmt")); - _res = simple_stmt_var; + _res = a; + if (_res == NULL && PyErr_Occurred()) { + p->error_indicator = 1; + D(p->level--); + return NULL; + } goto done; } p->mark = _mark; @@ -1273,7 +1283,7 @@ statement_rule(Parser *p) } // statement_newline: compound_stmt NEWLINE | simple_stmt | NEWLINE | $ -static asdl_seq* +static asdl_stmt_seq* statement_newline_rule(Parser *p) { D(p->level++); @@ -1281,7 +1291,7 @@ statement_newline_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_stmt_seq* _res = NULL; int _mark = p->mark; if (p->mark == p->fill && _PyPegen_fill_token(p) < 0) { p->error_indicator = 1; @@ -1307,7 +1317,7 @@ statement_newline_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ statement_newline[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "compound_stmt NEWLINE")); - _res = _PyPegen_singleton_seq ( p , a ); + _res = ( asdl_stmt_seq * ) _PyPegen_singleton_seq ( p , a ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -1325,7 +1335,7 @@ statement_newline_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> statement_newline[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "simple_stmt")); - asdl_seq* simple_stmt_var; + asdl_stmt_seq* simple_stmt_var; if ( (simple_stmt_var = simple_stmt_rule(p)) // simple_stmt ) @@ -1359,7 +1369,7 @@ statement_newline_rule(Parser *p) UNUSED(_end_lineno); // Only used by EXTRA macro int _end_col_offset = _token->end_col_offset; UNUSED(_end_col_offset); // Only used by EXTRA macro - _res = _PyPegen_singleton_seq ( p , CHECK ( _Py_Pass ( EXTRA ) ) ); + _res = ( asdl_stmt_seq * ) _PyPegen_singleton_seq ( p , CHECK ( _Py_Pass ( EXTRA ) ) ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -1402,7 +1412,7 @@ statement_newline_rule(Parser *p) } // simple_stmt: small_stmt !';' NEWLINE | ';'.small_stmt+ ';'? NEWLINE -static asdl_seq* +static asdl_stmt_seq* simple_stmt_rule(Parser *p) { D(p->level++); @@ -1410,7 +1420,7 @@ simple_stmt_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_stmt_seq* _res = NULL; int _mark = p->mark; { // small_stmt !';' NEWLINE if (p->error_indicator) { @@ -1429,7 +1439,7 @@ simple_stmt_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ simple_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "small_stmt !';' NEWLINE")); - _res = _PyPegen_singleton_seq ( p , a ); + _res = ( asdl_stmt_seq * ) _PyPegen_singleton_seq ( p , a ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -1449,10 +1459,10 @@ simple_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> simple_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "';'.small_stmt+ ';'? NEWLINE")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings - asdl_seq * a; + asdl_stmt_seq* a; Token * newline_var; if ( - (a = _gather_12_rule(p)) // ';'.small_stmt+ + (a = (asdl_stmt_seq*)_gather_12_rule(p)) // ';'.small_stmt+ && (_opt_var = _PyPegen_expect_token(p, 13), 1) // ';'? && @@ -2127,11 +2137,11 @@ assignment_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> assignment[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "((star_targets '='))+ (yield_expr | star_expressions) !'=' TYPE_COMMENT?")); - asdl_seq * a; + asdl_expr_seq* a; void *b; void *tc; if ( - (a = _loop1_22_rule(p)) // ((star_targets '='))+ + (a = (asdl_expr_seq*)_loop1_22_rule(p)) // ((star_targets '='))+ && (b = _tmp_23_rule(p)) // yield_expr | star_expressions && @@ -2602,11 +2612,11 @@ global_stmt_rule(Parser *p) } D(fprintf(stderr, "%*c> global_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'global' ','.NAME+")); Token * _keyword; - asdl_seq * a; + asdl_expr_seq* a; if ( (_keyword = _PyPegen_expect_token(p, 508)) // token='global' && - (a = _gather_25_rule(p)) // ','.NAME+ + (a = (asdl_expr_seq*)_gather_25_rule(p)) // ','.NAME+ ) { D(fprintf(stderr, "%*c+ global_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'global' ','.NAME+")); @@ -2664,11 +2674,11 @@ nonlocal_stmt_rule(Parser *p) } D(fprintf(stderr, "%*c> nonlocal_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'nonlocal' ','.NAME+")); Token * _keyword; - asdl_seq * a; + asdl_expr_seq* a; if ( (_keyword = _PyPegen_expect_token(p, 509)) // token='nonlocal' && - (a = _gather_27_rule(p)) // ','.NAME+ + (a = (asdl_expr_seq*)_gather_27_rule(p)) // ','.NAME+ ) { D(fprintf(stderr, "%*c+ nonlocal_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'nonlocal' ','.NAME+")); @@ -2850,7 +2860,7 @@ del_stmt_rule(Parser *p) } D(fprintf(stderr, "%*c> del_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'del' del_targets &(';' | NEWLINE)")); Token * _keyword; - asdl_seq* a; + asdl_expr_seq* a; if ( (_keyword = _PyPegen_expect_token(p, 503)) // token='del' && @@ -2988,7 +2998,7 @@ import_name_rule(Parser *p) } D(fprintf(stderr, "%*c> import_name[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'import' dotted_as_names")); Token * _keyword; - asdl_seq* a; + asdl_alias_seq* a; if ( (_keyword = _PyPegen_expect_token(p, 513)) // token='import' && @@ -3055,7 +3065,7 @@ import_from_rule(Parser *p) Token * _keyword_1; asdl_seq * a; expr_ty b; - asdl_seq* c; + asdl_alias_seq* c; if ( (_keyword = _PyPegen_expect_token(p, 514)) // token='from' && @@ -3099,7 +3109,7 @@ import_from_rule(Parser *p) Token * _keyword; Token * _keyword_1; asdl_seq * a; - asdl_seq* b; + asdl_alias_seq* b; if ( (_keyword = _PyPegen_expect_token(p, 514)) // token='from' && @@ -3143,7 +3153,7 @@ import_from_rule(Parser *p) // | import_from_as_names !',' // | '*' // | invalid_import_from_targets -static asdl_seq* +static asdl_alias_seq* import_from_targets_rule(Parser *p) { D(p->level++); @@ -3151,7 +3161,7 @@ import_from_targets_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_alias_seq* _res = NULL; int _mark = p->mark; { // '(' import_from_as_names ','? ')' if (p->error_indicator) { @@ -3163,7 +3173,7 @@ import_from_targets_rule(Parser *p) Token * _literal_1; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings - asdl_seq* a; + asdl_alias_seq* a; if ( (_literal = _PyPegen_expect_token(p, 7)) // token='(' && @@ -3193,7 +3203,7 @@ import_from_targets_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> import_from_targets[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "import_from_as_names !','")); - asdl_seq* import_from_as_names_var; + asdl_alias_seq* import_from_as_names_var; if ( (import_from_as_names_var = import_from_as_names_rule(p)) // import_from_as_names && @@ -3220,7 +3230,7 @@ import_from_targets_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ import_from_targets[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'*'")); - _res = _PyPegen_singleton_seq ( p , CHECK ( _PyPegen_alias_for_star ( p ) ) ); + _res = ( asdl_alias_seq * ) _PyPegen_singleton_seq ( p , CHECK ( _PyPegen_alias_for_star ( p ) ) ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -3258,7 +3268,7 @@ import_from_targets_rule(Parser *p) } // import_from_as_names: ','.import_from_as_name+ -static asdl_seq* +static asdl_alias_seq* import_from_as_names_rule(Parser *p) { D(p->level++); @@ -3266,7 +3276,7 @@ import_from_as_names_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_alias_seq* _res = NULL; int _mark = p->mark; { // ','.import_from_as_name+ if (p->error_indicator) { @@ -3274,9 +3284,9 @@ import_from_as_names_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> import_from_as_names[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.import_from_as_name+")); - asdl_seq * a; + asdl_alias_seq* a; if ( - (a = _gather_33_rule(p)) // ','.import_from_as_name+ + (a = (asdl_alias_seq*)_gather_33_rule(p)) // ','.import_from_as_name+ ) { D(fprintf(stderr, "%*c+ import_from_as_names[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.import_from_as_name+")); @@ -3343,7 +3353,7 @@ import_from_as_name_rule(Parser *p) } // dotted_as_names: ','.dotted_as_name+ -static asdl_seq* +static asdl_alias_seq* dotted_as_names_rule(Parser *p) { D(p->level++); @@ -3351,7 +3361,7 @@ dotted_as_names_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_alias_seq* _res = NULL; int _mark = p->mark; { // ','.dotted_as_name+ if (p->error_indicator) { @@ -3359,9 +3369,9 @@ dotted_as_names_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> dotted_as_names[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.dotted_as_name+")); - asdl_seq * a; + asdl_alias_seq* a; if ( - (a = _gather_36_rule(p)) // ','.dotted_as_name+ + (a = (asdl_alias_seq*)_gather_36_rule(p)) // ','.dotted_as_name+ ) { D(fprintf(stderr, "%*c+ dotted_as_names[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "','.dotted_as_name+")); @@ -3554,7 +3564,7 @@ if_stmt_rule(Parser *p) Token * _keyword; Token * _literal; expr_ty a; - asdl_seq* b; + asdl_stmt_seq* b; stmt_ty c; if ( (_keyword = _PyPegen_expect_token(p, 510)) // token='if' @@ -3578,7 +3588,7 @@ if_stmt_rule(Parser *p) UNUSED(_end_lineno); // Only used by EXTRA macro int _end_col_offset = _token->end_col_offset; UNUSED(_end_col_offset); // Only used by EXTRA macro - _res = _Py_If ( a , b , CHECK ( _PyPegen_singleton_seq ( p , c ) ) , EXTRA ); + _res = _Py_If ( a , b , CHECK ( ( asdl_stmt_seq * ) _PyPegen_singleton_seq ( p , c ) ) , EXTRA ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -3599,7 +3609,7 @@ if_stmt_rule(Parser *p) Token * _keyword; Token * _literal; expr_ty a; - asdl_seq* b; + asdl_stmt_seq* b; void *c; if ( (_keyword = _PyPegen_expect_token(p, 510)) // token='if' @@ -3672,7 +3682,7 @@ elif_stmt_rule(Parser *p) Token * _keyword; Token * _literal; expr_ty a; - asdl_seq* b; + asdl_stmt_seq* b; stmt_ty c; if ( (_keyword = _PyPegen_expect_token(p, 515)) // token='elif' @@ -3717,7 +3727,7 @@ elif_stmt_rule(Parser *p) Token * _keyword; Token * _literal; expr_ty a; - asdl_seq* b; + asdl_stmt_seq* b; void *c; if ( (_keyword = _PyPegen_expect_token(p, 515)) // token='elif' @@ -3760,7 +3770,7 @@ elif_stmt_rule(Parser *p) } // else_block: 'else' ':' block -static asdl_seq* +static asdl_stmt_seq* else_block_rule(Parser *p) { D(p->level++); @@ -3768,7 +3778,7 @@ else_block_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_stmt_seq* _res = NULL; int _mark = p->mark; { // 'else' ':' block if (p->error_indicator) { @@ -3778,7 +3788,7 @@ else_block_rule(Parser *p) D(fprintf(stderr, "%*c> else_block[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'else' ':' block")); Token * _keyword; Token * _literal; - asdl_seq* b; + asdl_stmt_seq* b; if ( (_keyword = _PyPegen_expect_token(p, 516)) // token='else' && @@ -3835,7 +3845,7 @@ while_stmt_rule(Parser *p) Token * _keyword; Token * _literal; expr_ty a; - asdl_seq* b; + asdl_stmt_seq* b; void *c; if ( (_keyword = _PyPegen_expect_token(p, 512)) // token='while' @@ -3910,7 +3920,7 @@ for_stmt_rule(Parser *p) Token * _keyword; Token * _keyword_1; Token * _literal; - asdl_seq* b; + asdl_stmt_seq* b; void *el; expr_ty ex; expr_ty t; @@ -3972,7 +3982,7 @@ for_stmt_rule(Parser *p) Token * _keyword_1; Token * _literal; Token * async_var; - asdl_seq* b; + asdl_stmt_seq* b; void *el; expr_ty ex; expr_ty t; @@ -4086,14 +4096,14 @@ with_stmt_rule(Parser *p) Token * _literal_2; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings - asdl_seq * a; - asdl_seq* b; + asdl_withitem_seq* a; + asdl_stmt_seq* b; if ( (_keyword = _PyPegen_expect_token(p, 519)) // token='with' && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && - (a = _gather_39_rule(p)) // ','.with_item+ + (a = (asdl_withitem_seq*)_gather_39_rule(p)) // ','.with_item+ && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? && @@ -4134,13 +4144,13 @@ with_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'with' ','.with_item+ ':' TYPE_COMMENT? block")); Token * _keyword; Token * _literal; - asdl_seq * a; - asdl_seq* b; + asdl_withitem_seq* a; + asdl_stmt_seq* b; void *tc; if ( (_keyword = _PyPegen_expect_token(p, 519)) // token='with' && - (a = _gather_41_rule(p)) // ','.with_item+ + (a = (asdl_withitem_seq*)_gather_41_rule(p)) // ','.with_item+ && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -4183,9 +4193,9 @@ with_stmt_rule(Parser *p) Token * _literal_2; void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings - asdl_seq * a; + asdl_withitem_seq* a; Token * async_var; - asdl_seq* b; + asdl_stmt_seq* b; if ( (async_var = _PyPegen_expect_token(p, ASYNC)) // token='ASYNC' && @@ -4193,7 +4203,7 @@ with_stmt_rule(Parser *p) && (_literal = _PyPegen_expect_token(p, 7)) // token='(' && - (a = _gather_43_rule(p)) // ','.with_item+ + (a = (asdl_withitem_seq*)_gather_43_rule(p)) // ','.with_item+ && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? && @@ -4234,16 +4244,16 @@ with_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> with_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "ASYNC 'with' ','.with_item+ ':' TYPE_COMMENT? block")); Token * _keyword; Token * _literal; - asdl_seq * a; + asdl_withitem_seq* a; Token * async_var; - asdl_seq* b; + asdl_stmt_seq* b; void *tc; if ( (async_var = _PyPegen_expect_token(p, ASYNC)) // token='ASYNC' && (_keyword = _PyPegen_expect_token(p, 519)) // token='with' && - (a = _gather_45_rule(p)) // ','.with_item+ + (a = (asdl_withitem_seq*)_gather_45_rule(p)) // ','.with_item+ && (_literal = _PyPegen_expect_token(p, 11)) // token=':' && @@ -4402,8 +4412,8 @@ try_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> try_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'try' ':' block finally_block")); Token * _keyword; Token * _literal; - asdl_seq* b; - asdl_seq* f; + asdl_stmt_seq* b; + asdl_stmt_seq* f; if ( (_keyword = _PyPegen_expect_token(p, 511)) // token='try' && @@ -4444,9 +4454,9 @@ try_stmt_rule(Parser *p) D(fprintf(stderr, "%*c> try_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'try' ':' block except_block+ else_block? finally_block?")); Token * _keyword; Token * _literal; - asdl_seq* b; + asdl_stmt_seq* b; void *el; - asdl_seq * ex; + asdl_excepthandler_seq* ex; void *f; if ( (_keyword = _PyPegen_expect_token(p, 511)) // token='try' @@ -4455,7 +4465,7 @@ try_stmt_rule(Parser *p) && (b = block_rule(p)) // block && - (ex = _loop1_48_rule(p)) // except_block+ + (ex = (asdl_excepthandler_seq*)_loop1_48_rule(p)) // except_block+ && (el = else_block_rule(p), 1) // else_block? && @@ -4518,7 +4528,7 @@ except_block_rule(Parser *p) D(fprintf(stderr, "%*c> except_block[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except' expression ['as' NAME] ':' block")); Token * _keyword; Token * _literal; - asdl_seq* b; + asdl_stmt_seq* b; expr_ty e; void *t; if ( @@ -4563,7 +4573,7 @@ except_block_rule(Parser *p) D(fprintf(stderr, "%*c> except_block[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'except' ':' block")); Token * _keyword; Token * _literal; - asdl_seq* b; + asdl_stmt_seq* b; if ( (_keyword = _PyPegen_expect_token(p, 521)) // token='except' && @@ -4601,7 +4611,7 @@ except_block_rule(Parser *p) } // finally_block: 'finally' ':' block -static asdl_seq* +static asdl_stmt_seq* finally_block_rule(Parser *p) { D(p->level++); @@ -4609,7 +4619,7 @@ finally_block_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_stmt_seq* _res = NULL; int _mark = p->mark; { // 'finally' ':' block if (p->error_indicator) { @@ -4619,7 +4629,7 @@ finally_block_rule(Parser *p) D(fprintf(stderr, "%*c> finally_block[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'finally' ':' block")); Token * _keyword; Token * _literal; - asdl_seq* a; + asdl_stmt_seq* a; if ( (_keyword = _PyPegen_expect_token(p, 522)) // token='finally' && @@ -4824,7 +4834,7 @@ function_def_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> function_def[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "decorators function_def_raw")); - asdl_seq* d; + asdl_expr_seq* d; stmt_ty f; if ( (d = decorators_rule(p)) // decorators @@ -4903,7 +4913,7 @@ function_def_raw_rule(Parser *p) Token * _literal_1; Token * _literal_2; void *a; - asdl_seq* b; + asdl_stmt_seq* b; expr_ty n; void *params; void *tc; @@ -4961,7 +4971,7 @@ function_def_raw_rule(Parser *p) Token * _literal_2; void *a; Token * async_var; - asdl_seq* b; + asdl_stmt_seq* b; expr_ty n; void *params; void *tc; @@ -5179,14 +5189,14 @@ parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "slash_no_default param_no_default* param_with_default* star_etc?")); - asdl_seq* a; - asdl_seq * b; + asdl_arg_seq* a; + asdl_arg_seq* b; asdl_seq * c; void *d; if ( (a = slash_no_default_rule(p)) // slash_no_default && - (b = _loop0_54_rule(p)) // param_no_default* + (b = (asdl_arg_seq*)_loop0_54_rule(p)) // param_no_default* && (c = _loop0_55_rule(p)) // param_with_default* && @@ -5242,11 +5252,11 @@ parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default+ param_with_default* star_etc?")); - asdl_seq * a; + asdl_arg_seq* a; asdl_seq * b; void *c; if ( - (a = _loop1_57_rule(p)) // param_no_default+ + (a = (asdl_arg_seq*)_loop1_57_rule(p)) // param_no_default+ && (b = _loop0_58_rule(p)) // param_with_default* && @@ -5324,7 +5334,7 @@ parameters_rule(Parser *p) } // slash_no_default: param_no_default+ '/' ',' | param_no_default+ '/' &')' -static asdl_seq* +static asdl_arg_seq* slash_no_default_rule(Parser *p) { D(p->level++); @@ -5332,7 +5342,7 @@ slash_no_default_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_arg_seq* _res = NULL; int _mark = p->mark; { // param_no_default+ '/' ',' if (p->error_indicator) { @@ -5342,9 +5352,9 @@ slash_no_default_rule(Parser *p) D(fprintf(stderr, "%*c> slash_no_default[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default+ '/' ','")); Token * _literal; Token * _literal_1; - asdl_seq * a; + asdl_arg_seq* a; if ( - (a = _loop1_60_rule(p)) // param_no_default+ + (a = (asdl_arg_seq*)_loop1_60_rule(p)) // param_no_default+ && (_literal = _PyPegen_expect_token(p, 17)) // token='/' && @@ -5371,9 +5381,9 @@ slash_no_default_rule(Parser *p) } D(fprintf(stderr, "%*c> slash_no_default[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "param_no_default+ '/' &')'")); Token * _literal; - asdl_seq * a; + asdl_arg_seq* a; if ( - (a = _loop1_61_rule(p)) // param_no_default+ + (a = (asdl_arg_seq*)_loop1_61_rule(p)) // param_no_default+ && (_literal = _PyPegen_expect_token(p, 17)) // token='/' && @@ -5433,7 +5443,7 @@ slash_with_default_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ slash_with_default[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "param_no_default* param_with_default+ '/' ','")); - _res = _PyPegen_slash_with_default ( p , a , b ); + _res = _PyPegen_slash_with_default ( p , ( asdl_arg_seq * ) a , b ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -5465,7 +5475,7 @@ slash_with_default_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ slash_with_default[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "param_no_default* param_with_default+ '/' &')'")); - _res = _PyPegen_slash_with_default ( p , a , b ); + _res = _PyPegen_slash_with_default ( p , ( asdl_arg_seq * ) a , b ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -6050,7 +6060,7 @@ default_rule(Parser *p) } // decorators: (('@' named_expression NEWLINE))+ -static asdl_seq* +static asdl_expr_seq* decorators_rule(Parser *p) { D(p->level++); @@ -6058,7 +6068,7 @@ decorators_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_expr_seq* _res = NULL; int _mark = p->mark; { // (('@' named_expression NEWLINE))+ if (p->error_indicator) { @@ -6066,9 +6076,9 @@ decorators_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> decorators[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "(('@' named_expression NEWLINE))+")); - asdl_seq * a; + asdl_expr_seq* a; if ( - (a = _loop1_68_rule(p)) // (('@' named_expression NEWLINE))+ + (a = (asdl_expr_seq*)_loop1_68_rule(p)) // (('@' named_expression NEWLINE))+ ) { D(fprintf(stderr, "%*c+ decorators[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "(('@' named_expression NEWLINE))+")); @@ -6107,7 +6117,7 @@ class_def_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> class_def[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "decorators class_def_raw")); - asdl_seq* a; + asdl_expr_seq* a; stmt_ty b; if ( (a = decorators_rule(p)) // decorators @@ -6183,7 +6193,7 @@ class_def_raw_rule(Parser *p) Token * _literal; expr_ty a; void *b; - asdl_seq* c; + asdl_stmt_seq* c; if ( (_keyword = _PyPegen_expect_token(p, 524)) // token='class' && @@ -6225,7 +6235,7 @@ class_def_raw_rule(Parser *p) } // block: NEWLINE INDENT statements DEDENT | simple_stmt | invalid_block -static asdl_seq* +static asdl_stmt_seq* block_rule(Parser *p) { D(p->level++); @@ -6233,7 +6243,7 @@ block_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_stmt_seq* _res = NULL; if (_PyPegen_is_memoized(p, block_type, &_res)) { D(p->level--); return _res; @@ -6245,7 +6255,7 @@ block_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> block[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "NEWLINE INDENT statements DEDENT")); - asdl_seq* a; + asdl_stmt_seq* a; Token * dedent_var; Token * indent_var; Token * newline_var; @@ -6278,7 +6288,7 @@ block_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> block[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "simple_stmt")); - asdl_seq* simple_stmt_var; + asdl_stmt_seq* simple_stmt_var; if ( (simple_stmt_var = simple_stmt_rule(p)) // simple_stmt ) @@ -6318,7 +6328,7 @@ block_rule(Parser *p) } // expressions_list: ','.star_expression+ ','? -static asdl_seq* +static asdl_expr_seq* expressions_list_rule(Parser *p) { D(p->level++); @@ -6326,7 +6336,7 @@ expressions_list_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_expr_seq* _res = NULL; int _mark = p->mark; { // ','.star_expression+ ','? if (p->error_indicator) { @@ -6336,9 +6346,9 @@ expressions_list_rule(Parser *p) D(fprintf(stderr, "%*c> expressions_list[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.star_expression+ ','?")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings - asdl_seq * a; + asdl_expr_seq* a; if ( - (a = _gather_70_rule(p)) // ','.star_expression+ + (a = (asdl_expr_seq*)_gather_70_rule(p)) // ','.star_expression+ && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? ) @@ -6573,7 +6583,7 @@ star_expression_rule(Parser *p) } // star_named_expressions: ','.star_named_expression+ ','? -static asdl_seq* +static asdl_expr_seq* star_named_expressions_rule(Parser *p) { D(p->level++); @@ -6581,7 +6591,7 @@ star_named_expressions_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_expr_seq* _res = NULL; int _mark = p->mark; { // ','.star_named_expression+ ','? if (p->error_indicator) { @@ -6591,9 +6601,9 @@ star_named_expressions_rule(Parser *p) D(fprintf(stderr, "%*c> star_named_expressions[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.star_named_expression+ ','?")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings - asdl_seq * a; + asdl_expr_seq* a; if ( - (a = _gather_73_rule(p)) // ','.star_named_expression+ + (a = (asdl_expr_seq*)_gather_73_rule(p)) // ','.star_named_expression+ && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? ) @@ -7245,14 +7255,14 @@ lambda_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> lambda_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_slash_no_default lambda_param_no_default* lambda_param_with_default* lambda_star_etc?")); - asdl_seq* a; - asdl_seq * b; + asdl_arg_seq* a; + asdl_arg_seq* b; asdl_seq * c; void *d; if ( (a = lambda_slash_no_default_rule(p)) // lambda_slash_no_default && - (b = _loop0_76_rule(p)) // lambda_param_no_default* + (b = (asdl_arg_seq*)_loop0_76_rule(p)) // lambda_param_no_default* && (c = _loop0_77_rule(p)) // lambda_param_with_default* && @@ -7308,11 +7318,11 @@ lambda_parameters_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> lambda_parameters[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default+ lambda_param_with_default* lambda_star_etc?")); - asdl_seq * a; + asdl_arg_seq* a; asdl_seq * b; void *c; if ( - (a = _loop1_79_rule(p)) // lambda_param_no_default+ + (a = (asdl_arg_seq*)_loop1_79_rule(p)) // lambda_param_no_default+ && (b = _loop0_80_rule(p)) // lambda_param_with_default* && @@ -7392,7 +7402,7 @@ lambda_parameters_rule(Parser *p) // lambda_slash_no_default: // | lambda_param_no_default+ '/' ',' // | lambda_param_no_default+ '/' &':' -static asdl_seq* +static asdl_arg_seq* lambda_slash_no_default_rule(Parser *p) { D(p->level++); @@ -7400,7 +7410,7 @@ lambda_slash_no_default_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_arg_seq* _res = NULL; int _mark = p->mark; { // lambda_param_no_default+ '/' ',' if (p->error_indicator) { @@ -7410,9 +7420,9 @@ lambda_slash_no_default_rule(Parser *p) D(fprintf(stderr, "%*c> lambda_slash_no_default[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default+ '/' ','")); Token * _literal; Token * _literal_1; - asdl_seq * a; + asdl_arg_seq* a; if ( - (a = _loop1_82_rule(p)) // lambda_param_no_default+ + (a = (asdl_arg_seq*)_loop1_82_rule(p)) // lambda_param_no_default+ && (_literal = _PyPegen_expect_token(p, 17)) // token='/' && @@ -7439,9 +7449,9 @@ lambda_slash_no_default_rule(Parser *p) } D(fprintf(stderr, "%*c> lambda_slash_no_default[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default+ '/' &':'")); Token * _literal; - asdl_seq * a; + asdl_arg_seq* a; if ( - (a = _loop1_83_rule(p)) // lambda_param_no_default+ + (a = (asdl_arg_seq*)_loop1_83_rule(p)) // lambda_param_no_default+ && (_literal = _PyPegen_expect_token(p, 17)) // token='/' && @@ -7501,7 +7511,7 @@ lambda_slash_with_default_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ lambda_slash_with_default[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default* lambda_param_with_default+ '/' ','")); - _res = _PyPegen_slash_with_default ( p , a , b ); + _res = _PyPegen_slash_with_default ( p , ( asdl_arg_seq * ) a , b ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -7533,7 +7543,7 @@ lambda_slash_with_default_rule(Parser *p) ) { D(fprintf(stderr, "%*c+ lambda_slash_with_default[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "lambda_param_no_default* lambda_param_with_default+ '/' &':'")); - _res = _PyPegen_slash_with_default ( p , a , b ); + _res = _PyPegen_slash_with_default ( p , ( asdl_arg_seq * ) a , b ); if (_res == NULL && PyErr_Occurred()) { p->error_indicator = 1; D(p->level--); @@ -10553,9 +10563,9 @@ slices_rule(Parser *p) D(fprintf(stderr, "%*c> slices[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.slice+ ','?")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings - asdl_seq * a; + asdl_expr_seq* a; if ( - (a = _gather_94_rule(p)) // ','.slice+ + (a = (asdl_expr_seq*)_gather_94_rule(p)) // ','.slice+ && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? ) @@ -11111,7 +11121,7 @@ listcomp_rule(Parser *p) Token * _literal; Token * _literal_1; expr_ty a; - asdl_seq* b; + asdl_comprehension_seq* b; if ( (_literal = _PyPegen_expect_token(p, 9)) // token='[' && @@ -11336,7 +11346,7 @@ genexp_rule(Parser *p) Token * _literal; Token * _literal_1; expr_ty a; - asdl_seq* b; + asdl_comprehension_seq* b; if ( (_literal = _PyPegen_expect_token(p, 7)) // token='(' && @@ -11428,7 +11438,7 @@ set_rule(Parser *p) D(fprintf(stderr, "%*c> set[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'{' expressions_list '}'")); Token * _literal; Token * _literal_1; - asdl_seq* a; + asdl_expr_seq* a; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' && @@ -11495,7 +11505,7 @@ setcomp_rule(Parser *p) Token * _literal; Token * _literal_1; expr_ty a; - asdl_seq* b; + asdl_comprehension_seq* b; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' && @@ -11653,7 +11663,7 @@ dictcomp_rule(Parser *p) Token * _literal; Token * _literal_1; KeyValuePair* a; - asdl_seq* b; + asdl_comprehension_seq* b; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' && @@ -11867,7 +11877,7 @@ kvpair_rule(Parser *p) } // for_if_clauses: for_if_clause+ -static asdl_seq* +static asdl_comprehension_seq* for_if_clauses_rule(Parser *p) { D(p->level++); @@ -11875,7 +11885,7 @@ for_if_clauses_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_comprehension_seq* _res = NULL; int _mark = p->mark; { // for_if_clause+ if (p->error_indicator) { @@ -11883,13 +11893,18 @@ for_if_clauses_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> for_if_clauses[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "for_if_clause+")); - asdl_seq * _loop1_105_var; + asdl_comprehension_seq* a; if ( - (_loop1_105_var = _loop1_105_rule(p)) // for_if_clause+ + (a = (asdl_comprehension_seq*)_loop1_105_rule(p)) // for_if_clause+ ) { D(fprintf(stderr, "%*c+ for_if_clauses[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "for_if_clause+")); - _res = _loop1_105_var; + _res = a; + if (_res == NULL && PyErr_Occurred()) { + p->error_indicator = 1; + D(p->level--); + return NULL; + } goto done; } p->mark = _mark; @@ -11928,7 +11943,7 @@ for_if_clause_rule(Parser *p) expr_ty a; Token * async_var; expr_ty b; - asdl_seq * c; + asdl_expr_seq* c; if ( (async_var = _PyPegen_expect_token(p, ASYNC)) // token='ASYNC' && @@ -11942,7 +11957,7 @@ for_if_clause_rule(Parser *p) && (b = disjunction_rule(p)) // disjunction && - (c = _loop0_106_rule(p)) // (('if' disjunction))* + (c = (asdl_expr_seq*)_loop0_106_rule(p)) // (('if' disjunction))* ) { D(fprintf(stderr, "%*c+ for_if_clause[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "ASYNC 'for' star_targets 'in' ~ disjunction (('if' disjunction))*")); @@ -11973,7 +11988,7 @@ for_if_clause_rule(Parser *p) Token * _keyword_1; expr_ty a; expr_ty b; - asdl_seq * c; + asdl_expr_seq* c; if ( (_keyword = _PyPegen_expect_token(p, 517)) // token='for' && @@ -11985,7 +12000,7 @@ for_if_clause_rule(Parser *p) && (b = disjunction_rule(p)) // disjunction && - (c = _loop0_107_rule(p)) // (('if' disjunction))* + (c = (asdl_expr_seq*)_loop0_107_rule(p)) // (('if' disjunction))* ) { D(fprintf(stderr, "%*c+ for_if_clause[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'for' star_targets 'in' ~ disjunction (('if' disjunction))*")); @@ -12228,10 +12243,10 @@ args_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> args[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.(starred_expression | named_expression !'=')+ [',' kwargs]")); - asdl_seq * a; + asdl_expr_seq* a; void *b; if ( - (a = _gather_108_rule(p)) // ','.(starred_expression | named_expression !'=')+ + (a = (asdl_expr_seq*)_gather_108_rule(p)) // ','.(starred_expression | named_expression !'=')+ && (b = _tmp_110_rule(p), 1) // [',' kwargs] ) @@ -12768,7 +12783,7 @@ star_targets_rule(Parser *p) } // star_targets_seq: ','.star_target+ ','? -static asdl_seq* +static asdl_expr_seq* star_targets_seq_rule(Parser *p) { D(p->level++); @@ -12776,7 +12791,7 @@ star_targets_seq_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_expr_seq* _res = NULL; int _mark = p->mark; { // ','.star_target+ ','? if (p->error_indicator) { @@ -12786,9 +12801,9 @@ star_targets_seq_rule(Parser *p) D(fprintf(stderr, "%*c> star_targets_seq[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.star_target+ ','?")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings - asdl_seq * a; + asdl_expr_seq* a; if ( - (a = _gather_120_rule(p)) // ','.star_target+ + (a = (asdl_expr_seq*)_gather_120_rule(p)) // ','.star_target+ && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? ) @@ -13353,7 +13368,7 @@ single_subscript_attribute_target_rule(Parser *p) } // del_targets: ','.del_target+ ','? -static asdl_seq* +static asdl_expr_seq* del_targets_rule(Parser *p) { D(p->level++); @@ -13361,7 +13376,7 @@ del_targets_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_expr_seq* _res = NULL; int _mark = p->mark; { // ','.del_target+ ','? if (p->error_indicator) { @@ -13371,9 +13386,9 @@ del_targets_rule(Parser *p) D(fprintf(stderr, "%*c> del_targets[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.del_target+ ','?")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings - asdl_seq * a; + asdl_expr_seq* a; if ( - (a = _gather_123_rule(p)) // ','.del_target+ + (a = (asdl_expr_seq*)_gather_123_rule(p)) // ','.del_target+ && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? ) @@ -13694,7 +13709,7 @@ del_t_atom_rule(Parser *p) } // targets: ','.target+ ','? -static asdl_seq* +static asdl_expr_seq* targets_rule(Parser *p) { D(p->level++); @@ -13702,7 +13717,7 @@ targets_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq* _res = NULL; + asdl_expr_seq* _res = NULL; int _mark = p->mark; { // ','.target+ ','? if (p->error_indicator) { @@ -13712,9 +13727,9 @@ targets_rule(Parser *p) D(fprintf(stderr, "%*c> targets[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "','.target+ ','?")); void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings - asdl_seq * a; + asdl_expr_seq* a; if ( - (a = _gather_125_rule(p)) // ','.target+ + (a = (asdl_expr_seq*)_gather_125_rule(p)) // ','.target+ && (_opt_var = _PyPegen_expect_token(p, 12), 1) // ','? ) @@ -14418,7 +14433,7 @@ incorrect_arguments_rule(Parser *p) void *_opt_var; UNUSED(_opt_var); // Silence compiler warnings expr_ty a; - asdl_seq* for_if_clauses_var; + asdl_comprehension_seq* for_if_clauses_var; if ( (a = expression_rule(p)) // expression && @@ -14449,7 +14464,7 @@ incorrect_arguments_rule(Parser *p) } D(fprintf(stderr, "%*c> incorrect_arguments[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "args for_if_clauses")); expr_ty a; - asdl_seq* for_if_clauses_var; + asdl_comprehension_seq* for_if_clauses_var; if ( (a = args_rule(p)) // args && @@ -14478,7 +14493,7 @@ incorrect_arguments_rule(Parser *p) Token * _literal; expr_ty a; expr_ty args_var; - asdl_seq* for_if_clauses_var; + asdl_comprehension_seq* for_if_clauses_var; if ( (args_var = args_rule(p)) // args && @@ -15029,7 +15044,7 @@ invalid_comprehension_rule(Parser *p) D(fprintf(stderr, "%*c> invalid_comprehension[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "('[' | '(' | '{') starred_expression for_if_clauses")); void *_tmp_132_var; expr_ty a; - asdl_seq* for_if_clauses_var; + asdl_comprehension_seq* for_if_clauses_var; if ( (_tmp_132_var = _tmp_132_rule(p)) // '[' | '(' | '{' && @@ -15078,7 +15093,7 @@ invalid_dict_comprehension_rule(Parser *p) Token * _literal_1; Token * a; expr_ty bitwise_or_var; - asdl_seq* for_if_clauses_var; + asdl_comprehension_seq* for_if_clauses_var; if ( (_literal = _PyPegen_expect_token(p, 25)) // token='{' && @@ -15537,7 +15552,7 @@ invalid_import_from_targets_rule(Parser *p) } D(fprintf(stderr, "%*c> invalid_import_from_targets[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "import_from_as_names ','")); Token * _literal; - asdl_seq* import_from_as_names_var; + asdl_alias_seq* import_from_as_names_var; if ( (import_from_as_names_var = import_from_as_names_rule(p)) // import_from_as_names && @@ -15614,7 +15629,7 @@ _loop0_1_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_1[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "NEWLINE")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -15622,7 +15637,7 @@ _loop0_1_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_1_type, _seq); D(p->level--); @@ -15680,7 +15695,7 @@ _loop0_2_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_2[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "NEWLINE")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -15688,7 +15703,7 @@ _loop0_2_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_2_type, _seq); D(p->level--); @@ -15755,7 +15770,7 @@ _loop0_4_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_4[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' expression")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -15763,7 +15778,7 @@ _loop0_4_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_4_type, _seq); D(p->level--); @@ -15869,7 +15884,7 @@ _loop0_6_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_6[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' expression")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -15877,7 +15892,7 @@ _loop0_6_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_6_type, _seq); D(p->level--); @@ -15983,7 +15998,7 @@ _loop0_8_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_8[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' expression")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -15991,7 +16006,7 @@ _loop0_8_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_8_type, _seq); D(p->level--); @@ -16097,7 +16112,7 @@ _loop0_10_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_10[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' expression")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -16105,7 +16120,7 @@ _loop0_10_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_10_type, _seq); D(p->level--); @@ -16178,7 +16193,7 @@ _loop1_11_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop1_11[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "statement")); - asdl_seq* statement_var; + asdl_stmt_seq* statement_var; while ( (statement_var = statement_rule(p)) // statement ) @@ -16207,7 +16222,7 @@ _loop1_11_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -16215,7 +16230,7 @@ _loop1_11_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_11_type, _seq); D(p->level--); @@ -16282,7 +16297,7 @@ _loop0_13_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_13[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "';' small_stmt")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -16290,7 +16305,7 @@ _loop0_13_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_13_type, _seq); D(p->level--); @@ -16840,7 +16855,7 @@ _loop1_22_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -16848,7 +16863,7 @@ _loop1_22_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_22_type, _seq); D(p->level--); @@ -17025,7 +17040,7 @@ _loop0_26_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_26[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' NAME")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -17033,7 +17048,7 @@ _loop0_26_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_26_type, _seq); D(p->level--); @@ -17139,7 +17154,7 @@ _loop0_28_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_28[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' NAME")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -17147,7 +17162,7 @@ _loop0_28_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_28_type, _seq); D(p->level--); @@ -17343,7 +17358,7 @@ _loop0_31_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_31[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "('.' | '...')")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -17351,7 +17366,7 @@ _loop0_31_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_31_type, _seq); D(p->level--); @@ -17414,7 +17429,7 @@ _loop1_32_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -17422,7 +17437,7 @@ _loop1_32_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_32_type, _seq); D(p->level--); @@ -17489,7 +17504,7 @@ _loop0_34_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_34[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' import_from_as_name")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -17497,7 +17512,7 @@ _loop0_34_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_34_type, _seq); D(p->level--); @@ -17647,7 +17662,7 @@ _loop0_37_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_37[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' dotted_as_name")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -17655,7 +17670,7 @@ _loop0_37_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_37_type, _seq); D(p->level--); @@ -17805,7 +17820,7 @@ _loop0_40_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_40[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' with_item")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -17813,7 +17828,7 @@ _loop0_40_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_40_type, _seq); D(p->level--); @@ -17919,7 +17934,7 @@ _loop0_42_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_42[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' with_item")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -17927,7 +17942,7 @@ _loop0_42_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_42_type, _seq); D(p->level--); @@ -18033,7 +18048,7 @@ _loop0_44_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_44[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' with_item")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -18041,7 +18056,7 @@ _loop0_44_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_44_type, _seq); D(p->level--); @@ -18147,7 +18162,7 @@ _loop0_46_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_46[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' with_item")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -18155,7 +18170,7 @@ _loop0_46_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_46_type, _seq); D(p->level--); @@ -18331,7 +18346,7 @@ _loop1_48_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -18339,7 +18354,7 @@ _loop1_48_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_48_type, _seq); D(p->level--); @@ -18612,7 +18627,7 @@ _loop0_54_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_54[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -18620,7 +18635,7 @@ _loop0_54_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_54_type, _seq); D(p->level--); @@ -18678,7 +18693,7 @@ _loop0_55_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_55[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_with_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -18686,7 +18701,7 @@ _loop0_55_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_55_type, _seq); D(p->level--); @@ -18744,7 +18759,7 @@ _loop0_56_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_56[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_with_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -18752,7 +18767,7 @@ _loop0_56_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_56_type, _seq); D(p->level--); @@ -18815,7 +18830,7 @@ _loop1_57_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -18823,7 +18838,7 @@ _loop1_57_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_57_type, _seq); D(p->level--); @@ -18881,7 +18896,7 @@ _loop0_58_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_58[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_with_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -18889,7 +18904,7 @@ _loop0_58_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_58_type, _seq); D(p->level--); @@ -18952,7 +18967,7 @@ _loop1_59_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -18960,7 +18975,7 @@ _loop1_59_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_59_type, _seq); D(p->level--); @@ -19023,7 +19038,7 @@ _loop1_60_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19031,7 +19046,7 @@ _loop1_60_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_60_type, _seq); D(p->level--); @@ -19094,7 +19109,7 @@ _loop1_61_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19102,7 +19117,7 @@ _loop1_61_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_61_type, _seq); D(p->level--); @@ -19160,7 +19175,7 @@ _loop0_62_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_62[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19168,7 +19183,7 @@ _loop0_62_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_62_type, _seq); D(p->level--); @@ -19231,7 +19246,7 @@ _loop1_63_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19239,7 +19254,7 @@ _loop1_63_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_63_type, _seq); D(p->level--); @@ -19297,7 +19312,7 @@ _loop0_64_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_64[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19305,7 +19320,7 @@ _loop0_64_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_64_type, _seq); D(p->level--); @@ -19368,7 +19383,7 @@ _loop1_65_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19376,7 +19391,7 @@ _loop1_65_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_65_type, _seq); D(p->level--); @@ -19434,7 +19449,7 @@ _loop0_66_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_66[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_maybe_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19442,7 +19457,7 @@ _loop0_66_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_66_type, _seq); D(p->level--); @@ -19505,7 +19520,7 @@ _loop1_67_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19513,7 +19528,7 @@ _loop1_67_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_67_type, _seq); D(p->level--); @@ -19576,7 +19591,7 @@ _loop1_68_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19584,7 +19599,7 @@ _loop1_68_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_68_type, _seq); D(p->level--); @@ -19698,7 +19713,7 @@ _loop0_71_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_71[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_expression")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19706,7 +19721,7 @@ _loop0_71_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_71_type, _seq); D(p->level--); @@ -19808,7 +19823,7 @@ _loop1_72_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19816,7 +19831,7 @@ _loop1_72_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_72_type, _seq); D(p->level--); @@ -19883,7 +19898,7 @@ _loop0_74_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_74[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_named_expression")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -19891,7 +19906,7 @@ _loop0_74_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_74_type, _seq); D(p->level--); @@ -19993,7 +20008,7 @@ _loop1_75_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20001,7 +20016,7 @@ _loop1_75_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_75_type, _seq); D(p->level--); @@ -20059,7 +20074,7 @@ _loop0_76_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_76[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_no_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20067,7 +20082,7 @@ _loop0_76_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_76_type, _seq); D(p->level--); @@ -20125,7 +20140,7 @@ _loop0_77_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_77[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_with_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20133,7 +20148,7 @@ _loop0_77_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_77_type, _seq); D(p->level--); @@ -20191,7 +20206,7 @@ _loop0_78_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_78[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_with_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20199,7 +20214,7 @@ _loop0_78_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_78_type, _seq); D(p->level--); @@ -20262,7 +20277,7 @@ _loop1_79_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20270,7 +20285,7 @@ _loop1_79_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_79_type, _seq); D(p->level--); @@ -20328,7 +20343,7 @@ _loop0_80_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_80[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_with_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20336,7 +20351,7 @@ _loop0_80_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_80_type, _seq); D(p->level--); @@ -20399,7 +20414,7 @@ _loop1_81_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20407,7 +20422,7 @@ _loop1_81_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_81_type, _seq); D(p->level--); @@ -20470,7 +20485,7 @@ _loop1_82_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20478,7 +20493,7 @@ _loop1_82_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_82_type, _seq); D(p->level--); @@ -20541,7 +20556,7 @@ _loop1_83_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20549,7 +20564,7 @@ _loop1_83_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_83_type, _seq); D(p->level--); @@ -20607,7 +20622,7 @@ _loop0_84_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_84[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_no_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20615,7 +20630,7 @@ _loop0_84_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_84_type, _seq); D(p->level--); @@ -20678,7 +20693,7 @@ _loop1_85_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20686,7 +20701,7 @@ _loop1_85_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_85_type, _seq); D(p->level--); @@ -20744,7 +20759,7 @@ _loop0_86_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_86[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_no_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20752,7 +20767,7 @@ _loop0_86_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_86_type, _seq); D(p->level--); @@ -20815,7 +20830,7 @@ _loop1_87_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20823,7 +20838,7 @@ _loop1_87_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_87_type, _seq); D(p->level--); @@ -20881,7 +20896,7 @@ _loop0_88_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_88[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_maybe_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20889,7 +20904,7 @@ _loop0_88_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_88_type, _seq); D(p->level--); @@ -20952,7 +20967,7 @@ _loop1_89_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -20960,7 +20975,7 @@ _loop1_89_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_89_type, _seq); D(p->level--); @@ -21023,7 +21038,7 @@ _loop1_90_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -21031,7 +21046,7 @@ _loop1_90_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_90_type, _seq); D(p->level--); @@ -21094,7 +21109,7 @@ _loop1_91_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -21102,7 +21117,7 @@ _loop1_91_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_91_type, _seq); D(p->level--); @@ -21165,7 +21180,7 @@ _loop1_92_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -21173,7 +21188,7 @@ _loop1_92_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_92_type, _seq); D(p->level--); @@ -21281,7 +21296,7 @@ _loop0_95_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_95[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' slice")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -21289,7 +21304,7 @@ _loop0_95_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_95_type, _seq); D(p->level--); @@ -21657,7 +21672,7 @@ _loop1_100_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -21665,7 +21680,7 @@ _loop1_100_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_100_type, _seq); D(p->level--); @@ -21834,7 +21849,7 @@ _loop0_104_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_104[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' double_starred_kvpair")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -21842,7 +21857,7 @@ _loop0_104_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_104_type, _seq); D(p->level--); @@ -21944,7 +21959,7 @@ _loop1_105_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -21952,7 +21967,7 @@ _loop1_105_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_105_type, _seq); D(p->level--); @@ -22010,7 +22025,7 @@ _loop0_106_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_106[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "('if' disjunction)")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -22018,7 +22033,7 @@ _loop0_106_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_106_type, _seq); D(p->level--); @@ -22076,7 +22091,7 @@ _loop0_107_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_107[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "('if' disjunction)")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -22084,7 +22099,7 @@ _loop0_107_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_107_type, _seq); D(p->level--); @@ -22151,7 +22166,7 @@ _loop0_109_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_109[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' (starred_expression | named_expression !'=')")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -22159,7 +22174,7 @@ _loop0_109_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_109_type, _seq); D(p->level--); @@ -22309,7 +22324,7 @@ _loop0_112_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_112[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' kwarg_or_starred")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -22317,7 +22332,7 @@ _loop0_112_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_112_type, _seq); D(p->level--); @@ -22423,7 +22438,7 @@ _loop0_114_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_114[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' kwarg_or_double_starred")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -22431,7 +22446,7 @@ _loop0_114_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_114_type, _seq); D(p->level--); @@ -22537,7 +22552,7 @@ _loop0_116_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_116[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' kwarg_or_starred")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -22545,7 +22560,7 @@ _loop0_116_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_116_type, _seq); D(p->level--); @@ -22651,7 +22666,7 @@ _loop0_118_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_118[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' kwarg_or_double_starred")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -22659,7 +22674,7 @@ _loop0_118_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_118_type, _seq); D(p->level--); @@ -22756,7 +22771,7 @@ _loop0_119_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_119[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(',' star_target)")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -22764,7 +22779,7 @@ _loop0_119_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_119_type, _seq); D(p->level--); @@ -22831,7 +22846,7 @@ _loop0_121_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_121[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' star_target")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -22839,7 +22854,7 @@ _loop0_121_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_121_type, _seq); D(p->level--); @@ -22983,7 +22998,7 @@ _loop0_124_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_124[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' del_target")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -22991,7 +23006,7 @@ _loop0_124_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_124_type, _seq); D(p->level--); @@ -23097,7 +23112,7 @@ _loop0_126_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_126[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "',' target")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -23105,7 +23120,7 @@ _loop0_126_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_126_type, _seq); D(p->level--); @@ -23188,7 +23203,7 @@ _tmp_127_rule(Parser *p) } D(fprintf(stderr, "%*c> _tmp_127[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "expression for_if_clauses")); expr_ty expression_var; - asdl_seq* for_if_clauses_var; + asdl_comprehension_seq* for_if_clauses_var; if ( (expression_var = expression_rule(p)) // expression && @@ -23236,7 +23251,7 @@ _loop0_128_rule(Parser *p) return NULL; } D(fprintf(stderr, "%*c> _loop0_128[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "star_named_expressions")); - asdl_seq* star_named_expressions_var; + asdl_expr_seq* star_named_expressions_var; while ( (star_named_expressions_var = star_named_expressions_rule(p)) // star_named_expressions ) @@ -23260,7 +23275,7 @@ _loop0_128_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_128[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "star_named_expressions")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -23268,7 +23283,7 @@ _loop0_128_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_128_type, _seq); D(p->level--); @@ -23326,7 +23341,7 @@ _loop0_129_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_129[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -23334,7 +23349,7 @@ _loop0_129_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_129_type, _seq); D(p->level--); @@ -23392,7 +23407,7 @@ _loop0_130_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_130[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "(star_targets '=')")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -23400,7 +23415,7 @@ _loop0_130_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_130_type, _seq); D(p->level--); @@ -23587,7 +23602,7 @@ _loop0_133_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_133[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "param_no_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -23595,7 +23610,7 @@ _loop0_133_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_133_type, _seq); D(p->level--); @@ -23708,7 +23723,7 @@ _loop0_135_rule(Parser *p) D(fprintf(stderr, "%*c%s _loop0_135[%d-%d]: %s failed!\n", p->level, ' ', p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "lambda_param_no_default")); } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -23716,7 +23731,7 @@ _loop0_135_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop0_135_type, _seq); D(p->level--); @@ -24594,7 +24609,7 @@ _loop1_153_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -24602,7 +24617,7 @@ _loop1_153_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_153_type, _seq); D(p->level--); @@ -24665,7 +24680,7 @@ _loop1_154_rule(Parser *p) D(p->level--); return NULL; } - asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena); + asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena); if (!_seq) { PyMem_Free(_children); p->error_indicator = 1; @@ -24673,7 +24688,7 @@ _loop1_154_rule(Parser *p) D(p->level--); return NULL; } - for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]); + for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]); PyMem_Free(_children); _PyPegen_insert_memo(p, _start_mark, _loop1_154_type, _seq); D(p->level--); diff --git a/Parser/pegen.c b/Parser/pegen.c index 4beb2abdd296a..1de495eaf398e 100644 --- a/Parser/pegen.c +++ b/Parser/pegen.c @@ -1243,7 +1243,7 @@ _PyPegen_run_parser_from_string(const char *str, int start_rule, PyObject *filen return result; } -void * +asdl_stmt_seq* _PyPegen_interactive_exit(Parser *p) { if (p->errcode) { @@ -1257,11 +1257,11 @@ asdl_seq * _PyPegen_singleton_seq(Parser *p, void *a) { assert(a != NULL); - asdl_seq *seq = _Py_asdl_seq_new(1, p->arena); + asdl_seq *seq = (asdl_seq*)_Py_asdl_generic_seq_new(1, p->arena); if (!seq) { return NULL; } - asdl_seq_SET(seq, 0, a); + asdl_seq_SET_UNTYPED(seq, 0, a); return seq; } @@ -1274,14 +1274,14 @@ _PyPegen_seq_insert_in_front(Parser *p, void *a, asdl_seq *seq) return _PyPegen_singleton_seq(p, a); } - asdl_seq *new_seq = _Py_asdl_seq_new(asdl_seq_LEN(seq) + 1, p->arena); + asdl_seq *new_seq = (asdl_seq*)_Py_asdl_generic_seq_new(asdl_seq_LEN(seq) + 1, p->arena); if (!new_seq) { return NULL; } - asdl_seq_SET(new_seq, 0, a); + asdl_seq_SET_UNTYPED(new_seq, 0, a); for (Py_ssize_t i = 1, l = asdl_seq_LEN(new_seq); i < l; i++) { - asdl_seq_SET(new_seq, i, asdl_seq_GET(seq, i - 1)); + asdl_seq_SET_UNTYPED(new_seq, i, asdl_seq_GET_UNTYPED(seq, i - 1)); } return new_seq; } @@ -1295,15 +1295,15 @@ _PyPegen_seq_append_to_end(Parser *p, asdl_seq *seq, void *a) return _PyPegen_singleton_seq(p, a); } - asdl_seq *new_seq = _Py_asdl_seq_new(asdl_seq_LEN(seq) + 1, p->arena); + asdl_seq *new_seq = (asdl_seq*)_Py_asdl_generic_seq_new(asdl_seq_LEN(seq) + 1, p->arena); if (!new_seq) { return NULL; } for (Py_ssize_t i = 0, l = asdl_seq_LEN(new_seq); i + 1 < l; i++) { - asdl_seq_SET(new_seq, i, asdl_seq_GET(seq, i)); + asdl_seq_SET_UNTYPED(new_seq, i, asdl_seq_GET_UNTYPED(seq, i)); } - asdl_seq_SET(new_seq, asdl_seq_LEN(new_seq) - 1, a); + asdl_seq_SET_UNTYPED(new_seq, asdl_seq_LEN(new_seq) - 1, a); return new_seq; } @@ -1312,7 +1312,7 @@ _get_flattened_seq_size(asdl_seq *seqs) { Py_ssize_t size = 0; for (Py_ssize_t i = 0, l = asdl_seq_LEN(seqs); i < l; i++) { - asdl_seq *inner_seq = asdl_seq_GET(seqs, i); + asdl_seq *inner_seq = asdl_seq_GET_UNTYPED(seqs, i); size += asdl_seq_LEN(inner_seq); } return size; @@ -1325,16 +1325,16 @@ _PyPegen_seq_flatten(Parser *p, asdl_seq *seqs) Py_ssize_t flattened_seq_size = _get_flattened_seq_size(seqs); assert(flattened_seq_size > 0); - asdl_seq *flattened_seq = _Py_asdl_seq_new(flattened_seq_size, p->arena); + asdl_seq *flattened_seq = (asdl_seq*)_Py_asdl_generic_seq_new(flattened_seq_size, p->arena); if (!flattened_seq) { return NULL; } int flattened_seq_idx = 0; for (Py_ssize_t i = 0, l = asdl_seq_LEN(seqs); i < l; i++) { - asdl_seq *inner_seq = asdl_seq_GET(seqs, i); + asdl_seq *inner_seq = asdl_seq_GET_UNTYPED(seqs, i); for (Py_ssize_t j = 0, li = asdl_seq_LEN(inner_seq); j < li; j++) { - asdl_seq_SET(flattened_seq, flattened_seq_idx++, asdl_seq_GET(inner_seq, j)); + asdl_seq_SET_UNTYPED(flattened_seq, flattened_seq_idx++, asdl_seq_GET_UNTYPED(inner_seq, j)); } } assert(flattened_seq_idx == flattened_seq_size); @@ -1403,7 +1403,7 @@ _PyPegen_seq_count_dots(asdl_seq *seq) { int number_of_dots = 0; for (Py_ssize_t i = 0, l = asdl_seq_LEN(seq); i < l; i++) { - Token *current_expr = asdl_seq_GET(seq, i); + Token *current_expr = asdl_seq_GET_UNTYPED(seq, i); switch (current_expr->type) { case ELLIPSIS: number_of_dots += 3; @@ -1435,13 +1435,13 @@ _PyPegen_alias_for_star(Parser *p) } /* Creates a new asdl_seq* with the identifiers of all the names in seq */ -asdl_seq * -_PyPegen_map_names_to_ids(Parser *p, asdl_seq *seq) +asdl_identifier_seq * +_PyPegen_map_names_to_ids(Parser *p, asdl_expr_seq *seq) { Py_ssize_t len = asdl_seq_LEN(seq); assert(len > 0); - asdl_seq *new_seq = _Py_asdl_seq_new(len, p->arena); + asdl_identifier_seq *new_seq = _Py_asdl_identifier_seq_new(len, p->arena); if (!new_seq) { return NULL; } @@ -1477,39 +1477,39 @@ _PyPegen_get_cmpops(Parser *p, asdl_seq *seq) return NULL; } for (Py_ssize_t i = 0; i < len; i++) { - CmpopExprPair *pair = asdl_seq_GET(seq, i); + CmpopExprPair *pair = asdl_seq_GET_UNTYPED(seq, i); asdl_seq_SET(new_seq, i, pair->cmpop); } return new_seq; } -asdl_seq * +asdl_expr_seq * _PyPegen_get_exprs(Parser *p, asdl_seq *seq) { Py_ssize_t len = asdl_seq_LEN(seq); assert(len > 0); - asdl_seq *new_seq = _Py_asdl_seq_new(len, p->arena); + asdl_expr_seq *new_seq = _Py_asdl_expr_seq_new(len, p->arena); if (!new_seq) { return NULL; } for (Py_ssize_t i = 0; i < len; i++) { - CmpopExprPair *pair = asdl_seq_GET(seq, i); + CmpopExprPair *pair = asdl_seq_GET_UNTYPED(seq, i); asdl_seq_SET(new_seq, i, pair->expr); } return new_seq; } /* Creates an asdl_seq* where all the elements have been changed to have ctx as context */ -static asdl_seq * -_set_seq_context(Parser *p, asdl_seq *seq, expr_context_ty ctx) +static asdl_expr_seq * +_set_seq_context(Parser *p, asdl_expr_seq *seq, expr_context_ty ctx) { Py_ssize_t len = asdl_seq_LEN(seq); if (len == 0) { return NULL; } - asdl_seq *new_seq = _Py_asdl_seq_new(len, p->arena); + asdl_expr_seq *new_seq = _Py_asdl_expr_seq_new(len, p->arena); if (!new_seq) { return NULL; } @@ -1529,13 +1529,19 @@ _set_name_context(Parser *p, expr_ty e, expr_context_ty ctx) static expr_ty _set_tuple_context(Parser *p, expr_ty e, expr_context_ty ctx) { - return _Py_Tuple(_set_seq_context(p, e->v.Tuple.elts, ctx), ctx, EXTRA_EXPR(e, e)); + return _Py_Tuple( + _set_seq_context(p, e->v.Tuple.elts, ctx), + ctx, + EXTRA_EXPR(e, e)); } static expr_ty _set_list_context(Parser *p, expr_ty e, expr_context_ty ctx) { - return _Py_List(_set_seq_context(p, e->v.List.elts, ctx), ctx, EXTRA_EXPR(e, e)); + return _Py_List( + _set_seq_context(p, e->v.List.elts, ctx), + ctx, + EXTRA_EXPR(e, e)); } static expr_ty @@ -1602,32 +1608,32 @@ _PyPegen_key_value_pair(Parser *p, expr_ty key, expr_ty value) } /* Extracts all keys from an asdl_seq* of KeyValuePair*'s */ -asdl_seq * +asdl_expr_seq * _PyPegen_get_keys(Parser *p, asdl_seq *seq) { Py_ssize_t len = asdl_seq_LEN(seq); - asdl_seq *new_seq = _Py_asdl_seq_new(len, p->arena); + asdl_expr_seq *new_seq = _Py_asdl_expr_seq_new(len, p->arena); if (!new_seq) { return NULL; } for (Py_ssize_t i = 0; i < len; i++) { - KeyValuePair *pair = asdl_seq_GET(seq, i); + KeyValuePair *pair = asdl_seq_GET_UNTYPED(seq, i); asdl_seq_SET(new_seq, i, pair->key); } return new_seq; } /* Extracts all values from an asdl_seq* of KeyValuePair*'s */ -asdl_seq * +asdl_expr_seq * _PyPegen_get_values(Parser *p, asdl_seq *seq) { Py_ssize_t len = asdl_seq_LEN(seq); - asdl_seq *new_seq = _Py_asdl_seq_new(len, p->arena); + asdl_expr_seq *new_seq = _Py_asdl_expr_seq_new(len, p->arena); if (!new_seq) { return NULL; } for (Py_ssize_t i = 0; i < len; i++) { - KeyValuePair *pair = asdl_seq_GET(seq, i); + KeyValuePair *pair = asdl_seq_GET_UNTYPED(seq, i); asdl_seq_SET(new_seq, i, pair->value); } return new_seq; @@ -1648,7 +1654,7 @@ _PyPegen_name_default_pair(Parser *p, arg_ty arg, expr_ty value, Token *tc) /* Constructs a SlashWithDefault */ SlashWithDefault * -_PyPegen_slash_with_default(Parser *p, asdl_seq *plain_names, asdl_seq *names_with_defaults) +_PyPegen_slash_with_default(Parser *p, asdl_arg_seq *plain_names, asdl_seq *names_with_defaults) { SlashWithDefault *a = PyArena_Malloc(p->arena, sizeof(SlashWithDefault)); if (!a) { @@ -1678,47 +1684,47 @@ _PyPegen_join_sequences(Parser *p, asdl_seq *a, asdl_seq *b) { Py_ssize_t first_len = asdl_seq_LEN(a); Py_ssize_t second_len = asdl_seq_LEN(b); - asdl_seq *new_seq = _Py_asdl_seq_new(first_len + second_len, p->arena); + asdl_seq *new_seq = (asdl_seq*)_Py_asdl_generic_seq_new(first_len + second_len, p->arena); if (!new_seq) { return NULL; } int k = 0; for (Py_ssize_t i = 0; i < first_len; i++) { - asdl_seq_SET(new_seq, k++, asdl_seq_GET(a, i)); + asdl_seq_SET_UNTYPED(new_seq, k++, asdl_seq_GET_UNTYPED(a, i)); } for (Py_ssize_t i = 0; i < second_len; i++) { - asdl_seq_SET(new_seq, k++, asdl_seq_GET(b, i)); + asdl_seq_SET_UNTYPED(new_seq, k++, asdl_seq_GET_UNTYPED(b, i)); } return new_seq; } -static asdl_seq * +static asdl_arg_seq* _get_names(Parser *p, asdl_seq *names_with_defaults) { Py_ssize_t len = asdl_seq_LEN(names_with_defaults); - asdl_seq *seq = _Py_asdl_seq_new(len, p->arena); + asdl_arg_seq *seq = _Py_asdl_arg_seq_new(len, p->arena); if (!seq) { return NULL; } for (Py_ssize_t i = 0; i < len; i++) { - NameDefaultPair *pair = asdl_seq_GET(names_with_defaults, i); + NameDefaultPair *pair = asdl_seq_GET_UNTYPED(names_with_defaults, i); asdl_seq_SET(seq, i, pair->arg); } return seq; } -static asdl_seq * +static asdl_expr_seq * _get_defaults(Parser *p, asdl_seq *names_with_defaults) { Py_ssize_t len = asdl_seq_LEN(names_with_defaults); - asdl_seq *seq = _Py_asdl_seq_new(len, p->arena); + asdl_expr_seq *seq = _Py_asdl_expr_seq_new(len, p->arena); if (!seq) { return NULL; } for (Py_ssize_t i = 0; i < len; i++) { - NameDefaultPair *pair = asdl_seq_GET(names_with_defaults, i); + NameDefaultPair *pair = asdl_seq_GET_UNTYPED(names_with_defaults, i); asdl_seq_SET(seq, i, pair->value); } return seq; @@ -1726,39 +1732,45 @@ _get_defaults(Parser *p, asdl_seq *names_with_defaults) /* Constructs an arguments_ty object out of all the parsed constructs in the parameters rule */ arguments_ty -_PyPegen_make_arguments(Parser *p, asdl_seq *slash_without_default, - SlashWithDefault *slash_with_default, asdl_seq *plain_names, +_PyPegen_make_arguments(Parser *p, asdl_arg_seq *slash_without_default, + SlashWithDefault *slash_with_default, asdl_arg_seq *plain_names, asdl_seq *names_with_default, StarEtc *star_etc) { - asdl_seq *posonlyargs; + asdl_arg_seq *posonlyargs; if (slash_without_default != NULL) { posonlyargs = slash_without_default; } else if (slash_with_default != NULL) { - asdl_seq *slash_with_default_names = + asdl_arg_seq *slash_with_default_names = _get_names(p, slash_with_default->names_with_defaults); if (!slash_with_default_names) { return NULL; } - posonlyargs = _PyPegen_join_sequences(p, slash_with_default->plain_names, slash_with_default_names); + posonlyargs = (asdl_arg_seq*)_PyPegen_join_sequences( + p, + (asdl_seq*)slash_with_default->plain_names, + (asdl_seq*)slash_with_default_names); if (!posonlyargs) { return NULL; } } else { - posonlyargs = _Py_asdl_seq_new(0, p->arena); + posonlyargs = _Py_asdl_arg_seq_new(0, p->arena); if (!posonlyargs) { return NULL; } } - asdl_seq *posargs; + asdl_arg_seq *posargs; if (plain_names != NULL && names_with_default != NULL) { - asdl_seq *names_with_default_names = _get_names(p, names_with_default); + asdl_arg_seq *names_with_default_names = _get_names(p, names_with_default); if (!names_with_default_names) { return NULL; } - posargs = _PyPegen_join_sequences(p, plain_names, names_with_default_names); + posargs = (asdl_arg_seq*)_PyPegen_join_sequences( + p, + (asdl_seq*)plain_names, + (asdl_seq*)names_with_default_names); if (!posargs) { return NULL; } @@ -1773,24 +1785,27 @@ _PyPegen_make_arguments(Parser *p, asdl_seq *slash_without_default, posargs = plain_names; } else { - posargs = _Py_asdl_seq_new(0, p->arena); + posargs = _Py_asdl_arg_seq_new(0, p->arena); if (!posargs) { return NULL; } } - asdl_seq *posdefaults; + asdl_expr_seq *posdefaults; if (slash_with_default != NULL && names_with_default != NULL) { - asdl_seq *slash_with_default_values = + asdl_expr_seq *slash_with_default_values = _get_defaults(p, slash_with_default->names_with_defaults); if (!slash_with_default_values) { return NULL; } - asdl_seq *names_with_default_values = _get_defaults(p, names_with_default); + asdl_expr_seq *names_with_default_values = _get_defaults(p, names_with_default); if (!names_with_default_values) { return NULL; } - posdefaults = _PyPegen_join_sequences(p, slash_with_default_values, names_with_default_values); + posdefaults = (asdl_expr_seq*)_PyPegen_join_sequences( + p, + (asdl_seq*)slash_with_default_values, + (asdl_seq*)names_with_default_values); if (!posdefaults) { return NULL; } @@ -1808,7 +1823,7 @@ _PyPegen_make_arguments(Parser *p, asdl_seq *slash_without_default, } } else { - posdefaults = _Py_asdl_seq_new(0, p->arena); + posdefaults = _Py_asdl_expr_seq_new(0, p->arena); if (!posdefaults) { return NULL; } @@ -1819,7 +1834,7 @@ _PyPegen_make_arguments(Parser *p, asdl_seq *slash_without_default, vararg = star_etc->vararg; } - asdl_seq *kwonlyargs; + asdl_arg_seq *kwonlyargs; if (star_etc != NULL && star_etc->kwonlyargs != NULL) { kwonlyargs = _get_names(p, star_etc->kwonlyargs); if (!kwonlyargs) { @@ -1827,13 +1842,13 @@ _PyPegen_make_arguments(Parser *p, asdl_seq *slash_without_default, } } else { - kwonlyargs = _Py_asdl_seq_new(0, p->arena); + kwonlyargs = _Py_asdl_arg_seq_new(0, p->arena); if (!kwonlyargs) { return NULL; } } - asdl_seq *kwdefaults; + asdl_expr_seq *kwdefaults; if (star_etc != NULL && star_etc->kwonlyargs != NULL) { kwdefaults = _get_defaults(p, star_etc->kwonlyargs); if (!kwdefaults) { @@ -1841,7 +1856,7 @@ _PyPegen_make_arguments(Parser *p, asdl_seq *slash_without_default, } } else { - kwdefaults = _Py_asdl_seq_new(0, p->arena); + kwdefaults = _Py_asdl_expr_seq_new(0, p->arena); if (!kwdefaults) { return NULL; } @@ -1861,23 +1876,23 @@ _PyPegen_make_arguments(Parser *p, asdl_seq *slash_without_default, arguments_ty _PyPegen_empty_arguments(Parser *p) { - asdl_seq *posonlyargs = _Py_asdl_seq_new(0, p->arena); + asdl_arg_seq *posonlyargs = _Py_asdl_arg_seq_new(0, p->arena); if (!posonlyargs) { return NULL; } - asdl_seq *posargs = _Py_asdl_seq_new(0, p->arena); + asdl_arg_seq *posargs = _Py_asdl_arg_seq_new(0, p->arena); if (!posargs) { return NULL; } - asdl_seq *posdefaults = _Py_asdl_seq_new(0, p->arena); + asdl_expr_seq *posdefaults = _Py_asdl_expr_seq_new(0, p->arena); if (!posdefaults) { return NULL; } - asdl_seq *kwonlyargs = _Py_asdl_seq_new(0, p->arena); + asdl_arg_seq *kwonlyargs = _Py_asdl_arg_seq_new(0, p->arena); if (!kwonlyargs) { return NULL; } - asdl_seq *kwdefaults = _Py_asdl_seq_new(0, p->arena); + asdl_expr_seq *kwdefaults = _Py_asdl_expr_seq_new(0, p->arena); if (!kwdefaults) { return NULL; } @@ -1900,7 +1915,7 @@ _PyPegen_augoperator(Parser *p, operator_ty kind) /* Construct a FunctionDef equivalent to function_def, but with decorators */ stmt_ty -_PyPegen_function_def_decorators(Parser *p, asdl_seq *decorators, stmt_ty function_def) +_PyPegen_function_def_decorators(Parser *p, asdl_expr_seq *decorators, stmt_ty function_def) { assert(function_def != NULL); if (function_def->kind == AsyncFunctionDef_kind) { @@ -1922,7 +1937,7 @@ _PyPegen_function_def_decorators(Parser *p, asdl_seq *decorators, stmt_ty functi /* Construct a ClassDef equivalent to class_def, but with decorators */ stmt_ty -_PyPegen_class_def_decorators(Parser *p, asdl_seq *decorators, stmt_ty class_def) +_PyPegen_class_def_decorators(Parser *p, asdl_expr_seq *decorators, stmt_ty class_def) { assert(class_def != NULL); return _Py_ClassDef(class_def->v.ClassDef.name, class_def->v.ClassDef.bases, @@ -1950,7 +1965,7 @@ _seq_number_of_starred_exprs(asdl_seq *seq) { int n = 0; for (Py_ssize_t i = 0, l = asdl_seq_LEN(seq); i < l; i++) { - KeywordOrStarred *k = asdl_seq_GET(seq, i); + KeywordOrStarred *k = asdl_seq_GET_UNTYPED(seq, i); if (!k->is_keyword) { n++; } @@ -1959,21 +1974,21 @@ _seq_number_of_starred_exprs(asdl_seq *seq) } /* Extract the starred expressions of an asdl_seq* of KeywordOrStarred*s */ -asdl_seq * +asdl_expr_seq * _PyPegen_seq_extract_starred_exprs(Parser *p, asdl_seq *kwargs) { int new_len = _seq_number_of_starred_exprs(kwargs); if (new_len == 0) { return NULL; } - asdl_seq *new_seq = _Py_asdl_seq_new(new_len, p->arena); + asdl_expr_seq *new_seq = _Py_asdl_expr_seq_new(new_len, p->arena); if (!new_seq) { return NULL; } int idx = 0; for (Py_ssize_t i = 0, len = asdl_seq_LEN(kwargs); i < len; i++) { - KeywordOrStarred *k = asdl_seq_GET(kwargs, i); + KeywordOrStarred *k = asdl_seq_GET_UNTYPED(kwargs, i); if (!k->is_keyword) { asdl_seq_SET(new_seq, idx++, k->element); } @@ -1982,7 +1997,7 @@ _PyPegen_seq_extract_starred_exprs(Parser *p, asdl_seq *kwargs) } /* Return a new asdl_seq* with only the keywords in kwargs */ -asdl_seq * +asdl_keyword_seq* _PyPegen_seq_delete_starred_exprs(Parser *p, asdl_seq *kwargs) { Py_ssize_t len = asdl_seq_LEN(kwargs); @@ -1990,14 +2005,14 @@ _PyPegen_seq_delete_starred_exprs(Parser *p, asdl_seq *kwargs) if (new_len == 0) { return NULL; } - asdl_seq *new_seq = _Py_asdl_seq_new(new_len, p->arena); + asdl_keyword_seq *new_seq = _Py_asdl_keyword_seq_new(new_len, p->arena); if (!new_seq) { return NULL; } int idx = 0; for (Py_ssize_t i = 0; i < len; i++) { - KeywordOrStarred *k = asdl_seq_GET(kwargs, i); + KeywordOrStarred *k = asdl_seq_GET_UNTYPED(kwargs, i); if (k->is_keyword) { asdl_seq_SET(new_seq, idx++, k->element); } @@ -2011,8 +2026,8 @@ _PyPegen_concatenate_strings(Parser *p, asdl_seq *strings) Py_ssize_t len = asdl_seq_LEN(strings); assert(len > 0); - Token *first = asdl_seq_GET(strings, 0); - Token *last = asdl_seq_GET(strings, len - 1); + Token *first = asdl_seq_GET_UNTYPED(strings, 0); + Token *last = asdl_seq_GET_UNTYPED(strings, len - 1); int bytesmode = 0; PyObject *bytes_str = NULL; @@ -2021,7 +2036,7 @@ _PyPegen_concatenate_strings(Parser *p, asdl_seq *strings) _PyPegen_FstringParser_Init(&state); for (Py_ssize_t i = 0; i < len; i++) { - Token *t = asdl_seq_GET(strings, i); + Token *t = asdl_seq_GET_UNTYPED(strings, i); int this_bytesmode; int this_rawmode; @@ -2095,12 +2110,12 @@ _PyPegen_concatenate_strings(Parser *p, asdl_seq *strings) } mod_ty -_PyPegen_make_module(Parser *p, asdl_seq *a) { - asdl_seq *type_ignores = NULL; +_PyPegen_make_module(Parser *p, asdl_stmt_seq *a) { + asdl_type_ignore_seq *type_ignores = NULL; Py_ssize_t num = p->type_ignore_comments.num_items; if (num > 0) { // Turn the raw (comment, lineno) pairs into TypeIgnore objects in the arena - type_ignores = _Py_asdl_seq_new(num, p->arena); + type_ignores = _Py_asdl_type_ignore_seq_new(num, p->arena); if (type_ignores == NULL) { return NULL; } @@ -2219,7 +2234,7 @@ _PyPegen_nonparen_genexp_in_call(Parser *p, expr_ty args) } -expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_seq *a, asdl_seq *b, +expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_expr_seq *a, asdl_seq *b, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { Py_ssize_t args_len = asdl_seq_LEN(a); @@ -2231,14 +2246,14 @@ expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_seq *a, asdl_seq *b, } - asdl_seq *starreds = _PyPegen_seq_extract_starred_exprs(p, b); - asdl_seq *keywords = _PyPegen_seq_delete_starred_exprs(p, b); + asdl_expr_seq *starreds = _PyPegen_seq_extract_starred_exprs(p, b); + asdl_keyword_seq *keywords = _PyPegen_seq_delete_starred_exprs(p, b); if (starreds) { total_len += asdl_seq_LEN(starreds); } - asdl_seq *args = _Py_asdl_seq_new(total_len, arena); + asdl_expr_seq *args = _Py_asdl_expr_seq_new(total_len, arena); Py_ssize_t i = 0; for (i = 0; i < args_len; i++) { @@ -2250,6 +2265,4 @@ expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_seq *a, asdl_seq *b, return _Py_Call(_PyPegen_dummy_name(p), args, keywords, lineno, col_offset, end_lineno, end_col_offset, arena); - - } diff --git a/Parser/pegen.h b/Parser/pegen.h index c81681efad208..000dc8c462b85 100644 --- a/Parser/pegen.h +++ b/Parser/pegen.h @@ -91,7 +91,7 @@ typedef struct { } NameDefaultPair; typedef struct { - asdl_seq *plain_names; + asdl_arg_seq *plain_names; asdl_seq *names_with_defaults; // asdl_seq* of NameDefaultsPair's } SlashWithDefault; @@ -229,7 +229,7 @@ mod_ty _PyPegen_run_parser_from_file_pointer(FILE *, int, PyObject *, const char void *_PyPegen_run_parser(Parser *); mod_ty _PyPegen_run_parser_from_file(const char *, int, PyObject *, PyCompilerFlags *, PyArena *); mod_ty _PyPegen_run_parser_from_string(const char *, int, PyObject *, PyCompilerFlags *, PyArena *); -void *_PyPegen_interactive_exit(Parser *); +asdl_stmt_seq *_PyPegen_interactive_exit(Parser *); asdl_seq *_PyPegen_singleton_seq(Parser *, void *); asdl_seq *_PyPegen_seq_insert_in_front(Parser *, void *, asdl_seq *); asdl_seq *_PyPegen_seq_append_to_end(Parser *, asdl_seq *, void *); @@ -237,33 +237,33 @@ asdl_seq *_PyPegen_seq_flatten(Parser *, asdl_seq *); expr_ty _PyPegen_join_names_with_dot(Parser *, expr_ty, expr_ty); int _PyPegen_seq_count_dots(asdl_seq *); alias_ty _PyPegen_alias_for_star(Parser *); -asdl_seq *_PyPegen_map_names_to_ids(Parser *, asdl_seq *); +asdl_identifier_seq *_PyPegen_map_names_to_ids(Parser *, asdl_expr_seq *); CmpopExprPair *_PyPegen_cmpop_expr_pair(Parser *, cmpop_ty, expr_ty); asdl_int_seq *_PyPegen_get_cmpops(Parser *p, asdl_seq *); -asdl_seq *_PyPegen_get_exprs(Parser *, asdl_seq *); +asdl_expr_seq *_PyPegen_get_exprs(Parser *, asdl_seq *); expr_ty _PyPegen_set_expr_context(Parser *, expr_ty, expr_context_ty); KeyValuePair *_PyPegen_key_value_pair(Parser *, expr_ty, expr_ty); -asdl_seq *_PyPegen_get_keys(Parser *, asdl_seq *); -asdl_seq *_PyPegen_get_values(Parser *, asdl_seq *); +asdl_expr_seq *_PyPegen_get_keys(Parser *, asdl_seq *); +asdl_expr_seq *_PyPegen_get_values(Parser *, asdl_seq *); NameDefaultPair *_PyPegen_name_default_pair(Parser *, arg_ty, expr_ty, Token *); -SlashWithDefault *_PyPegen_slash_with_default(Parser *, asdl_seq *, asdl_seq *); +SlashWithDefault *_PyPegen_slash_with_default(Parser *, asdl_arg_seq *, asdl_seq *); StarEtc *_PyPegen_star_etc(Parser *, arg_ty, asdl_seq *, arg_ty); -arguments_ty _PyPegen_make_arguments(Parser *, asdl_seq *, SlashWithDefault *, - asdl_seq *, asdl_seq *, StarEtc *); +arguments_ty _PyPegen_make_arguments(Parser *, asdl_arg_seq *, SlashWithDefault *, + asdl_arg_seq *, asdl_seq *, StarEtc *); arguments_ty _PyPegen_empty_arguments(Parser *); AugOperator *_PyPegen_augoperator(Parser*, operator_ty type); -stmt_ty _PyPegen_function_def_decorators(Parser *, asdl_seq *, stmt_ty); -stmt_ty _PyPegen_class_def_decorators(Parser *, asdl_seq *, stmt_ty); +stmt_ty _PyPegen_function_def_decorators(Parser *, asdl_expr_seq *, stmt_ty); +stmt_ty _PyPegen_class_def_decorators(Parser *, asdl_expr_seq *, stmt_ty); KeywordOrStarred *_PyPegen_keyword_or_starred(Parser *, void *, int); -asdl_seq *_PyPegen_seq_extract_starred_exprs(Parser *, asdl_seq *); -asdl_seq *_PyPegen_seq_delete_starred_exprs(Parser *, asdl_seq *); -expr_ty _PyPegen_collect_call_seqs(Parser *, asdl_seq *, asdl_seq *, +asdl_expr_seq *_PyPegen_seq_extract_starred_exprs(Parser *, asdl_seq *); +asdl_keyword_seq *_PyPegen_seq_delete_starred_exprs(Parser *, asdl_seq *); +expr_ty _PyPegen_collect_call_seqs(Parser *, asdl_expr_seq *, asdl_seq *, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena); expr_ty _PyPegen_concatenate_strings(Parser *p, asdl_seq *); asdl_seq *_PyPegen_join_sequences(Parser *, asdl_seq *, asdl_seq *); int _PyPegen_check_barry_as_flufl(Parser *); -mod_ty _PyPegen_make_module(Parser *, asdl_seq *); +mod_ty _PyPegen_make_module(Parser *, asdl_stmt_seq *); // Error reporting helpers typedef enum { diff --git a/Parser/string_parser.c b/Parser/string_parser.c index 2c35da590defb..1285968b31917 100644 --- a/Parser/string_parser.c +++ b/Parser/string_parser.c @@ -970,15 +970,15 @@ ExprList_Dealloc(ExprList *l) l->size = -1; } -static asdl_seq * +static asdl_expr_seq * ExprList_Finish(ExprList *l, PyArena *arena) { - asdl_seq *seq; + asdl_expr_seq *seq; ExprList_check_invariants(l); /* Allocate the asdl_seq and copy the expressions in to it. */ - seq = _Py_asdl_seq_new(l->size, arena); + seq = _Py_asdl_expr_seq_new(l->size, arena); if (seq) { Py_ssize_t i; for (i = 0; i < l->size; i++) { @@ -1167,7 +1167,7 @@ expr_ty _PyPegen_FstringParser_Finish(Parser *p, FstringParser *state, Token* first_token, Token *last_token) { - asdl_seq *seq; + asdl_expr_seq *seq; FstringParser_check_invariants(state); diff --git a/Python/Python-ast.c b/Python/Python-ast.c index 094010e6c9ddc..13657a6727566 100644 --- a/Python/Python-ast.c +++ b/Python/Python-ast.c @@ -549,6 +549,18 @@ static int init_identifiers(astmodulestate *state) return 1; }; +GENERATE_ASDL_SEQ_CONSTRUCTOR(mod, mod_ty) +GENERATE_ASDL_SEQ_CONSTRUCTOR(stmt, stmt_ty) +GENERATE_ASDL_SEQ_CONSTRUCTOR(expr, expr_ty) +GENERATE_ASDL_SEQ_CONSTRUCTOR(comprehension, comprehension_ty) +GENERATE_ASDL_SEQ_CONSTRUCTOR(excepthandler, excepthandler_ty) +GENERATE_ASDL_SEQ_CONSTRUCTOR(arguments, arguments_ty) +GENERATE_ASDL_SEQ_CONSTRUCTOR(arg, arg_ty) +GENERATE_ASDL_SEQ_CONSTRUCTOR(keyword, keyword_ty) +GENERATE_ASDL_SEQ_CONSTRUCTOR(alias, alias_ty) +GENERATE_ASDL_SEQ_CONSTRUCTOR(withitem, withitem_ty) +GENERATE_ASDL_SEQ_CONSTRUCTOR(type_ignore, type_ignore_ty) + static PyObject* ast2obj_mod(astmodulestate *state, void*); static const char * const Module_fields[]={ "body", @@ -1097,7 +1109,7 @@ static PyObject* ast2obj_list(astmodulestate *state, asdl_seq *seq, PyObject* (* if (!result) return NULL; for (i = 0; i < n; i++) { - value = func(state, asdl_seq_GET(seq, i)); + value = func(state, asdl_seq_GET_UNTYPED(seq, i)); if (!value) { Py_DECREF(result); return NULL; @@ -1912,7 +1924,8 @@ static int obj2ast_type_ignore(astmodulestate *state, PyObject* obj, type_ignore_ty* out, PyArena* arena); mod_ty -Module(asdl_seq * body, asdl_seq * type_ignores, PyArena *arena) +Module(asdl_stmt_seq * body, asdl_type_ignore_seq * type_ignores, PyArena + *arena) { mod_ty p; p = (mod_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -1925,7 +1938,7 @@ Module(asdl_seq * body, asdl_seq * type_ignores, PyArena *arena) } mod_ty -Interactive(asdl_seq * body, PyArena *arena) +Interactive(asdl_stmt_seq * body, PyArena *arena) { mod_ty p; p = (mod_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -1954,7 +1967,7 @@ Expression(expr_ty body, PyArena *arena) } mod_ty -FunctionType(asdl_seq * argtypes, expr_ty returns, PyArena *arena) +FunctionType(asdl_expr_seq * argtypes, expr_ty returns, PyArena *arena) { mod_ty p; if (!returns) { @@ -1972,9 +1985,10 @@ FunctionType(asdl_seq * argtypes, expr_ty returns, PyArena *arena) } stmt_ty -FunctionDef(identifier name, arguments_ty args, asdl_seq * body, asdl_seq * - decorator_list, expr_ty returns, string type_comment, int lineno, - int col_offset, int end_lineno, int end_col_offset, PyArena *arena) +FunctionDef(identifier name, arguments_ty args, asdl_stmt_seq * body, + asdl_expr_seq * decorator_list, expr_ty returns, string + type_comment, int lineno, int col_offset, int end_lineno, int + end_col_offset, PyArena *arena) { stmt_ty p; if (!name) { @@ -2005,10 +2019,10 @@ FunctionDef(identifier name, arguments_ty args, asdl_seq * body, asdl_seq * } stmt_ty -AsyncFunctionDef(identifier name, arguments_ty args, asdl_seq * body, asdl_seq - * decorator_list, expr_ty returns, string type_comment, int - lineno, int col_offset, int end_lineno, int end_col_offset, - PyArena *arena) +AsyncFunctionDef(identifier name, arguments_ty args, asdl_stmt_seq * body, + asdl_expr_seq * decorator_list, expr_ty returns, string + type_comment, int lineno, int col_offset, int end_lineno, int + end_col_offset, PyArena *arena) { stmt_ty p; if (!name) { @@ -2039,9 +2053,9 @@ AsyncFunctionDef(identifier name, arguments_ty args, asdl_seq * body, asdl_seq } stmt_ty -ClassDef(identifier name, asdl_seq * bases, asdl_seq * keywords, asdl_seq * - body, asdl_seq * decorator_list, int lineno, int col_offset, int - end_lineno, int end_col_offset, PyArena *arena) +ClassDef(identifier name, asdl_expr_seq * bases, asdl_keyword_seq * keywords, + asdl_stmt_seq * body, asdl_expr_seq * decorator_list, int lineno, int + col_offset, int end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; if (!name) { @@ -2083,7 +2097,7 @@ Return(expr_ty value, int lineno, int col_offset, int end_lineno, int } stmt_ty -Delete(asdl_seq * targets, int lineno, int col_offset, int end_lineno, int +Delete(asdl_expr_seq * targets, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; @@ -2100,8 +2114,8 @@ Delete(asdl_seq * targets, int lineno, int col_offset, int end_lineno, int } stmt_ty -Assign(asdl_seq * targets, expr_ty value, string type_comment, int lineno, int - col_offset, int end_lineno, int end_col_offset, PyArena *arena) +Assign(asdl_expr_seq * targets, expr_ty value, string type_comment, int lineno, + int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; if (!value) { @@ -2189,8 +2203,8 @@ AnnAssign(expr_ty target, expr_ty annotation, expr_ty value, int simple, int } stmt_ty -For(expr_ty target, expr_ty iter, asdl_seq * body, asdl_seq * orelse, string - type_comment, int lineno, int col_offset, int end_lineno, int +For(expr_ty target, expr_ty iter, asdl_stmt_seq * body, asdl_stmt_seq * orelse, + string type_comment, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; @@ -2221,9 +2235,9 @@ For(expr_ty target, expr_ty iter, asdl_seq * body, asdl_seq * orelse, string } stmt_ty -AsyncFor(expr_ty target, expr_ty iter, asdl_seq * body, asdl_seq * orelse, - string type_comment, int lineno, int col_offset, int end_lineno, int - end_col_offset, PyArena *arena) +AsyncFor(expr_ty target, expr_ty iter, asdl_stmt_seq * body, asdl_stmt_seq * + orelse, string type_comment, int lineno, int col_offset, int + end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; if (!target) { @@ -2253,8 +2267,8 @@ AsyncFor(expr_ty target, expr_ty iter, asdl_seq * body, asdl_seq * orelse, } stmt_ty -While(expr_ty test, asdl_seq * body, asdl_seq * orelse, int lineno, int - col_offset, int end_lineno, int end_col_offset, PyArena *arena) +While(expr_ty test, asdl_stmt_seq * body, asdl_stmt_seq * orelse, int lineno, + int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; if (!test) { @@ -2277,7 +2291,7 @@ While(expr_ty test, asdl_seq * body, asdl_seq * orelse, int lineno, int } stmt_ty -If(expr_ty test, asdl_seq * body, asdl_seq * orelse, int lineno, int +If(expr_ty test, asdl_stmt_seq * body, asdl_stmt_seq * orelse, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; @@ -2301,8 +2315,8 @@ If(expr_ty test, asdl_seq * body, asdl_seq * orelse, int lineno, int } stmt_ty -With(asdl_seq * items, asdl_seq * body, string type_comment, int lineno, int - col_offset, int end_lineno, int end_col_offset, PyArena *arena) +With(asdl_withitem_seq * items, asdl_stmt_seq * body, string type_comment, int + lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; p = (stmt_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -2320,8 +2334,9 @@ With(asdl_seq * items, asdl_seq * body, string type_comment, int lineno, int } stmt_ty -AsyncWith(asdl_seq * items, asdl_seq * body, string type_comment, int lineno, - int col_offset, int end_lineno, int end_col_offset, PyArena *arena) +AsyncWith(asdl_withitem_seq * items, asdl_stmt_seq * body, string type_comment, + int lineno, int col_offset, int end_lineno, int end_col_offset, + PyArena *arena) { stmt_ty p; p = (stmt_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -2357,9 +2372,9 @@ Raise(expr_ty exc, expr_ty cause, int lineno, int col_offset, int end_lineno, } stmt_ty -Try(asdl_seq * body, asdl_seq * handlers, asdl_seq * orelse, asdl_seq * - finalbody, int lineno, int col_offset, int end_lineno, int end_col_offset, - PyArena *arena) +Try(asdl_stmt_seq * body, asdl_excepthandler_seq * handlers, asdl_stmt_seq * + orelse, asdl_stmt_seq * finalbody, int lineno, int col_offset, int + end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; p = (stmt_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -2401,7 +2416,7 @@ Assert(expr_ty test, expr_ty msg, int lineno, int col_offset, int end_lineno, } stmt_ty -Import(asdl_seq * names, int lineno, int col_offset, int end_lineno, int +Import(asdl_alias_seq * names, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; @@ -2418,8 +2433,8 @@ Import(asdl_seq * names, int lineno, int col_offset, int end_lineno, int } stmt_ty -ImportFrom(identifier module, asdl_seq * names, int level, int lineno, int - col_offset, int end_lineno, int end_col_offset, PyArena *arena) +ImportFrom(identifier module, asdl_alias_seq * names, int level, int lineno, + int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; p = (stmt_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -2437,8 +2452,8 @@ ImportFrom(identifier module, asdl_seq * names, int level, int lineno, int } stmt_ty -Global(asdl_seq * names, int lineno, int col_offset, int end_lineno, int - end_col_offset, PyArena *arena) +Global(asdl_identifier_seq * names, int lineno, int col_offset, int end_lineno, + int end_col_offset, PyArena *arena) { stmt_ty p; p = (stmt_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -2454,8 +2469,8 @@ Global(asdl_seq * names, int lineno, int col_offset, int end_lineno, int } stmt_ty -Nonlocal(asdl_seq * names, int lineno, int col_offset, int end_lineno, int - end_col_offset, PyArena *arena) +Nonlocal(asdl_identifier_seq * names, int lineno, int col_offset, int + end_lineno, int end_col_offset, PyArena *arena) { stmt_ty p; p = (stmt_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -2541,7 +2556,7 @@ Continue(int lineno, int col_offset, int end_lineno, int end_col_offset, } expr_ty -BoolOp(boolop_ty op, asdl_seq * values, int lineno, int col_offset, int +BoolOp(boolop_ty op, asdl_expr_seq * values, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { expr_ty p; @@ -2716,8 +2731,8 @@ IfExp(expr_ty test, expr_ty body, expr_ty orelse, int lineno, int col_offset, } expr_ty -Dict(asdl_seq * keys, asdl_seq * values, int lineno, int col_offset, int - end_lineno, int end_col_offset, PyArena *arena) +Dict(asdl_expr_seq * keys, asdl_expr_seq * values, int lineno, int col_offset, + int end_lineno, int end_col_offset, PyArena *arena) { expr_ty p; p = (expr_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -2734,7 +2749,7 @@ Dict(asdl_seq * keys, asdl_seq * values, int lineno, int col_offset, int } expr_ty -Set(asdl_seq * elts, int lineno, int col_offset, int end_lineno, int +Set(asdl_expr_seq * elts, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { expr_ty p; @@ -2751,8 +2766,8 @@ Set(asdl_seq * elts, int lineno, int col_offset, int end_lineno, int } expr_ty -ListComp(expr_ty elt, asdl_seq * generators, int lineno, int col_offset, int - end_lineno, int end_col_offset, PyArena *arena) +ListComp(expr_ty elt, asdl_comprehension_seq * generators, int lineno, int + col_offset, int end_lineno, int end_col_offset, PyArena *arena) { expr_ty p; if (!elt) { @@ -2774,8 +2789,8 @@ ListComp(expr_ty elt, asdl_seq * generators, int lineno, int col_offset, int } expr_ty -SetComp(expr_ty elt, asdl_seq * generators, int lineno, int col_offset, int - end_lineno, int end_col_offset, PyArena *arena) +SetComp(expr_ty elt, asdl_comprehension_seq * generators, int lineno, int + col_offset, int end_lineno, int end_col_offset, PyArena *arena) { expr_ty p; if (!elt) { @@ -2797,8 +2812,9 @@ SetComp(expr_ty elt, asdl_seq * generators, int lineno, int col_offset, int } expr_ty -DictComp(expr_ty key, expr_ty value, asdl_seq * generators, int lineno, int - col_offset, int end_lineno, int end_col_offset, PyArena *arena) +DictComp(expr_ty key, expr_ty value, asdl_comprehension_seq * generators, int + lineno, int col_offset, int end_lineno, int end_col_offset, PyArena + *arena) { expr_ty p; if (!key) { @@ -2826,8 +2842,8 @@ DictComp(expr_ty key, expr_ty value, asdl_seq * generators, int lineno, int } expr_ty -GeneratorExp(expr_ty elt, asdl_seq * generators, int lineno, int col_offset, - int end_lineno, int end_col_offset, PyArena *arena) +GeneratorExp(expr_ty elt, asdl_comprehension_seq * generators, int lineno, int + col_offset, int end_lineno, int end_col_offset, PyArena *arena) { expr_ty p; if (!elt) { @@ -2910,8 +2926,9 @@ YieldFrom(expr_ty value, int lineno, int col_offset, int end_lineno, int } expr_ty -Compare(expr_ty left, asdl_int_seq * ops, asdl_seq * comparators, int lineno, - int col_offset, int end_lineno, int end_col_offset, PyArena *arena) +Compare(expr_ty left, asdl_int_seq * ops, asdl_expr_seq * comparators, int + lineno, int col_offset, int end_lineno, int end_col_offset, PyArena + *arena) { expr_ty p; if (!left) { @@ -2934,8 +2951,8 @@ Compare(expr_ty left, asdl_int_seq * ops, asdl_seq * comparators, int lineno, } expr_ty -Call(expr_ty func, asdl_seq * args, asdl_seq * keywords, int lineno, int - col_offset, int end_lineno, int end_col_offset, PyArena *arena) +Call(expr_ty func, asdl_expr_seq * args, asdl_keyword_seq * keywords, int + lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { expr_ty p; if (!func) { @@ -2983,8 +3000,8 @@ FormattedValue(expr_ty value, int conversion, expr_ty format_spec, int lineno, } expr_ty -JoinedStr(asdl_seq * values, int lineno, int col_offset, int end_lineno, int - end_col_offset, PyArena *arena) +JoinedStr(asdl_expr_seq * values, int lineno, int col_offset, int end_lineno, + int end_col_offset, PyArena *arena) { expr_ty p; p = (expr_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -3147,7 +3164,7 @@ Name(identifier id, expr_context_ty ctx, int lineno, int col_offset, int } expr_ty -List(asdl_seq * elts, expr_context_ty ctx, int lineno, int col_offset, int +List(asdl_expr_seq * elts, expr_context_ty ctx, int lineno, int col_offset, int end_lineno, int end_col_offset, PyArena *arena) { expr_ty p; @@ -3170,8 +3187,8 @@ List(asdl_seq * elts, expr_context_ty ctx, int lineno, int col_offset, int } expr_ty -Tuple(asdl_seq * elts, expr_context_ty ctx, int lineno, int col_offset, int - end_lineno, int end_col_offset, PyArena *arena) +Tuple(asdl_expr_seq * elts, expr_context_ty ctx, int lineno, int col_offset, + int end_lineno, int end_col_offset, PyArena *arena) { expr_ty p; if (!ctx) { @@ -3212,7 +3229,7 @@ Slice(expr_ty lower, expr_ty upper, expr_ty step, int lineno, int col_offset, } comprehension_ty -comprehension(expr_ty target, expr_ty iter, asdl_seq * ifs, int is_async, +comprehension(expr_ty target, expr_ty iter, asdl_expr_seq * ifs, int is_async, PyArena *arena) { comprehension_ty p; @@ -3237,8 +3254,9 @@ comprehension(expr_ty target, expr_ty iter, asdl_seq * ifs, int is_async, } excepthandler_ty -ExceptHandler(expr_ty type, identifier name, asdl_seq * body, int lineno, int - col_offset, int end_lineno, int end_col_offset, PyArena *arena) +ExceptHandler(expr_ty type, identifier name, asdl_stmt_seq * body, int lineno, + int col_offset, int end_lineno, int end_col_offset, PyArena + *arena) { excepthandler_ty p; p = (excepthandler_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -3256,9 +3274,9 @@ ExceptHandler(expr_ty type, identifier name, asdl_seq * body, int lineno, int } arguments_ty -arguments(asdl_seq * posonlyargs, asdl_seq * args, arg_ty vararg, asdl_seq * - kwonlyargs, asdl_seq * kw_defaults, arg_ty kwarg, asdl_seq * - defaults, PyArena *arena) +arguments(asdl_arg_seq * posonlyargs, asdl_arg_seq * args, arg_ty vararg, + asdl_arg_seq * kwonlyargs, asdl_expr_seq * kw_defaults, arg_ty kwarg, + asdl_expr_seq * defaults, PyArena *arena) { arguments_ty p; p = (arguments_ty)PyArena_Malloc(arena, sizeof(*p)); @@ -3386,12 +3404,12 @@ ast2obj_mod(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->Module_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.Module.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.Module.body, ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.Module.type_ignores, + value = ast2obj_list(state, (asdl_seq*)o->v.Module.type_ignores, ast2obj_type_ignore); if (!value) goto failed; if (PyObject_SetAttr(result, state->type_ignores, value) == -1) @@ -3402,7 +3420,8 @@ ast2obj_mod(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->Interactive_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.Interactive.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.Interactive.body, + ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; @@ -3422,7 +3441,8 @@ ast2obj_mod(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->FunctionType_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.FunctionType.argtypes, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.FunctionType.argtypes, + ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->argtypes, value) == -1) goto failed; @@ -3465,12 +3485,13 @@ ast2obj_stmt(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->args, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.FunctionDef.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.FunctionDef.body, + ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.FunctionDef.decorator_list, + value = ast2obj_list(state, (asdl_seq*)o->v.FunctionDef.decorator_list, ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->decorator_list, value) == -1) @@ -3501,12 +3522,14 @@ ast2obj_stmt(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->args, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.AsyncFunctionDef.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.AsyncFunctionDef.body, + ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.AsyncFunctionDef.decorator_list, + value = ast2obj_list(state, + (asdl_seq*)o->v.AsyncFunctionDef.decorator_list, ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->decorator_list, value) == -1) @@ -3532,22 +3555,26 @@ ast2obj_stmt(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->name, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.ClassDef.bases, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.ClassDef.bases, + ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->bases, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.ClassDef.keywords, ast2obj_keyword); + value = ast2obj_list(state, (asdl_seq*)o->v.ClassDef.keywords, + ast2obj_keyword); if (!value) goto failed; if (PyObject_SetAttr(result, state->keywords, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.ClassDef.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.ClassDef.body, + ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.ClassDef.decorator_list, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.ClassDef.decorator_list, + ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->decorator_list, value) == -1) goto failed; @@ -3567,7 +3594,8 @@ ast2obj_stmt(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->Delete_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.Delete.targets, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.Delete.targets, + ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->targets, value) == -1) goto failed; @@ -3577,7 +3605,8 @@ ast2obj_stmt(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->Assign_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.Assign.targets, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.Assign.targets, + ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->targets, value) == -1) goto failed; @@ -3652,12 +3681,12 @@ ast2obj_stmt(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->iter, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.For.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.For.body, ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.For.orelse, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.For.orelse, ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->orelse, value) == -1) goto failed; @@ -3682,12 +3711,14 @@ ast2obj_stmt(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->iter, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.AsyncFor.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.AsyncFor.body, + ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.AsyncFor.orelse, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.AsyncFor.orelse, + ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->orelse, value) == -1) goto failed; @@ -3707,12 +3738,12 @@ ast2obj_stmt(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->test, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.While.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.While.body, ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.While.orelse, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.While.orelse, ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->orelse, value) == -1) goto failed; @@ -3727,12 +3758,12 @@ ast2obj_stmt(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->test, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.If.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.If.body, ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.If.orelse, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.If.orelse, ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->orelse, value) == -1) goto failed; @@ -3742,12 +3773,13 @@ ast2obj_stmt(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->With_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.With.items, ast2obj_withitem); + value = ast2obj_list(state, (asdl_seq*)o->v.With.items, + ast2obj_withitem); if (!value) goto failed; if (PyObject_SetAttr(result, state->items, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.With.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.With.body, ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; @@ -3762,12 +3794,14 @@ ast2obj_stmt(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->AsyncWith_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.AsyncWith.items, ast2obj_withitem); + value = ast2obj_list(state, (asdl_seq*)o->v.AsyncWith.items, + ast2obj_withitem); if (!value) goto failed; if (PyObject_SetAttr(result, state->items, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.AsyncWith.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.AsyncWith.body, + ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; @@ -3797,22 +3831,24 @@ ast2obj_stmt(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->Try_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.Try.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.Try.body, ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.Try.handlers, ast2obj_excepthandler); + value = ast2obj_list(state, (asdl_seq*)o->v.Try.handlers, + ast2obj_excepthandler); if (!value) goto failed; if (PyObject_SetAttr(result, state->handlers, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.Try.orelse, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.Try.orelse, ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->orelse, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.Try.finalbody, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.Try.finalbody, + ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->finalbody, value) == -1) goto failed; @@ -3837,7 +3873,8 @@ ast2obj_stmt(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->Import_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.Import.names, ast2obj_alias); + value = ast2obj_list(state, (asdl_seq*)o->v.Import.names, + ast2obj_alias); if (!value) goto failed; if (PyObject_SetAttr(result, state->names, value) == -1) goto failed; @@ -3852,7 +3889,8 @@ ast2obj_stmt(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->module, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.ImportFrom.names, ast2obj_alias); + value = ast2obj_list(state, (asdl_seq*)o->v.ImportFrom.names, + ast2obj_alias); if (!value) goto failed; if (PyObject_SetAttr(result, state->names, value) == -1) goto failed; @@ -3867,7 +3905,8 @@ ast2obj_stmt(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->Global_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.Global.names, ast2obj_identifier); + value = ast2obj_list(state, (asdl_seq*)o->v.Global.names, + ast2obj_identifier); if (!value) goto failed; if (PyObject_SetAttr(result, state->names, value) == -1) goto failed; @@ -3877,7 +3916,8 @@ ast2obj_stmt(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->Nonlocal_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.Nonlocal.names, ast2obj_identifier); + value = ast2obj_list(state, (asdl_seq*)o->v.Nonlocal.names, + ast2obj_identifier); if (!value) goto failed; if (PyObject_SetAttr(result, state->names, value) == -1) goto failed; @@ -3955,7 +3995,8 @@ ast2obj_expr(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->op, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.BoolOp.values, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.BoolOp.values, + ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->values, value) == -1) goto failed; @@ -4050,12 +4091,12 @@ ast2obj_expr(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->Dict_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.Dict.keys, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.Dict.keys, ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->keys, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.Dict.values, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.Dict.values, ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->values, value) == -1) goto failed; @@ -4065,7 +4106,7 @@ ast2obj_expr(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->Set_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.Set.elts, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.Set.elts, ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->elts, value) == -1) goto failed; @@ -4080,7 +4121,7 @@ ast2obj_expr(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->elt, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.ListComp.generators, + value = ast2obj_list(state, (asdl_seq*)o->v.ListComp.generators, ast2obj_comprehension); if (!value) goto failed; if (PyObject_SetAttr(result, state->generators, value) == -1) @@ -4096,7 +4137,7 @@ ast2obj_expr(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->elt, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.SetComp.generators, + value = ast2obj_list(state, (asdl_seq*)o->v.SetComp.generators, ast2obj_comprehension); if (!value) goto failed; if (PyObject_SetAttr(result, state->generators, value) == -1) @@ -4117,7 +4158,7 @@ ast2obj_expr(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->value, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.DictComp.generators, + value = ast2obj_list(state, (asdl_seq*)o->v.DictComp.generators, ast2obj_comprehension); if (!value) goto failed; if (PyObject_SetAttr(result, state->generators, value) == -1) @@ -4133,7 +4174,7 @@ ast2obj_expr(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->elt, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.GeneratorExp.generators, + value = ast2obj_list(state, (asdl_seq*)o->v.GeneratorExp.generators, ast2obj_comprehension); if (!value) goto failed; if (PyObject_SetAttr(result, state->generators, value) == -1) @@ -4190,7 +4231,8 @@ ast2obj_expr(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->ops, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.Compare.comparators, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.Compare.comparators, + ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->comparators, value) == -1) goto failed; @@ -4205,12 +4247,13 @@ ast2obj_expr(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->func, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.Call.args, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.Call.args, ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->args, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.Call.keywords, ast2obj_keyword); + value = ast2obj_list(state, (asdl_seq*)o->v.Call.keywords, + ast2obj_keyword); if (!value) goto failed; if (PyObject_SetAttr(result, state->keywords, value) == -1) goto failed; @@ -4240,7 +4283,8 @@ ast2obj_expr(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->JoinedStr_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.JoinedStr.values, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.JoinedStr.values, + ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->values, value) == -1) goto failed; @@ -4335,7 +4379,7 @@ ast2obj_expr(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->List_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.List.elts, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.List.elts, ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->elts, value) == -1) goto failed; @@ -4350,7 +4394,7 @@ ast2obj_expr(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->Tuple_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) goto failed; - value = ast2obj_list(state, o->v.Tuple.elts, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->v.Tuple.elts, ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->elts, value) == -1) goto failed; @@ -4557,7 +4601,7 @@ ast2obj_comprehension(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->iter, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->ifs, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->ifs, ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->ifs, value) == -1) goto failed; @@ -4598,7 +4642,8 @@ ast2obj_excepthandler(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->name, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->v.ExceptHandler.body, ast2obj_stmt); + value = ast2obj_list(state, (asdl_seq*)o->v.ExceptHandler.body, + ast2obj_stmt); if (!value) goto failed; if (PyObject_SetAttr(result, state->body, value) == -1) goto failed; @@ -4644,12 +4689,12 @@ ast2obj_arguments(astmodulestate *state, void* _o) tp = (PyTypeObject *)state->arguments_type; result = PyType_GenericNew(tp, NULL, NULL); if (!result) return NULL; - value = ast2obj_list(state, o->posonlyargs, ast2obj_arg); + value = ast2obj_list(state, (asdl_seq*)o->posonlyargs, ast2obj_arg); if (!value) goto failed; if (PyObject_SetAttr(result, state->posonlyargs, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->args, ast2obj_arg); + value = ast2obj_list(state, (asdl_seq*)o->args, ast2obj_arg); if (!value) goto failed; if (PyObject_SetAttr(result, state->args, value) == -1) goto failed; @@ -4659,12 +4704,12 @@ ast2obj_arguments(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->vararg, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->kwonlyargs, ast2obj_arg); + value = ast2obj_list(state, (asdl_seq*)o->kwonlyargs, ast2obj_arg); if (!value) goto failed; if (PyObject_SetAttr(result, state->kwonlyargs, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->kw_defaults, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->kw_defaults, ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->kw_defaults, value) == -1) goto failed; @@ -4674,7 +4719,7 @@ ast2obj_arguments(astmodulestate *state, void* _o) if (PyObject_SetAttr(result, state->kwarg, value) == -1) goto failed; Py_DECREF(value); - value = ast2obj_list(state, o->defaults, ast2obj_expr); + value = ast2obj_list(state, (asdl_seq*)o->defaults, ast2obj_expr); if (!value) goto failed; if (PyObject_SetAttr(result, state->defaults, value) == -1) goto failed; @@ -4899,8 +4944,8 @@ obj2ast_mod(astmodulestate *state, PyObject* obj, mod_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* body; - asdl_seq* type_ignores; + asdl_stmt_seq* body; + asdl_type_ignore_seq* type_ignores; if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { return 1; @@ -4918,7 +4963,7 @@ obj2ast_mod(astmodulestate *state, PyObject* obj, mod_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -4951,7 +4996,7 @@ obj2ast_mod(astmodulestate *state, PyObject* obj, mod_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - type_ignores = _Py_asdl_seq_new(len, arena); + type_ignores = _Py_asdl_type_ignore_seq_new(len, arena); if (type_ignores == NULL) goto failed; for (i = 0; i < len; i++) { type_ignore_ty val; @@ -4978,7 +5023,7 @@ obj2ast_mod(astmodulestate *state, PyObject* obj, mod_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* body; + asdl_stmt_seq* body; if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { return 1; @@ -4996,7 +5041,7 @@ obj2ast_mod(astmodulestate *state, PyObject* obj, mod_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -5048,7 +5093,7 @@ obj2ast_mod(astmodulestate *state, PyObject* obj, mod_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* argtypes; + asdl_expr_seq* argtypes; expr_ty returns; if (_PyObject_LookupAttr(obj, state->argtypes, &tmp) < 0) { @@ -5067,7 +5112,7 @@ obj2ast_mod(astmodulestate *state, PyObject* obj, mod_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - argtypes = _Py_asdl_seq_new(len, arena); + argtypes = _Py_asdl_expr_seq_new(len, arena); if (argtypes == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -5184,8 +5229,8 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) if (isinstance) { identifier name; arguments_ty args; - asdl_seq* body; - asdl_seq* decorator_list; + asdl_stmt_seq* body; + asdl_expr_seq* decorator_list; expr_ty returns; string type_comment; @@ -5231,7 +5276,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -5264,7 +5309,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - decorator_list = _Py_asdl_seq_new(len, arena); + decorator_list = _Py_asdl_expr_seq_new(len, arena); if (decorator_list == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -5321,8 +5366,8 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) if (isinstance) { identifier name; arguments_ty args; - asdl_seq* body; - asdl_seq* decorator_list; + asdl_stmt_seq* body; + asdl_expr_seq* decorator_list; expr_ty returns; string type_comment; @@ -5368,7 +5413,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -5401,7 +5446,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - decorator_list = _Py_asdl_seq_new(len, arena); + decorator_list = _Py_asdl_expr_seq_new(len, arena); if (decorator_list == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -5457,10 +5502,10 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) } if (isinstance) { identifier name; - asdl_seq* bases; - asdl_seq* keywords; - asdl_seq* body; - asdl_seq* decorator_list; + asdl_expr_seq* bases; + asdl_keyword_seq* keywords; + asdl_stmt_seq* body; + asdl_expr_seq* decorator_list; if (_PyObject_LookupAttr(obj, state->name, &tmp) < 0) { return 1; @@ -5491,7 +5536,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - bases = _Py_asdl_seq_new(len, arena); + bases = _Py_asdl_expr_seq_new(len, arena); if (bases == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -5524,7 +5569,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - keywords = _Py_asdl_seq_new(len, arena); + keywords = _Py_asdl_keyword_seq_new(len, arena); if (keywords == NULL) goto failed; for (i = 0; i < len; i++) { keyword_ty val; @@ -5557,7 +5602,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -5590,7 +5635,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - decorator_list = _Py_asdl_seq_new(len, arena); + decorator_list = _Py_asdl_expr_seq_new(len, arena); if (decorator_list == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -5644,7 +5689,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* targets; + asdl_expr_seq* targets; if (_PyObject_LookupAttr(obj, state->targets, &tmp) < 0) { return 1; @@ -5662,7 +5707,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - targets = _Py_asdl_seq_new(len, arena); + targets = _Py_asdl_expr_seq_new(len, arena); if (targets == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -5690,7 +5735,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* targets; + asdl_expr_seq* targets; expr_ty value; string type_comment; @@ -5710,7 +5755,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - targets = _Py_asdl_seq_new(len, arena); + targets = _Py_asdl_expr_seq_new(len, arena); if (targets == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -5888,8 +5933,8 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) if (isinstance) { expr_ty target; expr_ty iter; - asdl_seq* body; - asdl_seq* orelse; + asdl_stmt_seq* body; + asdl_stmt_seq* orelse; string type_comment; if (_PyObject_LookupAttr(obj, state->target, &tmp) < 0) { @@ -5934,7 +5979,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -5967,7 +6012,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - orelse = _Py_asdl_seq_new(len, arena); + orelse = _Py_asdl_stmt_seq_new(len, arena); if (orelse == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6010,8 +6055,8 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) if (isinstance) { expr_ty target; expr_ty iter; - asdl_seq* body; - asdl_seq* orelse; + asdl_stmt_seq* body; + asdl_stmt_seq* orelse; string type_comment; if (_PyObject_LookupAttr(obj, state->target, &tmp) < 0) { @@ -6056,7 +6101,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6089,7 +6134,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - orelse = _Py_asdl_seq_new(len, arena); + orelse = _Py_asdl_stmt_seq_new(len, arena); if (orelse == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6131,8 +6176,8 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) } if (isinstance) { expr_ty test; - asdl_seq* body; - asdl_seq* orelse; + asdl_stmt_seq* body; + asdl_stmt_seq* orelse; if (_PyObject_LookupAttr(obj, state->test, &tmp) < 0) { return 1; @@ -6163,7 +6208,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6196,7 +6241,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - orelse = _Py_asdl_seq_new(len, arena); + orelse = _Py_asdl_stmt_seq_new(len, arena); if (orelse == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6225,8 +6270,8 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) } if (isinstance) { expr_ty test; - asdl_seq* body; - asdl_seq* orelse; + asdl_stmt_seq* body; + asdl_stmt_seq* orelse; if (_PyObject_LookupAttr(obj, state->test, &tmp) < 0) { return 1; @@ -6257,7 +6302,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6290,7 +6335,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - orelse = _Py_asdl_seq_new(len, arena); + orelse = _Py_asdl_stmt_seq_new(len, arena); if (orelse == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6318,8 +6363,8 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* items; - asdl_seq* body; + asdl_withitem_seq* items; + asdl_stmt_seq* body; string type_comment; if (_PyObject_LookupAttr(obj, state->items, &tmp) < 0) { @@ -6338,7 +6383,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - items = _Py_asdl_seq_new(len, arena); + items = _Py_asdl_withitem_seq_new(len, arena); if (items == NULL) goto failed; for (i = 0; i < len; i++) { withitem_ty val; @@ -6371,7 +6416,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6412,8 +6457,8 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* items; - asdl_seq* body; + asdl_withitem_seq* items; + asdl_stmt_seq* body; string type_comment; if (_PyObject_LookupAttr(obj, state->items, &tmp) < 0) { @@ -6432,7 +6477,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - items = _Py_asdl_seq_new(len, arena); + items = _Py_asdl_withitem_seq_new(len, arena); if (items == NULL) goto failed; for (i = 0; i < len; i++) { withitem_ty val; @@ -6465,7 +6510,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6546,10 +6591,10 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* body; - asdl_seq* handlers; - asdl_seq* orelse; - asdl_seq* finalbody; + asdl_stmt_seq* body; + asdl_excepthandler_seq* handlers; + asdl_stmt_seq* orelse; + asdl_stmt_seq* finalbody; if (_PyObject_LookupAttr(obj, state->body, &tmp) < 0) { return 1; @@ -6567,7 +6612,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6600,7 +6645,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - handlers = _Py_asdl_seq_new(len, arena); + handlers = _Py_asdl_excepthandler_seq_new(len, arena); if (handlers == NULL) goto failed; for (i = 0; i < len; i++) { excepthandler_ty val; @@ -6633,7 +6678,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - orelse = _Py_asdl_seq_new(len, arena); + orelse = _Py_asdl_stmt_seq_new(len, arena); if (orelse == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6666,7 +6711,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - finalbody = _Py_asdl_seq_new(len, arena); + finalbody = _Py_asdl_stmt_seq_new(len, arena); if (finalbody == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -6734,7 +6779,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* names; + asdl_alias_seq* names; if (_PyObject_LookupAttr(obj, state->names, &tmp) < 0) { return 1; @@ -6752,7 +6797,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - names = _Py_asdl_seq_new(len, arena); + names = _Py_asdl_alias_seq_new(len, arena); if (names == NULL) goto failed; for (i = 0; i < len; i++) { alias_ty val; @@ -6781,7 +6826,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) } if (isinstance) { identifier module; - asdl_seq* names; + asdl_alias_seq* names; int level; if (_PyObject_LookupAttr(obj, state->module, &tmp) < 0) { @@ -6813,7 +6858,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - names = _Py_asdl_seq_new(len, arena); + names = _Py_asdl_alias_seq_new(len, arena); if (names == NULL) goto failed; for (i = 0; i < len; i++) { alias_ty val; @@ -6854,7 +6899,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* names; + asdl_identifier_seq* names; if (_PyObject_LookupAttr(obj, state->names, &tmp) < 0) { return 1; @@ -6872,7 +6917,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - names = _Py_asdl_seq_new(len, arena); + names = _Py_asdl_identifier_seq_new(len, arena); if (names == NULL) goto failed; for (i = 0; i < len; i++) { identifier val; @@ -6900,7 +6945,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* names; + asdl_identifier_seq* names; if (_PyObject_LookupAttr(obj, state->names, &tmp) < 0) { return 1; @@ -6918,7 +6963,7 @@ obj2ast_stmt(astmodulestate *state, PyObject* obj, stmt_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - names = _Py_asdl_seq_new(len, arena); + names = _Py_asdl_identifier_seq_new(len, arena); if (names == NULL) goto failed; for (i = 0; i < len; i++) { identifier val; @@ -7081,7 +7126,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) } if (isinstance) { boolop_ty op; - asdl_seq* values; + asdl_expr_seq* values; if (_PyObject_LookupAttr(obj, state->op, &tmp) < 0) { return 1; @@ -7112,7 +7157,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - values = _Py_asdl_seq_new(len, arena); + values = _Py_asdl_expr_seq_new(len, arena); if (values == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -7368,8 +7413,8 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* keys; - asdl_seq* values; + asdl_expr_seq* keys; + asdl_expr_seq* values; if (_PyObject_LookupAttr(obj, state->keys, &tmp) < 0) { return 1; @@ -7387,7 +7432,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - keys = _Py_asdl_seq_new(len, arena); + keys = _Py_asdl_expr_seq_new(len, arena); if (keys == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -7420,7 +7465,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - values = _Py_asdl_seq_new(len, arena); + values = _Py_asdl_expr_seq_new(len, arena); if (values == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -7448,7 +7493,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* elts; + asdl_expr_seq* elts; if (_PyObject_LookupAttr(obj, state->elts, &tmp) < 0) { return 1; @@ -7466,7 +7511,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - elts = _Py_asdl_seq_new(len, arena); + elts = _Py_asdl_expr_seq_new(len, arena); if (elts == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -7494,7 +7539,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) } if (isinstance) { expr_ty elt; - asdl_seq* generators; + asdl_comprehension_seq* generators; if (_PyObject_LookupAttr(obj, state->elt, &tmp) < 0) { return 1; @@ -7525,7 +7570,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - generators = _Py_asdl_seq_new(len, arena); + generators = _Py_asdl_comprehension_seq_new(len, arena); if (generators == NULL) goto failed; for (i = 0; i < len; i++) { comprehension_ty val; @@ -7554,7 +7599,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) } if (isinstance) { expr_ty elt; - asdl_seq* generators; + asdl_comprehension_seq* generators; if (_PyObject_LookupAttr(obj, state->elt, &tmp) < 0) { return 1; @@ -7585,7 +7630,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - generators = _Py_asdl_seq_new(len, arena); + generators = _Py_asdl_comprehension_seq_new(len, arena); if (generators == NULL) goto failed; for (i = 0; i < len; i++) { comprehension_ty val; @@ -7615,7 +7660,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) if (isinstance) { expr_ty key; expr_ty value; - asdl_seq* generators; + asdl_comprehension_seq* generators; if (_PyObject_LookupAttr(obj, state->key, &tmp) < 0) { return 1; @@ -7659,7 +7704,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - generators = _Py_asdl_seq_new(len, arena); + generators = _Py_asdl_comprehension_seq_new(len, arena); if (generators == NULL) goto failed; for (i = 0; i < len; i++) { comprehension_ty val; @@ -7688,7 +7733,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) } if (isinstance) { expr_ty elt; - asdl_seq* generators; + asdl_comprehension_seq* generators; if (_PyObject_LookupAttr(obj, state->elt, &tmp) < 0) { return 1; @@ -7719,7 +7764,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - generators = _Py_asdl_seq_new(len, arena); + generators = _Py_asdl_comprehension_seq_new(len, arena); if (generators == NULL) goto failed; for (i = 0; i < len; i++) { comprehension_ty val; @@ -7827,7 +7872,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) if (isinstance) { expr_ty left; asdl_int_seq* ops; - asdl_seq* comparators; + asdl_expr_seq* comparators; if (_PyObject_LookupAttr(obj, state->left, &tmp) < 0) { return 1; @@ -7891,7 +7936,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - comparators = _Py_asdl_seq_new(len, arena); + comparators = _Py_asdl_expr_seq_new(len, arena); if (comparators == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -7920,8 +7965,8 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) } if (isinstance) { expr_ty func; - asdl_seq* args; - asdl_seq* keywords; + asdl_expr_seq* args; + asdl_keyword_seq* keywords; if (_PyObject_LookupAttr(obj, state->func, &tmp) < 0) { return 1; @@ -7952,7 +7997,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - args = _Py_asdl_seq_new(len, arena); + args = _Py_asdl_expr_seq_new(len, arena); if (args == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -7985,7 +8030,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - keywords = _Py_asdl_seq_new(len, arena); + keywords = _Py_asdl_keyword_seq_new(len, arena); if (keywords == NULL) goto failed; for (i = 0; i < len; i++) { keyword_ty val; @@ -8067,7 +8112,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* values; + asdl_expr_seq* values; if (_PyObject_LookupAttr(obj, state->values, &tmp) < 0) { return 1; @@ -8085,7 +8130,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - values = _Py_asdl_seq_new(len, arena); + values = _Py_asdl_expr_seq_new(len, arena); if (values == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -8341,7 +8386,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* elts; + asdl_expr_seq* elts; expr_context_ty ctx; if (_PyObject_LookupAttr(obj, state->elts, &tmp) < 0) { @@ -8360,7 +8405,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - elts = _Py_asdl_seq_new(len, arena); + elts = _Py_asdl_expr_seq_new(len, arena); if (elts == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -8401,7 +8446,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) return 1; } if (isinstance) { - asdl_seq* elts; + asdl_expr_seq* elts; expr_context_ty ctx; if (_PyObject_LookupAttr(obj, state->elts, &tmp) < 0) { @@ -8420,7 +8465,7 @@ obj2ast_expr(astmodulestate *state, PyObject* obj, expr_ty* out, PyArena* arena) goto failed; } len = PyList_GET_SIZE(tmp); - elts = _Py_asdl_seq_new(len, arena); + elts = _Py_asdl_expr_seq_new(len, arena); if (elts == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -8834,7 +8879,7 @@ obj2ast_comprehension(astmodulestate *state, PyObject* obj, comprehension_ty* PyObject* tmp = NULL; expr_ty target; expr_ty iter; - asdl_seq* ifs; + asdl_expr_seq* ifs; int is_async; if (_PyObject_LookupAttr(obj, state->target, &tmp) < 0) { @@ -8879,7 +8924,7 @@ obj2ast_comprehension(astmodulestate *state, PyObject* obj, comprehension_ty* goto failed; } len = PyList_GET_SIZE(tmp); - ifs = _Py_asdl_seq_new(len, arena); + ifs = _Py_asdl_expr_seq_new(len, arena); if (ifs == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -8993,7 +9038,7 @@ obj2ast_excepthandler(astmodulestate *state, PyObject* obj, excepthandler_ty* if (isinstance) { expr_ty type; identifier name; - asdl_seq* body; + asdl_stmt_seq* body; if (_PyObject_LookupAttr(obj, state->type, &tmp) < 0) { return 1; @@ -9037,7 +9082,7 @@ obj2ast_excepthandler(astmodulestate *state, PyObject* obj, excepthandler_ty* goto failed; } len = PyList_GET_SIZE(tmp); - body = _Py_asdl_seq_new(len, arena); + body = _Py_asdl_stmt_seq_new(len, arena); if (body == NULL) goto failed; for (i = 0; i < len; i++) { stmt_ty val; @@ -9071,13 +9116,13 @@ obj2ast_arguments(astmodulestate *state, PyObject* obj, arguments_ty* out, PyArena* arena) { PyObject* tmp = NULL; - asdl_seq* posonlyargs; - asdl_seq* args; + asdl_arg_seq* posonlyargs; + asdl_arg_seq* args; arg_ty vararg; - asdl_seq* kwonlyargs; - asdl_seq* kw_defaults; + asdl_arg_seq* kwonlyargs; + asdl_expr_seq* kw_defaults; arg_ty kwarg; - asdl_seq* defaults; + asdl_expr_seq* defaults; if (_PyObject_LookupAttr(obj, state->posonlyargs, &tmp) < 0) { return 1; @@ -9095,7 +9140,7 @@ obj2ast_arguments(astmodulestate *state, PyObject* obj, arguments_ty* out, goto failed; } len = PyList_GET_SIZE(tmp); - posonlyargs = _Py_asdl_seq_new(len, arena); + posonlyargs = _Py_asdl_arg_seq_new(len, arena); if (posonlyargs == NULL) goto failed; for (i = 0; i < len; i++) { arg_ty val; @@ -9128,7 +9173,7 @@ obj2ast_arguments(astmodulestate *state, PyObject* obj, arguments_ty* out, goto failed; } len = PyList_GET_SIZE(tmp); - args = _Py_asdl_seq_new(len, arena); + args = _Py_asdl_arg_seq_new(len, arena); if (args == NULL) goto failed; for (i = 0; i < len; i++) { arg_ty val; @@ -9174,7 +9219,7 @@ obj2ast_arguments(astmodulestate *state, PyObject* obj, arguments_ty* out, goto failed; } len = PyList_GET_SIZE(tmp); - kwonlyargs = _Py_asdl_seq_new(len, arena); + kwonlyargs = _Py_asdl_arg_seq_new(len, arena); if (kwonlyargs == NULL) goto failed; for (i = 0; i < len; i++) { arg_ty val; @@ -9207,7 +9252,7 @@ obj2ast_arguments(astmodulestate *state, PyObject* obj, arguments_ty* out, goto failed; } len = PyList_GET_SIZE(tmp); - kw_defaults = _Py_asdl_seq_new(len, arena); + kw_defaults = _Py_asdl_expr_seq_new(len, arena); if (kw_defaults == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; @@ -9253,7 +9298,7 @@ obj2ast_arguments(astmodulestate *state, PyObject* obj, arguments_ty* out, goto failed; } len = PyList_GET_SIZE(tmp); - defaults = _Py_asdl_seq_new(len, arena); + defaults = _Py_asdl_expr_seq_new(len, arena); if (defaults == NULL) goto failed; for (i = 0; i < len; i++) { expr_ty val; diff --git a/Python/asdl.c b/Python/asdl.c index c21107811813a..4ff07e4377b18 100644 --- a/Python/asdl.c +++ b/Python/asdl.c @@ -1,64 +1,6 @@ #include "Python.h" #include "asdl.h" -asdl_seq * -_Py_asdl_seq_new(Py_ssize_t size, PyArena *arena) -{ - asdl_seq *seq = NULL; - size_t n; - - /* check size is sane */ - if (size < 0 || - (size && (((size_t)size - 1) > (SIZE_MAX / sizeof(void *))))) { - PyErr_NoMemory(); - return NULL; - } - n = (size ? (sizeof(void *) * (size - 1)) : 0); - - /* check if size can be added safely */ - if (n > SIZE_MAX - sizeof(asdl_seq)) { - PyErr_NoMemory(); - return NULL; - } - n += sizeof(asdl_seq); - - seq = (asdl_seq *)PyArena_Malloc(arena, n); - if (!seq) { - PyErr_NoMemory(); - return NULL; - } - memset(seq, 0, n); - seq->size = size; - return seq; -} - -asdl_int_seq * -_Py_asdl_int_seq_new(Py_ssize_t size, PyArena *arena) -{ - asdl_int_seq *seq = NULL; - size_t n; - - /* check size is sane */ - if (size < 0 || - (size && (((size_t)size - 1) > (SIZE_MAX / sizeof(void *))))) { - PyErr_NoMemory(); - return NULL; - } - n = (size ? (sizeof(void *) * (size - 1)) : 0); - - /* check if size can be added safely */ - if (n > SIZE_MAX - sizeof(asdl_seq)) { - PyErr_NoMemory(); - return NULL; - } - n += sizeof(asdl_seq); - - seq = (asdl_int_seq *)PyArena_Malloc(arena, n); - if (!seq) { - PyErr_NoMemory(); - return NULL; - } - memset(seq, 0, n); - seq->size = size; - return seq; -} +GENERATE_ASDL_SEQ_CONSTRUCTOR(generic, void*); +GENERATE_ASDL_SEQ_CONSTRUCTOR(identifier, PyObject*); +GENERATE_ASDL_SEQ_CONSTRUCTOR(int, int); diff --git a/Python/ast.c b/Python/ast.c index 7bf66e50aa14d..4b7bbd229c99b 100644 --- a/Python/ast.c +++ b/Python/ast.c @@ -14,9 +14,9 @@ #define MAXLEVEL 200 /* Max parentheses level */ -static int validate_stmts(asdl_seq *); -static int validate_exprs(asdl_seq *, expr_context_ty, int); -static int validate_nonempty_seq(asdl_seq *, const char *, const char *); +static int validate_stmts(asdl_stmt_seq *); +static int validate_exprs(asdl_expr_seq*, expr_context_ty, int); +static int _validate_nonempty_seq(asdl_seq *, const char *, const char *); static int validate_stmt(stmt_ty); static int validate_expr(expr_ty, expr_context_ty); @@ -40,7 +40,7 @@ validate_name(PyObject *name) } static int -validate_comprehension(asdl_seq *gens) +validate_comprehension(asdl_comprehension_seq *gens) { Py_ssize_t i; if (!asdl_seq_LEN(gens)) { @@ -58,7 +58,7 @@ validate_comprehension(asdl_seq *gens) } static int -validate_keywords(asdl_seq *keywords) +validate_keywords(asdl_keyword_seq *keywords) { Py_ssize_t i; for (i = 0; i < asdl_seq_LEN(keywords); i++) @@ -68,7 +68,7 @@ validate_keywords(asdl_seq *keywords) } static int -validate_args(asdl_seq *args) +validate_args(asdl_arg_seq *args) { Py_ssize_t i; for (i = 0; i < asdl_seq_LEN(args); i++) { @@ -324,23 +324,24 @@ validate_expr(expr_ty exp, expr_context_ty ctx) } static int -validate_nonempty_seq(asdl_seq *seq, const char *what, const char *owner) +_validate_nonempty_seq(asdl_seq *seq, const char *what, const char *owner) { if (asdl_seq_LEN(seq)) return 1; PyErr_Format(PyExc_ValueError, "empty %s on %s", what, owner); return 0; } +#define validate_nonempty_seq(seq, what, owner) _validate_nonempty_seq((asdl_seq*)seq, what, owner) static int -validate_assignlist(asdl_seq *targets, expr_context_ty ctx) +validate_assignlist(asdl_expr_seq *targets, expr_context_ty ctx) { return validate_nonempty_seq(targets, "targets", ctx == Del ? "Delete" : "Assign") && validate_exprs(targets, ctx, 0); } static int -validate_body(asdl_seq *body, const char *owner) +validate_body(asdl_stmt_seq *body, const char *owner) { return validate_nonempty_seq(body, "body", owner) && validate_stmts(body); } @@ -488,7 +489,7 @@ validate_stmt(stmt_ty stmt) } static int -validate_stmts(asdl_seq *seq) +validate_stmts(asdl_stmt_seq *seq) { Py_ssize_t i; for (i = 0; i < asdl_seq_LEN(seq); i++) { @@ -507,7 +508,7 @@ validate_stmts(asdl_seq *seq) } static int -validate_exprs(asdl_seq *exprs, expr_context_ty ctx, int null_ok) +validate_exprs(asdl_expr_seq *exprs, expr_context_ty ctx, int null_ok) { Py_ssize_t i; for (i = 0; i < asdl_seq_LEN(exprs); i++) { @@ -550,7 +551,7 @@ PyAST_Validate(mod_ty mod) } PyObject * -_PyAST_GetDocString(asdl_seq *body) +_PyAST_GetDocString(asdl_stmt_seq *body) { if (!asdl_seq_LEN(body)) { return NULL; diff --git a/Python/ast_opt.c b/Python/ast_opt.c index ff786d6f8d63e..5efaac4c8925a 100644 --- a/Python/ast_opt.c +++ b/Python/ast_opt.c @@ -271,7 +271,7 @@ fold_binop(expr_ty node, PyArena *arena, _PyASTOptimizeState *state) } static PyObject* -make_const_tuple(asdl_seq *elts) +make_const_tuple(asdl_expr_seq *elts) { for (int i = 0; i < asdl_seq_LEN(elts); i++) { expr_ty e = (expr_ty)asdl_seq_GET(elts, i); @@ -337,7 +337,7 @@ fold_iter(expr_ty arg, PyArena *arena, _PyASTOptimizeState *state) PyObject *newval; if (arg->kind == List_kind) { /* First change a list into tuple. */ - asdl_seq *elts = arg->v.List.elts; + asdl_expr_seq *elts = arg->v.List.elts; Py_ssize_t n = asdl_seq_LEN(elts); for (Py_ssize_t i = 0; i < n; i++) { expr_ty e = (expr_ty)asdl_seq_GET(elts, i); @@ -368,7 +368,7 @@ static int fold_compare(expr_ty node, PyArena *arena, _PyASTOptimizeState *state) { asdl_int_seq *ops; - asdl_seq *args; + asdl_expr_seq *args; Py_ssize_t i; ops = node->v.Compare.ops; @@ -405,9 +405,9 @@ static int astfold_excepthandler(excepthandler_ty node_, PyArena *ctx_, _PyASTOp #define CALL_SEQ(FUNC, TYPE, ARG) { \ int i; \ - asdl_seq *seq = (ARG); /* avoid variable capture */ \ + asdl_ ## TYPE ## _seq *seq = (ARG); /* avoid variable capture */ \ for (i = 0; i < asdl_seq_LEN(seq); i++) { \ - TYPE elt = (TYPE)asdl_seq_GET(seq, i); \ + TYPE ## _ty elt = (TYPE ## _ty)asdl_seq_GET(seq, i); \ if (elt != NULL && !FUNC(elt, ctx_, state)) \ return 0; \ } \ @@ -424,13 +424,13 @@ static int astfold_excepthandler(excepthandler_ty node_, PyArena *ctx_, _PyASTOp } static int -astfold_body(asdl_seq *stmts, PyArena *ctx_, _PyASTOptimizeState *state) +astfold_body(asdl_stmt_seq *stmts, PyArena *ctx_, _PyASTOptimizeState *state) { int docstring = _PyAST_GetDocString(stmts) != NULL; - CALL_SEQ(astfold_stmt, stmt_ty, stmts); + CALL_SEQ(astfold_stmt, stmt, stmts); if (!docstring && _PyAST_GetDocString(stmts) != NULL) { stmt_ty st = (stmt_ty)asdl_seq_GET(stmts, 0); - asdl_seq *values = _Py_asdl_seq_new(1, ctx_); + asdl_expr_seq *values = _Py_asdl_expr_seq_new(1, ctx_); if (!values) { return 0; } @@ -453,7 +453,7 @@ astfold_mod(mod_ty node_, PyArena *ctx_, _PyASTOptimizeState *state) CALL(astfold_body, asdl_seq, node_->v.Module.body); break; case Interactive_kind: - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.Interactive.body); + CALL_SEQ(astfold_stmt, stmt, node_->v.Interactive.body); break; case Expression_kind: CALL(astfold_expr, expr_ty, node_->v.Expression.body); @@ -469,7 +469,7 @@ astfold_expr(expr_ty node_, PyArena *ctx_, _PyASTOptimizeState *state) { switch (node_->kind) { case BoolOp_kind: - CALL_SEQ(astfold_expr, expr_ty, node_->v.BoolOp.values); + CALL_SEQ(astfold_expr, expr, node_->v.BoolOp.values); break; case BinOp_kind: CALL(astfold_expr, expr_ty, node_->v.BinOp.left); @@ -490,28 +490,28 @@ astfold_expr(expr_ty node_, PyArena *ctx_, _PyASTOptimizeState *state) CALL(astfold_expr, expr_ty, node_->v.IfExp.orelse); break; case Dict_kind: - CALL_SEQ(astfold_expr, expr_ty, node_->v.Dict.keys); - CALL_SEQ(astfold_expr, expr_ty, node_->v.Dict.values); + CALL_SEQ(astfold_expr, expr, node_->v.Dict.keys); + CALL_SEQ(astfold_expr, expr, node_->v.Dict.values); break; case Set_kind: - CALL_SEQ(astfold_expr, expr_ty, node_->v.Set.elts); + CALL_SEQ(astfold_expr, expr, node_->v.Set.elts); break; case ListComp_kind: CALL(astfold_expr, expr_ty, node_->v.ListComp.elt); - CALL_SEQ(astfold_comprehension, comprehension_ty, node_->v.ListComp.generators); + CALL_SEQ(astfold_comprehension, comprehension, node_->v.ListComp.generators); break; case SetComp_kind: CALL(astfold_expr, expr_ty, node_->v.SetComp.elt); - CALL_SEQ(astfold_comprehension, comprehension_ty, node_->v.SetComp.generators); + CALL_SEQ(astfold_comprehension, comprehension, node_->v.SetComp.generators); break; case DictComp_kind: CALL(astfold_expr, expr_ty, node_->v.DictComp.key); CALL(astfold_expr, expr_ty, node_->v.DictComp.value); - CALL_SEQ(astfold_comprehension, comprehension_ty, node_->v.DictComp.generators); + CALL_SEQ(astfold_comprehension, comprehension, node_->v.DictComp.generators); break; case GeneratorExp_kind: CALL(astfold_expr, expr_ty, node_->v.GeneratorExp.elt); - CALL_SEQ(astfold_comprehension, comprehension_ty, node_->v.GeneratorExp.generators); + CALL_SEQ(astfold_comprehension, comprehension, node_->v.GeneratorExp.generators); break; case Await_kind: CALL(astfold_expr, expr_ty, node_->v.Await.value); @@ -524,20 +524,20 @@ astfold_expr(expr_ty node_, PyArena *ctx_, _PyASTOptimizeState *state) break; case Compare_kind: CALL(astfold_expr, expr_ty, node_->v.Compare.left); - CALL_SEQ(astfold_expr, expr_ty, node_->v.Compare.comparators); + CALL_SEQ(astfold_expr, expr, node_->v.Compare.comparators); CALL(fold_compare, expr_ty, node_); break; case Call_kind: CALL(astfold_expr, expr_ty, node_->v.Call.func); - CALL_SEQ(astfold_expr, expr_ty, node_->v.Call.args); - CALL_SEQ(astfold_keyword, keyword_ty, node_->v.Call.keywords); + CALL_SEQ(astfold_expr, expr, node_->v.Call.args); + CALL_SEQ(astfold_keyword, keyword, node_->v.Call.keywords); break; case FormattedValue_kind: CALL(astfold_expr, expr_ty, node_->v.FormattedValue.value); CALL_OPT(astfold_expr, expr_ty, node_->v.FormattedValue.format_spec); break; case JoinedStr_kind: - CALL_SEQ(astfold_expr, expr_ty, node_->v.JoinedStr.values); + CALL_SEQ(astfold_expr, expr, node_->v.JoinedStr.values); break; case Attribute_kind: CALL(astfold_expr, expr_ty, node_->v.Attribute.value); @@ -556,10 +556,10 @@ astfold_expr(expr_ty node_, PyArena *ctx_, _PyASTOptimizeState *state) CALL_OPT(astfold_expr, expr_ty, node_->v.Slice.step); break; case List_kind: - CALL_SEQ(astfold_expr, expr_ty, node_->v.List.elts); + CALL_SEQ(astfold_expr, expr, node_->v.List.elts); break; case Tuple_kind: - CALL_SEQ(astfold_expr, expr_ty, node_->v.Tuple.elts); + CALL_SEQ(astfold_expr, expr, node_->v.Tuple.elts); CALL(fold_tuple, expr_ty, node_); break; case Name_kind: @@ -586,7 +586,7 @@ astfold_comprehension(comprehension_ty node_, PyArena *ctx_, _PyASTOptimizeState { CALL(astfold_expr, expr_ty, node_->target); CALL(astfold_expr, expr_ty, node_->iter); - CALL_SEQ(astfold_expr, expr_ty, node_->ifs); + CALL_SEQ(astfold_expr, expr, node_->ifs); CALL(fold_iter, expr_ty, node_->iter); return 1; @@ -595,13 +595,13 @@ astfold_comprehension(comprehension_ty node_, PyArena *ctx_, _PyASTOptimizeState static int astfold_arguments(arguments_ty node_, PyArena *ctx_, _PyASTOptimizeState *state) { - CALL_SEQ(astfold_arg, arg_ty, node_->posonlyargs); - CALL_SEQ(astfold_arg, arg_ty, node_->args); + CALL_SEQ(astfold_arg, arg, node_->posonlyargs); + CALL_SEQ(astfold_arg, arg, node_->args); CALL_OPT(astfold_arg, arg_ty, node_->vararg); - CALL_SEQ(astfold_arg, arg_ty, node_->kwonlyargs); - CALL_SEQ(astfold_expr, expr_ty, node_->kw_defaults); + CALL_SEQ(astfold_arg, arg, node_->kwonlyargs); + CALL_SEQ(astfold_expr, expr, node_->kw_defaults); CALL_OPT(astfold_arg, arg_ty, node_->kwarg); - CALL_SEQ(astfold_expr, expr_ty, node_->defaults); + CALL_SEQ(astfold_expr, expr, node_->defaults); return 1; } @@ -621,7 +621,7 @@ astfold_stmt(stmt_ty node_, PyArena *ctx_, _PyASTOptimizeState *state) case FunctionDef_kind: CALL(astfold_arguments, arguments_ty, node_->v.FunctionDef.args); CALL(astfold_body, asdl_seq, node_->v.FunctionDef.body); - CALL_SEQ(astfold_expr, expr_ty, node_->v.FunctionDef.decorator_list); + CALL_SEQ(astfold_expr, expr, node_->v.FunctionDef.decorator_list); if (!(state->ff_features & CO_FUTURE_ANNOTATIONS)) { CALL_OPT(astfold_expr, expr_ty, node_->v.FunctionDef.returns); } @@ -629,25 +629,25 @@ astfold_stmt(stmt_ty node_, PyArena *ctx_, _PyASTOptimizeState *state) case AsyncFunctionDef_kind: CALL(astfold_arguments, arguments_ty, node_->v.AsyncFunctionDef.args); CALL(astfold_body, asdl_seq, node_->v.AsyncFunctionDef.body); - CALL_SEQ(astfold_expr, expr_ty, node_->v.AsyncFunctionDef.decorator_list); + CALL_SEQ(astfold_expr, expr, node_->v.AsyncFunctionDef.decorator_list); if (!(state->ff_features & CO_FUTURE_ANNOTATIONS)) { CALL_OPT(astfold_expr, expr_ty, node_->v.AsyncFunctionDef.returns); } break; case ClassDef_kind: - CALL_SEQ(astfold_expr, expr_ty, node_->v.ClassDef.bases); - CALL_SEQ(astfold_keyword, keyword_ty, node_->v.ClassDef.keywords); + CALL_SEQ(astfold_expr, expr, node_->v.ClassDef.bases); + CALL_SEQ(astfold_keyword, keyword, node_->v.ClassDef.keywords); CALL(astfold_body, asdl_seq, node_->v.ClassDef.body); - CALL_SEQ(astfold_expr, expr_ty, node_->v.ClassDef.decorator_list); + CALL_SEQ(astfold_expr, expr, node_->v.ClassDef.decorator_list); break; case Return_kind: CALL_OPT(astfold_expr, expr_ty, node_->v.Return.value); break; case Delete_kind: - CALL_SEQ(astfold_expr, expr_ty, node_->v.Delete.targets); + CALL_SEQ(astfold_expr, expr, node_->v.Delete.targets); break; case Assign_kind: - CALL_SEQ(astfold_expr, expr_ty, node_->v.Assign.targets); + CALL_SEQ(astfold_expr, expr, node_->v.Assign.targets); CALL(astfold_expr, expr_ty, node_->v.Assign.value); break; case AugAssign_kind: @@ -664,44 +664,44 @@ astfold_stmt(stmt_ty node_, PyArena *ctx_, _PyASTOptimizeState *state) case For_kind: CALL(astfold_expr, expr_ty, node_->v.For.target); CALL(astfold_expr, expr_ty, node_->v.For.iter); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.For.body); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.For.orelse); + CALL_SEQ(astfold_stmt, stmt, node_->v.For.body); + CALL_SEQ(astfold_stmt, stmt, node_->v.For.orelse); CALL(fold_iter, expr_ty, node_->v.For.iter); break; case AsyncFor_kind: CALL(astfold_expr, expr_ty, node_->v.AsyncFor.target); CALL(astfold_expr, expr_ty, node_->v.AsyncFor.iter); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.AsyncFor.body); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.AsyncFor.orelse); + CALL_SEQ(astfold_stmt, stmt, node_->v.AsyncFor.body); + CALL_SEQ(astfold_stmt, stmt, node_->v.AsyncFor.orelse); break; case While_kind: CALL(astfold_expr, expr_ty, node_->v.While.test); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.While.body); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.While.orelse); + CALL_SEQ(astfold_stmt, stmt, node_->v.While.body); + CALL_SEQ(astfold_stmt, stmt, node_->v.While.orelse); break; case If_kind: CALL(astfold_expr, expr_ty, node_->v.If.test); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.If.body); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.If.orelse); + CALL_SEQ(astfold_stmt, stmt, node_->v.If.body); + CALL_SEQ(astfold_stmt, stmt, node_->v.If.orelse); break; case With_kind: - CALL_SEQ(astfold_withitem, withitem_ty, node_->v.With.items); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.With.body); + CALL_SEQ(astfold_withitem, withitem, node_->v.With.items); + CALL_SEQ(astfold_stmt, stmt, node_->v.With.body); break; case AsyncWith_kind: - CALL_SEQ(astfold_withitem, withitem_ty, node_->v.AsyncWith.items); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.AsyncWith.body); + CALL_SEQ(astfold_withitem, withitem, node_->v.AsyncWith.items); + CALL_SEQ(astfold_stmt, stmt, node_->v.AsyncWith.body); break; case Raise_kind: CALL_OPT(astfold_expr, expr_ty, node_->v.Raise.exc); CALL_OPT(astfold_expr, expr_ty, node_->v.Raise.cause); break; case Try_kind: - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.Try.body); - CALL_SEQ(astfold_excepthandler, excepthandler_ty, node_->v.Try.handlers); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.Try.orelse); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.Try.finalbody); + CALL_SEQ(astfold_stmt, stmt, node_->v.Try.body); + CALL_SEQ(astfold_excepthandler, excepthandler, node_->v.Try.handlers); + CALL_SEQ(astfold_stmt, stmt, node_->v.Try.orelse); + CALL_SEQ(astfold_stmt, stmt, node_->v.Try.finalbody); break; case Assert_kind: CALL(astfold_expr, expr_ty, node_->v.Assert.test); @@ -722,7 +722,7 @@ astfold_excepthandler(excepthandler_ty node_, PyArena *ctx_, _PyASTOptimizeState switch (node_->kind) { case ExceptHandler_kind: CALL_OPT(astfold_expr, expr_ty, node_->v.ExceptHandler.type); - CALL_SEQ(astfold_stmt, stmt_ty, node_->v.ExceptHandler.body); + CALL_SEQ(astfold_stmt, stmt, node_->v.ExceptHandler.body); break; default: break; diff --git a/Python/ast_unparse.c b/Python/ast_unparse.c index e699751a05a05..a04ff93e9d9d9 100644 --- a/Python/ast_unparse.c +++ b/Python/ast_unparse.c @@ -117,7 +117,7 @@ static int append_ast_boolop(_PyUnicodeWriter *writer, expr_ty e, int level) { Py_ssize_t i, value_count; - asdl_seq *values; + asdl_expr_seq *values; const char *op = (e->v.BoolOp.op == And) ? " and " : " or "; int pr = (e->v.BoolOp.op == And) ? PR_AND : PR_OR; @@ -398,7 +398,7 @@ append_ast_comprehension(_PyUnicodeWriter *writer, comprehension_ty gen) } static int -append_ast_comprehensions(_PyUnicodeWriter *writer, asdl_seq *comprehensions) +append_ast_comprehensions(_PyUnicodeWriter *writer, asdl_comprehension_seq *comprehensions) { Py_ssize_t i, gen_count; gen_count = asdl_seq_LEN(comprehensions); @@ -453,7 +453,7 @@ append_ast_compare(_PyUnicodeWriter *writer, expr_ty e, int level) { const char *op; Py_ssize_t i, comparator_count; - asdl_seq *comparators; + asdl_expr_seq *comparators; asdl_int_seq *ops; APPEND_STR_IF(level > PR_CMP, "("); @@ -612,7 +612,7 @@ append_fstring_element(_PyUnicodeWriter *writer, expr_ty e, bool is_format_spec) /* Build body separately to enable wrapping the entire stream of Strs, Constants and FormattedValues in one opening and one closing quote. */ static PyObject * -build_fstring_body(asdl_seq *values, bool is_format_spec) +build_fstring_body(asdl_expr_seq *values, bool is_format_spec) { Py_ssize_t i, value_count; _PyUnicodeWriter body_writer; diff --git a/Python/compile.c b/Python/compile.c index 2c5326686f866..3ebf221cf02b7 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -222,27 +222,27 @@ static int compiler_subscript(struct compiler *, expr_ty); static int compiler_slice(struct compiler *, expr_ty); static int inplace_binop(operator_ty); -static int are_all_items_const(asdl_seq *, Py_ssize_t, Py_ssize_t); +static int are_all_items_const(asdl_expr_seq *, Py_ssize_t, Py_ssize_t); static int expr_constant(expr_ty); static int compiler_with(struct compiler *, stmt_ty, int); static int compiler_async_with(struct compiler *, stmt_ty, int); static int compiler_async_for(struct compiler *, stmt_ty); static int compiler_call_helper(struct compiler *c, int n, - asdl_seq *args, - asdl_seq *keywords); + asdl_expr_seq *args, + asdl_keyword_seq *keywords); static int compiler_try_except(struct compiler *, stmt_ty); static int compiler_set_qualname(struct compiler *); static int compiler_sync_comprehension_generator( struct compiler *c, - asdl_seq *generators, int gen_index, + asdl_comprehension_seq *generators, int gen_index, int depth, expr_ty elt, expr_ty val, int type); static int compiler_async_comprehension_generator( struct compiler *c, - asdl_seq *generators, int gen_index, + asdl_comprehension_seq *generators, int gen_index, int depth, expr_ty elt, expr_ty val, int type); @@ -1525,7 +1525,7 @@ compiler_addop_j(struct compiler *c, int opcode, basicblock *b) #define VISIT_SEQ(C, TYPE, SEQ) { \ int _i; \ - asdl_seq *seq = (SEQ); /* avoid variable capture */ \ + asdl_ ## TYPE ## _seq *seq = (SEQ); /* avoid variable capture */ \ for (_i = 0; _i < asdl_seq_LEN(seq); _i++) { \ TYPE ## _ty elt = (TYPE ## _ty)asdl_seq_GET(seq, _i); \ if (!compiler_visit_ ## TYPE((C), elt)) \ @@ -1535,7 +1535,7 @@ compiler_addop_j(struct compiler *c, int opcode, basicblock *b) #define VISIT_SEQ_IN_SCOPE(C, TYPE, SEQ) { \ int _i; \ - asdl_seq *seq = (SEQ); /* avoid variable capture */ \ + asdl_ ## TYPE ## _seq *seq = (SEQ); /* avoid variable capture */ \ for (_i = 0; _i < asdl_seq_LEN(seq); _i++) { \ TYPE ## _ty elt = (TYPE ## _ty)asdl_seq_GET(seq, _i); \ if (!compiler_visit_ ## TYPE((C), elt)) { \ @@ -1559,7 +1559,7 @@ compiler_addop_j(struct compiler *c, int opcode, basicblock *b) /* Search if variable annotations are present statically in a block. */ static int -find_ann(asdl_seq *stmts) +find_ann(asdl_stmt_seq *stmts) { int i, j, res = 0; stmt_ty st; @@ -1778,7 +1778,7 @@ compiler_unwind_fblock_stack(struct compiler *c, int preserve_tos, struct fblock and for annotations. */ static int -compiler_body(struct compiler *c, asdl_seq *stmts) +compiler_body(struct compiler *c, asdl_stmt_seq *stmts) { int i = 0; stmt_ty st; @@ -1841,8 +1841,7 @@ compiler_mod(struct compiler *c, mod_ty mod) ADDOP(c, SETUP_ANNOTATIONS); } c->c_interactive = 1; - VISIT_SEQ_IN_SCOPE(c, stmt, - mod->v.Interactive.body); + VISIT_SEQ_IN_SCOPE(c, stmt, mod->v.Interactive.body); break; case Expression_kind: VISIT_IN_SCOPE(c, expr, mod->v.Expression.body); @@ -1945,7 +1944,7 @@ compiler_make_closure(struct compiler *c, PyCodeObject *co, Py_ssize_t flags, Py } static int -compiler_decorators(struct compiler *c, asdl_seq* decos) +compiler_decorators(struct compiler *c, asdl_expr_seq* decos) { int i; @@ -1959,8 +1958,8 @@ compiler_decorators(struct compiler *c, asdl_seq* decos) } static int -compiler_visit_kwonlydefaults(struct compiler *c, asdl_seq *kwonlyargs, - asdl_seq *kw_defaults) +compiler_visit_kwonlydefaults(struct compiler *c, asdl_arg_seq *kwonlyargs, + asdl_expr_seq *kw_defaults) { /* Push a dict of keyword-only default values. @@ -2047,7 +2046,7 @@ compiler_visit_argannotation(struct compiler *c, identifier id, } static int -compiler_visit_argannotations(struct compiler *c, asdl_seq* args, +compiler_visit_argannotations(struct compiler *c, asdl_arg_seq* args, PyObject *names) { int i; @@ -2173,7 +2172,7 @@ compiler_check_debug_one_arg(struct compiler *c, arg_ty arg) } static int -compiler_check_debug_args_seq(struct compiler *c, asdl_seq *args) +compiler_check_debug_args_seq(struct compiler *c, asdl_arg_seq *args) { if (args != NULL) { for (Py_ssize_t i = 0, n = asdl_seq_LEN(args); i < n; i++) { @@ -2208,8 +2207,8 @@ compiler_function(struct compiler *c, stmt_ty s, int is_async) arguments_ty args; expr_ty returns; identifier name; - asdl_seq* decos; - asdl_seq *body; + asdl_expr_seq* decos; + asdl_stmt_seq *body; Py_ssize_t i, funcflags; int annotations; int scope_type; @@ -2306,7 +2305,7 @@ compiler_class(struct compiler *c, stmt_ty s) PyCodeObject *co; PyObject *str; int i, firstlineno; - asdl_seq* decos = s->v.ClassDef.decorator_list; + asdl_expr_seq *decos = s->v.ClassDef.decorator_list; if (!compiler_decorators(c, decos)) return 0; @@ -2418,9 +2417,7 @@ compiler_class(struct compiler *c, stmt_ty s) ADDOP_LOAD_CONST(c, s->v.ClassDef.name); /* 5. generate the rest of the code for the call */ - if (!compiler_call_helper(c, 2, - s->v.ClassDef.bases, - s->v.ClassDef.keywords)) + if (!compiler_call_helper(c, 2, s->v.ClassDef.bases, s->v.ClassDef.keywords)) return 0; /* 6. apply decorators */ @@ -2528,7 +2525,7 @@ compiler_jump_if(struct compiler *c, expr_ty e, basicblock *next, int cond) /* fallback to general implementation */ break; case BoolOp_kind: { - asdl_seq *s = e->v.BoolOp.values; + asdl_expr_seq *s = e->v.BoolOp.values; Py_ssize_t i, n = asdl_seq_LEN(s) - 1; assert(n >= 0); int cond2 = e->v.BoolOp.op == Or; @@ -3645,7 +3642,7 @@ compiler_boolop(struct compiler *c, expr_ty e) basicblock *end; int jumpi; Py_ssize_t i, n; - asdl_seq *s; + asdl_expr_seq *s; assert(e->kind == BoolOp_kind); if (e->v.BoolOp.op == And) @@ -3673,7 +3670,7 @@ compiler_boolop(struct compiler *c, expr_ty e) } static int -starunpack_helper(struct compiler *c, asdl_seq *elts, int pushed, +starunpack_helper(struct compiler *c, asdl_expr_seq *elts, int pushed, int build, int add, int extend, int tuple) { Py_ssize_t n = asdl_seq_LEN(elts); @@ -3750,7 +3747,7 @@ starunpack_helper(struct compiler *c, asdl_seq *elts, int pushed, } static int -assignment_helper(struct compiler *c, asdl_seq *elts) +assignment_helper(struct compiler *c, asdl_expr_seq *elts) { Py_ssize_t n = asdl_seq_LEN(elts); Py_ssize_t i; @@ -3784,7 +3781,7 @@ assignment_helper(struct compiler *c, asdl_seq *elts) static int compiler_list(struct compiler *c, expr_ty e) { - asdl_seq *elts = e->v.List.elts; + asdl_expr_seq *elts = e->v.List.elts; if (e->v.List.ctx == Store) { return assignment_helper(c, elts); } @@ -3800,7 +3797,7 @@ compiler_list(struct compiler *c, expr_ty e) static int compiler_tuple(struct compiler *c, expr_ty e) { - asdl_seq *elts = e->v.Tuple.elts; + asdl_expr_seq *elts = e->v.Tuple.elts; if (e->v.Tuple.ctx == Store) { return assignment_helper(c, elts); } @@ -3821,7 +3818,7 @@ compiler_set(struct compiler *c, expr_ty e) } static int -are_all_items_const(asdl_seq *seq, Py_ssize_t begin, Py_ssize_t end) +are_all_items_const(asdl_expr_seq *seq, Py_ssize_t begin, Py_ssize_t end) { Py_ssize_t i; for (i = begin; i < end; i++) { @@ -4084,7 +4081,7 @@ maybe_optimize_method_call(struct compiler *c, expr_ty e) { Py_ssize_t argsl, i; expr_ty meth = e->v.Call.func; - asdl_seq *args = e->v.Call.args; + asdl_expr_seq *args = e->v.Call.args; /* Check that the call node is an attribute access, and that the call doesn't have keyword parameters. */ @@ -4110,7 +4107,7 @@ maybe_optimize_method_call(struct compiler *c, expr_ty e) } static int -validate_keywords(struct compiler *c, asdl_seq *keywords) +validate_keywords(struct compiler *c, asdl_keyword_seq *keywords) { Py_ssize_t nkeywords = asdl_seq_LEN(keywords); for (Py_ssize_t i = 0; i < nkeywords; i++) { @@ -4210,7 +4207,7 @@ compiler_formatted_value(struct compiler *c, expr_ty e) } static int -compiler_subkwargs(struct compiler *c, asdl_seq *keywords, Py_ssize_t begin, Py_ssize_t end) +compiler_subkwargs(struct compiler *c, asdl_keyword_seq *keywords, Py_ssize_t begin, Py_ssize_t end) { Py_ssize_t i, n = end - begin; keyword_ty kw; @@ -4249,8 +4246,8 @@ compiler_subkwargs(struct compiler *c, asdl_seq *keywords, Py_ssize_t begin, Py_ static int compiler_call_helper(struct compiler *c, int n, /* Args already pushed */ - asdl_seq *args, - asdl_seq *keywords) + asdl_expr_seq *args, + asdl_keyword_seq *keywords) { Py_ssize_t i, nseen, nelts, nkwelts; @@ -4375,7 +4372,7 @@ compiler_call_helper(struct compiler *c, static int compiler_comprehension_generator(struct compiler *c, - asdl_seq *generators, int gen_index, + asdl_comprehension_seq *generators, int gen_index, int depth, expr_ty elt, expr_ty val, int type) { @@ -4392,7 +4389,7 @@ compiler_comprehension_generator(struct compiler *c, static int compiler_sync_comprehension_generator(struct compiler *c, - asdl_seq *generators, int gen_index, + asdl_comprehension_seq *generators, int gen_index, int depth, expr_ty elt, expr_ty val, int type) { @@ -4424,7 +4421,7 @@ compiler_sync_comprehension_generator(struct compiler *c, /* Fast path for the temporary variable assignment idiom: for y in [f(x)] */ - asdl_seq *elts; + asdl_expr_seq *elts; switch (gen->iter->kind) { case List_kind: elts = gen->iter->v.List.elts; @@ -4511,7 +4508,7 @@ compiler_sync_comprehension_generator(struct compiler *c, static int compiler_async_comprehension_generator(struct compiler *c, - asdl_seq *generators, int gen_index, + asdl_comprehension_seq *generators, int gen_index, int depth, expr_ty elt, expr_ty val, int type) { @@ -4602,7 +4599,7 @@ compiler_async_comprehension_generator(struct compiler *c, static int compiler_comprehension(struct compiler *c, expr_ty e, int type, - identifier name, asdl_seq *generators, expr_ty elt, + identifier name, asdl_comprehension_seq *generators, expr_ty elt, expr_ty val) { PyCodeObject *co = NULL; @@ -5226,7 +5223,7 @@ check_ann_subscr(struct compiler *c, expr_ty e) return 1; case Tuple_kind: { /* extended slice */ - asdl_seq *elts = e->v.Tuple.elts; + asdl_expr_seq *elts = e->v.Tuple.elts; Py_ssize_t i, n = asdl_seq_LEN(elts); for (i = 0; i < n; i++) { if (!check_ann_subscr(c, asdl_seq_GET(elts, i))) { diff --git a/Python/future.c b/Python/future.c index 56da4d8c798b8..3cea4fee78085 100644 --- a/Python/future.c +++ b/Python/future.c @@ -13,11 +13,10 @@ static int future_check_features(PyFutureFeatures *ff, stmt_ty s, PyObject *filename) { int i; - asdl_seq *names; assert(s->kind == ImportFrom_kind); - names = s->v.ImportFrom.names; + asdl_alias_seq *names = s->v.ImportFrom.names; for (i = 0; i < asdl_seq_LEN(names); i++) { alias_ty name = (alias_ty)asdl_seq_GET(names, i); const char *feature = PyUnicode_AsUTF8(name->name); diff --git a/Python/symtable.c b/Python/symtable.c index d192f31deefb7..4a98e79e74a25 100644 --- a/Python/symtable.c +++ b/Python/symtable.c @@ -202,8 +202,8 @@ static int symtable_visit_excepthandler(struct symtable *st, excepthandler_ty); static int symtable_visit_alias(struct symtable *st, alias_ty); static int symtable_visit_comprehension(struct symtable *st, comprehension_ty); static int symtable_visit_keyword(struct symtable *st, keyword_ty); -static int symtable_visit_params(struct symtable *st, asdl_seq *args); -static int symtable_visit_argannotations(struct symtable *st, asdl_seq *args); +static int symtable_visit_params(struct symtable *st, asdl_arg_seq *args); +static int symtable_visit_argannotations(struct symtable *st, asdl_arg_seq *args); static int symtable_implicit_arg(struct symtable *st, int pos); static int symtable_visit_annotations(struct symtable *st, arguments_ty, expr_ty); static int symtable_visit_withitem(struct symtable *st, withitem_ty item); @@ -261,7 +261,7 @@ struct symtable * PySymtable_BuildObject(mod_ty mod, PyObject *filename, PyFutureFeatures *future) { struct symtable *st = symtable_new(); - asdl_seq *seq; + asdl_stmt_seq *seq; int i; PyThreadState *tstate; int recursion_limit = Py_GetRecursionLimit(); @@ -1116,7 +1116,7 @@ symtable_add_def(struct symtable *st, PyObject *name, int flag) { #define VISIT_SEQ(ST, TYPE, SEQ) { \ int i; \ - asdl_seq *seq = (SEQ); /* avoid variable capture */ \ + asdl_ ## TYPE ## _seq *seq = (SEQ); /* avoid variable capture */ \ for (i = 0; i < asdl_seq_LEN(seq); i++) { \ TYPE ## _ty elt = (TYPE ## _ty)asdl_seq_GET(seq, i); \ if (!symtable_visit_ ## TYPE((ST), elt)) \ @@ -1126,7 +1126,7 @@ symtable_add_def(struct symtable *st, PyObject *name, int flag) { #define VISIT_SEQ_TAIL(ST, TYPE, SEQ, START) { \ int i; \ - asdl_seq *seq = (SEQ); /* avoid variable capture */ \ + asdl_ ## TYPE ## _seq *seq = (SEQ); /* avoid variable capture */ \ for (i = (START); i < asdl_seq_LEN(seq); i++) { \ TYPE ## _ty elt = (TYPE ## _ty)asdl_seq_GET(seq, i); \ if (!symtable_visit_ ## TYPE((ST), elt)) \ @@ -1136,7 +1136,7 @@ symtable_add_def(struct symtable *st, PyObject *name, int flag) { #define VISIT_SEQ_WITH_NULL(ST, TYPE, SEQ) { \ int i = 0; \ - asdl_seq *seq = (SEQ); /* avoid variable capture */ \ + asdl_ ## TYPE ## _seq *seq = (SEQ); /* avoid variable capture */ \ for (i = 0; i < asdl_seq_LEN(seq); i++) { \ TYPE ## _ty elt = (TYPE ## _ty)asdl_seq_GET(seq, i); \ if (!elt) continue; /* can be NULL */ \ @@ -1318,7 +1318,7 @@ symtable_visit_stmt(struct symtable *st, stmt_ty s) break; case Global_kind: { int i; - asdl_seq *seq = s->v.Global.names; + asdl_identifier_seq *seq = s->v.Global.names; for (i = 0; i < asdl_seq_LEN(seq); i++) { identifier name = (identifier)asdl_seq_GET(seq, i); long cur = symtable_lookup(st, name); @@ -1351,7 +1351,7 @@ symtable_visit_stmt(struct symtable *st, stmt_ty s) } case Nonlocal_kind: { int i; - asdl_seq *seq = s->v.Nonlocal.names; + asdl_identifier_seq *seq = s->v.Nonlocal.names; for (i = 0; i < asdl_seq_LEN(seq); i++) { identifier name = (identifier)asdl_seq_GET(seq, i); long cur = symtable_lookup(st, name); @@ -1683,7 +1683,7 @@ symtable_implicit_arg(struct symtable *st, int pos) } static int -symtable_visit_params(struct symtable *st, asdl_seq *args) +symtable_visit_params(struct symtable *st, asdl_arg_seq *args) { int i; @@ -1700,7 +1700,7 @@ symtable_visit_params(struct symtable *st, asdl_seq *args) } static int -symtable_visit_argannotations(struct symtable *st, asdl_seq *args) +symtable_visit_argannotations(struct symtable *st, asdl_arg_seq *args) { int i; @@ -1850,7 +1850,7 @@ symtable_visit_keyword(struct symtable *st, keyword_ty k) static int symtable_handle_comprehension(struct symtable *st, expr_ty e, - identifier scope_name, asdl_seq *generators, + identifier scope_name, asdl_comprehension_seq *generators, expr_ty elt, expr_ty value) { int is_generator = (e->kind == GeneratorExp_kind); diff --git a/Tools/peg_generator/pegen/c_generator.py b/Tools/peg_generator/pegen/c_generator.py index aee668c3f329a..1a814aad11ccc 100644 --- a/Tools/peg_generator/pegen/c_generator.py +++ b/Tools/peg_generator/pegen/c_generator.py @@ -74,6 +74,7 @@ class FunctionCall: function: str arguments: List[Any] = field(default_factory=list) assigned_variable: Optional[str] = None + assigned_variable_type: Optional[str] = None return_type: Optional[str] = None nodetype: Optional[NodeTypes] = None force_true: bool = False @@ -87,7 +88,10 @@ def __str__(self) -> str: if self.force_true: parts.append(", 1") if self.assigned_variable: - parts = ["(", self.assigned_variable, " = ", *parts, ")"] + if self.assigned_variable_type: + parts = ["(", self.assigned_variable, " = ", '(', self.assigned_variable_type, ')', *parts, ")"] + else: + parts = ["(", self.assigned_variable, " = ", *parts, ")"] if self.comment: parts.append(f" // {self.comment}") return "".join(parts) @@ -210,6 +214,8 @@ def visit_NamedItem(self, node: NamedItem) -> FunctionCall: call = self.generate_call(node.item) if node.name: call.assigned_variable = node.name + if node.type: + call.assigned_variable_type = node.type return call def lookahead_call_helper(self, node: Lookahead, positive: int) -> FunctionCall: @@ -568,9 +574,9 @@ def _handle_loop_rule_body(self, node: Rule, rhs: Rhs) -> None: self.print("PyMem_Free(_children);") self.add_return("NULL") self.print("}") - self.print("asdl_seq *_seq = _Py_asdl_seq_new(_n, p->arena);") + self.print("asdl_seq *_seq = (asdl_seq*)_Py_asdl_generic_seq_new(_n, p->arena);") self.out_of_memory_return(f"!_seq", cleanup_code="PyMem_Free(_children);") - self.print("for (int i = 0; i < _n; i++) asdl_seq_SET(_seq, i, _children[i]);") + self.print("for (int i = 0; i < _n; i++) asdl_seq_SET_UNTYPED(_seq, i, _children[i]);") self.print("PyMem_Free(_children);") if node.name: self.print(f"_PyPegen_insert_memo(p, _start_mark, {node.name}_type, _seq);") @@ -782,4 +788,5 @@ def add_var(self, node: NamedItem) -> Tuple[Optional[str], Optional[str]]: name = node.name if node.name else call.assigned_variable if name is not None: name = self.dedupe(name) - return name, call.return_type + return_type = call.return_type if node.type is None else node.type + return name, return_type diff --git a/Tools/peg_generator/pegen/grammar.py b/Tools/peg_generator/pegen/grammar.py index 78edf412ea6e4..332ee3c3eec5e 100644 --- a/Tools/peg_generator/pegen/grammar.py +++ b/Tools/peg_generator/pegen/grammar.py @@ -259,9 +259,10 @@ def collect_todo(self, gen: ParserGenerator) -> None: class NamedItem: - def __init__(self, name: Optional[str], item: Item): + def __init__(self, name: Optional[str], item: Item, type: Optional[str] = None): self.name = name self.item = item + self.type = type self.nullable = False def __str__(self) -> str: diff --git a/Tools/peg_generator/pegen/grammar_parser.py b/Tools/peg_generator/pegen/grammar_parser.py index c784cfdf3b266..6e3bc5068f5a7 100644 --- a/Tools/peg_generator/pegen/grammar_parser.py +++ b/Tools/peg_generator/pegen/grammar_parser.py @@ -402,9 +402,49 @@ def items(self) -> Optional[NamedItemList]: @memoize def named_item(self) -> Optional[NamedItem]: - # named_item: NAME '=' ~ item | item | lookahead + # named_item: NAME '[' NAME '*' ']' '=' ~ item | NAME '[' NAME ']' '=' ~ item | NAME '=' ~ item | item | lookahead mark = self.mark() cut = False + if ( + (name := self.name()) + and + (literal := self.expect('[')) + and + (type := self.name()) + and + (literal_1 := self.expect('*')) + and + (literal_2 := self.expect(']')) + and + (literal_3 := self.expect('=')) + and + (cut := True) + and + (item := self.item()) + ): + return NamedItem ( name . string , item , f"{type.string}*" ) + self.reset(mark) + if cut: return None + cut = False + if ( + (name := self.name()) + and + (literal := self.expect('[')) + and + (type := self.name()) + and + (literal_1 := self.expect(']')) + and + (literal_2 := self.expect('=')) + and + (cut := True) + and + (item := self.item()) + ): + return NamedItem ( name . string , item , type . string ) + self.reset(mark) + if cut: return None + cut = False if ( (name := self.name()) and diff --git a/Tools/peg_generator/pegen/metagrammar.gram b/Tools/peg_generator/pegen/metagrammar.gram index f0c5ac3ab390f..4802f56b68f7b 100644 --- a/Tools/peg_generator/pegen/metagrammar.gram +++ b/Tools/peg_generator/pegen/metagrammar.gram @@ -83,6 +83,8 @@ items[NamedItemList]: | named_item { [named_item] } named_item[NamedItem]: + | NAME '[' type=NAME '*' ']' '=' ~ item {NamedItem(name.string, item, f"{type.string}*")} + | NAME '[' type=NAME ']' '=' ~ item {NamedItem(name.string, item, type.string)} | NAME '=' ~ item {NamedItem(name.string, item)} | item {NamedItem(None, item)} | it=lookahead {NamedItem(None, it)} From webhook-mailer at python.org Wed Sep 16 15:38:15 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Wed, 16 Sep 2020 19:38:15 -0000 Subject: [Python-checkins] _auto_called cleanup (GH-22285) Message-ID: https://github.com/python/cpython/commit/fc23a9483ef0d7c98bea9f82392377d0b6ef7b18 commit: fc23a9483ef0d7c98bea9f82392377d0b6ef7b18 branch: master author: Ethan Furman committer: GitHub date: 2020-09-16T12:37:54-07:00 summary: _auto_called cleanup (GH-22285) files: M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 060b2a0dadf45..21a94caaee33f 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -105,9 +105,9 @@ def __setitem__(self, key, value): # enum overwriting a descriptor? raise TypeError('%r already defined as: %r' % (key, self[key])) if isinstance(value, auto): - self._auto_called = True if value.value == _auto_null: value.value = self._generate_next_value(key, 1, len(self._member_names), self._last_values[:]) + self._auto_called = True value = value.value self._member_names.append(key) self._last_values.append(value) diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 5d72d82cec27f..ebf76047972dc 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1837,6 +1837,17 @@ class Color(Enum): def _generate_next_value_(name, start, count, last): return name + def test_auto_order_wierd(self): + weird_auto = auto() + weird_auto.value = 'pathological case' + class Color(Enum): + red = weird_auto + def _generate_next_value_(name, start, count, last): + return name + blue = auto() + self.assertEqual(list(Color), [Color.red, Color.blue]) + self.assertEqual(Color.red.value, 'pathological case') + self.assertEqual(Color.blue.value, 'blue') def test_duplicate_auto(self): class Dupes(Enum): From webhook-mailer at python.org Wed Sep 16 16:01:09 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Wed, 16 Sep 2020 20:01:09 -0000 Subject: [Python-checkins] Enum: make `Flag` and `IntFlag` members iterable (GH-22221) Message-ID: https://github.com/python/cpython/commit/7219e27087baaa8a5693b5bef1b1357bddbffa53 commit: 7219e27087baaa8a5693b5bef1b1357bddbffa53 branch: master author: Ethan Furman committer: GitHub date: 2020-09-16T13:01:00-07:00 summary: Enum: make `Flag` and `IntFlag` members iterable (GH-22221) files: A Misc/NEWS.d/next/Library/2020-09-12-16-18-42.bpo-32218.IpYkEe.rst M Doc/library/enum.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 32e8bbf950927..2f84be229bc4d 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -656,6 +656,13 @@ be combined with them:: >>> Perm.X | 8 +:class:`IntFlag` members can also be iterated over:: + + >>> list(RW) + [, ] + +.. versionadded:: 3.10 + Flag ^^^^ @@ -709,6 +716,14 @@ value:: >>> bool(Color.BLACK) False +:class:`Flag` members can also be iterated over:: + + >>> purple = Color.RED | Color.BLUE + >>> list(purple) + [, ] + +.. versionadded:: 3.10 + .. note:: For the majority of new code, :class:`Enum` and :class:`Flag` are strongly diff --git a/Lib/enum.py b/Lib/enum.py index 21a94caaee33f..3c459ea4113d0 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -753,6 +753,10 @@ def __contains__(self, other): type(other).__qualname__, self.__class__.__qualname__)) return other._value_ & self._value_ == other._value_ + def __iter__(self): + members, extra_flags = _decompose(self.__class__, self.value) + return (m for m in members if m._value_ != 0) + def __repr__(self): cls = self.__class__ if self._name_ is not None: diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index ebf76047972dc..59789fb7bcc5f 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -2350,6 +2350,12 @@ def test_member_contains(self): self.assertFalse(W in RX) self.assertFalse(X in RW) + def test_member_iter(self): + Color = self.Color + self.assertEqual(list(Color.PURPLE), [Color.BLUE, Color.RED]) + self.assertEqual(list(Color.BLUE), [Color.BLUE]) + self.assertEqual(list(Color.GREEN), [Color.GREEN]) + def test_auto_number(self): class Color(Flag): red = auto() @@ -2805,6 +2811,12 @@ def test_member_contains(self): with self.assertRaises(TypeError): self.assertFalse('test' in RW) + def test_member_iter(self): + Color = self.Color + self.assertEqual(list(Color.PURPLE), [Color.BLUE, Color.RED]) + self.assertEqual(list(Color.BLUE), [Color.BLUE]) + self.assertEqual(list(Color.GREEN), [Color.GREEN]) + def test_bool(self): Perm = self.Perm for f in Perm: diff --git a/Misc/NEWS.d/next/Library/2020-09-12-16-18-42.bpo-32218.IpYkEe.rst b/Misc/NEWS.d/next/Library/2020-09-12-16-18-42.bpo-32218.IpYkEe.rst new file mode 100644 index 0000000000000..d5832b9767b70 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-12-16-18-42.bpo-32218.IpYkEe.rst @@ -0,0 +1 @@ +`enum.Flag` and `enum.IntFlag` members are now iterable From webhook-mailer at python.org Wed Sep 16 20:27:31 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 17 Sep 2020 00:27:31 -0000 Subject: [Python-checkins] _auto_called cleanup (GH-22285) Message-ID: https://github.com/python/cpython/commit/4465df626684fdc8eeb25ca248218cde2eeeb9ec commit: 4465df626684fdc8eeb25ca248218cde2eeeb9ec branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-16T17:27:09-07:00 summary: _auto_called cleanup (GH-22285) (cherry picked from commit fc23a9483ef0d7c98bea9f82392377d0b6ef7b18) Co-authored-by: Ethan Furman files: M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index bdbfbb69bdea6..da5a31f0b88ac 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -104,9 +104,9 @@ def __setitem__(self, key, value): # enum overwriting a descriptor? raise TypeError('%r already defined as: %r' % (key, self[key])) if isinstance(value, auto): - self._auto_called = True if value.value == _auto_null: value.value = self._generate_next_value(key, 1, len(self._member_names), self._last_values[:]) + self._auto_called = True value = value.value self._member_names.append(key) self._last_values.append(value) diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 0ecf6413b1313..6f547165acfee 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1813,6 +1813,17 @@ class Color(Enum): def _generate_next_value_(name, start, count, last): return name + def test_auto_order_wierd(self): + weird_auto = auto() + weird_auto.value = 'pathological case' + class Color(Enum): + red = weird_auto + def _generate_next_value_(name, start, count, last): + return name + blue = auto() + self.assertEqual(list(Color), [Color.red, Color.blue]) + self.assertEqual(Color.red.value, 'pathological case') + self.assertEqual(Color.blue.value, 'blue') def test_duplicate_auto(self): class Dupes(Enum): From webhook-mailer at python.org Wed Sep 16 20:28:36 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 17 Sep 2020 00:28:36 -0000 Subject: [Python-checkins] _auto_called cleanup (GH-22285) Message-ID: https://github.com/python/cpython/commit/007eddad3b809e1555391dc7e8168918933b6184 commit: 007eddad3b809e1555391dc7e8168918933b6184 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-16T17:28:32-07:00 summary: _auto_called cleanup (GH-22285) (cherry picked from commit fc23a9483ef0d7c98bea9f82392377d0b6ef7b18) Co-authored-by: Ethan Furman files: M Lib/enum.py M Lib/test/test_enum.py diff --git a/Lib/enum.py b/Lib/enum.py index 4c1acc6e360bd..c2adfd1c6c0bb 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -104,9 +104,9 @@ def __setitem__(self, key, value): # enum overwriting a descriptor? raise TypeError('%r already defined as: %r' % (key, self[key])) if isinstance(value, auto): - self._auto_called = True if value.value == _auto_null: value.value = self._generate_next_value(key, 1, len(self._member_names), self._last_values[:]) + self._auto_called = True value = value.value self._member_names.append(key) self._last_values.append(value) diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 11e88dba6b5b3..daa44ae6824ce 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1815,6 +1815,17 @@ class Color(Enum): def _generate_next_value_(name, start, count, last): return name + def test_auto_order_wierd(self): + weird_auto = auto() + weird_auto.value = 'pathological case' + class Color(Enum): + red = weird_auto + def _generate_next_value_(name, start, count, last): + return name + blue = auto() + self.assertEqual(list(Color), [Color.red, Color.blue]) + self.assertEqual(Color.red.value, 'pathological case') + self.assertEqual(Color.blue.value, 'blue') def test_duplicate_auto(self): class Dupes(Enum): From webhook-mailer at python.org Wed Sep 16 20:37:56 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Thu, 17 Sep 2020 00:37:56 -0000 Subject: [Python-checkins] [3.9] bpo-39728: Enum: fix duplicate `ValueError` (GH-22277) (GH-22282) Message-ID: https://github.com/python/cpython/commit/a9ba8ba9a71f3cb8d274c354ff67b6206abeb8ac commit: a9ba8ba9a71f3cb8d274c354ff67b6206abeb8ac branch: 3.9 author: Ethan Furman committer: GitHub date: 2020-09-16T17:37:51-07:00 summary: [3.9] bpo-39728: Enum: fix duplicate `ValueError` (GH-22277) (GH-22282) fix default `_missing_` to return `None` instead of raising a `ValueError` Co-authored-by: Andrey Darascheka (cherry picked from commit c95ad7a91fbd7636f33a098d3b39964ab083bf49) Co-authored-by: Ethan Furman files: A Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst M Lib/enum.py M Lib/test/test_enum.py M Misc/ACKS diff --git a/Lib/enum.py b/Lib/enum.py index da5a31f0b88ac..ebadd9f662a13 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -628,7 +628,7 @@ def _generate_next_value_(name, start, count, last_values): @classmethod def _missing_(cls, value): - raise ValueError("%r is not a valid %s" % (value, cls.__qualname__)) + return None def __repr__(self): return "<%s.%s: %r>" % ( diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 6f547165acfee..0e6b1b16ca323 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1832,6 +1832,18 @@ class Dupes(Enum): third = auto() self.assertEqual([Dupes.first, Dupes.second, Dupes.third], list(Dupes)) + def test_default_missing(self): + class Color(Enum): + RED = 1 + GREEN = 2 + BLUE = 3 + try: + Color(7) + except ValueError as exc: + self.assertTrue(exc.__context__ is None) + else: + raise Exception('Exception not raised.') + def test_missing(self): class Color(Enum): red = 1 @@ -1850,7 +1862,12 @@ def _missing_(cls, item): # trigger not found return None self.assertIs(Color('three'), Color.blue) - self.assertRaises(ValueError, Color, 7) + try: + Color(7) + except ValueError as exc: + self.assertTrue(exc.__context__ is None) + else: + raise Exception('Exception not raised.') try: Color('bad return') except TypeError as exc: diff --git a/Misc/ACKS b/Misc/ACKS index 2a180eb67cc5c..5e0953ae91b18 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -432,6 +432,7 @@ Marcos Donolo Dima Dorfman Yves Dorfsman Michael Dorman +Andrey Doroschenko Steve Dower Allen Downey Cesar Douady diff --git a/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst b/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst new file mode 100644 index 0000000000000..beb2016a85ba6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst @@ -0,0 +1 @@ +fix default `_missing_` so a duplicate `ValueError` is not set as the `__context__` of the original `ValueError` From webhook-mailer at python.org Wed Sep 16 20:38:19 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Thu, 17 Sep 2020 00:38:19 -0000 Subject: [Python-checkins] [3.8] bpo-39728: Enum: fix duplicate `ValueError` (GH-22277) (GH-22283) Message-ID: https://github.com/python/cpython/commit/5efb1a77e75648012f8b52960c8637fc296a5c6d commit: 5efb1a77e75648012f8b52960c8637fc296a5c6d branch: 3.8 author: Ethan Furman committer: GitHub date: 2020-09-16T17:38:14-07:00 summary: [3.8] bpo-39728: Enum: fix duplicate `ValueError` (GH-22277) (GH-22283) fix default `_missing_` to return `None` instead of raising a `ValueError` Co-authored-by: Andrey Darascheka . (cherry picked from commit c95ad7a91fbd7636f33a098d3b39964ab083bf49) Co-authored-by: Ethan Furman files: A Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst M Lib/enum.py M Lib/test/test_enum.py M Misc/ACKS diff --git a/Lib/enum.py b/Lib/enum.py index c2adfd1c6c0bb..de9ed4c1ec371 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -634,7 +634,7 @@ def _generate_next_value_(name, start, count, last_values): @classmethod def _missing_(cls, value): - raise ValueError("%r is not a valid %s" % (value, cls.__name__)) + return None def __repr__(self): return "<%s.%s: %r>" % ( diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index daa44ae6824ce..745962a1e66fe 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -1834,6 +1834,18 @@ class Dupes(Enum): third = auto() self.assertEqual([Dupes.first, Dupes.second, Dupes.third], list(Dupes)) + def test_default_missing(self): + class Color(Enum): + RED = 1 + GREEN = 2 + BLUE = 3 + try: + Color(7) + except ValueError as exc: + self.assertTrue(exc.__context__ is None) + else: + raise Exception('Exception not raised.') + def test_missing(self): class Color(Enum): red = 1 @@ -1852,7 +1864,12 @@ def _missing_(cls, item): # trigger not found return None self.assertIs(Color('three'), Color.blue) - self.assertRaises(ValueError, Color, 7) + try: + Color(7) + except ValueError as exc: + self.assertTrue(exc.__context__ is None) + else: + raise Exception('Exception not raised.') try: Color('bad return') except TypeError as exc: diff --git a/Misc/ACKS b/Misc/ACKS index e6d0c3ba12708..eee6b9accaa50 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -421,6 +421,7 @@ Marcos Donolo Dima Dorfman Yves Dorfsman Michael Dorman +Andrey Doroschenko Steve Dower Allen Downey Cesar Douady diff --git a/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst b/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst new file mode 100644 index 0000000000000..beb2016a85ba6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst @@ -0,0 +1 @@ +fix default `_missing_` so a duplicate `ValueError` is not set as the `__context__` of the original `ValueError` From webhook-mailer at python.org Thu Sep 17 03:34:43 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Thu, 17 Sep 2020 07:34:43 -0000 Subject: [Python-checkins] bpo-41715: Fix potential catastrofic backtracking in c_analyzer. (GH-22091) Message-ID: https://github.com/python/cpython/commit/dcfaa520c4638a67052a4ff4a2a820be68750ad7 commit: dcfaa520c4638a67052a4ff4a2a820be68750ad7 branch: master author: Serhiy Storchaka committer: GitHub date: 2020-09-17T10:34:20+03:00 summary: bpo-41715: Fix potential catastrofic backtracking in c_analyzer. (GH-22091) files: M Tools/c-analyzer/c_analyzer/common/info.py diff --git a/Tools/c-analyzer/c_analyzer/common/info.py b/Tools/c-analyzer/c_analyzer/common/info.py index 3f3f8c5b05de5..1a853a42ff2a2 100644 --- a/Tools/c-analyzer/c_analyzer/common/info.py +++ b/Tools/c-analyzer/c_analyzer/common/info.py @@ -9,7 +9,8 @@ UNKNOWN = '???' -NAME_RE = re.compile(r'^([a-zA-Z]|_\w*[a-zA-Z]\w*|[a-zA-Z]\w*)$') +# Does not start with digit and contains at least one letter. +NAME_RE = re.compile(r'(?!\d)(?=.*?[A-Za-z])\w+', re.ASCII) class ID(_NTBase, namedtuple('ID', 'filename funcname name')): @@ -50,17 +51,16 @@ def validate(self): """Fail if the object is invalid (i.e. init with bad data).""" if not self.name: raise TypeError('missing name') - else: - if not NAME_RE.match(self.name): - raise ValueError( - f'name must be an identifier, got {self.name!r}') + if not NAME_RE.fullmatch(self.name): + raise ValueError( + f'name must be an identifier, got {self.name!r}') # Symbols from a binary might not have filename/funcname info. if self.funcname: if not self.filename: raise TypeError('missing filename') - if not NAME_RE.match(self.funcname) and self.funcname != UNKNOWN: + if not NAME_RE.fullmatch(self.funcname) and self.funcname != UNKNOWN: raise ValueError( f'name must be an identifier, got {self.funcname!r}') From webhook-mailer at python.org Thu Sep 17 03:35:49 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Thu, 17 Sep 2020 07:35:49 -0000 Subject: [Python-checkins] bpo-41662: Fix bugs in binding parameters in sqlite3 (GH-21998) Message-ID: https://github.com/python/cpython/commit/0b419b791077414bbc011a412698ebb362b63761 commit: 0b419b791077414bbc011a412698ebb362b63761 branch: master author: Serhiy Storchaka committer: GitHub date: 2020-09-17T10:35:44+03:00 summary: bpo-41662: Fix bugs in binding parameters in sqlite3 (GH-21998) * When the parameters argument is a list, correctly handle the case of changing it during iteration. * When the parameters argument is a custom sequence, no longer override an exception raised in ``__len__()``. files: A Misc/NEWS.d/next/Library/2020-08-29-16-07-36.bpo-41662.Mn79zh.rst A Misc/NEWS.d/next/Library/2020-08-30-21-38-57.bpo-41662.6e9iZn.rst M Lib/sqlite3/test/dbapi.py M Lib/sqlite3/test/regression.py M Modules/_sqlite/statement.c diff --git a/Lib/sqlite3/test/dbapi.py b/Lib/sqlite3/test/dbapi.py index a8dfeb9b2d693..7867bf361e5ac 100644 --- a/Lib/sqlite3/test/dbapi.py +++ b/Lib/sqlite3/test/dbapi.py @@ -270,7 +270,7 @@ def CheckExecuteParamList(self): self.assertEqual(row[0], "foo") def CheckExecuteParamSequence(self): - class L(object): + class L: def __len__(self): return 1 def __getitem__(self, x): @@ -282,6 +282,18 @@ def __getitem__(self, x): row = self.cu.fetchone() self.assertEqual(row[0], "foo") + def CheckExecuteParamSequenceBadLen(self): + # Issue41662: Error in __len__() was overridden with ProgrammingError. + class L: + def __len__(self): + 1/0 + def __getitem__(slf, x): + raise AssertionError + + self.cu.execute("insert into test(name) values ('foo')") + with self.assertRaises(ZeroDivisionError): + self.cu.execute("select name from test where name=?", L()) + def CheckExecuteDictMapping(self): self.cu.execute("insert into test(name) values ('foo')") self.cu.execute("select name from test where name=:name", {"name": "foo"}) diff --git a/Lib/sqlite3/test/regression.py b/Lib/sqlite3/test/regression.py index 0735a5c129226..67557e19c796b 100644 --- a/Lib/sqlite3/test/regression.py +++ b/Lib/sqlite3/test/regression.py @@ -132,6 +132,19 @@ def CheckTypeMapUsage(self): con.execute("insert into foo(bar) values (5)") con.execute(SELECT) + def CheckBindMutatingList(self): + # Issue41662: Crash when mutate a list of parameters during iteration. + class X: + def __conform__(self, protocol): + parameters.clear() + return "..." + parameters = [X(), 0] + con = sqlite.connect(":memory:",detect_types=sqlite.PARSE_DECLTYPES) + con.execute("create table foo(bar X, baz integer)") + # Should not crash + with self.assertRaises(IndexError): + con.execute("insert into foo(bar, baz) values (?, ?)", parameters) + def CheckErrorMsgDecodeError(self): # When porting the module to Python 3.0, the error message about # decoding errors disappeared. This verifies they're back again. diff --git a/Misc/NEWS.d/next/Library/2020-08-29-16-07-36.bpo-41662.Mn79zh.rst b/Misc/NEWS.d/next/Library/2020-08-29-16-07-36.bpo-41662.Mn79zh.rst new file mode 100644 index 0000000000000..0571c2d110bee --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-08-29-16-07-36.bpo-41662.Mn79zh.rst @@ -0,0 +1 @@ +Fixed crash when mutate list of parameters during iteration in :mod:`sqlite3`. diff --git a/Misc/NEWS.d/next/Library/2020-08-30-21-38-57.bpo-41662.6e9iZn.rst b/Misc/NEWS.d/next/Library/2020-08-30-21-38-57.bpo-41662.6e9iZn.rst new file mode 100644 index 0000000000000..aecb0a1ea4d08 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-08-30-21-38-57.bpo-41662.6e9iZn.rst @@ -0,0 +1,2 @@ +No longer override exceptions raised in ``__len__()`` of a sequence of +parameters in :mod:`sqlite3` with :exc:`~sqlite3.ProgrammingError`. diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c index 26599b423eb8b..02e47a02b718c 100644 --- a/Modules/_sqlite/statement.c +++ b/Modules/_sqlite/statement.c @@ -227,6 +227,9 @@ void pysqlite_statement_bind_parameters(pysqlite_Statement* self, PyObject* para num_params = PyList_GET_SIZE(parameters); } else { num_params = PySequence_Size(parameters); + if (num_params == -1) { + return; + } } if (num_params != num_params_needed) { PyErr_Format(pysqlite_ProgrammingError, @@ -238,9 +241,9 @@ void pysqlite_statement_bind_parameters(pysqlite_Statement* self, PyObject* para for (i = 0; i < num_params; i++) { if (PyTuple_CheckExact(parameters)) { current_param = PyTuple_GET_ITEM(parameters, i); - Py_XINCREF(current_param); + Py_INCREF(current_param); } else if (PyList_CheckExact(parameters)) { - current_param = PyList_GET_ITEM(parameters, i); + current_param = PyList_GetItem(parameters, i); Py_XINCREF(current_param); } else { current_param = PySequence_GetItem(parameters, i); From webhook-mailer at python.org Thu Sep 17 03:57:22 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 17 Sep 2020 07:57:22 -0000 Subject: [Python-checkins] bpo-41662: Fix bugs in binding parameters in sqlite3 (GH-21998) Message-ID: https://github.com/python/cpython/commit/f76a3889d1fc6c5514323866a047b642c6da4e9b commit: f76a3889d1fc6c5514323866a047b642c6da4e9b branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-17T00:57:07-07:00 summary: bpo-41662: Fix bugs in binding parameters in sqlite3 (GH-21998) * When the parameters argument is a list, correctly handle the case of changing it during iteration. * When the parameters argument is a custom sequence, no longer override an exception raised in ``__len__()``. (cherry picked from commit 0b419b791077414bbc011a412698ebb362b63761) Co-authored-by: Serhiy Storchaka files: A Misc/NEWS.d/next/Library/2020-08-29-16-07-36.bpo-41662.Mn79zh.rst A Misc/NEWS.d/next/Library/2020-08-30-21-38-57.bpo-41662.6e9iZn.rst M Lib/sqlite3/test/dbapi.py M Lib/sqlite3/test/regression.py M Modules/_sqlite/statement.c diff --git a/Lib/sqlite3/test/dbapi.py b/Lib/sqlite3/test/dbapi.py index be11337154bdd..ad9c9f02a79a3 100644 --- a/Lib/sqlite3/test/dbapi.py +++ b/Lib/sqlite3/test/dbapi.py @@ -276,7 +276,7 @@ def CheckExecuteParamList(self): self.assertEqual(row[0], "foo") def CheckExecuteParamSequence(self): - class L(object): + class L: def __len__(self): return 1 def __getitem__(self, x): @@ -288,6 +288,18 @@ def __getitem__(self, x): row = self.cu.fetchone() self.assertEqual(row[0], "foo") + def CheckExecuteParamSequenceBadLen(self): + # Issue41662: Error in __len__() was overridden with ProgrammingError. + class L: + def __len__(self): + 1/0 + def __getitem__(slf, x): + raise AssertionError + + self.cu.execute("insert into test(name) values ('foo')") + with self.assertRaises(ZeroDivisionError): + self.cu.execute("select name from test where name=?", L()) + def CheckExecuteDictMapping(self): self.cu.execute("insert into test(name) values ('foo')") self.cu.execute("select name from test where name=:name", {"name": "foo"}) diff --git a/Lib/sqlite3/test/regression.py b/Lib/sqlite3/test/regression.py index cbd46d4978afb..6aa86d5766e56 100644 --- a/Lib/sqlite3/test/regression.py +++ b/Lib/sqlite3/test/regression.py @@ -133,6 +133,19 @@ def CheckTypeMapUsage(self): con.execute("insert into foo(bar) values (5)") con.execute(SELECT) + def CheckBindMutatingList(self): + # Issue41662: Crash when mutate a list of parameters during iteration. + class X: + def __conform__(self, protocol): + parameters.clear() + return "..." + parameters = [X(), 0] + con = sqlite.connect(":memory:",detect_types=sqlite.PARSE_DECLTYPES) + con.execute("create table foo(bar X, baz integer)") + # Should not crash + with self.assertRaises(IndexError): + con.execute("insert into foo(bar, baz) values (?, ?)", parameters) + def CheckErrorMsgDecodeError(self): # When porting the module to Python 3.0, the error message about # decoding errors disappeared. This verifies they're back again. diff --git a/Misc/NEWS.d/next/Library/2020-08-29-16-07-36.bpo-41662.Mn79zh.rst b/Misc/NEWS.d/next/Library/2020-08-29-16-07-36.bpo-41662.Mn79zh.rst new file mode 100644 index 0000000000000..0571c2d110bee --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-08-29-16-07-36.bpo-41662.Mn79zh.rst @@ -0,0 +1 @@ +Fixed crash when mutate list of parameters during iteration in :mod:`sqlite3`. diff --git a/Misc/NEWS.d/next/Library/2020-08-30-21-38-57.bpo-41662.6e9iZn.rst b/Misc/NEWS.d/next/Library/2020-08-30-21-38-57.bpo-41662.6e9iZn.rst new file mode 100644 index 0000000000000..aecb0a1ea4d08 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-08-30-21-38-57.bpo-41662.6e9iZn.rst @@ -0,0 +1,2 @@ +No longer override exceptions raised in ``__len__()`` of a sequence of +parameters in :mod:`sqlite3` with :exc:`~sqlite3.ProgrammingError`. diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c index 9de8f9b67228f..23c204e7521f0 100644 --- a/Modules/_sqlite/statement.c +++ b/Modules/_sqlite/statement.c @@ -227,6 +227,9 @@ void pysqlite_statement_bind_parameters(pysqlite_Statement* self, PyObject* para num_params = PyList_GET_SIZE(parameters); } else { num_params = PySequence_Size(parameters); + if (num_params == -1) { + return; + } } if (num_params != num_params_needed) { PyErr_Format(pysqlite_ProgrammingError, @@ -238,9 +241,9 @@ void pysqlite_statement_bind_parameters(pysqlite_Statement* self, PyObject* para for (i = 0; i < num_params; i++) { if (PyTuple_CheckExact(parameters)) { current_param = PyTuple_GET_ITEM(parameters, i); - Py_XINCREF(current_param); + Py_INCREF(current_param); } else if (PyList_CheckExact(parameters)) { - current_param = PyList_GET_ITEM(parameters, i); + current_param = PyList_GetItem(parameters, i); Py_XINCREF(current_param); } else { current_param = PySequence_GetItem(parameters, i); From webhook-mailer at python.org Thu Sep 17 04:49:07 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Thu, 17 Sep 2020 08:49:07 -0000 Subject: [Python-checkins] bpo-27032, bpo-37328: Document removing HTMLParser.unescape(). (GH-22288) Message-ID: https://github.com/python/cpython/commit/a33f2c2bae759fc9d06e1c032fd2026135f2df45 commit: a33f2c2bae759fc9d06e1c032fd2026135f2df45 branch: master author: Serhiy Storchaka committer: GitHub date: 2020-09-17T11:49:01+03:00 summary: bpo-27032, bpo-37328: Document removing HTMLParser.unescape(). (GH-22288) files: M Doc/whatsnew/3.9.rst diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index be7406e13c2cd..e1ea799b76296 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -887,6 +887,12 @@ Removed :func:`asyncio.current_task` and :func:`asyncio.all_tasks` instead. (Contributed by R?mi Lapeyre in :issue:`40967`) +* The ``unescape()`` method in the :class:`html.parser.HTMLParser` class + has been removed (it was deprecated since Python 3.4). :func:`html.unescape` + should be used for converting character references to the corresponding + unicode characters. + + Porting to Python 3.9 ===================== From webhook-mailer at python.org Thu Sep 17 04:50:54 2020 From: webhook-mailer at python.org (=?utf-8?q?=C5=81ukasz?= Langa) Date: Thu, 17 Sep 2020 08:50:54 -0000 Subject: [Python-checkins] Python 3.9.0rc2 Message-ID: https://github.com/python/cpython/commit/2bd31b5fde32194a63ca1e782e2578c6bd2ba759 commit: 2bd31b5fde32194a63ca1e782e2578c6bd2ba759 branch: 3.9 author: ?ukasz Langa committer: ?ukasz Langa date: 2020-09-16T23:23:13+02:00 summary: Python 3.9.0rc2 files: A Misc/NEWS.d/3.9.0rc2.rst D Misc/NEWS.d/next/Build/2020-08-24-18-34-01.bpo-41617.sKKXz7.rst D Misc/NEWS.d/next/C API/2020-08-12-17-09-06.bpo-41524.u6Xfr2.rst D Misc/NEWS.d/next/Core and Builtins/2020-05-03-22-26-00.bpo-29590.aRz3l7.rst D Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst D Misc/NEWS.d/next/Core and Builtins/2020-08-12-19-32-15.bpo-41531.WgPzjT.rst D Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-41533.4pcVAc.rst D Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst D Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst D Misc/NEWS.d/next/Core and Builtins/2020-08-31-17-49-02.bpo-41681.3-VJiH.rst D Misc/NEWS.d/next/Core and Builtins/2020-09-02-12-00-57.bpo-41690.Ny-Sfy.rst D Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst D Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst D Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst D Misc/NEWS.d/next/Documentation/2020-08-12-18-35-40.bpo-40204.C8A_pe.rst D Misc/NEWS.d/next/Documentation/2020-08-25-15-11-23.bpo-41624.ddjJlN.rst D Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst D Misc/NEWS.d/next/Library/2018-06-12-23-30-41.bpo-33660.AdDn5Z.rst D Misc/NEWS.d/next/Library/2020-03-11-07-44-06.bpo-31122.zIQ80l.rst D Misc/NEWS.d/next/Library/2020-06-18-10-34-59.bpo-41025.elf_nz.rst D Misc/NEWS.d/next/Library/2020-07-20-13-27-48.bpo-41344.iKipNd.rst D Misc/NEWS.d/next/Library/2020-08-07-15-18-16.bpo-41503.IYftcu.rst D Misc/NEWS.d/next/Library/2020-08-12-13-25-16.bpo-41520.BEUWa4.rst D Misc/NEWS.d/next/Library/2020-08-13-08-07-25.bpo-40782.aGZqmB.rst D Misc/NEWS.d/next/Library/2020-08-15-15-21-40.bpo-37658.f9nivB.rst D Misc/NEWS.d/next/Library/2020-08-15-15-50-12.bpo-32751.85je5X.rst D Misc/NEWS.d/next/Library/2020-08-21-15-51-15.bpo-41609.JmiUKG.rst D Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rst D Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rst D Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst D Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst D Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst D Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst D Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst D Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst D Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst D Misc/NEWS.d/next/Windows/2020-08-13-22-40-58.bpo-41526.-i2bwb.rst M Include/patchlevel.h M Lib/pydoc_data/topics.py M README.rst diff --git a/Include/patchlevel.h b/Include/patchlevel.h index 80124ed968f96..bed77a62c4171 100644 --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -20,10 +20,10 @@ #define PY_MINOR_VERSION 9 #define PY_MICRO_VERSION 0 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA -#define PY_RELEASE_SERIAL 1 +#define PY_RELEASE_SERIAL 2 /* Version as a string */ -#define PY_VERSION "3.9.0rc1+" +#define PY_VERSION "3.9.0rc2" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py index 08326400864c5..288bdfa195664 100644 --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Tue Aug 11 19:08:56 2020 +# Autogenerated by Sphinx on Wed Sep 16 23:08:58 2020 topics = {'assert': 'The "assert" statement\n' '**********************\n' '\n' @@ -6946,6 +6946,14 @@ 'the data\n' ' model.\n' '\n' + ' Note: Due to a bug in the dispatching mechanism for ' + '"**=", a\n' + ' class that defines "__ipow__()" but returns ' + '"NotImplemented"\n' + ' would fail to fall back to "x.__pow__(y)" and ' + '"y.__rpow__(x)".\n' + ' This bug is fixed in Python 3.10.\n' + '\n' 'object.__neg__(self)\n' 'object.__pos__(self)\n' 'object.__abs__(self)\n' @@ -9627,6 +9635,14 @@ 'the data\n' ' model.\n' '\n' + ' Note: Due to a bug in the dispatching mechanism for ' + '"**=", a\n' + ' class that defines "__ipow__()" but returns ' + '"NotImplemented"\n' + ' would fail to fall back to "x.__pow__(y)" and ' + '"y.__rpow__(x)".\n' + ' This bug is fixed in Python 3.10.\n' + '\n' 'object.__neg__(self)\n' 'object.__pos__(self)\n' 'object.__abs__(self)\n' @@ -9915,35 +9931,6 @@ '*start* and\n' ' *end* are interpreted as in slice notation.\n' '\n' - 'str.removeprefix(prefix, /)\n' - '\n' - ' If the string starts with the *prefix* string, return\n' - ' "string[len(prefix):]". Otherwise, return a copy of the ' - 'original\n' - ' string:\n' - '\n' - " >>> 'TestHook'.removeprefix('Test')\n" - " 'Hook'\n" - " >>> 'BaseTestCase'.removeprefix('Test')\n" - " 'BaseTestCase'\n" - '\n' - ' New in version 3.9.\n' - '\n' - 'str.removesuffix(suffix, /)\n' - '\n' - ' If the string ends with the *suffix* string and that ' - '*suffix* is\n' - ' not empty, return "string[:-len(suffix)]". Otherwise, ' - 'return a copy\n' - ' of the original string:\n' - '\n' - " >>> 'MiscTests'.removesuffix('Tests')\n" - " 'Misc'\n" - " >>> 'TmpDirMixin'.removesuffix('Tests')\n" - " 'TmpDirMixin'\n" - '\n' - ' New in version 3.9.\n' - '\n' 'str.encode(encoding="utf-8", errors="strict")\n' '\n' ' Return an encoded version of the string as a bytes ' @@ -10372,6 +10359,35 @@ 'followed by\n' ' two empty strings.\n' '\n' + 'str.removeprefix(prefix, /)\n' + '\n' + ' If the string starts with the *prefix* string, return\n' + ' "string[len(prefix):]". Otherwise, return a copy of the ' + 'original\n' + ' string:\n' + '\n' + " >>> 'TestHook'.removeprefix('Test')\n" + " 'Hook'\n" + " >>> 'BaseTestCase'.removeprefix('Test')\n" + " 'BaseTestCase'\n" + '\n' + ' New in version 3.9.\n' + '\n' + 'str.removesuffix(suffix, /)\n' + '\n' + ' If the string ends with the *suffix* string and that ' + '*suffix* is\n' + ' not empty, return "string[:-len(suffix)]". Otherwise, ' + 'return a copy\n' + ' of the original string:\n' + '\n' + " >>> 'MiscTests'.removesuffix('Tests')\n" + " 'Misc'\n" + " >>> 'TmpDirMixin'.removesuffix('Tests')\n" + " 'TmpDirMixin'\n" + '\n' + ' New in version 3.9.\n' + '\n' 'str.replace(old, new[, count])\n' '\n' ' Return a copy of the string with all occurrences of ' diff --git a/Misc/NEWS.d/3.9.0rc2.rst b/Misc/NEWS.d/3.9.0rc2.rst new file mode 100644 index 0000000000000..3da8441a02f4c --- /dev/null +++ b/Misc/NEWS.d/3.9.0rc2.rst @@ -0,0 +1,360 @@ +.. bpo: 41780 +.. date: 2020-09-15-23-29-49 +.. nonce: bOBUIH +.. release date: 2020-09-16 +.. section: Core and Builtins + +Fix :meth:`__dir__` of :class:`types.GenericAlias`. Patch by Batuhan +Taskaya. + +.. + +.. bpo: 41690 +.. date: 2020-09-02-12-00-57 +.. nonce: Ny-Sfy +.. section: Core and Builtins + +Fix a possible stack overflow in the parser when parsing functions and +classes with a huge ammount of arguments. Patch by Pablo Galindo. + +.. + +.. bpo: 41681 +.. date: 2020-08-31-17-49-02 +.. nonce: 3-VJiH +.. section: Core and Builtins + +Fixes the wrong error description in the error raised by using 2 `,` in +format string in f-string and :meth:`str.format`. + +.. + +.. bpo: 41654 +.. date: 2020-08-30-20-38-33 +.. nonce: HtnhAM +.. section: Core and Builtins + +Fix a crash that occurred when destroying subclasses of +:class:`MemoryError`. Patch by Pablo Galindo. + +.. + +.. bpo: 41631 +.. date: 2020-08-26-11-23-31 +.. nonce: 3jZcd9 +.. section: Core and Builtins + +The ``_ast`` module uses again a global state. Using a module state per +module instance is causing subtle practical problems. For example, the +Mercurial project replaces the ``__import__()`` function to implement lazy +import, whereas Python expected that ``import _ast`` always return a fully +initialized ``_ast`` module. + +.. + +.. bpo: 41533 +.. date: 2020-08-12-20-29-57 +.. nonce: 4pcVAc +.. section: Core and Builtins + +Free the stack allocated in ``va_build_stack`` if ``do_mkstack`` fails and +the stack is not a ``small_stack``. + +.. + +.. bpo: 41531 +.. date: 2020-08-12-19-32-15 +.. nonce: WgPzjT +.. section: Core and Builtins + +Fix a bug that was dropping keys when compiling dict literals with more than +0xFFFF elements. Patch by Pablo Galindo. + +.. + +.. bpo: 41525 +.. date: 2020-08-12-07-35-07 +.. nonce: d9q3XL +.. section: Core and Builtins + +The output of ``python --help`` contains now only ASCII characters. + +.. + +.. bpo: 29590 +.. date: 2020-05-03-22-26-00 +.. nonce: aRz3l7 +.. section: Core and Builtins + +Make the stack trace correct after calling :meth:`generator.throw` on a +generator that has yielded from a ``yield from``. + +.. + +.. bpo: 41517 +.. date: 2020-09-15-22-43-30 +.. nonce: sLBH7g +.. section: Library + +fix bug allowing Enums to be extended via multiple inheritance + +.. + +.. bpo: 39587 +.. date: 2020-09-15-14-56-13 +.. nonce: 69xzuh +.. section: Library + +use the correct mix-in data type when constructing Enums + +.. + +.. bpo: 41789 +.. date: 2020-09-14-19-27-46 +.. nonce: pI_uZQ +.. section: Library + +Honor `object` overrides in `Enum` class creation (specifically, `__str__`, +`__repr__`, `__format__`, and `__reduce_ex__`). + +.. + +.. bpo: 39651 +.. date: 2020-09-11-12-38-55 +.. nonce: JMp9l2 +.. section: Library + +Fix a race condition in the ``call_soon_threadsafe()`` method of +``asyncio.ProactorEventLoop``: do nothing if the self-pipe socket has been +closed. + +.. + +.. bpo: 41720 +.. date: 2020-09-04-20-45-38 +.. nonce: PW9MzZ +.. section: Library + +Fixed :meth:`turtle.Vec2D.__rmul__` for arguments which are not int or +float. + +.. + +.. bpo: 41696 +.. date: 2020-09-03-01-35-32 +.. nonce: zkYGre +.. section: Library + +Fix handling of debug mode in :func:`asyncio.run`. This allows setting +``PYTHONASYNCIODEBUG`` or ``-X dev`` to enable asyncio debug mode when using +:func:`asyncio.run`. + +.. + +.. bpo: 41687 +.. date: 2020-09-01-15-57-51 +.. nonce: m1b1KA +.. section: Library + +Fix implementation of sendfile to be compatible with Solaris. + +.. + +.. bpo: 39010 +.. date: 2020-08-30-10-24-26 +.. nonce: _mzXJW +.. section: Library + +Restarting a ``ProactorEventLoop`` on Windows no longer logs spurious +``ConnectionResetErrors``. + +.. + +.. bpo: 41609 +.. date: 2020-08-21-15-51-15 +.. nonce: JmiUKG +.. section: Library + +The pdb whatis command correctly reports instance methods as 'Method' rather +than 'Function'. + +.. + +.. bpo: 32751 +.. date: 2020-08-15-15-50-12 +.. nonce: 85je5X +.. section: Library + +When cancelling the task due to a timeout, :meth:`asyncio.wait_for` will now +wait until the cancellation is complete also in the case when *timeout* is +<= 0, like it does with positive timeouts. + +.. + +.. bpo: 37658 +.. date: 2020-08-15-15-21-40 +.. nonce: f9nivB +.. section: Library + +:meth:`asyncio.wait_for` now properly handles races between cancellation of +itself and the completion of the wrapped awaitable. + +.. + +.. bpo: 40782 +.. date: 2020-08-13-08-07-25 +.. nonce: aGZqmB +.. section: Library + +Change the method asyncio.AbstractEventLoop.run_in_executor to not be a +coroutine. + +.. + +.. bpo: 41520 +.. date: 2020-08-12-13-25-16 +.. nonce: BEUWa4 +.. section: Library + +Fix :mod:`codeop` regression that prevented turning compile warnings into +errors. + +.. + +.. bpo: 41503 +.. date: 2020-08-07-15-18-16 +.. nonce: IYftcu +.. section: Library + +Fixed a race between setTarget and flush in logging.handlers.MemoryHandler. + +.. + +.. bpo: 41344 +.. date: 2020-07-20-13-27-48 +.. nonce: iKipNd +.. section: Library + +Prevent creating :class:`shared_memory.SharedMemory` objects with +:code:`size=0`. + +.. + +.. bpo: 41025 +.. date: 2020-06-18-10-34-59 +.. nonce: elf_nz +.. section: Library + +Fixed an issue preventing the C implementation of :class:`zoneinfo.ZoneInfo` +from being subclassed. + +.. + +.. bpo: 31122 +.. date: 2020-03-11-07-44-06 +.. nonce: zIQ80l +.. section: Library + +ssl.wrap_socket() now raises ssl.SSLEOFError rather than OSError when peer +closes connection during TLS negotiation + +.. + +.. bpo: 33660 +.. date: 2018-06-12-23-30-41 +.. nonce: AdDn5Z +.. section: Library + +Fix pathlib.PosixPath to resolve a relative path located on the root +directory properly. + +.. + +.. bpo: 37149 +.. date: 2020-09-10-07-48-02 +.. nonce: VD0rCv +.. section: Documentation + +Change Shipman tkinter doc link from archive.org to TkDocs. (The doc has +been removed from the NMT server.) The new link responds much faster and +includes a short explanatory note. + +.. + +.. bpo: 41624 +.. date: 2020-08-25-15-11-23 +.. nonce: ddjJlN +.. section: Documentation + +Fix the signature of :class:`typing.Coroutine`. + +.. + +.. bpo: 40204 +.. date: 2020-08-12-18-35-40 +.. nonce: C8A_pe +.. section: Documentation + +Enable Sphinx 3.2 ``c_allow_pre_v3`` option and disable +``c_warn_on_allowed_pre_v3`` option to make the documentation compatible +with Sphinx 2 and Sphinx 3. + +.. + +.. bpo: 40979 +.. date: 2020-07-21-15-23-30 +.. nonce: pLA8rO +.. section: Documentation + +Refactored typing.rst, arranging more than 70 classes, functions, and +decorators into new sub-sections. + +.. + +.. bpo: 39883 +.. date: 2020-03-07-03-53-39 +.. nonce: 1tnb4- +.. section: Documentation + +Make code, examples, and recipes in the Python documentation be licensed +under the more permissive BSD0 license in addition to the existing Python +2.0 license. + +.. + +.. bpo: 41731 +.. date: 2020-09-11-19-12-31 +.. nonce: Ivxh4U +.. section: Tests + +Make test_cmd_line_script pass with option '-vv'. + +.. + +.. bpo: 41617 +.. date: 2020-08-24-18-34-01 +.. nonce: sKKXz7 +.. section: Build + +Fix ``pycore_byteswap.h`` header file to support old clang versions: +``__builtin_bswap16()`` is not available in LLVM clang 3.0. + +.. + +.. bpo: 41526 +.. date: 2020-08-13-22-40-58 +.. nonce: -i2bwb +.. section: Windows + +Fixed layout of final page of the installer by removing the special thanks +to Mark Hammond (with his permission). + +.. + +.. bpo: 41524 +.. date: 2020-08-12-17-09-06 +.. nonce: u6Xfr2 +.. section: C API + +Fix bug in PyOS_mystrnicmp and PyOS_mystricmp that incremented pointers +beyond the end of a string. diff --git a/Misc/NEWS.d/next/Build/2020-08-24-18-34-01.bpo-41617.sKKXz7.rst b/Misc/NEWS.d/next/Build/2020-08-24-18-34-01.bpo-41617.sKKXz7.rst deleted file mode 100644 index 9e4bc60fd8565..0000000000000 --- a/Misc/NEWS.d/next/Build/2020-08-24-18-34-01.bpo-41617.sKKXz7.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix ``pycore_byteswap.h`` header file to support old clang versions: -``__builtin_bswap16()`` is not available in LLVM clang 3.0. diff --git a/Misc/NEWS.d/next/C API/2020-08-12-17-09-06.bpo-41524.u6Xfr2.rst b/Misc/NEWS.d/next/C API/2020-08-12-17-09-06.bpo-41524.u6Xfr2.rst deleted file mode 100644 index 4704e29be29bb..0000000000000 --- a/Misc/NEWS.d/next/C API/2020-08-12-17-09-06.bpo-41524.u6Xfr2.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix bug in PyOS_mystrnicmp and PyOS_mystricmp that incremented -pointers beyond the end of a string. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-05-03-22-26-00.bpo-29590.aRz3l7.rst b/Misc/NEWS.d/next/Core and Builtins/2020-05-03-22-26-00.bpo-29590.aRz3l7.rst deleted file mode 100644 index 2570c4f2c7c0f..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-05-03-22-26-00.bpo-29590.aRz3l7.rst +++ /dev/null @@ -1,2 +0,0 @@ -Make the stack trace correct after calling :meth:`generator.throw` -on a generator that has yielded from a ``yield from``. diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst deleted file mode 100644 index acc00f8b992c9..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst +++ /dev/null @@ -1 +0,0 @@ -The output of ``python --help`` contains now only ASCII characters. diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-19-32-15.bpo-41531.WgPzjT.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-19-32-15.bpo-41531.WgPzjT.rst deleted file mode 100644 index 8544664f39335..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-19-32-15.bpo-41531.WgPzjT.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix a bug that was dropping keys when compiling dict literals with more than -0xFFFF elements. Patch by Pablo Galindo. diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-41533.4pcVAc.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-41533.4pcVAc.rst deleted file mode 100644 index e166f0c0b621a..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-20-29-57.bpo-41533.4pcVAc.rst +++ /dev/null @@ -1,2 +0,0 @@ -Free the stack allocated in ``va_build_stack`` if ``do_mkstack`` fails and -the stack is not a ``small_stack``. diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst deleted file mode 100644 index 68bb51024d9e7..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-08-26-11-23-31.bpo-41631.3jZcd9.rst +++ /dev/null @@ -1,5 +0,0 @@ -The ``_ast`` module uses again a global state. Using a module state per module -instance is causing subtle practical problems. For example, the Mercurial -project replaces the ``__import__()`` function to implement lazy import, -whereas Python expected that ``import _ast`` always return a fully initialized -``_ast`` module. diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst deleted file mode 100644 index e05c3133e1262..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-08-30-20-38-33.bpo-41654.HtnhAM.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix a crash that occurred when destroying subclasses of -:class:`MemoryError`. Patch by Pablo Galindo. diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-31-17-49-02.bpo-41681.3-VJiH.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-31-17-49-02.bpo-41681.3-VJiH.rst deleted file mode 100644 index ed557f92d85ca..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-08-31-17-49-02.bpo-41681.3-VJiH.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fixes the wrong error description in the error raised by using 2 `,` in -format string in f-string and :meth:`str.format`. diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-02-12-00-57.bpo-41690.Ny-Sfy.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-02-12-00-57.bpo-41690.Ny-Sfy.rst deleted file mode 100644 index 5711aa5a55f07..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-09-02-12-00-57.bpo-41690.Ny-Sfy.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix a possible stack overflow in the parser when parsing functions and -classes with a huge ammount of arguments. Patch by Pablo Galindo. diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst deleted file mode 100644 index 9a7594fc45338..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-09-15-23-29-49.bpo-41780.bOBUIH.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix :meth:`__dir__` of :class:`types.GenericAlias`. Patch by Batuhan -Taskaya. diff --git a/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst b/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst deleted file mode 100644 index 4941d50a560e2..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-03-07-03-53-39.bpo-39883.1tnb4-.rst +++ /dev/null @@ -1 +0,0 @@ -Make code, examples, and recipes in the Python documentation be licensed under the more permissive BSD0 license in addition to the existing Python 2.0 license. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst b/Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst deleted file mode 100644 index b0ca4327ad61a..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst +++ /dev/null @@ -1 +0,0 @@ -Refactored typing.rst, arranging more than 70 classes, functions, and decorators into new sub-sections. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Documentation/2020-08-12-18-35-40.bpo-40204.C8A_pe.rst b/Misc/NEWS.d/next/Documentation/2020-08-12-18-35-40.bpo-40204.C8A_pe.rst deleted file mode 100644 index 152f6c98b9004..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-08-12-18-35-40.bpo-40204.C8A_pe.rst +++ /dev/null @@ -1,3 +0,0 @@ -Enable Sphinx 3.2 ``c_allow_pre_v3`` option and disable -``c_warn_on_allowed_pre_v3`` option to make the documentation compatible -with Sphinx 2 and Sphinx 3. diff --git a/Misc/NEWS.d/next/Documentation/2020-08-25-15-11-23.bpo-41624.ddjJlN.rst b/Misc/NEWS.d/next/Documentation/2020-08-25-15-11-23.bpo-41624.ddjJlN.rst deleted file mode 100644 index bdbc5a445f339..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-08-25-15-11-23.bpo-41624.ddjJlN.rst +++ /dev/null @@ -1 +0,0 @@ -Fix the signature of :class:`typing.Coroutine`. diff --git a/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst b/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst deleted file mode 100644 index aeca652b4ed97..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst +++ /dev/null @@ -1 +0,0 @@ -Change Shipman tkinter doc link from archive.org to TkDocs. (The doc has been removed from the NMT server.) The new link responds much faster and includes a short explanatory note. diff --git a/Misc/NEWS.d/next/Library/2018-06-12-23-30-41.bpo-33660.AdDn5Z.rst b/Misc/NEWS.d/next/Library/2018-06-12-23-30-41.bpo-33660.AdDn5Z.rst deleted file mode 100644 index cce3dbb1c6ea5..0000000000000 --- a/Misc/NEWS.d/next/Library/2018-06-12-23-30-41.bpo-33660.AdDn5Z.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix pathlib.PosixPath to resolve a relative path located on the root -directory properly. diff --git a/Misc/NEWS.d/next/Library/2020-03-11-07-44-06.bpo-31122.zIQ80l.rst b/Misc/NEWS.d/next/Library/2020-03-11-07-44-06.bpo-31122.zIQ80l.rst deleted file mode 100644 index 2e70f7aee65c8..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-03-11-07-44-06.bpo-31122.zIQ80l.rst +++ /dev/null @@ -1 +0,0 @@ -ssl.wrap_socket() now raises ssl.SSLEOFError rather than OSError when peer closes connection during TLS negotiation \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2020-06-18-10-34-59.bpo-41025.elf_nz.rst b/Misc/NEWS.d/next/Library/2020-06-18-10-34-59.bpo-41025.elf_nz.rst deleted file mode 100644 index 21e184d0a4063..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-06-18-10-34-59.bpo-41025.elf_nz.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fixed an issue preventing the C implementation of :class:`zoneinfo.ZoneInfo` -from being subclassed. diff --git a/Misc/NEWS.d/next/Library/2020-07-20-13-27-48.bpo-41344.iKipNd.rst b/Misc/NEWS.d/next/Library/2020-07-20-13-27-48.bpo-41344.iKipNd.rst deleted file mode 100644 index 475bc9bddb0d5..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-07-20-13-27-48.bpo-41344.iKipNd.rst +++ /dev/null @@ -1 +0,0 @@ -Prevent creating :class:`shared_memory.SharedMemory` objects with :code:`size=0`. diff --git a/Misc/NEWS.d/next/Library/2020-08-07-15-18-16.bpo-41503.IYftcu.rst b/Misc/NEWS.d/next/Library/2020-08-07-15-18-16.bpo-41503.IYftcu.rst deleted file mode 100644 index c34996d881937..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-07-15-18-16.bpo-41503.IYftcu.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed a race between setTarget and flush in logging.handlers.MemoryHandler. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2020-08-12-13-25-16.bpo-41520.BEUWa4.rst b/Misc/NEWS.d/next/Library/2020-08-12-13-25-16.bpo-41520.BEUWa4.rst deleted file mode 100644 index 0e140d91bb4b6..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-12-13-25-16.bpo-41520.BEUWa4.rst +++ /dev/null @@ -1 +0,0 @@ -Fix :mod:`codeop` regression that prevented turning compile warnings into errors. diff --git a/Misc/NEWS.d/next/Library/2020-08-13-08-07-25.bpo-40782.aGZqmB.rst b/Misc/NEWS.d/next/Library/2020-08-13-08-07-25.bpo-40782.aGZqmB.rst deleted file mode 100644 index d4c7e0e2419df..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-13-08-07-25.bpo-40782.aGZqmB.rst +++ /dev/null @@ -1 +0,0 @@ -Change the method asyncio.AbstractEventLoop.run_in_executor to not be a coroutine. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2020-08-15-15-21-40.bpo-37658.f9nivB.rst b/Misc/NEWS.d/next/Library/2020-08-15-15-21-40.bpo-37658.f9nivB.rst deleted file mode 100644 index 694fbbbf346dc..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-15-15-21-40.bpo-37658.f9nivB.rst +++ /dev/null @@ -1,2 +0,0 @@ -:meth:`asyncio.wait_for` now properly handles races between cancellation of -itself and the completion of the wrapped awaitable. diff --git a/Misc/NEWS.d/next/Library/2020-08-15-15-50-12.bpo-32751.85je5X.rst b/Misc/NEWS.d/next/Library/2020-08-15-15-50-12.bpo-32751.85je5X.rst deleted file mode 100644 index c172ce5d9e948..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-15-15-50-12.bpo-32751.85je5X.rst +++ /dev/null @@ -1,3 +0,0 @@ -When cancelling the task due to a timeout, :meth:`asyncio.wait_for` will now -wait until the cancellation is complete also in the case when *timeout* is -<= 0, like it does with positive timeouts. diff --git a/Misc/NEWS.d/next/Library/2020-08-21-15-51-15.bpo-41609.JmiUKG.rst b/Misc/NEWS.d/next/Library/2020-08-21-15-51-15.bpo-41609.JmiUKG.rst deleted file mode 100644 index ecaf40eee7bab..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-21-15-51-15.bpo-41609.JmiUKG.rst +++ /dev/null @@ -1 +0,0 @@ -The pdb whatis command correctly reports instance methods as 'Method' rather than 'Function'. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rst b/Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rst deleted file mode 100644 index 0d9015b490e4c..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-08-30-10-24-26.bpo-39010._mzXJW.rst +++ /dev/null @@ -1,2 +0,0 @@ -Restarting a ``ProactorEventLoop`` on Windows no longer logs spurious -``ConnectionResetErrors``. diff --git a/Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rst b/Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rst deleted file mode 100644 index 284f500735701..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-01-15-57-51.bpo-41687.m1b1KA.rst +++ /dev/null @@ -1 +0,0 @@ -Fix implementation of sendfile to be compatible with Solaris. diff --git a/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst b/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst deleted file mode 100644 index 67bbbb857f18c..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-03-01-35-32.bpo-41696.zkYGre.rst +++ /dev/null @@ -1 +0,0 @@ -Fix handling of debug mode in :func:`asyncio.run`. This allows setting ``PYTHONASYNCIODEBUG`` or ``-X dev`` to enable asyncio debug mode when using :func:`asyncio.run`. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst b/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst deleted file mode 100644 index 5d2a5094ddeaa..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fixed :meth:`turtle.Vec2D.__rmul__` for arguments which are not int or -float. diff --git a/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst b/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst deleted file mode 100644 index 78dcff1370029..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst +++ /dev/null @@ -1,3 +0,0 @@ -Fix a race condition in the ``call_soon_threadsafe()`` method of -``asyncio.ProactorEventLoop``: do nothing if the self-pipe socket has been -closed. diff --git a/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst b/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst deleted file mode 100644 index 5ce7a3ca67b72..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst +++ /dev/null @@ -1,2 +0,0 @@ -Honor `object` overrides in `Enum` class creation (specifically, `__str__`, -`__repr__`, `__format__`, and `__reduce_ex__`). diff --git a/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst b/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst deleted file mode 100644 index e2f2b64867bed..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst +++ /dev/null @@ -1 +0,0 @@ -use the correct mix-in data type when constructing Enums diff --git a/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst b/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst deleted file mode 100644 index e7654711062ce..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst +++ /dev/null @@ -1 +0,0 @@ -fix bug allowing Enums to be extended via multiple inheritance diff --git a/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst b/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst deleted file mode 100644 index e368a60f77b1e..0000000000000 --- a/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst +++ /dev/null @@ -1 +0,0 @@ -Make test_cmd_line_script pass with option '-vv'. diff --git a/Misc/NEWS.d/next/Windows/2020-08-13-22-40-58.bpo-41526.-i2bwb.rst b/Misc/NEWS.d/next/Windows/2020-08-13-22-40-58.bpo-41526.-i2bwb.rst deleted file mode 100644 index 756c8270599f2..0000000000000 --- a/Misc/NEWS.d/next/Windows/2020-08-13-22-40-58.bpo-41526.-i2bwb.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fixed layout of final page of the installer by removing the special thanks -to Mark Hammond (with his permission). diff --git a/README.rst b/README.rst index ae09fea3469f4..b55529be4ae61 100644 --- a/README.rst +++ b/README.rst @@ -1,4 +1,4 @@ -This is Python version 3.9.0 release candidate 1 +This is Python version 3.9.0 release candidate 2 ================================================ .. image:: https://travis-ci.org/python/cpython.svg?branch=3.9 From webhook-mailer at python.org Thu Sep 17 20:47:44 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Fri, 18 Sep 2020 00:47:44 -0000 Subject: [Python-checkins] [3.9] bpo-27032, bpo-37328: Document removing HTMLParser.unescape() (GH-22288) Message-ID: https://github.com/python/cpython/commit/4daeb9eec294f73a5811ef6ab3a0f8e61174a24c commit: 4daeb9eec294f73a5811ef6ab3a0f8e61174a24c branch: 3.9 author: Terry Jan Reedy committer: GitHub date: 2020-09-17T20:47:22-04:00 summary: [3.9] bpo-27032, bpo-37328: Document removing HTMLParser.unescape() (GH-22288) (cherry picked from commit a33f2c2bae759fc9d06e1c032fd2026135f2df45) Co-authored-by: Serhiy Storchaka files: M Doc/whatsnew/3.9.rst diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 2b7cbaa48531a..40c1592af4d5c 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -892,6 +892,11 @@ Removed :func:`asyncio.current_task` and :func:`asyncio.all_tasks` instead. (Contributed by R?mi Lapeyre in :issue:`40967`) +* The ``unescape()`` method in the :class:`html.parser.HTMLParser` class + has been removed (it was deprecated since Python 3.4). :func:`html.unescape` + should be used for converting character references to the corresponding + unicode characters. + Porting to Python 3.9 ===================== From webhook-mailer at python.org Thu Sep 17 21:57:13 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Fri, 18 Sep 2020 01:57:13 -0000 Subject: [Python-checkins] bpo-41808: Add What's New 3.9 entry missing from master (#22294) Message-ID: https://github.com/python/cpython/commit/5a8364780b7e881385f6fabcf072d599e80f51b8 commit: 5a8364780b7e881385f6fabcf072d599e80f51b8 branch: master author: Terry Jan Reedy committer: GitHub date: 2020-09-17T21:56:58-04:00 summary: bpo-41808: Add What's New 3.9 entry missing from master (#22294) Entry was added by bpo-40939, #21012 and #21039. files: M Doc/whatsnew/3.9.rst diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index e1ea799b76296..b3cc84d07f38d 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -708,6 +708,11 @@ Deprecated users can leverage the Abstract Syntax Tree (AST) generation and compilation stage, using the :mod:`ast` module. +* The Public C API functions :c:func:`PyParser_SimpleParseStringFlags`, + :c:func:`PyParser_SimpleParseStringFlagsFilename`, + :c:func:`PyParser_SimpleParseFileFlags` and :c:func:`PyNode_Compile` + are deprecated and will be removed in Python 3.10 together with the old parser. + * Using :data:`NotImplemented` in a boolean context has been deprecated, as it is almost exclusively the result of incorrect rich comparator implementations. It will be made a :exc:`TypeError` in a future version From webhook-mailer at python.org Fri Sep 18 02:36:22 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Fri, 18 Sep 2020 06:36:22 -0000 Subject: [Python-checkins] [3.9] Remove duplicated words words (GH-20413). (GH-22296) Message-ID: https://github.com/python/cpython/commit/234ae8fe1b7ff0a00d41ad86d46dd753a235279e commit: 234ae8fe1b7ff0a00d41ad86d46dd753a235279e branch: 3.9 author: Serhiy Storchaka committer: GitHub date: 2020-09-18T09:36:14+03:00 summary: [3.9] Remove duplicated words words (GH-20413). (GH-22296) (cherry picked from commit 1c5d1d7304a119040fd3118128bdb412f0cce6a6) files: M Doc/library/ssl.rst M Doc/whatsnew/3.8.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/3.8.0a1.rst M Misc/NEWS.d/3.8.0b1.rst diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 952ee1653b924..852091c02ec9a 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -615,7 +615,7 @@ Constants Possible value for :attr:`SSLContext.verify_flags`. In this mode, only the peer cert is checked but none of the intermediate CA certificates. The mode requires a valid CRL that is signed by the peer cert's issuer (its direct - ancestor CA). If no proper CRL has has been loaded with + ancestor CA). If no proper CRL has been loaded with :attr:`SSLContext.load_verify_locations`, validation will fail. .. versionadded:: 3.4 diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 61aca479889b9..a2fa17811b3fc 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1693,7 +1693,7 @@ Deprecated :meth:`~gettext.NullTranslations.set_output_charset`, and the *codeset* parameter of functions :func:`~gettext.translation` and :func:`~gettext.install` are also deprecated, since they are only used for - for the ``l*gettext()`` functions. + the ``l*gettext()`` functions. (Contributed by Serhiy Storchaka in :issue:`33710`.) * The :meth:`~threading.Thread.isAlive()` method of :class:`threading.Thread` diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 40c1592af4d5c..b3cc84d07f38d 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -879,7 +879,7 @@ Removed (Contributed by Victor Stinner in :issue:`39489`.) * The ``_field_types`` attribute of the :class:`typing.NamedTuple` class - has been removed. It was deprecated deprecated since Python 3.8. Use + has been removed. It was deprecated since Python 3.8. Use the ``__annotations__`` attribute instead. (Contributed by Serhiy Storchaka in :issue:`40182`.) diff --git a/Misc/NEWS.d/3.8.0a1.rst b/Misc/NEWS.d/3.8.0a1.rst index c5ab5f4489c92..b7164a5b4e0fa 100644 --- a/Misc/NEWS.d/3.8.0a1.rst +++ b/Misc/NEWS.d/3.8.0a1.rst @@ -7034,7 +7034,7 @@ Fix ftplib test for TLS 1.3 by reading from data socket. .. nonce: g7TwYm .. section: Tests -Fix `test_socket` on AIX AIX 6.1 and later IPv6 zone id supports only +Fix `test_socket` on AIX 6.1 and later IPv6 zone id supports only supported by inet_pton6_zone() Switch to runtime-based platform.system() to establish current platform rather than build-time based sys.platform() diff --git a/Misc/NEWS.d/3.8.0b1.rst b/Misc/NEWS.d/3.8.0b1.rst index 43a88a37c5cb0..5400c07795292 100644 --- a/Misc/NEWS.d/3.8.0b1.rst +++ b/Misc/NEWS.d/3.8.0b1.rst @@ -1612,7 +1612,7 @@ versions. .. nonce: Zot4sx .. section: Documentation -Improve documentation of the stdin, stdout, and stderr arguments of of the +Improve documentation of the stdin, stdout, and stderr arguments of the ``asyncio.subprocess_exec`` function to specify which values are supported. Also mention that decoding as text is not supported. From webhook-mailer at python.org Fri Sep 18 02:37:08 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Fri, 18 Sep 2020 06:37:08 -0000 Subject: [Python-checkins] [3.8] Remove duplicated words words (GH-20413). (GH-22297) Message-ID: https://github.com/python/cpython/commit/038688a53bdd92eafd1edf47e6bed414b125a01e commit: 038688a53bdd92eafd1edf47e6bed414b125a01e branch: 3.8 author: Serhiy Storchaka committer: GitHub date: 2020-09-18T09:37:04+03:00 summary: [3.8] Remove duplicated words words (GH-20413). (GH-22297) (cherry picked from commit 1c5d1d7304a119040fd3118128bdb412f0cce6a6) files: M Doc/library/ssl.rst M Doc/whatsnew/3.8.rst M Misc/NEWS.d/3.8.0a1.rst M Misc/NEWS.d/3.8.0b1.rst diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index ea0f13c4f18b9..04523e914072e 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -615,7 +615,7 @@ Constants Possible value for :attr:`SSLContext.verify_flags`. In this mode, only the peer cert is checked but none of the intermediate CA certificates. The mode requires a valid CRL that is signed by the peer cert's issuer (its direct - ancestor CA). If no proper CRL has has been loaded with + ancestor CA). If no proper CRL has been loaded with :attr:`SSLContext.load_verify_locations`, validation will fail. .. versionadded:: 3.4 diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 475e3ab964312..7a86e51116f41 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1695,7 +1695,7 @@ Deprecated :meth:`~gettext.NullTranslations.set_output_charset`, and the *codeset* parameter of functions :func:`~gettext.translation` and :func:`~gettext.install` are also deprecated, since they are only used for - for the ``l*gettext()`` functions. + the ``l*gettext()`` functions. (Contributed by Serhiy Storchaka in :issue:`33710`.) * The :meth:`~threading.Thread.isAlive()` method of :class:`threading.Thread` diff --git a/Misc/NEWS.d/3.8.0a1.rst b/Misc/NEWS.d/3.8.0a1.rst index 96208c88d32b3..873f53b450376 100644 --- a/Misc/NEWS.d/3.8.0a1.rst +++ b/Misc/NEWS.d/3.8.0a1.rst @@ -7034,7 +7034,7 @@ Fix ftplib test for TLS 1.3 by reading from data socket. .. nonce: g7TwYm .. section: Tests -Fix `test_socket` on AIX AIX 6.1 and later IPv6 zone id supports only +Fix `test_socket` on AIX 6.1 and later IPv6 zone id supports only supported by inet_pton6_zone() Switch to runtime-based platform.system() to establish current platform rather than build-time based sys.platform() diff --git a/Misc/NEWS.d/3.8.0b1.rst b/Misc/NEWS.d/3.8.0b1.rst index 43a88a37c5cb0..5400c07795292 100644 --- a/Misc/NEWS.d/3.8.0b1.rst +++ b/Misc/NEWS.d/3.8.0b1.rst @@ -1612,7 +1612,7 @@ versions. .. nonce: Zot4sx .. section: Documentation -Improve documentation of the stdin, stdout, and stderr arguments of of the +Improve documentation of the stdin, stdout, and stderr arguments of the ``asyncio.subprocess_exec`` function to specify which values are supported. Also mention that decoding as text is not supported. From webhook-mailer at python.org Fri Sep 18 02:54:56 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Fri, 18 Sep 2020 06:54:56 -0000 Subject: [Python-checkins] Remove duplicated words words (GH-22298) Message-ID: https://github.com/python/cpython/commit/27201cddf3b25be7df25bbe36966531539757d60 commit: 27201cddf3b25be7df25bbe36966531539757d60 branch: master author: Serhiy Storchaka committer: GitHub date: 2020-09-18T09:54:42+03:00 summary: Remove duplicated words words (GH-22298) files: M Doc/library/tkinter.font.rst M Doc/whatsnew/3.9.rst M Misc/NEWS.d/next/Library/2020-06-02-23-49-07.bpo-32604.ZN4V4l.rst M Misc/NEWS.d/next/Library/2020-06-04-16-25-15.bpo-40807.yYyLWx.rst diff --git a/Doc/library/tkinter.font.rst b/Doc/library/tkinter.font.rst index 30c1e7b5f9eb4..b0f4505e9e3c6 100644 --- a/Doc/library/tkinter.font.rst +++ b/Doc/library/tkinter.font.rst @@ -38,8 +38,8 @@ The different font weights and slants are: | *family* - font family i.e. Courier, Times | *size* - font size | If *size* is positive it is interpreted as size in points. - | If *size* is a negative number its absolute value is treated as - as size in pixels. + | If *size* is a negative number its absolute value is treated + | as size in pixels. | *weight* - font emphasis (NORMAL, BOLD) | *slant* - ROMAN, ITALIC | *underline* - font underlining (0 - none, 1 - underline) diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index b3cc84d07f38d..e9fc496e47e24 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -887,7 +887,7 @@ Removed deprecated since 2006, and only returning ``False`` when it's called. (Contributed by Batuhan Taskaya in :issue:`40208`) -* The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks` have +* The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks` have been removed. They were deprecated since Python 3.7 and you can use :func:`asyncio.current_task` and :func:`asyncio.all_tasks` instead. (Contributed by R?mi Lapeyre in :issue:`40967`) diff --git a/Misc/NEWS.d/next/Library/2020-06-02-23-49-07.bpo-32604.ZN4V4l.rst b/Misc/NEWS.d/next/Library/2020-06-02-23-49-07.bpo-32604.ZN4V4l.rst index 6375276602e4a..af284b06eaed6 100644 --- a/Misc/NEWS.d/next/Library/2020-06-02-23-49-07.bpo-32604.ZN4V4l.rst +++ b/Misc/NEWS.d/next/Library/2020-06-02-23-49-07.bpo-32604.ZN4V4l.rst @@ -1,2 +1,2 @@ -Fix reference leak in the :mod:`select` module when the the module is +Fix reference leak in the :mod:`select` module when the module is imported in a subinterpreter. diff --git a/Misc/NEWS.d/next/Library/2020-06-04-16-25-15.bpo-40807.yYyLWx.rst b/Misc/NEWS.d/next/Library/2020-06-04-16-25-15.bpo-40807.yYyLWx.rst index 532b809b77eed..c64a86295d770 100644 --- a/Misc/NEWS.d/next/Library/2020-06-04-16-25-15.bpo-40807.yYyLWx.rst +++ b/Misc/NEWS.d/next/Library/2020-06-04-16-25-15.bpo-40807.yYyLWx.rst @@ -1,2 +1,2 @@ Stop codeop._maybe_compile, used by code.InteractiveInterpreter (and IDLE). -from from emitting each warning three times. +from emitting each warning three times. From webhook-mailer at python.org Fri Sep 18 03:10:25 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 18 Sep 2020 07:10:25 -0000 Subject: [Python-checkins] bpo-41762: Fix usage of productionlist markup in the doc (GH-22281) Message-ID: https://github.com/python/cpython/commit/8af239eacfcf52e4e0e2b0223e7cea4672309483 commit: 8af239eacfcf52e4e0e2b0223e7cea4672309483 branch: master author: Victor Stinner committer: GitHub date: 2020-09-18T09:10:15+02:00 summary: bpo-41762: Fix usage of productionlist markup in the doc (GH-22281) Use an unique identifier for the different grammars documented using the Sphinx productionlist markup. productionlist markups of the same grammar, like "expressions" or "compound statements", use the same identifier "python-grammar". files: M Doc/library/functions.rst M Doc/library/string.rst M Doc/reference/compound_stmts.rst M Doc/reference/expressions.rst M Doc/reference/introduction.rst M Doc/reference/lexical_analysis.rst M Doc/reference/simple_stmts.rst M Doc/reference/toplevel_components.rst diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index d381d43d8e99c..7543fc4b10d46 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -596,7 +596,7 @@ are always available. They are listed here in alphabetical order. input must conform to the following grammar after leading and trailing whitespace characters are removed: - .. productionlist:: + .. productionlist:: float sign: "+" | "-" infinity: "Infinity" | "inf" nan: "nan" diff --git a/Doc/library/string.rst b/Doc/library/string.rst index 62e86d6dd9706..91f43e9353d91 100644 --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -205,7 +205,7 @@ literal text, it can be escaped by doubling: ``{{`` and ``}}``. The grammar for a replacement field is as follows: - .. productionlist:: sf + .. productionlist:: format-string replacement_field: "{" [`field_name`] ["!" `conversion`] [":" `format_spec`] "}" field_name: arg_name ("." `attribute_name` | "[" `element_index` "]")* arg_name: [`identifier` | `digit`+] @@ -308,7 +308,7 @@ non-empty format specification typically modifies the result. The general form of a *standard format specifier* is: -.. productionlist:: +.. productionlist:: format-spec format_spec: [[`fill`]`align`][`sign`][#][0][`width`][`grouping_option`][.`precision`][`type`] fill: align: "<" | ">" | "=" | "^" diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index df720f6cc3238..158d6a8f164e2 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -44,7 +44,8 @@ executed:: Summarizing: -.. productionlist:: + +.. productionlist:: python-grammar compound_stmt: `if_stmt` : | `while_stmt` : | `for_stmt` @@ -89,7 +90,7 @@ The :keyword:`!if` statement The :keyword:`if` statement is used for conditional execution: -.. productionlist:: +.. productionlist:: python-grammar if_stmt: "if" `assignment_expression` ":" `suite` : ("elif" `assignment_expression` ":" `suite`)* : ["else" ":" `suite`] @@ -115,7 +116,7 @@ The :keyword:`!while` statement The :keyword:`while` statement is used for repeated execution as long as an expression is true: -.. productionlist:: +.. productionlist:: python-grammar while_stmt: "while" `assignment_expression` ":" `suite` : ["else" ":" `suite`] @@ -151,7 +152,7 @@ The :keyword:`!for` statement The :keyword:`for` statement is used to iterate over the elements of a sequence (such as a string, tuple or list) or other iterable object: -.. productionlist:: +.. productionlist:: python-grammar for_stmt: "for" `target_list` "in" `expression_list` ":" `suite` : ["else" ":" `suite`] @@ -234,7 +235,7 @@ The :keyword:`!try` statement The :keyword:`try` statement specifies exception handlers and/or cleanup code for a group of statements: -.. productionlist:: +.. productionlist:: python-grammar try_stmt: `try1_stmt` | `try2_stmt` try1_stmt: "try" ":" `suite` : ("except" [`expression` ["as" `identifier`]] ":" `suite`)+ @@ -390,7 +391,7 @@ methods defined by a context manager (see section :ref:`context-managers`). This allows common :keyword:`try`...\ :keyword:`except`...\ :keyword:`finally` usage patterns to be encapsulated for convenient reuse. -.. productionlist:: +.. productionlist:: python-grammar with_stmt: "with" `with_item` ("," `with_item`)* ":" `suite` with_item: `expression` ["as" `target`] @@ -503,7 +504,7 @@ Function definitions A function definition defines a user-defined function object (see section :ref:`types`): -.. productionlist:: +.. productionlist:: python-grammar funcdef: [`decorators`] "def" `funcname` "(" [`parameter_list`] ")" : ["->" `expression`] ":" `suite` decorators: `decorator`+ @@ -670,7 +671,7 @@ Class definitions A class definition defines a class object (see section :ref:`types`): -.. productionlist:: +.. productionlist:: python-grammar classdef: [`decorators`] "class" `classname` [`inheritance`] ":" `suite` inheritance: "(" [`argument_list`] ")" classname: `identifier` @@ -762,7 +763,7 @@ Coroutines Coroutine function definition ----------------------------- -.. productionlist:: +.. productionlist:: python-grammar async_funcdef: [`decorators`] "async" "def" `funcname` "(" [`parameter_list`] ")" : ["->" `expression`] ":" `suite` @@ -795,7 +796,7 @@ An example of a coroutine function:: The :keyword:`!async for` statement ----------------------------------- -.. productionlist:: +.. productionlist:: python-grammar async_for_stmt: "async" `for_stmt` An :term:`asynchronous iterable` is able to call asynchronous code in its @@ -840,7 +841,7 @@ body of a coroutine function. The :keyword:`!async with` statement ------------------------------------ -.. productionlist:: +.. productionlist:: python-grammar async_with_stmt: "async" `with_stmt` An :term:`asynchronous context manager` is a :term:`context manager` that is diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 18abce3c510bd..b68c29860cf33 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -13,7 +13,7 @@ This chapter explains the meaning of the elements of expressions in Python. be used to describe syntax, not lexical analysis. When (one alternative of) a syntax rule has the form -.. productionlist:: * +.. productionlist:: python-grammar name: `othername` and no semantics are given, the semantics of this form of ``name`` are the same @@ -54,7 +54,7 @@ Atoms are the most basic elements of expressions. The simplest atoms are identifiers or literals. Forms enclosed in parentheses, brackets or braces are also categorized syntactically as atoms. The syntax for atoms is: -.. productionlist:: +.. productionlist:: python-grammar atom: `identifier` | `literal` | `enclosure` enclosure: `parenth_form` | `list_display` | `dict_display` | `set_display` : | `generator_expression` | `yield_atom` @@ -103,7 +103,7 @@ Literals Python supports string and bytes literals and various numeric literals: -.. productionlist:: +.. productionlist:: python-grammar literal: `stringliteral` | `bytesliteral` : | `integer` | `floatnumber` | `imagnumber` @@ -134,7 +134,7 @@ Parenthesized forms A parenthesized form is an optional expression list enclosed in parentheses: -.. productionlist:: +.. productionlist:: python-grammar parenth_form: "(" [`starred_expression`] ")" A parenthesized expression list yields whatever that expression list yields: if @@ -177,7 +177,7 @@ called "displays", each of them in two flavors: Common syntax elements for comprehensions are: -.. productionlist:: +.. productionlist:: python-grammar comprehension: `assignment_expression` `comp_for` comp_for: ["async"] "for" `target_list` "in" `or_test` [`comp_iter`] comp_iter: `comp_for` | `comp_if` @@ -243,7 +243,7 @@ List displays A list display is a possibly empty series of expressions enclosed in square brackets: -.. productionlist:: +.. productionlist:: python-grammar list_display: "[" [`starred_list` | `comprehension`] "]" A list display yields a new list object, the contents being specified by either @@ -267,7 +267,7 @@ Set displays A set display is denoted by curly braces and distinguishable from dictionary displays by the lack of colons separating keys and values: -.. productionlist:: +.. productionlist:: python-grammar set_display: "{" (`starred_list` | `comprehension`) "}" A set display yields a new mutable set object, the contents being specified by @@ -296,7 +296,7 @@ Dictionary displays A dictionary display is a possibly empty series of key/datum pairs enclosed in curly braces: -.. productionlist:: +.. productionlist:: python-grammar dict_display: "{" [`key_datum_list` | `dict_comprehension`] "}" key_datum_list: `key_datum` ("," `key_datum`)* [","] key_datum: `expression` ":" `expression` | "**" `or_expr` @@ -355,7 +355,7 @@ Generator expressions A generator expression is a compact generator notation in parentheses: -.. productionlist:: +.. productionlist:: python-grammar generator_expression: "(" `expression` `comp_for` ")" A generator expression yields a new generator object. Its syntax is the same as @@ -409,7 +409,7 @@ Yield expressions pair: yield; expression pair: generator; function -.. productionlist:: +.. productionlist:: python-grammar yield_atom: "(" `yield_expression` ")" yield_expression: "yield" [`expression_list` | "from" `expression`] @@ -746,7 +746,7 @@ Primaries Primaries represent the most tightly bound operations of the language. Their syntax is: -.. productionlist:: +.. productionlist:: python-grammar primary: `atom` | `attributeref` | `subscription` | `slicing` | `call` @@ -761,7 +761,7 @@ Attribute references An attribute reference is a primary followed by a period and a name: -.. productionlist:: +.. productionlist:: python-grammar attributeref: `primary` "." `identifier` .. index:: @@ -799,7 +799,7 @@ Subscriptions A subscription selects an item of a sequence (string, tuple or list) or mapping (dictionary) object: -.. productionlist:: +.. productionlist:: python-grammar subscription: `primary` "[" `expression_list` "]" The primary must evaluate to an object that supports subscription (lists or @@ -855,7 +855,7 @@ A slicing selects a range of items in a sequence object (e.g., a string, tuple or list). Slicings may be used as expressions or as targets in assignment or :keyword:`del` statements. The syntax for a slicing: -.. productionlist:: +.. productionlist:: python-grammar slicing: `primary` "[" `slice_list` "]" slice_list: `slice_item` ("," `slice_item`)* [","] slice_item: `expression` | `proper_slice` @@ -905,7 +905,7 @@ Calls A call calls a callable object (e.g., a :term:`function`) with a possibly empty series of :term:`arguments `: -.. productionlist:: +.. productionlist:: python-grammar call: `primary` "(" [`argument_list` [","] | `comprehension`] ")" argument_list: `positional_arguments` ["," `starred_and_keywords`] : ["," `keywords_arguments`] @@ -1088,7 +1088,7 @@ Await expression Suspend the execution of :term:`coroutine` on an :term:`awaitable` object. Can only be used inside a :term:`coroutine function`. -.. productionlist:: +.. productionlist:: python-grammar await_expr: "await" `primary` .. versionadded:: 3.5 @@ -1106,7 +1106,7 @@ The power operator The power operator binds more tightly than unary operators on its left; it binds less tightly than unary operators on its right. The syntax is: -.. productionlist:: +.. productionlist:: python-grammar power: (`await_expr` | `primary`) ["**" `u_expr`] Thus, in an unparenthesized sequence of power and unary operators, the operators @@ -1139,7 +1139,7 @@ Unary arithmetic and bitwise operations All unary arithmetic and bitwise operations have the same priority: -.. productionlist:: +.. productionlist:: python-grammar u_expr: `power` | "-" `u_expr` | "+" `u_expr` | "~" `u_expr` .. index:: @@ -1183,7 +1183,7 @@ that some of these operations also apply to certain non-numeric types. Apart from the power operator, there are only two levels, one for multiplicative operators and one for additive operators: -.. productionlist:: +.. productionlist:: python-grammar m_expr: `u_expr` | `m_expr` "*" `u_expr` | `m_expr` "@" `m_expr` | : `m_expr` "//" `u_expr` | `m_expr` "/" `u_expr` | : `m_expr` "%" `u_expr` @@ -1279,7 +1279,7 @@ Shifting operations The shifting operations have lower priority than the arithmetic operations: -.. productionlist:: +.. productionlist:: python-grammar shift_expr: `a_expr` | `shift_expr` ("<<" | ">>") `a_expr` These operators accept integers as arguments. They shift the first argument to @@ -1300,7 +1300,7 @@ Binary bitwise operations Each of the three bitwise operations has a different priority level: -.. productionlist:: +.. productionlist:: python-grammar and_expr: `shift_expr` | `and_expr` "&" `shift_expr` xor_expr: `and_expr` | `xor_expr` "^" `and_expr` or_expr: `xor_expr` | `or_expr` "|" `xor_expr` @@ -1349,7 +1349,7 @@ lower than that of any arithmetic, shifting or bitwise operation. Also unlike C, expressions like ``a < b < c`` have the interpretation that is conventional in mathematics: -.. productionlist:: +.. productionlist:: python-grammar comparison: `or_expr` (`comp_operator` `or_expr`)* comp_operator: "<" | ">" | "==" | ">=" | "<=" | "!=" : | "is" ["not"] | ["not"] "in" @@ -1608,7 +1608,7 @@ Boolean operations pair: Conditional; expression pair: Boolean; operation -.. productionlist:: +.. productionlist:: python-grammar or_test: `and_test` | `or_test` "or" `and_test` and_test: `not_test` | `and_test` "and" `not_test` not_test: `comparison` | "not" `not_test` @@ -1647,7 +1647,7 @@ returns a boolean value regardless of the type of its argument Assignment expressions ====================== -.. productionlist:: +.. productionlist:: python-grammar assignment_expression: [`identifier` ":="] `expression` An assignment expression (sometimes also called a "named expression" or @@ -1683,7 +1683,7 @@ Conditional expressions single: if; conditional expression single: else; conditional expression -.. productionlist:: +.. productionlist:: python-grammar conditional_expression: `or_test` ["if" `or_test` "else" `expression`] expression: `conditional_expression` | `lambda_expr` expression_nocond: `or_test` | `lambda_expr_nocond` @@ -1710,7 +1710,7 @@ Lambdas pair: anonymous; function single: : (colon); lambda expression -.. productionlist:: +.. productionlist:: python-grammar lambda_expr: "lambda" [`parameter_list`] ":" `expression` lambda_expr_nocond: "lambda" [`parameter_list`] ":" `expression_nocond` @@ -1737,7 +1737,7 @@ Expression lists pair: expression; list single: , (comma); expression list -.. productionlist:: +.. productionlist:: python-grammar expression_list: `expression` ("," `expression`)* [","] starred_list: `starred_item` ("," `starred_item`)* [","] starred_expression: `expression` | (`starred_item` ",")* [`starred_item`] diff --git a/Doc/reference/introduction.rst b/Doc/reference/introduction.rst index 62480bd7dd9a6..72e874ee98e46 100644 --- a/Doc/reference/introduction.rst +++ b/Doc/reference/introduction.rst @@ -93,7 +93,7 @@ Notation The descriptions of lexical analysis and syntax use a modified BNF grammar notation. This uses the following style of definition: -.. productionlist:: +.. productionlist:: notation name: `lc_letter` (`lc_letter` | "_")* lc_letter: "a"..."z" diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 19ba83a5513d8..77e0578f5d89b 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -296,7 +296,7 @@ Unicode Character Database as included in the :mod:`unicodedata` module. Identifiers are unlimited in length. Case is significant. -.. productionlist:: +.. productionlist:: python-grammar identifier: `xid_start` `xid_continue`* id_start: id_continue: @@ -412,7 +412,7 @@ String and Bytes literals String literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar stringliteral: [`stringprefix`](`shortstring` | `longstring`) stringprefix: "r" | "u" | "R" | "U" | "f" | "F" : | "fr" | "Fr" | "fR" | "FR" | "rf" | "rF" | "Rf" | "RF" @@ -424,7 +424,7 @@ String literals are described by the following lexical definitions: longstringchar: stringescapeseq: "\" -.. productionlist:: +.. productionlist:: python-grammar bytesliteral: `bytesprefix`(`shortbytes` | `longbytes`) bytesprefix: "b" | "B" | "br" | "Br" | "bR" | "BR" | "rb" | "rB" | "Rb" | "RB" shortbytes: "'" `shortbytesitem`* "'" | '"' `shortbytesitem`* '"' @@ -659,7 +659,7 @@ Escape sequences are decoded like in ordinary string literals (except when a literal is also marked as a raw string). After decoding, the grammar for the contents of the string is: -.. productionlist:: +.. productionlist:: python-grammar f_string: (`literal_char` | "{{" | "}}" | `replacement_field`)* replacement_field: "{" `f_expression` ["="] ["!" `conversion`] [":" `format_spec`] "}" f_expression: (`conditional_expression` | "*" `or_expr`) @@ -820,7 +820,7 @@ Integer literals Integer literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar integer: `decinteger` | `bininteger` | `octinteger` | `hexinteger` decinteger: `nonzerodigit` (["_"] `digit`)* | "0"+ (["_"] "0")* bininteger: "0" ("b" | "B") (["_"] `bindigit`)+ @@ -864,7 +864,7 @@ Floating point literals Floating point literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar floatnumber: `pointfloat` | `exponentfloat` pointfloat: [`digitpart`] `fraction` | `digitpart` "." exponentfloat: (`digitpart` | `pointfloat`) `exponent` @@ -894,7 +894,7 @@ Imaginary literals Imaginary literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar imagnumber: (`floatnumber` | `digitpart`) ("j" | "J") An imaginary literal yields a complex number with a real part of 0.0. Complex diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index a8ec0fbe8b732..93be32713ff32 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -11,7 +11,7 @@ A simple statement is comprised within a single logical line. Several simple statements may occur on a single line separated by semicolons. The syntax for simple statements is: -.. productionlist:: +.. productionlist:: python-grammar simple_stmt: `expression_stmt` : | `assert_stmt` : | `assignment_stmt` @@ -46,7 +46,7 @@ result; in Python, procedures return the value ``None``). Other uses of expression statements are allowed and occasionally useful. The syntax for an expression statement is: -.. productionlist:: +.. productionlist:: python-grammar expression_stmt: `starred_expression` An expression statement evaluates the expression list (which may be a single @@ -82,7 +82,7 @@ Assignment statements Assignment statements are used to (re)bind names to values and to modify attributes or items of mutable objects: -.. productionlist:: +.. productionlist:: python-grammar assignment_stmt: (`target_list` "=")+ (`starred_expression` | `yield_expression`) target_list: `target` ("," `target`)* [","] target: `identifier` @@ -280,7 +280,7 @@ Augmented assignment statements Augmented assignment is the combination, in a single statement, of a binary operation and an assignment statement: -.. productionlist:: +.. productionlist:: python-grammar augmented_assignment_stmt: `augtarget` `augop` (`expression_list` | `yield_expression`) augtarget: `identifier` | `attributeref` | `subscription` | `slicing` augop: "+=" | "-=" | "*=" | "@=" | "/=" | "//=" | "%=" | "**=" @@ -328,7 +328,7 @@ Annotated assignment statements :term:`Annotation ` assignment is the combination, in a single statement, of a variable or attribute annotation and an optional assignment statement: -.. productionlist:: +.. productionlist:: python-grammar annotated_assignment_stmt: `augtarget` ":" `expression` : ["=" (`starred_expression` | `yield_expression`)] @@ -385,7 +385,7 @@ The :keyword:`!assert` statement Assert statements are a convenient way to insert debugging assertions into a program: -.. productionlist:: +.. productionlist:: python-grammar assert_stmt: "assert" `expression` ["," `expression`] The simple form, ``assert expression``, is equivalent to :: @@ -425,7 +425,7 @@ The :keyword:`!pass` statement pair: null; operation pair: null; operation -.. productionlist:: +.. productionlist:: python-grammar pass_stmt: "pass" :keyword:`pass` is a null operation --- when it is executed, nothing happens. @@ -447,7 +447,7 @@ The :keyword:`!del` statement pair: deletion; target triple: deletion; target; list -.. productionlist:: +.. productionlist:: python-grammar del_stmt: "del" `target_list` Deletion is recursively defined very similar to the way assignment is defined. @@ -486,7 +486,7 @@ The :keyword:`!return` statement pair: function; definition pair: class; definition -.. productionlist:: +.. productionlist:: python-grammar return_stmt: "return" [`expression_list`] :keyword:`return` may only occur syntactically nested in a function definition, @@ -525,7 +525,7 @@ The :keyword:`!yield` statement single: function; generator exception: StopIteration -.. productionlist:: +.. productionlist:: python-grammar yield_stmt: `yield_expression` A :keyword:`yield` statement is semantically equivalent to a :ref:`yield @@ -560,7 +560,7 @@ The :keyword:`!raise` statement pair: raising; exception single: __traceback__ (exception attribute) -.. productionlist:: +.. productionlist:: python-grammar raise_stmt: "raise" [`expression` ["from" `expression`]] If no expressions are present, :keyword:`raise` re-raises the last exception @@ -663,7 +663,7 @@ The :keyword:`!break` statement statement: while pair: loop; statement -.. productionlist:: +.. productionlist:: python-grammar break_stmt: "break" :keyword:`break` may only occur syntactically nested in a :keyword:`for` or @@ -698,7 +698,7 @@ The :keyword:`!continue` statement pair: loop; statement keyword: finally -.. productionlist:: +.. productionlist:: python-grammar continue_stmt: "continue" :keyword:`continue` may only occur syntactically nested in a :keyword:`for` or @@ -725,7 +725,7 @@ The :keyword:`!import` statement exception: ImportError single: , (comma); import statement -.. productionlist:: +.. productionlist:: python-grammar import_stmt: "import" `module` ["as" `identifier`] ("," `module` ["as" `identifier`])* : | "from" `relative_module` "import" `identifier` ["as" `identifier`] : ("," `identifier` ["as" `identifier`])* @@ -859,7 +859,7 @@ that introduce incompatible changes to the language. It allows use of the new features on a per-module basis before the release in which the feature becomes standard. -.. productionlist:: * +.. productionlist:: python-grammar future_stmt: "from" "__future__" "import" `feature` ["as" `identifier`] : ("," `feature` ["as" `identifier`])* : | "from" "__future__" "import" "(" `feature` ["as" `identifier`] @@ -937,7 +937,7 @@ The :keyword:`!global` statement triple: global; name; binding single: , (comma); identifier list -.. productionlist:: +.. productionlist:: python-grammar global_stmt: "global" `identifier` ("," `identifier`)* The :keyword:`global` statement is a declaration which holds for the entire @@ -982,7 +982,7 @@ The :keyword:`!nonlocal` statement .. index:: statement: nonlocal single: , (comma); identifier list -.. productionlist:: +.. productionlist:: python-grammar nonlocal_stmt: "nonlocal" `identifier` ("," `identifier`)* .. XXX add when implemented diff --git a/Doc/reference/toplevel_components.rst b/Doc/reference/toplevel_components.rst index d5ffb37b2e58c..319c9de484241 100644 --- a/Doc/reference/toplevel_components.rst +++ b/Doc/reference/toplevel_components.rst @@ -66,7 +66,7 @@ File input All input read from non-interactive files has the same form: -.. productionlist:: +.. productionlist:: python-grammar file_input: (NEWLINE | `statement`)* This syntax is used in the following situations: @@ -85,7 +85,7 @@ Interactive input Input in interactive mode is parsed using the following grammar: -.. productionlist:: +.. productionlist:: python-grammar interactive_input: [`stmt_list`] NEWLINE | `compound_stmt` NEWLINE Note that a (top-level) compound statement must be followed by a blank line in @@ -103,5 +103,5 @@ Expression input :func:`eval` is used for expression input. It ignores leading whitespace. The string argument to :func:`eval` must have the following form: -.. productionlist:: +.. productionlist:: python-grammar eval_input: `expression_list` NEWLINE* From webhook-mailer at python.org Fri Sep 18 03:14:04 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Fri, 18 Sep 2020 07:14:04 -0000 Subject: [Python-checkins] [3.9] Remove duplicated words words (GH-22298). (GH-22299) Message-ID: https://github.com/python/cpython/commit/6d893bab77497fcbdc751dc074c69df93ecab908 commit: 6d893bab77497fcbdc751dc074c69df93ecab908 branch: 3.9 author: Serhiy Storchaka committer: GitHub date: 2020-09-18T10:13:57+03:00 summary: [3.9] Remove duplicated words words (GH-22298). (GH-22299) (cherry picked from commit 27201cddf3b25be7df25bbe36966531539757d60) files: M Doc/library/tkinter.font.rst M Doc/whatsnew/3.9.rst diff --git a/Doc/library/tkinter.font.rst b/Doc/library/tkinter.font.rst index 30c1e7b5f9eb4..b0f4505e9e3c6 100644 --- a/Doc/library/tkinter.font.rst +++ b/Doc/library/tkinter.font.rst @@ -38,8 +38,8 @@ The different font weights and slants are: | *family* - font family i.e. Courier, Times | *size* - font size | If *size* is positive it is interpreted as size in points. - | If *size* is a negative number its absolute value is treated as - as size in pixels. + | If *size* is a negative number its absolute value is treated + | as size in pixels. | *weight* - font emphasis (NORMAL, BOLD) | *slant* - ROMAN, ITALIC | *underline* - font underlining (0 - none, 1 - underline) diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index b3cc84d07f38d..e9fc496e47e24 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -887,7 +887,7 @@ Removed deprecated since 2006, and only returning ``False`` when it's called. (Contributed by Batuhan Taskaya in :issue:`40208`) -* The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks` have +* The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks` have been removed. They were deprecated since Python 3.7 and you can use :func:`asyncio.current_task` and :func:`asyncio.all_tasks` instead. (Contributed by R?mi Lapeyre in :issue:`40967`) From webhook-mailer at python.org Fri Sep 18 03:27:33 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 18 Sep 2020 07:27:33 -0000 Subject: [Python-checkins] bpo-41762: Fix usage of productionlist markup in the doc (GH-22281) Message-ID: https://github.com/python/cpython/commit/c053402927d36f9f26160ded24999bf5109ea5eb commit: c053402927d36f9f26160ded24999bf5109ea5eb branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-18T00:27:21-07:00 summary: bpo-41762: Fix usage of productionlist markup in the doc (GH-22281) Use an unique identifier for the different grammars documented using the Sphinx productionlist markup. productionlist markups of the same grammar, like "expressions" or "compound statements", use the same identifier "python-grammar". (cherry picked from commit 8af239eacfcf52e4e0e2b0223e7cea4672309483) Co-authored-by: Victor Stinner files: M Doc/library/functions.rst M Doc/library/string.rst M Doc/reference/compound_stmts.rst M Doc/reference/expressions.rst M Doc/reference/introduction.rst M Doc/reference/lexical_analysis.rst M Doc/reference/simple_stmts.rst M Doc/reference/toplevel_components.rst diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index b88c56e3d5862..124085ed988de 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -582,7 +582,7 @@ are always available. They are listed here in alphabetical order. input must conform to the following grammar after leading and trailing whitespace characters are removed: - .. productionlist:: + .. productionlist:: float sign: "+" | "-" infinity: "Infinity" | "inf" nan: "nan" diff --git a/Doc/library/string.rst b/Doc/library/string.rst index 62e86d6dd9706..91f43e9353d91 100644 --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -205,7 +205,7 @@ literal text, it can be escaped by doubling: ``{{`` and ``}}``. The grammar for a replacement field is as follows: - .. productionlist:: sf + .. productionlist:: format-string replacement_field: "{" [`field_name`] ["!" `conversion`] [":" `format_spec`] "}" field_name: arg_name ("." `attribute_name` | "[" `element_index` "]")* arg_name: [`identifier` | `digit`+] @@ -308,7 +308,7 @@ non-empty format specification typically modifies the result. The general form of a *standard format specifier* is: -.. productionlist:: +.. productionlist:: format-spec format_spec: [[`fill`]`align`][`sign`][#][0][`width`][`grouping_option`][.`precision`][`type`] fill: align: "<" | ">" | "=" | "^" diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index c14e7c79fe14c..b4e06e5b10d5a 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -44,7 +44,8 @@ executed:: Summarizing: -.. productionlist:: + +.. productionlist:: python-grammar compound_stmt: `if_stmt` : | `while_stmt` : | `for_stmt` @@ -89,7 +90,7 @@ The :keyword:`!if` statement The :keyword:`if` statement is used for conditional execution: -.. productionlist:: +.. productionlist:: python-grammar if_stmt: "if" `assignment_expression` ":" `suite` : ("elif" `assignment_expression` ":" `suite`)* : ["else" ":" `suite`] @@ -115,7 +116,7 @@ The :keyword:`!while` statement The :keyword:`while` statement is used for repeated execution as long as an expression is true: -.. productionlist:: +.. productionlist:: python-grammar while_stmt: "while" `assignment_expression` ":" `suite` : ["else" ":" `suite`] @@ -151,7 +152,7 @@ The :keyword:`!for` statement The :keyword:`for` statement is used to iterate over the elements of a sequence (such as a string, tuple or list) or other iterable object: -.. productionlist:: +.. productionlist:: python-grammar for_stmt: "for" `target_list` "in" `expression_list` ":" `suite` : ["else" ":" `suite`] @@ -234,7 +235,7 @@ The :keyword:`!try` statement The :keyword:`try` statement specifies exception handlers and/or cleanup code for a group of statements: -.. productionlist:: +.. productionlist:: python-grammar try_stmt: `try1_stmt` | `try2_stmt` try1_stmt: "try" ":" `suite` : ("except" [`expression` ["as" `identifier`]] ":" `suite`)+ @@ -390,7 +391,7 @@ methods defined by a context manager (see section :ref:`context-managers`). This allows common :keyword:`try`...\ :keyword:`except`...\ :keyword:`finally` usage patterns to be encapsulated for convenient reuse. -.. productionlist:: +.. productionlist:: python-grammar with_stmt: "with" `with_item` ("," `with_item`)* ":" `suite` with_item: `expression` ["as" `target`] @@ -503,7 +504,7 @@ Function definitions A function definition defines a user-defined function object (see section :ref:`types`): -.. productionlist:: +.. productionlist:: python-grammar funcdef: [`decorators`] "def" `funcname` "(" [`parameter_list`] ")" : ["->" `expression`] ":" `suite` decorators: `decorator`+ @@ -670,7 +671,7 @@ Class definitions A class definition defines a class object (see section :ref:`types`): -.. productionlist:: +.. productionlist:: python-grammar classdef: [`decorators`] "class" `classname` [`inheritance`] ":" `suite` inheritance: "(" [`argument_list`] ")" classname: `identifier` @@ -762,7 +763,7 @@ Coroutines Coroutine function definition ----------------------------- -.. productionlist:: +.. productionlist:: python-grammar async_funcdef: [`decorators`] "async" "def" `funcname` "(" [`parameter_list`] ")" : ["->" `expression`] ":" `suite` @@ -795,7 +796,7 @@ An example of a coroutine function:: The :keyword:`!async for` statement ----------------------------------- -.. productionlist:: +.. productionlist:: python-grammar async_for_stmt: "async" `for_stmt` An :term:`asynchronous iterable` is able to call asynchronous code in its @@ -840,7 +841,7 @@ body of a coroutine function. The :keyword:`!async with` statement ------------------------------------ -.. productionlist:: +.. productionlist:: python-grammar async_with_stmt: "async" `with_stmt` An :term:`asynchronous context manager` is a :term:`context manager` that is diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 18abce3c510bd..b68c29860cf33 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -13,7 +13,7 @@ This chapter explains the meaning of the elements of expressions in Python. be used to describe syntax, not lexical analysis. When (one alternative of) a syntax rule has the form -.. productionlist:: * +.. productionlist:: python-grammar name: `othername` and no semantics are given, the semantics of this form of ``name`` are the same @@ -54,7 +54,7 @@ Atoms are the most basic elements of expressions. The simplest atoms are identifiers or literals. Forms enclosed in parentheses, brackets or braces are also categorized syntactically as atoms. The syntax for atoms is: -.. productionlist:: +.. productionlist:: python-grammar atom: `identifier` | `literal` | `enclosure` enclosure: `parenth_form` | `list_display` | `dict_display` | `set_display` : | `generator_expression` | `yield_atom` @@ -103,7 +103,7 @@ Literals Python supports string and bytes literals and various numeric literals: -.. productionlist:: +.. productionlist:: python-grammar literal: `stringliteral` | `bytesliteral` : | `integer` | `floatnumber` | `imagnumber` @@ -134,7 +134,7 @@ Parenthesized forms A parenthesized form is an optional expression list enclosed in parentheses: -.. productionlist:: +.. productionlist:: python-grammar parenth_form: "(" [`starred_expression`] ")" A parenthesized expression list yields whatever that expression list yields: if @@ -177,7 +177,7 @@ called "displays", each of them in two flavors: Common syntax elements for comprehensions are: -.. productionlist:: +.. productionlist:: python-grammar comprehension: `assignment_expression` `comp_for` comp_for: ["async"] "for" `target_list` "in" `or_test` [`comp_iter`] comp_iter: `comp_for` | `comp_if` @@ -243,7 +243,7 @@ List displays A list display is a possibly empty series of expressions enclosed in square brackets: -.. productionlist:: +.. productionlist:: python-grammar list_display: "[" [`starred_list` | `comprehension`] "]" A list display yields a new list object, the contents being specified by either @@ -267,7 +267,7 @@ Set displays A set display is denoted by curly braces and distinguishable from dictionary displays by the lack of colons separating keys and values: -.. productionlist:: +.. productionlist:: python-grammar set_display: "{" (`starred_list` | `comprehension`) "}" A set display yields a new mutable set object, the contents being specified by @@ -296,7 +296,7 @@ Dictionary displays A dictionary display is a possibly empty series of key/datum pairs enclosed in curly braces: -.. productionlist:: +.. productionlist:: python-grammar dict_display: "{" [`key_datum_list` | `dict_comprehension`] "}" key_datum_list: `key_datum` ("," `key_datum`)* [","] key_datum: `expression` ":" `expression` | "**" `or_expr` @@ -355,7 +355,7 @@ Generator expressions A generator expression is a compact generator notation in parentheses: -.. productionlist:: +.. productionlist:: python-grammar generator_expression: "(" `expression` `comp_for` ")" A generator expression yields a new generator object. Its syntax is the same as @@ -409,7 +409,7 @@ Yield expressions pair: yield; expression pair: generator; function -.. productionlist:: +.. productionlist:: python-grammar yield_atom: "(" `yield_expression` ")" yield_expression: "yield" [`expression_list` | "from" `expression`] @@ -746,7 +746,7 @@ Primaries Primaries represent the most tightly bound operations of the language. Their syntax is: -.. productionlist:: +.. productionlist:: python-grammar primary: `atom` | `attributeref` | `subscription` | `slicing` | `call` @@ -761,7 +761,7 @@ Attribute references An attribute reference is a primary followed by a period and a name: -.. productionlist:: +.. productionlist:: python-grammar attributeref: `primary` "." `identifier` .. index:: @@ -799,7 +799,7 @@ Subscriptions A subscription selects an item of a sequence (string, tuple or list) or mapping (dictionary) object: -.. productionlist:: +.. productionlist:: python-grammar subscription: `primary` "[" `expression_list` "]" The primary must evaluate to an object that supports subscription (lists or @@ -855,7 +855,7 @@ A slicing selects a range of items in a sequence object (e.g., a string, tuple or list). Slicings may be used as expressions or as targets in assignment or :keyword:`del` statements. The syntax for a slicing: -.. productionlist:: +.. productionlist:: python-grammar slicing: `primary` "[" `slice_list` "]" slice_list: `slice_item` ("," `slice_item`)* [","] slice_item: `expression` | `proper_slice` @@ -905,7 +905,7 @@ Calls A call calls a callable object (e.g., a :term:`function`) with a possibly empty series of :term:`arguments `: -.. productionlist:: +.. productionlist:: python-grammar call: `primary` "(" [`argument_list` [","] | `comprehension`] ")" argument_list: `positional_arguments` ["," `starred_and_keywords`] : ["," `keywords_arguments`] @@ -1088,7 +1088,7 @@ Await expression Suspend the execution of :term:`coroutine` on an :term:`awaitable` object. Can only be used inside a :term:`coroutine function`. -.. productionlist:: +.. productionlist:: python-grammar await_expr: "await" `primary` .. versionadded:: 3.5 @@ -1106,7 +1106,7 @@ The power operator The power operator binds more tightly than unary operators on its left; it binds less tightly than unary operators on its right. The syntax is: -.. productionlist:: +.. productionlist:: python-grammar power: (`await_expr` | `primary`) ["**" `u_expr`] Thus, in an unparenthesized sequence of power and unary operators, the operators @@ -1139,7 +1139,7 @@ Unary arithmetic and bitwise operations All unary arithmetic and bitwise operations have the same priority: -.. productionlist:: +.. productionlist:: python-grammar u_expr: `power` | "-" `u_expr` | "+" `u_expr` | "~" `u_expr` .. index:: @@ -1183,7 +1183,7 @@ that some of these operations also apply to certain non-numeric types. Apart from the power operator, there are only two levels, one for multiplicative operators and one for additive operators: -.. productionlist:: +.. productionlist:: python-grammar m_expr: `u_expr` | `m_expr` "*" `u_expr` | `m_expr` "@" `m_expr` | : `m_expr` "//" `u_expr` | `m_expr` "/" `u_expr` | : `m_expr` "%" `u_expr` @@ -1279,7 +1279,7 @@ Shifting operations The shifting operations have lower priority than the arithmetic operations: -.. productionlist:: +.. productionlist:: python-grammar shift_expr: `a_expr` | `shift_expr` ("<<" | ">>") `a_expr` These operators accept integers as arguments. They shift the first argument to @@ -1300,7 +1300,7 @@ Binary bitwise operations Each of the three bitwise operations has a different priority level: -.. productionlist:: +.. productionlist:: python-grammar and_expr: `shift_expr` | `and_expr` "&" `shift_expr` xor_expr: `and_expr` | `xor_expr` "^" `and_expr` or_expr: `xor_expr` | `or_expr` "|" `xor_expr` @@ -1349,7 +1349,7 @@ lower than that of any arithmetic, shifting or bitwise operation. Also unlike C, expressions like ``a < b < c`` have the interpretation that is conventional in mathematics: -.. productionlist:: +.. productionlist:: python-grammar comparison: `or_expr` (`comp_operator` `or_expr`)* comp_operator: "<" | ">" | "==" | ">=" | "<=" | "!=" : | "is" ["not"] | ["not"] "in" @@ -1608,7 +1608,7 @@ Boolean operations pair: Conditional; expression pair: Boolean; operation -.. productionlist:: +.. productionlist:: python-grammar or_test: `and_test` | `or_test` "or" `and_test` and_test: `not_test` | `and_test` "and" `not_test` not_test: `comparison` | "not" `not_test` @@ -1647,7 +1647,7 @@ returns a boolean value regardless of the type of its argument Assignment expressions ====================== -.. productionlist:: +.. productionlist:: python-grammar assignment_expression: [`identifier` ":="] `expression` An assignment expression (sometimes also called a "named expression" or @@ -1683,7 +1683,7 @@ Conditional expressions single: if; conditional expression single: else; conditional expression -.. productionlist:: +.. productionlist:: python-grammar conditional_expression: `or_test` ["if" `or_test` "else" `expression`] expression: `conditional_expression` | `lambda_expr` expression_nocond: `or_test` | `lambda_expr_nocond` @@ -1710,7 +1710,7 @@ Lambdas pair: anonymous; function single: : (colon); lambda expression -.. productionlist:: +.. productionlist:: python-grammar lambda_expr: "lambda" [`parameter_list`] ":" `expression` lambda_expr_nocond: "lambda" [`parameter_list`] ":" `expression_nocond` @@ -1737,7 +1737,7 @@ Expression lists pair: expression; list single: , (comma); expression list -.. productionlist:: +.. productionlist:: python-grammar expression_list: `expression` ("," `expression`)* [","] starred_list: `starred_item` ("," `starred_item`)* [","] starred_expression: `expression` | (`starred_item` ",")* [`starred_item`] diff --git a/Doc/reference/introduction.rst b/Doc/reference/introduction.rst index 62480bd7dd9a6..72e874ee98e46 100644 --- a/Doc/reference/introduction.rst +++ b/Doc/reference/introduction.rst @@ -93,7 +93,7 @@ Notation The descriptions of lexical analysis and syntax use a modified BNF grammar notation. This uses the following style of definition: -.. productionlist:: +.. productionlist:: notation name: `lc_letter` (`lc_letter` | "_")* lc_letter: "a"..."z" diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index 19ba83a5513d8..77e0578f5d89b 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -296,7 +296,7 @@ Unicode Character Database as included in the :mod:`unicodedata` module. Identifiers are unlimited in length. Case is significant. -.. productionlist:: +.. productionlist:: python-grammar identifier: `xid_start` `xid_continue`* id_start: id_continue: @@ -412,7 +412,7 @@ String and Bytes literals String literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar stringliteral: [`stringprefix`](`shortstring` | `longstring`) stringprefix: "r" | "u" | "R" | "U" | "f" | "F" : | "fr" | "Fr" | "fR" | "FR" | "rf" | "rF" | "Rf" | "RF" @@ -424,7 +424,7 @@ String literals are described by the following lexical definitions: longstringchar: stringescapeseq: "\" -.. productionlist:: +.. productionlist:: python-grammar bytesliteral: `bytesprefix`(`shortbytes` | `longbytes`) bytesprefix: "b" | "B" | "br" | "Br" | "bR" | "BR" | "rb" | "rB" | "Rb" | "RB" shortbytes: "'" `shortbytesitem`* "'" | '"' `shortbytesitem`* '"' @@ -659,7 +659,7 @@ Escape sequences are decoded like in ordinary string literals (except when a literal is also marked as a raw string). After decoding, the grammar for the contents of the string is: -.. productionlist:: +.. productionlist:: python-grammar f_string: (`literal_char` | "{{" | "}}" | `replacement_field`)* replacement_field: "{" `f_expression` ["="] ["!" `conversion`] [":" `format_spec`] "}" f_expression: (`conditional_expression` | "*" `or_expr`) @@ -820,7 +820,7 @@ Integer literals Integer literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar integer: `decinteger` | `bininteger` | `octinteger` | `hexinteger` decinteger: `nonzerodigit` (["_"] `digit`)* | "0"+ (["_"] "0")* bininteger: "0" ("b" | "B") (["_"] `bindigit`)+ @@ -864,7 +864,7 @@ Floating point literals Floating point literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar floatnumber: `pointfloat` | `exponentfloat` pointfloat: [`digitpart`] `fraction` | `digitpart` "." exponentfloat: (`digitpart` | `pointfloat`) `exponent` @@ -894,7 +894,7 @@ Imaginary literals Imaginary literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar imagnumber: (`floatnumber` | `digitpart`) ("j" | "J") An imaginary literal yields a complex number with a real part of 0.0. Complex diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index a8ec0fbe8b732..93be32713ff32 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -11,7 +11,7 @@ A simple statement is comprised within a single logical line. Several simple statements may occur on a single line separated by semicolons. The syntax for simple statements is: -.. productionlist:: +.. productionlist:: python-grammar simple_stmt: `expression_stmt` : | `assert_stmt` : | `assignment_stmt` @@ -46,7 +46,7 @@ result; in Python, procedures return the value ``None``). Other uses of expression statements are allowed and occasionally useful. The syntax for an expression statement is: -.. productionlist:: +.. productionlist:: python-grammar expression_stmt: `starred_expression` An expression statement evaluates the expression list (which may be a single @@ -82,7 +82,7 @@ Assignment statements Assignment statements are used to (re)bind names to values and to modify attributes or items of mutable objects: -.. productionlist:: +.. productionlist:: python-grammar assignment_stmt: (`target_list` "=")+ (`starred_expression` | `yield_expression`) target_list: `target` ("," `target`)* [","] target: `identifier` @@ -280,7 +280,7 @@ Augmented assignment statements Augmented assignment is the combination, in a single statement, of a binary operation and an assignment statement: -.. productionlist:: +.. productionlist:: python-grammar augmented_assignment_stmt: `augtarget` `augop` (`expression_list` | `yield_expression`) augtarget: `identifier` | `attributeref` | `subscription` | `slicing` augop: "+=" | "-=" | "*=" | "@=" | "/=" | "//=" | "%=" | "**=" @@ -328,7 +328,7 @@ Annotated assignment statements :term:`Annotation ` assignment is the combination, in a single statement, of a variable or attribute annotation and an optional assignment statement: -.. productionlist:: +.. productionlist:: python-grammar annotated_assignment_stmt: `augtarget` ":" `expression` : ["=" (`starred_expression` | `yield_expression`)] @@ -385,7 +385,7 @@ The :keyword:`!assert` statement Assert statements are a convenient way to insert debugging assertions into a program: -.. productionlist:: +.. productionlist:: python-grammar assert_stmt: "assert" `expression` ["," `expression`] The simple form, ``assert expression``, is equivalent to :: @@ -425,7 +425,7 @@ The :keyword:`!pass` statement pair: null; operation pair: null; operation -.. productionlist:: +.. productionlist:: python-grammar pass_stmt: "pass" :keyword:`pass` is a null operation --- when it is executed, nothing happens. @@ -447,7 +447,7 @@ The :keyword:`!del` statement pair: deletion; target triple: deletion; target; list -.. productionlist:: +.. productionlist:: python-grammar del_stmt: "del" `target_list` Deletion is recursively defined very similar to the way assignment is defined. @@ -486,7 +486,7 @@ The :keyword:`!return` statement pair: function; definition pair: class; definition -.. productionlist:: +.. productionlist:: python-grammar return_stmt: "return" [`expression_list`] :keyword:`return` may only occur syntactically nested in a function definition, @@ -525,7 +525,7 @@ The :keyword:`!yield` statement single: function; generator exception: StopIteration -.. productionlist:: +.. productionlist:: python-grammar yield_stmt: `yield_expression` A :keyword:`yield` statement is semantically equivalent to a :ref:`yield @@ -560,7 +560,7 @@ The :keyword:`!raise` statement pair: raising; exception single: __traceback__ (exception attribute) -.. productionlist:: +.. productionlist:: python-grammar raise_stmt: "raise" [`expression` ["from" `expression`]] If no expressions are present, :keyword:`raise` re-raises the last exception @@ -663,7 +663,7 @@ The :keyword:`!break` statement statement: while pair: loop; statement -.. productionlist:: +.. productionlist:: python-grammar break_stmt: "break" :keyword:`break` may only occur syntactically nested in a :keyword:`for` or @@ -698,7 +698,7 @@ The :keyword:`!continue` statement pair: loop; statement keyword: finally -.. productionlist:: +.. productionlist:: python-grammar continue_stmt: "continue" :keyword:`continue` may only occur syntactically nested in a :keyword:`for` or @@ -725,7 +725,7 @@ The :keyword:`!import` statement exception: ImportError single: , (comma); import statement -.. productionlist:: +.. productionlist:: python-grammar import_stmt: "import" `module` ["as" `identifier`] ("," `module` ["as" `identifier`])* : | "from" `relative_module` "import" `identifier` ["as" `identifier`] : ("," `identifier` ["as" `identifier`])* @@ -859,7 +859,7 @@ that introduce incompatible changes to the language. It allows use of the new features on a per-module basis before the release in which the feature becomes standard. -.. productionlist:: * +.. productionlist:: python-grammar future_stmt: "from" "__future__" "import" `feature` ["as" `identifier`] : ("," `feature` ["as" `identifier`])* : | "from" "__future__" "import" "(" `feature` ["as" `identifier`] @@ -937,7 +937,7 @@ The :keyword:`!global` statement triple: global; name; binding single: , (comma); identifier list -.. productionlist:: +.. productionlist:: python-grammar global_stmt: "global" `identifier` ("," `identifier`)* The :keyword:`global` statement is a declaration which holds for the entire @@ -982,7 +982,7 @@ The :keyword:`!nonlocal` statement .. index:: statement: nonlocal single: , (comma); identifier list -.. productionlist:: +.. productionlist:: python-grammar nonlocal_stmt: "nonlocal" `identifier` ("," `identifier`)* .. XXX add when implemented diff --git a/Doc/reference/toplevel_components.rst b/Doc/reference/toplevel_components.rst index d5ffb37b2e58c..319c9de484241 100644 --- a/Doc/reference/toplevel_components.rst +++ b/Doc/reference/toplevel_components.rst @@ -66,7 +66,7 @@ File input All input read from non-interactive files has the same form: -.. productionlist:: +.. productionlist:: python-grammar file_input: (NEWLINE | `statement`)* This syntax is used in the following situations: @@ -85,7 +85,7 @@ Interactive input Input in interactive mode is parsed using the following grammar: -.. productionlist:: +.. productionlist:: python-grammar interactive_input: [`stmt_list`] NEWLINE | `compound_stmt` NEWLINE Note that a (top-level) compound statement must be followed by a blank line in @@ -103,5 +103,5 @@ Expression input :func:`eval` is used for expression input. It ignores leading whitespace. The string argument to :func:`eval` must have the following form: -.. productionlist:: +.. productionlist:: python-grammar eval_input: `expression_list` NEWLINE* From webhook-mailer at python.org Fri Sep 18 04:31:33 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 18 Sep 2020 08:31:33 -0000 Subject: [Python-checkins] bpo-41762: Fix usage of productionlist markup in the doc (GH-22281) (GH-22300) Message-ID: https://github.com/python/cpython/commit/c5cddc17b5c046bb22072c1cf58b898c7ab92e07 commit: c5cddc17b5c046bb22072c1cf58b898c7ab92e07 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-18T10:31:28+02:00 summary: bpo-41762: Fix usage of productionlist markup in the doc (GH-22281) (GH-22300) Use an unique identifier for the different grammars documented using the Sphinx productionlist markup. productionlist markups of the same grammar, like "expressions" or "compound statements", use the same identifier "python-grammar". (cherry picked from commit 8af239eacfcf52e4e0e2b0223e7cea4672309483) Co-authored-by: Victor Stinner files: M Doc/library/functions.rst M Doc/library/string.rst M Doc/reference/compound_stmts.rst M Doc/reference/expressions.rst M Doc/reference/introduction.rst M Doc/reference/lexical_analysis.rst M Doc/reference/simple_stmts.rst M Doc/reference/toplevel_components.rst diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index c31efa96a92c1..6003898d5f6b8 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -581,7 +581,7 @@ are always available. They are listed here in alphabetical order. input must conform to the following grammar after leading and trailing whitespace characters are removed: - .. productionlist:: + .. productionlist:: float sign: "+" | "-" infinity: "Infinity" | "inf" nan: "nan" diff --git a/Doc/library/string.rst b/Doc/library/string.rst index 62e86d6dd9706..91f43e9353d91 100644 --- a/Doc/library/string.rst +++ b/Doc/library/string.rst @@ -205,7 +205,7 @@ literal text, it can be escaped by doubling: ``{{`` and ``}}``. The grammar for a replacement field is as follows: - .. productionlist:: sf + .. productionlist:: format-string replacement_field: "{" [`field_name`] ["!" `conversion`] [":" `format_spec`] "}" field_name: arg_name ("." `attribute_name` | "[" `element_index` "]")* arg_name: [`identifier` | `digit`+] @@ -308,7 +308,7 @@ non-empty format specification typically modifies the result. The general form of a *standard format specifier* is: -.. productionlist:: +.. productionlist:: format-spec format_spec: [[`fill`]`align`][`sign`][#][0][`width`][`grouping_option`][.`precision`][`type`] fill: align: "<" | ">" | "=" | "^" diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index feaf3109d5a24..6900c7ac7d071 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -44,7 +44,8 @@ executed:: Summarizing: -.. productionlist:: + +.. productionlist:: python-grammar compound_stmt: `if_stmt` : | `while_stmt` : | `for_stmt` @@ -89,7 +90,7 @@ The :keyword:`!if` statement The :keyword:`if` statement is used for conditional execution: -.. productionlist:: +.. productionlist:: python-grammar if_stmt: "if" `assignment_expression` ":" `suite` : ("elif" `assignment_expression` ":" `suite`)* : ["else" ":" `suite`] @@ -115,7 +116,7 @@ The :keyword:`!while` statement The :keyword:`while` statement is used for repeated execution as long as an expression is true: -.. productionlist:: +.. productionlist:: python-grammar while_stmt: "while" `assignment_expression` ":" `suite` : ["else" ":" `suite`] @@ -151,7 +152,7 @@ The :keyword:`!for` statement The :keyword:`for` statement is used to iterate over the elements of a sequence (such as a string, tuple or list) or other iterable object: -.. productionlist:: +.. productionlist:: python-grammar for_stmt: "for" `target_list` "in" `expression_list` ":" `suite` : ["else" ":" `suite`] @@ -234,7 +235,7 @@ The :keyword:`!try` statement The :keyword:`try` statement specifies exception handlers and/or cleanup code for a group of statements: -.. productionlist:: +.. productionlist:: python-grammar try_stmt: `try1_stmt` | `try2_stmt` try1_stmt: "try" ":" `suite` : ("except" [`expression` ["as" `identifier`]] ":" `suite`)+ @@ -390,7 +391,7 @@ methods defined by a context manager (see section :ref:`context-managers`). This allows common :keyword:`try`...\ :keyword:`except`...\ :keyword:`finally` usage patterns to be encapsulated for convenient reuse. -.. productionlist:: +.. productionlist:: python-grammar with_stmt: "with" `with_item` ("," `with_item`)* ":" `suite` with_item: `expression` ["as" `target`] @@ -503,7 +504,7 @@ Function definitions A function definition defines a user-defined function object (see section :ref:`types`): -.. productionlist:: +.. productionlist:: python-grammar funcdef: [`decorators`] "def" `funcname` "(" [`parameter_list`] ")" : ["->" `expression`] ":" `suite` decorators: `decorator`+ @@ -665,7 +666,7 @@ Class definitions A class definition defines a class object (see section :ref:`types`): -.. productionlist:: +.. productionlist:: python-grammar classdef: [`decorators`] "class" `classname` [`inheritance`] ":" `suite` inheritance: "(" [`argument_list`] ")" classname: `identifier` @@ -752,7 +753,7 @@ Coroutines Coroutine function definition ----------------------------- -.. productionlist:: +.. productionlist:: python-grammar async_funcdef: [`decorators`] "async" "def" `funcname` "(" [`parameter_list`] ")" : ["->" `expression`] ":" `suite` @@ -785,7 +786,7 @@ An example of a coroutine function:: The :keyword:`!async for` statement ----------------------------------- -.. productionlist:: +.. productionlist:: python-grammar async_for_stmt: "async" `for_stmt` An :term:`asynchronous iterable` is able to call asynchronous code in its @@ -830,7 +831,7 @@ body of a coroutine function. The :keyword:`!async with` statement ------------------------------------ -.. productionlist:: +.. productionlist:: python-grammar async_with_stmt: "async" `with_stmt` An :term:`asynchronous context manager` is a :term:`context manager` that is diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 18abce3c510bd..b68c29860cf33 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -13,7 +13,7 @@ This chapter explains the meaning of the elements of expressions in Python. be used to describe syntax, not lexical analysis. When (one alternative of) a syntax rule has the form -.. productionlist:: * +.. productionlist:: python-grammar name: `othername` and no semantics are given, the semantics of this form of ``name`` are the same @@ -54,7 +54,7 @@ Atoms are the most basic elements of expressions. The simplest atoms are identifiers or literals. Forms enclosed in parentheses, brackets or braces are also categorized syntactically as atoms. The syntax for atoms is: -.. productionlist:: +.. productionlist:: python-grammar atom: `identifier` | `literal` | `enclosure` enclosure: `parenth_form` | `list_display` | `dict_display` | `set_display` : | `generator_expression` | `yield_atom` @@ -103,7 +103,7 @@ Literals Python supports string and bytes literals and various numeric literals: -.. productionlist:: +.. productionlist:: python-grammar literal: `stringliteral` | `bytesliteral` : | `integer` | `floatnumber` | `imagnumber` @@ -134,7 +134,7 @@ Parenthesized forms A parenthesized form is an optional expression list enclosed in parentheses: -.. productionlist:: +.. productionlist:: python-grammar parenth_form: "(" [`starred_expression`] ")" A parenthesized expression list yields whatever that expression list yields: if @@ -177,7 +177,7 @@ called "displays", each of them in two flavors: Common syntax elements for comprehensions are: -.. productionlist:: +.. productionlist:: python-grammar comprehension: `assignment_expression` `comp_for` comp_for: ["async"] "for" `target_list` "in" `or_test` [`comp_iter`] comp_iter: `comp_for` | `comp_if` @@ -243,7 +243,7 @@ List displays A list display is a possibly empty series of expressions enclosed in square brackets: -.. productionlist:: +.. productionlist:: python-grammar list_display: "[" [`starred_list` | `comprehension`] "]" A list display yields a new list object, the contents being specified by either @@ -267,7 +267,7 @@ Set displays A set display is denoted by curly braces and distinguishable from dictionary displays by the lack of colons separating keys and values: -.. productionlist:: +.. productionlist:: python-grammar set_display: "{" (`starred_list` | `comprehension`) "}" A set display yields a new mutable set object, the contents being specified by @@ -296,7 +296,7 @@ Dictionary displays A dictionary display is a possibly empty series of key/datum pairs enclosed in curly braces: -.. productionlist:: +.. productionlist:: python-grammar dict_display: "{" [`key_datum_list` | `dict_comprehension`] "}" key_datum_list: `key_datum` ("," `key_datum`)* [","] key_datum: `expression` ":" `expression` | "**" `or_expr` @@ -355,7 +355,7 @@ Generator expressions A generator expression is a compact generator notation in parentheses: -.. productionlist:: +.. productionlist:: python-grammar generator_expression: "(" `expression` `comp_for` ")" A generator expression yields a new generator object. Its syntax is the same as @@ -409,7 +409,7 @@ Yield expressions pair: yield; expression pair: generator; function -.. productionlist:: +.. productionlist:: python-grammar yield_atom: "(" `yield_expression` ")" yield_expression: "yield" [`expression_list` | "from" `expression`] @@ -746,7 +746,7 @@ Primaries Primaries represent the most tightly bound operations of the language. Their syntax is: -.. productionlist:: +.. productionlist:: python-grammar primary: `atom` | `attributeref` | `subscription` | `slicing` | `call` @@ -761,7 +761,7 @@ Attribute references An attribute reference is a primary followed by a period and a name: -.. productionlist:: +.. productionlist:: python-grammar attributeref: `primary` "." `identifier` .. index:: @@ -799,7 +799,7 @@ Subscriptions A subscription selects an item of a sequence (string, tuple or list) or mapping (dictionary) object: -.. productionlist:: +.. productionlist:: python-grammar subscription: `primary` "[" `expression_list` "]" The primary must evaluate to an object that supports subscription (lists or @@ -855,7 +855,7 @@ A slicing selects a range of items in a sequence object (e.g., a string, tuple or list). Slicings may be used as expressions or as targets in assignment or :keyword:`del` statements. The syntax for a slicing: -.. productionlist:: +.. productionlist:: python-grammar slicing: `primary` "[" `slice_list` "]" slice_list: `slice_item` ("," `slice_item`)* [","] slice_item: `expression` | `proper_slice` @@ -905,7 +905,7 @@ Calls A call calls a callable object (e.g., a :term:`function`) with a possibly empty series of :term:`arguments `: -.. productionlist:: +.. productionlist:: python-grammar call: `primary` "(" [`argument_list` [","] | `comprehension`] ")" argument_list: `positional_arguments` ["," `starred_and_keywords`] : ["," `keywords_arguments`] @@ -1088,7 +1088,7 @@ Await expression Suspend the execution of :term:`coroutine` on an :term:`awaitable` object. Can only be used inside a :term:`coroutine function`. -.. productionlist:: +.. productionlist:: python-grammar await_expr: "await" `primary` .. versionadded:: 3.5 @@ -1106,7 +1106,7 @@ The power operator The power operator binds more tightly than unary operators on its left; it binds less tightly than unary operators on its right. The syntax is: -.. productionlist:: +.. productionlist:: python-grammar power: (`await_expr` | `primary`) ["**" `u_expr`] Thus, in an unparenthesized sequence of power and unary operators, the operators @@ -1139,7 +1139,7 @@ Unary arithmetic and bitwise operations All unary arithmetic and bitwise operations have the same priority: -.. productionlist:: +.. productionlist:: python-grammar u_expr: `power` | "-" `u_expr` | "+" `u_expr` | "~" `u_expr` .. index:: @@ -1183,7 +1183,7 @@ that some of these operations also apply to certain non-numeric types. Apart from the power operator, there are only two levels, one for multiplicative operators and one for additive operators: -.. productionlist:: +.. productionlist:: python-grammar m_expr: `u_expr` | `m_expr` "*" `u_expr` | `m_expr` "@" `m_expr` | : `m_expr` "//" `u_expr` | `m_expr` "/" `u_expr` | : `m_expr` "%" `u_expr` @@ -1279,7 +1279,7 @@ Shifting operations The shifting operations have lower priority than the arithmetic operations: -.. productionlist:: +.. productionlist:: python-grammar shift_expr: `a_expr` | `shift_expr` ("<<" | ">>") `a_expr` These operators accept integers as arguments. They shift the first argument to @@ -1300,7 +1300,7 @@ Binary bitwise operations Each of the three bitwise operations has a different priority level: -.. productionlist:: +.. productionlist:: python-grammar and_expr: `shift_expr` | `and_expr` "&" `shift_expr` xor_expr: `and_expr` | `xor_expr` "^" `and_expr` or_expr: `xor_expr` | `or_expr` "|" `xor_expr` @@ -1349,7 +1349,7 @@ lower than that of any arithmetic, shifting or bitwise operation. Also unlike C, expressions like ``a < b < c`` have the interpretation that is conventional in mathematics: -.. productionlist:: +.. productionlist:: python-grammar comparison: `or_expr` (`comp_operator` `or_expr`)* comp_operator: "<" | ">" | "==" | ">=" | "<=" | "!=" : | "is" ["not"] | ["not"] "in" @@ -1608,7 +1608,7 @@ Boolean operations pair: Conditional; expression pair: Boolean; operation -.. productionlist:: +.. productionlist:: python-grammar or_test: `and_test` | `or_test` "or" `and_test` and_test: `not_test` | `and_test` "and" `not_test` not_test: `comparison` | "not" `not_test` @@ -1647,7 +1647,7 @@ returns a boolean value regardless of the type of its argument Assignment expressions ====================== -.. productionlist:: +.. productionlist:: python-grammar assignment_expression: [`identifier` ":="] `expression` An assignment expression (sometimes also called a "named expression" or @@ -1683,7 +1683,7 @@ Conditional expressions single: if; conditional expression single: else; conditional expression -.. productionlist:: +.. productionlist:: python-grammar conditional_expression: `or_test` ["if" `or_test` "else" `expression`] expression: `conditional_expression` | `lambda_expr` expression_nocond: `or_test` | `lambda_expr_nocond` @@ -1710,7 +1710,7 @@ Lambdas pair: anonymous; function single: : (colon); lambda expression -.. productionlist:: +.. productionlist:: python-grammar lambda_expr: "lambda" [`parameter_list`] ":" `expression` lambda_expr_nocond: "lambda" [`parameter_list`] ":" `expression_nocond` @@ -1737,7 +1737,7 @@ Expression lists pair: expression; list single: , (comma); expression list -.. productionlist:: +.. productionlist:: python-grammar expression_list: `expression` ("," `expression`)* [","] starred_list: `starred_item` ("," `starred_item`)* [","] starred_expression: `expression` | (`starred_item` ",")* [`starred_item`] diff --git a/Doc/reference/introduction.rst b/Doc/reference/introduction.rst index 62480bd7dd9a6..72e874ee98e46 100644 --- a/Doc/reference/introduction.rst +++ b/Doc/reference/introduction.rst @@ -93,7 +93,7 @@ Notation The descriptions of lexical analysis and syntax use a modified BNF grammar notation. This uses the following style of definition: -.. productionlist:: +.. productionlist:: notation name: `lc_letter` (`lc_letter` | "_")* lc_letter: "a"..."z" diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst index ebacd7f471ee1..f74ff7949ef78 100644 --- a/Doc/reference/lexical_analysis.rst +++ b/Doc/reference/lexical_analysis.rst @@ -296,7 +296,7 @@ Unicode Character Database as included in the :mod:`unicodedata` module. Identifiers are unlimited in length. Case is significant. -.. productionlist:: +.. productionlist:: python-grammar identifier: `xid_start` `xid_continue`* id_start: id_continue: @@ -412,7 +412,7 @@ String and Bytes literals String literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar stringliteral: [`stringprefix`](`shortstring` | `longstring`) stringprefix: "r" | "u" | "R" | "U" | "f" | "F" : | "fr" | "Fr" | "fR" | "FR" | "rf" | "rF" | "Rf" | "RF" @@ -424,7 +424,7 @@ String literals are described by the following lexical definitions: longstringchar: stringescapeseq: "\" -.. productionlist:: +.. productionlist:: python-grammar bytesliteral: `bytesprefix`(`shortbytes` | `longbytes`) bytesprefix: "b" | "B" | "br" | "Br" | "bR" | "BR" | "rb" | "rB" | "Rb" | "RB" shortbytes: "'" `shortbytesitem`* "'" | '"' `shortbytesitem`* '"' @@ -659,7 +659,7 @@ Escape sequences are decoded like in ordinary string literals (except when a literal is also marked as a raw string). After decoding, the grammar for the contents of the string is: -.. productionlist:: +.. productionlist:: python-grammar f_string: (`literal_char` | "{{" | "}}" | `replacement_field`)* replacement_field: "{" `f_expression` ["="] ["!" `conversion`] [":" `format_spec`] "}" f_expression: (`conditional_expression` | "*" `or_expr`) @@ -820,7 +820,7 @@ Integer literals Integer literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar integer: `decinteger` | `bininteger` | `octinteger` | `hexinteger` decinteger: `nonzerodigit` (["_"] `digit`)* | "0"+ (["_"] "0")* bininteger: "0" ("b" | "B") (["_"] `bindigit`)+ @@ -864,7 +864,7 @@ Floating point literals Floating point literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar floatnumber: `pointfloat` | `exponentfloat` pointfloat: [`digitpart`] `fraction` | `digitpart` "." exponentfloat: (`digitpart` | `pointfloat`) `exponent` @@ -894,7 +894,7 @@ Imaginary literals Imaginary literals are described by the following lexical definitions: -.. productionlist:: +.. productionlist:: python-grammar imagnumber: (`floatnumber` | `digitpart`) ("j" | "J") An imaginary literal yields a complex number with a real part of 0.0. Complex diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index a8ec0fbe8b732..93be32713ff32 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -11,7 +11,7 @@ A simple statement is comprised within a single logical line. Several simple statements may occur on a single line separated by semicolons. The syntax for simple statements is: -.. productionlist:: +.. productionlist:: python-grammar simple_stmt: `expression_stmt` : | `assert_stmt` : | `assignment_stmt` @@ -46,7 +46,7 @@ result; in Python, procedures return the value ``None``). Other uses of expression statements are allowed and occasionally useful. The syntax for an expression statement is: -.. productionlist:: +.. productionlist:: python-grammar expression_stmt: `starred_expression` An expression statement evaluates the expression list (which may be a single @@ -82,7 +82,7 @@ Assignment statements Assignment statements are used to (re)bind names to values and to modify attributes or items of mutable objects: -.. productionlist:: +.. productionlist:: python-grammar assignment_stmt: (`target_list` "=")+ (`starred_expression` | `yield_expression`) target_list: `target` ("," `target`)* [","] target: `identifier` @@ -280,7 +280,7 @@ Augmented assignment statements Augmented assignment is the combination, in a single statement, of a binary operation and an assignment statement: -.. productionlist:: +.. productionlist:: python-grammar augmented_assignment_stmt: `augtarget` `augop` (`expression_list` | `yield_expression`) augtarget: `identifier` | `attributeref` | `subscription` | `slicing` augop: "+=" | "-=" | "*=" | "@=" | "/=" | "//=" | "%=" | "**=" @@ -328,7 +328,7 @@ Annotated assignment statements :term:`Annotation ` assignment is the combination, in a single statement, of a variable or attribute annotation and an optional assignment statement: -.. productionlist:: +.. productionlist:: python-grammar annotated_assignment_stmt: `augtarget` ":" `expression` : ["=" (`starred_expression` | `yield_expression`)] @@ -385,7 +385,7 @@ The :keyword:`!assert` statement Assert statements are a convenient way to insert debugging assertions into a program: -.. productionlist:: +.. productionlist:: python-grammar assert_stmt: "assert" `expression` ["," `expression`] The simple form, ``assert expression``, is equivalent to :: @@ -425,7 +425,7 @@ The :keyword:`!pass` statement pair: null; operation pair: null; operation -.. productionlist:: +.. productionlist:: python-grammar pass_stmt: "pass" :keyword:`pass` is a null operation --- when it is executed, nothing happens. @@ -447,7 +447,7 @@ The :keyword:`!del` statement pair: deletion; target triple: deletion; target; list -.. productionlist:: +.. productionlist:: python-grammar del_stmt: "del" `target_list` Deletion is recursively defined very similar to the way assignment is defined. @@ -486,7 +486,7 @@ The :keyword:`!return` statement pair: function; definition pair: class; definition -.. productionlist:: +.. productionlist:: python-grammar return_stmt: "return" [`expression_list`] :keyword:`return` may only occur syntactically nested in a function definition, @@ -525,7 +525,7 @@ The :keyword:`!yield` statement single: function; generator exception: StopIteration -.. productionlist:: +.. productionlist:: python-grammar yield_stmt: `yield_expression` A :keyword:`yield` statement is semantically equivalent to a :ref:`yield @@ -560,7 +560,7 @@ The :keyword:`!raise` statement pair: raising; exception single: __traceback__ (exception attribute) -.. productionlist:: +.. productionlist:: python-grammar raise_stmt: "raise" [`expression` ["from" `expression`]] If no expressions are present, :keyword:`raise` re-raises the last exception @@ -663,7 +663,7 @@ The :keyword:`!break` statement statement: while pair: loop; statement -.. productionlist:: +.. productionlist:: python-grammar break_stmt: "break" :keyword:`break` may only occur syntactically nested in a :keyword:`for` or @@ -698,7 +698,7 @@ The :keyword:`!continue` statement pair: loop; statement keyword: finally -.. productionlist:: +.. productionlist:: python-grammar continue_stmt: "continue" :keyword:`continue` may only occur syntactically nested in a :keyword:`for` or @@ -725,7 +725,7 @@ The :keyword:`!import` statement exception: ImportError single: , (comma); import statement -.. productionlist:: +.. productionlist:: python-grammar import_stmt: "import" `module` ["as" `identifier`] ("," `module` ["as" `identifier`])* : | "from" `relative_module` "import" `identifier` ["as" `identifier`] : ("," `identifier` ["as" `identifier`])* @@ -859,7 +859,7 @@ that introduce incompatible changes to the language. It allows use of the new features on a per-module basis before the release in which the feature becomes standard. -.. productionlist:: * +.. productionlist:: python-grammar future_stmt: "from" "__future__" "import" `feature` ["as" `identifier`] : ("," `feature` ["as" `identifier`])* : | "from" "__future__" "import" "(" `feature` ["as" `identifier`] @@ -937,7 +937,7 @@ The :keyword:`!global` statement triple: global; name; binding single: , (comma); identifier list -.. productionlist:: +.. productionlist:: python-grammar global_stmt: "global" `identifier` ("," `identifier`)* The :keyword:`global` statement is a declaration which holds for the entire @@ -982,7 +982,7 @@ The :keyword:`!nonlocal` statement .. index:: statement: nonlocal single: , (comma); identifier list -.. productionlist:: +.. productionlist:: python-grammar nonlocal_stmt: "nonlocal" `identifier` ("," `identifier`)* .. XXX add when implemented diff --git a/Doc/reference/toplevel_components.rst b/Doc/reference/toplevel_components.rst index d5ffb37b2e58c..319c9de484241 100644 --- a/Doc/reference/toplevel_components.rst +++ b/Doc/reference/toplevel_components.rst @@ -66,7 +66,7 @@ File input All input read from non-interactive files has the same form: -.. productionlist:: +.. productionlist:: python-grammar file_input: (NEWLINE | `statement`)* This syntax is used in the following situations: @@ -85,7 +85,7 @@ Interactive input Input in interactive mode is parsed using the following grammar: -.. productionlist:: +.. productionlist:: python-grammar interactive_input: [`stmt_list`] NEWLINE | `compound_stmt` NEWLINE Note that a (top-level) compound statement must be followed by a blank line in @@ -103,5 +103,5 @@ Expression input :func:`eval` is used for expression input. It ignores leading whitespace. The string argument to :func:`eval` must have the following form: -.. productionlist:: +.. productionlist:: python-grammar eval_input: `expression_list` NEWLINE* From webhook-mailer at python.org Fri Sep 18 05:22:45 2020 From: webhook-mailer at python.org (Dong-hee Na) Date: Fri, 18 Sep 2020 09:22:45 -0000 Subject: [Python-checkins] bpo-35293: Remove RemovedInSphinx40Warning (GH-22198) Message-ID: https://github.com/python/cpython/commit/6595cb0af4c51c0381c233b97884fe916a4ddd35 commit: 6595cb0af4c51c0381c233b97884fe916a4ddd35 branch: master author: Dong-hee Na committer: GitHub date: 2020-09-18T18:22:36+09:00 summary: bpo-35293: Remove RemovedInSphinx40Warning (GH-22198) * bpo-35293: Remove RemovedInSphinx40Warning * Update Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst Co-authored-by: Victor Stinner * bpo-35293: Apply Victor's review Co-authored-by: Victor Stinner files: A Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst M Doc/tools/extensions/pyspecific.py diff --git a/Doc/tools/extensions/pyspecific.py b/Doc/tools/extensions/pyspecific.py index 008dd8a6f7c8c..80fbd96d56fdc 100644 --- a/Doc/tools/extensions/pyspecific.py +++ b/Doc/tools/extensions/pyspecific.py @@ -31,7 +31,12 @@ from sphinx.util.nodes import split_explicit_title from sphinx.writers.text import TextWriter, TextTranslator from sphinx.writers.latex import LaTeXTranslator -from sphinx.domains.python import PyModulelevel, PyClassmember + +try: + from sphinx.domains.python import PyFunction, PyMethod +except ImportError: + from sphinx.domains.python import PyClassmember as PyMethod + from sphinx.domains.python import PyModulelevel as PyFunction # Support for checking for suspicious markup @@ -271,17 +276,18 @@ def needs_arglist(self): return False -class PyDecoratorFunction(PyDecoratorMixin, PyModulelevel): +class PyDecoratorFunction(PyDecoratorMixin, PyFunction): def run(self): # a decorator function is a function after all self.name = 'py:function' - return PyModulelevel.run(self) + return PyFunction.run(self) -class PyDecoratorMethod(PyDecoratorMixin, PyClassmember): +# TODO: Use sphinx.domains.python.PyDecoratorMethod when possible +class PyDecoratorMethod(PyDecoratorMixin, PyMethod): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) class PyCoroutineMixin(object): @@ -298,31 +304,31 @@ def handle_signature(self, sig, signode): return ret -class PyCoroutineFunction(PyCoroutineMixin, PyModulelevel): +class PyCoroutineFunction(PyCoroutineMixin, PyFunction): def run(self): self.name = 'py:function' - return PyModulelevel.run(self) + return PyFunction.run(self) -class PyCoroutineMethod(PyCoroutineMixin, PyClassmember): +class PyCoroutineMethod(PyCoroutineMixin, PyMethod): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) -class PyAwaitableFunction(PyAwaitableMixin, PyClassmember): +class PyAwaitableFunction(PyAwaitableMixin, PyFunction): def run(self): self.name = 'py:function' - return PyClassmember.run(self) + return PyFunction.run(self) -class PyAwaitableMethod(PyAwaitableMixin, PyClassmember): +class PyAwaitableMethod(PyAwaitableMixin, PyMethod): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) -class PyAbstractMethod(PyClassmember): +class PyAbstractMethod(PyMethod): def handle_signature(self, sig, signode): ret = super(PyAbstractMethod, self).handle_signature(sig, signode) @@ -332,7 +338,7 @@ def handle_signature(self, sig, signode): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) # Support for documenting version of removal in deprecations diff --git a/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst b/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst new file mode 100644 index 0000000000000..089d44e35d2ba --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst @@ -0,0 +1 @@ +Fix RemovedInSphinx40Warning when building the documentation. Patch by Dong-hee Na. From webhook-mailer at python.org Fri Sep 18 05:31:38 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 18 Sep 2020 09:31:38 -0000 Subject: [Python-checkins] bpo-35293: Remove RemovedInSphinx40Warning (GH-22198) Message-ID: https://github.com/python/cpython/commit/b7cdea8f087699aaa6486ce5a4f6e5a143dc355e commit: b7cdea8f087699aaa6486ce5a4f6e5a143dc355e branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-18T02:31:30-07:00 summary: bpo-35293: Remove RemovedInSphinx40Warning (GH-22198) * bpo-35293: Remove RemovedInSphinx40Warning * Update Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst Co-authored-by: Victor Stinner * bpo-35293: Apply Victor's review Co-authored-by: Victor Stinner (cherry picked from commit 6595cb0af4c51c0381c233b97884fe916a4ddd35) Co-authored-by: Dong-hee Na files: A Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst M Doc/tools/extensions/pyspecific.py diff --git a/Doc/tools/extensions/pyspecific.py b/Doc/tools/extensions/pyspecific.py index a128c705847e9..e3369cdc1ebe1 100644 --- a/Doc/tools/extensions/pyspecific.py +++ b/Doc/tools/extensions/pyspecific.py @@ -31,7 +31,12 @@ from sphinx.util.nodes import split_explicit_title from sphinx.writers.text import TextWriter, TextTranslator from sphinx.writers.latex import LaTeXTranslator -from sphinx.domains.python import PyModulelevel, PyClassmember + +try: + from sphinx.domains.python import PyFunction, PyMethod +except ImportError: + from sphinx.domains.python import PyClassmember as PyMethod + from sphinx.domains.python import PyModulelevel as PyFunction # Support for checking for suspicious markup @@ -238,17 +243,18 @@ def needs_arglist(self): return False -class PyDecoratorFunction(PyDecoratorMixin, PyModulelevel): +class PyDecoratorFunction(PyDecoratorMixin, PyFunction): def run(self): # a decorator function is a function after all self.name = 'py:function' - return PyModulelevel.run(self) + return PyFunction.run(self) -class PyDecoratorMethod(PyDecoratorMixin, PyClassmember): +# TODO: Use sphinx.domains.python.PyDecoratorMethod when possible +class PyDecoratorMethod(PyDecoratorMixin, PyMethod): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) class PyCoroutineMixin(object): @@ -265,31 +271,31 @@ def handle_signature(self, sig, signode): return ret -class PyCoroutineFunction(PyCoroutineMixin, PyModulelevel): +class PyCoroutineFunction(PyCoroutineMixin, PyFunction): def run(self): self.name = 'py:function' - return PyModulelevel.run(self) + return PyFunction.run(self) -class PyCoroutineMethod(PyCoroutineMixin, PyClassmember): +class PyCoroutineMethod(PyCoroutineMixin, PyMethod): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) -class PyAwaitableFunction(PyAwaitableMixin, PyClassmember): +class PyAwaitableFunction(PyAwaitableMixin, PyFunction): def run(self): self.name = 'py:function' - return PyClassmember.run(self) + return PyFunction.run(self) -class PyAwaitableMethod(PyAwaitableMixin, PyClassmember): +class PyAwaitableMethod(PyAwaitableMixin, PyMethod): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) -class PyAbstractMethod(PyClassmember): +class PyAbstractMethod(PyMethod): def handle_signature(self, sig, signode): ret = super(PyAbstractMethod, self).handle_signature(sig, signode) @@ -299,7 +305,7 @@ def handle_signature(self, sig, signode): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) # Support for documenting version of removal in deprecations diff --git a/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst b/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst new file mode 100644 index 0000000000000..089d44e35d2ba --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst @@ -0,0 +1 @@ +Fix RemovedInSphinx40Warning when building the documentation. Patch by Dong-hee Na. From webhook-mailer at python.org Fri Sep 18 05:38:47 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Fri, 18 Sep 2020 09:38:47 -0000 Subject: [Python-checkins] bpo-35293: Remove RemovedInSphinx40Warning (GH-22198) Message-ID: https://github.com/python/cpython/commit/f3e0eceff6207d5072befeef0e1ec195c3e2b0ab commit: f3e0eceff6207d5072befeef0e1ec195c3e2b0ab branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-18T02:38:41-07:00 summary: bpo-35293: Remove RemovedInSphinx40Warning (GH-22198) * bpo-35293: Remove RemovedInSphinx40Warning * Update Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst Co-authored-by: Victor Stinner * bpo-35293: Apply Victor's review Co-authored-by: Victor Stinner (cherry picked from commit 6595cb0af4c51c0381c233b97884fe916a4ddd35) Co-authored-by: Dong-hee Na files: A Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst M Doc/tools/extensions/pyspecific.py diff --git a/Doc/tools/extensions/pyspecific.py b/Doc/tools/extensions/pyspecific.py index f08f4abbfa0f1..2fad9ec00cd56 100644 --- a/Doc/tools/extensions/pyspecific.py +++ b/Doc/tools/extensions/pyspecific.py @@ -31,7 +31,12 @@ from sphinx.util.nodes import split_explicit_title from sphinx.writers.text import TextWriter, TextTranslator from sphinx.writers.latex import LaTeXTranslator -from sphinx.domains.python import PyModulelevel, PyClassmember + +try: + from sphinx.domains.python import PyFunction, PyMethod +except ImportError: + from sphinx.domains.python import PyClassmember as PyMethod + from sphinx.domains.python import PyModulelevel as PyFunction # Support for checking for suspicious markup @@ -238,17 +243,18 @@ def needs_arglist(self): return False -class PyDecoratorFunction(PyDecoratorMixin, PyModulelevel): +class PyDecoratorFunction(PyDecoratorMixin, PyFunction): def run(self): # a decorator function is a function after all self.name = 'py:function' - return PyModulelevel.run(self) + return PyFunction.run(self) -class PyDecoratorMethod(PyDecoratorMixin, PyClassmember): +# TODO: Use sphinx.domains.python.PyDecoratorMethod when possible +class PyDecoratorMethod(PyDecoratorMixin, PyMethod): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) class PyCoroutineMixin(object): @@ -265,31 +271,31 @@ def handle_signature(self, sig, signode): return ret -class PyCoroutineFunction(PyCoroutineMixin, PyModulelevel): +class PyCoroutineFunction(PyCoroutineMixin, PyFunction): def run(self): self.name = 'py:function' - return PyModulelevel.run(self) + return PyFunction.run(self) -class PyCoroutineMethod(PyCoroutineMixin, PyClassmember): +class PyCoroutineMethod(PyCoroutineMixin, PyMethod): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) -class PyAwaitableFunction(PyAwaitableMixin, PyClassmember): +class PyAwaitableFunction(PyAwaitableMixin, PyFunction): def run(self): self.name = 'py:function' - return PyClassmember.run(self) + return PyFunction.run(self) -class PyAwaitableMethod(PyAwaitableMixin, PyClassmember): +class PyAwaitableMethod(PyAwaitableMixin, PyMethod): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) -class PyAbstractMethod(PyClassmember): +class PyAbstractMethod(PyMethod): def handle_signature(self, sig, signode): ret = super(PyAbstractMethod, self).handle_signature(sig, signode) @@ -299,7 +305,7 @@ def handle_signature(self, sig, signode): def run(self): self.name = 'py:method' - return PyClassmember.run(self) + return PyMethod.run(self) # Support for documenting version of removal in deprecations diff --git a/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst b/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst new file mode 100644 index 0000000000000..089d44e35d2ba --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst @@ -0,0 +1 @@ +Fix RemovedInSphinx40Warning when building the documentation. Patch by Dong-hee Na. From webhook-mailer at python.org Fri Sep 18 10:23:23 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 18 Sep 2020 14:23:23 -0000 Subject: [Python-checkins] bpo-35293: Travis CI uses "make venv" for the doc (GH-22307) Message-ID: https://github.com/python/cpython/commit/8394500cca56490cc347604d39ca40abcdce46c3 commit: 8394500cca56490cc347604d39ca40abcdce46c3 branch: master author: Victor Stinner committer: GitHub date: 2020-09-18T16:23:18+02:00 summary: bpo-35293: Travis CI uses "make venv" for the doc (GH-22307) Doc/requirements.txt becomes the reference for packages and package versions needed to build the Python documentation. * Doc/Makefile now uses Doc/requirements.txt * .travis.yml now uses "make env" of Doc/Makefile files: M .travis.yml M Doc/Makefile M Doc/requirements.txt diff --git a/.travis.yml b/.travis.yml index a915f7a46ec3d..254a4ea35ab94 100644 --- a/.travis.yml +++ b/.travis.yml @@ -51,10 +51,7 @@ matrix: env: TESTING=docs before_script: - cd Doc - # Sphinx is pinned so that new versions that introduce new warnings won't suddenly cause build failures. - # (Updating the version is fine as long as no warnings are raised by doing so.) - # The theme used by the docs is stored separately, so we need to install that as well. - - python -m pip install sphinx==2.2.0 blurb python-docs-theme + - make venv PYTHON=python script: - make check suspicious html SPHINXOPTS="-q -W -j4" - name: "Documentation tests" diff --git a/Doc/Makefile b/Doc/Makefile index c11a7ca5c1bcb..f653d70674eb1 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -143,7 +143,7 @@ clean: venv: $(PYTHON) -m venv $(VENVDIR) $(VENVDIR)/bin/python3 -m pip install -U pip setuptools - $(VENVDIR)/bin/python3 -m pip install -U Sphinx==3.2.1 blurb python-docs-theme + $(VENVDIR)/bin/python3 -m pip install -r requirements.txt @echo "The venv has been created in the $(VENVDIR) directory" dist: diff --git a/Doc/requirements.txt b/Doc/requirements.txt index 198446b350ff2..2b70af3a4fc6b 100644 --- a/Doc/requirements.txt +++ b/Doc/requirements.txt @@ -1,5 +1,12 @@ -# Requirements for docs build on netlify -# Pin sphinx to version specified in .travis.yml -sphinx==2.2.0 +# Requirements to build the Python documentation + +# Sphinx version is pinned so that new versions that introduce new warnings +# won't suddenly cause build failures. Updating the version is fine as long +# as no warnings are raised by doing so. +sphinx==3.2.1 + blurb + +# The theme used by the documentation is stored separately, so we need +# to install that as well. python-docs-theme From webhook-mailer at python.org Fri Sep 18 10:53:51 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 18 Sep 2020 14:53:51 -0000 Subject: [Python-checkins] bpo-35293: Travis CI uses "make venv" for the doc (GH-22307) (GH-22309) Message-ID: https://github.com/python/cpython/commit/9e73cac173e5e9010bd18c8334fffeee1cade3a4 commit: 9e73cac173e5e9010bd18c8334fffeee1cade3a4 branch: 3.9 author: Victor Stinner committer: GitHub date: 2020-09-18T16:53:46+02:00 summary: bpo-35293: Travis CI uses "make venv" for the doc (GH-22307) (GH-22309) Doc/requirements.txt becomes the reference for packages and package versions needed to build the Python documentation. * Doc/Makefile now uses Doc/requirements.txt * .travis.yml now uses "make env" of Doc/Makefile * Update Sphinx to version 2.4.4 (cherry picked from commit 8394500cca56490cc347604d39ca40abcdce46c3) files: M .travis.yml M Doc/Makefile M Doc/requirements.txt diff --git a/.travis.yml b/.travis.yml index e565f49262fef..5d9f4208e0431 100644 --- a/.travis.yml +++ b/.travis.yml @@ -51,10 +51,7 @@ matrix: env: TESTING=docs before_script: - cd Doc - # Sphinx is pinned so that new versions that introduce new warnings won't suddenly cause build failures. - # (Updating the version is fine as long as no warnings are raised by doing so.) - # The theme used by the docs is stored separately, so we need to install that as well. - - python -m pip install sphinx==2.2.0 blurb python-docs-theme + - make venv PYTHON=python script: - make check suspicious html SPHINXOPTS="-q -W -j4" - name: "Documentation tests" diff --git a/Doc/Makefile b/Doc/Makefile index b8ca1edfbc60a..f653d70674eb1 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -143,7 +143,7 @@ clean: venv: $(PYTHON) -m venv $(VENVDIR) $(VENVDIR)/bin/python3 -m pip install -U pip setuptools - $(VENVDIR)/bin/python3 -m pip install -U Sphinx==2.3.1 blurb python-docs-theme + $(VENVDIR)/bin/python3 -m pip install -r requirements.txt @echo "The venv has been created in the $(VENVDIR) directory" dist: diff --git a/Doc/requirements.txt b/Doc/requirements.txt index 198446b350ff2..47b78eeac817e 100644 --- a/Doc/requirements.txt +++ b/Doc/requirements.txt @@ -1,5 +1,12 @@ -# Requirements for docs build on netlify -# Pin sphinx to version specified in .travis.yml -sphinx==2.2.0 +# Requirements to build the Python documentation + +# Sphinx version is pinned so that new versions that introduce new warnings +# won't suddenly cause build failures. Updating the version is fine as long +# as no warnings are raised by doing so. +sphinx==2.4.4 + blurb + +# The theme used by the documentation is stored separately, so we need +# to install that as well. python-docs-theme From webhook-mailer at python.org Fri Sep 18 11:21:34 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Fri, 18 Sep 2020 15:21:34 -0000 Subject: [Python-checkins] bpo-35293: Travis CI uses "make venv" for the doc (GH-22307) (GH-22309) (GH-22310) Message-ID: https://github.com/python/cpython/commit/3acc403f21642f1e916f035abf24c3fb81495b3c commit: 3acc403f21642f1e916f035abf24c3fb81495b3c branch: 3.8 author: Victor Stinner committer: GitHub date: 2020-09-18T17:21:24+02:00 summary: bpo-35293: Travis CI uses "make venv" for the doc (GH-22307) (GH-22309) (GH-22310) Doc/requirements.txt becomes the reference for packages and package versions needed to build the Python documentation. * Doc/Makefile now uses Doc/requirements.txt * .travis.yml now uses "make env" of Doc/Makefile * Update Sphinx to version 2.4.4 (cherry picked from commit 8394500cca56490cc347604d39ca40abcdce46c3) (cherry picked from commit 9e73cac173e5e9010bd18c8334fffeee1cade3a4) files: A Doc/requirements.txt M .travis.yml M Doc/Makefile diff --git a/.travis.yml b/.travis.yml index 2b6fed6175265..39a3cf7e92964 100644 --- a/.travis.yml +++ b/.travis.yml @@ -51,10 +51,7 @@ matrix: env: TESTING=docs before_script: - cd Doc - # Sphinx is pinned so that new versions that introduce new warnings won't suddenly cause build failures. - # (Updating the version is fine as long as no warnings are raised by doing so.) - # The theme used by the docs is stored separately, so we need to install that as well. - - python -m pip install sphinx==1.8.2 blurb python-docs-theme + - make venv PYTHON=python script: - make check suspicious html SPHINXOPTS="-q -W -j4" - name: "Documentation tests" diff --git a/Doc/Makefile b/Doc/Makefile index b8ca1edfbc60a..f653d70674eb1 100644 --- a/Doc/Makefile +++ b/Doc/Makefile @@ -143,7 +143,7 @@ clean: venv: $(PYTHON) -m venv $(VENVDIR) $(VENVDIR)/bin/python3 -m pip install -U pip setuptools - $(VENVDIR)/bin/python3 -m pip install -U Sphinx==2.3.1 blurb python-docs-theme + $(VENVDIR)/bin/python3 -m pip install -r requirements.txt @echo "The venv has been created in the $(VENVDIR) directory" dist: diff --git a/Doc/requirements.txt b/Doc/requirements.txt new file mode 100644 index 0000000000000..47b78eeac817e --- /dev/null +++ b/Doc/requirements.txt @@ -0,0 +1,12 @@ +# Requirements to build the Python documentation + +# Sphinx version is pinned so that new versions that introduce new warnings +# won't suddenly cause build failures. Updating the version is fine as long +# as no warnings are raised by doing so. +sphinx==2.4.4 + +blurb + +# The theme used by the documentation is stored separately, so we need +# to install that as well. +python-docs-theme From webhook-mailer at python.org Fri Sep 18 20:57:36 2020 From: webhook-mailer at python.org (Raymond Hettinger) Date: Sat, 19 Sep 2020 00:57:36 -0000 Subject: [Python-checkins] Make fractional value accumulation consistent inside and outside the loop. (GH-22315) Message-ID: https://github.com/python/cpython/commit/ec8a15b034124f3b58d1addda789fa4c20313006 commit: ec8a15b034124f3b58d1addda789fa4c20313006 branch: master author: Raymond Hettinger committer: GitHub date: 2020-09-18T17:57:28-07:00 summary: Make fractional value accumulation consistent inside and outside the loop. (GH-22315) files: M Modules/mathmodule.c diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index ecd291ecd1b4b..935759ec671ca 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -2550,8 +2550,7 @@ vector_norm(Py_ssize_t n, double *vec, double max, int found_nan) assert(csum + lo * lo == csum); frac_lo += lo * lo; } - frac += frac_lo + frac_mid; - h = sqrt(csum - 1.0 + frac); + h = sqrt(csum - 1.0 + (frac_lo + frac_mid + frac)); x = h; t = x * T27; @@ -2569,15 +2568,15 @@ vector_norm(Py_ssize_t n, double *vec, double max, int found_nan) assert(fabs(csum) >= fabs(x)); oldcsum = csum; csum += x; - frac += (oldcsum - csum) + x; + frac_mid += (oldcsum - csum) + x; x = -lo * lo; assert(fabs(csum) >= fabs(x)); oldcsum = csum; csum += x; - frac += (oldcsum - csum) + x; + frac_lo += (oldcsum - csum) + x; - x = csum - 1.0 + frac; + x = csum - 1.0 + (frac_lo + frac_mid + frac); return (h + x / (2.0 * h)) / scale; } /* When max_e < -1023, ldexp(1.0, -max_e) overflows. From webhook-mailer at python.org Fri Sep 18 21:38:46 2020 From: webhook-mailer at python.org (Vladimir Matveev) Date: Sat, 19 Sep 2020 01:38:46 -0000 Subject: [Python-checkins] bpo-41756: Introduce PyGen_Send C API (GH-22196) Message-ID: https://github.com/python/cpython/commit/2b05361bf7cbbd76035206fd9befe87f37489f1e commit: 2b05361bf7cbbd76035206fd9befe87f37489f1e branch: master author: Vladimir Matveev committer: GitHub date: 2020-09-18T18:38:38-07:00 summary: bpo-41756: Introduce PyGen_Send C API (GH-22196) The new API allows to efficiently send values into native generators and coroutines avoiding use of StopIteration exceptions to signal returns. ceval loop now uses this method instead of the old "private" _PyGen_Send C API. This translates to 1.6x increased performance of 'await' calls in micro-benchmarks. Aside from CPython core improvements, this new API will also allow Cython to generate more efficient code, benefiting high-performance IO libraries like uvloop. files: A Misc/NEWS.d/next/Core and Builtins/2020-09-12-12-55-45.bpo-41756.1h0tbV.rst M Doc/c-api/gen.rst M Doc/data/refcounts.dat M Include/genobject.h M Modules/_asynciomodule.c M Objects/genobject.c M Python/ceval.c diff --git a/Doc/c-api/gen.rst b/Doc/c-api/gen.rst index 74410927bfde1..e098425e6364d 100644 --- a/Doc/c-api/gen.rst +++ b/Doc/c-api/gen.rst @@ -15,6 +15,11 @@ than explicitly calling :c:func:`PyGen_New` or :c:func:`PyGen_NewWithQualName`. The C structure used for generator objects. +.. c:type:: PySendResult + + The enum value used to represent different results of :c:func:`PyGen_Send`. + + .. c:var:: PyTypeObject PyGen_Type The type object corresponding to generator objects. @@ -42,3 +47,13 @@ than explicitly calling :c:func:`PyGen_New` or :c:func:`PyGen_NewWithQualName`. with ``__name__`` and ``__qualname__`` set to *name* and *qualname*. A reference to *frame* is stolen by this function. The *frame* argument must not be ``NULL``. + +.. c:function:: PySendResult PyGen_Send(PyGenObject *gen, PyObject *arg, PyObject **presult) + + Sends the *arg* value into the generator *gen*. Coroutine objects + are also allowed to be as the *gen* argument but they need to be + explicitly casted to PyGenObject*. Returns: + + - ``PYGEN_RETURN`` if generator returns. Return value is returned via *presult*. + - ``PYGEN_NEXT`` if generator yields. Yielded value is returned via *presult*. + - ``PYGEN_ERROR`` if generator has raised and exception. *presult* is set to ``NULL``. diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat index 355a4d6d3fa7b..6b1bde37967ae 100644 --- a/Doc/data/refcounts.dat +++ b/Doc/data/refcounts.dat @@ -959,6 +959,11 @@ PyGen_NewWithQualName:PyFrameObject*:frame:0: PyGen_NewWithQualName:PyObject*:name:0: PyGen_NewWithQualName:PyObject*:qualname:0: +PyGen_Send:int::: +PyGen_Send:PyGenObject*:gen:0: +PyGen_Send:PyObject*:arg:0: +PyGen_Send:PyObject**:presult:+1: + PyCoro_CheckExact:int::: PyCoro_CheckExact:PyObject*:ob:0: diff --git a/Include/genobject.h b/Include/genobject.h index a76dc92e811c4..7488054c68fcd 100644 --- a/Include/genobject.h +++ b/Include/genobject.h @@ -45,6 +45,21 @@ PyAPI_FUNC(PyObject *) _PyGen_Send(PyGenObject *, PyObject *); PyObject *_PyGen_yf(PyGenObject *); PyAPI_FUNC(void) _PyGen_Finalize(PyObject *self); +typedef enum { + PYGEN_RETURN = 0, + PYGEN_ERROR = -1, + PYGEN_NEXT = 1, +} PySendResult; + +/* Sends the value into the generator or the coroutine. Returns: + - PYGEN_RETURN (0) if generator has returned. + 'result' parameter is filled with return value + - PYGEN_ERROR (-1) if exception was raised. + 'result' parameter is NULL + - PYGEN_NEXT (1) if generator has yielded. + 'result' parameter is filled with yielded value. */ +PyAPI_FUNC(PySendResult) PyGen_Send(PyGenObject *, PyObject *, PyObject **); + #ifndef Py_LIMITED_API typedef struct { _PyGenObject_HEAD(cr) diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-12-12-55-45.bpo-41756.1h0tbV.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-12-12-55-45.bpo-41756.1h0tbV.rst new file mode 100644 index 0000000000000..b387cfd94033c --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-12-12-55-45.bpo-41756.1h0tbV.rst @@ -0,0 +1,2 @@ +Add PyGen_Send function to allow sending value into generator/coroutine +without raising StopIteration exception to signal return diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 4a1c91e9eddd6..2151f20281a31 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -2621,6 +2621,20 @@ task_set_error_soon(TaskObj *task, PyObject *et, const char *format, ...) Py_RETURN_NONE; } +static inline int +gen_status_from_result(PyObject **result) +{ + if (*result != NULL) { + return PYGEN_NEXT; + } + if (_PyGen_FetchStopIterationValue(result) == 0) { + return PYGEN_RETURN; + } + + assert(PyErr_Occurred()); + return PYGEN_ERROR; +} + static PyObject * task_step_impl(TaskObj *task, PyObject *exc) { @@ -2679,26 +2693,29 @@ task_step_impl(TaskObj *task, PyObject *exc) return NULL; } + int gen_status = PYGEN_ERROR; if (exc == NULL) { if (PyGen_CheckExact(coro) || PyCoro_CheckExact(coro)) { - result = _PyGen_Send((PyGenObject*)coro, Py_None); + gen_status = PyGen_Send((PyGenObject*)coro, Py_None, &result); } else { result = _PyObject_CallMethodIdOneArg(coro, &PyId_send, Py_None); + gen_status = gen_status_from_result(&result); } } else { result = _PyObject_CallMethodIdOneArg(coro, &PyId_throw, exc); + gen_status = gen_status_from_result(&result); if (clear_exc) { /* We created 'exc' during this call */ Py_DECREF(exc); } } - if (result == NULL) { + if (gen_status == PYGEN_RETURN || gen_status == PYGEN_ERROR) { PyObject *et, *ev, *tb; - if (_PyGen_FetchStopIterationValue(&o) == 0) { + if (result != NULL) { /* The error is StopIteration and that means that the underlying coroutine has resolved */ @@ -2709,10 +2726,10 @@ task_step_impl(TaskObj *task, PyObject *exc) res = future_cancel((FutureObj*)task, task->task_cancel_msg); } else { - res = future_set_result((FutureObj*)task, o); + res = future_set_result((FutureObj*)task, result); } - Py_DECREF(o); + Py_DECREF(result); if (res == NULL) { return NULL; diff --git a/Objects/genobject.c b/Objects/genobject.c index 809838a4cd2f3..24aca988354c5 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -137,7 +137,7 @@ gen_dealloc(PyGenObject *gen) } static PyObject * -gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing) +gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing, int *is_return_value) { PyThreadState *tstate = _PyThreadState_GET(); PyFrameObject *f = gen->gi_frame; @@ -170,6 +170,10 @@ gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing) PyErr_SetNone(PyExc_StopAsyncIteration); } else { + if (is_return_value != NULL) { + *is_return_value = 1; + Py_RETURN_NONE; + } PyErr_SetNone(PyExc_StopIteration); } } @@ -230,18 +234,33 @@ gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing) /* Delay exception instantiation if we can */ if (PyAsyncGen_CheckExact(gen)) { PyErr_SetNone(PyExc_StopAsyncIteration); + Py_CLEAR(result); } else if (arg) { - /* Set exception if not called by gen_iternext() */ - PyErr_SetNone(PyExc_StopIteration); + if (is_return_value != NULL) { + *is_return_value = 1; + } + else { + /* Set exception if not called by gen_iternext() */ + PyErr_SetNone(PyExc_StopIteration); + Py_CLEAR(result); + } + } + else { + Py_CLEAR(result); } } else { /* Async generators cannot return anything but None */ assert(!PyAsyncGen_CheckExact(gen)); - _PyGen_SetStopIterationValue(result); + if (is_return_value != NULL) { + *is_return_value = 1; + } + else { + _PyGen_SetStopIterationValue(result); + Py_CLEAR(result); + } } - Py_CLEAR(result); } else if (!result && PyErr_ExceptionMatches(PyExc_StopIteration)) { const char *msg = "generator raised StopIteration"; @@ -264,7 +283,7 @@ gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing) _PyErr_FormatFromCause(PyExc_RuntimeError, "%s", msg); } - if (!result || _PyFrameHasCompleted(f)) { + if ((is_return_value && *is_return_value) || !result || _PyFrameHasCompleted(f)) { /* generator can't be rerun, so release the frame */ /* first clean reference cycle through stored exception traceback */ _PyErr_ClearExcState(&gen->gi_exc_state); @@ -283,7 +302,19 @@ return next yielded value or raise StopIteration."); PyObject * _PyGen_Send(PyGenObject *gen, PyObject *arg) { - return gen_send_ex(gen, arg, 0, 0); + return gen_send_ex(gen, arg, 0, 0, NULL); +} + +PySendResult +PyGen_Send(PyGenObject *gen, PyObject *arg, PyObject **result) +{ + assert(result != NULL); + + int is_return_value = 0; + if ((*result = gen_send_ex(gen, arg, 0, 0, &is_return_value)) == NULL) { + return PYGEN_ERROR; + } + return is_return_value ? PYGEN_RETURN : PYGEN_NEXT; } PyDoc_STRVAR(close_doc, @@ -365,7 +396,7 @@ gen_close(PyGenObject *gen, PyObject *args) } if (err == 0) PyErr_SetNone(PyExc_GeneratorExit); - retval = gen_send_ex(gen, Py_None, 1, 1); + retval = gen_send_ex(gen, Py_None, 1, 1, NULL); if (retval) { const char *msg = "generator ignored GeneratorExit"; if (PyCoro_CheckExact(gen)) { @@ -413,7 +444,7 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, gen->gi_frame->f_state = state; Py_DECREF(yf); if (err < 0) - return gen_send_ex(gen, Py_None, 1, 0); + return gen_send_ex(gen, Py_None, 1, 0, NULL); goto throw_here; } if (PyGen_CheckExact(yf) || PyCoro_CheckExact(yf)) { @@ -465,10 +496,10 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, assert(gen->gi_frame->f_lasti >= 0); gen->gi_frame->f_lasti += sizeof(_Py_CODEUNIT); if (_PyGen_FetchStopIterationValue(&val) == 0) { - ret = gen_send_ex(gen, val, 0, 0); + ret = gen_send_ex(gen, val, 0, 0, NULL); Py_DECREF(val); } else { - ret = gen_send_ex(gen, Py_None, 1, 0); + ret = gen_send_ex(gen, Py_None, 1, 0, NULL); } } return ret; @@ -522,7 +553,7 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, } PyErr_Restore(typ, val, tb); - return gen_send_ex(gen, Py_None, 1, 0); + return gen_send_ex(gen, Py_None, 1, 0, NULL); failed_throw: /* Didn't use our arguments, so restore their original refcounts */ @@ -551,7 +582,7 @@ gen_throw(PyGenObject *gen, PyObject *args) static PyObject * gen_iternext(PyGenObject *gen) { - return gen_send_ex(gen, NULL, 0, 0); + return gen_send_ex(gen, NULL, 0, 0, NULL); } /* @@ -1051,13 +1082,13 @@ coro_wrapper_dealloc(PyCoroWrapper *cw) static PyObject * coro_wrapper_iternext(PyCoroWrapper *cw) { - return gen_send_ex((PyGenObject *)cw->cw_coroutine, NULL, 0, 0); + return gen_send_ex((PyGenObject *)cw->cw_coroutine, NULL, 0, 0, NULL); } static PyObject * coro_wrapper_send(PyCoroWrapper *cw, PyObject *arg) { - return gen_send_ex((PyGenObject *)cw->cw_coroutine, arg, 0, 0); + return gen_send_ex((PyGenObject *)cw->cw_coroutine, arg, 0, 0, NULL); } static PyObject * @@ -1570,7 +1601,7 @@ async_gen_asend_send(PyAsyncGenASend *o, PyObject *arg) } o->ags_gen->ag_running_async = 1; - result = gen_send_ex((PyGenObject*)o->ags_gen, arg, 0, 0); + result = gen_send_ex((PyGenObject*)o->ags_gen, arg, 0, 0, NULL); result = async_gen_unwrap_value(o->ags_gen, result); if (result == NULL) { @@ -1926,7 +1957,7 @@ async_gen_athrow_send(PyAsyncGenAThrow *o, PyObject *arg) assert(o->agt_state == AWAITABLE_STATE_ITER); - retval = gen_send_ex((PyGenObject *)gen, arg, 0, 0); + retval = gen_send_ex((PyGenObject *)gen, arg, 0, 0, NULL); if (o->agt_args) { return async_gen_unwrap_value(o->agt_gen, retval); } else { diff --git a/Python/ceval.c b/Python/ceval.c index f747faaebf024..3de372f45a251 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2223,29 +2223,53 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag) case TARGET(YIELD_FROM): { PyObject *v = POP(); PyObject *receiver = TOP(); - int err; - if (PyGen_CheckExact(receiver) || PyCoro_CheckExact(receiver)) { - retval = _PyGen_Send((PyGenObject *)receiver, v); + int is_gen_or_coro = PyGen_CheckExact(receiver) || PyCoro_CheckExact(receiver); + int gen_status; + if (tstate->c_tracefunc == NULL && is_gen_or_coro) { + gen_status = PyGen_Send((PyGenObject *)receiver, v, &retval); } else { - _Py_IDENTIFIER(send); - if (v == Py_None) - retval = Py_TYPE(receiver)->tp_iternext(receiver); - else - retval = _PyObject_CallMethodIdOneArg(receiver, &PyId_send, v); + if (is_gen_or_coro) { + retval = _PyGen_Send((PyGenObject *)receiver, v); + } + else { + _Py_IDENTIFIER(send); + if (v == Py_None) { + retval = Py_TYPE(receiver)->tp_iternext(receiver); + } + else { + retval = _PyObject_CallMethodIdOneArg(receiver, &PyId_send, v); + } + } + + if (retval == NULL) { + if (tstate->c_tracefunc != NULL + && _PyErr_ExceptionMatches(tstate, PyExc_StopIteration)) + call_exc_trace(tstate->c_tracefunc, tstate->c_traceobj, tstate, f); + if (_PyGen_FetchStopIterationValue(&retval) == 0) { + gen_status = PYGEN_RETURN; + } + else { + gen_status = PYGEN_ERROR; + } + } + else { + gen_status = PYGEN_NEXT; + } } Py_DECREF(v); - if (retval == NULL) { - PyObject *val; - if (tstate->c_tracefunc != NULL - && _PyErr_ExceptionMatches(tstate, PyExc_StopIteration)) - call_exc_trace(tstate->c_tracefunc, tstate->c_traceobj, tstate, f); - err = _PyGen_FetchStopIterationValue(&val); - if (err < 0) - goto error; + if (gen_status == PYGEN_ERROR) { + assert (retval == NULL); + goto error; + } + if (gen_status == PYGEN_RETURN) { + assert (retval != NULL); + Py_DECREF(receiver); - SET_TOP(val); + SET_TOP(retval); + retval = NULL; DISPATCH(); } + assert (gen_status == PYGEN_NEXT); /* receiver remains on stack, retval is value to be yielded */ /* and repeat... */ assert(f->f_lasti >= (int)sizeof(_Py_CODEUNIT)); From webhook-mailer at python.org Sat Sep 19 14:13:24 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Sat, 19 Sep 2020 18:13:24 -0000 Subject: [Python-checkins] bpo-41811: create SortKey members using first given value (GH-22316) Message-ID: https://github.com/python/cpython/commit/ae0d2a33ec05aece939a959d36fcf1df1e210a08 commit: ae0d2a33ec05aece939a959d36fcf1df1e210a08 branch: master author: Ethan Furman committer: GitHub date: 2020-09-19T11:12:57-07:00 summary: bpo-41811: create SortKey members using first given value (GH-22316) files: M Lib/pstats.py M Lib/test/test_pstats.py diff --git a/Lib/pstats.py b/Lib/pstats.py index e781b91c6052c..0f93ae02c9507 100644 --- a/Lib/pstats.py +++ b/Lib/pstats.py @@ -45,9 +45,9 @@ class SortKey(str, Enum): TIME = 'time', 'tottime' def __new__(cls, *values): - obj = str.__new__(cls) - - obj._value_ = values[0] + value = values[0] + obj = str.__new__(cls, value) + obj._value_ = value for other_value in values[1:]: cls._value2member_map_[other_value] = obj obj._all_values = values diff --git a/Lib/test/test_pstats.py b/Lib/test/test_pstats.py index 10559deb6bcb2..4f78b99fd1cae 100644 --- a/Lib/test/test_pstats.py +++ b/Lib/test/test_pstats.py @@ -95,5 +95,9 @@ def pass3(): pass self.assertIn('pass2', funcs_called) self.assertIn('pass3', funcs_called) + def test_SortKey_enum(self): + self.assertEqual(SortKey.FILENAME, 'filename') + self.assertNotEqual(SortKey.FILENAME, SortKey.CALLS) + if __name__ == "__main__": unittest.main() From webhook-mailer at python.org Sat Sep 19 15:13:37 2020 From: webhook-mailer at python.org (idomic) Date: Sat, 19 Sep 2020 19:13:37 -0000 Subject: [Python-checkins] bpo-33689: Blank lines in .pth file cause a duplicate sys.path entry (GH-20679) Message-ID: https://github.com/python/cpython/commit/0c71a66b53f6ea262aa5a60784c8c625ae0bb98c commit: 0c71a66b53f6ea262aa5a60784c8c625ae0bb98c branch: master author: idomic committer: GitHub date: 2020-09-19T22:13:29+03:00 summary: bpo-33689: Blank lines in .pth file cause a duplicate sys.path entry (GH-20679) files: A Misc/NEWS.d/next/Library/2020-06-06-14-09-55.bpo-33689.EFUDH7.rst M Lib/site.py M Lib/test/test_site.py diff --git a/Lib/site.py b/Lib/site.py index 4c095774729c5..4d3b869fff77a 100644 --- a/Lib/site.py +++ b/Lib/site.py @@ -170,6 +170,8 @@ def addpackage(sitedir, name, known_paths): for n, line in enumerate(f): if line.startswith("#"): continue + if line.strip() == "": + continue try: if line.startswith(("import ", "import\t")): exec(line) diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py index 97b5c5de95bbc..d3ee68facdbc3 100644 --- a/Lib/test/test_site.py +++ b/Lib/test/test_site.py @@ -161,6 +161,12 @@ def test_addpackage_import_bad_exec(self): self.assertRegex(err_out.getvalue(), 'Traceback') self.assertRegex(err_out.getvalue(), 'ModuleNotFoundError') + def test_addpackage_empty_lines(self): + # Issue 33689 + pth_dir, pth_fn = self.make_pth("\n\n \n\n") + known_paths = site.addpackage(pth_dir, pth_fn, set()) + self.assertEqual(known_paths, set()) + def test_addpackage_import_bad_pth_file(self): # Issue 5258 pth_dir, pth_fn = self.make_pth("abc\x00def\n") @@ -595,7 +601,6 @@ def test_startup_interactivehook_isolated_explicit(self): 'import site, sys; site.enablerlcompleter(); sys.exit(hasattr(sys, "__interactivehook__"))']).wait() self.assertTrue(r, "'__interactivehook__' not added by enablerlcompleter()") - @unittest.skipUnless(sys.platform == 'win32', "only supported on Windows") class _pthFileTests(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Library/2020-06-06-14-09-55.bpo-33689.EFUDH7.rst b/Misc/NEWS.d/next/Library/2020-06-06-14-09-55.bpo-33689.EFUDH7.rst new file mode 100644 index 0000000000000..bc0756d02ddc9 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-06-06-14-09-55.bpo-33689.EFUDH7.rst @@ -0,0 +1,4 @@ +Ignore empty or whitespace-only lines in .pth files. This matches the +documentated behavior. Before, empty lines caused the site-packages +dir to appear multiple times in sys.path. +By Ido Michael, contributors Malcolm Smith and Tal Einat. From webhook-mailer at python.org Sat Sep 19 15:56:21 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 19 Sep 2020 19:56:21 -0000 Subject: [Python-checkins] bpo-41811: create SortKey members using first given value (GH-22316) (GH-22325) Message-ID: https://github.com/python/cpython/commit/0e4d526de47eae24b75f3623cd56bb0453fc8c74 commit: 0e4d526de47eae24b75f3623cd56bb0453fc8c74 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-19T12:56:13-07:00 summary: bpo-41811: create SortKey members using first given value (GH-22316) (GH-22325) (cherry picked from commit ae0d2a33ec05aece939a959d36fcf1df1e210a08) Co-authored-by: Ethan Furman files: M Lib/pstats.py M Lib/test/test_pstats.py diff --git a/Lib/pstats.py b/Lib/pstats.py index e781b91c6052c..0f93ae02c9507 100644 --- a/Lib/pstats.py +++ b/Lib/pstats.py @@ -45,9 +45,9 @@ class SortKey(str, Enum): TIME = 'time', 'tottime' def __new__(cls, *values): - obj = str.__new__(cls) - - obj._value_ = values[0] + value = values[0] + obj = str.__new__(cls, value) + obj._value_ = value for other_value in values[1:]: cls._value2member_map_[other_value] = obj obj._all_values = values diff --git a/Lib/test/test_pstats.py b/Lib/test/test_pstats.py index 10559deb6bcb2..4f78b99fd1cae 100644 --- a/Lib/test/test_pstats.py +++ b/Lib/test/test_pstats.py @@ -95,5 +95,9 @@ def pass3(): pass self.assertIn('pass2', funcs_called) self.assertIn('pass3', funcs_called) + def test_SortKey_enum(self): + self.assertEqual(SortKey.FILENAME, 'filename') + self.assertNotEqual(SortKey.FILENAME, SortKey.CALLS) + if __name__ == "__main__": unittest.main() From webhook-mailer at python.org Sat Sep 19 15:56:34 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sat, 19 Sep 2020 19:56:34 -0000 Subject: [Python-checkins] bpo-41811: create SortKey members using first given value (GH-22316) (GH-22326) Message-ID: https://github.com/python/cpython/commit/488e3eb70d24d5ce55ae0d7aed13a3721d3eb8a1 commit: 488e3eb70d24d5ce55ae0d7aed13a3721d3eb8a1 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-19T12:56:30-07:00 summary: bpo-41811: create SortKey members using first given value (GH-22316) (GH-22326) (cherry picked from commit ae0d2a33ec05aece939a959d36fcf1df1e210a08) Co-authored-by: Ethan Furman files: M Lib/pstats.py M Lib/test/test_pstats.py diff --git a/Lib/pstats.py b/Lib/pstats.py index 4b419a8ecdb6c..67a68ebe750b5 100644 --- a/Lib/pstats.py +++ b/Lib/pstats.py @@ -43,9 +43,9 @@ class SortKey(str, Enum): TIME = 'time', 'tottime' def __new__(cls, *values): - obj = str.__new__(cls) - - obj._value_ = values[0] + value = values[0] + obj = str.__new__(cls, value) + obj._value_ = value for other_value in values[1:]: cls._value2member_map_[other_value] = obj obj._all_values = values diff --git a/Lib/test/test_pstats.py b/Lib/test/test_pstats.py index f835ce309a60e..052a3b9c3d5ee 100644 --- a/Lib/test/test_pstats.py +++ b/Lib/test/test_pstats.py @@ -76,5 +76,9 @@ def test_sort_starts_mix(self): 'calls') + def test_SortKey_enum(self): + self.assertEqual(SortKey.FILENAME, 'filename') + self.assertNotEqual(SortKey.FILENAME, SortKey.CALLS) + if __name__ == "__main__": unittest.main() From webhook-mailer at python.org Sat Sep 19 16:38:15 2020 From: webhook-mailer at python.org (Mark Dickinson) Date: Sat, 19 Sep 2020 20:38:15 -0000 Subject: [Python-checkins] Add missing whatsnew entry for TestCase.assertNoLogs (GH-22317) Message-ID: https://github.com/python/cpython/commit/c8c70e78762315643fcd132911bad38842a3e8af commit: c8c70e78762315643fcd132911bad38842a3e8af branch: master author: Mark Dickinson committer: GitHub date: 2020-09-19T21:38:11+01:00 summary: Add missing whatsnew entry for TestCase.assertNoLogs (GH-22317) files: M Doc/whatsnew/3.10.rst M Misc/ACKS diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index f6f276a8bfa49..ce888fec1d8c9 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -145,6 +145,13 @@ Add :data:`sys.orig_argv` attribute: the list of the original command line arguments passed to the Python executable. (Contributed by Victor Stinner in :issue:`23427`.) +unittest +-------- + +Add new method :meth:`~unittest.TestCase.assertNoLogs` to complement the +existing :meth:`~unittest.TestCase.assertLogs`. (Contributed by Kit Yan Choi +in :issue:`39385`.) + xml --- diff --git a/Misc/ACKS b/Misc/ACKS index d5bdb084a1e1e..d23797a2f5557 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -306,6 +306,7 @@ Albert Chin-A-Young Adal Chiriliuc Matt Chisholm Lita Cho +Kit Yan Choi Sayan Chowdhury Yuan-Chao Chou Anders Chrigstr?m From webhook-mailer at python.org Sat Sep 19 23:41:00 2020 From: webhook-mailer at python.org (Peter McCormick) Date: Sun, 20 Sep 2020 03:41:00 -0000 Subject: [Python-checkins] bpo-41815: SQLite: segfault if backup called on closed database (GH-22322) Message-ID: https://github.com/python/cpython/commit/bfee9fad84531a471fd7864e88947320669f68e2 commit: bfee9fad84531a471fd7864e88947320669f68e2 branch: master author: Peter McCormick committer: GitHub date: 2020-09-19T20:40:46-07:00 summary: bpo-41815: SQLite: segfault if backup called on closed database (GH-22322) # [bpo-41815](): SQLite: fix segfault if backup called on closed database Attempting to backup a closed database will trigger segfault: ```python import sqlite3 target = sqlite3.connect(':memory:') source = sqlite3.connect(':memory:') source.close() source.backup(target) ``` files: A Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst M Lib/sqlite3/test/backup.py M Modules/_sqlite/connection.c diff --git a/Lib/sqlite3/test/backup.py b/Lib/sqlite3/test/backup.py index 2752a4db337dd..3637c4bb21b6d 100644 --- a/Lib/sqlite3/test/backup.py +++ b/Lib/sqlite3/test/backup.py @@ -35,6 +35,13 @@ def test_bad_target_closed_connection(self): with self.assertRaises(sqlite.ProgrammingError): self.cx.backup(bck) + def test_bad_source_closed_connection(self): + bck = sqlite.connect(':memory:') + source = sqlite.connect(":memory:") + source.close() + with self.assertRaises(sqlite.ProgrammingError): + source.backup(bck) + def test_bad_target_in_transaction(self): bck = sqlite.connect(':memory:') bck.execute('CREATE TABLE bar (key INTEGER)') diff --git a/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst b/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst new file mode 100644 index 0000000000000..3560db9bc5d35 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst @@ -0,0 +1,2 @@ +Fix SQLite3 segfault when backing up closed database. Patch contributed by +Peter David McCormick. diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index f765ba1df2466..81fc1335371a6 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -1514,6 +1514,10 @@ pysqlite_connection_backup(pysqlite_Connection *self, PyObject *args, PyObject * sleep_ms = (int)ms; } + if (!pysqlite_check_thread(self) || !pysqlite_check_connection(self)) { + return NULL; + } + if (!pysqlite_check_connection((pysqlite_Connection *)target)) { return NULL; } From webhook-mailer at python.org Sun Sep 20 02:38:15 2020 From: webhook-mailer at python.org (Berker Peksag) Date: Sun, 20 Sep 2020 06:38:15 -0000 Subject: [Python-checkins] bpo-12178: Fix escaping of escapechar in csv.writer() (GH-13710) Message-ID: https://github.com/python/cpython/commit/5c0eed7375fdd791cc5e19ceabfab4170ad44062 commit: 5c0eed7375fdd791cc5e19ceabfab4170ad44062 branch: master author: Berker Peksag committer: GitHub date: 2020-09-20T09:38:07+03:00 summary: bpo-12178: Fix escaping of escapechar in csv.writer() (GH-13710) Co-authored-by: Itay Elbirt files: A Misc/NEWS.d/next/Library/2019-05-31-23-54-28.bpo-12178.N6FLCZ.rst M Lib/test/test_csv.py M Modules/_csv.c diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py index 38160220853ea..a98707ce3caed 100644 --- a/Lib/test/test_csv.py +++ b/Lib/test/test_csv.py @@ -202,6 +202,20 @@ def test_write_escape(self): escapechar='\\', quoting = csv.QUOTE_NONE) self._write_test(['a',1,'p,q'], 'a,1,p\\,q', escapechar='\\', quoting = csv.QUOTE_NONE) + self._write_test(['\\', 'a'], '\\\\,a', + escapechar='\\', quoting=csv.QUOTE_NONE) + self._write_test(['\\', 'a'], '\\\\,a', + escapechar='\\', quoting=csv.QUOTE_MINIMAL) + self._write_test(['\\', 'a'], '"\\\\","a"', + escapechar='\\', quoting=csv.QUOTE_ALL) + self._write_test(['\\ ', 'a'], '\\\\ ,a', + escapechar='\\', quoting=csv.QUOTE_MINIMAL) + self._write_test(['\\,', 'a'], '\\\\\\,,a', + escapechar='\\', quoting=csv.QUOTE_NONE) + self._write_test([',\\', 'a'], '",\\\\",a', + escapechar='\\', quoting=csv.QUOTE_MINIMAL) + self._write_test(['C\\', '6', '7', 'X"'], 'C\\\\,6,7,"X"""', + escapechar='\\', quoting=csv.QUOTE_MINIMAL) def test_write_iterable(self): self._write_test(iter(['a', 1, 'p,q']), 'a,1,"p,q"') diff --git a/Misc/NEWS.d/next/Library/2019-05-31-23-54-28.bpo-12178.N6FLCZ.rst b/Misc/NEWS.d/next/Library/2019-05-31-23-54-28.bpo-12178.N6FLCZ.rst new file mode 100644 index 0000000000000..80e2a7b5fbb2c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-05-31-23-54-28.bpo-12178.N6FLCZ.rst @@ -0,0 +1,3 @@ +:func:`csv.writer` now correctly escapes *escapechar* when input +contains *escapechar*. Patch by Catalin Iacob, Berker Peksag, +and Itay Elbirt. diff --git a/Modules/_csv.c b/Modules/_csv.c index da61db9377f94..594f6c1472726 100644 --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -1040,6 +1040,9 @@ join_append_data(WriterObj *self, unsigned int field_kind, const void *field_dat else want_escape = 1; } + else if (c == dialect->escapechar) { + want_escape = 1; + } if (!want_escape) *quoted = 1; } From webhook-mailer at python.org Sun Sep 20 13:09:55 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Sun, 20 Sep 2020 17:09:55 -0000 Subject: [Python-checkins] [doc] Teach 0-args form of super in Programming FAQ (GH-22176) Message-ID: https://github.com/python/cpython/commit/778ad926cbdda0632f50acb7fac3a2a8b80dcf69 commit: 778ad926cbdda0632f50acb7fac3a2a8b80dcf69 branch: master author: Andre Delfino committer: GitHub date: 2020-09-20T10:09:50-07:00 summary: [doc] Teach 0-args form of super in Programming FAQ (GH-22176) files: M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index eecbbf4a5c4ff..fd0adc378bfa6 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1504,20 +1504,19 @@ Most :meth:`__setattr__` implementations must modify ``self.__dict__`` to store local state for self without causing an infinite recursion. -How do I call a method defined in a base class from a derived class that overrides it? --------------------------------------------------------------------------------------- +How do I call a method defined in a base class from a derived class that extends it? +------------------------------------------------------------------------------------ Use the built-in :func:`super` function:: class Derived(Base): def meth(self): - super(Derived, self).meth() + super().meth() # calls Base.meth -For version prior to 3.0, you may be using classic classes: For a class -definition such as ``class Derived(Base): ...`` you can call method ``meth()`` -defined in ``Base`` (or one of ``Base``'s base classes) as ``Base.meth(self, -arguments...)``. Here, ``Base.meth`` is an unbound method, so you need to -provide the ``self`` argument. +In the example, :func:`super` will automatically determine the instance from +which it was called (the ``self`` value), look up the :term:`method resolution +order` (MRO) with ``type(self).__mro__``, and return the next in line after +``Derived`` in the MRO: ``Base``. How can I organize my code to make it easier to change the base class? From webhook-mailer at python.org Mon Sep 21 00:48:04 2020 From: webhook-mailer at python.org (Raymond Hettinger) Date: Mon, 21 Sep 2020 04:48:04 -0000 Subject: [Python-checkins] bpo-41513: Add accuracy tests for math.hypot() (GH-22327) Message-ID: https://github.com/python/cpython/commit/bc6b7fa6d7fb8957eb4ff809366af40dfb12b8cd commit: bc6b7fa6d7fb8957eb4ff809366af40dfb12b8cd branch: master author: Raymond Hettinger committer: GitHub date: 2020-09-20T21:47:56-07:00 summary: bpo-41513: Add accuracy tests for math.hypot() (GH-22327) files: M Lib/test/test_math.py diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py index f5283c5e0dcb6..2abe5b028b355 100644 --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -803,6 +803,69 @@ def testHypot(self): scale = FLOAT_MIN / 2.0 ** exp self.assertEqual(math.hypot(4*scale, 3*scale), 5*scale) + @requires_IEEE_754 + @unittest.skipIf(HAVE_DOUBLE_ROUNDING, + "hypot() loses accuracy on machines with double rounding") + def testHypotAccuracy(self): + # Verify improved accuracy in cases that were known to be inaccurate. + # + # The new algorithm's accuracy depends on IEEE 754 arithmetic + # guarantees, on having the usual ROUND HALF EVEN rounding mode, on + # the system not having double rounding due to extended precision, + # and on the compiler maintaining the specified order of operations. + # + # This test is known to succeed on most of our builds. If it fails + # some build, we either need to add another skipIf if the cause is + # identifiable; otherwise, we can remove this test entirely. + + hypot = math.hypot + Decimal = decimal.Decimal + high_precision = decimal.Context(prec=500) + + for hx, hy in [ + # Cases with a 1 ulp error in Python 3.7 compiled with Clang + ('0x1.10e89518dca48p+29', '0x1.1970f7565b7efp+30'), + ('0x1.10106eb4b44a2p+29', '0x1.ef0596cdc97f8p+29'), + ('0x1.459c058e20bb7p+30', '0x1.993ca009b9178p+29'), + ('0x1.378371ae67c0cp+30', '0x1.fbe6619854b4cp+29'), + ('0x1.f4cd0574fb97ap+29', '0x1.50fe31669340ep+30'), + ('0x1.494b2cdd3d446p+29', '0x1.212a5367b4c7cp+29'), + ('0x1.f84e649f1e46dp+29', '0x1.1fa56bef8eec4p+30'), + ('0x1.2e817edd3d6fap+30', '0x1.eb0814f1e9602p+29'), + ('0x1.0d3a6e3d04245p+29', '0x1.32a62fea52352p+30'), + ('0x1.888e19611bfc5p+29', '0x1.52b8e70b24353p+29'), + + # Cases with 2 ulp error in Python 3.8 + ('0x1.538816d48a13fp+29', '0x1.7967c5ca43e16p+29'), + ('0x1.57b47b7234530p+29', '0x1.74e2c7040e772p+29'), + ('0x1.821b685e9b168p+30', '0x1.677dc1c1e3dc6p+29'), + ('0x1.9e8247f67097bp+29', '0x1.24bd2dc4f4baep+29'), + ('0x1.b73b59e0cb5f9p+29', '0x1.da899ab784a97p+28'), + ('0x1.94a8d2842a7cfp+30', '0x1.326a51d4d8d8ap+30'), + ('0x1.e930b9cd99035p+29', '0x1.5a1030e18dff9p+30'), + ('0x1.1592bbb0e4690p+29', '0x1.a9c337b33fb9ap+29'), + ('0x1.1243a50751fd4p+29', '0x1.a5a10175622d9p+29'), + ('0x1.57a8596e74722p+30', '0x1.42d1af9d04da9p+30'), + + # Cases with 1 ulp error in version fff3c28052e6b0 + ('0x1.ee7dbd9565899p+29', '0x1.7ab4d6fc6e4b4p+29'), + ('0x1.5c6bfbec5c4dcp+30', '0x1.02511184b4970p+30'), + ('0x1.59dcebba995cap+30', '0x1.50ca7e7c38854p+29'), + ('0x1.768cdd94cf5aap+29', '0x1.9cfdc5571d38ep+29'), + ('0x1.dcf137d60262ep+29', '0x1.1101621990b3ep+30'), + ('0x1.3a2d006e288b0p+30', '0x1.e9a240914326cp+29'), + ('0x1.62a32f7f53c61p+29', '0x1.47eb6cd72684fp+29'), + ('0x1.d3bcb60748ef2p+29', '0x1.3f13c4056312cp+30'), + ('0x1.282bdb82f17f3p+30', '0x1.640ba4c4eed3ap+30'), + ('0x1.89d8c423ea0c6p+29', '0x1.d35dcfe902bc3p+29'), + ]: + x = float.fromhex(hx) + y = float.fromhex(hy) + with self.subTest(hx=hx, hy=hy, x=x, y=y): + with decimal.localcontext(high_precision): + z = float((Decimal(x)**2 + Decimal(y)**2).sqrt()) + self.assertEqual(hypot(x, y), z) + def testDist(self): from decimal import Decimal as D from fractions import Fraction as F From webhook-mailer at python.org Mon Sep 21 04:35:34 2020 From: webhook-mailer at python.org (Samuel Marks) Date: Mon, 21 Sep 2020 08:35:34 -0000 Subject: [Python-checkins] bpo-41819: Fix compiler warning in init_dump_ascii_wstr() (GH-22332) Message-ID: https://github.com/python/cpython/commit/c322948892438a387d752ec18d1eb512699a4d67 commit: c322948892438a387d752ec18d1eb512699a4d67 branch: master author: Samuel Marks <807580+SamuelMarks at users.noreply.github.com> committer: GitHub date: 2020-09-21T10:35:17+02:00 summary: bpo-41819: Fix compiler warning in init_dump_ascii_wstr() (GH-22332) Fix the compiler warning: format specifies type `wint_t` (aka `int`) but the argument has type `unsigned int` files: M Python/initconfig.c diff --git a/Python/initconfig.c b/Python/initconfig.c index 880e145ec031c..6a13dc52ed776 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -2674,7 +2674,7 @@ init_dump_ascii_wstr(const wchar_t *str) if (ch == L'\'') { PySys_WriteStderr("\\'"); } else if (0x20 <= ch && ch < 0x7f) { - PySys_WriteStderr("%lc", ch); + PySys_WriteStderr("%c", ch); } else if (ch <= 0xff) { PySys_WriteStderr("\\x%02x", ch); From webhook-mailer at python.org Mon Sep 21 04:58:56 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 21 Sep 2020 08:58:56 -0000 Subject: [Python-checkins] bpo-41819: Fix compiler warning in init_dump_ascii_wstr() (GH-22332) Message-ID: https://github.com/python/cpython/commit/7aa534c9567201d896408bc7d5a718b529868540 commit: 7aa534c9567201d896408bc7d5a718b529868540 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-21T01:58:27-07:00 summary: bpo-41819: Fix compiler warning in init_dump_ascii_wstr() (GH-22332) Fix the compiler warning: format specifies type `wint_t` (aka `int`) but the argument has type `unsigned int` (cherry picked from commit c322948892438a387d752ec18d1eb512699a4d67) Co-authored-by: Samuel Marks <807580+SamuelMarks at users.noreply.github.com> files: M Python/initconfig.c diff --git a/Python/initconfig.c b/Python/initconfig.c index 9887079a86fd1..e882a1fba5396 100644 --- a/Python/initconfig.c +++ b/Python/initconfig.c @@ -2676,7 +2676,7 @@ init_dump_ascii_wstr(const wchar_t *str) if (ch == L'\'') { PySys_WriteStderr("\\'"); } else if (0x20 <= ch && ch < 0x7f) { - PySys_WriteStderr("%lc", ch); + PySys_WriteStderr("%c", ch); } else if (ch <= 0xff) { PySys_WriteStderr("\\x%02x", ch); From webhook-mailer at python.org Mon Sep 21 06:57:33 2020 From: webhook-mailer at python.org (Pablo Galindo) Date: Mon, 21 Sep 2020 10:57:33 -0000 Subject: [Python-checkins] Fix a compiler warning in pycore_pylifecycle.h (GH-22331) Message-ID: https://github.com/python/cpython/commit/724df8325d5fffc953280363c28ed84d5a7f3008 commit: 724df8325d5fffc953280363c28ed84d5a7f3008 branch: 3.9 author: Pablo Galindo committer: GitHub date: 2020-09-21T11:57:22+01:00 summary: Fix a compiler warning in pycore_pylifecycle.h (GH-22331) files: M Include/internal/pycore_pylifecycle.h diff --git a/Include/internal/pycore_pylifecycle.h b/Include/internal/pycore_pylifecycle.h index b0adc4562c447..b76bb2c7778ae 100644 --- a/Include/internal/pycore_pylifecycle.h +++ b/Include/internal/pycore_pylifecycle.h @@ -82,7 +82,7 @@ extern void _PyFaulthandler_Fini(void); extern void _PyHash_Fini(void); extern void _PyTraceMalloc_Fini(void); extern void _PyWarnings_Fini(PyInterpreterState *interp); -extern void _PyAST_Fini(); +extern void _PyAST_Fini(void); extern PyStatus _PyGILState_Init(PyThreadState *tstate); extern void _PyGILState_Fini(PyThreadState *tstate); From webhook-mailer at python.org Mon Sep 21 08:40:46 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Mon, 21 Sep 2020 12:40:46 -0000 Subject: [Python-checkins] bpo-1635741: Convert an _lsprof method to argument clinic (GH-22240) Message-ID: https://github.com/python/cpython/commit/1b328ea9a7d15de4a8c9d0eb8aee94f6c75c1b46 commit: 1b328ea9a7d15de4a8c9d0eb8aee94f6c75c1b46 branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-21T14:40:42+02:00 summary: bpo-1635741: Convert an _lsprof method to argument clinic (GH-22240) files: A Modules/clinic/_lsprof.c.h M Modules/_lsprof.c diff --git a/Modules/_lsprof.c b/Modules/_lsprof.c index 5e53d839640d9..a4ba7d5230033 100644 --- a/Modules/_lsprof.c +++ b/Modules/_lsprof.c @@ -50,8 +50,15 @@ typedef struct { #define POF_BUILTINS 0x004 #define POF_NOMEMORY 0x100 +/*[clinic input] +module _lsprof +class _lsprof.Profiler "ProfilerObject *" "&ProfilerType" +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=e349ac952152f336]*/ static PyTypeObject PyProfiler_Type; +#include "clinic/_lsprof.c.h" + #define PyProfiler_Check(op) PyObject_TypeCheck(op, &PyProfiler_Type) #define PyProfiler_CheckExact(op) Py_IS_TYPE(op, &PyProfiler_Type) @@ -556,49 +563,54 @@ static int statsForEntry(rotating_node_t *node, void *arg) return err; } -PyDoc_STRVAR(getstats_doc, "\ -getstats() -> list of profiler_entry objects\n\ -\n\ -Return all information collected by the profiler.\n\ -Each profiler_entry is a tuple-like object with the\n\ -following attributes:\n\ -\n\ - code code object\n\ - callcount how many times this was called\n\ - reccallcount how many times called recursively\n\ - totaltime total time in this entry\n\ - inlinetime inline time in this entry (not in subcalls)\n\ - calls details of the calls\n\ -\n\ -The calls attribute is either None or a list of\n\ -profiler_subentry objects:\n\ -\n\ - code called code object\n\ - callcount how many times this is called\n\ - reccallcount how many times this is called recursively\n\ - totaltime total time spent in this call\n\ - inlinetime inline time (not in further subcalls)\n\ -"); +/*[clinic input] +_lsprof.Profiler.getstats -static PyObject* -profiler_getstats(ProfilerObject *pObj, PyObject* noarg) +list of profiler_entry objects. + +getstats() -> list of profiler_entry objects + +Return all information collected by the profiler. +Each profiler_entry is a tuple-like object with the +following attributes: + + code code object + callcount how many times this was called + reccallcount how many times called recursively + totaltime total time in this entry + inlinetime inline time in this entry (not in subcalls) + calls details of the calls + +The calls attribute is either None or a list of +profiler_subentry objects: + + code called code object + callcount how many times this is called + reccallcount how many times this is called recursively + totaltime total time spent in this call + inlinetime inline time (not in further subcalls) +[clinic start generated code]*/ + +static PyObject * +_lsprof_Profiler_getstats_impl(ProfilerObject *self) +/*[clinic end generated code: output=9461b451e9ef0f24 input=ade04fa384ce450a]*/ { statscollector_t collect; - if (pending_exception(pObj)) { + if (pending_exception(self)) { return NULL; } - if (!pObj->externalTimer || pObj->externalTimerUnit == 0.0) { + if (!self->externalTimer || self->externalTimerUnit == 0.0) { _PyTime_t onesec = _PyTime_FromSeconds(1); collect.factor = (double)1 / onesec; } else { - collect.factor = pObj->externalTimerUnit; + collect.factor = self->externalTimerUnit; } collect.list = PyList_New(0); if (collect.list == NULL) return NULL; - if (RotatingTree_Enum(pObj->profilerEntries, statsForEntry, &collect) + if (RotatingTree_Enum(self->profilerEntries, statsForEntry, &collect) != 0) { Py_DECREF(collect.list); return NULL; @@ -750,8 +762,7 @@ profiler_init(ProfilerObject *pObj, PyObject *args, PyObject *kw) } static PyMethodDef profiler_methods[] = { - {"getstats", (PyCFunction)profiler_getstats, - METH_NOARGS, getstats_doc}, + _LSPROF_PROFILER_GETSTATS_METHODDEF {"enable", (PyCFunction)(void(*)(void))profiler_enable, METH_VARARGS | METH_KEYWORDS, enable_doc}, {"disable", (PyCFunction)profiler_disable, diff --git a/Modules/clinic/_lsprof.c.h b/Modules/clinic/_lsprof.c.h new file mode 100644 index 0000000000000..50762e3ff3596 --- /dev/null +++ b/Modules/clinic/_lsprof.c.h @@ -0,0 +1,44 @@ +/*[clinic input] +preserve +[clinic start generated code]*/ + +PyDoc_STRVAR(_lsprof_Profiler_getstats__doc__, +"getstats($self, /)\n" +"--\n" +"\n" +"list of profiler_entry objects.\n" +"\n" +"getstats() -> list of profiler_entry objects\n" +"\n" +"Return all information collected by the profiler.\n" +"Each profiler_entry is a tuple-like object with the\n" +"following attributes:\n" +"\n" +" code code object\n" +" callcount how many times this was called\n" +" reccallcount how many times called recursively\n" +" totaltime total time in this entry\n" +" inlinetime inline time in this entry (not in subcalls)\n" +" calls details of the calls\n" +"\n" +"The calls attribute is either None or a list of\n" +"profiler_subentry objects:\n" +"\n" +" code called code object\n" +" callcount how many times this is called\n" +" reccallcount how many times this is called recursively\n" +" totaltime total time spent in this call\n" +" inlinetime inline time (not in further subcalls)"); + +#define _LSPROF_PROFILER_GETSTATS_METHODDEF \ + {"getstats", (PyCFunction)_lsprof_Profiler_getstats, METH_NOARGS, _lsprof_Profiler_getstats__doc__}, + +static PyObject * +_lsprof_Profiler_getstats_impl(ProfilerObject *self); + +static PyObject * +_lsprof_Profiler_getstats(ProfilerObject *self, PyObject *Py_UNUSED(ignored)) +{ + return _lsprof_Profiler_getstats_impl(self); +} +/*[clinic end generated code: output=24c525812713e00f input=a9049054013a1b77]*/ From webhook-mailer at python.org Mon Sep 21 09:11:11 2020 From: webhook-mailer at python.org (Angelin BOOZ) Date: Mon, 21 Sep 2020 13:11:11 -0000 Subject: [Python-checkins] bpo-40084: Enum - dir() includes member attributes (GH-19219) Message-ID: https://github.com/python/cpython/commit/68526fe258da8c01196fd7cf48e8e5f1280bf8fd commit: 68526fe258da8c01196fd7cf48e8e5f1280bf8fd branch: master author: Angelin BOOZ <9497359+lem2clide at users.noreply.github.com> committer: GitHub date: 2020-09-21T06:11:06-07:00 summary: bpo-40084: Enum - dir() includes member attributes (GH-19219) files: A Misc/NEWS.d/next/Library/2020-03-29-21-32-00.bpo-40084.MCYwcv.rst M Lib/enum.py M Lib/test/test_enum.py M Lib/test/test_httplib.py M Misc/ACKS diff --git a/Lib/enum.py b/Lib/enum.py index 3c459ea4113d0..e8603a43420b0 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -644,7 +644,7 @@ def __dir__(self): for cls in self.__class__.mro() for m in cls.__dict__ if m[0] != '_' and m not in self._member_map_ - ] + ] + [m for m in self.__dict__ if m[0] != '_'] return (['__class__', '__doc__', '__module__'] + added_behavior) def __format__(self, format_spec): diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 59789fb7bcc5f..3f39073f5d564 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -216,6 +216,18 @@ class SubEnum(SuperEnum): set(['__class__', '__doc__', '__module__', 'name', 'value', 'invisible']), ) + def test_dir_on_sub_with_behavior_including_instance_dict_on_super(self): + # see issue40084 + class SuperEnum(IntEnum): + def __new__(cls, value, description=""): + obj = int.__new__(cls, value) + obj._value_ = value + obj.description = description + return obj + class SubEnum(SuperEnum): + sample = 5 + self.assertTrue({'description'} <= set(dir(SubEnum.sample))) + def test_enum_in_enum_out(self): Season = self.Season self.assertIs(Season(Season.WINTER), Season.WINTER) diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py index a3f268be97921..4abff60230b54 100644 --- a/Lib/test/test_httplib.py +++ b/Lib/test/test_httplib.py @@ -1,5 +1,5 @@ import errno -from http import client +from http import client, HTTPStatus import io import itertools import os @@ -519,6 +519,10 @@ def _parse_chunked(self, data): class BasicTest(TestCase): + def test_dir_with_added_behavior_on_status(self): + # see issue40084 + self.assertTrue({'description', 'name', 'phrase', 'value'} <= set(dir(HTTPStatus(404)))) + def test_status_lines(self): # Test HTTP status lines diff --git a/Misc/ACKS b/Misc/ACKS index d23797a2f5557..01ee1cb42d39d 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -191,6 +191,7 @@ Gawain Bolton Carl Friedrich Bolz-Tereick Forest Bond Gregory Bond +Angelin Booz M?d?ric Boquien Matias Bordese Jonas Borgstr?m diff --git a/Misc/NEWS.d/next/Library/2020-03-29-21-32-00.bpo-40084.MCYwcv.rst b/Misc/NEWS.d/next/Library/2020-03-29-21-32-00.bpo-40084.MCYwcv.rst new file mode 100644 index 0000000000000..65ff4ce36e82e --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-03-29-21-32-00.bpo-40084.MCYwcv.rst @@ -0,0 +1 @@ +Fix ``Enum.__dir__``: dir(Enum.member) now includes attributes as well as methods. From webhook-mailer at python.org Mon Sep 21 18:00:45 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 21 Sep 2020 22:00:45 -0000 Subject: [Python-checkins] bpo-41815: SQLite: segfault if backup called on closed database (GH-22322) Message-ID: https://github.com/python/cpython/commit/ca2d99d091a5b7768e92ee5ce7104aa6d8a3ed86 commit: ca2d99d091a5b7768e92ee5ce7104aa6d8a3ed86 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-21T15:00:34-07:00 summary: bpo-41815: SQLite: segfault if backup called on closed database (GH-22322) GH- [bpo-41815](): SQLite: fix segfault if backup called on closed database Attempting to backup a closed database will trigger segfault: ```python import sqlite3 target = sqlite3.connect(':memory:') source = sqlite3.connect(':memory:') source.close() source.backup(target) ``` (cherry picked from commit bfee9fad84531a471fd7864e88947320669f68e2) Co-authored-by: Peter McCormick files: A Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst M Lib/sqlite3/test/backup.py M Modules/_sqlite/connection.c diff --git a/Lib/sqlite3/test/backup.py b/Lib/sqlite3/test/backup.py index 903bacf490301..ad1da9721527c 100644 --- a/Lib/sqlite3/test/backup.py +++ b/Lib/sqlite3/test/backup.py @@ -36,6 +36,13 @@ def test_bad_target_closed_connection(self): with self.assertRaises(sqlite.ProgrammingError): self.cx.backup(bck) + def test_bad_source_closed_connection(self): + bck = sqlite.connect(':memory:') + source = sqlite.connect(":memory:") + source.close() + with self.assertRaises(sqlite.ProgrammingError): + source.backup(bck) + def test_bad_target_in_transaction(self): bck = sqlite.connect(':memory:') bck.execute('CREATE TABLE bar (key INTEGER)') diff --git a/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst b/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst new file mode 100644 index 0000000000000..3560db9bc5d35 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst @@ -0,0 +1,2 @@ +Fix SQLite3 segfault when backing up closed database. Patch contributed by +Peter David McCormick. diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index ebe073f644aa5..b6188a36733ef 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -1523,6 +1523,10 @@ pysqlite_connection_backup(pysqlite_Connection *self, PyObject *args, PyObject * sleep_ms = (int)ms; } + if (!pysqlite_check_thread(self) || !pysqlite_check_connection(self)) { + return NULL; + } + if (!pysqlite_check_connection((pysqlite_Connection *)target)) { return NULL; } From webhook-mailer at python.org Mon Sep 21 18:05:26 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 21 Sep 2020 22:05:26 -0000 Subject: [Python-checkins] bpo-41815: SQLite: segfault if backup called on closed database (GH-22322) Message-ID: https://github.com/python/cpython/commit/4ee30c42441c3e5066d5080a22da53675aeae716 commit: 4ee30c42441c3e5066d5080a22da53675aeae716 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-21T15:05:17-07:00 summary: bpo-41815: SQLite: segfault if backup called on closed database (GH-22322) GH- [bpo-41815](): SQLite: fix segfault if backup called on closed database Attempting to backup a closed database will trigger segfault: ```python import sqlite3 target = sqlite3.connect(':memory:') source = sqlite3.connect(':memory:') source.close() source.backup(target) ``` (cherry picked from commit bfee9fad84531a471fd7864e88947320669f68e2) Co-authored-by: Peter McCormick files: A Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst M Lib/sqlite3/test/backup.py M Modules/_sqlite/connection.c diff --git a/Lib/sqlite3/test/backup.py b/Lib/sqlite3/test/backup.py index 903bacf490301..ad1da9721527c 100644 --- a/Lib/sqlite3/test/backup.py +++ b/Lib/sqlite3/test/backup.py @@ -36,6 +36,13 @@ def test_bad_target_closed_connection(self): with self.assertRaises(sqlite.ProgrammingError): self.cx.backup(bck) + def test_bad_source_closed_connection(self): + bck = sqlite.connect(':memory:') + source = sqlite.connect(":memory:") + source.close() + with self.assertRaises(sqlite.ProgrammingError): + source.backup(bck) + def test_bad_target_in_transaction(self): bck = sqlite.connect(':memory:') bck.execute('CREATE TABLE bar (key INTEGER)') diff --git a/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst b/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst new file mode 100644 index 0000000000000..3560db9bc5d35 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst @@ -0,0 +1,2 @@ +Fix SQLite3 segfault when backing up closed database. Patch contributed by +Peter David McCormick. diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index 958be7d869794..b80037347f13b 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -1514,6 +1514,10 @@ pysqlite_connection_backup(pysqlite_Connection *self, PyObject *args, PyObject * sleep_ms = (int)ms; } + if (!pysqlite_check_thread(self) || !pysqlite_check_connection(self)) { + return NULL; + } + if (!pysqlite_check_connection((pysqlite_Connection *)target)) { return NULL; } From webhook-mailer at python.org Mon Sep 21 20:23:26 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Tue, 22 Sep 2020 00:23:26 -0000 Subject: [Python-checkins] bpo-41816: add `StrEnum` (GH-22337) Message-ID: https://github.com/python/cpython/commit/0063ff4e583505e69473caa978e476ea4c559b83 commit: 0063ff4e583505e69473caa978e476ea4c559b83 branch: master author: Ethan Furman committer: GitHub date: 2020-09-21T17:23:13-07:00 summary: bpo-41816: add `StrEnum` (GH-22337) `StrEnum` ensures that its members were already strings, or intended to be strings. files: A Misc/NEWS.d/next/Library/2020-09-19-12-22-08.bpo-41816.ynynXJ.rst M Doc/library/enum.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 2f84be229bc4d..843d961afc4f7 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -44,6 +44,11 @@ helper, :class:`auto`. Base class for creating enumerated constants that are also subclasses of :class:`int`. +.. class:: StrEnum + + Base class for creating enumerated constants that are also + subclasses of :class:`str`. + .. class:: IntFlag Base class for creating enumerated constants that can be combined using @@ -601,6 +606,25 @@ However, they still can't be compared to standard :class:`Enum` enumerations:: [0, 1] +StrEnum +^^^^^^^ + +The second variation of :class:`Enum` that is provided is also a subclass of +:class:`str`. Members of a :class:`StrEnum` can be compared to strings; +by extension, string enumerations of different types can also be compared +to each other. :class:`StrEnum` exists to help avoid the problem of getting +an incorrect member:: + + >>> class Directions(StrEnum): + ... NORTH = 'north', # notice the trailing comma + ... SOUTH = 'south' + +Before :class:`StrEnum`, ``Directions.NORTH`` would have been the :class:`tuple` +``('north',)``. + +.. versionadded:: 3.10 + + IntFlag ^^^^^^^ @@ -1132,6 +1156,20 @@ all-uppercase names for members):: .. versionchanged:: 3.5 +Creating members that are mixed with other data types +""""""""""""""""""""""""""""""""""""""""""""""""""""" + +When subclassing other data types, such as :class:`int` or :class:`str`, with +an :class:`Enum`, all values after the `=` are passed to that data type's +constructor. For example:: + + >>> class MyEnum(IntEnum): + ... example = '11', 16 # '11' will be interpreted as a hexadecimal + ... # number + >>> MyEnum.example + + + Boolean value of ``Enum`` classes and members """"""""""""""""""""""""""""""""""""""""""""" diff --git a/Lib/enum.py b/Lib/enum.py index e8603a43420b0..589b17fd69777 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -4,7 +4,7 @@ __all__ = [ 'EnumMeta', - 'Enum', 'IntEnum', 'Flag', 'IntFlag', + 'Enum', 'IntEnum', 'StrEnum', 'Flag', 'IntFlag', 'auto', 'unique', ] @@ -688,7 +688,35 @@ def value(self): class IntEnum(int, Enum): - """Enum where members are also (and must be) ints""" + """ + Enum where members are also (and must be) ints + """ + + +class StrEnum(str, Enum): + """ + Enum where members are also (and must be) strings + """ + + def __new__(cls, *values): + if len(values) > 3: + raise TypeError('too many arguments for str(): %r' % (values, )) + if len(values) == 1: + # it must be a string + if not isinstance(values[0], str): + raise TypeError('%r is not a string' % (values[0], )) + if len(values) > 1: + # check that encoding argument is a string + if not isinstance(values[1], str): + raise TypeError('encoding must be a string, not %r' % (values[1], )) + if len(values) > 2: + # check that errors argument is a string + if not isinstance(values[2], str): + raise TypeError('errors must be a string, not %r' % (values[2], )) + value = str(*values) + member = str.__new__(cls, value) + member._value_ = value + return member def _reduce_ex_by_name(self, proto): diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 3f39073f5d564..8e84d929429eb 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -5,7 +5,7 @@ import unittest import threading from collections import OrderedDict -from enum import Enum, IntEnum, EnumMeta, Flag, IntFlag, unique, auto +from enum import Enum, IntEnum, StrEnum, EnumMeta, Flag, IntFlag, unique, auto from io import StringIO from pickle import dumps, loads, PicklingError, HIGHEST_PROTOCOL from test import support @@ -48,14 +48,9 @@ class FlagStooges(Flag): FlagStooges = exc # for pickle test and subclass tests -try: - class StrEnum(str, Enum): - 'accepts only string values' - class Name(StrEnum): - BDFL = 'Guido van Rossum' - FLUFL = 'Barry Warsaw' -except Exception as exc: - Name = exc +class Name(StrEnum): + BDFL = 'Guido van Rossum' + FLUFL = 'Barry Warsaw' try: Question = Enum('Question', 'who what when where why', module=__name__) @@ -665,14 +660,13 @@ class phy(str, Enum): tau = 'Tau' self.assertTrue(phy.pi < phy.tau) - def test_strenum_inherited(self): - class StrEnum(str, Enum): - pass + def test_strenum_inherited_methods(self): class phy(StrEnum): pi = 'Pi' tau = 'Tau' self.assertTrue(phy.pi < phy.tau) - + self.assertEqual(phy.pi.upper(), 'PI') + self.assertEqual(phy.tau.count('a'), 1) def test_intenum(self): class WeekDay(IntEnum): @@ -2014,13 +2008,6 @@ class ReformedColor(StrMixin, IntEnum, SomeEnum, AnotherEnum): self.assertTrue(issubclass(ReformedColor, int)) def test_multiple_inherited_mixin(self): - class StrEnum(str, Enum): - def __new__(cls, *args, **kwargs): - for a in args: - if not isinstance(a, str): - raise TypeError("Enumeration '%s' (%s) is not" - " a string" % (a, type(a).__name__)) - return str.__new__(cls, *args, **kwargs) @unique class Decision1(StrEnum): REVERT = "REVERT" @@ -2043,6 +2030,33 @@ def test_empty_globals(self): local_ls = {} exec(code, global_ns, local_ls) + def test_strenum(self): + class GoodStrEnum(StrEnum): + one = '1' + two = '2' + three = b'3', 'ascii' + four = b'4', 'latin1', 'strict' + with self.assertRaisesRegex(TypeError, '1 is not a string'): + class FirstFailedStrEnum(StrEnum): + one = 1 + two = '2' + with self.assertRaisesRegex(TypeError, "2 is not a string"): + class SecondFailedStrEnum(StrEnum): + one = '1' + two = 2, + three = '3' + with self.assertRaisesRegex(TypeError, '2 is not a string'): + class ThirdFailedStrEnum(StrEnum): + one = '1' + two = 2 + with self.assertRaisesRegex(TypeError, 'encoding must be a string, not %r' % (sys.getdefaultencoding, )): + class ThirdFailedStrEnum(StrEnum): + one = '1' + two = b'2', sys.getdefaultencoding + with self.assertRaisesRegex(TypeError, 'errors must be a string, not 9'): + class ThirdFailedStrEnum(StrEnum): + one = '1' + two = b'2', 'ascii', 9 class TestOrder(unittest.TestCase): diff --git a/Misc/NEWS.d/next/Library/2020-09-19-12-22-08.bpo-41816.ynynXJ.rst b/Misc/NEWS.d/next/Library/2020-09-19-12-22-08.bpo-41816.ynynXJ.rst new file mode 100644 index 0000000000000..605c346f37a81 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-19-12-22-08.bpo-41816.ynynXJ.rst @@ -0,0 +1,2 @@ +StrEnum added: it ensures that all members are already strings or string +candidates From webhook-mailer at python.org Tue Sep 22 01:09:04 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Tue, 22 Sep 2020 05:09:04 -0000 Subject: [Python-checkins] bpo-41756: Refactor gen_send_ex(). (GH-22330) Message-ID: https://github.com/python/cpython/commit/6c33385e3a1dce31d7b9037eebfc584075795dba commit: 6c33385e3a1dce31d7b9037eebfc584075795dba branch: master author: Serhiy Storchaka committer: GitHub date: 2020-09-22T08:08:54+03:00 summary: bpo-41756: Refactor gen_send_ex(). (GH-22330) files: M Objects/genobject.c diff --git a/Objects/genobject.c b/Objects/genobject.c index 24aca988354c5..f0943ae847c54 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -136,13 +136,15 @@ gen_dealloc(PyGenObject *gen) PyObject_GC_Del(gen); } -static PyObject * -gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing, int *is_return_value) +static PySendResult +gen_send_ex2(PyGenObject *gen, PyObject *arg, PyObject **presult, + int exc, int closing) { PyThreadState *tstate = _PyThreadState_GET(); PyFrameObject *f = gen->gi_frame; PyObject *result; + *presult = NULL; if (f != NULL && _PyFrame_IsExecuting(f)) { const char *msg = "generator already executing"; if (PyCoro_CheckExact(gen)) { @@ -152,7 +154,7 @@ gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing, int *is_retur msg = "async generator already executing"; } PyErr_SetString(PyExc_ValueError, msg); - return NULL; + return PYGEN_ERROR; } if (f == NULL || _PyFrameHasCompleted(f)) { if (PyCoro_CheckExact(gen) && !closing) { @@ -165,19 +167,12 @@ gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing, int *is_retur } else if (arg && !exc) { /* `gen` is an exhausted generator: - only set exception if called from send(). */ - if (PyAsyncGen_CheckExact(gen)) { - PyErr_SetNone(PyExc_StopAsyncIteration); - } - else { - if (is_return_value != NULL) { - *is_return_value = 1; - Py_RETURN_NONE; - } - PyErr_SetNone(PyExc_StopIteration); - } + only return value if called from send(). */ + *presult = Py_None; + Py_INCREF(*presult); + return PYGEN_RETURN; } - return NULL; + return PYGEN_ERROR; } assert(_PyFrame_IsRunnable(f)); @@ -193,7 +188,7 @@ gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing, int *is_retur "just-started async generator"; } PyErr_SetString(PyExc_TypeError, msg); - return NULL; + return PYGEN_ERROR; } } else { /* Push arg onto the frame's value stack */ @@ -229,69 +224,77 @@ gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing, int *is_retur /* If the generator just returned (as opposed to yielding), signal * that the generator is exhausted. */ - if (result && _PyFrameHasCompleted(f)) { - if (result == Py_None) { - /* Delay exception instantiation if we can */ - if (PyAsyncGen_CheckExact(gen)) { - PyErr_SetNone(PyExc_StopAsyncIteration); - Py_CLEAR(result); - } - else if (arg) { - if (is_return_value != NULL) { - *is_return_value = 1; - } - else { - /* Set exception if not called by gen_iternext() */ - PyErr_SetNone(PyExc_StopIteration); - Py_CLEAR(result); - } - } - else { - Py_CLEAR(result); - } + if (result) { + if (!_PyFrameHasCompleted(f)) { + *presult = result; + return PYGEN_NEXT; } - else { - /* Async generators cannot return anything but None */ - assert(!PyAsyncGen_CheckExact(gen)); - if (is_return_value != NULL) { - *is_return_value = 1; - } - else { - _PyGen_SetStopIterationValue(result); - Py_CLEAR(result); - } + assert(result == Py_None || !PyAsyncGen_CheckExact(gen)); + if (result == Py_None && !PyAsyncGen_CheckExact(gen) && !arg) { + /* Return NULL if called by gen_iternext() */ + Py_CLEAR(result); } } - else if (!result && PyErr_ExceptionMatches(PyExc_StopIteration)) { - const char *msg = "generator raised StopIteration"; - if (PyCoro_CheckExact(gen)) { - msg = "coroutine raised StopIteration"; + else { + if (PyErr_ExceptionMatches(PyExc_StopIteration)) { + const char *msg = "generator raised StopIteration"; + if (PyCoro_CheckExact(gen)) { + msg = "coroutine raised StopIteration"; + } + else if (PyAsyncGen_CheckExact(gen)) { + msg = "async generator raised StopIteration"; + } + _PyErr_FormatFromCause(PyExc_RuntimeError, "%s", msg); } - else if (PyAsyncGen_CheckExact(gen)) { - msg = "async generator raised StopIteration"; + else if (PyAsyncGen_CheckExact(gen) && + PyErr_ExceptionMatches(PyExc_StopAsyncIteration)) + { + /* code in `gen` raised a StopAsyncIteration error: + raise a RuntimeError. + */ + const char *msg = "async generator raised StopAsyncIteration"; + _PyErr_FormatFromCause(PyExc_RuntimeError, "%s", msg); } - _PyErr_FormatFromCause(PyExc_RuntimeError, "%s", msg); - - } - else if (!result && PyAsyncGen_CheckExact(gen) && - PyErr_ExceptionMatches(PyExc_StopAsyncIteration)) - { - /* code in `gen` raised a StopAsyncIteration error: - raise a RuntimeError. - */ - const char *msg = "async generator raised StopAsyncIteration"; - _PyErr_FormatFromCause(PyExc_RuntimeError, "%s", msg); } - if ((is_return_value && *is_return_value) || !result || _PyFrameHasCompleted(f)) { - /* generator can't be rerun, so release the frame */ - /* first clean reference cycle through stored exception traceback */ - _PyErr_ClearExcState(&gen->gi_exc_state); - gen->gi_frame->f_gen = NULL; - gen->gi_frame = NULL; - Py_DECREF(f); - } + /* generator can't be rerun, so release the frame */ + /* first clean reference cycle through stored exception traceback */ + _PyErr_ClearExcState(&gen->gi_exc_state); + gen->gi_frame->f_gen = NULL; + gen->gi_frame = NULL; + Py_DECREF(f); + + *presult = result; + return result ? PYGEN_RETURN : PYGEN_ERROR; +} + +PySendResult +PyGen_Send(PyGenObject *gen, PyObject *arg, PyObject **result) +{ + assert(PyGen_CheckExact(gen) || PyCoro_CheckExact(gen)); + assert(result != NULL); + assert(arg != NULL); + + return gen_send_ex2(gen, arg, result, 0, 0); +} +static PyObject * +gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing) +{ + PyObject *result; + if (gen_send_ex2(gen, arg, &result, exc, closing) == PYGEN_RETURN) { + if (PyAsyncGen_CheckExact(gen)) { + assert(result == Py_None); + PyErr_SetNone(PyExc_StopAsyncIteration); + } + else if (result == Py_None) { + PyErr_SetNone(PyExc_StopIteration); + } + else { + _PyGen_SetStopIterationValue(result); + } + Py_CLEAR(result); + } return result; } @@ -299,22 +302,16 @@ PyDoc_STRVAR(send_doc, "send(arg) -> send 'arg' into generator,\n\ return next yielded value or raise StopIteration."); -PyObject * -_PyGen_Send(PyGenObject *gen, PyObject *arg) +static PyObject * +gen_send(PyGenObject *gen, PyObject *arg) { - return gen_send_ex(gen, arg, 0, 0, NULL); + return gen_send_ex(gen, arg, 0, 0); } -PySendResult -PyGen_Send(PyGenObject *gen, PyObject *arg, PyObject **result) +PyObject * +_PyGen_Send(PyGenObject *gen, PyObject *arg) { - assert(result != NULL); - - int is_return_value = 0; - if ((*result = gen_send_ex(gen, arg, 0, 0, &is_return_value)) == NULL) { - return PYGEN_ERROR; - } - return is_return_value ? PYGEN_RETURN : PYGEN_NEXT; + return gen_send(gen, arg); } PyDoc_STRVAR(close_doc, @@ -396,7 +393,7 @@ gen_close(PyGenObject *gen, PyObject *args) } if (err == 0) PyErr_SetNone(PyExc_GeneratorExit); - retval = gen_send_ex(gen, Py_None, 1, 1, NULL); + retval = gen_send_ex(gen, Py_None, 1, 1); if (retval) { const char *msg = "generator ignored GeneratorExit"; if (PyCoro_CheckExact(gen)) { @@ -444,7 +441,7 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, gen->gi_frame->f_state = state; Py_DECREF(yf); if (err < 0) - return gen_send_ex(gen, Py_None, 1, 0, NULL); + return gen_send_ex(gen, Py_None, 1, 0); goto throw_here; } if (PyGen_CheckExact(yf) || PyCoro_CheckExact(yf)) { @@ -496,10 +493,10 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, assert(gen->gi_frame->f_lasti >= 0); gen->gi_frame->f_lasti += sizeof(_Py_CODEUNIT); if (_PyGen_FetchStopIterationValue(&val) == 0) { - ret = gen_send_ex(gen, val, 0, 0, NULL); + ret = gen_send(gen, val); Py_DECREF(val); } else { - ret = gen_send_ex(gen, Py_None, 1, 0, NULL); + ret = gen_send_ex(gen, Py_None, 1, 0); } } return ret; @@ -553,7 +550,7 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, } PyErr_Restore(typ, val, tb); - return gen_send_ex(gen, Py_None, 1, 0, NULL); + return gen_send_ex(gen, Py_None, 1, 0); failed_throw: /* Didn't use our arguments, so restore their original refcounts */ @@ -582,7 +579,15 @@ gen_throw(PyGenObject *gen, PyObject *args) static PyObject * gen_iternext(PyGenObject *gen) { - return gen_send_ex(gen, NULL, 0, 0, NULL); + PyObject *result; + assert(PyGen_CheckExact(gen) || PyCoro_CheckExact(gen)); + if (gen_send_ex2(gen, NULL, &result, 0, 0) == PYGEN_RETURN) { + if (result != Py_None) { + _PyGen_SetStopIterationValue(result); + } + Py_CLEAR(result); + } + return result; } /* @@ -767,7 +772,7 @@ static PyMemberDef gen_memberlist[] = { }; static PyMethodDef gen_methods[] = { - {"send",(PyCFunction)_PyGen_Send, METH_O, send_doc}, + {"send",(PyCFunction)gen_send, METH_O, send_doc}, {"throw",(PyCFunction)gen_throw, METH_VARARGS, throw_doc}, {"close",(PyCFunction)gen_close, METH_NOARGS, close_doc}, {NULL, NULL} /* Sentinel */ @@ -1082,13 +1087,13 @@ coro_wrapper_dealloc(PyCoroWrapper *cw) static PyObject * coro_wrapper_iternext(PyCoroWrapper *cw) { - return gen_send_ex((PyGenObject *)cw->cw_coroutine, NULL, 0, 0, NULL); + return gen_iternext((PyGenObject *)cw->cw_coroutine); } static PyObject * coro_wrapper_send(PyCoroWrapper *cw, PyObject *arg) { - return gen_send_ex((PyGenObject *)cw->cw_coroutine, arg, 0, 0, NULL); + return gen_send((PyGenObject *)cw->cw_coroutine, arg); } static PyObject * @@ -1601,7 +1606,7 @@ async_gen_asend_send(PyAsyncGenASend *o, PyObject *arg) } o->ags_gen->ag_running_async = 1; - result = gen_send_ex((PyGenObject*)o->ags_gen, arg, 0, 0, NULL); + result = gen_send((PyGenObject*)o->ags_gen, arg); result = async_gen_unwrap_value(o->ags_gen, result); if (result == NULL) { @@ -1957,7 +1962,7 @@ async_gen_athrow_send(PyAsyncGenAThrow *o, PyObject *arg) assert(o->agt_state == AWAITABLE_STATE_ITER); - retval = gen_send_ex((PyGenObject *)gen, arg, 0, 0, NULL); + retval = gen_send((PyGenObject *)gen, arg); if (o->agt_args) { return async_gen_unwrap_value(o->agt_gen, retval); } else { From webhook-mailer at python.org Tue Sep 22 01:44:04 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Tue, 22 Sep 2020 05:44:04 -0000 Subject: [Python-checkins] bpo-40181: Remove '/' reminder in IDLE calltips. (GH-22350) Message-ID: https://github.com/python/cpython/commit/40a0625792e795cd41c4ba20475e3b770b53817a commit: 40a0625792e795cd41c4ba20475e3b770b53817a branch: master author: Terry Jan Reedy committer: GitHub date: 2020-09-22T01:43:55-04:00 summary: bpo-40181: Remove '/' reminder in IDLE calltips. (GH-22350) The marker was added to the language in 3.8 and 3.7 only gets security patches. files: A Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/calltip.py M Lib/idlelib/idle_test/test_calltip.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index fd762077b1b3c..f8ec6ab505297 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,9 @@ Released on 2020-10-05? ====================================== +bpo-40181: In calltips, stop reminding that '/' marks the end of +positional-only arguments. + bpo-41468: Improve IDLE run crash error message (which users should never see). diff --git a/Lib/idlelib/calltip.py b/Lib/idlelib/calltip.py index d4092c7847186..b02f87207d8db 100644 --- a/Lib/idlelib/calltip.py +++ b/Lib/idlelib/calltip.py @@ -118,7 +118,6 @@ def get_entity(expression): _first_param = re.compile(r'(?<=\()\w*\,?\s*') _default_callable_argspec = "See source or doc" _invalid_method = "invalid method signature" -_argument_positional = " # '/' marks preceding args as positional-only." def get_argspec(ob): '''Return a string describing the signature of a callable object, or ''. @@ -146,9 +145,6 @@ def get_argspec(ob): else: argspec = '' - if '/' in argspec and len(argspec) < _MAX_COLS - len(_argument_positional): - # Add explanation TODO remove after 3.7, before 3.9. - argspec += _argument_positional if isinstance(fob, type) and argspec == '()': # If fob has no argument, use default callable argspec. argspec = _default_callable_argspec diff --git a/Lib/idlelib/idle_test/test_calltip.py b/Lib/idlelib/idle_test/test_calltip.py index d386b5cd81321..4d53df17d8cc7 100644 --- a/Lib/idlelib/idle_test/test_calltip.py +++ b/Lib/idlelib/idle_test/test_calltip.py @@ -61,18 +61,16 @@ class SB: __call__ = None if List.__doc__ is not None: tiptest(List, - f'(iterable=(), /){calltip._argument_positional}' + f'(iterable=(), /)' f'\n{List.__doc__}') tiptest(list.__new__, '(*args, **kwargs)\n' 'Create and return a new object. ' 'See help(type) for accurate signature.') tiptest(list.__init__, - '(self, /, *args, **kwargs)' - + calltip._argument_positional + '\n' + + '(self, /, *args, **kwargs)\n' 'Initialize self. See help(type(self)) for accurate signature.') - append_doc = (calltip._argument_positional - + "\nAppend object to the end of the list.") + append_doc = "\nAppend object to the end of the list." tiptest(list.append, '(self, object, /)' + append_doc) tiptest(List.append, '(self, object, /)' + append_doc) tiptest([].append, '(object, /)' + append_doc) diff --git a/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst b/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst new file mode 100644 index 0000000000000..b6866e19c4d41 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst @@ -0,0 +1,2 @@ +In calltips, stop reminding that '/' marks the end of positional-only +arguments. From webhook-mailer at python.org Tue Sep 22 02:02:54 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 22 Sep 2020 06:02:54 -0000 Subject: [Python-checkins] bpo-40181: Remove '/' reminder in IDLE calltips. (GH-22350) Message-ID: https://github.com/python/cpython/commit/7bcbb536dde2308558c6a80c011042c6c3efc626 commit: 7bcbb536dde2308558c6a80c011042c6c3efc626 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-21T23:02:45-07:00 summary: bpo-40181: Remove '/' reminder in IDLE calltips. (GH-22350) The marker was added to the language in 3.8 and 3.7 only gets security patches. (cherry picked from commit 40a0625792e795cd41c4ba20475e3b770b53817a) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/calltip.py M Lib/idlelib/idle_test/test_calltip.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index f395f0c72f662..2cb1910c6c3f3 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,9 @@ Released on 2020-09-14? ====================================== +bpo-40181: In calltips, stop reminding that '/' marks the end of +positional-only arguments. + bpo-41468: Improve IDLE run crash error message (which users should never see). diff --git a/Lib/idlelib/calltip.py b/Lib/idlelib/calltip.py index d4092c7847186..b02f87207d8db 100644 --- a/Lib/idlelib/calltip.py +++ b/Lib/idlelib/calltip.py @@ -118,7 +118,6 @@ def get_entity(expression): _first_param = re.compile(r'(?<=\()\w*\,?\s*') _default_callable_argspec = "See source or doc" _invalid_method = "invalid method signature" -_argument_positional = " # '/' marks preceding args as positional-only." def get_argspec(ob): '''Return a string describing the signature of a callable object, or ''. @@ -146,9 +145,6 @@ def get_argspec(ob): else: argspec = '' - if '/' in argspec and len(argspec) < _MAX_COLS - len(_argument_positional): - # Add explanation TODO remove after 3.7, before 3.9. - argspec += _argument_positional if isinstance(fob, type) and argspec == '()': # If fob has no argument, use default callable argspec. argspec = _default_callable_argspec diff --git a/Lib/idlelib/idle_test/test_calltip.py b/Lib/idlelib/idle_test/test_calltip.py index d386b5cd81321..4d53df17d8cc7 100644 --- a/Lib/idlelib/idle_test/test_calltip.py +++ b/Lib/idlelib/idle_test/test_calltip.py @@ -61,18 +61,16 @@ class SB: __call__ = None if List.__doc__ is not None: tiptest(List, - f'(iterable=(), /){calltip._argument_positional}' + f'(iterable=(), /)' f'\n{List.__doc__}') tiptest(list.__new__, '(*args, **kwargs)\n' 'Create and return a new object. ' 'See help(type) for accurate signature.') tiptest(list.__init__, - '(self, /, *args, **kwargs)' - + calltip._argument_positional + '\n' + + '(self, /, *args, **kwargs)\n' 'Initialize self. See help(type(self)) for accurate signature.') - append_doc = (calltip._argument_positional - + "\nAppend object to the end of the list.") + append_doc = "\nAppend object to the end of the list." tiptest(list.append, '(self, object, /)' + append_doc) tiptest(List.append, '(self, object, /)' + append_doc) tiptest([].append, '(object, /)' + append_doc) diff --git a/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst b/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst new file mode 100644 index 0000000000000..b6866e19c4d41 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst @@ -0,0 +1,2 @@ +In calltips, stop reminding that '/' marks the end of positional-only +arguments. From webhook-mailer at python.org Tue Sep 22 02:05:16 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 22 Sep 2020 06:05:16 -0000 Subject: [Python-checkins] bpo-40181: Remove '/' reminder in IDLE calltips. (GH-22350) Message-ID: https://github.com/python/cpython/commit/6b731c48786bd30e9f8b86302dfb95a2640979a1 commit: 6b731c48786bd30e9f8b86302dfb95a2640979a1 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-21T23:05:07-07:00 summary: bpo-40181: Remove '/' reminder in IDLE calltips. (GH-22350) The marker was added to the language in 3.8 and 3.7 only gets security patches. (cherry picked from commit 40a0625792e795cd41c4ba20475e3b770b53817a) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/calltip.py M Lib/idlelib/idle_test/test_calltip.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index fd762077b1b3c..f8ec6ab505297 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,9 @@ Released on 2020-10-05? ====================================== +bpo-40181: In calltips, stop reminding that '/' marks the end of +positional-only arguments. + bpo-41468: Improve IDLE run crash error message (which users should never see). diff --git a/Lib/idlelib/calltip.py b/Lib/idlelib/calltip.py index d4092c7847186..b02f87207d8db 100644 --- a/Lib/idlelib/calltip.py +++ b/Lib/idlelib/calltip.py @@ -118,7 +118,6 @@ def get_entity(expression): _first_param = re.compile(r'(?<=\()\w*\,?\s*') _default_callable_argspec = "See source or doc" _invalid_method = "invalid method signature" -_argument_positional = " # '/' marks preceding args as positional-only." def get_argspec(ob): '''Return a string describing the signature of a callable object, or ''. @@ -146,9 +145,6 @@ def get_argspec(ob): else: argspec = '' - if '/' in argspec and len(argspec) < _MAX_COLS - len(_argument_positional): - # Add explanation TODO remove after 3.7, before 3.9. - argspec += _argument_positional if isinstance(fob, type) and argspec == '()': # If fob has no argument, use default callable argspec. argspec = _default_callable_argspec diff --git a/Lib/idlelib/idle_test/test_calltip.py b/Lib/idlelib/idle_test/test_calltip.py index d386b5cd81321..4d53df17d8cc7 100644 --- a/Lib/idlelib/idle_test/test_calltip.py +++ b/Lib/idlelib/idle_test/test_calltip.py @@ -61,18 +61,16 @@ class SB: __call__ = None if List.__doc__ is not None: tiptest(List, - f'(iterable=(), /){calltip._argument_positional}' + f'(iterable=(), /)' f'\n{List.__doc__}') tiptest(list.__new__, '(*args, **kwargs)\n' 'Create and return a new object. ' 'See help(type) for accurate signature.') tiptest(list.__init__, - '(self, /, *args, **kwargs)' - + calltip._argument_positional + '\n' + + '(self, /, *args, **kwargs)\n' 'Initialize self. See help(type(self)) for accurate signature.') - append_doc = (calltip._argument_positional - + "\nAppend object to the end of the list.") + append_doc = "\nAppend object to the end of the list." tiptest(list.append, '(self, object, /)' + append_doc) tiptest(List.append, '(self, object, /)' + append_doc) tiptest([].append, '(object, /)' + append_doc) diff --git a/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst b/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst new file mode 100644 index 0000000000000..b6866e19c4d41 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst @@ -0,0 +1,2 @@ +In calltips, stop reminding that '/' marks the end of positional-only +arguments. From webhook-mailer at python.org Tue Sep 22 03:05:34 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Tue, 22 Sep 2020 07:05:34 -0000 Subject: [Python-checkins] Enum: add extended AutoNumber example (GH-22349) Message-ID: https://github.com/python/cpython/commit/62e40d8450b9c78346ec3617de7fe3f0ad381510 commit: 62e40d8450b9c78346ec3617de7fe3f0ad381510 branch: master author: Ethan Furman committer: GitHub date: 2020-09-22T00:05:27-07:00 summary: Enum: add extended AutoNumber example (GH-22349) files: M Doc/library/enum.rst M Misc/ACKS diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 843d961afc4f7..3e9b1f9db3550 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -925,6 +925,32 @@ Using an auto-numbering :meth:`__new__` would look like:: >>> Color.GREEN.value 2 +To make a more general purpose ``AutoNumber``, add ``*args`` to the signature:: + + >>> class AutoNumber(NoValue): + ... def __new__(cls, *args): # this is the only change from above + ... value = len(cls.__members__) + 1 + ... obj = object.__new__(cls) + ... obj._value_ = value + ... return obj + ... + +Then when you inherit from ``AutoNumber`` you can write your own ``__init__`` +to handle any extra arguments:: + + >>> class Swatch(AutoNumber): + ... def __init__(self, pantone='unknown'): + ... self.pantone = pantone + ... AUBURN = '3497' + ... SEA_GREEN = '1246' + ... BLEACHED_CORAL = () # New color, no Pantone code yet! + ... + >>> Swatch.SEA_GREEN + + >>> Swatch.SEA_GREEN.pantone + '1246' + >>> Swatch.BLEACHED_CORAL.pantone + 'unknown' .. note:: diff --git a/Misc/ACKS b/Misc/ACKS index 01ee1cb42d39d..e4bd3da6b6c40 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1723,6 +1723,7 @@ F?vry Thibault Lowe Thiderman Nicolas M. Thi?ry James Thomas +Reuben Thomas Robin Thomas Brian Thorne Christopher Thorne From webhook-mailer at python.org Tue Sep 22 06:42:33 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Tue, 22 Sep 2020 10:42:33 -0000 Subject: [Python-checkins] Py_IS_TYPE() macro uses Py_TYPE() (GH-22341) Message-ID: https://github.com/python/cpython/commit/c5cb077ab3c88394b7ac8ed4e746bd31b53e39f1 commit: c5cb077ab3c88394b7ac8ed4e746bd31b53e39f1 branch: master author: Victor Stinner committer: GitHub date: 2020-09-22T12:42:28+02:00 summary: Py_IS_TYPE() macro uses Py_TYPE() (GH-22341) files: M Include/object.h diff --git a/Include/object.h b/Include/object.h index 10f1d6a3dff2d..6ee4ee7848551 100644 --- a/Include/object.h +++ b/Include/object.h @@ -141,7 +141,7 @@ static inline PyTypeObject* _Py_TYPE(const PyObject *ob) { static inline int _Py_IS_TYPE(const PyObject *ob, const PyTypeObject *type) { - return ob->ob_type == type; + return Py_TYPE(ob) == type; } #define Py_IS_TYPE(ob, type) _Py_IS_TYPE(_PyObject_CAST_CONST(ob), type) From webhook-mailer at python.org Tue Sep 22 09:16:55 2020 From: webhook-mailer at python.org (Serhiy Storchaka) Date: Tue, 22 Sep 2020 13:16:55 -0000 Subject: [Python-checkins] bpo-40670: More reliable validation of statements in timeit.Timer. (GH-22358) Message-ID: https://github.com/python/cpython/commit/557b9a52edc4445cec293f2cc2c268c4f564adcf commit: 557b9a52edc4445cec293f2cc2c268c4f564adcf branch: master author: Serhiy Storchaka committer: GitHub date: 2020-09-22T16:16:46+03:00 summary: bpo-40670: More reliable validation of statements in timeit.Timer. (GH-22358) It now accepts "empty" statements (only whitespaces and comments) and rejects misindentent statements. files: A Misc/NEWS.d/next/Library/2020-09-22-14-55-34.bpo-40670.R5sm68.rst M Lib/test/test_timeit.py M Lib/timeit.py diff --git a/Lib/test/test_timeit.py b/Lib/test/test_timeit.py index e02d4a71a9ba7..72a104fc1a679 100644 --- a/Lib/test/test_timeit.py +++ b/Lib/test/test_timeit.py @@ -77,6 +77,9 @@ def test_timer_invalid_stmt(self): self.assertRaises(SyntaxError, timeit.Timer, stmt='break') self.assertRaises(SyntaxError, timeit.Timer, stmt='continue') self.assertRaises(SyntaxError, timeit.Timer, stmt='from timeit import *') + self.assertRaises(SyntaxError, timeit.Timer, stmt=' pass') + self.assertRaises(SyntaxError, timeit.Timer, + setup='while False:\n pass', stmt=' break') def test_timer_invalid_setup(self): self.assertRaises(ValueError, timeit.Timer, setup=None) @@ -86,6 +89,12 @@ def test_timer_invalid_setup(self): self.assertRaises(SyntaxError, timeit.Timer, setup='break') self.assertRaises(SyntaxError, timeit.Timer, setup='continue') self.assertRaises(SyntaxError, timeit.Timer, setup='from timeit import *') + self.assertRaises(SyntaxError, timeit.Timer, setup=' pass') + + def test_timer_empty_stmt(self): + timeit.Timer(stmt='') + timeit.Timer(stmt=' \n\t\f') + timeit.Timer(stmt='# comment') fake_setup = "import timeit\ntimeit._fake_timer.setup()" fake_stmt = "import timeit\ntimeit._fake_timer.inc()" diff --git a/Lib/timeit.py b/Lib/timeit.py index 6c3ec01067f2d..9dfd454936e6b 100755 --- a/Lib/timeit.py +++ b/Lib/timeit.py @@ -72,6 +72,7 @@ def inner(_it, _timer{init}): _t0 = _timer() for _i in _it: {stmt} + pass _t1 = _timer() return _t1 - _t0 """ diff --git a/Misc/NEWS.d/next/Library/2020-09-22-14-55-34.bpo-40670.R5sm68.rst b/Misc/NEWS.d/next/Library/2020-09-22-14-55-34.bpo-40670.R5sm68.rst new file mode 100644 index 0000000000000..0436194d736ab --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-22-14-55-34.bpo-40670.R5sm68.rst @@ -0,0 +1,3 @@ +More reliable validation of statements in :class:`timeit.Timer`. It now +accepts "empty" statements (only whitespaces and comments) and rejects +misindentent statements. From webhook-mailer at python.org Tue Sep 22 11:01:29 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Tue, 22 Sep 2020 15:01:29 -0000 Subject: [Python-checkins] bpo-41817: use new StrEnum to ensure all members are strings (GH-22348) Message-ID: https://github.com/python/cpython/commit/ea0711a9f9f207d6d4ca037d90de6ec60db131b0 commit: ea0711a9f9f207d6d4ca037d90de6ec60db131b0 branch: master author: Ethan Furman committer: GitHub date: 2020-09-22T08:01:17-07:00 summary: bpo-41817: use new StrEnum to ensure all members are strings (GH-22348) * use new StrEnum to ensure all members are strings files: A Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst M Lib/tkinter/__init__.py diff --git a/Lib/tkinter/__init__.py b/Lib/tkinter/__init__.py index 1067ab6a8b8a1..3919397d3cead 100644 --- a/Lib/tkinter/__init__.py +++ b/Lib/tkinter/__init__.py @@ -144,12 +144,12 @@ def _splitdict(tk, v, cut_minus=True, conv=None): return dict -class EventType(str, enum.Enum): +class EventType(enum.StrEnum): KeyPress = '2' - Key = KeyPress, + Key = KeyPress KeyRelease = '3' ButtonPress = '4' - Button = ButtonPress, + Button = ButtonPress ButtonRelease = '5' Motion = '6' Enter = '7' @@ -180,10 +180,10 @@ class EventType(str, enum.Enum): Colormap = '32' ClientMessage = '33' # undocumented Mapping = '34' # undocumented - VirtualEvent = '35', # undocumented - Activate = '36', - Deactivate = '37', - MouseWheel = '38', + VirtualEvent = '35' # undocumented + Activate = '36' + Deactivate = '37' + MouseWheel = '38' def __str__(self): return self.name diff --git a/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst b/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst new file mode 100644 index 0000000000000..6a634bb613260 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst @@ -0,0 +1 @@ +fix `tkinter.EventType` Enum so all members are strings, and none are tuples From webhook-mailer at python.org Tue Sep 22 11:53:12 2020 From: webhook-mailer at python.org (Thomas Grainger) Date: Tue, 22 Sep 2020 15:53:12 -0000 Subject: [Python-checkins] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (#21956) Message-ID: https://github.com/python/cpython/commit/a68a2ad19c891faa891904b3da537911cc77df21 commit: a68a2ad19c891faa891904b3da537911cc77df21 branch: master author: Thomas Grainger committer: GitHub date: 2020-09-22T08:53:03-07:00 summary: bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (#21956) Closes bpo issue 41602 files: A Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst M Lib/test/test_runpy.py M Modules/main.c diff --git a/Lib/test/test_runpy.py b/Lib/test/test_runpy.py index f8274a981cb1c..2954dfedc7e42 100644 --- a/Lib/test/test_runpy.py +++ b/Lib/test/test_runpy.py @@ -1,15 +1,18 @@ # Test the runpy module -import unittest -import os +import contextlib +import importlib.machinery, importlib.util import os.path -import sys +import pathlib +import py_compile import re +import signal +import subprocess +import sys import tempfile -import importlib, importlib.machinery, importlib.util -import py_compile +import textwrap +import unittest import warnings -import pathlib -from test.support import verbose, no_tracing +from test.support import no_tracing, verbose from test.support.import_helper import forget, make_legacy_pyc, unload from test.support.os_helper import create_empty_file, temp_dir from test.support.script_helper import make_script, make_zip_script @@ -752,5 +755,82 @@ def test_encoding(self): self.assertEqual(result['s'], "non-ASCII: h\xe9") +class TestExit(unittest.TestCase): + STATUS_CONTROL_C_EXIT = 0xC000013A + EXPECTED_CODE = ( + STATUS_CONTROL_C_EXIT + if sys.platform == "win32" + else -signal.SIGINT + ) + @staticmethod + @contextlib.contextmanager + def tmp_path(*args, **kwargs): + with temp_dir() as tmp_fn: + yield pathlib.Path(tmp_fn) + + + def run(self, *args, **kwargs): + with self.tmp_path() as tmp: + self.ham = ham = tmp / "ham.py" + ham.write_text( + textwrap.dedent( + """\ + raise KeyboardInterrupt + """ + ) + ) + super().run(*args, **kwargs) + + def assertSigInt(self, *args, **kwargs): + proc = subprocess.run(*args, **kwargs, text=True, stderr=subprocess.PIPE) + self.assertTrue(proc.stderr.endswith("\nKeyboardInterrupt\n")) + self.assertEqual(proc.returncode, self.EXPECTED_CODE) + + def test_pymain_run_file(self): + self.assertSigInt([sys.executable, self.ham]) + + def test_pymain_run_file_runpy_run_module(self): + tmp = self.ham.parent + run_module = tmp / "run_module.py" + run_module.write_text( + textwrap.dedent( + """\ + import runpy + runpy.run_module("ham") + """ + ) + ) + self.assertSigInt([sys.executable, run_module], cwd=tmp) + + def test_pymain_run_file_runpy_run_module_as_main(self): + tmp = self.ham.parent + run_module_as_main = tmp / "run_module_as_main.py" + run_module_as_main.write_text( + textwrap.dedent( + """\ + import runpy + runpy._run_module_as_main("ham") + """ + ) + ) + self.assertSigInt([sys.executable, run_module_as_main], cwd=tmp) + + def test_pymain_run_command_run_module(self): + self.assertSigInt( + [sys.executable, "-c", "import runpy; runpy.run_module('ham')"], + cwd=self.ham.parent, + ) + + def test_pymain_run_command(self): + self.assertSigInt([sys.executable, "-c", "import ham"], cwd=self.ham.parent) + + def test_pymain_run_stdin(self): + self.assertSigInt([sys.executable], input="import ham", cwd=self.ham.parent) + + def test_pymain_run_module(self): + ham = self.ham + self.assertSigInt([sys.executable, "-m", ham.stem], cwd=ham.parent) + + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst b/Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst new file mode 100644 index 0000000000000..fa3d2f1aa374e --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst @@ -0,0 +1 @@ +Add tests for SIGINT handling in the runpy module. diff --git a/Modules/main.c b/Modules/main.c index 4a76f4461bf61..2cc891f61aadd 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -287,7 +287,11 @@ pymain_run_module(const wchar_t *modname, int set_argv0) Py_DECREF(module); return pymain_exit_err_print(); } + _Py_UnhandledKeyboardInterrupt = 0; result = PyObject_Call(runmodule, runargs, NULL); + if (!result && PyErr_Occurred() == PyExc_KeyboardInterrupt) { + _Py_UnhandledKeyboardInterrupt = 1; + } Py_DECREF(runpy); Py_DECREF(runmodule); Py_DECREF(module); From webhook-mailer at python.org Tue Sep 22 11:55:38 2020 From: webhook-mailer at python.org (Bas van Beek) Date: Tue, 22 Sep 2020 15:55:38 -0000 Subject: [Python-checkins] bpo-41810: Reintroduce `types.EllipsisType`, `.NoneType` & `.NotImplementedType` (GH-22336) Message-ID: https://github.com/python/cpython/commit/0d0e9fe2ffc1683758a1985ef6dedeef5ecafdbc commit: 0d0e9fe2ffc1683758a1985ef6dedeef5ecafdbc branch: master author: Bas van Beek <43369155+BvB93 at users.noreply.github.com> committer: GitHub date: 2020-09-22T08:55:34-07:00 summary: bpo-41810: Reintroduce `types.EllipsisType`, `.NoneType` & `.NotImplementedType` (GH-22336) closes issue 41810 files: A Misc/NEWS.d/next/Library/2020-09-20-15-14-05.bpo-41810.7l8lyV.rst M Doc/library/constants.rst M Doc/library/types.rst M Doc/whatsnew/3.10.rst M Lib/test/test_types.py M Lib/types.py M Misc/ACKS diff --git a/Doc/library/constants.rst b/Doc/library/constants.rst index f17e1a3787516..38dd552a0363a 100644 --- a/Doc/library/constants.rst +++ b/Doc/library/constants.rst @@ -19,19 +19,21 @@ A small number of constants live in the built-in namespace. They are: .. data:: None - The sole value of the type ``NoneType``. ``None`` is frequently used to - represent the absence of a value, as when default arguments are not passed to a - function. Assignments to ``None`` are illegal and raise a :exc:`SyntaxError`. + An object frequently used to represent the absence of a value, as when + default arguments are not passed to a function. Assignments to ``None`` + are illegal and raise a :exc:`SyntaxError`. + ``None`` is the sole instance of the :data:`NoneType` type. .. data:: NotImplemented - Special value which should be returned by the binary special methods + A special value which should be returned by the binary special methods (e.g. :meth:`__eq__`, :meth:`__lt__`, :meth:`__add__`, :meth:`__rsub__`, etc.) to indicate that the operation is not implemented with respect to the other type; may be returned by the in-place binary special methods (e.g. :meth:`__imul__`, :meth:`__iand__`, etc.) for the same purpose. It should not be evaluated in a boolean context. + ``NotImplemented`` is the sole instance of the :data:`types.NotImplementedType` type. .. note:: @@ -59,8 +61,9 @@ A small number of constants live in the built-in namespace. They are: .. index:: single: ...; ellipsis literal .. data:: Ellipsis - The same as the ellipsis literal "``...``". Special value used mostly in conjunction + The same as the ellipsis literal "``...``". Special value used mostly in conjunction with extended slicing syntax for user-defined container data types. + ``Ellipsis`` is the sole instance of the :data:`types.EllipsisType` type. .. data:: __debug__ diff --git a/Doc/library/types.rst b/Doc/library/types.rst index 79acdf4499afd..25fa750f2ccac 100644 --- a/Doc/library/types.rst +++ b/Doc/library/types.rst @@ -103,6 +103,13 @@ If you instantiate any of these types, note that signatures may vary between Pyt Standard names are defined for the following types: +.. data:: NoneType + + The type of :data:`None`. + + .. versionadded:: 3.10 + + .. data:: FunctionType LambdaType @@ -186,6 +193,13 @@ Standard names are defined for the following types: .. versionadded:: 3.7 +.. data:: NotImplementedType + + The type of :data:`NotImplemented`. + + .. versionadded:: 3.10 + + .. data:: MethodDescriptorType The type of methods of some built-in data types such as :meth:`str.join`. @@ -236,6 +250,13 @@ Standard names are defined for the following types: Defaults to ``None``. Previously the attribute was optional. +.. data:: EllipsisType + + The type of :data:`Ellipsis`. + + .. versionadded:: 3.10 + + .. class:: TracebackType(tb_next, tb_frame, tb_lasti, tb_lineno) The type of traceback objects such as found in ``sys.exc_info()[2]``. diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index ce888fec1d8c9..f88281a934ca1 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -145,6 +145,14 @@ Add :data:`sys.orig_argv` attribute: the list of the original command line arguments passed to the Python executable. (Contributed by Victor Stinner in :issue:`23427`.) +types +----- + +Reintroduced the :data:`types.EllipsisType`, :data:`types.NoneType` +and :data:`types.NotImplementedType` classes, providing a new set +of types readily interpretable by type checkers. +(Contributed by Bas van Beek in :issue:`41810`.) + unittest -------- diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py index f499fb9c8c51a..52a59d54f044d 100644 --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -713,6 +713,16 @@ def test_or_type_repr(self): assert repr(int | None) == "int | None" assert repr(int | typing.GenericAlias(list, int)) == "int | list[int]" + def test_ellipsis_type(self): + self.assertIsInstance(Ellipsis, types.EllipsisType) + + def test_notimplemented_type(self): + self.assertIsInstance(NotImplemented, types.NotImplementedType) + + def test_none_type(self): + self.assertIsInstance(None, types.NoneType) + + class MappingProxyTests(unittest.TestCase): mappingproxy = types.MappingProxyType diff --git a/Lib/types.py b/Lib/types.py index 9642e7212caac..532f4806fc022 100644 --- a/Lib/types.py +++ b/Lib/types.py @@ -296,5 +296,8 @@ def wrapped(*args, **kwargs): GenericAlias = type(list[int]) Union = type(int | str) +EllipsisType = type(Ellipsis) +NoneType = type(None) +NotImplementedType = type(NotImplemented) __all__ = [n for n in globals() if n[:1] != '_'] diff --git a/Misc/ACKS b/Misc/ACKS index e4bd3da6b6c40..7b743464c1c1c 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -134,6 +134,7 @@ Robin Becker Torsten Becker Bill Bedford Micha? Bednarski +Bas van Beek Ian Beer Stefan Behnel Reimer Behrends diff --git a/Misc/NEWS.d/next/Library/2020-09-20-15-14-05.bpo-41810.7l8lyV.rst b/Misc/NEWS.d/next/Library/2020-09-20-15-14-05.bpo-41810.7l8lyV.rst new file mode 100644 index 0000000000000..515aea9e36ce9 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-20-15-14-05.bpo-41810.7l8lyV.rst @@ -0,0 +1,3 @@ +:data:`types.EllipsisType`, :data:`types.NotImplementedType` and +:data:`types.NoneType` have been reintroduced, providing a new set +of types readily interpretable by static type checkers. From webhook-mailer at python.org Tue Sep 22 13:22:16 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Tue, 22 Sep 2020 17:22:16 -0000 Subject: [Python-checkins] bpo-35764: Rewrite the IDLE Calltips doc section (GH-22363) Message-ID: https://github.com/python/cpython/commit/947adcaa13080790167757664912c3a6c2d4c201 commit: 947adcaa13080790167757664912c3a6c2d4c201 branch: master author: Terry Jan Reedy committer: GitHub date: 2020-09-22T13:21:58-04:00 summary: bpo-35764: Rewrite the IDLE Calltips doc section (GH-22363) files: A Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst M Doc/library/idle.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/help.html diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 43096b014fed3..a59a5d3a46570 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -527,30 +527,33 @@ by typing '_' after '.', either before or after the box is opened. Calltips ^^^^^^^^ -A calltip is shown when one types :kbd:`(` after the name of an *accessible* -function. A name expression may include dots and subscripts. A calltip -remains until it is clicked, the cursor is moved out of the argument area, -or :kbd:`)` is typed. When the cursor is in the argument part of a definition, -the menu or shortcut display a calltip. - -A calltip consists of the function signature and the first line of the -docstring. For builtins without an accessible signature, the calltip -consists of all lines up the fifth line or the first blank line. These -details may change. - -The set of *accessible* functions depends on what modules have been imported -into the user process, including those imported by Idle itself, -and what definitions have been run, all since the last restart. +A calltip is shown automatically when one types :kbd:`(` after the name +of an *accessible* function. A function name expression may include +dots and subscripts. A calltip remains until it is clicked, the cursor +is moved out of the argument area, or :kbd:`)` is typed. Whenever the +cursor is in the argument part of a definition, select Edit and "Show +Call Tip" on the menu or enter its shortcut to display a calltip. + +The calltip consists of the function's signature and docstring up to +the latter's first blank line or the fifth non-blank line. (Some builtin +functions lack an accessible signature.) A '/' or '*' in the signature +indicates that the preceding or following arguments are passed by +position or name (keyword) only. Details are subject to change. + +In Shell, the accessible functions depends on what modules have been +imported into the user process, including those imported by Idle itself, +and which definitions have been run, all since the last restart. For example, restart the Shell and enter ``itertools.count(``. A calltip -appears because Idle imports itertools into the user process for its own use. -(This could change.) Enter ``turtle.write(`` and nothing appears. Idle does -not import turtle. The menu or shortcut do nothing either. Enter -``import turtle`` and then ``turtle.write(`` will work. - -In an editor, import statements have no effect until one runs the file. One -might want to run a file after writing the import statements at the top, -or immediately run an existing file before editing. +appears because Idle imports itertools into the user process for its own +use. (This could change.) Enter ``turtle.write(`` and nothing appears. +Idle does not itself import turtle. The menu entry and shortcut also do +nothing. Enter ``import turtle``. Thereafter, ``turtle.write(`` +will display a calltip. + +In an editor, import statements have no effect until one runs the file. +One might want to run a file after writing import statements, after +adding function definitions, or after opening an existing file. .. _code-context: diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index f8ec6ab505297..7eea0a47aa6e8 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,8 @@ Released on 2020-10-05? ====================================== +bpo-35764: Rewrite the Calltips doc section. + bpo-40181: In calltips, stop reminding that '/' marks the end of positional-only arguments. diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html index b2853cffe0c26..0edd3917e1ffa 100644 --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -509,26 +509,29 @@

Automatic indentation

Calltips?

-

A calltip is shown when one types ( after the name of an accessible -function. A name expression may include dots and subscripts. A calltip -remains until it is clicked, the cursor is moved out of the argument area, -or ) is typed. When the cursor is in the argument part of a definition, -the menu or shortcut display a calltip.

-

A calltip consists of the function signature and the first line of the -docstring. For builtins without an accessible signature, the calltip -consists of all lines up the fifth line or the first blank line. These -details may change.

-

The set of accessible functions depends on what modules have been imported -into the user process, including those imported by Idle itself, -and what definitions have been run, all since the last restart.

+

A calltip is shown automatically when one types ( after the name +of an accessible function. A function name expression may include +dots and subscripts. A calltip remains until it is clicked, the cursor +is moved out of the argument area, or ) is typed. Whenever the +cursor is in the argument part of a definition, select Edit and ?Show +Call Tip? on the menu or enter its shortcut to display a calltip.

+

The calltip consists of the function?s signature and docstring up to +the latter?s first blank line or the fifth non-blank line. (Some builtin +functions lack an accessible signature.) A ?/? or ?*? in the signature +indicates that the preceding or following arguments are passed by +position or name (keyword) only. Details are subject to change.

+

In Shell, the accessible functions depends on what modules have been +imported into the user process, including those imported by Idle itself, +and which definitions have been run, all since the last restart.

For example, restart the Shell and enter itertools.count(. A calltip -appears because Idle imports itertools into the user process for its own use. -(This could change.) Enter turtle.write( and nothing appears. Idle does -not import turtle. The menu or shortcut do nothing either. Enter -import turtle and then turtle.write( will work.

-

In an editor, import statements have no effect until one runs the file. One -might want to run a file after writing the import statements at the top, -or immediately run an existing file before editing.

+appears because Idle imports itertools into the user process for its own +use. (This could change.) Enter turtle.write( and nothing appears. +Idle does not itself import turtle. The menu entry and shortcut also do +nothing. Enter import turtle. Thereafter, turtle.write( +will display a calltip.

+

In an editor, import statements have no effect until one runs the file. +One might want to run a file after writing import statements, after +adding function definitions, or after opening an existing file.

Code Context?

@@ -975,7 +978,7 @@

Navigation



- Last updated on Sep 09, 2020. + Last updated on Sep 22, 2020. Found a bug?
diff --git a/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst b/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst new file mode 100644 index 0000000000000..eb62d3699d5fe --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst @@ -0,0 +1 @@ +Rewrite the Calltips doc section. From webhook-mailer at python.org Tue Sep 22 14:30:56 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 22 Sep 2020 18:30:56 -0000 Subject: [Python-checkins] fix `EventType` bases Message-ID: https://github.com/python/cpython/commit/6d372ef19468b6f5239479b595dfb587c642b594 commit: 6d372ef19468b6f5239479b595dfb587c642b594 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-22T11:30:47-07:00 summary: fix `EventType` bases `StrEnum` does not exist in 3.8, so use original `(str, enum.Enum)` bases. files: A Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst M Lib/tkinter/__init__.py diff --git a/Lib/tkinter/__init__.py b/Lib/tkinter/__init__.py index 9d3bf4d49298f..479eb016da7f0 100644 --- a/Lib/tkinter/__init__.py +++ b/Lib/tkinter/__init__.py @@ -146,10 +146,10 @@ def _splitdict(tk, v, cut_minus=True, conv=None): class EventType(str, enum.Enum): KeyPress = '2' - Key = KeyPress, + Key = KeyPress KeyRelease = '3' ButtonPress = '4' - Button = ButtonPress, + Button = ButtonPress ButtonRelease = '5' Motion = '6' Enter = '7' @@ -180,10 +180,10 @@ class EventType(str, enum.Enum): Colormap = '32' ClientMessage = '33' # undocumented Mapping = '34' # undocumented - VirtualEvent = '35', # undocumented - Activate = '36', - Deactivate = '37', - MouseWheel = '38', + VirtualEvent = '35' # undocumented + Activate = '36' + Deactivate = '37' + MouseWheel = '38' def __str__(self): return self.name diff --git a/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst b/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst new file mode 100644 index 0000000000000..6a634bb613260 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst @@ -0,0 +1 @@ +fix `tkinter.EventType` Enum so all members are strings, and none are tuples From webhook-mailer at python.org Tue Sep 22 14:31:26 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 22 Sep 2020 18:31:26 -0000 Subject: [Python-checkins] fix `EventType` bases Message-ID: https://github.com/python/cpython/commit/b38ad227051d03e731a80db3942a729485262bd4 commit: b38ad227051d03e731a80db3942a729485262bd4 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-22T11:31:22-07:00 summary: fix `EventType` bases `StrEnum` does not exist in 3.8, so use original `(str, enum.Enum)` bases. files: A Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst M Lib/tkinter/__init__.py diff --git a/Lib/tkinter/__init__.py b/Lib/tkinter/__init__.py index 1067ab6a8b8a1..11ced6a3aff81 100644 --- a/Lib/tkinter/__init__.py +++ b/Lib/tkinter/__init__.py @@ -146,10 +146,10 @@ def _splitdict(tk, v, cut_minus=True, conv=None): class EventType(str, enum.Enum): KeyPress = '2' - Key = KeyPress, + Key = KeyPress KeyRelease = '3' ButtonPress = '4' - Button = ButtonPress, + Button = ButtonPress ButtonRelease = '5' Motion = '6' Enter = '7' @@ -180,10 +180,10 @@ class EventType(str, enum.Enum): Colormap = '32' ClientMessage = '33' # undocumented Mapping = '34' # undocumented - VirtualEvent = '35', # undocumented - Activate = '36', - Deactivate = '37', - MouseWheel = '38', + VirtualEvent = '35' # undocumented + Activate = '36' + Deactivate = '37' + MouseWheel = '38' def __str__(self): return self.name diff --git a/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst b/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst new file mode 100644 index 0000000000000..6a634bb613260 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst @@ -0,0 +1 @@ +fix `tkinter.EventType` Enum so all members are strings, and none are tuples From webhook-mailer at python.org Tue Sep 22 14:35:46 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 22 Sep 2020 18:35:46 -0000 Subject: [Python-checkins] bpo-35764: Rewrite the IDLE Calltips doc section (GH-22363) Message-ID: https://github.com/python/cpython/commit/f27a1577d32f88c959e1ef6d0b12f25b2a54cdca commit: f27a1577d32f88c959e1ef6d0b12f25b2a54cdca branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-22T14:35:38-04:00 summary: bpo-35764: Rewrite the IDLE Calltips doc section (GH-22363) files: A Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst M Doc/library/idle.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/help.html diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 43096b014fed3..a59a5d3a46570 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -527,30 +527,33 @@ by typing '_' after '.', either before or after the box is opened. Calltips ^^^^^^^^ -A calltip is shown when one types :kbd:`(` after the name of an *accessible* -function. A name expression may include dots and subscripts. A calltip -remains until it is clicked, the cursor is moved out of the argument area, -or :kbd:`)` is typed. When the cursor is in the argument part of a definition, -the menu or shortcut display a calltip. - -A calltip consists of the function signature and the first line of the -docstring. For builtins without an accessible signature, the calltip -consists of all lines up the fifth line or the first blank line. These -details may change. - -The set of *accessible* functions depends on what modules have been imported -into the user process, including those imported by Idle itself, -and what definitions have been run, all since the last restart. +A calltip is shown automatically when one types :kbd:`(` after the name +of an *accessible* function. A function name expression may include +dots and subscripts. A calltip remains until it is clicked, the cursor +is moved out of the argument area, or :kbd:`)` is typed. Whenever the +cursor is in the argument part of a definition, select Edit and "Show +Call Tip" on the menu or enter its shortcut to display a calltip. + +The calltip consists of the function's signature and docstring up to +the latter's first blank line or the fifth non-blank line. (Some builtin +functions lack an accessible signature.) A '/' or '*' in the signature +indicates that the preceding or following arguments are passed by +position or name (keyword) only. Details are subject to change. + +In Shell, the accessible functions depends on what modules have been +imported into the user process, including those imported by Idle itself, +and which definitions have been run, all since the last restart. For example, restart the Shell and enter ``itertools.count(``. A calltip -appears because Idle imports itertools into the user process for its own use. -(This could change.) Enter ``turtle.write(`` and nothing appears. Idle does -not import turtle. The menu or shortcut do nothing either. Enter -``import turtle`` and then ``turtle.write(`` will work. - -In an editor, import statements have no effect until one runs the file. One -might want to run a file after writing the import statements at the top, -or immediately run an existing file before editing. +appears because Idle imports itertools into the user process for its own +use. (This could change.) Enter ``turtle.write(`` and nothing appears. +Idle does not itself import turtle. The menu entry and shortcut also do +nothing. Enter ``import turtle``. Thereafter, ``turtle.write(`` +will display a calltip. + +In an editor, import statements have no effect until one runs the file. +One might want to run a file after writing import statements, after +adding function definitions, or after opening an existing file. .. _code-context: diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index f8ec6ab505297..7eea0a47aa6e8 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,8 @@ Released on 2020-10-05? ====================================== +bpo-35764: Rewrite the Calltips doc section. + bpo-40181: In calltips, stop reminding that '/' marks the end of positional-only arguments. diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html index b2853cffe0c26..0edd3917e1ffa 100644 --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -509,26 +509,29 @@

Automatic indentation

Calltips?

-

A calltip is shown when one types ( after the name of an accessible -function. A name expression may include dots and subscripts. A calltip -remains until it is clicked, the cursor is moved out of the argument area, -or ) is typed. When the cursor is in the argument part of a definition, -the menu or shortcut display a calltip.

-

A calltip consists of the function signature and the first line of the -docstring. For builtins without an accessible signature, the calltip -consists of all lines up the fifth line or the first blank line. These -details may change.

-

The set of accessible functions depends on what modules have been imported -into the user process, including those imported by Idle itself, -and what definitions have been run, all since the last restart.

+

A calltip is shown automatically when one types ( after the name +of an accessible function. A function name expression may include +dots and subscripts. A calltip remains until it is clicked, the cursor +is moved out of the argument area, or ) is typed. Whenever the +cursor is in the argument part of a definition, select Edit and ?Show +Call Tip? on the menu or enter its shortcut to display a calltip.

+

The calltip consists of the function?s signature and docstring up to +the latter?s first blank line or the fifth non-blank line. (Some builtin +functions lack an accessible signature.) A ?/? or ?*? in the signature +indicates that the preceding or following arguments are passed by +position or name (keyword) only. Details are subject to change.

+

In Shell, the accessible functions depends on what modules have been +imported into the user process, including those imported by Idle itself, +and which definitions have been run, all since the last restart.

For example, restart the Shell and enter itertools.count(. A calltip -appears because Idle imports itertools into the user process for its own use. -(This could change.) Enter turtle.write( and nothing appears. Idle does -not import turtle. The menu or shortcut do nothing either. Enter -import turtle and then turtle.write( will work.

-

In an editor, import statements have no effect until one runs the file. One -might want to run a file after writing the import statements at the top, -or immediately run an existing file before editing.

+appears because Idle imports itertools into the user process for its own +use. (This could change.) Enter turtle.write( and nothing appears. +Idle does not itself import turtle. The menu entry and shortcut also do +nothing. Enter import turtle. Thereafter, turtle.write( +will display a calltip.

+

In an editor, import statements have no effect until one runs the file. +One might want to run a file after writing import statements, after +adding function definitions, or after opening an existing file.

Code Context?

@@ -975,7 +978,7 @@

Navigation



- Last updated on Sep 09, 2020. + Last updated on Sep 22, 2020. Found a bug?
diff --git a/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst b/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst new file mode 100644 index 0000000000000..eb62d3699d5fe --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst @@ -0,0 +1 @@ +Rewrite the Calltips doc section. From webhook-mailer at python.org Tue Sep 22 14:36:07 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 22 Sep 2020 18:36:07 -0000 Subject: [Python-checkins] bpo-35764: Rewrite the IDLE Calltips doc section (GH-22363) Message-ID: https://github.com/python/cpython/commit/2466a7ae6bb1e4049c3d045a30a0503dda7654c5 commit: 2466a7ae6bb1e4049c3d045a30a0503dda7654c5 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-22T14:36:02-04:00 summary: bpo-35764: Rewrite the IDLE Calltips doc section (GH-22363) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst M Doc/library/idle.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/help.html diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst index 43096b014fed3..a59a5d3a46570 100644 --- a/Doc/library/idle.rst +++ b/Doc/library/idle.rst @@ -527,30 +527,33 @@ by typing '_' after '.', either before or after the box is opened. Calltips ^^^^^^^^ -A calltip is shown when one types :kbd:`(` after the name of an *accessible* -function. A name expression may include dots and subscripts. A calltip -remains until it is clicked, the cursor is moved out of the argument area, -or :kbd:`)` is typed. When the cursor is in the argument part of a definition, -the menu or shortcut display a calltip. - -A calltip consists of the function signature and the first line of the -docstring. For builtins without an accessible signature, the calltip -consists of all lines up the fifth line or the first blank line. These -details may change. - -The set of *accessible* functions depends on what modules have been imported -into the user process, including those imported by Idle itself, -and what definitions have been run, all since the last restart. +A calltip is shown automatically when one types :kbd:`(` after the name +of an *accessible* function. A function name expression may include +dots and subscripts. A calltip remains until it is clicked, the cursor +is moved out of the argument area, or :kbd:`)` is typed. Whenever the +cursor is in the argument part of a definition, select Edit and "Show +Call Tip" on the menu or enter its shortcut to display a calltip. + +The calltip consists of the function's signature and docstring up to +the latter's first blank line or the fifth non-blank line. (Some builtin +functions lack an accessible signature.) A '/' or '*' in the signature +indicates that the preceding or following arguments are passed by +position or name (keyword) only. Details are subject to change. + +In Shell, the accessible functions depends on what modules have been +imported into the user process, including those imported by Idle itself, +and which definitions have been run, all since the last restart. For example, restart the Shell and enter ``itertools.count(``. A calltip -appears because Idle imports itertools into the user process for its own use. -(This could change.) Enter ``turtle.write(`` and nothing appears. Idle does -not import turtle. The menu or shortcut do nothing either. Enter -``import turtle`` and then ``turtle.write(`` will work. - -In an editor, import statements have no effect until one runs the file. One -might want to run a file after writing the import statements at the top, -or immediately run an existing file before editing. +appears because Idle imports itertools into the user process for its own +use. (This could change.) Enter ``turtle.write(`` and nothing appears. +Idle does not itself import turtle. The menu entry and shortcut also do +nothing. Enter ``import turtle``. Thereafter, ``turtle.write(`` +will display a calltip. + +In an editor, import statements have no effect until one runs the file. +One might want to run a file after writing import statements, after +adding function definitions, or after opening an existing file. .. _code-context: diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 2cb1910c6c3f3..5d9602e707b07 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,8 @@ Released on 2020-09-14? ====================================== +bpo-35764: Rewrite the Calltips doc section. + bpo-40181: In calltips, stop reminding that '/' marks the end of positional-only arguments. diff --git a/Lib/idlelib/help.html b/Lib/idlelib/help.html index b2853cffe0c26..0edd3917e1ffa 100644 --- a/Lib/idlelib/help.html +++ b/Lib/idlelib/help.html @@ -509,26 +509,29 @@

Automatic indentation

Calltips?

-

A calltip is shown when one types ( after the name of an accessible -function. A name expression may include dots and subscripts. A calltip -remains until it is clicked, the cursor is moved out of the argument area, -or ) is typed. When the cursor is in the argument part of a definition, -the menu or shortcut display a calltip.

-

A calltip consists of the function signature and the first line of the -docstring. For builtins without an accessible signature, the calltip -consists of all lines up the fifth line or the first blank line. These -details may change.

-

The set of accessible functions depends on what modules have been imported -into the user process, including those imported by Idle itself, -and what definitions have been run, all since the last restart.

+

A calltip is shown automatically when one types ( after the name +of an accessible function. A function name expression may include +dots and subscripts. A calltip remains until it is clicked, the cursor +is moved out of the argument area, or ) is typed. Whenever the +cursor is in the argument part of a definition, select Edit and ?Show +Call Tip? on the menu or enter its shortcut to display a calltip.

+

The calltip consists of the function?s signature and docstring up to +the latter?s first blank line or the fifth non-blank line. (Some builtin +functions lack an accessible signature.) A ?/? or ?*? in the signature +indicates that the preceding or following arguments are passed by +position or name (keyword) only. Details are subject to change.

+

In Shell, the accessible functions depends on what modules have been +imported into the user process, including those imported by Idle itself, +and which definitions have been run, all since the last restart.

For example, restart the Shell and enter itertools.count(. A calltip -appears because Idle imports itertools into the user process for its own use. -(This could change.) Enter turtle.write( and nothing appears. Idle does -not import turtle. The menu or shortcut do nothing either. Enter -import turtle and then turtle.write( will work.

-

In an editor, import statements have no effect until one runs the file. One -might want to run a file after writing the import statements at the top, -or immediately run an existing file before editing.

+appears because Idle imports itertools into the user process for its own +use. (This could change.) Enter turtle.write( and nothing appears. +Idle does not itself import turtle. The menu entry and shortcut also do +nothing. Enter import turtle. Thereafter, turtle.write( +will display a calltip.

+

In an editor, import statements have no effect until one runs the file. +One might want to run a file after writing import statements, after +adding function definitions, or after opening an existing file.

Code Context?

@@ -975,7 +978,7 @@

Navigation



- Last updated on Sep 09, 2020. + Last updated on Sep 22, 2020. Found a bug?
diff --git a/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst b/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst new file mode 100644 index 0000000000000..eb62d3699d5fe --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst @@ -0,0 +1 @@ +Rewrite the Calltips doc section. From webhook-mailer at python.org Tue Sep 22 16:00:26 2020 From: webhook-mailer at python.org (Ethan Furman) Date: Tue, 22 Sep 2020 20:00:26 -0000 Subject: [Python-checkins] bpo-41816: `StrEnum.__str__` is `str.__str__` (GH-22362) Message-ID: https://github.com/python/cpython/commit/d986d1657e1e7b50807d0633cb31d96a2d866d42 commit: d986d1657e1e7b50807d0633cb31d96a2d866d42 branch: master author: Ethan Furman committer: GitHub date: 2020-09-22T13:00:07-07:00 summary: bpo-41816: `StrEnum.__str__` is `str.__str__` (GH-22362) use `str.__str__` for `StrEnum` so that `str(StrEnum.member)` is the same as directly accessing the string value of the `StrEnum` member files: M Doc/library/enum.rst M Lib/enum.py M Lib/test/test_enum.py diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 3e9b1f9db3550..118002bef19f8 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -622,6 +622,11 @@ an incorrect member:: Before :class:`StrEnum`, ``Directions.NORTH`` would have been the :class:`tuple` ``('north',)``. +.. note:: + + Unlike other Enum's, ``str(StrEnum.member)`` will return the value of the + member instead of the usual ``"EnumClass.member"``. + .. versionadded:: 3.10 @@ -1243,3 +1248,13 @@ all named flags and all named combinations of flags that are in the value:: >>> Color(7) # not named combination +``StrEnum`` and :meth:`str.__str__` +""""""""""""""""""""""""""""""""""" + +An important difference between :class:`StrEnum` and other Enums is the +:meth:`__str__` method; because :class:`StrEnum` members are strings, some +parts of Python will read the string data directly, while others will call +:meth:`str()`. To make those two operations have the same result, +:meth:`StrEnum.__str__` will be the same as :meth:`str.__str__` so that +``str(StrEnum.member) == StrEnum.member`` is true. + diff --git a/Lib/enum.py b/Lib/enum.py index 589b17fd69777..40ff25b9cdad3 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -718,6 +718,8 @@ def __new__(cls, *values): member._value_ = value return member + __str__ = str.__str__ + def _reduce_ex_by_name(self, proto): return self.name diff --git a/Lib/test/test_enum.py b/Lib/test/test_enum.py index 8e84d929429eb..3431040f98a72 100644 --- a/Lib/test/test_enum.py +++ b/Lib/test/test_enum.py @@ -2036,6 +2036,37 @@ class GoodStrEnum(StrEnum): two = '2' three = b'3', 'ascii' four = b'4', 'latin1', 'strict' + self.assertEqual(GoodStrEnum.one, '1') + self.assertEqual(str(GoodStrEnum.one), '1') + self.assertEqual(GoodStrEnum.one, str(GoodStrEnum.one)) + self.assertEqual(GoodStrEnum.one, '{}'.format(GoodStrEnum.one)) + # + class DumbMixin: + def __str__(self): + return "don't do this" + class DumbStrEnum(DumbMixin, StrEnum): + five = '5' + six = '6' + seven = '7' + self.assertEqual(DumbStrEnum.seven, '7') + self.assertEqual(str(DumbStrEnum.seven), "don't do this") + # + class EnumMixin(Enum): + def hello(self): + print('hello from %s' % (self, )) + class HelloEnum(EnumMixin, StrEnum): + eight = '8' + self.assertEqual(HelloEnum.eight, '8') + self.assertEqual(HelloEnum.eight, str(HelloEnum.eight)) + # + class GoodbyeMixin: + def goodbye(self): + print('%s wishes you a fond farewell') + class GoodbyeEnum(GoodbyeMixin, EnumMixin, StrEnum): + nine = '9' + self.assertEqual(GoodbyeEnum.nine, '9') + self.assertEqual(GoodbyeEnum.nine, str(GoodbyeEnum.nine)) + # with self.assertRaisesRegex(TypeError, '1 is not a string'): class FirstFailedStrEnum(StrEnum): one = 1 From webhook-mailer at python.org Tue Sep 22 23:01:32 2020 From: webhook-mailer at python.org (Raymond Hettinger) Date: Wed, 23 Sep 2020 03:01:32 -0000 Subject: [Python-checkins] bpo-41513: Improve order of adding fractional values. Improve variable names. (GH-22368) Message-ID: https://github.com/python/cpython/commit/438e9fc66f664eff0526a16a6d900349bfd1f9d2 commit: 438e9fc66f664eff0526a16a6d900349bfd1f9d2 branch: master author: Raymond Hettinger committer: GitHub date: 2020-09-22T20:01:12-07:00 summary: bpo-41513: Improve order of adding fractional values. Improve variable names. (GH-22368) files: M Modules/mathmodule.c diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index 935759ec671ca..5cd31b7dd4acf 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -2502,8 +2502,8 @@ exactly equal) was verified for 1 billion random inputs with n=5. [7] static inline double vector_norm(Py_ssize_t n, double *vec, double max, int found_nan) { - const double T27 = 134217729.0; /* ldexp(1.0, 27)+1.0) */ - double x, csum = 1.0, oldcsum, scale, frac=0.0, frac_mid=0.0, frac_lo=0.0; + const double T27 = 134217729.0; /* ldexp(1.0, 27) + 1.0) */ + double x, scale, oldcsum, csum = 1.0, frac1 = 0.0, frac2 = 0.0, frac3 = 0.0; double t, hi, lo, h; int max_e; Py_ssize_t i; @@ -2539,18 +2539,18 @@ vector_norm(Py_ssize_t n, double *vec, double max, int found_nan) assert(fabs(csum) >= fabs(x)); oldcsum = csum; csum += x; - frac += (oldcsum - csum) + x; + frac1 += (oldcsum - csum) + x; x = 2.0 * hi * lo; assert(fabs(csum) >= fabs(x)); oldcsum = csum; csum += x; - frac_mid += (oldcsum - csum) + x; + frac2 += (oldcsum - csum) + x; assert(csum + lo * lo == csum); - frac_lo += lo * lo; + frac3 += lo * lo; } - h = sqrt(csum - 1.0 + (frac_lo + frac_mid + frac)); + h = sqrt(csum - 1.0 + (frac1 + frac2 + frac3)); x = h; t = x * T27; @@ -2562,21 +2562,21 @@ vector_norm(Py_ssize_t n, double *vec, double max, int found_nan) assert(fabs(csum) >= fabs(x)); oldcsum = csum; csum += x; - frac += (oldcsum - csum) + x; + frac1 += (oldcsum - csum) + x; x = -2.0 * hi * lo; assert(fabs(csum) >= fabs(x)); oldcsum = csum; csum += x; - frac_mid += (oldcsum - csum) + x; + frac2 += (oldcsum - csum) + x; x = -lo * lo; assert(fabs(csum) >= fabs(x)); oldcsum = csum; csum += x; - frac_lo += (oldcsum - csum) + x; + frac3 += (oldcsum - csum) + x; - x = csum - 1.0 + (frac_lo + frac_mid + frac); + x = csum - 1.0 + (frac1 + frac2 + frac3); return (h + x / (2.0 * h)) / scale; } /* When max_e < -1023, ldexp(1.0, -max_e) overflows. @@ -2591,9 +2591,9 @@ vector_norm(Py_ssize_t n, double *vec, double max, int found_nan) assert(fabs(csum) >= fabs(x)); oldcsum = csum; csum += x; - frac += (oldcsum - csum) + x; + frac1 += (oldcsum - csum) + x; } - return max * sqrt(csum - 1.0 + frac); + return max * sqrt(csum - 1.0 + frac1); } #define NUM_STACK_ELEMS 16 From webhook-mailer at python.org Tue Sep 22 23:57:57 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 23 Sep 2020 03:57:57 -0000 Subject: [Python-checkins] bpo-37062: Enum: add extended AutoNumber example (GH-22349) (GH-22369) Message-ID: https://github.com/python/cpython/commit/5acc1b5f0b62eef3258e4bc31eba3b9c659108c9 commit: 5acc1b5f0b62eef3258e4bc31eba3b9c659108c9 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-22T20:57:48-07:00 summary: bpo-37062: Enum: add extended AutoNumber example (GH-22349) (GH-22369) (cherry picked from commit 62e40d8450b9c78346ec3617de7fe3f0ad381510) Co-authored-by: Ethan Furman files: M Doc/library/enum.rst M Misc/ACKS diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 8753416a27416..bd10016b937b4 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -888,6 +888,32 @@ Using an auto-numbering :meth:`__new__` would look like:: >>> Color.GREEN.value 2 +To make a more general purpose ``AutoNumber``, add ``*args`` to the signature:: + + >>> class AutoNumber(NoValue): + ... def __new__(cls, *args): # this is the only change from above + ... value = len(cls.__members__) + 1 + ... obj = object.__new__(cls) + ... obj._value_ = value + ... return obj + ... + +Then when you inherit from ``AutoNumber`` you can write your own ``__init__`` +to handle any extra arguments:: + + >>> class Swatch(AutoNumber): + ... def __init__(self, pantone='unknown'): + ... self.pantone = pantone + ... AUBURN = '3497' + ... SEA_GREEN = '1246' + ... BLEACHED_CORAL = () # New color, no Pantone code yet! + ... + >>> Swatch.SEA_GREEN + + >>> Swatch.SEA_GREEN.pantone + '1246' + >>> Swatch.BLEACHED_CORAL.pantone + 'unknown' .. note:: diff --git a/Misc/ACKS b/Misc/ACKS index eee6b9accaa50..8d355da8fde05 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1678,6 +1678,7 @@ F?vry Thibault Lowe Thiderman Nicolas M. Thi?ry James Thomas +Reuben Thomas Robin Thomas Brian Thorne Christopher Thorne From webhook-mailer at python.org Tue Sep 22 23:58:39 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 23 Sep 2020 03:58:39 -0000 Subject: [Python-checkins] bpo-37062: Enum: add extended AutoNumber example (GH-22349) (GH-22370) Message-ID: https://github.com/python/cpython/commit/64362c2e435eddc5e84cea313d92bc0b96d227f7 commit: 64362c2e435eddc5e84cea313d92bc0b96d227f7 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-22T20:58:32-07:00 summary: bpo-37062: Enum: add extended AutoNumber example (GH-22349) (GH-22370) (cherry picked from commit 62e40d8450b9c78346ec3617de7fe3f0ad381510) Co-authored-by: Ethan Furman files: M Doc/library/enum.rst M Misc/ACKS diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst index 5f0762edc2590..a3c51655576ba 100644 --- a/Doc/library/enum.rst +++ b/Doc/library/enum.rst @@ -887,6 +887,32 @@ Using an auto-numbering :meth:`__new__` would look like:: >>> Color.GREEN.value 2 +To make a more general purpose ``AutoNumber``, add ``*args`` to the signature:: + + >>> class AutoNumber(NoValue): + ... def __new__(cls, *args): # this is the only change from above + ... value = len(cls.__members__) + 1 + ... obj = object.__new__(cls) + ... obj._value_ = value + ... return obj + ... + +Then when you inherit from ``AutoNumber`` you can write your own ``__init__`` +to handle any extra arguments:: + + >>> class Swatch(AutoNumber): + ... def __init__(self, pantone='unknown'): + ... self.pantone = pantone + ... AUBURN = '3497' + ... SEA_GREEN = '1246' + ... BLEACHED_CORAL = () # New color, no Pantone code yet! + ... + >>> Swatch.SEA_GREEN + + >>> Swatch.SEA_GREEN.pantone + '1246' + >>> Swatch.BLEACHED_CORAL.pantone + 'unknown' .. note:: diff --git a/Misc/ACKS b/Misc/ACKS index 5e0953ae91b18..a16f15a74ebdc 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1713,6 +1713,7 @@ F?vry Thibault Lowe Thiderman Nicolas M. Thi?ry James Thomas +Reuben Thomas Robin Thomas Brian Thorne Christopher Thorne From webhook-mailer at python.org Wed Sep 23 06:33:49 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Wed, 23 Sep 2020 10:33:49 -0000 Subject: [Python-checkins] bpo-1635741: Port _lsprof extension to multi-phase init (PEP 489) (GH-22220) Message-ID: https://github.com/python/cpython/commit/83de110dce94a9196dccc01d526628615714e362 commit: 83de110dce94a9196dccc01d526628615714e362 branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-23T12:33:21+02:00 summary: bpo-1635741: Port _lsprof extension to multi-phase init (PEP 489) (GH-22220) files: A Misc/NEWS.d/next/Core and Builtins/2020-09-12-18-34-34.bpo-1635741.lh335O.rst M Modules/_lsprof.c M Modules/clinic/_lsprof.c.h diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-12-18-34-34.bpo-1635741.lh335O.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-12-18-34-34.bpo-1635741.lh335O.rst new file mode 100644 index 0000000000000..ba61819df9e08 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-12-18-34-34.bpo-1635741.lh335O.rst @@ -0,0 +1,2 @@ +Port the :mod:`_lsprof` extension module to multi-phase initialization +(:pep:`489`). diff --git a/Modules/_lsprof.c b/Modules/_lsprof.c index a4ba7d5230033..78d464d1481d7 100644 --- a/Modules/_lsprof.c +++ b/Modules/_lsprof.c @@ -55,12 +55,22 @@ module _lsprof class _lsprof.Profiler "ProfilerObject *" "&ProfilerType" [clinic start generated code]*/ /*[clinic end generated code: output=da39a3ee5e6b4b0d input=e349ac952152f336]*/ -static PyTypeObject PyProfiler_Type; #include "clinic/_lsprof.c.h" -#define PyProfiler_Check(op) PyObject_TypeCheck(op, &PyProfiler_Type) -#define PyProfiler_CheckExact(op) Py_IS_TYPE(op, &PyProfiler_Type) +typedef struct { + PyTypeObject *profiler_type; + PyTypeObject *stats_entry_type; + PyTypeObject *stats_subentry_type; +} _lsprof_state; + +static inline _lsprof_state* +_lsprof_get_state(PyObject *module) +{ + void *state = PyModule_GetState(module); + assert(state != NULL); + return (_lsprof_state *)state; +} /*** External Timers ***/ @@ -478,28 +488,24 @@ static PyStructSequence_Field profiler_subentry_fields[] = { }; static PyStructSequence_Desc profiler_entry_desc = { - "_lsprof.profiler_entry", /* name */ - NULL, /* doc */ - profiler_entry_fields, - 6 + .name = "_lsprof.profiler_entry", + .doc = "", + .fields = profiler_entry_fields, + .n_in_sequence = 6 }; static PyStructSequence_Desc profiler_subentry_desc = { - "_lsprof.profiler_subentry", /* name */ - NULL, /* doc */ - profiler_subentry_fields, - 5 + .name = "_lsprof.profiler_subentry", + .doc = "", + .fields = profiler_subentry_fields, + .n_in_sequence = 5 }; -static int initialized; -static PyTypeObject StatsEntryType; -static PyTypeObject StatsSubEntryType; - - typedef struct { PyObject *list; PyObject *sublist; double factor; + _lsprof_state *state; } statscollector_t; static int statsForSubEntry(rotating_node_t *node, void *arg) @@ -509,7 +515,7 @@ static int statsForSubEntry(rotating_node_t *node, void *arg) ProfilerEntry *entry = (ProfilerEntry*) sentry->header.key; int err; PyObject *sinfo; - sinfo = PyObject_CallFunction((PyObject*) &StatsSubEntryType, + sinfo = PyObject_CallFunction((PyObject*) collect->state->stats_subentry_type, "((Olldd))", entry->userObj, sentry->callcount, @@ -547,7 +553,7 @@ static int statsForEntry(rotating_node_t *node, void *arg) collect->sublist = Py_None; } - info = PyObject_CallFunction((PyObject*) &StatsEntryType, + info = PyObject_CallFunction((PyObject*) collect->state->stats_entry_type, "((OllddO))", entry->userObj, entry->callcount, @@ -566,6 +572,8 @@ static int statsForEntry(rotating_node_t *node, void *arg) /*[clinic input] _lsprof.Profiler.getstats + cls: defining_class + list of profiler_entry objects. getstats() -> list of profiler_entry objects @@ -592,10 +600,11 @@ profiler_subentry objects: [clinic start generated code]*/ static PyObject * -_lsprof_Profiler_getstats_impl(ProfilerObject *self) -/*[clinic end generated code: output=9461b451e9ef0f24 input=ade04fa384ce450a]*/ +_lsprof_Profiler_getstats_impl(ProfilerObject *self, PyTypeObject *cls) +/*[clinic end generated code: output=1806ef720019ee03 input=445e193ef4522902]*/ { statscollector_t collect; + collect.state = PyType_GetModuleState(cls); if (pending_exception(self)) { return NULL; } @@ -735,7 +744,9 @@ profiler_dealloc(ProfilerObject *op) flush_unmatched(op); clearEntries(op); Py_XDECREF(op->externalTimer); - Py_TYPE(op)->tp_free(op); + PyTypeObject *tp = Py_TYPE(op); + tp->tp_free(op); + Py_DECREF(tp); } static int @@ -782,91 +793,107 @@ Profiler(timer=None, timeunit=None, subcalls=True, builtins=True)\n\ is, in seconds).\n\ "); -static PyTypeObject PyProfiler_Type = { - PyVarObject_HEAD_INIT(NULL, 0) - "_lsprof.Profiler", /* tp_name */ - sizeof(ProfilerObject), /* tp_basicsize */ - 0, /* tp_itemsize */ - (destructor)profiler_dealloc, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags */ - profiler_doc, /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - profiler_methods, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - (initproc)profiler_init, /* tp_init */ - PyType_GenericAlloc, /* tp_alloc */ - PyType_GenericNew, /* tp_new */ - PyObject_Del, /* tp_free */ +static PyType_Slot _lsprof_profiler_type_spec_slots[] = { + {Py_tp_doc, (void *)profiler_doc}, + {Py_tp_methods, profiler_methods}, + {Py_tp_dealloc, profiler_dealloc}, + {Py_tp_init, profiler_init}, + {Py_tp_alloc, PyType_GenericAlloc}, + {Py_tp_new, PyType_GenericNew}, + {Py_tp_free, PyObject_Del}, + {0, 0} +}; + +static PyType_Spec _lsprof_profiler_type_spec = { + .name = "_lsprof.Profiler", + .basicsize = sizeof(ProfilerObject), + .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, + .slots = _lsprof_profiler_type_spec_slots, }; static PyMethodDef moduleMethods[] = { {NULL, NULL} }; +static int +_lsprof_traverse(PyObject *module, visitproc visit, void *arg) +{ + _lsprof_state *state = _lsprof_get_state(module); + Py_VISIT(state->profiler_type); + Py_VISIT(state->stats_entry_type); + Py_VISIT(state->stats_subentry_type); + return 0; +} + +static int +_lsprof_clear(PyObject *module) +{ + _lsprof_state *state = _lsprof_get_state(module); + Py_CLEAR(state->profiler_type); + Py_CLEAR(state->stats_entry_type); + Py_CLEAR(state->stats_subentry_type); + return 0; +} + +static void +_lsprof_free(void *module) +{ + _lsprof_clear((PyObject *)module); +} + +static int +_lsprof_exec(PyObject *module) +{ + _lsprof_state *state = PyModule_GetState(module); + + state->profiler_type = (PyTypeObject *)PyType_FromModuleAndSpec( + module, &_lsprof_profiler_type_spec, NULL); + if (state->profiler_type == NULL) { + return -1; + } + + if (PyModule_AddType(module, state->profiler_type) < 0) { + return -1; + } + + state->stats_entry_type = PyStructSequence_NewType(&profiler_entry_desc); + if (state->stats_entry_type == NULL) { + return -1; + } + if (PyModule_AddType(module, state->stats_entry_type) < 0) { + return -1; + } + + state->stats_subentry_type = PyStructSequence_NewType(&profiler_subentry_desc); + if (state->stats_subentry_type == NULL) { + return -1; + } + if (PyModule_AddType(module, state->stats_subentry_type) < 0) { + return -1; + } + + return 0; +} + +static PyModuleDef_Slot _lsprofslots[] = { + {Py_mod_exec, _lsprof_exec}, + {0, NULL} +}; static struct PyModuleDef _lsprofmodule = { PyModuleDef_HEAD_INIT, - "_lsprof", - "Fast profiler", - -1, - moduleMethods, - NULL, - NULL, - NULL, - NULL + .m_name = "_lsprof", + .m_doc = "Fast profiler", + .m_size = sizeof(_lsprof_state), + .m_methods = moduleMethods, + .m_slots = _lsprofslots, + .m_traverse = _lsprof_traverse, + .m_clear = _lsprof_clear, + .m_free = _lsprof_free }; PyMODINIT_FUNC PyInit__lsprof(void) { - PyObject *module, *d; - module = PyModule_Create(&_lsprofmodule); - if (module == NULL) - return NULL; - d = PyModule_GetDict(module); - if (PyType_Ready(&PyProfiler_Type) < 0) - return NULL; - PyDict_SetItemString(d, "Profiler", (PyObject *)&PyProfiler_Type); - - if (!initialized) { - if (PyStructSequence_InitType2(&StatsEntryType, - &profiler_entry_desc) < 0) - return NULL; - if (PyStructSequence_InitType2(&StatsSubEntryType, - &profiler_subentry_desc) < 0) - return NULL; - } - Py_INCREF((PyObject*) &StatsEntryType); - Py_INCREF((PyObject*) &StatsSubEntryType); - PyModule_AddObject(module, "profiler_entry", - (PyObject*) &StatsEntryType); - PyModule_AddObject(module, "profiler_subentry", - (PyObject*) &StatsSubEntryType); - initialized = 1; - return module; + return PyModuleDef_Init(&_lsprofmodule); } diff --git a/Modules/clinic/_lsprof.c.h b/Modules/clinic/_lsprof.c.h index 50762e3ff3596..5d9c209eab856 100644 --- a/Modules/clinic/_lsprof.c.h +++ b/Modules/clinic/_lsprof.c.h @@ -31,14 +31,25 @@ PyDoc_STRVAR(_lsprof_Profiler_getstats__doc__, " inlinetime inline time (not in further subcalls)"); #define _LSPROF_PROFILER_GETSTATS_METHODDEF \ - {"getstats", (PyCFunction)_lsprof_Profiler_getstats, METH_NOARGS, _lsprof_Profiler_getstats__doc__}, + {"getstats", (PyCFunction)(void(*)(void))_lsprof_Profiler_getstats, METH_METHOD|METH_FASTCALL|METH_KEYWORDS, _lsprof_Profiler_getstats__doc__}, static PyObject * -_lsprof_Profiler_getstats_impl(ProfilerObject *self); +_lsprof_Profiler_getstats_impl(ProfilerObject *self, PyTypeObject *cls); static PyObject * -_lsprof_Profiler_getstats(ProfilerObject *self, PyObject *Py_UNUSED(ignored)) +_lsprof_Profiler_getstats(ProfilerObject *self, PyTypeObject *cls, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) { - return _lsprof_Profiler_getstats_impl(self); + PyObject *return_value = NULL; + static const char * const _keywords[] = { NULL}; + static _PyArg_Parser _parser = {":getstats", _keywords, 0}; + + if (!_PyArg_ParseStackAndKeywords(args, nargs, kwnames, &_parser + )) { + goto exit; + } + return_value = _lsprof_Profiler_getstats_impl(self, cls); + +exit: + return return_value; } -/*[clinic end generated code: output=24c525812713e00f input=a9049054013a1b77]*/ +/*[clinic end generated code: output=b4727cfebecdd22d input=a9049054013a1b77]*/ From webhook-mailer at python.org Wed Sep 23 06:38:20 2020 From: webhook-mailer at python.org (Mohamed Koubaa) Date: Wed, 23 Sep 2020 10:38:20 -0000 Subject: [Python-checkins] bpo-1635741, unicodedata: add ucd_type parameter to UCD_Check() macro (GH-22328) Message-ID: https://github.com/python/cpython/commit/ddc0dd001a4224274ba6f83568b45a1dd88c6fc6 commit: ddc0dd001a4224274ba6f83568b45a1dd88c6fc6 branch: master author: Mohamed Koubaa committer: GitHub date: 2020-09-23T12:38:16+02:00 summary: bpo-1635741, unicodedata: add ucd_type parameter to UCD_Check() macro (GH-22328) Co-authored-by: Victor Stinner files: M Modules/unicodedata.c diff --git a/Modules/unicodedata.c b/Modules/unicodedata.c index 8a1198a2b712d..8e11cfc4dafa9 100644 --- a/Modules/unicodedata.c +++ b/Modules/unicodedata.c @@ -92,7 +92,10 @@ static PyMemberDef DB_members[] = { /* forward declaration */ static PyTypeObject UCD_Type; -#define UCD_Check(o) Py_IS_TYPE(o, &UCD_Type) + +// Check if self is an instance of UCD_Type. +// Return 0 if self is NULL (when the PyCapsule C API is used). +#define UCD_Check(self, ucd_type) (self != NULL && Py_IS_TYPE(self, ucd_type)) static PyObject* new_previous_version(const char*name, const change_record* (*getrecord)(Py_UCS4), @@ -135,7 +138,7 @@ unicodedata_UCD_decimal_impl(PyObject *self, int chr, long rc; Py_UCS4 c = (Py_UCS4)chr; - if (self && UCD_Check(self)) { + if (UCD_Check(self, &UCD_Type)) { const change_record *old = get_old_record(self, c); if (old->category_changed == 0) { /* unassigned */ @@ -223,7 +226,7 @@ unicodedata_UCD_numeric_impl(PyObject *self, int chr, double rc; Py_UCS4 c = (Py_UCS4)chr; - if (self && UCD_Check(self)) { + if (UCD_Check(self, &UCD_Type)) { const change_record *old = get_old_record(self, c); if (old->category_changed == 0) { /* unassigned */ @@ -268,7 +271,7 @@ unicodedata_UCD_category_impl(PyObject *self, int chr) int index; Py_UCS4 c = (Py_UCS4)chr; index = (int) _getrecord_ex(c)->category; - if (self && UCD_Check(self)) { + if (UCD_Check(self, &UCD_Type)) { const change_record *old = get_old_record(self, c); if (old->category_changed != 0xFF) index = old->category_changed; @@ -295,7 +298,7 @@ unicodedata_UCD_bidirectional_impl(PyObject *self, int chr) int index; Py_UCS4 c = (Py_UCS4)chr; index = (int) _getrecord_ex(c)->bidirectional; - if (self && UCD_Check(self)) { + if (UCD_Check(self, &UCD_Type)) { const change_record *old = get_old_record(self, c); if (old->category_changed == 0) index = 0; /* unassigned */ @@ -324,7 +327,7 @@ unicodedata_UCD_combining_impl(PyObject *self, int chr) int index; Py_UCS4 c = (Py_UCS4)chr; index = (int) _getrecord_ex(c)->combining; - if (self && UCD_Check(self)) { + if (UCD_Check(self, &UCD_Type)) { const change_record *old = get_old_record(self, c); if (old->category_changed == 0) index = 0; /* unassigned */ @@ -352,7 +355,7 @@ unicodedata_UCD_mirrored_impl(PyObject *self, int chr) int index; Py_UCS4 c = (Py_UCS4)chr; index = (int) _getrecord_ex(c)->mirrored; - if (self && UCD_Check(self)) { + if (UCD_Check(self, &UCD_Type)) { const change_record *old = get_old_record(self, c); if (old->category_changed == 0) index = 0; /* unassigned */ @@ -379,7 +382,7 @@ unicodedata_UCD_east_asian_width_impl(PyObject *self, int chr) int index; Py_UCS4 c = (Py_UCS4)chr; index = (int) _getrecord_ex(c)->east_asian_width; - if (self && UCD_Check(self)) { + if (UCD_Check(self, &UCD_Type)) { const change_record *old = get_old_record(self, c); if (old->category_changed == 0) index = 0; /* unassigned */ @@ -413,7 +416,7 @@ unicodedata_UCD_decomposition_impl(PyObject *self, int chr) code = (int)c; - if (self && UCD_Check(self)) { + if (UCD_Check(self, &UCD_Type)) { const change_record *old = get_old_record(self, c); if (old->category_changed == 0) return PyUnicode_FromString(""); /* unassigned */ @@ -460,7 +463,7 @@ get_decomp_record(PyObject *self, Py_UCS4 code, int *index, int *prefix, int *co { if (code >= 0x110000) { *index = 0; - } else if (self && UCD_Check(self) && + } else if (UCD_Check(self, &UCD_Type) && get_old_record(self, code)->category_changed==0) { /* unassigned in old version */ *index = 0; @@ -558,7 +561,7 @@ nfd_nfkd(PyObject *self, PyObject *input, int k) continue; } /* normalization changes */ - if (self && UCD_Check(self)) { + if (UCD_Check(self, &UCD_Type)) { Py_UCS4 value = ((PreviousDBVersion*)self)->normalization(code); if (value != 0) { stack[stackptr++] = value; @@ -799,7 +802,7 @@ is_normalized_quickcheck(PyObject *self, PyObject *input, { /* An older version of the database is requested, quickchecks must be disabled. */ - if (self && UCD_Check(self)) + if (UCD_Check(self, &UCD_Type)) return NO; Py_ssize_t i, len; @@ -1066,7 +1069,7 @@ _getucname(PyObject *self, Py_UCS4 code, char* buffer, int buflen, if (!with_alias_and_seq && (IS_ALIAS(code) || IS_NAMED_SEQ(code))) return 0; - if (self && UCD_Check(self)) { + if (UCD_Check(self, &UCD_Type)) { /* in 3.2.0 there are no aliases and named sequences */ const change_record *old; if (IS_ALIAS(code) || IS_NAMED_SEQ(code)) From webhook-mailer at python.org Wed Sep 23 08:05:02 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 23 Sep 2020 12:05:02 -0000 Subject: [Python-checkins] bpo-41834: Remove _Py_CheckRecursionLimit variable (GH-22359) Message-ID: https://github.com/python/cpython/commit/19c3ac92bf73f1902cff846988552fd7bb8a8621 commit: 19c3ac92bf73f1902cff846988552fd7bb8a8621 branch: master author: Victor Stinner committer: GitHub date: 2020-09-23T14:04:57+02:00 summary: bpo-41834: Remove _Py_CheckRecursionLimit variable (GH-22359) Remove the global _Py_CheckRecursionLimit variable: it has been replaced by ceval.recursion_limit of the PyInterpreterState structure. There is no need to keep the variable for the stable ABI, since Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() were not usable in Python 3.8 and older: these macros accessed PyThreadState members, whereas the PyThreadState structure is opaque in the limited C API. files: A Misc/NEWS.d/next/C API/2020-09-22-14-47-12.bpo-41834.nrOrDU.rst M Doc/whatsnew/3.10.rst M Include/internal/pycore_ceval.h M PC/python3dll.c M Python/ceval.c M Tools/c-analyzer/TODO M Tools/c-analyzer/known.tsv diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index f88281a934ca1..606fad2fdcf05 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -314,3 +314,7 @@ Removed * Removed ``PyUnicode_AsUnicodeCopy()``. Please use :c:func:`PyUnicode_AsUCS4Copy` or :c:func:`PyUnicode_AsWideCharString` (Contributed by Inada Naoki in :issue:`41103`.) + +* Removed ``_Py_CheckRecursionLimit`` variable: it has been replaced by + ``ceval.recursion_limit`` of the :c:type:`PyInterpreterState` structure. + (Contributed by Victor Stinner in :issue:`41834`.) diff --git a/Include/internal/pycore_ceval.h b/Include/internal/pycore_ceval.h index aafb533b57d5f..bbb667ea32d27 100644 --- a/Include/internal/pycore_ceval.h +++ b/Include/internal/pycore_ceval.h @@ -63,8 +63,6 @@ extern void _PyEval_ReleaseLock(PyThreadState *tstate); /* --- _Py_EnterRecursiveCall() ----------------------------------------- */ -PyAPI_DATA(int) _Py_CheckRecursionLimit; - #ifdef USE_STACKCHECK /* With USE_STACKCHECK macro defined, trigger stack checks in _Py_CheckRecursiveCall() on every 64th call to Py_EnterRecursiveCall. */ diff --git a/Misc/NEWS.d/next/C API/2020-09-22-14-47-12.bpo-41834.nrOrDU.rst b/Misc/NEWS.d/next/C API/2020-09-22-14-47-12.bpo-41834.nrOrDU.rst new file mode 100644 index 0000000000000..07043dce5a627 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2020-09-22-14-47-12.bpo-41834.nrOrDU.rst @@ -0,0 +1,3 @@ +Remove the ``_Py_CheckRecursionLimit`` variable: it has been replaced by +``ceval.recursion_limit`` of the :c:type:`PyInterpreterState` +structure. Patch by Victor Stinner. diff --git a/PC/python3dll.c b/PC/python3dll.c index f72f2c8af19d2..ff69ea7ca5efa 100644 --- a/PC/python3dll.c +++ b/PC/python3dll.c @@ -663,7 +663,6 @@ EXPORT_FUNC(PyWeakref_NewProxy) EXPORT_FUNC(PyWeakref_NewRef) EXPORT_FUNC(PyWrapper_New) -EXPORT_DATA(_Py_CheckRecursionLimit) EXPORT_DATA(_Py_EllipsisObject) EXPORT_DATA(_Py_FalseStruct) EXPORT_DATA(_Py_NoneStruct) diff --git a/Python/ceval.c b/Python/ceval.c index 3de372f45a251..4bb4b820b8e17 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -741,15 +741,12 @@ Py_MakePendingCalls(void) /* The interpreter's recursion limit */ #ifndef Py_DEFAULT_RECURSION_LIMIT -#define Py_DEFAULT_RECURSION_LIMIT 1000 +# define Py_DEFAULT_RECURSION_LIMIT 1000 #endif -int _Py_CheckRecursionLimit = Py_DEFAULT_RECURSION_LIMIT; - void _PyEval_InitRuntimeState(struct _ceval_runtime_state *ceval) { - _Py_CheckRecursionLimit = Py_DEFAULT_RECURSION_LIMIT; #ifndef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS _gil_initialize(&ceval->gil); #endif @@ -797,14 +794,11 @@ Py_SetRecursionLimit(int new_limit) { PyThreadState *tstate = _PyThreadState_GET(); tstate->interp->ceval.recursion_limit = new_limit; - if (_Py_IsMainInterpreter(tstate)) { - _Py_CheckRecursionLimit = new_limit; - } } /* The function _Py_EnterRecursiveCall() only calls _Py_CheckRecursiveCall() - if the recursion_depth reaches _Py_CheckRecursionLimit. - If USE_STACKCHECK, the macro decrements _Py_CheckRecursionLimit + if the recursion_depth reaches recursion_limit. + If USE_STACKCHECK, the macro decrements recursion_limit to guarantee that _Py_CheckRecursiveCall() is regularly called. Without USE_STACKCHECK, there is no need for this. */ int @@ -819,10 +813,6 @@ _Py_CheckRecursiveCall(PyThreadState *tstate, const char *where) _PyErr_SetString(tstate, PyExc_MemoryError, "Stack overflow"); return -1; } - if (_Py_IsMainInterpreter(tstate)) { - /* Needed for ABI backwards-compatibility (see bpo-31857) */ - _Py_CheckRecursionLimit = recursion_limit; - } #endif if (tstate->recursion_critical) /* Somebody asked that we don't check for recursion. */ diff --git a/Tools/c-analyzer/TODO b/Tools/c-analyzer/TODO index 1134055f9a8fd..f5c1a92f85695 100644 --- a/Tools/c-analyzer/TODO +++ b/Tools/c-analyzer/TODO @@ -66,7 +66,6 @@ Objects/tupleobject.c:_Py_tuple_zero_allocs Py_ssize_t _Py_ Objects/typeobject.c:next_version_tag static unsigned int next_version_tag Python/Python-ast.c:init_types():initialized static int initialized Python/bootstrap_hash.c:urandom_cache static struct { int fd; dev_t st_dev; ino_t st_ino; } urandom_cache -Python/ceval.c:_Py_CheckRecursionLimit int _Py_CheckRecursionLimit Python/ceval.c:lltrace static int lltrace Python/ceval.c:make_pending_calls():busy static int busy Python/dynload_shlib.c:handles static struct { dev_t dev; ino_t ino; void *handle; } handles[128] diff --git a/Tools/c-analyzer/known.tsv b/Tools/c-analyzer/known.tsv index f92b45e5b4d85..f8c12a3944d9b 100644 --- a/Tools/c-analyzer/known.tsv +++ b/Tools/c-analyzer/known.tsv @@ -805,7 +805,6 @@ Objects/iterobject.c - PyCallIter_Type variable PyTypeObject PyCallIter_Type Objects/capsule.c - PyCapsule_Type variable PyTypeObject PyCapsule_Type Objects/cellobject.c - PyCell_Type variable PyTypeObject PyCell_Type Objects/methodobject.c - PyCFunction_Type variable PyTypeObject PyCFunction_Type -Python/ceval.c - _Py_CheckRecursionLimit variable int _Py_CheckRecursionLimit Objects/descrobject.c - PyClassMethodDescr_Type variable PyTypeObject PyClassMethodDescr_Type Objects/funcobject.c - PyClassMethod_Type variable PyTypeObject PyClassMethod_Type Objects/codeobject.c - PyCode_Type variable PyTypeObject PyCode_Type From webhook-mailer at python.org Wed Sep 23 08:05:37 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 23 Sep 2020 12:05:37 -0000 Subject: [Python-checkins] bpo-40521: Fix PyUnicode_InternInPlace() (GH-22376) Message-ID: https://github.com/python/cpython/commit/7f413a5d95e6d7ddddd6e2c9844c33594d6288f4 commit: 7f413a5d95e6d7ddddd6e2c9844c33594d6288f4 branch: master author: Victor Stinner committer: GitHub date: 2020-09-23T14:05:32+02:00 summary: bpo-40521: Fix PyUnicode_InternInPlace() (GH-22376) Fix PyUnicode_InternInPlace() when the INTERNED_STRINGS macro is not defined (when the EXPERIMENTAL_ISOLATED_SUBINTERPRETERS macro is defined). files: M Objects/unicodeobject.c diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index fd0e8e008adae..f32ab417c364c 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -15754,6 +15754,10 @@ PyUnicode_InternInPlace(PyObject **p) this. */ Py_SET_REFCNT(s, Py_REFCNT(s) - 2); _PyUnicode_STATE(s).interned = SSTATE_INTERNED_MORTAL; +#else + // PyDict expects that interned strings have their hash + // (PyASCIIObject.hash) already computed. + (void)unicode_hash(s); #endif } From webhook-mailer at python.org Wed Sep 23 08:06:59 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 23 Sep 2020 12:06:59 -0000 Subject: [Python-checkins] bpo-40941: Fix fold_tuple_on_constants() compiler warnings (GH-22378) Message-ID: https://github.com/python/cpython/commit/71f2ff4ccf4ff8bdb56cc30d115ca2ddc602b12f commit: 71f2ff4ccf4ff8bdb56cc30d115ca2ddc602b12f branch: master author: Victor Stinner committer: GitHub date: 2020-09-23T14:06:55+02:00 summary: bpo-40941: Fix fold_tuple_on_constants() compiler warnings (GH-22378) Add explicit casts to fix compiler warnings in fold_tuple_on_constants(). The limit of constants per code is now INT_MAX, rather than UINT_MAX. files: M Python/compile.c diff --git a/Python/compile.c b/Python/compile.c index 3ebf221cf02b7..0f9e5c276c7b7 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -6100,13 +6100,11 @@ fold_tuple_on_constants(struct instr *inst, PyTuple_SET_ITEM(newconst, i, constant); } Py_ssize_t index = PyList_GET_SIZE(consts); -#if SIZEOF_SIZE_T > SIZEOF_INT - if ((size_t)index >= UINT_MAX - 1) { + if ((size_t)index >= (size_t)INT_MAX - 1) { Py_DECREF(newconst); PyErr_SetString(PyExc_OverflowError, "too many constants"); return -1; } -#endif if (PyList_Append(consts, newconst)) { Py_DECREF(newconst); return -1; @@ -6116,7 +6114,7 @@ fold_tuple_on_constants(struct instr *inst, inst[i].i_opcode = NOP; } inst[n].i_opcode = LOAD_CONST; - inst[n].i_oparg = index; + inst[n].i_oparg = (int)index; return 0; } From webhook-mailer at python.org Wed Sep 23 08:07:21 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 23 Sep 2020 12:07:21 -0000 Subject: [Python-checkins] bpo-40941: Fix stackdepth compiler warnings (GH-22377) Message-ID: https://github.com/python/cpython/commit/b7d8d8dbfe83040087a63662e0b908f4b5ac24b0 commit: b7d8d8dbfe83040087a63662e0b908f4b5ac24b0 branch: master author: Victor Stinner committer: GitHub date: 2020-09-23T14:07:16+02:00 summary: bpo-40941: Fix stackdepth compiler warnings (GH-22377) Explicitly cast a difference of two pointers to int: PyFrameObject.f_stackdepth is an int. files: M Python/ceval.c diff --git a/Python/ceval.c b/Python/ceval.c index 4bb4b820b8e17..6430e792b8c5d 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -1433,9 +1433,9 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag) if (_Py_TracingPossible(ceval2) && tstate->c_tracefunc != NULL && !tstate->tracing) { int err; - /* see maybe_call_line_trace + /* see maybe_call_line_trace() for expository comments */ - f->f_stackdepth = stack_pointer-f->f_valuestack; + f->f_stackdepth = (int)(stack_pointer - f->f_valuestack); err = maybe_call_line_trace(tstate->c_tracefunc, tstate->c_traceobj, @@ -2265,7 +2265,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag) assert(f->f_lasti >= (int)sizeof(_Py_CODEUNIT)); f->f_lasti -= sizeof(_Py_CODEUNIT); f->f_state = FRAME_SUSPENDED; - f->f_stackdepth = stack_pointer-f->f_valuestack; + f->f_stackdepth = (int)(stack_pointer - f->f_valuestack); goto exiting; } @@ -2282,7 +2282,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag) retval = w; } f->f_state = FRAME_SUSPENDED; - f->f_stackdepth = stack_pointer-f->f_valuestack; + f->f_stackdepth = (int)(stack_pointer - f->f_valuestack); goto exiting; } From webhook-mailer at python.org Wed Sep 23 08:08:42 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 23 Sep 2020 12:08:42 -0000 Subject: [Python-checkins] bpo-40170: Use inline _PyType_HasFeature() function (GH-22375) Message-ID: https://github.com/python/cpython/commit/97d15ae1d8411b49b1fcdc0c67c51849dccce9c9 commit: 97d15ae1d8411b49b1fcdc0c67c51849dccce9c9 branch: master author: Victor Stinner committer: GitHub date: 2020-09-23T14:08:38+02:00 summary: bpo-40170: Use inline _PyType_HasFeature() function (GH-22375) Use _PyType_HasFeature() in the _io module and in structseq implementation. Replace PyType_HasFeature() opaque function call with _PyType_HasFeature() inlined function. files: M Modules/_io/iobase.c M Objects/structseq.c diff --git a/Modules/_io/iobase.c b/Modules/_io/iobase.c index a8e55c34799bd..195862df5dc06 100644 --- a/Modules/_io/iobase.c +++ b/Modules/_io/iobase.c @@ -349,8 +349,9 @@ iobase_dealloc(iobase *self) if (_PyIOBase_finalize((PyObject *) self) < 0) { /* When called from a heap type's dealloc, the type will be decref'ed on return (see e.g. subtype_dealloc in typeobject.c). */ - if (PyType_HasFeature(Py_TYPE(self), Py_TPFLAGS_HEAPTYPE)) + if (_PyType_HasFeature(Py_TYPE(self), Py_TPFLAGS_HEAPTYPE)) { Py_INCREF(Py_TYPE(self)); + } return; } _PyObject_GC_UNTRACK(self); diff --git a/Objects/structseq.c b/Objects/structseq.c index bd20ce3fbdcb9..8ae8f28cbc580 100644 --- a/Objects/structseq.c +++ b/Objects/structseq.c @@ -94,7 +94,7 @@ structseq_dealloc(PyStructSequence *obj) Py_XDECREF(obj->ob_item[i]); } PyObject_GC_Del(obj); - if (PyType_GetFlags(tp) & Py_TPFLAGS_HEAPTYPE) { + if (_PyType_HasFeature(tp, Py_TPFLAGS_HEAPTYPE)) { Py_DECREF(tp); } } From webhook-mailer at python.org Wed Sep 23 08:29:07 2020 From: webhook-mailer at python.org (sblondon) Date: Wed, 23 Sep 2020 12:29:07 -0000 Subject: [Python-checkins] bpo-37779 : Add information about the overriding behavior of ConfigParser.read (GH-15177) Message-ID: https://github.com/python/cpython/commit/48b0b1b121f26f811265f9eb06f195a3df38ef4b commit: 48b0b1b121f26f811265f9eb06f195a3df38ef4b branch: master author: sblondon committer: GitHub date: 2020-09-23T14:28:58+02:00 summary: bpo-37779 : Add information about the overriding behavior of ConfigParser.read (GH-15177) Co-Authored-By: Kyle Stanley Co-Authored-By: Paul Ganssle files: M Doc/library/configparser.rst diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst index 2e22a549ee281..646e8a317f52c 100644 --- a/Doc/library/configparser.rst +++ b/Doc/library/configparser.rst @@ -135,6 +135,30 @@ involves the ``DEFAULT`` section which provides default values for all other sections [1]_. Note also that keys in sections are case-insensitive and stored in lowercase [1]_. +It is possible to read several configurations into a single +:class:`ConfigParser`, where the most recently added configuration has the +highest priority. Any conflicting keys are taken from the more recent +configuration while the previously existing keys are retained. + +.. doctest:: + + >>> another_config = configparser.ConfigParser() + >>> another_config.read('example.ini') + ['example.ini'] + >>> another_config['topsecret.server.com']['Port'] + '50022' + >>> another_config.read_string("[topsecret.server.com]\nPort=48484") + >>> another_config['topsecret.server.com']['Port'] + '48484' + >>> another_config.read_dict({"topsecret.server.com": {"Port": 21212}}) + >>> another_config['topsecret.server.com']['Port'] + '21212' + >>> another_config['topsecret.server.com']['ForwardX11'] + 'no' + +This behaviour is equivalent to a :meth:`ConfigParser.read` call with several +files passed to the *filenames* parameter. + Supported Datatypes ------------------- From webhook-mailer at python.org Wed Sep 23 09:44:36 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Wed, 23 Sep 2020 13:44:36 -0000 Subject: [Python-checkins] bpo-41841: Prepare IDLE News for 3.10 (GH-22379) Message-ID: https://github.com/python/cpython/commit/7af072c3b60dd3aac2835122bc37454f0973351e commit: 7af072c3b60dd3aac2835122bc37454f0973351e branch: master author: Terry Jan Reedy committer: GitHub date: 2020-09-23T09:44:31-04:00 summary: bpo-41841: Prepare IDLE News for 3.10 (GH-22379) files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 7eea0a47aa6e8..38b98644942f9 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -1,5 +1,5 @@ -What's New in IDLE 3.9.0 (since 3.8.0) -Released on 2020-10-05? +What's New in IDLE 3.10.0 (since 3.9.0) +Released on 2021-10-04? ====================================== @@ -8,6 +8,11 @@ bpo-35764: Rewrite the Calltips doc section. bpo-40181: In calltips, stop reminding that '/' marks the end of positional-only arguments. + +What's New in IDLE 3.9.0 (since 3.8.0) +Released on 2020-10-05? +====================================== + bpo-41468: Improve IDLE run crash error message (which users should never see). From webhook-mailer at python.org Wed Sep 23 10:07:27 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Wed, 23 Sep 2020 14:07:27 -0000 Subject: [Python-checkins] [3.9] bpo-41841: Prepare IDLE News for 3.9 (GH-22380) Message-ID: https://github.com/python/cpython/commit/d7a4e5d330df49774019e25313e0c1c4e9bf505d commit: d7a4e5d330df49774019e25313e0c1c4e9bf505d branch: 3.9 author: Terry Jan Reedy committer: GitHub date: 2020-09-23T10:07:19-04:00 summary: [3.9] bpo-41841: Prepare IDLE News for 3.9 (GH-22380) files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 7eea0a47aa6e8..ed1619001d59e 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -1,5 +1,5 @@ -What's New in IDLE 3.9.0 (since 3.8.0) -Released on 2020-10-05? +What's New in IDLE 3.9.1 +Released on 2020-12-07? ====================================== @@ -8,6 +8,11 @@ bpo-35764: Rewrite the Calltips doc section. bpo-40181: In calltips, stop reminding that '/' marks the end of positional-only arguments. + +What's New in IDLE 3.9.0 (since 3.8.0) +Released on 2020-10-05? +====================================== + bpo-41468: Improve IDLE run crash error message (which users should never see). From webhook-mailer at python.org Wed Sep 23 10:35:06 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Wed, 23 Sep 2020 14:35:06 -0000 Subject: [Python-checkins] [3.8] bpo-41841: Prepare IDLE News for 3.8.7 (GH-22381) Message-ID: https://github.com/python/cpython/commit/0a2e11aecfb300615ab813f5d2d77a799f6e852c commit: 0a2e11aecfb300615ab813f5d2d77a799f6e852c branch: 3.8 author: Terry Jan Reedy committer: GitHub date: 2020-09-23T10:34:56-04:00 summary: [3.8] bpo-41841: Prepare IDLE News for 3.8.7 (GH-22381) files: M Lib/idlelib/NEWS.txt diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 5d9602e707b07..5d38a500fe618 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -1,5 +1,5 @@ -What's New in IDLE 3.8.6 -Released on 2020-09-14? +What's New in IDLE 3.8.7 +Released on 2020-12-?? ====================================== @@ -8,6 +8,11 @@ bpo-35764: Rewrite the Calltips doc section. bpo-40181: In calltips, stop reminding that '/' marks the end of positional-only arguments. + +What's New in IDLE 3.8.6 +Released on 2020-09-?? +====================================== + bpo-41468: Improve IDLE run crash error message (which users should never see). From webhook-mailer at python.org Wed Sep 23 11:17:36 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Wed, 23 Sep 2020 15:17:36 -0000 Subject: [Python-checkins] bpo-41844: Add IDLE section to What's New 3.9 (GN-22382) Message-ID: https://github.com/python/cpython/commit/22b7616f26694ac707fd53ece500febfad665426 commit: 22b7616f26694ac707fd53ece500febfad665426 branch: master author: Terry Jan Reedy committer: GitHub date: 2020-09-23T11:17:26-04:00 summary: bpo-41844: Add IDLE section to What's New 3.9 (GN-22382) files: M Doc/whatsnew/3.9.rst diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index e9fc496e47e24..95188b7493ad4 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -387,6 +387,20 @@ http HTTP status codes ``103 EARLY_HINTS``, ``418 IM_A_TEAPOT`` and ``425 TOO_EARLY`` are added to :class:`http.HTTPStatus`. (Contributed by Dong-hee Na in :issue:`39509` and Ross Rhodes in :issue:`39507`.) +IDLE and idlelib +---------------- + +Add option to toggle cursor blink off. (Contributed by Zackery Spytz +in :issue:`4603`.) + +Escape key now closes IDLE completion windows. (Contributed by Johnny +Najera in :issue:`38944`.) + +Add keywords to module name completion list. (Contributed by Terry J. +Reedy in :issue:`37765`.) + +The changes above have been backported to 3.8 maintenance releases. + imaplib ------- From webhook-mailer at python.org Wed Sep 23 11:19:13 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Wed, 23 Sep 2020 15:19:13 -0000 Subject: [Python-checkins] bpo-33822: Update IDLE section of What's New 3.8 (GH-22383) Message-ID: https://github.com/python/cpython/commit/9c4eac7f02ddcf32fc1cdaf7c08c37fe9718c1fb commit: 9c4eac7f02ddcf32fc1cdaf7c08c37fe9718c1fb branch: master author: Terry Jan Reedy committer: GitHub date: 2020-09-23T11:19:05-04:00 summary: bpo-33822: Update IDLE section of What's New 3.8 (GH-22383) files: M Doc/whatsnew/3.8.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index a2fa17811b3fc..6a9fa34156946 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -870,8 +870,18 @@ clipboard. Converting strings from Tcl to Python and back now never fails. (Many people worked on this for eight years but the problem was finally solved by Serhiy Storchaka in :issue:`13153`.) +New in 3.8.1: + +Add option to toggle cursor blink off. (Contributed by Zackery Spytz +in :issue:`4603`.) + +Escape key now closes IDLE completion windows. (Contributed by Johnny +Najera in :issue:`38944`.) + The changes above have been backported to 3.7 maintenance releases. +Add keywords to module name completion list. (Contributed by Terry J. +Reedy in :issue:`37765`.) inspect ------- From webhook-mailer at python.org Wed Sep 23 11:28:17 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 23 Sep 2020 15:28:17 -0000 Subject: [Python-checkins] bpo-33822: Update IDLE section of What's New 3.8 (GH-22383) Message-ID: https://github.com/python/cpython/commit/e678ca960b44ec3ddf2ac01fdb333018646ed8dd commit: e678ca960b44ec3ddf2ac01fdb333018646ed8dd branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-23T08:28:08-07:00 summary: bpo-33822: Update IDLE section of What's New 3.8 (GH-22383) (cherry picked from commit 9c4eac7f02ddcf32fc1cdaf7c08c37fe9718c1fb) Co-authored-by: Terry Jan Reedy files: M Doc/whatsnew/3.8.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 7a86e51116f41..ce386c0556cc2 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -872,8 +872,18 @@ clipboard. Converting strings from Tcl to Python and back now never fails. (Many people worked on this for eight years but the problem was finally solved by Serhiy Storchaka in :issue:`13153`.) +New in 3.8.1: + +Add option to toggle cursor blink off. (Contributed by Zackery Spytz +in :issue:`4603`.) + +Escape key now closes IDLE completion windows. (Contributed by Johnny +Najera in :issue:`38944`.) + The changes above have been backported to 3.7 maintenance releases. +Add keywords to module name completion list. (Contributed by Terry J. +Reedy in :issue:`37765`.) inspect ------- From webhook-mailer at python.org Wed Sep 23 11:38:00 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 23 Sep 2020 15:38:00 -0000 Subject: [Python-checkins] bpo-33822: Update IDLE section of What's New 3.8 (GH-22383) Message-ID: https://github.com/python/cpython/commit/e33fe6c3d793688d46edf7c2e31a88fc2e719d65 commit: e33fe6c3d793688d46edf7c2e31a88fc2e719d65 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-23T08:37:52-07:00 summary: bpo-33822: Update IDLE section of What's New 3.8 (GH-22383) (cherry picked from commit 9c4eac7f02ddcf32fc1cdaf7c08c37fe9718c1fb) Co-authored-by: Terry Jan Reedy files: M Doc/whatsnew/3.8.rst diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index a2fa17811b3fc..6a9fa34156946 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -870,8 +870,18 @@ clipboard. Converting strings from Tcl to Python and back now never fails. (Many people worked on this for eight years but the problem was finally solved by Serhiy Storchaka in :issue:`13153`.) +New in 3.8.1: + +Add option to toggle cursor blink off. (Contributed by Zackery Spytz +in :issue:`4603`.) + +Escape key now closes IDLE completion windows. (Contributed by Johnny +Najera in :issue:`38944`.) + The changes above have been backported to 3.7 maintenance releases. +Add keywords to module name completion list. (Contributed by Terry J. +Reedy in :issue:`37765`.) inspect ------- From webhook-mailer at python.org Wed Sep 23 11:51:59 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 23 Sep 2020 15:51:59 -0000 Subject: [Python-checkins] bpo-41844: Add IDLE section to What's New 3.9 (GN-22382) Message-ID: https://github.com/python/cpython/commit/57e7d5c19e934f450f5a3539eff48162a78af994 commit: 57e7d5c19e934f450f5a3539eff48162a78af994 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-23T11:51:50-04:00 summary: bpo-41844: Add IDLE section to What's New 3.9 (GN-22382) (cherry picked from commit 22b7616f26694ac707fd53ece500febfad665426) Co-authored-by: Terry Jan Reedy files: M Doc/whatsnew/3.9.rst diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index e9fc496e47e24..95188b7493ad4 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -387,6 +387,20 @@ http HTTP status codes ``103 EARLY_HINTS``, ``418 IM_A_TEAPOT`` and ``425 TOO_EARLY`` are added to :class:`http.HTTPStatus`. (Contributed by Dong-hee Na in :issue:`39509` and Ross Rhodes in :issue:`39507`.) +IDLE and idlelib +---------------- + +Add option to toggle cursor blink off. (Contributed by Zackery Spytz +in :issue:`4603`.) + +Escape key now closes IDLE completion windows. (Contributed by Johnny +Najera in :issue:`38944`.) + +Add keywords to module name completion list. (Contributed by Terry J. +Reedy in :issue:`37765`.) + +The changes above have been backported to 3.8 maintenance releases. + imaplib ------- From webhook-mailer at python.org Wed Sep 23 14:43:54 2020 From: webhook-mailer at python.org (Zackery Spytz) Date: Wed, 23 Sep 2020 18:43:54 -0000 Subject: [Python-checkins] bpo-30155: Add macros to get tzinfo from datetime instances (GH-21633) Message-ID: https://github.com/python/cpython/commit/2e4dd336e5b50fd30947fdecb605ddcd71f7f6f5 commit: 2e4dd336e5b50fd30947fdecb605ddcd71f7f6f5 branch: master author: Zackery Spytz committer: GitHub date: 2020-09-23T14:43:45-04:00 summary: bpo-30155: Add macros to get tzinfo from datetime instances (GH-21633) Add PyDateTime_DATE_GET_TZINFO() and PyDateTime_TIME_GET_TZINFO() macros. files: A Misc/NEWS.d/next/C API/2020-07-26-19-39-45.bpo-30155.rHZRJ_.rst M Doc/c-api/datetime.rst M Doc/whatsnew/3.10.rst M Include/datetime.h M Lib/test/datetimetester.py M Modules/_datetimemodule.c M Modules/_testcapimodule.c M Modules/_zoneinfo.c diff --git a/Doc/c-api/datetime.rst b/Doc/c-api/datetime.rst index bd4f1ff446bcf..415ce4cac67f1 100644 --- a/Doc/c-api/datetime.rst +++ b/Doc/c-api/datetime.rst @@ -185,6 +185,11 @@ must not be ``NULL``, and the type is not checked: Return the microsecond, as an int from 0 through 999999. +.. c:function:: PyObject* PyDateTime_DATE_GET_TZINFO(PyDateTime_DateTime *o) + + Return the tzinfo (which may be ``None``). + + .. versionadded:: 3.10 Macros to extract fields from time objects. The argument must be an instance of :c:data:`PyDateTime_Time`, including subclasses. The argument must not be ``NULL``, @@ -209,6 +214,12 @@ and the type is not checked: Return the microsecond, as an int from 0 through 999999. +.. c:function:: PyObject* PyDateTime_TIME_GET_TZINFO(PyDateTime_Time *o) + + Return the tzinfo (which may be ``None``). + + .. versionadded:: 3.10 + Macros to extract fields from time delta objects. The argument must be an instance of :c:data:`PyDateTime_Delta`, including subclasses. The argument must diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 606fad2fdcf05..1228f2695496c 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -231,6 +231,12 @@ New Features Python executable. (Contributed by Victor Stinner in :issue:`23427`.) +* The :c:func:`PyDateTime_DATE_GET_TZINFO` and + :c:func:`PyDateTime_TIME_GET_TZINFO` macros have been added for accessing + the ``tzinfo`` attributes of :class:`datetime.datetime` and + :class:`datetime.time` objects. + (Contributed by Zackery Spytz in :issue:`30155`.) + Porting to Python 3.10 ---------------------- diff --git a/Include/datetime.h b/Include/datetime.h index 5d9f2558f924d..bb565201a164d 100644 --- a/Include/datetime.h +++ b/Include/datetime.h @@ -115,6 +115,10 @@ typedef struct /* Apply for date and datetime instances. */ + +// o is a pointer to a time or a datetime object. +#define _PyDateTime_HAS_TZINFO(o) (((_PyDateTime_BaseTZInfo *)(o))->hastzinfo) + #define PyDateTime_GET_YEAR(o) ((((PyDateTime_Date*)o)->data[0] << 8) | \ ((PyDateTime_Date*)o)->data[1]) #define PyDateTime_GET_MONTH(o) (((PyDateTime_Date*)o)->data[2]) @@ -128,6 +132,8 @@ typedef struct (((PyDateTime_DateTime*)o)->data[8] << 8) | \ ((PyDateTime_DateTime*)o)->data[9]) #define PyDateTime_DATE_GET_FOLD(o) (((PyDateTime_DateTime*)o)->fold) +#define PyDateTime_DATE_GET_TZINFO(o) (_PyDateTime_HAS_TZINFO(o) ? \ + ((PyDateTime_DateTime *)(o))->tzinfo : Py_None) /* Apply for time instances. */ #define PyDateTime_TIME_GET_HOUR(o) (((PyDateTime_Time*)o)->data[0]) @@ -138,6 +144,8 @@ typedef struct (((PyDateTime_Time*)o)->data[4] << 8) | \ ((PyDateTime_Time*)o)->data[5]) #define PyDateTime_TIME_GET_FOLD(o) (((PyDateTime_Time*)o)->fold) +#define PyDateTime_TIME_GET_TZINFO(o) (_PyDateTime_HAS_TZINFO(o) ? \ + ((PyDateTime_Time *)(o))->tzinfo : Py_None) /* Apply for time delta instances */ #define PyDateTime_DELTA_GET_DAYS(o) (((PyDateTime_Delta*)o)->days) diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 520a51df87999..8b61c26f9e5c2 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -5991,30 +5991,36 @@ class DateTimeSubclass(datetime): for klass in [datetime, DateTimeSubclass]: for args in [(1993, 8, 26, 22, 12, 55, 99999), - (1993, 8, 26, 22, 12, 55, 99999)]: + (1993, 8, 26, 22, 12, 55, 99999, + timezone.utc)]: d = klass(*args) with self.subTest(cls=klass, date=args): - hour, minute, second, microsecond = _testcapi.PyDateTime_DATE_GET(d) + hour, minute, second, microsecond, tzinfo = \ + _testcapi.PyDateTime_DATE_GET(d) self.assertEqual(hour, d.hour) self.assertEqual(minute, d.minute) self.assertEqual(second, d.second) self.assertEqual(microsecond, d.microsecond) + self.assertIs(tzinfo, d.tzinfo) def test_PyDateTime_TIME_GET(self): class TimeSubclass(time): pass for klass in [time, TimeSubclass]: - for args in [(12, 30, 20, 10), (12, 30, 20, 10)]: + for args in [(12, 30, 20, 10), + (12, 30, 20, 10, timezone.utc)]: d = klass(*args) with self.subTest(cls=klass, date=args): - hour, minute, second, microsecond = _testcapi.PyDateTime_TIME_GET(d) + hour, minute, second, microsecond, tzinfo = \ + _testcapi.PyDateTime_TIME_GET(d) self.assertEqual(hour, d.hour) self.assertEqual(minute, d.minute) self.assertEqual(second, d.second) self.assertEqual(microsecond, d.microsecond) + self.assertIs(tzinfo, d.tzinfo) def test_timezones_offset_zero(self): utc0, utc1, non_utc = _testcapi.get_timezones_offset_zero() diff --git a/Misc/NEWS.d/next/C API/2020-07-26-19-39-45.bpo-30155.rHZRJ_.rst b/Misc/NEWS.d/next/C API/2020-07-26-19-39-45.bpo-30155.rHZRJ_.rst new file mode 100644 index 0000000000000..a276759da7933 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2020-07-26-19-39-45.bpo-30155.rHZRJ_.rst @@ -0,0 +1,3 @@ +Add :c:func:`PyDateTime_DATE_GET_TZINFO` and +:c:func:`PyDateTime_TIME_GET_TZINFO` macros for accessing the ``tzinfo`` +attributes of :class:`datetime.datetime` and :class:`datetime.time` objects. diff --git a/Modules/_datetimemodule.c b/Modules/_datetimemodule.c index 74a54e74ae0fe..0631272429f4f 100644 --- a/Modules/_datetimemodule.c +++ b/Modules/_datetimemodule.c @@ -115,14 +115,9 @@ class datetime.IsoCalendarDate "PyDateTime_IsoCalendarDate *" "&PyDateTime_IsoCa #define SET_TD_SECONDS(o, v) ((o)->seconds = (v)) #define SET_TD_MICROSECONDS(o, v) ((o)->microseconds = (v)) -/* p is a pointer to a time or a datetime object; HASTZINFO(p) returns - * p->hastzinfo. - */ -#define HASTZINFO(p) (((_PyDateTime_BaseTZInfo *)(p))->hastzinfo) -#define GET_TIME_TZINFO(p) (HASTZINFO(p) ? \ - ((PyDateTime_Time *)(p))->tzinfo : Py_None) -#define GET_DT_TZINFO(p) (HASTZINFO(p) ? \ - ((PyDateTime_DateTime *)(p))->tzinfo : Py_None) +#define HASTZINFO _PyDateTime_HAS_TZINFO +#define GET_TIME_TZINFO PyDateTime_TIME_GET_TZINFO +#define GET_DT_TZINFO PyDateTime_DATE_GET_TZINFO /* M is a char or int claiming to be a valid month. The macro is equivalent * to the two-sided Python test * 1 <= M <= 12 diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 7536d29535038..0e098779696b7 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -2677,8 +2677,9 @@ test_PyDateTime_DATE_GET(PyObject *self, PyObject *obj) minute = PyDateTime_DATE_GET_MINUTE(obj); second = PyDateTime_DATE_GET_SECOND(obj); microsecond = PyDateTime_DATE_GET_MICROSECOND(obj); + PyObject *tzinfo = PyDateTime_DATE_GET_TZINFO(obj); - return Py_BuildValue("(llll)", hour, minute, second, microsecond); + return Py_BuildValue("(llllO)", hour, minute, second, microsecond, tzinfo); } static PyObject * @@ -2690,8 +2691,9 @@ test_PyDateTime_TIME_GET(PyObject *self, PyObject *obj) minute = PyDateTime_TIME_GET_MINUTE(obj); second = PyDateTime_TIME_GET_SECOND(obj); microsecond = PyDateTime_TIME_GET_MICROSECOND(obj); + PyObject *tzinfo = PyDateTime_TIME_GET_TZINFO(obj); - return Py_BuildValue("(llll)", hour, minute, second, microsecond); + return Py_BuildValue("(llllO)", hour, minute, second, microsecond, tzinfo); } static PyObject * diff --git a/Modules/_zoneinfo.c b/Modules/_zoneinfo.c index 2cee65fac6dd0..bee59b8d2ae0c 100644 --- a/Modules/_zoneinfo.c +++ b/Modules/_zoneinfo.c @@ -484,9 +484,7 @@ zoneinfo_tzname(PyObject *self, PyObject *dt) return tti->tzname; } -#define HASTZINFO(p) (((_PyDateTime_BaseTZInfo *)(p))->hastzinfo) -#define GET_DT_TZINFO(p) \ - (HASTZINFO(p) ? ((PyDateTime_DateTime *)(p))->tzinfo : Py_None) +#define GET_DT_TZINFO PyDateTime_DATE_GET_TZINFO static PyObject * zoneinfo_fromutc(PyObject *obj_self, PyObject *dt) From webhook-mailer at python.org Wed Sep 23 17:21:30 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 23 Sep 2020 21:21:30 -0000 Subject: [Python-checkins] bpo-41833: threading.Thread now uses the target name (GH-22357) Message-ID: https://github.com/python/cpython/commit/98c16c991d6e70a48f4280a7cd464d807bdd9f2b commit: 98c16c991d6e70a48f4280a7cd464d807bdd9f2b branch: master author: Victor Stinner committer: GitHub date: 2020-09-23T23:21:19+02:00 summary: bpo-41833: threading.Thread now uses the target name (GH-22357) files: A Misc/NEWS.d/next/Library/2020-09-22-13-51-14.bpo-41833.6HVDjT.rst M Doc/library/threading.rst M Lib/test/test_threading.py M Lib/threading.py diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index 7fcf93d74610e..7eb12fe116bd2 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -264,8 +264,10 @@ since it is impossible to detect the termination of alien threads. *target* is the callable object to be invoked by the :meth:`run` method. Defaults to ``None``, meaning nothing is called. - *name* is the thread name. By default, a unique name is constructed of the - form "Thread-*N*" where *N* is a small decimal number. + *name* is the thread name. By default, a unique name is constructed + of the form "Thread-*N*" where *N* is a small decimal number, + or "Thread-*N* (target)" where "target" is ``target.__name__`` if the + *target* argument is specified. *args* is the argument tuple for the target invocation. Defaults to ``()``. @@ -280,6 +282,9 @@ since it is impossible to detect the termination of alien threads. base class constructor (``Thread.__init__()``) before doing anything else to the thread. + .. versionchanged:: 3.10 + Use the *target* name if *name* argument is omitted. + .. versionchanged:: 3.3 Added the *daemon* argument. diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index d02d3b346f9cc..2f0f3ae0946a5 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -20,6 +20,7 @@ import signal import textwrap +from unittest import mock from test import lock_tests from test import support @@ -86,6 +87,33 @@ def tearDown(self): class ThreadTests(BaseTestCase): + @cpython_only + def test_name(self): + def func(): pass + + thread = threading.Thread(name="myname1") + self.assertEqual(thread.name, "myname1") + + # Convert int name to str + thread = threading.Thread(name=123) + self.assertEqual(thread.name, "123") + + # target name is ignored if name is specified + thread = threading.Thread(target=func, name="myname2") + self.assertEqual(thread.name, "myname2") + + with mock.patch.object(threading, '_counter', return_value=2): + thread = threading.Thread(name="") + self.assertEqual(thread.name, "Thread-2") + + with mock.patch.object(threading, '_counter', return_value=3): + thread = threading.Thread() + self.assertEqual(thread.name, "Thread-3") + + with mock.patch.object(threading, '_counter', return_value=5): + thread = threading.Thread(target=func) + self.assertEqual(thread.name, "Thread-5 (func)") + # Create a bunch of threads, let each do some work, wait until all are # done. def test_various_ops(self): @@ -531,7 +559,7 @@ def test_main_thread_after_fork_from_nonmain_thread(self): import os, threading, sys from test import support - def f(): + def func(): pid = os.fork() if pid == 0: main = threading.main_thread() @@ -544,14 +572,14 @@ def f(): else: support.wait_process(pid, exitcode=0) - th = threading.Thread(target=f) + th = threading.Thread(target=func) th.start() th.join() """ _, out, err = assert_python_ok("-c", code) data = out.decode().replace('\r', '') self.assertEqual(err, b"") - self.assertEqual(data, "Thread-1\nTrue\nTrue\n") + self.assertEqual(data, "Thread-1 (func)\nTrue\nTrue\n") def test_main_thread_during_shutdown(self): # bpo-31516: current_thread() should still point to the main thread diff --git a/Lib/threading.py b/Lib/threading.py index ab29db77a747a..06c77f70fe74f 100644 --- a/Lib/threading.py +++ b/Lib/threading.py @@ -745,10 +745,9 @@ class BrokenBarrierError(RuntimeError): # Helper to generate new thread names -_counter = _count().__next__ -_counter() # Consume 0 so first non-main thread has id 1. -def _newname(template="Thread-%d"): - return template % _counter() +_counter = _count(1).__next__ +def _newname(name_template): + return name_template % _counter() # Active thread administration _active_limbo_lock = _allocate_lock() @@ -800,8 +799,19 @@ class is implemented. assert group is None, "group argument must be None for now" if kwargs is None: kwargs = {} + if name: + name = str(name) + else: + name = _newname("Thread-%d") + if target is not None: + try: + target_name = target.__name__ + name += f" ({target_name})" + except AttributeError: + pass + self._target = target - self._name = str(name or _newname()) + self._name = name self._args = args self._kwargs = kwargs if daemon is not None: diff --git a/Misc/NEWS.d/next/Library/2020-09-22-13-51-14.bpo-41833.6HVDjT.rst b/Misc/NEWS.d/next/Library/2020-09-22-13-51-14.bpo-41833.6HVDjT.rst new file mode 100644 index 0000000000000..abb3a077d91b8 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-22-13-51-14.bpo-41833.6HVDjT.rst @@ -0,0 +1,2 @@ +The :class:`threading.Thread` constructor now uses the target name if the +*target* argument is specified but the *name* argument is omitted. From webhook-mailer at python.org Wed Sep 23 17:25:46 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 23 Sep 2020 21:25:46 -0000 Subject: [Python-checkins] bpo-41654: Fix compiler warning in MemoryError_dealloc() (GH-22387) Message-ID: https://github.com/python/cpython/commit/bbeb223e9a5e9f9374df384efa386b4068a65c0e commit: bbeb223e9a5e9f9374df384efa386b4068a65c0e branch: master author: Victor Stinner committer: GitHub date: 2020-09-23T23:25:40+02:00 summary: bpo-41654: Fix compiler warning in MemoryError_dealloc() (GH-22387) Fix warning: Objects\exceptions.c(2324,56): warning C4098: 'MemoryError_dealloc': 'void' function returning a value files: M Objects/exceptions.c diff --git a/Objects/exceptions.c b/Objects/exceptions.c index b08cbdd6aed35..b14da20db0c4e 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -2321,7 +2321,8 @@ MemoryError_dealloc(PyBaseExceptionObject *self) /* If this is a subclass of MemoryError, we don't need to * do anything in the free-list*/ if (!Py_IS_TYPE(self, (PyTypeObject *) PyExc_MemoryError)) { - return Py_TYPE(self)->tp_free((PyObject *)self); + Py_TYPE(self)->tp_free((PyObject *)self); + return; } _PyObject_GC_UNTRACK(self); From webhook-mailer at python.org Wed Sep 23 17:25:58 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Wed, 23 Sep 2020 21:25:58 -0000 Subject: [Python-checkins] bpo-41428: Fix compiler warnings in unionobject.c (GH-22388) Message-ID: https://github.com/python/cpython/commit/d67de0a30d76c6a28056bae22fd7d13f2e111b77 commit: d67de0a30d76c6a28056bae22fd7d13f2e111b77 branch: master author: Victor Stinner committer: GitHub date: 2020-09-23T23:25:54+02:00 summary: bpo-41428: Fix compiler warnings in unionobject.c (GH-22388) Use Py_ssize_t type rather than int, to store lengths in unionobject.c. Fix warnings: Objects\unionobject.c(189,71): warning C4244: '+=': conversion from 'Py_ssize_t' to 'int', possible loss of data Objects\unionobject.c(182,1): warning C4244: 'initializing': conversion from 'Py_ssize_t' to 'int', possible loss of data Objects\unionobject.c(205,1): warning C4244: 'initializing': conversion from 'Py_ssize_t' to 'int', possible loss of data Objects\unionobject.c(437,1): warning C4244: 'initializing': conversion from 'Py_ssize_t' to 'int', possible loss of data files: M Objects/unionobject.c diff --git a/Objects/unionobject.c b/Objects/unionobject.c index 0ef7abb4c55c2..e055a55e91715 100644 --- a/Objects/unionobject.c +++ b/Objects/unionobject.c @@ -179,8 +179,8 @@ union_richcompare(PyObject *a, PyObject *b, int op) static PyObject* flatten_args(PyObject* args) { - int arg_length = PyTuple_GET_SIZE(args); - int total_args = 0; + Py_ssize_t arg_length = PyTuple_GET_SIZE(args); + Py_ssize_t total_args = 0; // Get number of total args once it's flattened. for (Py_ssize_t i = 0; i < arg_length; i++) { PyObject *arg = PyTuple_GET_ITEM(args, i); @@ -434,7 +434,7 @@ _Py_Union(PyObject *args) unionobject* result = NULL; // Check arguments are unionable. - int nargs = PyTuple_GET_SIZE(args); + Py_ssize_t nargs = PyTuple_GET_SIZE(args); for (Py_ssize_t iarg = 0; iarg < nargs; iarg++) { PyObject *arg = PyTuple_GET_ITEM(args, iarg); if (arg == NULL) { From webhook-mailer at python.org Thu Sep 24 06:40:05 2020 From: webhook-mailer at python.org (=?utf-8?q?=C5=81ukasz?= Langa) Date: Thu, 24 Sep 2020 10:40:05 -0000 Subject: [Python-checkins] Python 3.8.6 Message-ID: https://github.com/python/cpython/commit/db455296be5f792b8c12b7cd7f3962b52e4f44ee commit: db455296be5f792b8c12b7cd7f3962b52e4f44ee branch: 3.8 author: ?ukasz Langa committer: ?ukasz Langa date: 2020-09-23T14:36:32+02:00 summary: Python 3.8.6 files: A Misc/NEWS.d/3.8.6.rst D Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst D Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst D Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst D Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst D Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst D Misc/NEWS.d/next/Library/2019-07-02-12-43-57.bpo-37479.O53a5S.rst D Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst D Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst D Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst D Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst D Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst D Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst D Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst D Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst D Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst D Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst M Include/patchlevel.h M Lib/pydoc_data/topics.py M README.rst diff --git a/Include/patchlevel.h b/Include/patchlevel.h index 6a89ef436b0aa..5123b2601b252 100644 --- a/Include/patchlevel.h +++ b/Include/patchlevel.h @@ -19,11 +19,11 @@ #define PY_MAJOR_VERSION 3 #define PY_MINOR_VERSION 8 #define PY_MICRO_VERSION 6 -#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA -#define PY_RELEASE_SERIAL 1 +#define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL +#define PY_RELEASE_SERIAL 0 /* Version as a string */ -#define PY_VERSION "3.8.6rc1+" +#define PY_VERSION "3.8.6" /*--end constants--*/ /* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2. diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py index ab53a563a43bd..d760056197ac1 100644 --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Autogenerated by Sphinx on Mon Sep 7 17:49:24 2020 +# Autogenerated by Sphinx on Wed Sep 23 14:35:51 2020 topics = {'assert': 'The "assert" statement\n' '**********************\n' '\n' @@ -6923,6 +6923,14 @@ 'the data\n' ' model.\n' '\n' + ' Note: Due to a bug in the dispatching mechanism for ' + '"**=", a\n' + ' class that defines "__ipow__()" but returns ' + '"NotImplemented"\n' + ' would fail to fall back to "x.__pow__(y)" and ' + '"y.__rpow__(x)".\n' + ' This bug is fixed in Python 3.10.\n' + '\n' 'object.__neg__(self)\n' 'object.__pos__(self)\n' 'object.__abs__(self)\n' @@ -9603,6 +9611,14 @@ 'the data\n' ' model.\n' '\n' + ' Note: Due to a bug in the dispatching mechanism for ' + '"**=", a\n' + ' class that defines "__ipow__()" but returns ' + '"NotImplemented"\n' + ' would fail to fall back to "x.__pow__(y)" and ' + '"y.__rpow__(x)".\n' + ' This bug is fixed in Python 3.10.\n' + '\n' 'object.__neg__(self)\n' 'object.__pos__(self)\n' 'object.__abs__(self)\n' diff --git a/Misc/NEWS.d/3.8.6.rst b/Misc/NEWS.d/3.8.6.rst new file mode 100644 index 0000000000000..641ca33631342 --- /dev/null +++ b/Misc/NEWS.d/3.8.6.rst @@ -0,0 +1,153 @@ +.. bpo: 41525 +.. date: 2020-08-12-07-35-07 +.. nonce: d9q3XL +.. release date: 2020-09-23 +.. section: Core and Builtins + +The output of ``python --help`` contains now only ASCII characters. + +.. + +.. bpo: 41817 +.. date: 2020-09-22-00-23-30 +.. nonce: bnh-VG +.. section: Library + +fix `tkinter.EventType` Enum so all members are strings, and none are tuples + +.. + +.. bpo: 41815 +.. date: 2020-09-19-23-14-54 +.. nonce: RNpuX3 +.. section: Library + +Fix SQLite3 segfault when backing up closed database. Patch contributed by +Peter David McCormick. + +.. + +.. bpo: 41517 +.. date: 2020-09-15-22-43-30 +.. nonce: sLBH7g +.. section: Library + +fix bug allowing Enums to be extended via multiple inheritance + +.. + +.. bpo: 39587 +.. date: 2020-09-15-14-56-13 +.. nonce: 69xzuh +.. section: Library + +use the correct mix-in data type when constructing Enums + +.. + +.. bpo: 41789 +.. date: 2020-09-14-19-27-46 +.. nonce: pI_uZQ +.. section: Library + +Honor `object` overrides in `Enum` class creation (specifically, `__str__`, +`__repr__`, `__format__`, and `__reduce_ex__`). + +.. + +.. bpo: 39651 +.. date: 2020-09-11-12-38-55 +.. nonce: JMp9l2 +.. section: Library + +Fix a race condition in the ``call_soon_threadsafe()`` method of +``asyncio.ProactorEventLoop``: do nothing if the self-pipe socket has been +closed. + +.. + +.. bpo: 41720 +.. date: 2020-09-04-20-45-38 +.. nonce: PW9MzZ +.. section: Library + +Fixed :meth:`turtle.Vec2D.__rmul__` for arguments which are not int or +float. + +.. + +.. bpo: 39728 +.. date: 2020-02-24-10-58-34 +.. nonce: kOOaHn +.. section: Library + +fix default `_missing_` so a duplicate `ValueError` is not set as the +`__context__` of the original `ValueError` + +.. + +.. bpo: 37479 +.. date: 2019-07-02-12-43-57 +.. nonce: O53a5S +.. section: Library + +When `Enum.__str__` is overridden in a derived class, the override will be +used by `Enum.__format__` regardless of whether mixin classes are present. + +.. + +.. bpo: 35293 +.. date: 2020-09-12-17-37-13 +.. nonce: _cOwPD +.. section: Documentation + +Fix RemovedInSphinx40Warning when building the documentation. Patch by +Dong-hee Na. + +.. + +.. bpo: 37149 +.. date: 2020-09-10-07-48-02 +.. nonce: VD0rCv +.. section: Documentation + +Change Shipman tkinter doc link from archive.org to TkDocs. (The doc has +been removed from the NMT server.) The new link responds much faster and +includes a short explanatory note. + +.. + +.. bpo: 41731 +.. date: 2020-09-11-19-12-31 +.. nonce: Ivxh4U +.. section: Tests + +Make test_cmd_line_script pass with option '-vv'. + +.. + +.. bpo: 41744 +.. date: 2020-09-11-17-59-33 +.. nonce: e_ugDQ +.. section: Windows + +Fixes automatic import of props file when using the Nuget package. + +.. + +.. bpo: 35764 +.. date: 2020-09-22-11-13-45 +.. nonce: VoNa8y +.. section: IDLE + +Rewrite the Calltips doc section. + +.. + +.. bpo: 40181 +.. date: 2020-09-22-00-45-40 +.. nonce: hhQi3z +.. section: IDLE + +In calltips, stop reminding that '/' marks the end of positional-only +arguments. diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst deleted file mode 100644 index acc00f8b992c9..0000000000000 --- a/Misc/NEWS.d/next/Core and Builtins/2020-08-12-07-35-07.bpo-41525.d9q3XL.rst +++ /dev/null @@ -1 +0,0 @@ -The output of ``python --help`` contains now only ASCII characters. diff --git a/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst b/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst deleted file mode 100644 index aeca652b4ed97..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-09-10-07-48-02.bpo-37149.VD0rCv.rst +++ /dev/null @@ -1 +0,0 @@ -Change Shipman tkinter doc link from archive.org to TkDocs. (The doc has been removed from the NMT server.) The new link responds much faster and includes a short explanatory note. diff --git a/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst b/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst deleted file mode 100644 index 089d44e35d2ba..0000000000000 --- a/Misc/NEWS.d/next/Documentation/2020-09-12-17-37-13.bpo-35293._cOwPD.rst +++ /dev/null @@ -1 +0,0 @@ -Fix RemovedInSphinx40Warning when building the documentation. Patch by Dong-hee Na. diff --git a/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst b/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst deleted file mode 100644 index b6866e19c4d41..0000000000000 --- a/Misc/NEWS.d/next/IDLE/2020-09-22-00-45-40.bpo-40181.hhQi3z.rst +++ /dev/null @@ -1,2 +0,0 @@ -In calltips, stop reminding that '/' marks the end of positional-only -arguments. diff --git a/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst b/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst deleted file mode 100644 index eb62d3699d5fe..0000000000000 --- a/Misc/NEWS.d/next/IDLE/2020-09-22-11-13-45.bpo-35764.VoNa8y.rst +++ /dev/null @@ -1 +0,0 @@ -Rewrite the Calltips doc section. diff --git a/Misc/NEWS.d/next/Library/2019-07-02-12-43-57.bpo-37479.O53a5S.rst b/Misc/NEWS.d/next/Library/2019-07-02-12-43-57.bpo-37479.O53a5S.rst deleted file mode 100644 index cf23155c33958..0000000000000 --- a/Misc/NEWS.d/next/Library/2019-07-02-12-43-57.bpo-37479.O53a5S.rst +++ /dev/null @@ -1,2 +0,0 @@ -When `Enum.__str__` is overridden in a derived class, the override will be -used by `Enum.__format__` regardless of whether mixin classes are present. \ No newline at end of file diff --git a/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst b/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst deleted file mode 100644 index beb2016a85ba6..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-02-24-10-58-34.bpo-39728.kOOaHn.rst +++ /dev/null @@ -1 +0,0 @@ -fix default `_missing_` so a duplicate `ValueError` is not set as the `__context__` of the original `ValueError` diff --git a/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst b/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst deleted file mode 100644 index 5d2a5094ddeaa..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-04-20-45-38.bpo-41720.PW9MzZ.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fixed :meth:`turtle.Vec2D.__rmul__` for arguments which are not int or -float. diff --git a/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst b/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst deleted file mode 100644 index 78dcff1370029..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-11-12-38-55.bpo-39651.JMp9l2.rst +++ /dev/null @@ -1,3 +0,0 @@ -Fix a race condition in the ``call_soon_threadsafe()`` method of -``asyncio.ProactorEventLoop``: do nothing if the self-pipe socket has been -closed. diff --git a/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst b/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst deleted file mode 100644 index 5ce7a3ca67b72..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-14-19-27-46.bpo-41789.pI_uZQ.rst +++ /dev/null @@ -1,2 +0,0 @@ -Honor `object` overrides in `Enum` class creation (specifically, `__str__`, -`__repr__`, `__format__`, and `__reduce_ex__`). diff --git a/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst b/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst deleted file mode 100644 index e2f2b64867bed..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-15-14-56-13.bpo-39587.69xzuh.rst +++ /dev/null @@ -1 +0,0 @@ -use the correct mix-in data type when constructing Enums diff --git a/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst b/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst deleted file mode 100644 index e7654711062ce..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-15-22-43-30.bpo-41517.sLBH7g.rst +++ /dev/null @@ -1 +0,0 @@ -fix bug allowing Enums to be extended via multiple inheritance diff --git a/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst b/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst deleted file mode 100644 index 3560db9bc5d35..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-19-23-14-54.bpo-41815.RNpuX3.rst +++ /dev/null @@ -1,2 +0,0 @@ -Fix SQLite3 segfault when backing up closed database. Patch contributed by -Peter David McCormick. diff --git a/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst b/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst deleted file mode 100644 index 6a634bb613260..0000000000000 --- a/Misc/NEWS.d/next/Library/2020-09-22-00-23-30.bpo-41817.bnh-VG.rst +++ /dev/null @@ -1 +0,0 @@ -fix `tkinter.EventType` Enum so all members are strings, and none are tuples diff --git a/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst b/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst deleted file mode 100644 index e368a60f77b1e..0000000000000 --- a/Misc/NEWS.d/next/Tests/2020-09-11-19-12-31.bpo-41731.Ivxh4U.rst +++ /dev/null @@ -1 +0,0 @@ -Make test_cmd_line_script pass with option '-vv'. diff --git a/Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst b/Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst deleted file mode 100644 index 6106d6604c7dd..0000000000000 --- a/Misc/NEWS.d/next/Windows/2020-09-11-17-59-33.bpo-41744.e_ugDQ.rst +++ /dev/null @@ -1 +0,0 @@ -Fixes automatic import of props file when using the Nuget package. \ No newline at end of file diff --git a/README.rst b/README.rst index f2b1e36aef1f6..97a7191ca6af9 100644 --- a/README.rst +++ b/README.rst @@ -1,5 +1,5 @@ -This is Python version 3.8.6rc1 -=============================== +This is Python version 3.8.6 +============================ .. image:: https://travis-ci.org/python/cpython.svg?branch=3.8 :alt: CPython build status on Travis CI From webhook-mailer at python.org Thu Sep 24 10:34:44 2020 From: webhook-mailer at python.org (=?utf-8?q?=C5=81ukasz?= Langa) Date: Thu, 24 Sep 2020 14:34:44 -0000 Subject: [Python-checkins] [3.9] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (#22397) Message-ID: https://github.com/python/cpython/commit/ca8d46dd422e5aa10f444796d93faec5a8cbc1e0 commit: ca8d46dd422e5aa10f444796d93faec5a8cbc1e0 branch: 3.9 author: ?ukasz Langa committer: GitHub date: 2020-09-24T16:34:21+02:00 summary: [3.9] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (#22397) Closes bpo issue 41602. (cherry picked from commit a68a2ad19c891faa891904b3da537911cc77df21) Co-authored-by: Thomas Grainger files: A Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst M Lib/test/test_runpy.py M Modules/main.c diff --git a/Lib/test/test_runpy.py b/Lib/test/test_runpy.py index 84834d39b1c21..4aaf75b1bceb6 100644 --- a/Lib/test/test_runpy.py +++ b/Lib/test/test_runpy.py @@ -1,14 +1,17 @@ # Test the runpy module -import unittest -import os +import contextlib +import importlib.machinery, importlib.util import os.path -import sys +import pathlib +import py_compile import re +import signal +import subprocess +import sys import tempfile -import importlib, importlib.machinery, importlib.util -import py_compile +import textwrap +import unittest import warnings -import pathlib from test.support import ( forget, make_legacy_pyc, unload, verbose, no_tracing, create_empty_file, temp_dir) @@ -752,5 +755,82 @@ def test_encoding(self): self.assertEqual(result['s'], "non-ASCII: h\xe9") +class TestExit(unittest.TestCase): + STATUS_CONTROL_C_EXIT = 0xC000013A + EXPECTED_CODE = ( + STATUS_CONTROL_C_EXIT + if sys.platform == "win32" + else -signal.SIGINT + ) + @staticmethod + @contextlib.contextmanager + def tmp_path(*args, **kwargs): + with temp_dir() as tmp_fn: + yield pathlib.Path(tmp_fn) + + + def run(self, *args, **kwargs): + with self.tmp_path() as tmp: + self.ham = ham = tmp / "ham.py" + ham.write_text( + textwrap.dedent( + """\ + raise KeyboardInterrupt + """ + ) + ) + super().run(*args, **kwargs) + + def assertSigInt(self, *args, **kwargs): + proc = subprocess.run(*args, **kwargs, text=True, stderr=subprocess.PIPE) + self.assertTrue(proc.stderr.endswith("\nKeyboardInterrupt\n")) + self.assertEqual(proc.returncode, self.EXPECTED_CODE) + + def test_pymain_run_file(self): + self.assertSigInt([sys.executable, self.ham]) + + def test_pymain_run_file_runpy_run_module(self): + tmp = self.ham.parent + run_module = tmp / "run_module.py" + run_module.write_text( + textwrap.dedent( + """\ + import runpy + runpy.run_module("ham") + """ + ) + ) + self.assertSigInt([sys.executable, run_module], cwd=tmp) + + def test_pymain_run_file_runpy_run_module_as_main(self): + tmp = self.ham.parent + run_module_as_main = tmp / "run_module_as_main.py" + run_module_as_main.write_text( + textwrap.dedent( + """\ + import runpy + runpy._run_module_as_main("ham") + """ + ) + ) + self.assertSigInt([sys.executable, run_module_as_main], cwd=tmp) + + def test_pymain_run_command_run_module(self): + self.assertSigInt( + [sys.executable, "-c", "import runpy; runpy.run_module('ham')"], + cwd=self.ham.parent, + ) + + def test_pymain_run_command(self): + self.assertSigInt([sys.executable, "-c", "import ham"], cwd=self.ham.parent) + + def test_pymain_run_stdin(self): + self.assertSigInt([sys.executable], input="import ham", cwd=self.ham.parent) + + def test_pymain_run_module(self): + ham = self.ham + self.assertSigInt([sys.executable, "-m", ham.stem], cwd=ham.parent) + + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst b/Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst new file mode 100644 index 0000000000000..fa3d2f1aa374e --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst @@ -0,0 +1 @@ +Add tests for SIGINT handling in the runpy module. diff --git a/Modules/main.c b/Modules/main.c index 4a76f4461bf61..2cc891f61aadd 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -287,7 +287,11 @@ pymain_run_module(const wchar_t *modname, int set_argv0) Py_DECREF(module); return pymain_exit_err_print(); } + _Py_UnhandledKeyboardInterrupt = 0; result = PyObject_Call(runmodule, runargs, NULL); + if (!result && PyErr_Occurred() == PyExc_KeyboardInterrupt) { + _Py_UnhandledKeyboardInterrupt = 1; + } Py_DECREF(runpy); Py_DECREF(runmodule); Py_DECREF(module); From webhook-mailer at python.org Thu Sep 24 12:44:23 2020 From: webhook-mailer at python.org (Thomas Grainger) Date: Thu, 24 Sep 2020 16:44:23 -0000 Subject: [Python-checkins] [3.8] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (#22398) Message-ID: https://github.com/python/cpython/commit/ae462297c08515f2c0dacf77b04816504142ba3e commit: ae462297c08515f2c0dacf77b04816504142ba3e branch: 3.8 author: Thomas Grainger committer: GitHub date: 2020-09-24T18:44:14+02:00 summary: [3.8] bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (GH-21956) (#22398) Closes bpo issue 41602. (cherry picked from commit a68a2ad19c891faa891904b3da537911cc77df21) Co-authored-by: Thomas Grainger files: A Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst M Lib/test/test_runpy.py M Modules/main.c diff --git a/Lib/test/test_runpy.py b/Lib/test/test_runpy.py index 84834d39b1c21..4aaf75b1bceb6 100644 --- a/Lib/test/test_runpy.py +++ b/Lib/test/test_runpy.py @@ -1,14 +1,17 @@ # Test the runpy module -import unittest -import os +import contextlib +import importlib.machinery, importlib.util import os.path -import sys +import pathlib +import py_compile import re +import signal +import subprocess +import sys import tempfile -import importlib, importlib.machinery, importlib.util -import py_compile +import textwrap +import unittest import warnings -import pathlib from test.support import ( forget, make_legacy_pyc, unload, verbose, no_tracing, create_empty_file, temp_dir) @@ -752,5 +755,82 @@ def test_encoding(self): self.assertEqual(result['s'], "non-ASCII: h\xe9") +class TestExit(unittest.TestCase): + STATUS_CONTROL_C_EXIT = 0xC000013A + EXPECTED_CODE = ( + STATUS_CONTROL_C_EXIT + if sys.platform == "win32" + else -signal.SIGINT + ) + @staticmethod + @contextlib.contextmanager + def tmp_path(*args, **kwargs): + with temp_dir() as tmp_fn: + yield pathlib.Path(tmp_fn) + + + def run(self, *args, **kwargs): + with self.tmp_path() as tmp: + self.ham = ham = tmp / "ham.py" + ham.write_text( + textwrap.dedent( + """\ + raise KeyboardInterrupt + """ + ) + ) + super().run(*args, **kwargs) + + def assertSigInt(self, *args, **kwargs): + proc = subprocess.run(*args, **kwargs, text=True, stderr=subprocess.PIPE) + self.assertTrue(proc.stderr.endswith("\nKeyboardInterrupt\n")) + self.assertEqual(proc.returncode, self.EXPECTED_CODE) + + def test_pymain_run_file(self): + self.assertSigInt([sys.executable, self.ham]) + + def test_pymain_run_file_runpy_run_module(self): + tmp = self.ham.parent + run_module = tmp / "run_module.py" + run_module.write_text( + textwrap.dedent( + """\ + import runpy + runpy.run_module("ham") + """ + ) + ) + self.assertSigInt([sys.executable, run_module], cwd=tmp) + + def test_pymain_run_file_runpy_run_module_as_main(self): + tmp = self.ham.parent + run_module_as_main = tmp / "run_module_as_main.py" + run_module_as_main.write_text( + textwrap.dedent( + """\ + import runpy + runpy._run_module_as_main("ham") + """ + ) + ) + self.assertSigInt([sys.executable, run_module_as_main], cwd=tmp) + + def test_pymain_run_command_run_module(self): + self.assertSigInt( + [sys.executable, "-c", "import runpy; runpy.run_module('ham')"], + cwd=self.ham.parent, + ) + + def test_pymain_run_command(self): + self.assertSigInt([sys.executable, "-c", "import ham"], cwd=self.ham.parent) + + def test_pymain_run_stdin(self): + self.assertSigInt([sys.executable], input="import ham", cwd=self.ham.parent) + + def test_pymain_run_module(self): + ham = self.ham + self.assertSigInt([sys.executable, "-m", ham.stem], cwd=ham.parent) + + if __name__ == "__main__": unittest.main() diff --git a/Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst b/Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst new file mode 100644 index 0000000000000..fa3d2f1aa374e --- /dev/null +++ b/Misc/NEWS.d/next/Tests/2020-08-25-19-25-36.bpo-41602.Z64s0I.rst @@ -0,0 +1 @@ +Add tests for SIGINT handling in the runpy module. diff --git a/Modules/main.c b/Modules/main.c index 70be4cfacbf82..ea6250e28c1b8 100644 --- a/Modules/main.c +++ b/Modules/main.c @@ -299,7 +299,11 @@ pymain_run_module(const wchar_t *modname, int set_argv0) Py_DECREF(module); return pymain_exit_err_print(); } + _Py_UnhandledKeyboardInterrupt = 0; result = PyObject_Call(runmodule, runargs, NULL); + if (!result && PyErr_Occurred() == PyExc_KeyboardInterrupt) { + _Py_UnhandledKeyboardInterrupt = 1; + } Py_DECREF(runpy); Py_DECREF(runmodule); Py_DECREF(module); From webhook-mailer at python.org Thu Sep 24 15:30:32 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Thu, 24 Sep 2020 19:30:32 -0000 Subject: [Python-checkins] bpo-41775: Make 'IDLE Shell' the shell title (#22399) Message-ID: https://github.com/python/cpython/commit/05cc881cbcf5d73a312568c78c7149eae3195072 commit: 05cc881cbcf5d73a312568c78c7149eae3195072 branch: master author: Terry Jan Reedy committer: GitHub date: 2020-09-24T15:30:09-04:00 summary: bpo-41775: Make 'IDLE Shell' the shell title (#22399) 'Python Shell' may have contributed to some beginners confusing 'IDLE' with ' Python'. files: A Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/pyshell.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 38b98644942f9..754034200a1f6 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,8 @@ Released on 2021-10-04? ====================================== +bpo-41775: Make 'IDLE Shell' the shell title. + bpo-35764: Rewrite the Calltips doc section. bpo-40181: In calltips, stop reminding that '/' marks the end of diff --git a/Lib/idlelib/pyshell.py b/Lib/idlelib/pyshell.py index 66ae0f7435dab..b69916dbe876c 100755 --- a/Lib/idlelib/pyshell.py +++ b/Lib/idlelib/pyshell.py @@ -833,7 +833,7 @@ def display_executing_dialog(self): class PyShell(OutputWindow): - shell_title = "Python " + python_version() + " Shell" + shell_title = "IDLE Shell " + python_version() # Override classes ColorDelegator = ModifiedColorDelegator diff --git a/Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst b/Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst new file mode 100644 index 0000000000000..59605fa40235f --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst @@ -0,0 +1 @@ +Use 'IDLE Shell' as shell title From webhook-mailer at python.org Thu Sep 24 16:07:57 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 24 Sep 2020 20:07:57 -0000 Subject: [Python-checkins] bpo-41775: Make 'IDLE Shell' the shell title (GH-22399) Message-ID: https://github.com/python/cpython/commit/4080f12ccccb17cee36732758912dc4346ec9361 commit: 4080f12ccccb17cee36732758912dc4346ec9361 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-24T13:07:49-07:00 summary: bpo-41775: Make 'IDLE Shell' the shell title (GH-22399) 'Python Shell' may have contributed to some beginners confusing 'IDLE' with ' Python'. (cherry picked from commit 05cc881cbcf5d73a312568c78c7149eae3195072) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/pyshell.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index ed1619001d59e..5ea33226cc0c3 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,8 @@ Released on 2020-12-07? ====================================== +bpo-41775: Make 'IDLE Shell' the shell title. + bpo-35764: Rewrite the Calltips doc section. bpo-40181: In calltips, stop reminding that '/' marks the end of diff --git a/Lib/idlelib/pyshell.py b/Lib/idlelib/pyshell.py index 66ae0f7435dab..b69916dbe876c 100755 --- a/Lib/idlelib/pyshell.py +++ b/Lib/idlelib/pyshell.py @@ -833,7 +833,7 @@ def display_executing_dialog(self): class PyShell(OutputWindow): - shell_title = "Python " + python_version() + " Shell" + shell_title = "IDLE Shell " + python_version() # Override classes ColorDelegator = ModifiedColorDelegator diff --git a/Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst b/Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst new file mode 100644 index 0000000000000..59605fa40235f --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst @@ -0,0 +1 @@ +Use 'IDLE Shell' as shell title From webhook-mailer at python.org Thu Sep 24 16:08:10 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Thu, 24 Sep 2020 20:08:10 -0000 Subject: [Python-checkins] bpo-41775: Make 'IDLE Shell' the shell title (GH-22399) Message-ID: https://github.com/python/cpython/commit/18e59f587b86297d4fc99765be10d92a717d4775 commit: 18e59f587b86297d4fc99765be10d92a717d4775 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-24T13:08:06-07:00 summary: bpo-41775: Make 'IDLE Shell' the shell title (GH-22399) 'Python Shell' may have contributed to some beginners confusing 'IDLE' with ' Python'. (cherry picked from commit 05cc881cbcf5d73a312568c78c7149eae3195072) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst M Lib/idlelib/NEWS.txt M Lib/idlelib/pyshell.py diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt index 5d38a500fe618..e538329d978b3 100644 --- a/Lib/idlelib/NEWS.txt +++ b/Lib/idlelib/NEWS.txt @@ -3,6 +3,8 @@ Released on 2020-12-?? ====================================== +bpo-41775: Make 'IDLE Shell' the shell title. + bpo-35764: Rewrite the Calltips doc section. bpo-40181: In calltips, stop reminding that '/' marks the end of diff --git a/Lib/idlelib/pyshell.py b/Lib/idlelib/pyshell.py index 66ae0f7435dab..b69916dbe876c 100755 --- a/Lib/idlelib/pyshell.py +++ b/Lib/idlelib/pyshell.py @@ -833,7 +833,7 @@ def display_executing_dialog(self): class PyShell(OutputWindow): - shell_title = "Python " + python_version() + " Shell" + shell_title = "IDLE Shell " + python_version() # Override classes ColorDelegator = ModifiedColorDelegator diff --git a/Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst b/Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst new file mode 100644 index 0000000000000..59605fa40235f --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2020-09-24-14-31-16.bpo-41775.sB8Vre.rst @@ -0,0 +1 @@ +Use 'IDLE Shell' as shell title From webhook-mailer at python.org Fri Sep 25 09:04:29 2020 From: webhook-mailer at python.org (Mark Shannon) Date: Fri, 25 Sep 2020 13:04:29 -0000 Subject: [Python-checkins] bpo-39934: Account for control blocks in 'except' in compiler. (GH-22395) Message-ID: https://github.com/python/cpython/commit/02d126aa09d96d03dcf9c5b51c858ce5ef386601 commit: 02d126aa09d96d03dcf9c5b51c858ce5ef386601 branch: master author: Mark Shannon committer: GitHub date: 2020-09-25T14:04:19+01:00 summary: bpo-39934: Account for control blocks in 'except' in compiler. (GH-22395) * Account for control blocks in 'except' in compiler. Fixes #39934. files: A Misc/NEWS.d/next/Core and Builtins/2020-09-24-12-15-45.bpo-39934.YVHTCF.rst M Lib/test/test_syntax.py M Python/compile.c diff --git a/Lib/test/test_syntax.py b/Lib/test/test_syntax.py index 4657fd1c0d8a7..09c6eb3375409 100644 --- a/Lib/test/test_syntax.py +++ b/Lib/test/test_syntax.py @@ -950,6 +950,15 @@ def test_empty_line_after_linecont(self): except SyntaxError: self.fail("Empty line after a line continuation character is valid.") + @support.cpython_only + def test_nested_named_except_blocks(self): + code = "" + for i in range(12): + code += f"{' '*i}try:\n" + code += f"{' '*(i+1)}raise Exception\n" + code += f"{' '*i}except Exception as e:\n" + code += f"{' '*4*12}pass" + self._check_error(code, "too many statically nested blocks") def test_main(): support.run_unittest(SyntaxTestCase) diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-24-12-15-45.bpo-39934.YVHTCF.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-24-12-15-45.bpo-39934.YVHTCF.rst new file mode 100644 index 0000000000000..92cd1ba234d21 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-24-12-15-45.bpo-39934.YVHTCF.rst @@ -0,0 +1,3 @@ +Correctly count control blocks in 'except' in compiler. Ensures that a +syntax error, rather a fatal error, occurs for deeply nested, named +exception handlers. diff --git a/Python/compile.c b/Python/compile.c index 0f9e5c276c7b7..f2563d7f7a411 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -108,8 +108,8 @@ It's called a frame block to distinguish it from a basic block in the compiler IR. */ -enum fblocktype { WHILE_LOOP, FOR_LOOP, EXCEPT, FINALLY_TRY, FINALLY_END, - WITH, ASYNC_WITH, HANDLER_CLEANUP, POP_VALUE }; +enum fblocktype { WHILE_LOOP, FOR_LOOP, TRY_EXCEPT, FINALLY_TRY, FINALLY_END, + WITH, ASYNC_WITH, HANDLER_CLEANUP, POP_VALUE, EXCEPTION_HANDLER }; struct fblockinfo { enum fblocktype fb_type; @@ -1623,9 +1623,7 @@ compiler_push_fblock(struct compiler *c, enum fblocktype t, basicblock *b, { struct fblockinfo *f; if (c->u->u_nfblocks >= CO_MAXBLOCKS) { - PyErr_SetString(PyExc_SyntaxError, - "too many statically nested blocks"); - return 0; + return compiler_error(c, "too many statically nested blocks"); } f = &c->u->u_fblock[c->u->u_nfblocks++]; f->fb_type = t; @@ -1665,6 +1663,7 @@ compiler_unwind_fblock(struct compiler *c, struct fblockinfo *info, { switch (info->fb_type) { case WHILE_LOOP: + case EXCEPTION_HANDLER: return 1; case FOR_LOOP: @@ -1675,7 +1674,7 @@ compiler_unwind_fblock(struct compiler *c, struct fblockinfo *info, ADDOP(c, POP_TOP); return 1; - case EXCEPT: + case TRY_EXCEPT: ADDOP(c, POP_BLOCK); return 1; @@ -3060,14 +3059,17 @@ compiler_try_except(struct compiler *c, stmt_ty s) return 0; ADDOP_JUMP(c, SETUP_FINALLY, except); compiler_use_next_block(c, body); - if (!compiler_push_fblock(c, EXCEPT, body, NULL, NULL)) + if (!compiler_push_fblock(c, TRY_EXCEPT, body, NULL, NULL)) return 0; VISIT_SEQ(c, stmt, s->v.Try.body); ADDOP(c, POP_BLOCK); - compiler_pop_fblock(c, EXCEPT, body); + compiler_pop_fblock(c, TRY_EXCEPT, body); ADDOP_JUMP(c, JUMP_FORWARD, orelse); n = asdl_seq_LEN(s->v.Try.handlers); compiler_use_next_block(c, except); + /* Runtime will push a block here, so we need to account for that */ + if (!compiler_push_fblock(c, EXCEPTION_HANDLER, NULL, NULL, NULL)) + return 0; for (i = 0; i < n; i++) { excepthandler_ty handler = (excepthandler_ty)asdl_seq_GET( s->v.Try.handlers, i); @@ -3152,6 +3154,7 @@ compiler_try_except(struct compiler *c, stmt_ty s) } compiler_use_next_block(c, except); } + compiler_pop_fblock(c, EXCEPTION_HANDLER, NULL); ADDOP(c, RERAISE); compiler_use_next_block(c, orelse); VISIT_SEQ(c, stmt, s->v.Try.orelse); From webhook-mailer at python.org Fri Sep 25 14:08:55 2020 From: webhook-mailer at python.org (Eric Larson) Date: Fri, 25 Sep 2020 18:08:55 -0000 Subject: [Python-checkins] Fix logging error message (GH-22410) Message-ID: https://github.com/python/cpython/commit/9fdb76c34c44ac27531229b72aa2aa424d10e846 commit: 9fdb76c34c44ac27531229b72aa2aa424d10e846 branch: master author: Eric Larson committer: GitHub date: 2020-09-25T11:08:50-07:00 summary: Fix logging error message (GH-22410) Same changes as #22276 squashed to a single commit. Just hoping to get Travis to cooperate by opening a new PR... Automerge-Triggered-By: @vsajip files: M Lib/logging/__init__.py M Lib/test/test_logging.py diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py index 94361ca75f4f3..d8a88db378436 100644 --- a/Lib/logging/__init__.py +++ b/Lib/logging/__init__.py @@ -194,7 +194,8 @@ def _checkLevel(level): raise ValueError("Unknown level: %r" % level) rv = _nameToLevel[level] else: - raise TypeError("Level not an integer or a valid string: %r" % level) + raise TypeError("Level not an integer or a valid string: %r" + % (level,)) return rv #--------------------------------------------------------------------------- diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index d23fbfb4fe281..4cd8c7e25daa9 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -3720,7 +3720,15 @@ def tzname(self, dt): utc = UTC() -class FormatterTest(unittest.TestCase): +class AssertErrorMessage: + + def assert_error_message(self, exception, message, *args, **kwargs): + try: + self.assertRaises((), *args, **kwargs) + except exception as e: + self.assertEqual(message, str(e)) + +class FormatterTest(unittest.TestCase, AssertErrorMessage): def setUp(self): self.common = { 'name': 'formatter.test', @@ -3744,12 +3752,6 @@ def get_record(self, name=None): result.update(self.variants[name]) return logging.makeLogRecord(result) - def assert_error_message(self, exception, message, *args, **kwargs): - try: - self.assertRaises(exception, *args, **kwargs) - except exception as e: - self.assertEqual(message, e.message) - def test_percent(self): # Test %-formatting r = self.get_record() @@ -3868,7 +3870,7 @@ def test_format_validate(self): # Testing failure for '-' in field name self.assert_error_message( ValueError, - "invalid field name/expression: 'name-thing'", + "invalid format: invalid field name/expression: 'name-thing'", logging.Formatter, "{name-thing}", style="{" ) # Testing failure for style mismatch @@ -3891,7 +3893,7 @@ def test_format_validate(self): # Testing failure for invalid spec self.assert_error_message( ValueError, - "bad specifier: '.2ff'", + "invalid format: bad specifier: '.2ff'", logging.Formatter, '{process:.2ff}', style='{' ) self.assertRaises(ValueError, logging.Formatter, '{process:.2Z}', style='{') @@ -3901,12 +3903,12 @@ def test_format_validate(self): # Testing failure for mismatch braces self.assert_error_message( ValueError, - "invalid format: unmatched '{' in format spec", + "invalid format: expected '}' before end of string", logging.Formatter, '{process', style='{' ) self.assert_error_message( ValueError, - "invalid format: unmatched '{' in format spec", + "invalid format: Single '}' encountered in format string", logging.Formatter, 'process}', style='{' ) self.assertRaises(ValueError, logging.Formatter, '{{foo!r:4.2}', style='{') @@ -4867,7 +4869,7 @@ def process(self, msg, kwargs): self.assertIs(self.logger.manager, orig_manager) -class LoggerTest(BaseTest): +class LoggerTest(BaseTest, AssertErrorMessage): def setUp(self): super(LoggerTest, self).setUp() @@ -4879,7 +4881,12 @@ def setUp(self): self.addCleanup(logging.shutdown) def test_set_invalid_level(self): - self.assertRaises(TypeError, self.logger.setLevel, object()) + self.assert_error_message( + TypeError, 'Level not an integer or a valid string: None', + self.logger.setLevel, None) + self.assert_error_message( + TypeError, 'Level not an integer or a valid string: (0, 0)', + self.logger.setLevel, (0, 0)) def test_exception(self): msg = 'testing exception: %r' From webhook-mailer at python.org Sat Sep 26 06:49:12 2020 From: webhook-mailer at python.org (Victor Stinner) Date: Sat, 26 Sep 2020 10:49:12 -0000 Subject: [Python-checkins] bpo-41428: Fix compiler warning in unionobject.c (GH-22416) Message-ID: https://github.com/python/cpython/commit/d73cf7ca85fb60b739e671597aabe72cc36d397a commit: d73cf7ca85fb60b739e671597aabe72cc36d397a branch: master author: Victor Stinner committer: GitHub date: 2020-09-26T12:48:41+02:00 summary: bpo-41428: Fix compiler warning in unionobject.c (GH-22416) Use Py_ssize_t type rather than int, to store lengths in unionobject.c. Fix the warning: Objects\unionobject.c(205,1): warning C4244: 'initializing': conversion from 'Py_ssize_t' to 'int', possible loss of data files: M Objects/unionobject.c diff --git a/Objects/unionobject.c b/Objects/unionobject.c index e055a55e91715..8cfb2a664753f 100644 --- a/Objects/unionobject.c +++ b/Objects/unionobject.c @@ -202,8 +202,8 @@ flatten_args(PyObject* args) PyTypeObject* arg_type = Py_TYPE(arg); if (arg_type == &_Py_UnionType) { PyObject* nested_args = ((unionobject*)arg)->args; - int nested_arg_length = PyTuple_GET_SIZE(nested_args); - for (int j = 0; j < nested_arg_length; j++) { + Py_ssize_t nested_arg_length = PyTuple_GET_SIZE(nested_args); + for (Py_ssize_t j = 0; j < nested_arg_length; j++) { PyObject* nested_arg = PyTuple_GET_ITEM(nested_args, j); Py_INCREF(nested_arg); PyTuple_SET_ITEM(flattened_args, pos, nested_arg); @@ -231,7 +231,7 @@ dedup_and_flatten_args(PyObject* args) return NULL; } // Add unique elements to an array. - int added_items = 0; + Py_ssize_t added_items = 0; for (Py_ssize_t i = 0; i < arg_length; i++) { int is_duplicate = 0; PyObject* i_element = PyTuple_GET_ITEM(args, i); From webhook-mailer at python.org Sat Sep 26 06:56:34 2020 From: webhook-mailer at python.org (Dong-hee Na) Date: Sat, 26 Sep 2020 10:56:34 -0000 Subject: [Python-checkins] bpo-1635741: Port _bisect module to multi-phase init (GH-22415) Message-ID: https://github.com/python/cpython/commit/2afd1751dd9a35d4ec03b708e3e5cddd72c43f7e commit: 2afd1751dd9a35d4ec03b708e3e5cddd72c43f7e branch: master author: Dong-hee Na committer: GitHub date: 2020-09-26T19:56:26+09:00 summary: bpo-1635741: Port _bisect module to multi-phase init (GH-22415) files: A Misc/NEWS.d/next/Core and Builtins/2020-09-26-14-43-30.bpo-1635741.aJS9B3.rst M Modules/_bisectmodule.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-26-14-43-30.bpo-1635741.aJS9B3.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-26-14-43-30.bpo-1635741.aJS9B3.rst new file mode 100644 index 0000000000000..252dab35a1368 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-26-14-43-30.bpo-1635741.aJS9B3.rst @@ -0,0 +1 @@ +Port the :mod:`_bisect` module to the multi-phase initialization API (:pep:`489`). diff --git a/Modules/_bisectmodule.c b/Modules/_bisectmodule.c index 82d800d9a8790..277e9755f2721 100644 --- a/Modules/_bisectmodule.c +++ b/Modules/_bisectmodule.c @@ -237,18 +237,14 @@ common approach.\n"); static struct PyModuleDef _bisectmodule = { PyModuleDef_HEAD_INIT, - "_bisect", - module_doc, - -1, - bisect_methods, - NULL, - NULL, - NULL, - NULL + .m_name = "_bisect", + .m_doc = module_doc, + .m_methods = bisect_methods, + .m_size = 0 }; PyMODINIT_FUNC PyInit__bisect(void) { - return PyModule_Create(&_bisectmodule); + return PyModuleDef_Init(&_bisectmodule); } From webhook-mailer at python.org Sat Sep 26 20:47:34 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Sun, 27 Sep 2020 00:47:34 -0000 Subject: [Python-checkins] Revert "Fix all Python Cookbook links (#22205)" (GH-22424) Message-ID: https://github.com/python/cpython/commit/e8a2076e149703ea043db5a9dfaaf8d9b7f37c3e commit: e8a2076e149703ea043db5a9dfaaf8d9b7f37c3e branch: master author: Andre Delfino committer: GitHub date: 2020-09-27T01:47:25+01:00 summary: Revert "Fix all Python Cookbook links (#22205)" (GH-22424) This commit reverts commit ac0333e1e117b7f61ed7ef1dbcdb6e515ada603b as the original links are working again and they provide extended features such as comments and alternative versions. files: M Doc/faq/programming.rst M Doc/howto/urllib2.rst M Doc/library/bisect.rst M Doc/library/collections.abc.rst M Doc/library/collections.rst M Doc/library/difflib.rst M Doc/library/math.rst M Doc/library/random.rst M Doc/library/shelve.rst M Doc/library/stdtypes.rst M Doc/library/sys.rst M Doc/tutorial/whatnow.rst M Doc/whatsnew/3.2.rst M Lib/collections/__init__.py M Lib/heapq.py M Lib/test/test_math.py M Tools/peg_generator/pegen/sccutils.py diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index fd0adc378bfa6..4f4ea8b18176c 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1141,7 +1141,7 @@ How do you remove duplicates from a list? See the Python Cookbook for a long discussion of many ways to do this: - https://github.com/ActiveState/code/tree/master/recipes/Python/52560_Remove_duplicates/recipe-52560.py + https://code.activestate.com/recipes/52560/ If you don't mind reordering the list, sort it and then scan from the end of the list, deleting duplicates as you go:: diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst index 38623371fbabf..046a88af62f0b 100644 --- a/Doc/howto/urllib2.rst +++ b/Doc/howto/urllib2.rst @@ -601,5 +601,5 @@ This document was reviewed and revised by John Lee. scripts with a localhost server, I have to prevent urllib from using the proxy. .. [#] urllib opener for SSL proxy (CONNECT method): `ASPN Cookbook Recipe - `_. + `_. diff --git a/Doc/library/bisect.rst b/Doc/library/bisect.rst index 6666d55abe2e5..6bf7814b257f4 100644 --- a/Doc/library/bisect.rst +++ b/Doc/library/bisect.rst @@ -60,7 +60,7 @@ The following functions are provided: .. seealso:: `SortedCollection recipe - `_ that uses + `_ that uses bisect to build a full-featured collection class with straight-forward search methods and support for a key-function. The keys are precomputed to save unnecessary calls to the key function during searches. diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst index a6038098675da..db0e25bb0772e 100644 --- a/Doc/library/collections.abc.rst +++ b/Doc/library/collections.abc.rst @@ -308,7 +308,7 @@ Notes on using :class:`Set` and :class:`MutableSet` as a mixin: .. seealso:: - * `OrderedSet recipe `_ for an + * `OrderedSet recipe `_ for an example built on :class:`MutableSet`. * For more about ABCs, see the :mod:`abc` module and :pep:`3119`. diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index a7d01b3f397a7..f538da5e1c9fa 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -135,12 +135,12 @@ The class can be used to simulate nested scopes and is useful in templating. :attr:`~collections.ChainMap.parents` property. * The `Nested Contexts recipe - `_ has options to control + `_ has options to control whether writes and other mutations apply only to the first mapping or to any mapping in the chain. * A `greatly simplified read-only version of Chainmap - `_. + `_. :class:`ChainMap` Examples and Recipes diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst index 009b7976dff15..aa08988c8b36f 100644 --- a/Doc/library/difflib.rst +++ b/Doc/library/difflib.rst @@ -633,7 +633,7 @@ If you want to know how to change the first sequence into the second, use work. * `Simple version control recipe - `_ for a small application + `_ for a small application built with :class:`SequenceMatcher`. diff --git a/Doc/library/math.rst b/Doc/library/math.rst index f152c45a87aa3..bbf64643ff59f 100644 --- a/Doc/library/math.rst +++ b/Doc/library/math.rst @@ -123,7 +123,7 @@ Number-theoretic and representation functions For further discussion and two alternative approaches, see the `ASPN cookbook recipes for accurate floating point summation - `_\. + `_\. .. function:: gcd(*integers) diff --git a/Doc/library/random.rst b/Doc/library/random.rst index 4e97b1dbad85c..0cdf0a6ac4a47 100644 --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -57,7 +57,7 @@ from sources provided by the operating system. `Complementary-Multiply-with-Carry recipe - `_ for a compatible alternative + `_ for a compatible alternative random number generator with a long period and comparatively simple update operations. diff --git a/Doc/library/shelve.rst b/Doc/library/shelve.rst index a94255bbf698e..f08c58179a2f9 100644 --- a/Doc/library/shelve.rst +++ b/Doc/library/shelve.rst @@ -75,7 +75,7 @@ Two additional methods are supported: .. seealso:: - `Persistent dictionary recipe `_ + `Persistent dictionary recipe `_ with widely supported storage formats and having the speed of native dictionaries. diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 2eee22c79af76..0ffe7b7526fa7 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1404,7 +1404,7 @@ objects that compare equal might have different :attr:`~range.start`, .. seealso:: - * The `linspace recipe `_ + * The `linspace recipe `_ shows how to implement a lazy version of range suitable for floating point applications. diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index aa417ede40228..d201d7061f980 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -679,7 +679,7 @@ always available. additional garbage collector overhead if the object is managed by the garbage collector. - See `recursive sizeof recipe `_ + See `recursive sizeof recipe `_ for an example of using :func:`getsizeof` recursively to find the size of containers and all their contents. diff --git a/Doc/tutorial/whatnow.rst b/Doc/tutorial/whatnow.rst index 38ce9f0a900c2..3208201312b87 100644 --- a/Doc/tutorial/whatnow.rst +++ b/Doc/tutorial/whatnow.rst @@ -43,7 +43,7 @@ More Python resources: for download. Once you begin releasing code, you can register it here so that others can find it. -* https://github.com/ActiveState/code/tree/master/recipes/Python: The Python Cookbook is a +* https://code.activestate.com/recipes/langs/python/: The Python Cookbook is a sizable collection of code examples, larger modules, and useful scripts. Particularly notable contributions are collected in a book also titled Python Cookbook (O'Reilly & Associates, ISBN 0-596-00797-3.) diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index 37bae34ce74ad..06bee9966c0be 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -781,8 +781,8 @@ functools (Contributed by Raymond Hettinger and incorporating design ideas from Jim Baker, Miki Tebeka, and Nick Coghlan; see `recipe 498245 - `_\, `recipe 577479 - `_\, :issue:`10586`, and + `_\, `recipe 577479 + `_\, :issue:`10586`, and :issue:`10593`.) * The :func:`functools.wraps` decorator now adds a :attr:`__wrapped__` attribute diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py index f4da9d0cefd6b..5d75501645fc4 100644 --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -574,7 +574,7 @@ class Counter(dict): # http://en.wikipedia.org/wiki/Multiset # http://www.gnu.org/software/smalltalk/manual-base/html_node/Bag.html # http://www.demo2s.com/Tutorial/Cpp/0380__set-multiset/Catalog0380__set-multiset.htm - # https://github.com/ActiveState/code/tree/master/recipes/Python/259174_bag_collection_class/recipe-259174.py + # http://code.activestate.com/recipes/259174/ # Knuth, TAOCP Vol. II section 4.6.3 def __init__(self, iterable=None, /, **kwds): diff --git a/Lib/heapq.py b/Lib/heapq.py index 5895562db4142..fabefd87f8bf8 100644 --- a/Lib/heapq.py +++ b/Lib/heapq.py @@ -456,7 +456,7 @@ def merge(*iterables, key=None, reverse=False): # 2) Made multiple passes over the data. # 3) Made more comparisons in common cases (small k, large n, semi-random input). # See the more detailed comparison of approach at: -# https://github.com/ActiveState/code/tree/master/recipes/Python/577573_Compare_algorithms/recipe-577573.py +# http://code.activestate.com/recipes/577573-compare-algorithms-for-heapqsmallest def nsmallest(n, iterable, key=None): """Find the n smallest elements in a dataset. diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py index 2abe5b028b355..a6f6483f55d89 100644 --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -611,7 +611,7 @@ def testFsum(self): def msum(iterable): """Full precision summation. Compute sum(iterable) without any intermediate accumulation of error. Based on the 'lsum' function - at https://github.com/ActiveState/code/tree/master/recipes/Python/393090_Binary_floating_point_summatiaccurate_full/recipe-393090.py + at http://code.activestate.com/recipes/393090/ """ tmant, texp = 0, 0 diff --git a/Tools/peg_generator/pegen/sccutils.py b/Tools/peg_generator/pegen/sccutils.py index 0c295196607ec..1f0586bb2f7d6 100644 --- a/Tools/peg_generator/pegen/sccutils.py +++ b/Tools/peg_generator/pegen/sccutils.py @@ -18,7 +18,7 @@ def strongly_connected_components( exactly once; vertices not part of a SCC are returned as singleton sets. - From https://github.com/ActiveState/code/tree/master/recipes/Python/578507_Strongly_connected_components_directed/recipe-578507.py. + From http://code.activestate.com/recipes/578507/. """ identified: Set[str] = set() stack: List[str] = [] @@ -81,7 +81,7 @@ def topsort( {B, C} {A} - From https://github.com/ActiveState/code/tree/master/recipes/Python/577413_Topological_Sort/recipe-577413.py. + From http://code.activestate.com/recipes/577413/. """ # TODO: Use a faster algorithm? for k, v in data.items(): From webhook-mailer at python.org Sat Sep 26 23:43:22 2020 From: webhook-mailer at python.org (Emmanuel Arias) Date: Sun, 27 Sep 2020 03:43:22 -0000 Subject: [Python-checkins] bpo-41858: Clarify line in optparse doc (GH-22407) Message-ID: https://github.com/python/cpython/commit/00eb063b6600fdb4ba5cfb99da83cc1660ec69bf commit: 00eb063b6600fdb4ba5cfb99da83cc1660ec69bf branch: master author: Emmanuel Arias committer: GitHub date: 2020-09-26T23:43:18-04:00 summary: bpo-41858: Clarify line in optparse doc (GH-22407) The existing line is easily read as being incomplete. files: M Doc/library/optparse.rst diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index c1a18e0147431..b1094198f4c84 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -55,7 +55,7 @@ equivalent to the above example:: -q -foutfile -qfoutfile -Additionally, users can run one of :: +Additionally, users can run one of the following :: -h --help From webhook-mailer at python.org Sat Sep 26 23:52:06 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 27 Sep 2020 03:52:06 -0000 Subject: [Python-checkins] bpo-41858: Clarify line in optparse doc (GH-22407) Message-ID: https://github.com/python/cpython/commit/e0e614ca99c39e107b97c7be779efea08f22ace3 commit: e0e614ca99c39e107b97c7be779efea08f22ace3 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-26T20:52:01-07:00 summary: bpo-41858: Clarify line in optparse doc (GH-22407) The existing line is easily read as being incomplete. (cherry picked from commit 00eb063b6600fdb4ba5cfb99da83cc1660ec69bf) Co-authored-by: Emmanuel Arias files: M Doc/library/optparse.rst diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index c1a18e0147431..b1094198f4c84 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -55,7 +55,7 @@ equivalent to the above example:: -q -foutfile -qfoutfile -Additionally, users can run one of :: +Additionally, users can run one of the following :: -h --help From webhook-mailer at python.org Sun Sep 27 00:04:01 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 27 Sep 2020 04:04:01 -0000 Subject: [Python-checkins] bpo-41858: Clarify line in optparse doc (GH-22407) Message-ID: https://github.com/python/cpython/commit/8b2595f68af059afdd61b2bb3d32a2bf4f1662c8 commit: 8b2595f68af059afdd61b2bb3d32a2bf4f1662c8 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-26T21:03:56-07:00 summary: bpo-41858: Clarify line in optparse doc (GH-22407) The existing line is easily read as being incomplete. (cherry picked from commit 00eb063b6600fdb4ba5cfb99da83cc1660ec69bf) Co-authored-by: Emmanuel Arias files: M Doc/library/optparse.rst diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst index c1a18e0147431..b1094198f4c84 100644 --- a/Doc/library/optparse.rst +++ b/Doc/library/optparse.rst @@ -55,7 +55,7 @@ equivalent to the above example:: -q -foutfile -qfoutfile -Additionally, users can run one of :: +Additionally, users can run one of the following :: -h --help From webhook-mailer at python.org Sun Sep 27 08:15:09 2020 From: webhook-mailer at python.org (Erlend Egeberg Aasland) Date: Sun, 27 Sep 2020 12:15:09 -0000 Subject: [Python-checkins] bpo-41861: Convert _sqlite3 cache and node static types to heap types (GH-22417) Message-ID: https://github.com/python/cpython/commit/a937ab45d6af4f99ff16fec73d5d75d93c569456 commit: a937ab45d6af4f99ff16fec73d5d75d93c569456 branch: master author: Erlend Egeberg Aasland committer: GitHub date: 2020-09-27T14:14:50+02:00 summary: bpo-41861: Convert _sqlite3 cache and node static types to heap types (GH-22417) files: M Modules/_sqlite/cache.c M Modules/_sqlite/cache.h M Modules/_sqlite/connection.c M Modules/_sqlite/module.c diff --git a/Modules/_sqlite/cache.c b/Modules/_sqlite/cache.c index 758fc022f7810..c417ce872d368 100644 --- a/Modules/_sqlite/cache.c +++ b/Modules/_sqlite/cache.c @@ -29,7 +29,7 @@ pysqlite_Node* pysqlite_new_node(PyObject* key, PyObject* data) { pysqlite_Node* node; - node = (pysqlite_Node*) (pysqlite_NodeType.tp_alloc(&pysqlite_NodeType, 0)); + node = (pysqlite_Node*) (pysqlite_NodeType->tp_alloc(pysqlite_NodeType, 0)); if (!node) { return NULL; } @@ -48,10 +48,13 @@ pysqlite_Node* pysqlite_new_node(PyObject* key, PyObject* data) void pysqlite_node_dealloc(pysqlite_Node* self) { + PyTypeObject *tp = Py_TYPE(self); + Py_DECREF(self->key); Py_DECREF(self->data); - Py_TYPE(self)->tp_free((PyObject*)self); + tp->tp_free(self); + Py_DECREF(tp); } int pysqlite_cache_init(pysqlite_Cache* self, PyObject* args, PyObject* kwargs) @@ -88,6 +91,7 @@ int pysqlite_cache_init(pysqlite_Cache* self, PyObject* args, PyObject* kwargs) void pysqlite_cache_dealloc(pysqlite_Cache* self) { + PyTypeObject *tp = Py_TYPE(self); pysqlite_Node* node; pysqlite_Node* delete_node; @@ -109,7 +113,8 @@ void pysqlite_cache_dealloc(pysqlite_Cache* self) } Py_DECREF(self->mapping); - Py_TYPE(self)->tp_free((PyObject*)self); + tp->tp_free(self); + Py_DECREF(tp); } PyObject* pysqlite_cache_get(pysqlite_Cache* self, PyObject* key) @@ -253,6 +258,20 @@ PyObject* pysqlite_cache_display(pysqlite_Cache* self, PyObject* args) Py_RETURN_NONE; } +static PyType_Slot pysqlite_NodeType_slots[] = { + {Py_tp_dealloc, pysqlite_node_dealloc}, + {Py_tp_new, PyType_GenericNew}, + {0, NULL}, +}; + +static PyType_Spec pysqlite_NodeType_spec = { + .name = MODULE_NAME ".Node", + .basicsize = sizeof(pysqlite_Node), + .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HEAPTYPE, + .slots = pysqlite_NodeType_slots, +}; +PyTypeObject *pysqlite_NodeType = NULL; + static PyMethodDef cache_methods[] = { {"get", (PyCFunction)pysqlite_cache_get, METH_O, PyDoc_STR("Gets an entry from the cache or calls the factory function to produce one.")}, @@ -261,102 +280,32 @@ static PyMethodDef cache_methods[] = { {NULL, NULL} }; -PyTypeObject pysqlite_NodeType = { - PyVarObject_HEAD_INIT(NULL, 0) - MODULE_NAME "Node", /* tp_name */ - sizeof(pysqlite_Node), /* tp_basicsize */ - 0, /* tp_itemsize */ - (destructor)pysqlite_node_dealloc, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, /* tp_flags */ - 0, /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - 0, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - (initproc)0, /* tp_init */ - 0, /* tp_alloc */ - 0, /* tp_new */ - 0 /* tp_free */ +static PyType_Slot pysqlite_CacheType_slots[] = { + {Py_tp_dealloc, pysqlite_cache_dealloc}, + {Py_tp_methods, cache_methods}, + {Py_tp_new, PyType_GenericNew}, + {Py_tp_init, pysqlite_cache_init}, + {0, NULL}, }; -PyTypeObject pysqlite_CacheType = { - PyVarObject_HEAD_INIT(NULL, 0) - MODULE_NAME ".Cache", /* tp_name */ - sizeof(pysqlite_Cache), /* tp_basicsize */ - 0, /* tp_itemsize */ - (destructor)pysqlite_cache_dealloc, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, /* tp_flags */ - 0, /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - cache_methods, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - (initproc)pysqlite_cache_init, /* tp_init */ - 0, /* tp_alloc */ - 0, /* tp_new */ - 0 /* tp_free */ +static PyType_Spec pysqlite_CacheType_spec = { + .name = MODULE_NAME ".Cache", + .basicsize = sizeof(pysqlite_Cache), + .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HEAPTYPE, + .slots = pysqlite_CacheType_slots, }; +PyTypeObject *pysqlite_CacheType = NULL; -extern int pysqlite_cache_setup_types(void) +extern int pysqlite_cache_setup_types(PyObject *mod) { - int rc; - - pysqlite_NodeType.tp_new = PyType_GenericNew; - pysqlite_CacheType.tp_new = PyType_GenericNew; - - rc = PyType_Ready(&pysqlite_NodeType); - if (rc < 0) { - return rc; + pysqlite_NodeType = (PyTypeObject *)PyType_FromModuleAndSpec(mod, &pysqlite_NodeType_spec, NULL); + if (pysqlite_NodeType == NULL) { + return -1; } - rc = PyType_Ready(&pysqlite_CacheType); - return rc; + pysqlite_CacheType = (PyTypeObject *)PyType_FromModuleAndSpec(mod, &pysqlite_CacheType_spec, NULL); + if (pysqlite_CacheType == NULL) { + return -1; + } + return 0; } diff --git a/Modules/_sqlite/cache.h b/Modules/_sqlite/cache.h index 529010967c4f3..0afdf7f09b65c 100644 --- a/Modules/_sqlite/cache.h +++ b/Modules/_sqlite/cache.h @@ -59,8 +59,8 @@ typedef struct int decref_factory; } pysqlite_Cache; -extern PyTypeObject pysqlite_NodeType; -extern PyTypeObject pysqlite_CacheType; +extern PyTypeObject *pysqlite_NodeType; +extern PyTypeObject *pysqlite_CacheType; int pysqlite_node_init(pysqlite_Node* self, PyObject* args, PyObject* kwargs); void pysqlite_node_dealloc(pysqlite_Node* self); @@ -69,6 +69,6 @@ int pysqlite_cache_init(pysqlite_Cache* self, PyObject* args, PyObject* kwargs); void pysqlite_cache_dealloc(pysqlite_Cache* self); PyObject* pysqlite_cache_get(pysqlite_Cache* self, PyObject* args); -int pysqlite_cache_setup_types(void); +int pysqlite_cache_setup_types(PyObject *module); #endif diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c index 81fc1335371a6..121850ae7e1f3 100644 --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -133,7 +133,7 @@ int pysqlite_connection_init(pysqlite_Connection* self, PyObject* args, PyObject } Py_DECREF(isolation_level); - self->statement_cache = (pysqlite_Cache*)PyObject_CallFunction((PyObject*)&pysqlite_CacheType, "Oi", self, cached_statements); + self->statement_cache = (pysqlite_Cache*)PyObject_CallFunction((PyObject*)pysqlite_CacheType, "Oi", self, cached_statements); if (PyErr_Occurred()) { return -1; } diff --git a/Modules/_sqlite/module.c b/Modules/_sqlite/module.c index 82f58eb248026..625d065a317ba 100644 --- a/Modules/_sqlite/module.c +++ b/Modules/_sqlite/module.c @@ -355,7 +355,7 @@ PyMODINIT_FUNC PyInit__sqlite3(void) (pysqlite_row_setup_types() < 0) || (pysqlite_cursor_setup_types() < 0) || (pysqlite_connection_setup_types() < 0) || - (pysqlite_cache_setup_types() < 0) || + (pysqlite_cache_setup_types(module) < 0) || (pysqlite_statement_setup_types() < 0) || (pysqlite_prepare_protocol_setup_types() < 0) ) { From webhook-mailer at python.org Sun Sep 27 15:07:08 2020 From: webhook-mailer at python.org (Andre Delfino) Date: Sun, 27 Sep 2020 19:07:08 -0000 Subject: [Python-checkins] [doc] Leverage the fact that the actual types can now be indexed for typing (GH-22340) Message-ID: https://github.com/python/cpython/commit/d9ab95ff1fe81efdf70e545d536d9f6927d1ba81 commit: d9ab95ff1fe81efdf70e545d536d9f6927d1ba81 branch: master author: Andre Delfino committer: GitHub date: 2020-09-27T12:07:04-07:00 summary: [doc] Leverage the fact that the actual types can now be indexed for typing (GH-22340) This shows users that they can use the actual types. Using deprecated types is confusing. This also prefers colections.abc.Sized instead of the alias typing.Sized. I guess the aliases were created to make it convenient to import all collections related types from the same place. This should be backported to 3.9. Automerge-Triggered-By: @gvanrossum files: M Doc/glossary.rst M Doc/library/typing.rst diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 7be755e411310..9fdbdb1a83f28 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -1084,19 +1084,15 @@ Glossary Type aliases are useful for simplifying :term:`type hints `. For example:: - from typing import List, Tuple - def remove_gray_shades( - colors: List[Tuple[int, int, int]]) -> List[Tuple[int, int, int]]: + colors: list[tuple[int, int, int]]) -> list[tuple[int, int, int]]: pass could be made more readable like this:: - from typing import List, Tuple - - Color = Tuple[int, int, int] + Color = tuple[int, int, int] - def remove_gray_shades(colors: List[Color]) -> List[Color]: + def remove_gray_shades(colors: list[Color]) -> list[Color]: pass See :mod:`typing` and :pep:`484`, which describe this functionality. diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index d31c65d38e1ab..3b824d0a4a8da 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -38,10 +38,9 @@ Type aliases ============ A type alias is defined by assigning the type to the alias. In this example, -``Vector`` and ``List[float]`` will be treated as interchangeable synonyms:: +``Vector`` and ``list[float]`` will be treated as interchangeable synonyms:: - from typing import List - Vector = List[float] + Vector = list[float] def scale(scalar: float, vector: Vector) -> Vector: return [scalar * num for num in vector] @@ -51,11 +50,11 @@ A type alias is defined by assigning the type to the alias. In this example, Type aliases are useful for simplifying complex type signatures. For example:: - from typing import Dict, Tuple, Sequence + from collections.abc import Sequence - ConnectionOptions = Dict[str, str] - Address = Tuple[str, int] - Server = Tuple[Address, ConnectionOptions] + ConnectionOptions = dict[str, str] + Address = tuple[str, int] + Server = tuple[Address, ConnectionOptions] def broadcast_message(message: str, servers: Sequence[Server]) -> None: ... @@ -64,7 +63,7 @@ Type aliases are useful for simplifying complex type signatures. For example:: # being exactly equivalent to this one. def broadcast_message( message: str, - servers: Sequence[Tuple[Tuple[str, int], Dict[str, str]]]) -> None: + servers: Sequence[tuple[tuple[str, int], dict[str, str]]]) -> None: ... Note that ``None`` as a type hint is a special case and is replaced by @@ -157,7 +156,7 @@ type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. For example:: - from typing import Callable + from collections.abc import Callable def feeder(get_next_item: Callable[[], str]) -> None: # Body @@ -181,7 +180,7 @@ subscription to denote expected types for container elements. :: - from typing import Mapping, Sequence + from collections.abc import Mapping, Sequence def notify_by_email(employees: Sequence[Employee], overrides: Mapping[str, str]) -> None: ... @@ -191,7 +190,8 @@ called :class:`TypeVar`. :: - from typing import Sequence, TypeVar + from collections.abc import Sequence + from typing import TypeVar T = TypeVar('T') # Declare type variable @@ -235,7 +235,7 @@ class body. The :class:`Generic` base class defines :meth:`__class_getitem__` so that ``LoggedVar[t]`` is valid as a type:: - from typing import Iterable + from collections.abc import Iterable def zero_all_vars(vars: Iterable[LoggedVar[int]]) -> None: for var in vars: @@ -266,7 +266,8 @@ This is thus invalid:: You can use multiple inheritance with :class:`Generic`:: - from typing import TypeVar, Generic, Sized + from collections.abc import Sized + from typing import TypeVar, Generic T = TypeVar('T') @@ -275,7 +276,8 @@ You can use multiple inheritance with :class:`Generic`:: When inheriting from generic classes, some type variables could be fixed:: - from typing import TypeVar, Mapping + from collections.abc import Mapping + from typing import TypeVar T = TypeVar('T') @@ -288,13 +290,14 @@ Using a generic class without specifying type parameters assumes :data:`Any` for each position. In the following example, ``MyIterable`` is not generic but implicitly inherits from ``Iterable[Any]``:: - from typing import Iterable + from collections.abc import Iterable class MyIterable(Iterable): # Same as Iterable[Any] User defined generic type aliases are also supported. Examples:: - from typing import TypeVar, Iterable, Tuple, Union + from collections.abc import Iterable + from typing import TypeVar, Union S = TypeVar('S') Response = Union[Iterable[S], int] @@ -303,9 +306,9 @@ User defined generic type aliases are also supported. Examples:: ... T = TypeVar('T', int, float, complex) - Vec = Iterable[Tuple[T, T]] + Vec = Iterable[tuple[T, T]] - def inproduct(v: Vec[T]) -> T: # Same as Iterable[Tuple[T, T]] + def inproduct(v: Vec[T]) -> T: # Same as Iterable[tuple[T, T]] return sum(x*y for x, y in v) .. versionchanged:: 3.7 @@ -408,7 +411,7 @@ to be explicitly marked to support them, which is unpythonic and unlike what one would normally do in idiomatic dynamically typed Python code. For example, this conforms to the :pep:`484`:: - from typing import Sized, Iterable, Iterator + from collections.abc import Sized, Iterable, Iterator class Bucket(Sized, Iterable[int]): ... @@ -421,7 +424,7 @@ allowing ``Bucket`` to be implicitly considered a subtype of both ``Sized`` and ``Iterable[int]`` by static type checkers. This is known as *structural subtyping* (or static duck-typing):: - from typing import Iterator, Iterable + from collections.abc import Iterator, Iterable class Bucket: # Note: no base classes ... @@ -1371,10 +1374,10 @@ Asynchronous programming The variance and order of type variables correspond to those of :class:`Generator`, for example:: - from typing import List, Coroutine - c = None # type: Coroutine[List[str], str, int] + from collections.abc import Coroutine + c = None # type: Coroutine[list[str], str, int] ... - x = c.send('hi') # type: List[str] + x = c.send('hi') # type: list[str] async def bar() -> None: x = await c # type: int From webhook-mailer at python.org Sun Sep 27 15:42:33 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Sun, 27 Sep 2020 19:42:33 -0000 Subject: [Python-checkins] [doc] Leverage the fact that the actual types can now be indexed for typing (GH-22340) Message-ID: https://github.com/python/cpython/commit/bdf46bc7e12c08aaab676a2c947f03daffe035a7 commit: bdf46bc7e12c08aaab676a2c947f03daffe035a7 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-27T12:42:29-07:00 summary: [doc] Leverage the fact that the actual types can now be indexed for typing (GH-22340) This shows users that they can use the actual types. Using deprecated types is confusing. This also prefers colections.abc.Sized instead of the alias typing.Sized. I guess the aliases were created to make it convenient to import all collections related types from the same place. This should be backported to 3.9. Automerge-Triggered-By: @gvanrossum (cherry picked from commit d9ab95ff1fe81efdf70e545d536d9f6927d1ba81) Co-authored-by: Andre Delfino files: M Doc/glossary.rst M Doc/library/typing.rst diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 7be755e411310..9fdbdb1a83f28 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -1084,19 +1084,15 @@ Glossary Type aliases are useful for simplifying :term:`type hints `. For example:: - from typing import List, Tuple - def remove_gray_shades( - colors: List[Tuple[int, int, int]]) -> List[Tuple[int, int, int]]: + colors: list[tuple[int, int, int]]) -> list[tuple[int, int, int]]: pass could be made more readable like this:: - from typing import List, Tuple - - Color = Tuple[int, int, int] + Color = tuple[int, int, int] - def remove_gray_shades(colors: List[Color]) -> List[Color]: + def remove_gray_shades(colors: list[Color]) -> list[Color]: pass See :mod:`typing` and :pep:`484`, which describe this functionality. diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index 6d6b76c1d0895..78d90d19f74e0 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -38,10 +38,9 @@ Type aliases ============ A type alias is defined by assigning the type to the alias. In this example, -``Vector`` and ``List[float]`` will be treated as interchangeable synonyms:: +``Vector`` and ``list[float]`` will be treated as interchangeable synonyms:: - from typing import List - Vector = List[float] + Vector = list[float] def scale(scalar: float, vector: Vector) -> Vector: return [scalar * num for num in vector] @@ -51,11 +50,11 @@ A type alias is defined by assigning the type to the alias. In this example, Type aliases are useful for simplifying complex type signatures. For example:: - from typing import Dict, Tuple, Sequence + from collections.abc import Sequence - ConnectionOptions = Dict[str, str] - Address = Tuple[str, int] - Server = Tuple[Address, ConnectionOptions] + ConnectionOptions = dict[str, str] + Address = tuple[str, int] + Server = tuple[Address, ConnectionOptions] def broadcast_message(message: str, servers: Sequence[Server]) -> None: ... @@ -64,7 +63,7 @@ Type aliases are useful for simplifying complex type signatures. For example:: # being exactly equivalent to this one. def broadcast_message( message: str, - servers: Sequence[Tuple[Tuple[str, int], Dict[str, str]]]) -> None: + servers: Sequence[tuple[tuple[str, int], dict[str, str]]]) -> None: ... Note that ``None`` as a type hint is a special case and is replaced by @@ -157,7 +156,7 @@ type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. For example:: - from typing import Callable + from collections.abc import Callable def feeder(get_next_item: Callable[[], str]) -> None: # Body @@ -181,7 +180,7 @@ subscription to denote expected types for container elements. :: - from typing import Mapping, Sequence + from collections.abc import Mapping, Sequence def notify_by_email(employees: Sequence[Employee], overrides: Mapping[str, str]) -> None: ... @@ -191,7 +190,8 @@ called :class:`TypeVar`. :: - from typing import Sequence, TypeVar + from collections.abc import Sequence + from typing import TypeVar T = TypeVar('T') # Declare type variable @@ -235,7 +235,7 @@ class body. The :class:`Generic` base class defines :meth:`__class_getitem__` so that ``LoggedVar[t]`` is valid as a type:: - from typing import Iterable + from collections.abc import Iterable def zero_all_vars(vars: Iterable[LoggedVar[int]]) -> None: for var in vars: @@ -266,7 +266,8 @@ This is thus invalid:: You can use multiple inheritance with :class:`Generic`:: - from typing import TypeVar, Generic, Sized + from collections.abc import Sized + from typing import TypeVar, Generic T = TypeVar('T') @@ -275,7 +276,8 @@ You can use multiple inheritance with :class:`Generic`:: When inheriting from generic classes, some type variables could be fixed:: - from typing import TypeVar, Mapping + from collections.abc import Mapping + from typing import TypeVar T = TypeVar('T') @@ -288,13 +290,14 @@ Using a generic class without specifying type parameters assumes :data:`Any` for each position. In the following example, ``MyIterable`` is not generic but implicitly inherits from ``Iterable[Any]``:: - from typing import Iterable + from collections.abc import Iterable class MyIterable(Iterable): # Same as Iterable[Any] User defined generic type aliases are also supported. Examples:: - from typing import TypeVar, Iterable, Tuple, Union + from collections.abc import Iterable + from typing import TypeVar, Union S = TypeVar('S') Response = Union[Iterable[S], int] @@ -303,9 +306,9 @@ User defined generic type aliases are also supported. Examples:: ... T = TypeVar('T', int, float, complex) - Vec = Iterable[Tuple[T, T]] + Vec = Iterable[tuple[T, T]] - def inproduct(v: Vec[T]) -> T: # Same as Iterable[Tuple[T, T]] + def inproduct(v: Vec[T]) -> T: # Same as Iterable[tuple[T, T]] return sum(x*y for x, y in v) .. versionchanged:: 3.7 @@ -408,7 +411,7 @@ to be explicitly marked to support them, which is unpythonic and unlike what one would normally do in idiomatic dynamically typed Python code. For example, this conforms to the :pep:`484`:: - from typing import Sized, Iterable, Iterator + from collections.abc import Sized, Iterable, Iterator class Bucket(Sized, Iterable[int]): ... @@ -421,7 +424,7 @@ allowing ``Bucket`` to be implicitly considered a subtype of both ``Sized`` and ``Iterable[int]`` by static type checkers. This is known as *structural subtyping* (or static duck-typing):: - from typing import Iterator, Iterable + from collections.abc import Iterator, Iterable class Bucket: # Note: no base classes ... @@ -1371,10 +1374,10 @@ Asynchronous programming The variance and order of type variables correspond to those of :class:`Generator`, for example:: - from typing import List, Coroutine - c = None # type: Coroutine[List[str], str, int] + from collections.abc import Coroutine + c = None # type: Coroutine[list[str], str, int] ... - x = c.send('hi') # type: List[str] + x = c.send('hi') # type: list[str] async def bar() -> None: x = await c # type: int From webhook-mailer at python.org Mon Sep 28 12:16:26 2020 From: webhook-mailer at python.org (Dong-hee Na) Date: Mon, 28 Sep 2020 16:16:26 -0000 Subject: [Python-checkins] bpo-41870: Use PEP 590 vectorcall to speed up bool() (GH-22427) Message-ID: https://github.com/python/cpython/commit/a195bceff7b552c5f86dec7894ff24dcc87235da commit: a195bceff7b552c5f86dec7894ff24dcc87235da branch: master author: Dong-hee Na committer: GitHub date: 2020-09-29T01:16:21+09:00 summary: bpo-41870: Use PEP 590 vectorcall to speed up bool() (GH-22427) * bpo-41870: Use PEP 590 vectorcall to speed up bool() * bpo-41870: Add NEWS.d files: A Misc/NEWS.d/next/Core and Builtins/2020-09-27-22-23-14.bpo-41870.2v6_v4.rst M Objects/boolobject.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-27-22-23-14.bpo-41870.2v6_v4.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-27-22-23-14.bpo-41870.2v6_v4.rst new file mode 100644 index 0000000000000..13a6bb04a28fd --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-27-22-23-14.bpo-41870.2v6_v4.rst @@ -0,0 +1,2 @@ +Speed up calls to ``bool()`` by using the :pep:`590` ``vectorcall`` calling +convention. Patch by Dong-hee Na. diff --git a/Objects/boolobject.c b/Objects/boolobject.c index 720835b98aa65..ab7669cb240de 100644 --- a/Objects/boolobject.c +++ b/Objects/boolobject.c @@ -55,6 +55,30 @@ bool_new(PyTypeObject *type, PyObject *args, PyObject *kwds) return PyBool_FromLong(ok); } +static PyObject * +bool_vectorcall(PyObject *type, PyObject * const*args, + size_t nargsf, PyObject *kwnames) +{ + long ok = 0; + if (!_PyArg_NoKwnames("bool", kwnames)) { + return NULL; + } + + Py_ssize_t nargs = PyVectorcall_NARGS(nargsf); + if (!_PyArg_CheckPositional("bool", nargs, 0, 1)) { + return NULL; + } + + assert(PyType_Check(type)); + if (nargs) { + ok = PyObject_IsTrue(args[0]); + } + if (ok < 0) { + return NULL; + } + return PyBool_FromLong(ok); +} + /* Arithmetic operations redefined to return bool if both args are bool. */ static PyObject * @@ -170,6 +194,7 @@ PyTypeObject PyBool_Type = { 0, /* tp_init */ 0, /* tp_alloc */ bool_new, /* tp_new */ + .tp_vectorcall = bool_vectorcall, }; /* The objects representing bool values False and True */ From webhook-mailer at python.org Mon Sep 28 14:53:42 2020 From: webhook-mailer at python.org (Jan Mazur) Date: Mon, 28 Sep 2020 18:53:42 -0000 Subject: [Python-checkins] bpo-40105: ZipFile truncate in append mode with shorter comment (GH-19337) Message-ID: https://github.com/python/cpython/commit/ff9147d93b868f0e13b9fe14e2a76c2879f6787b commit: ff9147d93b868f0e13b9fe14e2a76c2879f6787b branch: master author: Jan Mazur <16736821+mzr at users.noreply.github.com> committer: GitHub date: 2020-09-28T21:53:33+03:00 summary: bpo-40105: ZipFile truncate in append mode with shorter comment (GH-19337) files: A Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst M Lib/test/test_zipfile.py M Lib/zipfile.py diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py index 2851051425bf1..687e43df780d6 100644 --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -1856,11 +1856,14 @@ def test_comments(self): self.assertEqual(zipf.comment, b"an updated comment") # check that comments are correctly shortened in append mode + # and the file is indeed truncated with zipfile.ZipFile(TESTFN,mode="w") as zipf: zipf.comment = b"original comment that's longer" zipf.writestr("foo.txt", "O, for a Muse of Fire!") + original_zip_size = os.path.getsize(TESTFN) with zipfile.ZipFile(TESTFN,mode="a") as zipf: zipf.comment = b"shorter comment" + self.assertTrue(original_zip_size > os.path.getsize(TESTFN)) with zipfile.ZipFile(TESTFN,mode="r") as zipf: self.assertEqual(zipf.comment, b"shorter comment") diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 915698f9e0588..816f8582bbf6d 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -1918,6 +1918,8 @@ def _write_end_record(self): centDirSize, centDirOffset, len(self._comment)) self.fp.write(endrec) self.fp.write(self._comment) + if self.mode == "a": + self.fp.truncate() self.fp.flush() def _fpclose(self, fp): diff --git a/Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst b/Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst new file mode 100644 index 0000000000000..f71a7a1e697a4 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst @@ -0,0 +1,2 @@ +ZipFile truncates files to avoid corruption when a shorter comment is provided +in append ("a") mode. Patch by Jan Mazur. From webhook-mailer at python.org Mon Sep 28 15:18:47 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 28 Sep 2020 19:18:47 -0000 Subject: [Python-checkins] bpo-40105: ZipFile truncate in append mode with shorter comment (GH-19337) Message-ID: https://github.com/python/cpython/commit/e4008404fbc0002c49becc565d42e93eca11dd75 commit: e4008404fbc0002c49becc565d42e93eca11dd75 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-28T12:18:39-07:00 summary: bpo-40105: ZipFile truncate in append mode with shorter comment (GH-19337) (cherry picked from commit ff9147d93b868f0e13b9fe14e2a76c2879f6787b) Co-authored-by: Jan Mazur <16736821+mzr at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst M Lib/test/test_zipfile.py M Lib/zipfile.py diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py index e2e37a1770418..31153445697fd 100644 --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -1844,11 +1844,14 @@ def test_comments(self): self.assertEqual(zipf.comment, b"an updated comment") # check that comments are correctly shortened in append mode + # and the file is indeed truncated with zipfile.ZipFile(TESTFN,mode="w") as zipf: zipf.comment = b"original comment that's longer" zipf.writestr("foo.txt", "O, for a Muse of Fire!") + original_zip_size = os.path.getsize(TESTFN) with zipfile.ZipFile(TESTFN,mode="a") as zipf: zipf.comment = b"shorter comment" + self.assertTrue(original_zip_size > os.path.getsize(TESTFN)) with zipfile.ZipFile(TESTFN,mode="r") as zipf: self.assertEqual(zipf.comment, b"shorter comment") diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 73e89666309ff..e95e2b2d452df 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -1942,6 +1942,8 @@ def _write_end_record(self): centDirSize, centDirOffset, len(self._comment)) self.fp.write(endrec) self.fp.write(self._comment) + if self.mode == "a": + self.fp.truncate() self.fp.flush() def _fpclose(self, fp): diff --git a/Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst b/Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst new file mode 100644 index 0000000000000..f71a7a1e697a4 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst @@ -0,0 +1,2 @@ +ZipFile truncates files to avoid corruption when a shorter comment is provided +in append ("a") mode. Patch by Jan Mazur. From webhook-mailer at python.org Mon Sep 28 16:18:51 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 28 Sep 2020 20:18:51 -0000 Subject: [Python-checkins] bpo-40105: ZipFile truncate in append mode with shorter comment (GH-19337) Message-ID: https://github.com/python/cpython/commit/048f54dc75d51e8a1c5822ab7b2828295192aaa5 commit: 048f54dc75d51e8a1c5822ab7b2828295192aaa5 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-28T13:18:47-07:00 summary: bpo-40105: ZipFile truncate in append mode with shorter comment (GH-19337) (cherry picked from commit ff9147d93b868f0e13b9fe14e2a76c2879f6787b) Co-authored-by: Jan Mazur <16736821+mzr at users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst M Lib/test/test_zipfile.py M Lib/zipfile.py diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py index b7bc218d17a3d..999197a0e32dc 100644 --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -1855,11 +1855,14 @@ def test_comments(self): self.assertEqual(zipf.comment, b"an updated comment") # check that comments are correctly shortened in append mode + # and the file is indeed truncated with zipfile.ZipFile(TESTFN,mode="w") as zipf: zipf.comment = b"original comment that's longer" zipf.writestr("foo.txt", "O, for a Muse of Fire!") + original_zip_size = os.path.getsize(TESTFN) with zipfile.ZipFile(TESTFN,mode="a") as zipf: zipf.comment = b"shorter comment" + self.assertTrue(original_zip_size > os.path.getsize(TESTFN)) with zipfile.ZipFile(TESTFN,mode="r") as zipf: self.assertEqual(zipf.comment, b"shorter comment") diff --git a/Lib/zipfile.py b/Lib/zipfile.py index 915698f9e0588..816f8582bbf6d 100644 --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -1918,6 +1918,8 @@ def _write_end_record(self): centDirSize, centDirOffset, len(self._comment)) self.fp.write(endrec) self.fp.write(self._comment) + if self.mode == "a": + self.fp.truncate() self.fp.flush() def _fpclose(self, fp): diff --git a/Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst b/Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst new file mode 100644 index 0000000000000..f71a7a1e697a4 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-04-03-16-13-59.bpo-40105.hfM2c0.rst @@ -0,0 +1,2 @@ +ZipFile truncates files to avoid corruption when a shorter comment is provided +in append ("a") mode. Patch by Jan Mazur. From webhook-mailer at python.org Mon Sep 28 16:41:33 2020 From: webhook-mailer at python.org (Dong-hee Na) Date: Mon, 28 Sep 2020 20:41:33 -0000 Subject: [Python-checkins] bpo-41875: Use __builtin_unreachable when possible (GH-22433) Message-ID: https://github.com/python/cpython/commit/24ba3b0df5e5f2f237d7b23b4017ba12f16320ae commit: 24ba3b0df5e5f2f237d7b23b4017ba12f16320ae branch: master author: Dong-hee Na committer: GitHub date: 2020-09-29T05:41:23+09:00 summary: bpo-41875: Use __builtin_unreachable when possible (GH-22433) files: A Misc/NEWS.d/next/Build/2020-09-28-21-56-51.bpo-38249.uzMCaZ.rst M Include/pymacro.h diff --git a/Include/pymacro.h b/Include/pymacro.h index 856cae774d61c..202b936d964f0 100644 --- a/Include/pymacro.h +++ b/Include/pymacro.h @@ -118,7 +118,9 @@ "We've reached an unreachable state. Anything is possible.\n" \ "The limits were in our heads all along. Follow your dreams.\n" \ "https://xkcd.com/2200") -#elif defined(__GNUC__) || defined(__clang__) || defined(__INTEL_COMPILER) +#elif defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 5)) +# define Py_UNREACHABLE() __builtin_unreachable() +#elif defined(__clang__) || defined(__INTEL_COMPILER) # define Py_UNREACHABLE() __builtin_unreachable() #elif defined(_MSC_VER) # define Py_UNREACHABLE() __assume(0) diff --git a/Misc/NEWS.d/next/Build/2020-09-28-21-56-51.bpo-38249.uzMCaZ.rst b/Misc/NEWS.d/next/Build/2020-09-28-21-56-51.bpo-38249.uzMCaZ.rst new file mode 100644 index 0000000000000..3e409ec2e7c20 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2020-09-28-21-56-51.bpo-38249.uzMCaZ.rst @@ -0,0 +1,2 @@ +Update :c:macro:`Py_UNREACHABLE` to use __builtin_unreachable() if only the +compiler is able to use it. Patch by Dong-hee Na. From webhook-mailer at python.org Mon Sep 28 17:41:19 2020 From: webhook-mailer at python.org (Hai Shi) Date: Mon, 28 Sep 2020 21:41:19 -0000 Subject: [Python-checkins] bpo-41842: Add codecs.unregister() function (GH-22360) Message-ID: https://github.com/python/cpython/commit/d332e7b8164c3c9c885b9e631f33d9517b628b75 commit: d332e7b8164c3c9c885b9e631f33d9517b628b75 branch: master author: Hai Shi committer: GitHub date: 2020-09-28T23:41:11+02:00 summary: bpo-41842: Add codecs.unregister() function (GH-22360) Add codecs.unregister() and PyCodec_Unregister() functions to unregister a codec search function. files: A Misc/NEWS.d/next/C API/2020-09-27-20-43-16.bpo-41842.bCakAj.rst A Misc/NEWS.d/next/Library/2020-09-23-22-52-24.bpo-41842.lIuhC9.rst M Doc/c-api/codec.rst M Doc/library/codecs.rst M Doc/whatsnew/3.10.rst M Include/codecs.h M Lib/test/test_codecs.py M Misc/ACKS M Modules/_codecsmodule.c M Modules/clinic/_codecsmodule.c.h M Python/codecs.c diff --git a/Doc/c-api/codec.rst b/Doc/c-api/codec.rst index 172dcb326a4bc..235c77c945cc5 100644 --- a/Doc/c-api/codec.rst +++ b/Doc/c-api/codec.rst @@ -10,6 +10,14 @@ Codec registry and support functions As side effect, this tries to load the :mod:`encodings` package, if not yet done, to make sure that it is always first in the list of search functions. +.. c:function:: int PyCodec_Unregister(PyObject *search_function) + + Unregister a codec search function and clear the registry's cache. + If the search function is not registered, do nothing. + Return 0 on success. Raise an exception and return -1 on error. + + .. versionadded:: 3.10 + .. c:function:: int PyCodec_KnownEncoding(const char *encoding) Return ``1`` or ``0`` depending on whether there is a registered codec for diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst index f071057293eec..a026513520590 100644 --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -163,11 +163,14 @@ function: :class:`CodecInfo` object. In case a search function cannot find a given encoding, it should return ``None``. - .. note:: - Search function registration is not currently reversible, - which may cause problems in some cases, such as unit testing or - module reloading. +.. function:: unregister(search_function) + + Unregister a codec search function and clear the registry's cache. + If the search function is not registered, do nothing. + + .. versionadded:: 3.10 + While the builtin :func:`open` and the associated :mod:`io` module are the recommended approach for working with encoded text files, this module diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index 1228f2695496c..f74dd1aa247a3 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -109,6 +109,12 @@ base64 Add :func:`base64.b32hexencode` and :func:`base64.b32hexdecode` to support the Base32 Encoding with Extended Hex Alphabet. +codecs +------ + +Add a :func:`codecs.unregister` function to unregister a codec search function. +(Contributed by Hai Shi in :issue:`41842`.) + curses ------ @@ -237,6 +243,10 @@ New Features :class:`datetime.time` objects. (Contributed by Zackery Spytz in :issue:`30155`.) +* Add a :c:func:`PyCodec_Unregister` function to unregister a codec + search function. + (Contributed by Hai Shi in :issue:`41842`.) + Porting to Python 3.10 ---------------------- diff --git a/Include/codecs.h b/Include/codecs.h index 3ad0f2b5aae79..37ecfb4ab757b 100644 --- a/Include/codecs.h +++ b/Include/codecs.h @@ -27,6 +27,14 @@ PyAPI_FUNC(int) PyCodec_Register( PyObject *search_function ); +/* Unregister a codec search function and clear the registry's cache. + If the search function is not registered, do nothing. + Return 0 on success. Raise an exception and return -1 on error. */ + +PyAPI_FUNC(int) PyCodec_Unregister( + PyObject *search_function + ); + /* Codec registry lookup API. Looks up the given encoding and returns a CodecInfo object with diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py index 3dd56820cd107..ed508f36ad423 100644 --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -1641,6 +1641,18 @@ def test_register(self): self.assertRaises(TypeError, codecs.register) self.assertRaises(TypeError, codecs.register, 42) + def test_unregister(self): + name = "nonexistent_codec_name" + search_function = mock.Mock() + codecs.register(search_function) + self.assertRaises(TypeError, codecs.lookup, name) + search_function.assert_called_with(name) + search_function.reset_mock() + + codecs.unregister(search_function) + self.assertRaises(LookupError, codecs.lookup, name) + search_function.assert_not_called() + def test_lookup(self): self.assertRaises(TypeError, codecs.lookup) self.assertRaises(LookupError, codecs.lookup, "__spam__") diff --git a/Misc/ACKS b/Misc/ACKS index 7b743464c1c1c..85001daf67d23 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1575,6 +1575,7 @@ Akash Shende Charlie Shepherd Bruce Sherwood Gregory Shevchenko +Hai Shi Alexander Shigin Pete Shinners Michael Shiplett diff --git a/Misc/NEWS.d/next/C API/2020-09-27-20-43-16.bpo-41842.bCakAj.rst b/Misc/NEWS.d/next/C API/2020-09-27-20-43-16.bpo-41842.bCakAj.rst new file mode 100644 index 0000000000000..116d08f490859 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2020-09-27-20-43-16.bpo-41842.bCakAj.rst @@ -0,0 +1,2 @@ +Add :c:func:`PyCodec_Unregister` function to unregister a codec search +function. diff --git a/Misc/NEWS.d/next/Library/2020-09-23-22-52-24.bpo-41842.lIuhC9.rst b/Misc/NEWS.d/next/Library/2020-09-23-22-52-24.bpo-41842.lIuhC9.rst new file mode 100644 index 0000000000000..306b02d76fffe --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-23-22-52-24.bpo-41842.lIuhC9.rst @@ -0,0 +1 @@ +Add :func:`codecs.unregister` function to unregister a codec search function. diff --git a/Modules/_codecsmodule.c b/Modules/_codecsmodule.c index 952072102d5d8..08a3d4ab024cc 100644 --- a/Modules/_codecsmodule.c +++ b/Modules/_codecsmodule.c @@ -68,6 +68,27 @@ _codecs_register(PyObject *module, PyObject *search_function) Py_RETURN_NONE; } +/*[clinic input] +_codecs.unregister + search_function: object + / + +Unregister a codec search function and clear the registry's cache. + +If the search function is not registered, do nothing. +[clinic start generated code]*/ + +static PyObject * +_codecs_unregister(PyObject *module, PyObject *search_function) +/*[clinic end generated code: output=1f0edee9cf246399 input=dd7c004c652d345e]*/ +{ + if (PyCodec_Unregister(search_function) < 0) { + return NULL; + } + + Py_RETURN_NONE; +} + /*[clinic input] _codecs.lookup encoding: str @@ -992,6 +1013,7 @@ _codecs_lookup_error_impl(PyObject *module, const char *name) static PyMethodDef _codecs_functions[] = { _CODECS_REGISTER_METHODDEF + _CODECS_UNREGISTER_METHODDEF _CODECS_LOOKUP_METHODDEF _CODECS_ENCODE_METHODDEF _CODECS_DECODE_METHODDEF diff --git a/Modules/clinic/_codecsmodule.c.h b/Modules/clinic/_codecsmodule.c.h index 249065c9fd05a..e2ebb6861299e 100644 --- a/Modules/clinic/_codecsmodule.c.h +++ b/Modules/clinic/_codecsmodule.c.h @@ -15,6 +15,17 @@ PyDoc_STRVAR(_codecs_register__doc__, #define _CODECS_REGISTER_METHODDEF \ {"register", (PyCFunction)_codecs_register, METH_O, _codecs_register__doc__}, +PyDoc_STRVAR(_codecs_unregister__doc__, +"unregister($module, search_function, /)\n" +"--\n" +"\n" +"Unregister a codec search function and clear the registry\'s cache.\n" +"\n" +"If the search function is not registered, do nothing."); + +#define _CODECS_UNREGISTER_METHODDEF \ + {"unregister", (PyCFunction)_codecs_unregister, METH_O, _codecs_unregister__doc__}, + PyDoc_STRVAR(_codecs_lookup__doc__, "lookup($module, encoding, /)\n" "--\n" @@ -2827,4 +2838,4 @@ _codecs_lookup_error(PyObject *module, PyObject *arg) #ifndef _CODECS_CODE_PAGE_ENCODE_METHODDEF #define _CODECS_CODE_PAGE_ENCODE_METHODDEF #endif /* !defined(_CODECS_CODE_PAGE_ENCODE_METHODDEF) */ -/*[clinic end generated code: output=eeead01414be6e42 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=9a97e2ddf3e69072 input=a9049054013a1b77]*/ diff --git a/Python/codecs.c b/Python/codecs.c index 0f18c27e5fe46..a8233a73c4ed3 100644 --- a/Python/codecs.c +++ b/Python/codecs.c @@ -50,6 +50,31 @@ int PyCodec_Register(PyObject *search_function) return -1; } +int +PyCodec_Unregister(PyObject *search_function) +{ + PyInterpreterState *interp = PyInterpreterState_Get(); + PyObject *codec_search_path = interp->codec_search_path; + /* Do nothing if codec_search_path is not created yet or was cleared. */ + if (codec_search_path == NULL) { + return 0; + } + + assert(PyList_CheckExact(codec_search_path)); + Py_ssize_t n = PyList_GET_SIZE(codec_search_path); + for (Py_ssize_t i = 0; i < n; i++) { + PyObject *item = PyList_GET_ITEM(codec_search_path, i); + if (item == search_function) { + if (interp->codec_search_cache != NULL) { + assert(PyDict_CheckExact(interp->codec_search_cache)); + PyDict_Clear(interp->codec_search_cache); + } + return PyList_SetSlice(codec_search_path, i, i+1, NULL); + } + } + return 0; +} + extern int _Py_normalize_encoding(const char *, char *, size_t); /* Convert a string to a normalized Python string(decoded from UTF-8): all characters are From webhook-mailer at python.org Mon Sep 28 18:05:11 2020 From: webhook-mailer at python.org (Erlend Egeberg Aasland) Date: Mon, 28 Sep 2020 22:05:11 -0000 Subject: [Python-checkins] bpo-41861: Convert _sqlite3 PrepareProtocolType to heap type (GH-22428) Message-ID: https://github.com/python/cpython/commit/cb6db8b6ae47dccc1aa97830d0f05d29f31e0cbc commit: cb6db8b6ae47dccc1aa97830d0f05d29f31e0cbc branch: master author: Erlend Egeberg Aasland committer: GitHub date: 2020-09-29T00:05:04+02:00 summary: bpo-41861: Convert _sqlite3 PrepareProtocolType to heap type (GH-22428) files: M Modules/_sqlite/microprotocols.c M Modules/_sqlite/module.c M Modules/_sqlite/prepare_protocol.c M Modules/_sqlite/prepare_protocol.h M Modules/_sqlite/statement.c diff --git a/Modules/_sqlite/microprotocols.c b/Modules/_sqlite/microprotocols.c index 3b2d7f42b8735..64095adb4db2b 100644 --- a/Modules/_sqlite/microprotocols.c +++ b/Modules/_sqlite/microprotocols.c @@ -56,7 +56,7 @@ pysqlite_microprotocols_add(PyTypeObject *type, PyObject *proto, PyObject *cast) PyObject* key; int rc; - if (proto == NULL) proto = (PyObject*)&pysqlite_PrepareProtocolType; + if (proto == NULL) proto = (PyObject*)pysqlite_PrepareProtocolType; key = Py_BuildValue("(OO)", (PyObject*)type, proto); if (!key) { @@ -152,7 +152,7 @@ PyObject * pysqlite_adapt(pysqlite_Cursor *self, PyObject *args) { PyObject *obj, *alt = NULL; - PyObject *proto = (PyObject*)&pysqlite_PrepareProtocolType; + PyObject *proto = (PyObject*)pysqlite_PrepareProtocolType; if (!PyArg_ParseTuple(args, "O|OO", &obj, &proto, &alt)) return NULL; return pysqlite_microprotocols_adapt(obj, proto, alt); diff --git a/Modules/_sqlite/module.c b/Modules/_sqlite/module.c index 625d065a317ba..d0a546c008de2 100644 --- a/Modules/_sqlite/module.c +++ b/Modules/_sqlite/module.c @@ -176,7 +176,7 @@ static PyObject* module_register_adapter(PyObject* self, PyObject* args) pysqlite_BaseTypeAdapted = 1; } - rc = pysqlite_microprotocols_add(type, (PyObject*)&pysqlite_PrepareProtocolType, caster); + rc = pysqlite_microprotocols_add(type, (PyObject*)pysqlite_PrepareProtocolType, caster); if (rc == -1) return NULL; @@ -357,7 +357,7 @@ PyMODINIT_FUNC PyInit__sqlite3(void) (pysqlite_connection_setup_types() < 0) || (pysqlite_cache_setup_types(module) < 0) || (pysqlite_statement_setup_types() < 0) || - (pysqlite_prepare_protocol_setup_types() < 0) + (pysqlite_prepare_protocol_setup_types(module) < 0) ) { Py_XDECREF(module); return NULL; @@ -365,7 +365,7 @@ PyMODINIT_FUNC PyInit__sqlite3(void) ADD_TYPE(module, pysqlite_ConnectionType); ADD_TYPE(module, pysqlite_CursorType); - ADD_TYPE(module, pysqlite_PrepareProtocolType); + ADD_TYPE(module, *pysqlite_PrepareProtocolType); ADD_TYPE(module, pysqlite_RowType); if (!(dict = PyModule_GetDict(module))) { diff --git a/Modules/_sqlite/prepare_protocol.c b/Modules/_sqlite/prepare_protocol.c index 05a2ca5a652f5..7daf8a620096a 100644 --- a/Modules/_sqlite/prepare_protocol.c +++ b/Modules/_sqlite/prepare_protocol.c @@ -30,54 +30,33 @@ int pysqlite_prepare_protocol_init(pysqlite_PrepareProtocol* self, PyObject* arg void pysqlite_prepare_protocol_dealloc(pysqlite_PrepareProtocol* self) { - Py_TYPE(self)->tp_free((PyObject*)self); + PyTypeObject *tp = Py_TYPE(self); + + tp->tp_free(self); + Py_DECREF(tp); } -PyTypeObject pysqlite_PrepareProtocolType= { - PyVarObject_HEAD_INIT(NULL, 0) - MODULE_NAME ".PrepareProtocol", /* tp_name */ - sizeof(pysqlite_PrepareProtocol), /* tp_basicsize */ - 0, /* tp_itemsize */ - (destructor)pysqlite_prepare_protocol_dealloc, /* tp_dealloc */ - 0, /* tp_vectorcall_offset */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_as_async */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT, /* tp_flags */ - 0, /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - 0, /* tp_methods */ - 0, /* tp_members */ - 0, /* tp_getset */ - 0, /* tp_base */ - 0, /* tp_dict */ - 0, /* tp_descr_get */ - 0, /* tp_descr_set */ - 0, /* tp_dictoffset */ - (initproc)pysqlite_prepare_protocol_init, /* tp_init */ - 0, /* tp_alloc */ - 0, /* tp_new */ - 0 /* tp_free */ +static PyType_Slot type_slots[] = { + {Py_tp_dealloc, pysqlite_prepare_protocol_dealloc}, + {Py_tp_new, PyType_GenericNew}, + {Py_tp_init, pysqlite_prepare_protocol_init}, + {0, NULL}, +}; + +static PyType_Spec type_spec = { + .name = MODULE_NAME ".PrepareProtocol", + .basicsize = sizeof(pysqlite_PrepareProtocol), + .flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HEAPTYPE, + .slots = type_slots, }; -extern int pysqlite_prepare_protocol_setup_types(void) +PyTypeObject *pysqlite_PrepareProtocolType = NULL; + +extern int pysqlite_prepare_protocol_setup_types(PyObject *module) { - pysqlite_PrepareProtocolType.tp_new = PyType_GenericNew; - Py_SET_TYPE(&pysqlite_PrepareProtocolType, &PyType_Type); - return PyType_Ready(&pysqlite_PrepareProtocolType); + pysqlite_PrepareProtocolType = (PyTypeObject *)PyType_FromModuleAndSpec(module, &type_spec, NULL); + if (pysqlite_PrepareProtocolType == NULL) { + return -1; + } + return 0; } diff --git a/Modules/_sqlite/prepare_protocol.h b/Modules/_sqlite/prepare_protocol.h index 3998a55e51caf..d0f717c754c1d 100644 --- a/Modules/_sqlite/prepare_protocol.h +++ b/Modules/_sqlite/prepare_protocol.h @@ -31,12 +31,12 @@ typedef struct PyObject_HEAD } pysqlite_PrepareProtocol; -extern PyTypeObject pysqlite_PrepareProtocolType; +extern PyTypeObject *pysqlite_PrepareProtocolType; int pysqlite_prepare_protocol_init(pysqlite_PrepareProtocol* self, PyObject* args, PyObject* kwargs); void pysqlite_prepare_protocol_dealloc(pysqlite_PrepareProtocol* self); -int pysqlite_prepare_protocol_setup_types(void); +int pysqlite_prepare_protocol_setup_types(PyObject *module); #define UNKNOWN (-1) #endif diff --git a/Modules/_sqlite/statement.c b/Modules/_sqlite/statement.c index 02e47a02b718c..0458e1171ebdd 100644 --- a/Modules/_sqlite/statement.c +++ b/Modules/_sqlite/statement.c @@ -255,7 +255,7 @@ void pysqlite_statement_bind_parameters(pysqlite_Statement* self, PyObject* para if (!_need_adapt(current_param)) { adapted = current_param; } else { - adapted = pysqlite_microprotocols_adapt(current_param, (PyObject*)&pysqlite_PrepareProtocolType, current_param); + adapted = pysqlite_microprotocols_adapt(current_param, (PyObject*)pysqlite_PrepareProtocolType, current_param); Py_DECREF(current_param); if (!adapted) { return; @@ -306,7 +306,7 @@ void pysqlite_statement_bind_parameters(pysqlite_Statement* self, PyObject* para if (!_need_adapt(current_param)) { adapted = current_param; } else { - adapted = pysqlite_microprotocols_adapt(current_param, (PyObject*)&pysqlite_PrepareProtocolType, current_param); + adapted = pysqlite_microprotocols_adapt(current_param, (PyObject*)pysqlite_PrepareProtocolType, current_param); Py_DECREF(current_param); if (!adapted) { return; From webhook-mailer at python.org Mon Sep 28 18:16:33 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Mon, 28 Sep 2020 22:16:33 -0000 Subject: [Python-checkins] bpo-41875: Use __builtin_unreachable when possible (GH-22433) Message-ID: https://github.com/python/cpython/commit/cca896e13b230a934fdb709b7f10c5451ffc25ba commit: cca896e13b230a934fdb709b7f10c5451ffc25ba branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-28T15:16:24-07:00 summary: bpo-41875: Use __builtin_unreachable when possible (GH-22433) (cherry picked from commit 24ba3b0df5e5f2f237d7b23b4017ba12f16320ae) Co-authored-by: Dong-hee Na files: A Misc/NEWS.d/next/Build/2020-09-28-21-56-51.bpo-38249.uzMCaZ.rst M Include/pymacro.h diff --git a/Include/pymacro.h b/Include/pymacro.h index 856cae774d61c..202b936d964f0 100644 --- a/Include/pymacro.h +++ b/Include/pymacro.h @@ -118,7 +118,9 @@ "We've reached an unreachable state. Anything is possible.\n" \ "The limits were in our heads all along. Follow your dreams.\n" \ "https://xkcd.com/2200") -#elif defined(__GNUC__) || defined(__clang__) || defined(__INTEL_COMPILER) +#elif defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 5)) +# define Py_UNREACHABLE() __builtin_unreachable() +#elif defined(__clang__) || defined(__INTEL_COMPILER) # define Py_UNREACHABLE() __builtin_unreachable() #elif defined(_MSC_VER) # define Py_UNREACHABLE() __assume(0) diff --git a/Misc/NEWS.d/next/Build/2020-09-28-21-56-51.bpo-38249.uzMCaZ.rst b/Misc/NEWS.d/next/Build/2020-09-28-21-56-51.bpo-38249.uzMCaZ.rst new file mode 100644 index 0000000000000..3e409ec2e7c20 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2020-09-28-21-56-51.bpo-38249.uzMCaZ.rst @@ -0,0 +1,2 @@ +Update :c:macro:`Py_UNREACHABLE` to use __builtin_unreachable() if only the +compiler is able to use it. Patch by Dong-hee Na. From webhook-mailer at python.org Mon Sep 28 20:56:03 2020 From: webhook-mailer at python.org (Dennis Sweeney) Date: Tue, 29 Sep 2020 00:56:03 -0000 Subject: [Python-checkins] bpo-41873: Add vectorcall for float() (GH-22432) Message-ID: https://github.com/python/cpython/commit/e8acc355d430b45f1c3ff83312e72272262a854f commit: e8acc355d430b45f1c3ff83312e72272262a854f branch: master author: Dennis Sweeney <36520290+sweeneyde at users.noreply.github.com> committer: GitHub date: 2020-09-29T09:55:52+09:00 summary: bpo-41873: Add vectorcall for float() (GH-22432) files: A Misc/NEWS.d/next/Core and Builtins/2020-09-28-08-58-28.bpo-41873.VzEDhA.rst M Lib/test/test_float.py M Objects/floatobject.c diff --git a/Lib/test/test_float.py b/Lib/test/test_float.py index 9651281e24edb..99c81f0b72a5a 100644 --- a/Lib/test/test_float.py +++ b/Lib/test/test_float.py @@ -64,6 +64,9 @@ def test_float(self): # See bpo-34087 self.assertRaises(ValueError, float, '\u3053\u3093\u306b\u3061\u306f') + def test_noargs(self): + self.assertEqual(float(), 0.0) + def test_underscores(self): for lit in VALID_UNDERSCORE_LITERALS: if not any(ch in lit for ch in 'jJxXoObB'): diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-09-28-08-58-28.bpo-41873.VzEDhA.rst b/Misc/NEWS.d/next/Core and Builtins/2020-09-28-08-58-28.bpo-41873.VzEDhA.rst new file mode 100644 index 0000000000000..ee2636704c299 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-09-28-08-58-28.bpo-41873.VzEDhA.rst @@ -0,0 +1 @@ +Calls to ``float()`` are now faster due to the ``vectorcall`` calling convention. Patch by Dennis Sweeney. \ No newline at end of file diff --git a/Objects/floatobject.c b/Objects/floatobject.c index 0606f29ff5408..d0af0ea1a9825 100644 --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -1649,6 +1649,24 @@ float_subtype_new(PyTypeObject *type, PyObject *x) return newobj; } +static PyObject * +float_vectorcall(PyObject *type, PyObject * const*args, + size_t nargsf, PyObject *kwnames) +{ + if (!_PyArg_NoKwnames("float", kwnames)) { + return NULL; + } + + Py_ssize_t nargs = PyVectorcall_NARGS(nargsf); + if (!_PyArg_CheckPositional("float", nargs, 0, 1)) { + return NULL; + } + + PyObject *x = nargs >= 1 ? args[0] : _PyLong_Zero; + return float_new_impl((PyTypeObject *)type, x); +} + + /*[clinic input] float.__getnewargs__ [clinic start generated code]*/ @@ -1937,6 +1955,7 @@ PyTypeObject PyFloat_Type = { 0, /* tp_init */ 0, /* tp_alloc */ float_new, /* tp_new */ + .tp_vectorcall = (vectorcallfunc)float_vectorcall, }; int From webhook-mailer at python.org Mon Sep 28 21:32:19 2020 From: webhook-mailer at python.org (Ram Rachum) Date: Tue, 29 Sep 2020 01:32:19 -0000 Subject: [Python-checkins] bpo-41773: Raise exception for non-finite weights in random.choices(). (GH-22441) Message-ID: https://github.com/python/cpython/commit/b0dfc7581697f20385813582de7e92ba6ba0105f commit: b0dfc7581697f20385813582de7e92ba6ba0105f branch: master author: Ram Rachum committer: GitHub date: 2020-09-28T18:32:10-07:00 summary: bpo-41773: Raise exception for non-finite weights in random.choices(). (GH-22441) files: A Misc/NEWS.d/next/Library/2020-09-28-23-22-25.bpo-41773.oKkus0.rst M Doc/library/random.rst M Lib/random.py M Lib/test/test_random.py diff --git a/Doc/library/random.rst b/Doc/library/random.rst index 0cdf0a6ac4a47..af5131df280c2 100644 --- a/Doc/library/random.rst +++ b/Doc/library/random.rst @@ -180,8 +180,8 @@ Functions for sequences The *weights* or *cum_weights* can use any numeric type that interoperates with the :class:`float` values returned by :func:`random` (that includes - integers, floats, and fractions but excludes decimals). Behavior is - undefined if any weight is negative. A :exc:`ValueError` is raised if all + integers, floats, and fractions but excludes decimals). Weights are assumed + to be non-negative and finite. A :exc:`ValueError` is raised if all weights are zero. For a given seed, the :func:`choices` function with equal weighting diff --git a/Lib/random.py b/Lib/random.py index 3ea369b81b3e5..139e8a40bb272 100644 --- a/Lib/random.py +++ b/Lib/random.py @@ -48,7 +48,7 @@ from warnings import warn as _warn from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil from math import sqrt as _sqrt, acos as _acos, cos as _cos, sin as _sin -from math import tau as TWOPI, floor as _floor +from math import tau as TWOPI, floor as _floor, isfinite as _isfinite from os import urandom as _urandom from _collections_abc import Set as _Set, Sequence as _Sequence from itertools import accumulate as _accumulate, repeat as _repeat @@ -492,6 +492,8 @@ def choices(self, population, weights=None, *, cum_weights=None, k=1): total = cum_weights[-1] + 0.0 # convert to float if total <= 0.0: raise ValueError('Total of weights must be greater than zero') + if not _isfinite(total): + raise ValueError('Total of weights must be finite') bisect = _bisect hi = n - 1 return [population[bisect(cum_weights, random() * total, 0, hi)] diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py index a80e71e67e4c6..0c1fdeec9915e 100644 --- a/Lib/test/test_random.py +++ b/Lib/test/test_random.py @@ -324,6 +324,22 @@ def test_choices_with_all_zero_weights(self): with self.assertRaises(ValueError): self.gen.choices('AB', [0.0, 0.0]) + def test_choices_negative_total(self): + with self.assertRaises(ValueError): + self.gen.choices('ABC', [3, -5, 1]) + + def test_choices_infinite_total(self): + with self.assertRaises(ValueError): + self.gen.choices('A', [float('inf')]) + with self.assertRaises(ValueError): + self.gen.choices('AB', [0.0, float('inf')]) + with self.assertRaises(ValueError): + self.gen.choices('AB', [-float('inf'), 123]) + with self.assertRaises(ValueError): + self.gen.choices('AB', [0.0, float('nan')]) + with self.assertRaises(ValueError): + self.gen.choices('AB', [float('-inf'), float('inf')]) + def test_gauss(self): # Ensure that the seed() method initializes all the hidden state. In # particular, through 2.2.1 it failed to reset a piece of state used diff --git a/Misc/NEWS.d/next/Library/2020-09-28-23-22-25.bpo-41773.oKkus0.rst b/Misc/NEWS.d/next/Library/2020-09-28-23-22-25.bpo-41773.oKkus0.rst new file mode 100644 index 0000000000000..cef7ff0188354 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-09-28-23-22-25.bpo-41773.oKkus0.rst @@ -0,0 +1,2 @@ +Note in documentation that :func:`random.choices` doesn't support non-finite +weights, raise :exc:`ValueError` when given non-finite weights. From webhook-mailer at python.org Tue Sep 29 01:02:55 2020 From: webhook-mailer at python.org (Terry Jan Reedy) Date: Tue, 29 Sep 2020 05:02:55 -0000 Subject: [Python-checkins] bpo-41774: Add programming FAQ entry (GH-22402) Message-ID: https://github.com/python/cpython/commit/5b0181d1f6474c2cb9b80bdaf3bc56a78bf5fbe7 commit: 5b0181d1f6474c2cb9b80bdaf3bc56a78bf5fbe7 branch: master author: Terry Jan Reedy committer: GitHub date: 2020-09-29T01:02:44-04:00 summary: bpo-41774: Add programming FAQ entry (GH-22402) In the "Sequences (Tuples/Lists)" section, add "How do you remove multiple items from a list". files: A Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 4f4ea8b18176c..76ae4d260fad4 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1164,6 +1164,21 @@ This converts the list into a set, thereby removing duplicates, and then back into a list. +How do you remove multiple items from a list +-------------------------------------------- + +As with removing duplicates, explicitly iterating in reverse with a +delete condition is one possibility. However, it is easier and faster +to use slice replacement with an implicit or explicit forward iteration. +Here are three variations.:: + + mylist[:] = filter(keep_function, mylist) + mylist[:] = (x for x in mylist if keep_condition) + mylist[:] = [x for x in mylist if keep_condition] + +If space is not an issue, the list comprehension may be fastest. + + How do you make an array in Python? ----------------------------------- diff --git a/Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst b/Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst new file mode 100644 index 0000000000000..af8e02437cb2b --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst @@ -0,0 +1,2 @@ +In Programming FAQ "Sequences (Tuples/Lists)" section, add "How do you +remove multiple items from a list". From webhook-mailer at python.org Tue Sep 29 01:11:15 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 29 Sep 2020 05:11:15 -0000 Subject: [Python-checkins] bpo-41774: Add programming FAQ entry (GH-22402) Message-ID: https://github.com/python/cpython/commit/d50a0700265536a20bcce3fb108c954746d97625 commit: d50a0700265536a20bcce3fb108c954746d97625 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-28T22:11:06-07:00 summary: bpo-41774: Add programming FAQ entry (GH-22402) In the "Sequences (Tuples/Lists)" section, add "How do you remove multiple items from a list". (cherry picked from commit 5b0181d1f6474c2cb9b80bdaf3bc56a78bf5fbe7) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 70e9190e5a520..1af04483eb329 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1163,6 +1163,21 @@ This converts the list into a set, thereby removing duplicates, and then back into a list. +How do you remove multiple items from a list +-------------------------------------------- + +As with removing duplicates, explicitly iterating in reverse with a +delete condition is one possibility. However, it is easier and faster +to use slice replacement with an implicit or explicit forward iteration. +Here are three variations.:: + + mylist[:] = filter(keep_function, mylist) + mylist[:] = (x for x in mylist if keep_condition) + mylist[:] = [x for x in mylist if keep_condition] + +If space is not an issue, the list comprehension may be fastest. + + How do you make an array in Python? ----------------------------------- diff --git a/Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst b/Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst new file mode 100644 index 0000000000000..af8e02437cb2b --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst @@ -0,0 +1,2 @@ +In Programming FAQ "Sequences (Tuples/Lists)" section, add "How do you +remove multiple items from a list". From webhook-mailer at python.org Tue Sep 29 01:27:15 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Tue, 29 Sep 2020 05:27:15 -0000 Subject: [Python-checkins] bpo-41774: Add programming FAQ entry (GH-22402) Message-ID: https://github.com/python/cpython/commit/868c8e41eb1d7dc032679ae06e62c0d60edd7725 commit: 868c8e41eb1d7dc032679ae06e62c0d60edd7725 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-28T22:27:06-07:00 summary: bpo-41774: Add programming FAQ entry (GH-22402) In the "Sequences (Tuples/Lists)" section, add "How do you remove multiple items from a list". (cherry picked from commit 5b0181d1f6474c2cb9b80bdaf3bc56a78bf5fbe7) Co-authored-by: Terry Jan Reedy files: A Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst M Doc/faq/programming.rst diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 66d210a55fac7..1435765063f73 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1164,6 +1164,21 @@ This converts the list into a set, thereby removing duplicates, and then back into a list. +How do you remove multiple items from a list +-------------------------------------------- + +As with removing duplicates, explicitly iterating in reverse with a +delete condition is one possibility. However, it is easier and faster +to use slice replacement with an implicit or explicit forward iteration. +Here are three variations.:: + + mylist[:] = filter(keep_function, mylist) + mylist[:] = (x for x in mylist if keep_condition) + mylist[:] = [x for x in mylist if keep_condition] + +If space is not an issue, the list comprehension may be fastest. + + How do you make an array in Python? ----------------------------------- diff --git a/Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst b/Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst new file mode 100644 index 0000000000000..af8e02437cb2b --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-09-24-15-35-13.bpo-41774.5IqdGP.rst @@ -0,0 +1,2 @@ +In Programming FAQ "Sequences (Tuples/Lists)" section, add "How do you +remove multiple items from a list". From webhook-mailer at python.org Tue Sep 29 05:09:31 2020 From: webhook-mailer at python.org (Mark Shannon) Date: Tue, 29 Sep 2020 09:09:31 -0000 Subject: [Python-checkins] bpo-41670: Remove outdated predict macro invocation. (GH-22026) Message-ID: https://github.com/python/cpython/commit/17b5be0c0a3f74141014e06a660f1b5ddb002fec commit: 17b5be0c0a3f74141014e06a660f1b5ddb002fec branch: master author: Mark Shannon committer: GitHub date: 2020-09-29T10:09:13+01:00 summary: bpo-41670: Remove outdated predict macro invocation. (GH-22026) Remove PREDICTion of POP_BLOCK from FOR_ITER. files: A Misc/NEWS.d/next/Core and Builtins/2020-08-31-11-37-59.bpo-41670.vmRJRx.rst M Lib/test/test_sys_settrace.py M Python/ceval.c diff --git a/Lib/test/test_sys_settrace.py b/Lib/test/test_sys_settrace.py index 3f902b1fe74ce..dd4418dd98b22 100644 --- a/Lib/test/test_sys_settrace.py +++ b/Lib/test/test_sys_settrace.py @@ -602,6 +602,23 @@ def run(tracer): self.compare_events(doit_async.__code__.co_firstlineno, tracer.events, events) + def test_loop_in_try_except(self): + # https://bugs.python.org/issue41670 + + def func(): + try: + for i in []: pass + return 1 + except: + return 2 + + self.run_and_compare(func, + [(0, 'call'), + (1, 'line'), + (2, 'line'), + (3, 'line'), + (3, 'return')]) + class SkipLineEventsTraceTestCase(TraceTestCase): """Repeat the trace tests, but with per-line events skipped""" diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-08-31-11-37-59.bpo-41670.vmRJRx.rst b/Misc/NEWS.d/next/Core and Builtins/2020-08-31-11-37-59.bpo-41670.vmRJRx.rst new file mode 100644 index 0000000000000..6ad5fb6dc9bb4 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-08-31-11-37-59.bpo-41670.vmRJRx.rst @@ -0,0 +1,4 @@ +Prevent line trace being skipped on platforms not compiled +with ``USE_COMPUTED_GOTOS``. +Fixes issue where some lines nested within a try-except block +were not being traced on Windows. diff --git a/Python/ceval.c b/Python/ceval.c index 6430e792b8c5d..6bd2d6bc13d86 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2311,7 +2311,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag) } case TARGET(POP_BLOCK): { - PREDICTED(POP_BLOCK); PyFrame_BlockPop(f); DISPATCH(); } @@ -3366,7 +3365,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag) STACK_SHRINK(1); Py_DECREF(iter); JUMPBY(oparg); - PREDICT(POP_BLOCK); DISPATCH(); } From webhook-mailer at python.org Wed Sep 30 18:06:00 2020 From: webhook-mailer at python.org (Max Smolens) Date: Wed, 30 Sep 2020 22:06:00 -0000 Subject: [Python-checkins] Fix grammar in secrets module documentation (GH-22467) Message-ID: https://github.com/python/cpython/commit/63298930fb531ba2bb4f23bc3b915dbf1e17e9e1 commit: 63298930fb531ba2bb4f23bc3b915dbf1e17e9e1 branch: master author: Max Smolens committer: GitHub date: 2020-09-30T15:05:51-07:00 summary: Fix grammar in secrets module documentation (GH-22467) >From `In particularly,` to `In particular,` files: M Doc/library/secrets.rst diff --git a/Doc/library/secrets.rst b/Doc/library/secrets.rst index bc4766da2785b..afa8e2d385fa4 100644 --- a/Doc/library/secrets.rst +++ b/Doc/library/secrets.rst @@ -21,7 +21,7 @@ The :mod:`secrets` module is used for generating cryptographically strong random numbers suitable for managing data such as passwords, account authentication, security tokens, and related secrets. -In particularly, :mod:`secrets` should be used in preference to the +In particular, :mod:`secrets` should be used in preference to the default pseudo-random number generator in the :mod:`random` module, which is designed for modelling and simulation, not security or cryptography. From webhook-mailer at python.org Wed Sep 30 18:50:05 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 30 Sep 2020 22:50:05 -0000 Subject: [Python-checkins] Fix grammar in secrets module documentation (GH-22467) Message-ID: https://github.com/python/cpython/commit/8fc828ec4e867c111cef137a2daeea6c3d5db2c7 commit: 8fc828ec4e867c111cef137a2daeea6c3d5db2c7 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-30T15:49:52-07:00 summary: Fix grammar in secrets module documentation (GH-22467) >From `In particularly,` to `In particular,` (cherry picked from commit 63298930fb531ba2bb4f23bc3b915dbf1e17e9e1) Co-authored-by: Max Smolens files: M Doc/library/secrets.rst diff --git a/Doc/library/secrets.rst b/Doc/library/secrets.rst index bc4766da2785b..afa8e2d385fa4 100644 --- a/Doc/library/secrets.rst +++ b/Doc/library/secrets.rst @@ -21,7 +21,7 @@ The :mod:`secrets` module is used for generating cryptographically strong random numbers suitable for managing data such as passwords, account authentication, security tokens, and related secrets. -In particularly, :mod:`secrets` should be used in preference to the +In particular, :mod:`secrets` should be used in preference to the default pseudo-random number generator in the :mod:`random` module, which is designed for modelling and simulation, not security or cryptography. From webhook-mailer at python.org Wed Sep 30 18:50:12 2020 From: webhook-mailer at python.org (Miss Islington (bot)) Date: Wed, 30 Sep 2020 22:50:12 -0000 Subject: [Python-checkins] Fix grammar in secrets module documentation (GH-22467) Message-ID: https://github.com/python/cpython/commit/4c2e299d80c53591f05de2669c0edeaf8acd8544 commit: 4c2e299d80c53591f05de2669c0edeaf8acd8544 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com> committer: GitHub date: 2020-09-30T15:50:07-07:00 summary: Fix grammar in secrets module documentation (GH-22467) >From `In particularly,` to `In particular,` (cherry picked from commit 63298930fb531ba2bb4f23bc3b915dbf1e17e9e1) Co-authored-by: Max Smolens files: M Doc/library/secrets.rst diff --git a/Doc/library/secrets.rst b/Doc/library/secrets.rst index bc4766da2785b..afa8e2d385fa4 100644 --- a/Doc/library/secrets.rst +++ b/Doc/library/secrets.rst @@ -21,7 +21,7 @@ The :mod:`secrets` module is used for generating cryptographically strong random numbers suitable for managing data such as passwords, account authentication, security tokens, and related secrets. -In particularly, :mod:`secrets` should be used in preference to the +In particular, :mod:`secrets` should be used in preference to the default pseudo-random number generator in the :mod:`random` module, which is designed for modelling and simulation, not security or cryptography.