Mailman 3 February 2020 - Python-checkins

bpo-39576: Clarify the word size for the 32-bit build. (GH-18616) (#18617)
by Miss Islington (bot) 23 Feb '20

23 Feb '20

https://github.com/python/cpython/commit/24c570bbb82a7cb70576c253a73390accf… commit: 24c570bbb82a7cb70576c253a73390accfa7ed78 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2020-02-23T14:42:57+01:00 summary: bpo-39576: Clarify the word size for the 32-bit build. (GH-18616) (#18617) (cherry picked from commit b76518d43fb82ed9e5d27025d18c90a23d525c90) Authored-by: Stefan Krah <skrah(a)bytereef.org> files: M Doc/library/decimal.rst diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst index 2a51429bdff5c..4e640cc695990 100644 --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -2155,8 +2155,8 @@ RAM and expect 10 simultaneous operands using a maximum of 500MB each:: >>> import sys >>> - >>> # Maximum number of digits for a single operand using 500MB in 8 byte words - >>> # with 19 (9 for the 32-bit version) digits per word: + >>> # Maximum number of digits for a single operand using 500MB in 8-byte words + >>> # with 19 digits per word (4-byte and 9 digits for the 32-bit build): >>> maxdigits = 19 * ((500 * 1024**2) // 8) >>> >>> # Check that this works:

1 0

bpo-39576: Clarify the word size for the 32-bit build. (#18616)
by Stefan Krah 23 Feb '20

23 Feb '20

https://github.com/python/cpython/commit/b76518d43fb82ed9e5d27025d18c90a23d… commit: b76518d43fb82ed9e5d27025d18c90a23d525c90 branch: master author: Stefan Krah <skrah(a)bytereef.org> committer: GitHub <noreply(a)github.com> date: 2020-02-23T14:36:54+01:00 summary: bpo-39576: Clarify the word size for the 32-bit build. (#18616) files: M Doc/library/decimal.rst diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst index 2a51429bdff5c..4e640cc695990 100644 --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -2155,8 +2155,8 @@ RAM and expect 10 simultaneous operands using a maximum of 500MB each:: >>> import sys >>> - >>> # Maximum number of digits for a single operand using 500MB in 8 byte words - >>> # with 19 (9 for the 32-bit version) digits per word: + >>> # Maximum number of digits for a single operand using 500MB in 8-byte words + >>> # with 19 digits per word (4-byte and 9 digits for the 32-bit build): >>> maxdigits = 19 * ((500 * 1024**2) // 8) >>> >>> # Check that this works:

1 0

bpo-39648: Expand math.gcd() and math.lcm() to handle multiple arguments. (GH-18604)
by Serhiy Storchaka 23 Feb '20

23 Feb '20

https://github.com/python/cpython/commit/559e7f165ad03731e6bc2211c0e6d8d9c0… commit: 559e7f165ad03731e6bc2211c0e6d8d9c02fb549 branch: master author: Serhiy Storchaka <storchaka(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2020-02-23T11:21:29Z summary: bpo-39648: Expand math.gcd() and math.lcm() to handle multiple arguments. (GH-18604) * bpo-39648: Expand math.gcd() and math.lcm() to handle multiple arguments. * Simplify fast path. * Difine lcm() without arguments returning 1. * Apply suggestions from code review Co-Authored-By: Mark Dickinson <dickinsm(a)gmail.com> Co-authored-by: Mark Dickinson <dickinsm(a)gmail.com> files: A Misc/NEWS.d/next/Library/2020-02-22-12-49-04.bpo-39648.Y-9N7F.rst M Doc/library/math.rst M Doc/whatsnew/3.9.rst M Lib/test/test_math.py M Modules/clinic/mathmodule.c.h M Modules/mathmodule.c diff --git a/Doc/library/math.rst b/Doc/library/math.rst index d8ac35256d1b4..6ec1feee35a6d 100644 --- a/Doc/library/math.rst +++ b/Doc/library/math.rst @@ -126,23 +126,19 @@ Number-theoretic and representation functions <https://code.activestate.com/recipes/393090/>`_\. -.. function:: gcd(a, b) +.. function:: gcd(*integers) - Return the greatest common divisor of the integers *a* and *b*. If either - *a* or *b* is nonzero, then the value of ``gcd(a, b)`` is the largest - positive integer that divides both *a* and *b*. ``gcd(0, 0)`` returns - ``0``. + Return the greatest common divisor of the specified integer arguments. + If any of the arguments is nonzero, then the returned value is the largest + positive integer that is a divisor af all arguments. If all arguments + are zero, then the returned value is ``0``. ``gcd()`` without arguments + returns ``0``. .. versionadded:: 3.5 - -.. function:: lcm(a, b) - - Return the least common multiple of integers *a* and *b*. The value of - ``lcm(a, b)`` is the smallest nonnegative integer that is a multiple of - both *a* and *b*. If either *a* or *b* is zero then ``lcm(a, b)`` is zero. - - .. versionadded:: 3.9 + .. versionchanged:: 3.9 + Added support for an arbitrary number of arguments. Formerly, only two + arguments were supported. .. function:: isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0) @@ -210,6 +206,17 @@ Number-theoretic and representation functions .. versionadded:: 3.8 +.. function:: lcm(*integers) + + Return the least common multiple of the specified integer arguments. + If all arguments are nonzero, then the returned value is the smallest + positive integer that is a multiple of all arguments. If any of the arguments + is zero, then the returned value is ``0``. ``lcm()`` without arguments + returns ``1``. + + .. versionadded:: 3.9 + + .. function:: ldexp(x, i) Return ``x * (2**i)``. This is essentially the inverse of function diff --git a/Doc/whatsnew/3.9.rst b/Doc/whatsnew/3.9.rst index 161675d32f670..a0ae4eb9b825a 100644 --- a/Doc/whatsnew/3.9.rst +++ b/Doc/whatsnew/3.9.rst @@ -216,8 +216,13 @@ import attempts. math ---- -Add :func:`math.lcm`: return the least common multiple of *a* and *b*. -(Contributed by Ananthakrishnan in :issue:`39479`.) +Expanded the :func:`math.gcd` function to handle multiple arguments. +Formerly, it only supported two arguments. +(Contributed by Serhiy Storchaka in :issue:`39648`.) + +Add :func:`math.lcm`: return the least common multiple of specified arguments. +(Contributed by Mark Dickinson, Ananthakrishnan and Serhiy Storchaka in +:issue:`39479` and :issue:`39648`.) Add :func:`math.nextafter`: return the next floating-point value after *x* towards *y*. diff --git a/Lib/test/test_math.py b/Lib/test/test_math.py index ad8273d50cc39..cc39402b3c60b 100644 --- a/Lib/test/test_math.py +++ b/Lib/test/test_math.py @@ -705,33 +705,32 @@ def testGcd(self): self.assertEqual(gcd(84, -120), 12) self.assertEqual(gcd(1216342683557601535506311712, 436522681849110124616458784), 32) - c = 652560 + x = 434610456570399902378880679233098819019853229470286994367836600566 y = 1064502245825115327754847244914921553977 - a = x * c - b = y * c - self.assertEqual(gcd(a, b), c) - self.assertEqual(gcd(b, a), c) - self.assertEqual(gcd(-a, b), c) - self.assertEqual(gcd(b, -a), c) - self.assertEqual(gcd(a, -b), c) - self.assertEqual(gcd(-b, a), c) - self.assertEqual(gcd(-a, -b), c) - self.assertEqual(gcd(-b, -a), c) - c = 576559230871654959816130551884856912003141446781646602790216406874 - a = x * c - b = y * c - self.assertEqual(gcd(a, b), c) - self.assertEqual(gcd(b, a), c) - self.assertEqual(gcd(-a, b), c) - self.assertEqual(gcd(b, -a), c) - self.assertEqual(gcd(a, -b), c) - self.assertEqual(gcd(-b, a), c) - self.assertEqual(gcd(-a, -b), c) - self.assertEqual(gcd(-b, -a), c) - + for c in (652560, + 576559230871654959816130551884856912003141446781646602790216406874): + a = x * c + b = y * c + self.assertEqual(gcd(a, b), c) + self.assertEqual(gcd(b, a), c) + self.assertEqual(gcd(-a, b), c) + self.assertEqual(gcd(b, -a), c) + self.assertEqual(gcd(a, -b), c) + self.assertEqual(gcd(-b, a), c) + self.assertEqual(gcd(-a, -b), c) + self.assertEqual(gcd(-b, -a), c) + + self.assertEqual(gcd(), 0) + self.assertEqual(gcd(120), 120) + self.assertEqual(gcd(-120), 120) + self.assertEqual(gcd(120, 84, 102), 6) + self.assertEqual(gcd(120, 1, 84), 1) + + self.assertRaises(TypeError, gcd, 120.0) self.assertRaises(TypeError, gcd, 120.0, 84) self.assertRaises(TypeError, gcd, 120, 84.0) + self.assertRaises(TypeError, gcd, 120, 1, 84.0) self.assertEqual(gcd(MyIndexable(120), MyIndexable(84)), 12) def testHypot(self): @@ -989,9 +988,9 @@ def test_lcm(self): self.assertEqual(lcm(1216342683557601535506311712, 436522681849110124616458784), 16592536571065866494401400422922201534178938447014944) + x = 43461045657039990237 y = 10645022458251153277 - for c in (652560, 57655923087165495981): a = x * c @@ -1005,9 +1004,18 @@ def test_lcm(self): self.assertEqual(lcm(-b, a), d) self.assertEqual(lcm(-a, -b), d) self.assertEqual(lcm(-b, -a), d) - self.assertEqual(lcm(MyIndexable(120), MyIndexable(84)), 840) + + self.assertEqual(lcm(), 1) + self.assertEqual(lcm(120), 120) + self.assertEqual(lcm(-120), 120) + self.assertEqual(lcm(120, 84, 102), 14280) + self.assertEqual(lcm(120, 0, 84), 0) + + self.assertRaises(TypeError, lcm, 120.0) self.assertRaises(TypeError, lcm, 120.0, 84) self.assertRaises(TypeError, lcm, 120, 84.0) + self.assertRaises(TypeError, lcm, 120, 0, 84.0) + self.assertEqual(lcm(MyIndexable(120), MyIndexable(84)), 840) def testLdexp(self): self.assertRaises(TypeError, math.ldexp) diff --git a/Misc/NEWS.d/next/Library/2020-02-22-12-49-04.bpo-39648.Y-9N7F.rst b/Misc/NEWS.d/next/Library/2020-02-22-12-49-04.bpo-39648.Y-9N7F.rst new file mode 100644 index 0000000000000..f205911ad9bfd --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-02-22-12-49-04.bpo-39648.Y-9N7F.rst @@ -0,0 +1 @@ +Expanded :func:`math.gcd` and :func:`math.lcm` to handle multiple arguments. diff --git a/Modules/clinic/mathmodule.c.h b/Modules/clinic/mathmodule.c.h index df45a1a0c5e85..65f3dd4f520ae 100644 --- a/Modules/clinic/mathmodule.c.h +++ b/Modules/clinic/mathmodule.c.h @@ -2,36 +2,6 @@ preserve [clinic start generated code]*/ -PyDoc_STRVAR(math_gcd__doc__, -"gcd($module, x, y, /)\n" -"--\n" -"\n" -"greatest common divisor of x and y"); - -#define MATH_GCD_METHODDEF \ - {"gcd", (PyCFunction)(void(*)(void))math_gcd, METH_FASTCALL, math_gcd__doc__}, - -static PyObject * -math_gcd_impl(PyObject *module, PyObject *a, PyObject *b); - -static PyObject * -math_gcd(PyObject *module, PyObject *const *args, Py_ssize_t nargs) -{ - PyObject *return_value = NULL; - PyObject *a; - PyObject *b; - - if (!_PyArg_CheckPositional("gcd", nargs, 2, 2)) { - goto exit; - } - a = args[0]; - b = args[1]; - return_value = math_gcd_impl(module, a, b); - -exit: - return return_value; -} - PyDoc_STRVAR(math_ceil__doc__, "ceil($module, x, /)\n" "--\n" @@ -85,36 +55,6 @@ PyDoc_STRVAR(math_factorial__doc__, #define MATH_FACTORIAL_METHODDEF \ {"factorial", (PyCFunction)math_factorial, METH_O, math_factorial__doc__}, -PyDoc_STRVAR(math_lcm__doc__, -"lcm($module, x, y, /)\n" -"--\n" -"\n" -"least common multiple of x and y"); - -#define MATH_LCM_METHODDEF \ - {"lcm", (PyCFunction)(void(*)(void))math_lcm, METH_FASTCALL, math_lcm__doc__}, - -static PyObject * -math_lcm_impl(PyObject *module, PyObject *a, PyObject *b); - -static PyObject * -math_lcm(PyObject *module, PyObject *const *args, Py_ssize_t nargs) -{ - PyObject *return_value = NULL; - PyObject *a; - PyObject *b; - - if (!_PyArg_CheckPositional("lcm", nargs, 2, 2)) { - goto exit; - } - a = args[0]; - b = args[1]; - return_value = math_lcm_impl(module, a, b); - -exit: - return return_value; -} - PyDoc_STRVAR(math_trunc__doc__, "trunc($module, x, /)\n" "--\n" @@ -925,4 +865,4 @@ math_ulp(PyObject *module, PyObject *arg) exit: return return_value; } -/*[clinic end generated code: output=f8daa185c043a7b7 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=1eae2b3ef19568fa input=a9049054013a1b77]*/ diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index f74b7e1a34203..77e325cf0c63d 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -826,37 +826,125 @@ m_log10(double x) } -/*[clinic input] -math.gcd +static PyObject * +math_gcd(PyObject *module, PyObject * const *args, Py_ssize_t nargs) +{ + PyObject *res, *x; + Py_ssize_t i; - x as a: object - y as b: object - / + if (nargs == 0) { + return PyLong_FromLong(0); + } + res = PyNumber_Index(args[0]); + if (res == NULL) { + return NULL; + } + if (nargs == 1) { + Py_SETREF(res, PyNumber_Absolute(res)); + return res; + } + for (i = 1; i < nargs; i++) { + x = PyNumber_Index(args[i]); + if (x == NULL) { + Py_DECREF(res); + return NULL; + } + if (res == _PyLong_One) { + /* Fast path: just check arguments. + It is okay to use identity comparison here. */ + Py_DECREF(x); + continue; + } + Py_SETREF(res, _PyLong_GCD(res, x)); + Py_DECREF(x); + if (res == NULL) { + return NULL; + } + } + return res; +} + +PyDoc_STRVAR(math_gcd_doc, +"gcd($module, *integers)\n" +"--\n" +"\n" +"Greatest Common Divisor."); -greatest common divisor of x and y -[clinic start generated code]*/ static PyObject * -math_gcd_impl(PyObject *module, PyObject *a, PyObject *b) -/*[clinic end generated code: output=7b2e0c151bd7a5d8 input=c2691e57fb2a98fa]*/ +long_lcm(PyObject *a, PyObject *b) { - PyObject *g; + PyObject *g, *m, *f, *ab; - a = PyNumber_Index(a); - if (a == NULL) + if (Py_SIZE(a) == 0 || Py_SIZE(b) == 0) { + return PyLong_FromLong(0); + } + g = _PyLong_GCD(a, b); + if (g == NULL) { return NULL; - b = PyNumber_Index(b); - if (b == NULL) { - Py_DECREF(a); + } + f = PyNumber_FloorDivide(a, g); + Py_DECREF(g); + if (f == NULL) { return NULL; } - g = _PyLong_GCD(a, b); - Py_DECREF(a); - Py_DECREF(b); - return g; + m = PyNumber_Multiply(f, b); + Py_DECREF(f); + if (m == NULL) { + return NULL; + } + ab = PyNumber_Absolute(m); + Py_DECREF(m); + return ab; } +static PyObject * +math_lcm(PyObject *module, PyObject * const *args, Py_ssize_t nargs) +{ + PyObject *res, *x; + Py_ssize_t i; + + if (nargs == 0) { + return PyLong_FromLong(1); + } + res = PyNumber_Index(args[0]); + if (res == NULL) { + return NULL; + } + if (nargs == 1) { + Py_SETREF(res, PyNumber_Absolute(res)); + return res; + } + for (i = 1; i < nargs; i++) { + x = PyNumber_Index(args[i]); + if (x == NULL) { + Py_DECREF(res); + return NULL; + } + if (res == _PyLong_Zero) { + /* Fast path: just check arguments. + It is okay to use identity comparison here. */ + Py_DECREF(x); + continue; + } + Py_SETREF(res, long_lcm(res, x)); + Py_DECREF(x); + if (res == NULL) { + return NULL; + } + } + return res; +} + + +PyDoc_STRVAR(math_lcm_doc, +"lcm($module, *integers)\n" +"--\n" +"\n" +"Least Common Multiple."); + + /* Call is_error when errno != 0, and where x is the result libm * returned. is_error will usually set up an exception and return * true (1), but may return false (0) without setting up an exception. @@ -2017,59 +2105,6 @@ math_factorial(PyObject *module, PyObject *arg) } -/*[clinic input] -math.lcm - x as a: object - y as b: object - / -least common multiple of x and y -[clinic start generated code]*/ - -static PyObject * -math_lcm_impl(PyObject *module, PyObject *a, PyObject *b) -/*[clinic end generated code: output=6f83fb6d671074ba input=efb3d7b7334b7118]*/ -{ - PyObject *g, *m, *f, *ab; - - a = PyNumber_Index(a); - if (a == NULL) { - return NULL; - } - b = PyNumber_Index(b); - if (b == NULL) { - Py_DECREF(a); - return NULL; - } - if (_PyLong_Sign(a) == 0 || _PyLong_Sign(b) == 0) { - Py_DECREF(a); - Py_DECREF(b); - return PyLong_FromLong(0); - } - g = _PyLong_GCD(a, b); - if (g == NULL) { - Py_DECREF(a); - Py_DECREF(b); - return NULL; - } - f = PyNumber_FloorDivide(a, g); - Py_DECREF(g); - Py_DECREF(a); - if (f == NULL) { - Py_DECREF(b); - return NULL; - } - m = PyNumber_Multiply(f, b); - Py_DECREF(f); - Py_DECREF(b); - if (m == NULL) { - return NULL; - } - ab = PyNumber_Absolute(m); - Py_DECREF(m); - return ab; -} - - /*[clinic input] math.trunc @@ -3408,14 +3443,14 @@ static PyMethodDef math_methods[] = { MATH_FREXP_METHODDEF MATH_FSUM_METHODDEF {"gamma", math_gamma, METH_O, math_gamma_doc}, - MATH_GCD_METHODDEF + {"gcd", (PyCFunction)(void(*)(void))math_gcd, METH_FASTCALL, math_gcd_doc}, {"hypot", (PyCFunction)(void(*)(void))math_hypot, METH_FASTCALL, math_hypot_doc}, MATH_ISCLOSE_METHODDEF MATH_ISFINITE_METHODDEF MATH_ISINF_METHODDEF MATH_ISNAN_METHODDEF MATH_ISQRT_METHODDEF - MATH_LCM_METHODDEF + {"lcm", (PyCFunction)(void(*)(void))math_lcm, METH_FASTCALL, math_lcm_doc}, MATH_LDEXP_METHODDEF {"lgamma", math_lgamma, METH_O, math_lgamma_doc}, MATH_LOG_METHODDEF

1 0

bpo-17422: Language reference should specify restrictions on class namespace (GH-18559)
by Miss Islington (bot) 22 Feb '20

22 Feb '20

https://github.com/python/cpython/commit/36a120d343c1cb69b8d02913569c153e8b… commit: 36a120d343c1cb69b8d02913569c153e8bb969c8 branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2020-02-22T13:16:43-08:00 summary: bpo-17422: Language reference should specify restrictions on class namespace (GH-18559) The language reference now specifies restrictions on class namespaces. Adapted from a patch by Ethan Furman. (cherry picked from commit fbe2e0bb8a7ee75d0f9d57682436dac7d69e202e) Co-authored-by: ananthan-123 <ananthakrishnan15.2001(a)gmail.com> files: A Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst M Doc/reference/datamodel.rst diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index d25357db43468..b682465d3ddc8 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1932,10 +1932,12 @@ Once the appropriate metaclass has been identified, then the class namespace is prepared. If the metaclass has a ``__prepare__`` attribute, it is called as ``namespace = metaclass.__prepare__(name, bases, **kwds)`` (where the additional keyword arguments, if any, come from the class definition). The -``__prepare__`` method should be implemented as a :func:`classmethod`. +``__prepare__`` method should be implemented as a :func:`classmethod`. The +namespace returned by ``__prepare__`` is passed in to ``__new__``, but when +the final class object is created the namespace is copied into a new ``dict``. If the metaclass has no ``__prepare__`` attribute, then the class namespace -is initialised as an empty ordered mapping. +is initialised as an empty :func:`dict`. .. seealso:: diff --git a/Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst b/Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst new file mode 100644 index 0000000000000..f071d286176ae --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst @@ -0,0 +1 @@ +The language reference now specifies restrictions on class namespaces. Adapted from a patch by Ethan Furman.

1 0

bpo-17422: Language reference should specify restrictions on class namespace (GH-18559)
by Miss Islington (bot) 22 Feb '20

22 Feb '20

https://github.com/python/cpython/commit/91ba44623e5cb8f4a7b0d3fc1fab8ff8d8… commit: 91ba44623e5cb8f4a7b0d3fc1fab8ff8d8a849c2 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2020-02-22T13:16:17-08:00 summary: bpo-17422: Language reference should specify restrictions on class namespace (GH-18559) The language reference now specifies restrictions on class namespaces. Adapted from a patch by Ethan Furman. (cherry picked from commit fbe2e0bb8a7ee75d0f9d57682436dac7d69e202e) Co-authored-by: ananthan-123 <ananthakrishnan15.2001(a)gmail.com> files: A Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst M Doc/reference/datamodel.rst diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 7b3bf0d2f5f36..06f4992cf544a 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1946,10 +1946,12 @@ Once the appropriate metaclass has been identified, then the class namespace is prepared. If the metaclass has a ``__prepare__`` attribute, it is called as ``namespace = metaclass.__prepare__(name, bases, **kwds)`` (where the additional keyword arguments, if any, come from the class definition). The -``__prepare__`` method should be implemented as a :func:`classmethod`. +``__prepare__`` method should be implemented as a :func:`classmethod`. The +namespace returned by ``__prepare__`` is passed in to ``__new__``, but when +the final class object is created the namespace is copied into a new ``dict``. If the metaclass has no ``__prepare__`` attribute, then the class namespace -is initialised as an empty ordered mapping. +is initialised as an empty :func:`dict`. .. seealso:: diff --git a/Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst b/Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst new file mode 100644 index 0000000000000..f071d286176ae --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst @@ -0,0 +1 @@ +The language reference now specifies restrictions on class namespaces. Adapted from a patch by Ethan Furman.

1 0

bpo-17422: Language reference should specify restrictions on class namespace (#18559)
by ananthan-123 22 Feb '20

22 Feb '20

https://github.com/python/cpython/commit/fbe2e0bb8a7ee75d0f9d57682436dac7d6… commit: fbe2e0bb8a7ee75d0f9d57682436dac7d69e202e branch: master author: ananthan-123 <ananthakrishnan15.2001(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2020-02-22T09:56:01-08:00 summary: bpo-17422: Language reference should specify restrictions on class namespace (#18559) The language reference now specifies restrictions on class namespaces. Adapted from a patch by Ethan Furman. files: A Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst M Doc/reference/datamodel.rst diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 9520f824287f6..5b3b669c5eb91 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -1946,10 +1946,12 @@ Once the appropriate metaclass has been identified, then the class namespace is prepared. If the metaclass has a ``__prepare__`` attribute, it is called as ``namespace = metaclass.__prepare__(name, bases, **kwds)`` (where the additional keyword arguments, if any, come from the class definition). The -``__prepare__`` method should be implemented as a :func:`classmethod`. +``__prepare__`` method should be implemented as a :func:`classmethod`. The +namespace returned by ``__prepare__`` is passed in to ``__new__``, but when +the final class object is created the namespace is copied into a new ``dict``. If the metaclass has no ``__prepare__`` attribute, then the class namespace -is initialised as an empty ordered mapping. +is initialised as an empty :func:`dict`. .. seealso:: diff --git a/Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst b/Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst new file mode 100644 index 0000000000000..f071d286176ae --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-02-19-11-13-47.bpo-17422.g7_9zz.rst @@ -0,0 +1 @@ +The language reference now specifies restrictions on class namespaces. Adapted from a patch by Ethan Furman.

1 0

bpo-39382: Avoid dangling object use in abstract_issubclass() (GH-18530)
by Miss Islington (bot) 22 Feb '20

22 Feb '20

https://github.com/python/cpython/commit/0c1827e70c1c05ce1982a34380cea7d391… commit: 0c1827e70c1c05ce1982a34380cea7d391904293 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2020-02-22T05:34:06-08:00 summary: bpo-39382: Avoid dangling object use in abstract_issubclass() (GH-18530) Hold reference of __bases__ tuple until tuple item is done with, because by dropping the reference the item may be destroyed. (cherry picked from commit 1c56f8ffad44478b4214a2bf8eb7cf51c28a347a) Co-authored-by: Yonatan Goldschmidt <yon.goldschmidt(a)gmail.com> files: A Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst M Lib/test/test_isinstance.py M Misc/ACKS M Objects/abstract.c diff --git a/Lib/test/test_isinstance.py b/Lib/test/test_isinstance.py index 65751ab916855..53639e984e48a 100644 --- a/Lib/test/test_isinstance.py +++ b/Lib/test/test_isinstance.py @@ -251,6 +251,27 @@ def test_isinstance_recursion_limit(self): # blown self.assertRaises(RecursionError, blowstack, isinstance, '', str) + def test_issubclass_refcount_handling(self): + # bpo-39382: abstract_issubclass() didn't hold item reference while + # peeking in the bases tuple, in the single inheritance case. + class A: + @property + def __bases__(self): + return (int, ) + + class B: + def __init__(self): + # setting this here increases the chances of exhibiting the bug, + # probably due to memory layout changes. + self.x = 1 + + @property + def __bases__(self): + return (A(), ) + + self.assertEqual(True, issubclass(B(), int)) + + def blowstack(fxn, arg, compare_to): # Make sure that calling isinstance with a deeply nested tuple for its # argument will raise RecursionError eventually. diff --git a/Misc/ACKS b/Misc/ACKS index 69ed251b988db..8fd4df5bf3f19 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -584,6 +584,7 @@ Karan Goel Jeroen Van Goey Christoph Gohlke Tim Golden +Yonatan Goldschmidt Mark Gollahon Guilherme Gonçalves Tiago Gonçalves diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst b/Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst new file mode 100644 index 0000000000000..605f4c8e5dfd1 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst @@ -0,0 +1,3 @@ +Fix a use-after-free in the single inheritance path of ``issubclass()``, when +the ``__bases__`` of an object has a single reference, and so does its first item. +Patch by Yonatan Goldschmidt. diff --git a/Objects/abstract.c b/Objects/abstract.c index 77d09143aa076..bc1ebd930c016 100644 --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2336,9 +2336,16 @@ abstract_issubclass(PyObject *derived, PyObject *cls) int r = 0; while (1) { - if (derived == cls) + if (derived == cls) { + Py_XDECREF(bases); /* See below comment */ return 1; - bases = abstract_get_bases(derived); + } + /* Use XSETREF to drop bases reference *after* finishing with + derived; bases might be the only reference to it. + XSETREF is used instead of SETREF, because bases is NULL on the + first iteration of the loop. + */ + Py_XSETREF(bases, abstract_get_bases(derived)); if (bases == NULL) { if (PyErr_Occurred()) return -1; @@ -2352,7 +2359,6 @@ abstract_issubclass(PyObject *derived, PyObject *cls) /* Avoid recursivity in the single inheritance case */ if (n == 1) { derived = PyTuple_GET_ITEM(bases, 0); - Py_DECREF(bases); continue; } for (i = 0; i < n; i++) {

1 0

bpo-39382: Avoid dangling object use in abstract_issubclass() (GH-18530)
by Miss Islington (bot) 22 Feb '20

22 Feb '20

https://github.com/python/cpython/commit/43a0137c87b997c6ba8b23cc3281ce2de1… commit: 43a0137c87b997c6ba8b23cc3281ce2de18f008a branch: 3.7 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2020-02-22T05:32:36-08:00 summary: bpo-39382: Avoid dangling object use in abstract_issubclass() (GH-18530) Hold reference of __bases__ tuple until tuple item is done with, because by dropping the reference the item may be destroyed. (cherry picked from commit 1c56f8ffad44478b4214a2bf8eb7cf51c28a347a) Co-authored-by: Yonatan Goldschmidt <yon.goldschmidt(a)gmail.com> files: A Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst M Lib/test/test_isinstance.py M Misc/ACKS M Objects/abstract.c diff --git a/Lib/test/test_isinstance.py b/Lib/test/test_isinstance.py index 65751ab916855..53639e984e48a 100644 --- a/Lib/test/test_isinstance.py +++ b/Lib/test/test_isinstance.py @@ -251,6 +251,27 @@ def test_isinstance_recursion_limit(self): # blown self.assertRaises(RecursionError, blowstack, isinstance, '', str) + def test_issubclass_refcount_handling(self): + # bpo-39382: abstract_issubclass() didn't hold item reference while + # peeking in the bases tuple, in the single inheritance case. + class A: + @property + def __bases__(self): + return (int, ) + + class B: + def __init__(self): + # setting this here increases the chances of exhibiting the bug, + # probably due to memory layout changes. + self.x = 1 + + @property + def __bases__(self): + return (A(), ) + + self.assertEqual(True, issubclass(B(), int)) + + def blowstack(fxn, arg, compare_to): # Make sure that calling isinstance with a deeply nested tuple for its # argument will raise RecursionError eventually. diff --git a/Misc/ACKS b/Misc/ACKS index 613763195b687..2107435cf093b 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -568,6 +568,7 @@ Karan Goel Jeroen Van Goey Christoph Gohlke Tim Golden +Yonatan Goldschmidt Mark Gollahon Guilherme Gonçalves Tiago Gonçalves diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst b/Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst new file mode 100644 index 0000000000000..605f4c8e5dfd1 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst @@ -0,0 +1,3 @@ +Fix a use-after-free in the single inheritance path of ``issubclass()``, when +the ``__bases__`` of an object has a single reference, and so does its first item. +Patch by Yonatan Goldschmidt. diff --git a/Objects/abstract.c b/Objects/abstract.c index 9416df4321f3f..e6831feb82433 100644 --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2287,9 +2287,16 @@ abstract_issubclass(PyObject *derived, PyObject *cls) int r = 0; while (1) { - if (derived == cls) + if (derived == cls) { + Py_XDECREF(bases); /* See below comment */ return 1; - bases = abstract_get_bases(derived); + } + /* Use XSETREF to drop bases reference *after* finishing with + derived; bases might be the only reference to it. + XSETREF is used instead of SETREF, because bases is NULL on the + first iteration of the loop. + */ + Py_XSETREF(bases, abstract_get_bases(derived)); if (bases == NULL) { if (PyErr_Occurred()) return -1; @@ -2303,7 +2310,6 @@ abstract_issubclass(PyObject *derived, PyObject *cls) /* Avoid recursivity in the single inheritance case */ if (n == 1) { derived = PyTuple_GET_ITEM(bases, 0); - Py_DECREF(bases); continue; } for (i = 0; i < n; i++) {

1 0

bpo-39382: Avoid dangling object use in abstract_issubclass() (GH-18530)
by Yonatan Goldschmidt 22 Feb '20

22 Feb '20

https://github.com/python/cpython/commit/1c56f8ffad44478b4214a2bf8eb7cf51c2… commit: 1c56f8ffad44478b4214a2bf8eb7cf51c28a347a branch: master author: Yonatan Goldschmidt <yon.goldschmidt(a)gmail.com> committer: GitHub <noreply(a)github.com> date: 2020-02-22T15:11:48+02:00 summary: bpo-39382: Avoid dangling object use in abstract_issubclass() (GH-18530) Hold reference of __bases__ tuple until tuple item is done with, because by dropping the reference the item may be destroyed. files: A Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst M Lib/test/test_isinstance.py M Misc/ACKS M Objects/abstract.c diff --git a/Lib/test/test_isinstance.py b/Lib/test/test_isinstance.py index 65751ab916855..53639e984e48a 100644 --- a/Lib/test/test_isinstance.py +++ b/Lib/test/test_isinstance.py @@ -251,6 +251,27 @@ def test_isinstance_recursion_limit(self): # blown self.assertRaises(RecursionError, blowstack, isinstance, '', str) + def test_issubclass_refcount_handling(self): + # bpo-39382: abstract_issubclass() didn't hold item reference while + # peeking in the bases tuple, in the single inheritance case. + class A: + @property + def __bases__(self): + return (int, ) + + class B: + def __init__(self): + # setting this here increases the chances of exhibiting the bug, + # probably due to memory layout changes. + self.x = 1 + + @property + def __bases__(self): + return (A(), ) + + self.assertEqual(True, issubclass(B(), int)) + + def blowstack(fxn, arg, compare_to): # Make sure that calling isinstance with a deeply nested tuple for its # argument will raise RecursionError eventually. diff --git a/Misc/ACKS b/Misc/ACKS index f329a2d4a7d33..fe24a5636ccc2 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -594,6 +594,7 @@ Karan Goel Jeroen Van Goey Christoph Gohlke Tim Golden +Yonatan Goldschmidt Mark Gollahon Guilherme Gonçalves Tiago Gonçalves diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst b/Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst new file mode 100644 index 0000000000000..605f4c8e5dfd1 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-02-18-01-40-13.bpo-39382.OLSJu9.rst @@ -0,0 +1,3 @@ +Fix a use-after-free in the single inheritance path of ``issubclass()``, when +the ``__bases__`` of an object has a single reference, and so does its first item. +Patch by Yonatan Goldschmidt. diff --git a/Objects/abstract.c b/Objects/abstract.c index f0e01f7691bb3..de5652f3e685f 100644 --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -2379,9 +2379,16 @@ abstract_issubclass(PyObject *derived, PyObject *cls) int r = 0; while (1) { - if (derived == cls) + if (derived == cls) { + Py_XDECREF(bases); /* See below comment */ return 1; - bases = abstract_get_bases(derived); + } + /* Use XSETREF to drop bases reference *after* finishing with + derived; bases might be the only reference to it. + XSETREF is used instead of SETREF, because bases is NULL on the + first iteration of the loop. + */ + Py_XSETREF(bases, abstract_get_bases(derived)); if (bases == NULL) { if (PyErr_Occurred()) return -1; @@ -2395,7 +2402,6 @@ abstract_issubclass(PyObject *derived, PyObject *cls) /* Avoid recursivity in the single inheritance case */ if (n == 1) { derived = PyTuple_GET_ITEM(bases, 0); - Py_DECREF(bases); continue; } for (i = 0; i < n; i++) {

1 0

bpo-39576: docs: set context for decimal arbitrary precision arithmetic (GH-18594) (#18597)
by Miss Islington (bot) 21 Feb '20

21 Feb '20

https://github.com/python/cpython/commit/d6965ff026f35498e554bc964ef2be8f4d… commit: d6965ff026f35498e554bc964ef2be8f4d80eb7f branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2020-02-21T21:36:42+01:00 summary: bpo-39576: docs: set context for decimal arbitrary precision arithmetic (GH-18594) (#18597) (cherry picked from commit a025d4ca99fb4c652465368e0b4eb03cf4b316b9) Authored-by: Stefan Krah <skrah(a)bytereef.org> files: M Doc/library/decimal.rst diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst index bcae55eb82178..2a51429bdff5c 100644 --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -2121,17 +2121,67 @@ Q. Is the CPython implementation fast for large numbers? A. Yes. In the CPython and PyPy3 implementations, the C/CFFI versions of the decimal module integrate the high speed `libmpdec <https://www.bytereef.org/mpdecimal/doc/libmpdec/index.html>`_ library for -arbitrary precision correctly-rounded decimal floating point arithmetic. +arbitrary precision correctly-rounded decimal floating point arithmetic [#]_. ``libmpdec`` uses `Karatsuba multiplication <https://en.wikipedia.org/wiki/Karatsuba_algorithm>`_ for medium-sized numbers and the `Number Theoretic Transform <https://en.wikipedia.org/wiki/Discrete_Fourier_transform_(general)#Number-t…>`_ -for very large numbers. However, to realize this performance gain, the -context needs to be set for unrounded calculations. +for very large numbers. - >>> c = getcontext() - >>> c.prec = MAX_PREC - >>> c.Emax = MAX_EMAX - >>> c.Emin = MIN_EMIN +The context must be adapted for exact arbitrary precision arithmetic. :attr:`Emin` +and :attr:`Emax` should always be set to the maximum values, :attr:`clamp` +should always be 0 (the default). Setting :attr:`prec` requires some care. -.. versionadded:: 3.3 \ No newline at end of file +The easiest approach for trying out bignum arithmetic is to use the maximum +value for :attr:`prec` as well [#]_:: + + >>> setcontext(Context(prec=MAX_PREC, Emax=MAX_EMAX, Emin=MIN_EMIN)) + >>> x = Decimal(2) ** 256 + >>> x / 128 + Decimal('904625697166532776746648320380374280103671755200316906558262375061821325312') + + +For inexact results, :attr:`MAX_PREC` is far too large on 64-bit platforms and +the available memory will be insufficient:: + + >>> Decimal(1) / 3 + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + MemoryError + +On systems with overallocation (e.g. Linux), a more sophisticated approach is to +adjust :attr:`prec` to the amount of available RAM. Suppose that you have 8GB of +RAM and expect 10 simultaneous operands using a maximum of 500MB each:: + + >>> import sys + >>> + >>> # Maximum number of digits for a single operand using 500MB in 8 byte words + >>> # with 19 (9 for the 32-bit version) digits per word: + >>> maxdigits = 19 * ((500 * 1024**2) // 8) + >>> + >>> # Check that this works: + >>> c = Context(prec=maxdigits, Emax=MAX_EMAX, Emin=MIN_EMIN) + >>> c.traps[Inexact] = True + >>> setcontext(c) + >>> + >>> # Fill the available precision with nines: + >>> x = Decimal(0).logical_invert() * 9 + >>> sys.getsizeof(x) + 524288112 + >>> x + 2 + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + decimal.Inexact: [<class 'decimal.Inexact'>] + +In general (and especially on systems without overallocation), it is recommended +to estimate even tighter bounds and set the :attr:`Inexact` trap if all calculations +are expected to be exact. + + +.. [#] + .. versionadded:: 3.3 + +.. [#] + .. versionchanged:: 3.9 + This approach now works for all exact results except for non-integer powers. + Also backported to 3.7 and 3.8.

1 0