Mailman 3 October 2024 - Python-checkins

[3.13] gh-126240: handle `NULL` returned by `_Py_asdl_expr_seq_new` (GH-126241) (#126243)
by sobolevn Oct. 31, 2024

Oct. 31, 2024

https://github.com/python/cpython/commit/80042384b477519ee2b137a841b5a82567… commit: 80042384b477519ee2b137a841b5a825675bedf5 branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: sobolevn <mail(a)sobolevn.me> date: 2024-10-31T14:57:11Z summary: [3.13] gh-126240: handle `NULL` returned by `_Py_asdl_expr_seq_new` (GH-126241) (#126243) gh-126240: handle `NULL` returned by `_Py_asdl_expr_seq_new` (GH-126241) check return value of `_Py_asdl_expr_seq_new` (cherry picked from commit 94639f6b7182c2e1a82f2f907b03b5b15202acfa) Co-authored-by: Bénédikt Tran <10796600+picnixz(a)users.noreply.github.com> files: M Parser/action_helpers.c diff --git a/Parser/action_helpers.c b/Parser/action_helpers.c index 34f744516c1f30..8dfc919da71b5a 100644 --- a/Parser/action_helpers.c +++ b/Parser/action_helpers.c @@ -1120,6 +1120,9 @@ expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_expr_seq *a, asdl_seq *b, } asdl_expr_seq *args = _Py_asdl_expr_seq_new(total_len, arena); + if (args == NULL) { + return NULL; + } Py_ssize_t i = 0; for (i = 0; i < args_len; i++) { @@ -1290,6 +1293,9 @@ unpack_top_level_joined_strs(Parser *p, asdl_expr_seq *raw_expressions) } asdl_expr_seq *expressions = _Py_asdl_expr_seq_new(req_size, p->arena); + if (expressions == NULL) { + return NULL; + } Py_ssize_t raw_index, req_index = 0; for (raw_index = 0; raw_index < raw_size; raw_index++) { @@ -1482,6 +1488,9 @@ expr_ty _PyPegen_formatted_value(Parser *p, expr_ty expression, Token *debug, Re } asdl_expr_seq *values = _Py_asdl_expr_seq_new(2, arena); + if (values == NULL) { + return NULL; + } asdl_seq_SET(values, 0, debug_text); asdl_seq_SET(values, 1, formatted_value); return _PyAST_JoinedStr(values, lineno, col_offset, debug_end_line, debug_end_offset, p->arena);

1 0

[3.13] gh-126223: Propagate unicode errors in `_interpreters.create()` (GH-126224) (#126242)
by sobolevn Oct. 31, 2024

Oct. 31, 2024

https://github.com/python/cpython/commit/89664d4d482b9873b078df29654e4c3fa8… commit: 89664d4d482b9873b078df29654e4c3fa8ca066f branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: sobolevn <mail(a)sobolevn.me> date: 2024-10-31T14:46:36Z summary: [3.13] gh-126223: Propagate unicode errors in `_interpreters.create()` (GH-126224) (#126242) gh-126223: Propagate unicode errors in `_interpreters.create()` (GH-126224) (cherry picked from commit 01415213d72504eafc159721a8f55d57b374fd9c) Co-authored-by: Peter Bierma <zintensitydev(a)gmail.com> Co-authored-by: sobolevn <mail(a)sobolevn.me> Co-authored-by: Bénédikt Tran <10796600+picnixz(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2024-10-30-23-42-44.gh-issue-126223.k2qooc.rst M Lib/test/test_interpreters/test_api.py M Modules/_interpretersmodule.c diff --git a/Lib/test/test_interpreters/test_api.py b/Lib/test/test_interpreters/test_api.py index 719c1c721cad7c..42819ae9b180b4 100644 --- a/Lib/test/test_interpreters/test_api.py +++ b/Lib/test/test_interpreters/test_api.py @@ -55,6 +55,9 @@ def test_in_main(self): self.assertIsInstance(interp, interpreters.Interpreter) self.assertIn(interp, interpreters.list_all()) + # GH-126221: Passing an invalid Unicode character used to cause a SystemError + self.assertRaises(UnicodeEncodeError, _interpreters.create, '\udc80') + def test_in_thread(self): lock = threading.Lock() interp = None diff --git a/Misc/NEWS.d/next/Library/2024-10-30-23-42-44.gh-issue-126223.k2qooc.rst b/Misc/NEWS.d/next/Library/2024-10-30-23-42-44.gh-issue-126223.k2qooc.rst new file mode 100644 index 00000000000000..fee391c030b941 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-30-23-42-44.gh-issue-126223.k2qooc.rst @@ -0,0 +1,2 @@ +Raise a :exc:`UnicodeEncodeError` instead of a :exc:`SystemError` upon +calling :func:`!_interpreters.create` with an invalid Unicode character. diff --git a/Modules/_interpretersmodule.c b/Modules/_interpretersmodule.c index 86a4113dcc16f1..7933febf3cacfc 100644 --- a/Modules/_interpretersmodule.c +++ b/Modules/_interpretersmodule.c @@ -403,7 +403,11 @@ config_from_object(PyObject *configobj, PyInterpreterConfig *config) } } else if (PyUnicode_Check(configobj)) { - if (init_named_config(config, PyUnicode_AsUTF8(configobj)) < 0) { + const char *utf8name = PyUnicode_AsUTF8(configobj); + if (utf8name == NULL) { + return -1; + } + if (init_named_config(config, utf8name) < 0) { return -1; } }

1 0

gh-101123: Adapt vararg functions in the math module to Argument Clinic (#126235)
by erlend-aasland Oct. 31, 2024

Oct. 31, 2024

https://github.com/python/cpython/commit/3275cb19530fb5c7115cf8313f1ada9621… commit: 3275cb19530fb5c7115cf8313f1ada9621ed3a92 branch: main author: Sergey B Kirpichev <skirpichev(a)gmail.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2024-10-31T14:37:47Z summary: gh-101123: Adapt vararg functions in the math module to Argument Clinic (#126235) This implicitly fixes the math.hypot signature, which was previously incomprehensible to inspect.signature(). files: M Modules/clinic/mathmodule.c.h M Modules/mathmodule.c diff --git a/Modules/clinic/mathmodule.c.h b/Modules/clinic/mathmodule.c.h index 81eec310ddb21d..7d0b98d5502267 100644 --- a/Modules/clinic/mathmodule.c.h +++ b/Modules/clinic/mathmodule.c.h @@ -8,6 +8,64 @@ preserve #endif #include "pycore_modsupport.h" // _PyArg_CheckPositional() +PyDoc_STRVAR(math_gcd__doc__, +"gcd($module, /, *integers)\n" +"--\n" +"\n" +"Greatest Common Divisor."); + +#define MATH_GCD_METHODDEF \ + {"gcd", _PyCFunction_CAST(math_gcd), METH_FASTCALL, math_gcd__doc__}, + +static PyObject * +math_gcd_impl(PyObject *module, Py_ssize_t nargs, PyObject *const *args); + +static PyObject * +math_gcd(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; + + if (!_PyArg_CheckPositional("gcd", nargs, 0, PY_SSIZE_T_MAX)) { + goto exit; + } + __clinic_args = args + 0; + return_value = math_gcd_impl(module, nvararg, __clinic_args); + +exit: + return return_value; +} + +PyDoc_STRVAR(math_lcm__doc__, +"lcm($module, /, *integers)\n" +"--\n" +"\n" +"Least Common Multiple."); + +#define MATH_LCM_METHODDEF \ + {"lcm", _PyCFunction_CAST(math_lcm), METH_FASTCALL, math_lcm__doc__}, + +static PyObject * +math_lcm_impl(PyObject *module, Py_ssize_t nargs, PyObject *const *args); + +static PyObject * +math_lcm(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; + + if (!_PyArg_CheckPositional("lcm", nargs, 0, PY_SSIZE_T_MAX)) { + goto exit; + } + __clinic_args = args + 0; + return_value = math_lcm_impl(module, nvararg, __clinic_args); + +exit: + return return_value; +} + PyDoc_STRVAR(math_ceil__doc__, "ceil($module, x, /)\n" "--\n" @@ -351,6 +409,46 @@ math_dist(PyObject *module, PyObject *const *args, Py_ssize_t nargs) return return_value; } +PyDoc_STRVAR(math_hypot__doc__, +"hypot($module, /, *coordinates)\n" +"--\n" +"\n" +"Multidimensional Euclidean distance from the origin to a point.\n" +"\n" +"Roughly equivalent to:\n" +" sqrt(sum(x**2 for x in coordinates))\n" +"\n" +"For a two dimensional point (x, y), gives the hypotenuse\n" +"using the Pythagorean theorem: sqrt(x*x + y*y).\n" +"\n" +"For example, the hypotenuse of a 3/4/5 right triangle is:\n" +"\n" +" >>> hypot(3.0, 4.0)\n" +" 5.0"); + +#define MATH_HYPOT_METHODDEF \ + {"hypot", _PyCFunction_CAST(math_hypot), METH_FASTCALL, math_hypot__doc__}, + +static PyObject * +math_hypot_impl(PyObject *module, Py_ssize_t nargs, PyObject *const *args); + +static PyObject * +math_hypot(PyObject *module, PyObject *const *args, Py_ssize_t nargs) +{ + PyObject *return_value = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; + + if (!_PyArg_CheckPositional("hypot", nargs, 0, PY_SSIZE_T_MAX)) { + goto exit; + } + __clinic_args = args + 0; + return_value = math_hypot_impl(module, nvararg, __clinic_args); + +exit: + return return_value; +} + PyDoc_STRVAR(math_sumprod__doc__, "sumprod($module, p, q, /)\n" "--\n" @@ -1011,4 +1109,4 @@ math_ulp(PyObject *module, PyObject *arg) exit: return return_value; } -/*[clinic end generated code: output=755da3b1dbd9e45f input=a9049054013a1b77]*/ +/*[clinic end generated code: output=ee0a2f6bd1220061 input=a9049054013a1b77]*/ diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c index 058f57770755aa..ad23dadd7b86cc 100644 --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -719,8 +719,17 @@ m_log10(double x) } +/*[clinic input] +math.gcd + + *integers as args: object + +Greatest Common Divisor. +[clinic start generated code]*/ + static PyObject * -math_gcd(PyObject *module, PyObject * const *args, Py_ssize_t nargs) +math_gcd_impl(PyObject *module, Py_ssize_t nargs, PyObject *const *args) +/*[clinic end generated code: output=b57687fcf431c1b8 input=94e675b7ceeaf0c9]*/ { // Fast-path for the common case: gcd(int, int) if (nargs == 2 && PyLong_CheckExact(args[0]) && PyLong_CheckExact(args[1])) @@ -763,12 +772,6 @@ math_gcd(PyObject *module, PyObject * const *args, Py_ssize_t nargs) return res; } -PyDoc_STRVAR(math_gcd_doc, -"gcd($module, *integers)\n" -"--\n" -"\n" -"Greatest Common Divisor."); - static PyObject * long_lcm(PyObject *a, PyObject *b) @@ -798,8 +801,17 @@ long_lcm(PyObject *a, PyObject *b) } +/*[clinic input] +math.lcm + + *integers as args: object + +Least Common Multiple. +[clinic start generated code]*/ + static PyObject * -math_lcm(PyObject *module, PyObject * const *args, Py_ssize_t nargs) +math_lcm_impl(PyObject *module, Py_ssize_t nargs, PyObject *const *args) +/*[clinic end generated code: output=f3eff0c25e4d7030 input=e64c33e85f4c47c6]*/ { PyObject *res, *x; Py_ssize_t i; @@ -839,13 +851,6 @@ math_lcm(PyObject *module, PyObject * const *args, Py_ssize_t nargs) } -PyDoc_STRVAR(math_lcm_doc, -"lcm($module, *integers)\n" -"--\n" -"\n" -"Least Common Multiple."); - - /* Call is_error when errno != 0, and where x is the result libm * returned. is_error will usually set up an exception and return * true (1), but may return false (0) without setting up an exception. @@ -2621,9 +2626,28 @@ math_dist_impl(PyObject *module, PyObject *p, PyObject *q) return NULL; } -/* AC: cannot convert yet, waiting for *args support */ +/*[clinic input] +math.hypot + + *coordinates as args: object + +Multidimensional Euclidean distance from the origin to a point. + +Roughly equivalent to: + sqrt(sum(x**2 for x in coordinates)) + +For a two dimensional point (x, y), gives the hypotenuse +using the Pythagorean theorem: sqrt(x*x + y*y). + +For example, the hypotenuse of a 3/4/5 right triangle is: + + >>> hypot(3.0, 4.0) + 5.0 +[clinic start generated code]*/ + static PyObject * -math_hypot(PyObject *self, PyObject *const *args, Py_ssize_t nargs) +math_hypot_impl(PyObject *module, Py_ssize_t nargs, PyObject *const *args) +/*[clinic end generated code: output=dcb6d4b7a1102ee1 input=5c0061a2d11235ed]*/ { Py_ssize_t i; PyObject *item; @@ -2664,22 +2688,6 @@ math_hypot(PyObject *self, PyObject *const *args, Py_ssize_t nargs) #undef NUM_STACK_ELEMS -PyDoc_STRVAR(math_hypot_doc, - "hypot(*coordinates) -> value\n\n\ -Multidimensional Euclidean distance from the origin to a point.\n\ -\n\ -Roughly equivalent to:\n\ - sqrt(sum(x**2 for x in coordinates))\n\ -\n\ -For a two dimensional point (x, y), gives the hypotenuse\n\ -using the Pythagorean theorem: sqrt(x*x + y*y).\n\ -\n\ -For example, the hypotenuse of a 3/4/5 right triangle is:\n\ -\n\ - >>> hypot(3.0, 4.0)\n\ - 5.0\n\ -"); - /** sumprod() ***************************************************************/ /* Forward declaration */ @@ -4112,14 +4120,14 @@ static PyMethodDef math_methods[] = { MATH_FREXP_METHODDEF MATH_FSUM_METHODDEF {"gamma", math_gamma, METH_O, math_gamma_doc}, - {"gcd", _PyCFunction_CAST(math_gcd), METH_FASTCALL, math_gcd_doc}, - {"hypot", _PyCFunction_CAST(math_hypot), METH_FASTCALL, math_hypot_doc}, + MATH_GCD_METHODDEF + MATH_HYPOT_METHODDEF MATH_ISCLOSE_METHODDEF MATH_ISFINITE_METHODDEF MATH_ISINF_METHODDEF MATH_ISNAN_METHODDEF MATH_ISQRT_METHODDEF - {"lcm", _PyCFunction_CAST(math_lcm), METH_FASTCALL, math_lcm_doc}, + MATH_LCM_METHODDEF MATH_LDEXP_METHODDEF {"lgamma", math_lgamma, METH_O, math_lgamma_doc}, {"log", _PyCFunction_CAST(math_log), METH_FASTCALL, math_log_doc},

1 0

gh-126240: handle `NULL` returned by `_Py_asdl_expr_seq_new` (#126241)
by sobolevn Oct. 31, 2024

Oct. 31, 2024

https://github.com/python/cpython/commit/94639f6b7182c2e1a82f2f907b03b5b152… commit: 94639f6b7182c2e1a82f2f907b03b5b15202acfa branch: main author: Bénédikt Tran <10796600+picnixz(a)users.noreply.github.com> committer: sobolevn <mail(a)sobolevn.me> date: 2024-10-31T17:24:07+03:00 summary: gh-126240: handle `NULL` returned by `_Py_asdl_expr_seq_new` (#126241) check return value of `_Py_asdl_expr_seq_new` files: M Parser/action_helpers.c diff --git a/Parser/action_helpers.c b/Parser/action_helpers.c index cb21777f566189..5ac1dd7813689c 100644 --- a/Parser/action_helpers.c +++ b/Parser/action_helpers.c @@ -1128,6 +1128,9 @@ expr_ty _PyPegen_collect_call_seqs(Parser *p, asdl_expr_seq *a, asdl_seq *b, } asdl_expr_seq *args = _Py_asdl_expr_seq_new(total_len, arena); + if (args == NULL) { + return NULL; + } Py_ssize_t i = 0; for (i = 0; i < args_len; i++) { @@ -1298,6 +1301,9 @@ unpack_top_level_joined_strs(Parser *p, asdl_expr_seq *raw_expressions) } asdl_expr_seq *expressions = _Py_asdl_expr_seq_new(req_size, p->arena); + if (expressions == NULL) { + return NULL; + } Py_ssize_t raw_index, req_index = 0; for (raw_index = 0; raw_index < raw_size; raw_index++) { @@ -1490,6 +1496,9 @@ expr_ty _PyPegen_formatted_value(Parser *p, expr_ty expression, Token *debug, Re } asdl_expr_seq *values = _Py_asdl_expr_seq_new(2, arena); + if (values == NULL) { + return NULL; + } asdl_seq_SET(values, 0, debug_text); asdl_seq_SET(values, 1, formatted_value); return _PyAST_JoinedStr(values, lineno, col_offset, debug_end_line, debug_end_offset, p->arena);

1 0

gh-126223: Propagate unicode errors in `_interpreters.create()` (#126224)
by sobolevn Oct. 31, 2024

Oct. 31, 2024

https://github.com/python/cpython/commit/01415213d72504eafc159721a8f55d57b3… commit: 01415213d72504eafc159721a8f55d57b374fd9c branch: main author: Peter Bierma <zintensitydev(a)gmail.com> committer: sobolevn <mail(a)sobolevn.me> date: 2024-10-31T17:14:37+03:00 summary: gh-126223: Propagate unicode errors in `_interpreters.create()` (#126224) Co-authored-by: sobolevn <mail(a)sobolevn.me> Co-authored-by: Bénédikt Tran <10796600+picnixz(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2024-10-30-23-42-44.gh-issue-126223.k2qooc.rst M Lib/test/test_interpreters/test_api.py M Modules/_interpretersmodule.c diff --git a/Lib/test/test_interpreters/test_api.py b/Lib/test/test_interpreters/test_api.py index 5e3d7a052bae91..a9befbba64daa0 100644 --- a/Lib/test/test_interpreters/test_api.py +++ b/Lib/test/test_interpreters/test_api.py @@ -54,6 +54,9 @@ def test_in_main(self): self.assertIsInstance(interp, interpreters.Interpreter) self.assertIn(interp, interpreters.list_all()) + # GH-126221: Passing an invalid Unicode character used to cause a SystemError + self.assertRaises(UnicodeEncodeError, _interpreters.create, '\udc80') + def test_in_thread(self): lock = threading.Lock() interp = None diff --git a/Misc/NEWS.d/next/Library/2024-10-30-23-42-44.gh-issue-126223.k2qooc.rst b/Misc/NEWS.d/next/Library/2024-10-30-23-42-44.gh-issue-126223.k2qooc.rst new file mode 100644 index 00000000000000..fee391c030b941 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-30-23-42-44.gh-issue-126223.k2qooc.rst @@ -0,0 +1,2 @@ +Raise a :exc:`UnicodeEncodeError` instead of a :exc:`SystemError` upon +calling :func:`!_interpreters.create` with an invalid Unicode character. diff --git a/Modules/_interpretersmodule.c b/Modules/_interpretersmodule.c index 6f3392fe6ea28d..63f2bb38768511 100644 --- a/Modules/_interpretersmodule.c +++ b/Modules/_interpretersmodule.c @@ -402,7 +402,11 @@ config_from_object(PyObject *configobj, PyInterpreterConfig *config) } } else if (PyUnicode_Check(configobj)) { - if (init_named_config(config, PyUnicode_AsUTF8(configobj)) < 0) { + const char *utf8name = PyUnicode_AsUTF8(configobj); + if (utf8name == NULL) { + return -1; + } + if (init_named_config(config, utf8name) < 0) { return -1; } }

1 0

gh-90370: Argument Clinic: avoid temporary tuple creation for varargs (#126064)
by erlend-aasland Oct. 31, 2024

Oct. 31, 2024

https://github.com/python/cpython/commit/8c22eba877225ab29cd64672dcb97f09a5… commit: 8c22eba877225ab29cd64672dcb97f09a5f7336f branch: main author: Sergey B Kirpichev <skirpichev(a)gmail.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2024-10-31T11:37:03+01:00 summary: gh-90370: Argument Clinic: avoid temporary tuple creation for varargs (#126064) Avoid temporary tuple creation when all arguments either positional-only or vararg. Objects/setobject.c and Modules/gcmodule.c adapted. This fixes slight performance regression for set methods, introduced by gh-115112. files: A Misc/NEWS.d/next/Core_and_Builtins/2024-10-27-20-31-43.gh-issue-90370.IP_W3a.rst M Lib/test/clinic.test.c M Lib/test/test_clinic.py M Modules/_testclinic.c M Modules/clinic/_testclinic.c.h M Modules/clinic/gcmodule.c.h M Modules/gcmodule.c M Objects/clinic/setobject.c.h M Objects/setobject.c M Tools/clinic/libclinic/clanguage.py M Tools/clinic/libclinic/parse_args.py diff --git a/Lib/test/clinic.test.c b/Lib/test/clinic.test.c index 2a071f8485a2b8..e22324efc490be 100644 --- a/Lib/test/clinic.test.c +++ b/Lib/test/clinic.test.c @@ -4148,36 +4148,32 @@ PyDoc_STRVAR(test_vararg_and_posonly__doc__, {"test_vararg_and_posonly", _PyCFunction_CAST(test_vararg_and_posonly), METH_FASTCALL, test_vararg_and_posonly__doc__}, static PyObject * -test_vararg_and_posonly_impl(PyObject *module, PyObject *a, PyObject *args); +test_vararg_and_posonly_impl(PyObject *module, PyObject *a, Py_ssize_t nargs, + PyObject *const *args); static PyObject * test_vararg_and_posonly(PyObject *module, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; + Py_ssize_t nvararg = nargs - 1; PyObject *a; - PyObject *__clinic_args = NULL; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("test_vararg_and_posonly", nargs, 1, PY_SSIZE_T_MAX)) { goto exit; } a = args[0]; - __clinic_args = PyTuple_New(nargs - 1); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 1; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[1 + i])); - } - return_value = test_vararg_and_posonly_impl(module, a, __clinic_args); + __clinic_args = args + 1; + return_value = test_vararg_and_posonly_impl(module, a, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } static PyObject * -test_vararg_and_posonly_impl(PyObject *module, PyObject *a, PyObject *args) -/*[clinic end generated code: output=79b75dc07decc8d6 input=9cfa748bbff09877]*/ +test_vararg_and_posonly_impl(PyObject *module, PyObject *a, Py_ssize_t nargs, + PyObject *const *args) +/*[clinic end generated code: output=dc2dd9483cc0459e input=9cfa748bbff09877]*/ /*[clinic input] test_vararg @@ -4931,14 +4927,14 @@ PyDoc_STRVAR(Test___init____doc__, "Varargs init method. For example, nargs is translated to PyTuple_GET_SIZE."); static int -Test___init___impl(TestObj *self, PyObject *args); +Test___init___impl(TestObj *self, Py_ssize_t nargs, PyObject *const *args); static int Test___init__(PyObject *self, PyObject *args, PyObject *kwargs) { int return_value = -1; PyTypeObject *base_tp = TestType; - PyObject *__clinic_args = NULL; + PyObject *const *__clinic_args = NULL; if ((Py_IS_TYPE(self, base_tp) || Py_TYPE(self)->tp_new == base_tp->tp_new) && @@ -4948,17 +4944,16 @@ Test___init__(PyObject *self, PyObject *args, PyObject *kwargs) if (!_PyArg_CheckPositional("Test", PyTuple_GET_SIZE(args), 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_GetSlice(0, -1); - return_value = Test___init___impl((TestObj *)self, __clinic_args); + __clinic_args = _PyTuple_CAST(args)->ob_item; + return_value = Test___init___impl((TestObj *)self, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } static int -Test___init___impl(TestObj *self, PyObject *args) -/*[clinic end generated code: output=0ed1009fe0dcf98d input=2a8bd0033c9ac772]*/ +Test___init___impl(TestObj *self, Py_ssize_t nargs, PyObject *const *args) +/*[clinic end generated code: output=6a64b417c9080a73 input=2a8bd0033c9ac772]*/ /*[clinic input] @@ -4976,14 +4971,14 @@ PyDoc_STRVAR(Test__doc__, "Varargs new method. For example, nargs is translated to PyTuple_GET_SIZE."); static PyObject * -Test_impl(PyTypeObject *type, PyObject *args); +Test_impl(PyTypeObject *type, Py_ssize_t nargs, PyObject *const *args); static PyObject * Test(PyTypeObject *type, PyObject *args, PyObject *kwargs) { PyObject *return_value = NULL; PyTypeObject *base_tp = TestType; - PyObject *__clinic_args = NULL; + PyObject *const *__clinic_args = NULL; if ((type == base_tp || type->tp_init == base_tp->tp_init) && !_PyArg_NoKeywords("Test", kwargs)) { @@ -4992,17 +4987,16 @@ Test(PyTypeObject *type, PyObject *args, PyObject *kwargs) if (!_PyArg_CheckPositional("Test", PyTuple_GET_SIZE(args), 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_GetSlice(0, -1); - return_value = Test_impl(type, __clinic_args); + __clinic_args = _PyTuple_CAST(args)->ob_item; + return_value = Test_impl(type, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } static PyObject * -Test_impl(PyTypeObject *type, PyObject *args) -/*[clinic end generated code: output=8b219f6633e2a2e9 input=70ad829df3dd9b84]*/ +Test_impl(PyTypeObject *type, Py_ssize_t nargs, PyObject *const *args) +/*[clinic end generated code: output=bf22f942407383a5 input=70ad829df3dd9b84]*/ /*[clinic input] diff --git a/Lib/test/test_clinic.py b/Lib/test/test_clinic.py index 402106194f169f..d492ea1d76aa3a 100644 --- a/Lib/test/test_clinic.py +++ b/Lib/test/test_clinic.py @@ -3381,8 +3381,8 @@ def test_keyword_only_parameter(self): def test_varpos(self): # fn(*args) fn = ac_tester.varpos - self.assertEqual(fn(), ()) - self.assertEqual(fn(1, 2), (1, 2)) + self.assertEqual(fn(), ((),)) + self.assertEqual(fn(1, 2), ((1, 2),)) def test_posonly_varpos(self): # fn(a, b, /, *args) diff --git a/Misc/NEWS.d/next/Core_and_Builtins/2024-10-27-20-31-43.gh-issue-90370.IP_W3a.rst b/Misc/NEWS.d/next/Core_and_Builtins/2024-10-27-20-31-43.gh-issue-90370.IP_W3a.rst new file mode 100644 index 00000000000000..b6a19c06a228ca --- /dev/null +++ b/Misc/NEWS.d/next/Core_and_Builtins/2024-10-27-20-31-43.gh-issue-90370.IP_W3a.rst @@ -0,0 +1,2 @@ +Avoid temporary tuple creation for vararg in argument passing with Argument +Clinic generated code (if arguments either vararg or positional-only). diff --git a/Modules/_testclinic.c b/Modules/_testclinic.c index ca884af1aa29b8..e3c8ba9b0b5074 100644 --- a/Modules/_testclinic.c +++ b/Modules/_testclinic.c @@ -58,6 +58,20 @@ pack_arguments_newref(int argc, ...) return tuple; } +static PyObject * +pack_varargs_to_tuple(Py_ssize_t varargssize, PyObject *const *args) +{ + assert(!PyErr_Occurred()); + PyObject *tuple = PyTuple_New(varargssize); + if (!tuple) { + return NULL; + } + for (Py_ssize_t i = 0; i < varargssize; i++) { + PyTuple_SET_ITEM(tuple, i, Py_NewRef(args[i])); + } + return tuple; +} + /* Pack arguments to a tuple. * `wrapper` is function which converts primitive type to PyObject. * `arg_type` is type that arguments should be converted to before wrapped. */ @@ -970,10 +984,16 @@ varpos [clinic start generated code]*/ static PyObject * -varpos_impl(PyObject *module, PyObject *args) -/*[clinic end generated code: output=7b0b9545872bdca4 input=f87cd674145d394c]*/ +varpos_impl(PyObject *module, Py_ssize_t nargs, PyObject *const *args) +/*[clinic end generated code: output=b65096f423fb5dcc input=f87cd674145d394c]*/ { - return Py_NewRef(args); + PyObject *vararg_tuple = pack_varargs_to_tuple(nargs, args); + if (!vararg_tuple) { + return NULL; + } + PyObject *result = pack_arguments_newref(1, vararg_tuple); + Py_DECREF(vararg_tuple); + return result; } @@ -989,10 +1009,16 @@ posonly_varpos static PyObject * posonly_varpos_impl(PyObject *module, PyObject *a, PyObject *b, - PyObject *args) -/*[clinic end generated code: output=5dae5eb2a0d623cd input=c9fd7895cfbaabba]*/ + Py_ssize_t nargs, PyObject *const *args) +/*[clinic end generated code: output=d10d43d86d117ab3 input=c9fd7895cfbaabba]*/ { - return pack_arguments_newref(3, a, b, args); + PyObject *vararg_tuple = pack_varargs_to_tuple(nargs, args); + if (!vararg_tuple) { + return NULL; + } + PyObject *result = pack_arguments_newref(3, a, b, vararg_tuple); + Py_DECREF(vararg_tuple); + return result; } @@ -1157,8 +1183,9 @@ Proof-of-concept of GH-99233 refcount error bug. [clinic start generated code]*/ static PyObject * -gh_99233_refcount_impl(PyObject *module, PyObject *args) -/*[clinic end generated code: output=585855abfbca9a7f input=eecfdc2092d90dc3]*/ +gh_99233_refcount_impl(PyObject *module, Py_ssize_t nargs, + PyObject *const *args) +/*[clinic end generated code: output=b570007e61e5c670 input=eecfdc2092d90dc3]*/ { Py_RETURN_NONE; } diff --git a/Modules/clinic/_testclinic.c.h b/Modules/clinic/_testclinic.c.h index 1988c06971087d..7e29998c7db520 100644 --- a/Modules/clinic/_testclinic.c.h +++ b/Modules/clinic/_testclinic.c.h @@ -2530,28 +2530,22 @@ PyDoc_STRVAR(varpos__doc__, {"varpos", _PyCFunction_CAST(varpos), METH_FASTCALL, varpos__doc__}, static PyObject * -varpos_impl(PyObject *module, PyObject *args); +varpos_impl(PyObject *module, Py_ssize_t nargs, PyObject *const *args); static PyObject * varpos(PyObject *module, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; - PyObject *__clinic_args = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("varpos", nargs, 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_New(nargs - 0); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 0; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[0 + i])); - } - return_value = varpos_impl(module, __clinic_args); + __clinic_args = args + 0; + return_value = varpos_impl(module, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } @@ -2565,32 +2559,26 @@ PyDoc_STRVAR(posonly_varpos__doc__, static PyObject * posonly_varpos_impl(PyObject *module, PyObject *a, PyObject *b, - PyObject *args); + Py_ssize_t nargs, PyObject *const *args); static PyObject * posonly_varpos(PyObject *module, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; + Py_ssize_t nvararg = nargs - 2; PyObject *a; PyObject *b; - PyObject *__clinic_args = NULL; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("posonly_varpos", nargs, 2, PY_SSIZE_T_MAX)) { goto exit; } a = args[0]; b = args[1]; - __clinic_args = PyTuple_New(nargs - 2); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 2; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[2 + i])); - } - return_value = posonly_varpos_impl(module, a, b, __clinic_args); + __clinic_args = args + 2; + return_value = posonly_varpos_impl(module, a, b, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } @@ -3136,28 +3124,23 @@ PyDoc_STRVAR(gh_99233_refcount__doc__, {"gh_99233_refcount", _PyCFunction_CAST(gh_99233_refcount), METH_FASTCALL, gh_99233_refcount__doc__}, static PyObject * -gh_99233_refcount_impl(PyObject *module, PyObject *args); +gh_99233_refcount_impl(PyObject *module, Py_ssize_t nargs, + PyObject *const *args); static PyObject * gh_99233_refcount(PyObject *module, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; - PyObject *__clinic_args = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("gh_99233_refcount", nargs, 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_New(nargs - 0); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 0; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[0 + i])); - } - return_value = gh_99233_refcount_impl(module, __clinic_args); + __clinic_args = args + 0; + return_value = gh_99233_refcount_impl(module, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } @@ -3693,4 +3676,4 @@ _testclinic_TestClass_defclass_posonly_varpos(PyObject *self, PyTypeObject *cls, Py_XDECREF(__clinic_args); return return_value; } -/*[clinic end generated code: output=76ecbb38c632bde8 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=7662d07e7d29cbeb input=a9049054013a1b77]*/ diff --git a/Modules/clinic/gcmodule.c.h b/Modules/clinic/gcmodule.c.h index 9fff4da616ba00..be3bd35b4ffd44 100644 --- a/Modules/clinic/gcmodule.c.h +++ b/Modules/clinic/gcmodule.c.h @@ -312,28 +312,23 @@ PyDoc_STRVAR(gc_get_referrers__doc__, {"get_referrers", _PyCFunction_CAST(gc_get_referrers), METH_FASTCALL, gc_get_referrers__doc__}, static PyObject * -gc_get_referrers_impl(PyObject *module, PyObject *args); +gc_get_referrers_impl(PyObject *module, Py_ssize_t nargs, + PyObject *const *args); static PyObject * gc_get_referrers(PyObject *module, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; - PyObject *__clinic_args = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("get_referrers", nargs, 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_New(nargs - 0); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 0; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[0 + i])); - } - return_value = gc_get_referrers_impl(module, __clinic_args); + __clinic_args = args + 0; + return_value = gc_get_referrers_impl(module, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } @@ -347,28 +342,23 @@ PyDoc_STRVAR(gc_get_referents__doc__, {"get_referents", _PyCFunction_CAST(gc_get_referents), METH_FASTCALL, gc_get_referents__doc__}, static PyObject * -gc_get_referents_impl(PyObject *module, PyObject *args); +gc_get_referents_impl(PyObject *module, Py_ssize_t nargs, + PyObject *const *args); static PyObject * gc_get_referents(PyObject *module, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; - PyObject *__clinic_args = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("get_referents", nargs, 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_New(nargs - 0); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 0; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[0 + i])); - } - return_value = gc_get_referents_impl(module, __clinic_args); + __clinic_args = args + 0; + return_value = gc_get_referents_impl(module, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } @@ -585,4 +575,4 @@ gc_get_freeze_count(PyObject *module, PyObject *Py_UNUSED(ignored)) exit: return return_value; } -/*[clinic end generated code: output=0a7e91917adcb937 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=f488a0d4d6bd3687 input=a9049054013a1b77]*/ diff --git a/Modules/gcmodule.c b/Modules/gcmodule.c index 57e4aae9ed557e..f5fea5aa4dde08 100644 --- a/Modules/gcmodule.c +++ b/Modules/gcmodule.c @@ -8,6 +8,7 @@ #include "pycore_gc.h" #include "pycore_object.h" // _PyObject_IS_GC() #include "pycore_pystate.h" // _PyInterpreterState_GET() +#include "pycore_tuple.h" // _PyTuple_FromArray() typedef struct _gc_runtime_state GCState; @@ -221,15 +222,25 @@ Return the list of objects that directly refer to any of 'objs'. [clinic start generated code]*/ static PyObject * -gc_get_referrers_impl(PyObject *module, PyObject *args) -/*[clinic end generated code: output=296a09587f6a86b5 input=bae96961b14a0922]*/ +gc_get_referrers_impl(PyObject *module, Py_ssize_t nargs, + PyObject *const *args) +/*[clinic end generated code: output=1d44a7695ea25c40 input=bae96961b14a0922]*/ { - if (PySys_Audit("gc.get_referrers", "(O)", args) < 0) { + PyObject *varargs = _PyTuple_FromArray(args, nargs); + + if (!varargs) { + return NULL; + } + if (PySys_Audit("gc.get_referrers", "(O)", varargs) < 0) { + Py_DECREF(varargs); return NULL; } PyInterpreterState *interp = _PyInterpreterState_GET(); - return _PyGC_GetReferrers(interp, args); + PyObject *result = _PyGC_GetReferrers(interp, varargs); + + Py_DECREF(varargs); + return result; } /* Append obj to list; return true if error (out of memory), false if OK. */ @@ -269,27 +280,37 @@ Return the list of objects that are directly referred to by 'objs'. [clinic start generated code]*/ static PyObject * -gc_get_referents_impl(PyObject *module, PyObject *args) -/*[clinic end generated code: output=d47dc02cefd06fe8 input=b3ceab0c34038cbf]*/ +gc_get_referents_impl(PyObject *module, Py_ssize_t nargs, + PyObject *const *args) +/*[clinic end generated code: output=e459f3e8c0d19311 input=b3ceab0c34038cbf]*/ { - if (PySys_Audit("gc.get_referents", "(O)", args) < 0) { + PyObject *varargs = _PyTuple_FromArray(args, nargs); + + if (!varargs) { + return NULL; + } + if (PySys_Audit("gc.get_referents", "(O)", varargs) < 0) { + Py_DECREF(varargs); return NULL; } PyInterpreterState *interp = _PyInterpreterState_GET(); PyObject *result = PyList_New(0); - if (result == NULL) + if (result == NULL) { + Py_DECREF(varargs); return NULL; + } // NOTE: stop the world is a no-op in default build _PyEval_StopTheWorld(interp); - int err = append_referrents(result, args); + int err = append_referrents(result, varargs); _PyEval_StartTheWorld(interp); if (err < 0) { Py_CLEAR(result); } + Py_DECREF(varargs); return result; } diff --git a/Objects/clinic/setobject.c.h b/Objects/clinic/setobject.c.h index 3853ce3bce685b..d6e381a9975050 100644 --- a/Objects/clinic/setobject.c.h +++ b/Objects/clinic/setobject.c.h @@ -41,28 +41,22 @@ PyDoc_STRVAR(set_update__doc__, {"update", _PyCFunction_CAST(set_update), METH_FASTCALL, set_update__doc__}, static PyObject * -set_update_impl(PySetObject *so, PyObject *args); +set_update_impl(PySetObject *so, Py_ssize_t nargs, PyObject *const *args); static PyObject * set_update(PySetObject *so, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; - PyObject *__clinic_args = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("update", nargs, 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_New(nargs - 0); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 0; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[0 + i])); - } - return_value = set_update_impl(so, __clinic_args); + __clinic_args = args + 0; + return_value = set_update_impl(so, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } @@ -148,28 +142,22 @@ PyDoc_STRVAR(set_union__doc__, {"union", _PyCFunction_CAST(set_union), METH_FASTCALL, set_union__doc__}, static PyObject * -set_union_impl(PySetObject *so, PyObject *args); +set_union_impl(PySetObject *so, Py_ssize_t nargs, PyObject *const *args); static PyObject * set_union(PySetObject *so, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; - PyObject *__clinic_args = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("union", nargs, 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_New(nargs - 0); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 0; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[0 + i])); - } - return_value = set_union_impl(so, __clinic_args); + __clinic_args = args + 0; + return_value = set_union_impl(so, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } @@ -183,28 +171,23 @@ PyDoc_STRVAR(set_intersection_multi__doc__, {"intersection", _PyCFunction_CAST(set_intersection_multi), METH_FASTCALL, set_intersection_multi__doc__}, static PyObject * -set_intersection_multi_impl(PySetObject *so, PyObject *args); +set_intersection_multi_impl(PySetObject *so, Py_ssize_t nargs, + PyObject *const *args); static PyObject * set_intersection_multi(PySetObject *so, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; - PyObject *__clinic_args = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("intersection", nargs, 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_New(nargs - 0); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 0; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[0 + i])); - } - return_value = set_intersection_multi_impl(so, __clinic_args); + __clinic_args = args + 0; + return_value = set_intersection_multi_impl(so, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } @@ -218,28 +201,23 @@ PyDoc_STRVAR(set_intersection_update_multi__doc__, {"intersection_update", _PyCFunction_CAST(set_intersection_update_multi), METH_FASTCALL, set_intersection_update_multi__doc__}, static PyObject * -set_intersection_update_multi_impl(PySetObject *so, PyObject *args); +set_intersection_update_multi_impl(PySetObject *so, Py_ssize_t nargs, + PyObject *const *args); static PyObject * set_intersection_update_multi(PySetObject *so, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; - PyObject *__clinic_args = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("intersection_update", nargs, 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_New(nargs - 0); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 0; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[0 + i])); - } - return_value = set_intersection_update_multi_impl(so, __clinic_args); + __clinic_args = args + 0; + return_value = set_intersection_update_multi_impl(so, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } @@ -277,28 +255,23 @@ PyDoc_STRVAR(set_difference_update__doc__, {"difference_update", _PyCFunction_CAST(set_difference_update), METH_FASTCALL, set_difference_update__doc__}, static PyObject * -set_difference_update_impl(PySetObject *so, PyObject *args); +set_difference_update_impl(PySetObject *so, Py_ssize_t nargs, + PyObject *const *args); static PyObject * set_difference_update(PySetObject *so, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; - PyObject *__clinic_args = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("difference_update", nargs, 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_New(nargs - 0); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 0; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[0 + i])); - } - return_value = set_difference_update_impl(so, __clinic_args); + __clinic_args = args + 0; + return_value = set_difference_update_impl(so, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } @@ -312,28 +285,23 @@ PyDoc_STRVAR(set_difference_multi__doc__, {"difference", _PyCFunction_CAST(set_difference_multi), METH_FASTCALL, set_difference_multi__doc__}, static PyObject * -set_difference_multi_impl(PySetObject *so, PyObject *args); +set_difference_multi_impl(PySetObject *so, Py_ssize_t nargs, + PyObject *const *args); static PyObject * set_difference_multi(PySetObject *so, PyObject *const *args, Py_ssize_t nargs) { PyObject *return_value = NULL; - PyObject *__clinic_args = NULL; + Py_ssize_t nvararg = nargs - 0; + PyObject *const *__clinic_args = NULL; if (!_PyArg_CheckPositional("difference", nargs, 0, PY_SSIZE_T_MAX)) { goto exit; } - __clinic_args = PyTuple_New(nargs - 0); - if (!__clinic_args) { - goto exit; - } - for (Py_ssize_t i = 0; i < nargs - 0; ++i) { - PyTuple_SET_ITEM(__clinic_args, i, Py_NewRef(args[0 + i])); - } - return_value = set_difference_multi_impl(so, __clinic_args); + __clinic_args = args + 0; + return_value = set_difference_multi_impl(so, nvararg, __clinic_args); exit: - Py_XDECREF(__clinic_args); return return_value; } @@ -568,4 +536,4 @@ set___sizeof__(PySetObject *so, PyObject *Py_UNUSED(ignored)) return return_value; } -/*[clinic end generated code: output=de4ee725bd29f758 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=9d4b41191b2c602f input=a9049054013a1b77]*/ diff --git a/Objects/setobject.c b/Objects/setobject.c index 9f40e085f06fa6..66d7fc730c555c 100644 --- a/Objects/setobject.c +++ b/Objects/setobject.c @@ -1060,13 +1060,13 @@ Update the set, adding elements from all others. [clinic start generated code]*/ static PyObject * -set_update_impl(PySetObject *so, PyObject *args) -/*[clinic end generated code: output=34f6371704974c8a input=df4fe486e38cd337]*/ +set_update_impl(PySetObject *so, Py_ssize_t nargs, PyObject *const *args) +/*[clinic end generated code: output=050e2a21f8d7d16a input=df4fe486e38cd337]*/ { Py_ssize_t i; - for (i=0 ; i<PyTuple_GET_SIZE(args) ; i++) { - PyObject *other = PyTuple_GET_ITEM(args, i); + for (i = 0; i < nargs; i++) { + PyObject *other = args[i]; if (set_update_internal(so, other)) return NULL; } @@ -1289,8 +1289,8 @@ Return a new set with elements from the set and all others. [clinic start generated code]*/ static PyObject * -set_union_impl(PySetObject *so, PyObject *args) -/*[clinic end generated code: output=2c83d05a446a1477 input=ddf088706e9577b2]*/ +set_union_impl(PySetObject *so, Py_ssize_t nargs, PyObject *const *args) +/*[clinic end generated code: output=f68ec24d5c19d404 input=ddf088706e9577b2]*/ { PySetObject *result; PyObject *other; @@ -1300,8 +1300,8 @@ set_union_impl(PySetObject *so, PyObject *args) if (result == NULL) return NULL; - for (i=0 ; i<PyTuple_GET_SIZE(args) ; i++) { - other = PyTuple_GET_ITEM(args, i); + for (i = 0; i < nargs; i++) { + other = args[i]; if ((PyObject *)so == other) continue; if (set_update_local(result, other)) { @@ -1440,18 +1440,19 @@ Return a new set with elements common to the set and all others. [clinic start generated code]*/ static PyObject * -set_intersection_multi_impl(PySetObject *so, PyObject *args) -/*[clinic end generated code: output=2406ef3387adbe2f input=0d9f3805ccbba6a4]*/ +set_intersection_multi_impl(PySetObject *so, Py_ssize_t nargs, + PyObject *const *args) +/*[clinic end generated code: output=ef0756ddb5f2dee9 input=0d9f3805ccbba6a4]*/ { Py_ssize_t i; - if (PyTuple_GET_SIZE(args) == 0) { + if (nargs == 0) { return set_copy(so, NULL); } PyObject *result = Py_NewRef(so); - for (i=0 ; i<PyTuple_GET_SIZE(args) ; i++) { - PyObject *other = PyTuple_GET_ITEM(args, i); + for (i = 0; i < nargs; i++) { + PyObject *other = args[i]; PyObject *newresult; Py_BEGIN_CRITICAL_SECTION2(result, other); newresult = set_intersection((PySetObject *)result, other); @@ -1487,12 +1488,13 @@ Update the set, keeping only elements found in it and all others. [clinic start generated code]*/ static PyObject * -set_intersection_update_multi_impl(PySetObject *so, PyObject *args) -/*[clinic end generated code: output=251c1f729063609d input=223c1e086aa669a9]*/ +set_intersection_update_multi_impl(PySetObject *so, Py_ssize_t nargs, + PyObject *const *args) +/*[clinic end generated code: output=808d7ad1935b1dfe input=223c1e086aa669a9]*/ { PyObject *tmp; - tmp = set_intersection_multi_impl(so, args); + tmp = set_intersection_multi_impl(so, nargs, args); if (tmp == NULL) return NULL; Py_BEGIN_CRITICAL_SECTION(so); @@ -1676,13 +1678,14 @@ Update the set, removing elements found in others. [clinic start generated code]*/ static PyObject * -set_difference_update_impl(PySetObject *so, PyObject *args) -/*[clinic end generated code: output=28685b2fc63e41c4 input=024e6baa6fbcbb3d]*/ +set_difference_update_impl(PySetObject *so, Py_ssize_t nargs, + PyObject *const *args) +/*[clinic end generated code: output=55f850c27748d312 input=024e6baa6fbcbb3d]*/ { Py_ssize_t i; - for (i=0 ; i<PyTuple_GET_SIZE(args) ; i++) { - PyObject *other = PyTuple_GET_ITEM(args, i); + for (i = 0; i < nargs; i++) { + PyObject *other = args[i]; int rv; Py_BEGIN_CRITICAL_SECTION2(so, other); rv = set_difference_update_internal(so, other); @@ -1793,25 +1796,26 @@ Return a new set with elements in the set that are not in the others. [clinic start generated code]*/ static PyObject * -set_difference_multi_impl(PySetObject *so, PyObject *args) -/*[clinic end generated code: output=3130c3bb3cac873d input=ba78ea5f099e58df]*/ +set_difference_multi_impl(PySetObject *so, Py_ssize_t nargs, + PyObject *const *args) +/*[clinic end generated code: output=8150d008c00523f3 input=ba78ea5f099e58df]*/ { Py_ssize_t i; PyObject *result, *other; - if (PyTuple_GET_SIZE(args) == 0) { + if (nargs == 0) { return set_copy(so, NULL); } - other = PyTuple_GET_ITEM(args, 0); + other = args[0]; Py_BEGIN_CRITICAL_SECTION2(so, other); result = set_difference(so, other); Py_END_CRITICAL_SECTION2(); if (result == NULL) return NULL; - for (i=1 ; i<PyTuple_GET_SIZE(args) ; i++) { - other = PyTuple_GET_ITEM(args, i); + for (i = 1; i < nargs; i++) { + other = args[i]; int rv; Py_BEGIN_CRITICAL_SECTION(other); rv = set_difference_update_internal((PySetObject *)result, other); diff --git a/Tools/clinic/libclinic/clanguage.py b/Tools/clinic/libclinic/clanguage.py index 73d47833d97294..32aba81ab8a850 100644 --- a/Tools/clinic/libclinic/clanguage.py +++ b/Tools/clinic/libclinic/clanguage.py @@ -15,7 +15,7 @@ Module, Class, Function, Parameter, permute_optional_groups, GETTER, SETTER, METHOD_INIT) -from libclinic.converters import self_converter +from libclinic.converters import defining_class_converter, self_converter from libclinic.parse_args import ParseArgsCodeGen if TYPE_CHECKING: from libclinic.app import Clinic @@ -396,6 +396,12 @@ def render_function( first_optional = len(selfless) positional = selfless and selfless[-1].is_positional_only() has_option_groups = False + requires_defining_class = (len(selfless) + and isinstance(selfless[0].converter, + defining_class_converter)) + pass_vararg_directly = (all(p.is_positional_only() or p.is_vararg() + for p in selfless) + and not requires_defining_class) # offset i by -1 because first_optional needs to ignore self for i, p in enumerate(parameters, -1): @@ -404,7 +410,7 @@ def render_function( if (i != -1) and (p.default is not unspecified): first_optional = min(first_optional, i) - if p.is_vararg(): + if p.is_vararg() and not pass_vararg_directly: data.cleanup.append(f"Py_XDECREF({c.parser_name});") # insert group variable @@ -418,6 +424,11 @@ def render_function( data.impl_parameters.append("int " + group_name) has_option_groups = True + if p.is_vararg() and pass_vararg_directly: + data.impl_arguments.append('nvararg') + data.impl_parameters.append('Py_ssize_t nargs') + p.converter.type = 'PyObject *const *' + c.render(p, data) if has_option_groups and (not positional): diff --git a/Tools/clinic/libclinic/parse_args.py b/Tools/clinic/libclinic/parse_args.py index 96c9b919bff811..559d4fbdd09c57 100644 --- a/Tools/clinic/libclinic/parse_args.py +++ b/Tools/clinic/libclinic/parse_args.py @@ -469,7 +469,11 @@ def parse_pos_only(self) -> None: nargs = 'PyTuple_GET_SIZE(args)' argname_fmt = 'PyTuple_GET_ITEM(args, %d)' - left_args = f"{nargs} - {self.max_pos}" + if self.vararg != NO_VARARG: + self.declarations = f"Py_ssize_t nvararg = {nargs} - {self.max_pos};" + else: + self.declarations = "" + max_args = NO_VARARG if (self.vararg != NO_VARARG) else self.max_pos if self.limited_capi: parser_code = [] @@ -518,30 +522,13 @@ def parse_pos_only(self) -> None: use_parser_code = True for i, p in enumerate(self.parameters): if p.is_vararg(): + var = p.converter.parser_name if self.fastcall: - parser_code.append(libclinic.normalize_snippet(""" - %s = PyTuple_New(%s); - if (!%s) {{ - goto exit; - }} - for (Py_ssize_t i = 0; i < %s; ++i) {{ - PyTuple_SET_ITEM(%s, i, Py_NewRef(args[%d + i])); - }} - """ % ( - p.converter.parser_name, - left_args, - p.converter.parser_name, - left_args, - p.converter.parser_name, - self.max_pos - ), indent=4)) + code = f"{var} = args + {self.vararg};" else: - parser_code.append(libclinic.normalize_snippet(""" - %s = PyTuple_GetSlice(%d, -1); - """ % ( - p.converter.parser_name, - self.max_pos - ), indent=4)) + code = f"{var} = _PyTuple_CAST(args)->ob_item;" + formatted_code = libclinic.normalize_snippet(code, indent=4) + parser_code.append(formatted_code) continue displayname = p.get_displayname(i+1) @@ -588,7 +575,7 @@ def parse_pos_only(self) -> None: goto exit; }} """, indent=4)] - self.parser_body(*parser_code) + self.parser_body(*parser_code, declarations=self.declarations) def parse_general(self, clang: CLanguage) -> None: parsearg: str | None

1 0

[3.12] gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103) (#126230)
by sobolevn Oct. 31, 2024

Oct. 31, 2024

https://github.com/python/cpython/commit/90de322b32a72c69afafa2fb8796769233… commit: 90de322b32a72c69afafa2fb879676923357cf3a branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: sobolevn <mail(a)sobolevn.me> date: 2024-10-31T08:19:56Z summary: [3.12] gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103) (#126230) gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103) (cherry picked from commit d07dcce6935364cab807e0df931ed09b088ade69) Co-authored-by: Nico-Posada <102486290+Nico-Posada(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst M Lib/test/test_asyncio/test_tasks.py M Modules/_asynciomodule.c diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py index 6e8a51ce2555d5..792ab4fbede0a2 100644 --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -2489,6 +2489,28 @@ def test_get_context(self): finally: loop.close() + def test_proper_refcounts(self): + # see: https://github.com/python/cpython/issues/126083 + class Break: + def __str__(self): + raise RuntimeError("break") + + obj = object() + initial_refcount = sys.getrefcount(obj) + + coro = coroutine_function() + loop = asyncio.new_event_loop() + task = asyncio.Task.__new__(asyncio.Task) + + for _ in range(5): + with self.assertRaisesRegex(RuntimeError, 'break'): + task.__init__(coro, loop=loop, context=obj, name=Break()) + + coro.close() + del task + + self.assertEqual(sys.getrefcount(obj), initial_refcount) + def add_subclass_tests(cls): BaseTask = cls.Task diff --git a/Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst b/Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst new file mode 100644 index 00000000000000..d64b7dd2fedbd6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst @@ -0,0 +1 @@ +Fixed a reference leak in :class:`asyncio.Task` objects when reinitializing the same object with a non-``None`` context. Patch by Nico Posada. diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index 9016c077e0d657..96821762c5ae68 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -2143,7 +2143,7 @@ _asyncio_Task___init___impl(TaskObj *self, PyObject *coro, PyObject *loop, return -1; } } else { - self->task_context = Py_NewRef(context); + Py_XSETREF(self->task_context, Py_NewRef(context)); } Py_CLEAR(self->task_fut_waiter);

1 0

[3.13] gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103) (#126229)
by sobolevn Oct. 31, 2024

Oct. 31, 2024

https://github.com/python/cpython/commit/ac00bf0e96d93adf20879217dc7022c7c3… commit: ac00bf0e96d93adf20879217dc7022c7c3a5dc89 branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: sobolevn <mail(a)sobolevn.me> date: 2024-10-31T08:14:00Z summary: [3.13] gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103) (#126229) gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (GH-126103) (cherry picked from commit d07dcce6935364cab807e0df931ed09b088ade69) Co-authored-by: Nico-Posada <102486290+Nico-Posada(a)users.noreply.github.com> files: A Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst M Lib/test/test_asyncio/test_tasks.py M Modules/_asynciomodule.c diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py index 5b09c81faef62a..313eed9d715158 100644 --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -2684,6 +2684,28 @@ def test_get_context(self): finally: loop.close() + def test_proper_refcounts(self): + # see: https://github.com/python/cpython/issues/126083 + class Break: + def __str__(self): + raise RuntimeError("break") + + obj = object() + initial_refcount = sys.getrefcount(obj) + + coro = coroutine_function() + loop = asyncio.new_event_loop() + task = asyncio.Task.__new__(asyncio.Task) + + for _ in range(5): + with self.assertRaisesRegex(RuntimeError, 'break'): + task.__init__(coro, loop=loop, context=obj, name=Break()) + + coro.close() + del task + + self.assertEqual(sys.getrefcount(obj), initial_refcount) + def add_subclass_tests(cls): BaseTask = cls.Task diff --git a/Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst b/Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst new file mode 100644 index 00000000000000..d64b7dd2fedbd6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst @@ -0,0 +1 @@ +Fixed a reference leak in :class:`asyncio.Task` objects when reinitializing the same object with a non-``None`` context. Patch by Nico Posada. diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index c47b921d72ae3a..44529f633fbe2e 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -2070,7 +2070,7 @@ _asyncio_Task___init___impl(TaskObj *self, PyObject *coro, PyObject *loop, return -1; } } else { - self->task_context = Py_NewRef(context); + Py_XSETREF(self->task_context, Py_NewRef(context)); } Py_CLEAR(self->task_fut_waiter);

1 0

gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (#126103)
by sobolevn Oct. 31, 2024

Oct. 31, 2024

https://github.com/python/cpython/commit/d07dcce6935364cab807e0df931ed09b08… commit: d07dcce6935364cab807e0df931ed09b088ade69 branch: main author: Nico-Posada <102486290+Nico-Posada(a)users.noreply.github.com> committer: sobolevn <mail(a)sobolevn.me> date: 2024-10-31T10:47:57+03:00 summary: gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (#126103) files: A Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst M Lib/test/test_asyncio/test_tasks.py M Modules/_asynciomodule.c diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py index a1013ab803348d..9d2d356631b42c 100644 --- a/Lib/test/test_asyncio/test_tasks.py +++ b/Lib/test/test_asyncio/test_tasks.py @@ -2688,6 +2688,28 @@ def test_get_context(self): finally: loop.close() + def test_proper_refcounts(self): + # see: https://github.com/python/cpython/issues/126083 + class Break: + def __str__(self): + raise RuntimeError("break") + + obj = object() + initial_refcount = sys.getrefcount(obj) + + coro = coroutine_function() + loop = asyncio.new_event_loop() + task = asyncio.Task.__new__(asyncio.Task) + + for _ in range(5): + with self.assertRaisesRegex(RuntimeError, 'break'): + task.__init__(coro, loop=loop, context=obj, name=Break()) + + coro.close() + del task + + self.assertEqual(sys.getrefcount(obj), initial_refcount) + def add_subclass_tests(cls): BaseTask = cls.Task diff --git a/Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst b/Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst new file mode 100644 index 00000000000000..d64b7dd2fedbd6 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-28-22-35-22.gh-issue-126083.TuI--n.rst @@ -0,0 +1 @@ +Fixed a reference leak in :class:`asyncio.Task` objects when reinitializing the same object with a non-``None`` context. Patch by Nico Posada. diff --git a/Modules/_asynciomodule.c b/Modules/_asynciomodule.c index d4135f04e56575..c2500fbd692d4d 100644 --- a/Modules/_asynciomodule.c +++ b/Modules/_asynciomodule.c @@ -2120,7 +2120,7 @@ _asyncio_Task___init___impl(TaskObj *self, PyObject *coro, PyObject *loop, return -1; } } else { - self->task_context = Py_NewRef(context); + Py_XSETREF(self->task_context, Py_NewRef(context)); } Py_CLEAR(self->task_fut_waiter);

1 0

[3.13] gh-125818: Fix incorrect signature of argument `skip_file_prefixes` in warnings docs (GH-125823) (#126216)
by willingc Oct. 31, 2024

Oct. 31, 2024

https://github.com/python/cpython/commit/57668a4be586fad22b5d6337a1e5e1e724… commit: 57668a4be586fad22b5d6337a1e5e1e7246caa92 branch: 3.13 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: willingc <carolcode(a)willingconsulting.com> date: 2024-10-30T18:32:04-07:00 summary: [3.13] gh-125818: Fix incorrect signature of argument `skip_file_prefixes` in warnings docs (GH-125823) (#126216) gh-125818: Fix incorrect signature of argument `skip_file_prefixes` in warnings docs (GH-125823) Change documentation (cherry picked from commit d467d9246cbe0ce5dc149c4c74223bb8374ece73) Co-authored-by: RUANG (James Roy) <longjinyii(a)outlook.com> files: M Doc/library/warnings.rst diff --git a/Doc/library/warnings.rst b/Doc/library/warnings.rst index 83163cc688297c..e4942fa0c184bc 100644 --- a/Doc/library/warnings.rst +++ b/Doc/library/warnings.rst @@ -396,7 +396,7 @@ Available Functions ------------------- -.. function:: warn(message, category=None, stacklevel=1, source=None, *, skip_file_prefixes=None) +.. function:: warn(message, category=None, stacklevel=1, source=None, *, skip_file_prefixes=()) Issue a warning, or maybe ignore it or raise an exception. The *category* argument, if given, must be a :ref:`warning category class <warning-categories>`; it

1 0