Mailman 3 December 2020 - Python-checkins

Minor improvements to the convolve() recipe (GH-24012)
by rhettinger Dec. 30, 2020

Dec. 30, 2020

https://github.com/python/cpython/commit/f421bfce80730cb0ff5cbe14727ac30cf0… commit: f421bfce80730cb0ff5cbe14727ac30cf0462eed branch: master author: Raymond Hettinger <rhettinger(a)users.noreply.github.com> committer: rhettinger <rhettinger(a)users.noreply.github.com> date: 2020-12-30T12:51:19-08:00 summary: Minor improvements to the convolve() recipe (GH-24012) * Minor improvement to speed and space efficiency for the convolve() recipe * Don't require convolve's kernel to be a sequence. files: M Doc/library/itertools.rst diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 03416a0a7fbcb..fd77f99a88f57 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -791,9 +791,9 @@ which incur interpreter overhead. # convolve(data, [0.25, 0.25, 0.25, 0.25]) --> Moving average (blur) # convolve(data, [1, -1]) --> 1st finite difference (1st derivative) # convolve(data, [1, -2, 1]) --> 2nd finite difference (2nd derivative) - kernel = list(reversed(kernel)) + kernel = tuple(kernel)[::-1] n = len(kernel) - window = collections.deque([0] * n, maxlen=n) + window = collections.deque([0], maxlen=n) * n for x in chain(signal, repeat(0, n-1)): window.append(x) yield sum(map(operator.mul, kernel, window))

1 0

bpo-42773: fix tests not being run on pushes (GH-24004)
by miss-islington Dec. 30, 2020

Dec. 30, 2020

https://github.com/python/cpython/commit/4ac923f2756f835f512339ee181348cc53… commit: 4ac923f2756f835f512339ee181348cc535ab07f branch: master author: Filipe Laíns <lains(a)riseup.net> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2020-12-30T06:53:58-08:00 summary: bpo-42773: fix tests not being run on pushes (GH-24004) There was a typo, we were checking if the "GITHUB_BASE_REF" string literal was empty instead of the $GITHUB_BASE_REF value. When $GITHUB_BASE_REF is empty, the action that triggered the run was not a pull request, so we always run the full test suite. Signed-off-by: Filipe Laíns <lains(a)riseup.net> files: M .github/workflows/build.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 1b865a17b09dd..6f05310e6e586 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -28,7 +28,7 @@ jobs: - name: Check for source changes id: check run: | - if [ -z "GITHUB_BASE_REF" ]; then + if [ -z "$GITHUB_BASE_REF" ]; then echo '::set-output name=run_tests::true' else git fetch origin $GITHUB_BASE_REF --depth=1

1 0

bpo-27640: Add --disable-test-modules configure option (GH-23886)
by vstinner Dec. 30, 2020

Dec. 30, 2020

https://github.com/python/cpython/commit/277ce3060becc120f1c877346562bd6880… commit: 277ce3060becc120f1c877346562bd6880f4be71 branch: master author: pxinwr <peixing.xin(a)windriver.com> committer: vstinner <vstinner(a)python.org> date: 2020-12-30T13:50:39+01:00 summary: bpo-27640: Add --disable-test-modules configure option (GH-23886) Added --disable-test-modules option to the configure script: don't build nor install test modules. Patch by Xavier de Gaye, Thomas Petazzoni and Peixing Xin. Co-Authored-By: Thomas Petazzoni <thomas.petazzoni(a)free-electrons.com> Co-Authored-By: Xavier de Gaye <xdegaye(a)gmail.com> files: A Misc/NEWS.d/next/Build/2020-12-22-17-57-04.bpo-27640.j3a8r0.rst M Doc/whatsnew/3.10.rst M Makefile.pre.in M configure M configure.ac M setup.py diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index b5fb1e9a629c1..db34b33b84a49 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -558,6 +558,10 @@ Build Changes * The :mod:`atexit` module must now always be built as a built-in module. (Contributed by Victor Stinner in :issue:`42639`.) +* Added ``--disable-test-modules`` option to the ``configure`` script: + don't build nor install test modules. + (Contributed by Xavier de Gaye, Thomas Petazzoni and Peixing Xin in :issue:`27640`.) + C API Changes ============= diff --git a/Makefile.pre.in b/Makefile.pre.in index 5c93b0b3fa9c6..d8b9e8498d51b 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -1366,18 +1366,54 @@ maninstall: altmaninstall # Install the library XMLLIBSUBDIRS= xml xml/dom xml/etree xml/parsers xml/sax -LIBSUBDIRS= tkinter tkinter/test tkinter/test/test_tkinter \ - tkinter/test/test_ttk site-packages test \ - test/audiodata \ - test/capath test/data \ - test/cjkencodings test/decimaltestdata \ - test/xmltestdata test/xmltestdata/c14n-20 \ - test/dtracedata \ - test/eintrdata \ - test/imghdrdata \ - test/libregrtest \ - test/subprocessdata test/sndhdrdata test/support \ - test/tracedmodules test/encoded_modules \ +LIBSUBDIRS= asyncio \ + collections \ + concurrent concurrent/futures \ + csv \ + ctypes ctypes/macholib \ + curses \ + dbm \ + distutils distutils/command \ + email email/mime \ + encodings \ + ensurepip ensurepip/_bundled \ + html \ + http \ + idlelib idlelib/Icons \ + importlib importlib/metadata \ + json \ + lib2to3 lib2to3/fixes lib2to3/pgen2 \ + logging \ + multiprocessing multiprocessing/dummy \ + pydoc_data \ + site-packages \ + sqlite3 \ + tkinter \ + turtledemo \ + unittest \ + urllib \ + venv venv/scripts venv/scripts/common venv/scripts/posix \ + wsgiref \ + $(XMLLIBSUBDIRS) \ + xmlrpc \ + zoneinfo +TESTSUBDIRS= ctypes/test \ + distutils/tests \ + idlelib/idle_test \ + lib2to3/tests \ + lib2to3/tests/data \ + lib2to3/tests/data/fixers \ + lib2to3/tests/data/fixers/myfixes \ + sqlite3/test \ + test test/audiodata \ + test/capath test/cjkencodings \ + test/data test/decimaltestdata \ + test/dtracedata test/eintrdata \ + test/encoded_modules test/imghdrdata \ + test/libregrtest test/sndhdrdata \ + test/subprocessdata test/support \ + test/test_asyncio \ + test/test_email test/test_email/data \ test/test_import \ test/test_import/data \ test/test_import/data/circular_imports \ @@ -1385,8 +1421,6 @@ LIBSUBDIRS= tkinter tkinter/test tkinter/test/test_tkinter \ test/test_import/data/package \ test/test_import/data/package2 \ test/test_import/data/unwritable \ - importlib \ - importlib/metadata \ test/test_importlib \ test/test_importlib/builtin \ test/test_importlib/data \ @@ -1425,30 +1459,19 @@ LIBSUBDIRS= tkinter tkinter/test tkinter/test/test_tkinter \ test/test_importlib/source \ test/test_importlib/zipdata01 \ test/test_importlib/zipdata02 \ + test/test_json \ + test/test_peg_generator \ + test/test_tools \ + test/test_warnings test/test_warnings/data \ test/test_zoneinfo test/test_zoneinfo/data \ + test/tracedmodules \ + test/xmltestdata test/xmltestdata/c14n-20 \ test/ziptestdata \ - asyncio \ - test/test_asyncio \ - collections concurrent concurrent/futures encodings \ - email email/mime test/test_email test/test_email/data \ - ensurepip ensurepip/_bundled \ - html json test/test_json http dbm xmlrpc \ - sqlite3 sqlite3/test \ - logging csv wsgiref urllib \ - lib2to3 lib2to3/fixes lib2to3/pgen2 lib2to3/tests \ - lib2to3/tests/data lib2to3/tests/data/fixers \ - lib2to3/tests/data/fixers/myfixes \ - ctypes ctypes/test ctypes/macholib \ - idlelib idlelib/Icons idlelib/idle_test \ - distutils distutils/command distutils/tests $(XMLLIBSUBDIRS) \ - test/test_peg_generator \ - test/test_tools test/test_warnings test/test_warnings/data \ - turtledemo \ - multiprocessing multiprocessing/dummy \ - unittest unittest/test unittest/test/testmock \ - venv venv/scripts venv/scripts/common venv/scripts/posix \ - curses pydoc_data \ - zoneinfo + tkinter/test tkinter/test/test_tkinter \ + tkinter/test/test_ttk \ + unittest/test unittest/test/testmock + +TEST_MODULES=@TEST_MODULES@ libinstall: build_all $(srcdir)/Modules/xxmodule.c @for i in $(SCRIPTDIR) $(LIBDEST); \ do \ @@ -1458,7 +1481,12 @@ libinstall: build_all $(srcdir)/Modules/xxmodule.c else true; \ fi; \ done - @for d in $(LIBSUBDIRS); \ + @if test "$(TEST_MODULES)" = yes; then \ + subdirs="$(LIBSUBDIRS) $(TESTSUBDIRS)"; \ + else \ + subdirs="$(LIBSUBDIRS)"; \ + fi; \ + for d in $$subdirs; \ do \ a=$(srcdir)/Lib/$$d; \ if test ! -d $$a; then continue; else true; fi; \ @@ -1479,7 +1507,12 @@ libinstall: build_all $(srcdir)/Modules/xxmodule.c echo $(INSTALL_DATA) $$i $(LIBDEST); \ fi; \ done - @for d in $(LIBSUBDIRS); \ + @if test "$(TEST_MODULES)" = yes; then \ + subdirs="$(LIBSUBDIRS) $(TESTSUBDIRS)"; \ + else \ + subdirs="$(LIBSUBDIRS)"; \ + fi; \ + for d in $$subdirs; \ do \ a=$(srcdir)/Lib/$$d; \ if test ! -d $$a; then continue; else true; fi; \ diff --git a/Misc/NEWS.d/next/Build/2020-12-22-17-57-04.bpo-27640.j3a8r0.rst b/Misc/NEWS.d/next/Build/2020-12-22-17-57-04.bpo-27640.j3a8r0.rst new file mode 100644 index 0000000000000..142590d05ef85 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2020-12-22-17-57-04.bpo-27640.j3a8r0.rst @@ -0,0 +1,3 @@ +Added ``--disable-test-modules`` option to the ``configure`` script: +don't build nor install test modules. +Patch by Xavier de Gaye, Thomas Petazzoni and Peixing Xin. diff --git a/configure b/configure index 530c04a0edee3..5691c27cf66fe 100755 --- a/configure +++ b/configure @@ -623,6 +623,7 @@ ac_includes_default="\ #endif" ac_subst_vars='LTLIBOBJS +TEST_MODULES OPENSSL_LDFLAGS OPENSSL_LIBS OPENSSL_INCLUDES @@ -852,6 +853,7 @@ with_openssl with_ssl_default_suites with_builtin_hashlib_hashes with_experimental_isolated_subinterpreters +enable_test_modules ' ac_precious_vars='build_alias host_alias @@ -1503,6 +1505,7 @@ Optional Features: --enable-big-digits[=15|30] use big digits (30 or 15 bits) for Python longs (default is system-dependent)] + --disable-test-modules don't build nor install test modules Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] @@ -17725,6 +17728,26 @@ $as_echo "no" >&6; } fi +# Check whether to disable test modules. Once set, setup.py will not build +# test extension modules and "make install" will not install test suites. +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --disable-test-modules" >&5 +$as_echo_n "checking for --disable-test-modules... " >&6; } +# Check whether --enable-test-modules was given. +if test "${enable_test_modules+set}" = set; then : + enableval=$enable_test_modules; +fi + +if test "$enable_test_modules" = no; then + TEST_MODULES=no + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } +else + TEST_MODULES=yes + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + # generate output files ac_config_files="$ac_config_files Makefile.pre Misc/python.pc Misc/python-embed.pc Misc/python-config.sh" @@ -19051,4 +19074,3 @@ if test "$Py_OPT" = 'false' -a "$Py_DEBUG" != 'true'; then echo "" >&6 echo "" >&6 fi - diff --git a/configure.ac b/configure.ac index 39eadfedfba02..990d6bfdd81b7 100644 --- a/configure.ac +++ b/configure.ac @@ -5834,6 +5834,20 @@ else fi], [AC_MSG_RESULT(no)]) +# Check whether to disable test modules. Once set, setup.py will not build +# test extension modules and "make install" will not install test suites. +AC_MSG_CHECKING(for --disable-test-modules) +AC_ARG_ENABLE(test-modules, + AS_HELP_STRING([--disable-test-modules], [don't build nor install test modules])) +if test "$enable_test_modules" = no; then + TEST_MODULES=no + AC_MSG_RESULT(yes) +else + TEST_MODULES=yes + AC_MSG_RESULT(no) +fi +AC_SUBST(TEST_MODULES) + # generate output files AC_CONFIG_FILES(Makefile.pre Misc/python.pc Misc/python-embed.pc Misc/python-config.sh) @@ -5860,4 +5874,3 @@ if test "$Py_OPT" = 'false' -a "$Py_DEBUG" != 'true'; then echo "" >&AS_MESSAGE_FD echo "" >&AS_MESSAGE_FD fi - diff --git a/setup.py b/setup.py index e055e44b0f175..8598d2aa5dbce 100644 --- a/setup.py +++ b/setup.py @@ -41,7 +41,7 @@ # Compile extensions used to test Python? -TEST_EXTENSIONS = True +TEST_EXTENSIONS = (sysconfig.get_config_var('TEST_MODULES') == 'yes') # This global variable is used to hold the list of modules to be disabled. DISABLED_MODULE_LIST = []

1 0

bpo-27794: Add `name` attribute to `property` class (GH-23967)
by rhettinger Dec. 30, 2020

Dec. 30, 2020

https://github.com/python/cpython/commit/c56387f80c5aabf8100ceaffe365cc004c… commit: c56387f80c5aabf8100ceaffe365cc004ce0d7e0 branch: master author: Yurii Karabas <1998uriyyo(a)gmail.com> committer: rhettinger <rhettinger(a)users.noreply.github.com> date: 2020-12-30T01:51:24-08:00 summary: bpo-27794: Add `name` attribute to `property` class (GH-23967) files: A Misc/NEWS.d/next/Core and Builtins/2020-12-27-18-07-43.bpo-27794.sxgfGi.rst M Doc/howto/descriptor.rst M Lib/test/test_property.py M Lib/test/test_sys.py M Objects/descrobject.c diff --git a/Doc/howto/descriptor.rst b/Doc/howto/descriptor.rst index ab5a573c6a06d..d172c9b181c1c 100644 --- a/Doc/howto/descriptor.rst +++ b/Doc/howto/descriptor.rst @@ -934,32 +934,42 @@ here is a pure Python equivalent: if doc is None and fget is not None: doc = fget.__doc__ self.__doc__ = doc + self._name = '' + + def __set_name__(self, owner, name): + self._name = name def __get__(self, obj, objtype=None): if obj is None: return self if self.fget is None: - raise AttributeError("unreadable attribute") + raise AttributeError(f'unreadable attribute {self._name}') return self.fget(obj) def __set__(self, obj, value): if self.fset is None: - raise AttributeError("can't set attribute") + raise AttributeError(f"can't set attribute {self._name}") self.fset(obj, value) def __delete__(self, obj): if self.fdel is None: - raise AttributeError("can't delete attribute") + raise AttributeError(f"can't delete attribute {self._name}") self.fdel(obj) def getter(self, fget): - return type(self)(fget, self.fset, self.fdel, self.__doc__) + prop = type(self)(fget, self.fset, self.fdel, self.__doc__) + prop._name = self._name + return prop def setter(self, fset): - return type(self)(self.fget, fset, self.fdel, self.__doc__) + prop = type(self)(self.fget, fset, self.fdel, self.__doc__) + prop._name = self._name + return prop def deleter(self, fdel): - return type(self)(self.fget, self.fset, fdel, self.__doc__) + prop = type(self)(self.fget, self.fset, fdel, self.__doc__) + prop._name = self._name + return prop .. testcode:: :hide: diff --git a/Lib/test/test_property.py b/Lib/test/test_property.py index 172737ade143f..7f3813fc8cd15 100644 --- a/Lib/test/test_property.py +++ b/Lib/test/test_property.py @@ -204,6 +204,16 @@ def __doc__(cls): return 'Second' self.assertEqual(A.__doc__, 'Second') + def test_property_set_name_incorrect_args(self): + p = property() + + for i in (0, 1, 3): + with self.assertRaisesRegex( + TypeError, + fr'^__set_name__ takes 2 positional arguments but {i} were given$' + ): + p.__set_name__(*([0] * i)) + # Issue 5890: subclasses of property do not preserve method __doc__ strings class PropertySub(property): @@ -299,6 +309,46 @@ def spam(self): self.assertEqual(Foo.spam.__doc__, "a new docstring") +class _PropertyUnreachableAttribute: + msg_format = None + obj = None + cls = None + + def _format_exc_msg(self, msg): + return self.msg_format.format(msg) + + @classmethod + def setUpClass(cls): + cls.obj = cls.cls() + + def test_get_property(self): + with self.assertRaisesRegex(AttributeError, self._format_exc_msg("unreadable attribute")): + self.obj.foo + + def test_set_property(self): + with self.assertRaisesRegex(AttributeError, self._format_exc_msg("can't set attribute")): + self.obj.foo = None + + def test_del_property(self): + with self.assertRaisesRegex(AttributeError, self._format_exc_msg("can't delete attribute")): + del self.obj.foo + + +class PropertyUnreachableAttributeWithName(_PropertyUnreachableAttribute, unittest.TestCase): + msg_format = "^{} 'foo'$" + + class cls: + foo = property() + + +class PropertyUnreachableAttributeNoName(_PropertyUnreachableAttribute, unittest.TestCase): + msg_format = "^{}$" + + class cls: + pass + + cls.foo = property() + if __name__ == '__main__': unittest.main() diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py index 3860656c181c2..3af5b117affde 100644 --- a/Lib/test/test_sys.py +++ b/Lib/test/test_sys.py @@ -1329,7 +1329,7 @@ def getx(self): return self.__x def setx(self, value): self.__x = value def delx(self): del self.__x x = property(getx, setx, delx, "") - check(x, size('4Pi')) + check(x, size('5Pi')) # PyCapsule # XXX # rangeiterator diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-12-27-18-07-43.bpo-27794.sxgfGi.rst b/Misc/NEWS.d/next/Core and Builtins/2020-12-27-18-07-43.bpo-27794.sxgfGi.rst new file mode 100644 index 0000000000000..0f66b4effc5df --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2020-12-27-18-07-43.bpo-27794.sxgfGi.rst @@ -0,0 +1,3 @@ +Improve the error message for failed writes/deletes to property objects. +When possible, the attribute name is now shown. Patch provided by +Yurii Karabas. diff --git a/Objects/descrobject.c b/Objects/descrobject.c index a8ce13c7aa4ba..16c695a08f47d 100644 --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -1490,6 +1490,7 @@ typedef struct { PyObject *prop_set; PyObject *prop_del; PyObject *prop_doc; + PyObject *prop_name; int getter_doc; } propertyobject; @@ -1535,10 +1536,33 @@ property_deleter(PyObject *self, PyObject *deleter) } +PyDoc_STRVAR(set_name_doc, + "Method to set name of a property."); + +static PyObject * +property_set_name(PyObject *self, PyObject *args) { + if (PyTuple_GET_SIZE(args) != 2) { + PyErr_Format( + PyExc_TypeError, + "__set_name__() takes 2 positional arguments but %d were given", + PyTuple_GET_SIZE(args)); + return NULL; + } + + propertyobject *prop = (propertyobject *)self; + PyObject *name = PyTuple_GET_ITEM(args, 1); + + Py_XINCREF(name); + Py_XSETREF(prop->prop_name, name); + + Py_RETURN_NONE; +} + static PyMethodDef property_methods[] = { {"getter", property_getter, METH_O, getter_doc}, {"setter", property_setter, METH_O, setter_doc}, {"deleter", property_deleter, METH_O, deleter_doc}, + {"__set_name__", property_set_name, METH_VARARGS, set_name_doc}, {0} }; @@ -1553,6 +1577,7 @@ property_dealloc(PyObject *self) Py_XDECREF(gs->prop_set); Py_XDECREF(gs->prop_del); Py_XDECREF(gs->prop_doc); + Py_XDECREF(gs->prop_name); Py_TYPE(self)->tp_free(self); } @@ -1566,7 +1591,12 @@ property_descr_get(PyObject *self, PyObject *obj, PyObject *type) propertyobject *gs = (propertyobject *)self; if (gs->prop_get == NULL) { - PyErr_SetString(PyExc_AttributeError, "unreadable attribute"); + if (gs->prop_name != NULL) { + PyErr_Format(PyExc_AttributeError, "unreadable attribute %R", gs->prop_name); + } else { + PyErr_SetString(PyExc_AttributeError, "unreadable attribute"); + } + return NULL; } @@ -1584,10 +1614,18 @@ property_descr_set(PyObject *self, PyObject *obj, PyObject *value) else func = gs->prop_set; if (func == NULL) { - PyErr_SetString(PyExc_AttributeError, + if (gs->prop_name != NULL) { + PyErr_Format(PyExc_AttributeError, value == NULL ? - "can't delete attribute" : - "can't set attribute"); + "can't delete attribute %R" : + "can't set attribute %R", + gs->prop_name); + } else { + PyErr_SetString(PyExc_AttributeError, + value == NULL ? + "can't delete attribute" : + "can't set attribute"); + } return -1; } if (value == NULL) @@ -1634,6 +1672,9 @@ property_copy(PyObject *old, PyObject *get, PyObject *set, PyObject *del) Py_DECREF(type); if (new == NULL) return NULL; + + Py_XINCREF(pold->prop_name); + Py_XSETREF(((propertyobject *) new)->prop_name, pold->prop_name); return new; } @@ -1695,6 +1736,8 @@ property_init_impl(propertyobject *self, PyObject *fget, PyObject *fset, Py_XSETREF(self->prop_set, fset); Py_XSETREF(self->prop_del, fdel); Py_XSETREF(self->prop_doc, doc); + Py_XSETREF(self->prop_name, NULL); + self->getter_doc = 0; /* if no docstring given and the getter has one, use that one */ @@ -1769,6 +1812,7 @@ property_traverse(PyObject *self, visitproc visit, void *arg) Py_VISIT(pp->prop_set); Py_VISIT(pp->prop_del); Py_VISIT(pp->prop_doc); + Py_VISIT(pp->prop_name); return 0; }

1 0

bpo-40137: Fix refleak in _functools_exec() (GH-24006)
by vstinner Dec. 30, 2020

Dec. 30, 2020

https://github.com/python/cpython/commit/ba0e49a4648e727d1a16b3ce479499eb39… commit: ba0e49a4648e727d1a16b3ce479499eb39f22311 branch: master author: Victor Stinner <vstinner(a)python.org> committer: vstinner <vstinner(a)python.org> date: 2020-12-30T02:24:43+01:00 summary: bpo-40137: Fix refleak in _functools_exec() (GH-24006) files: M Modules/_functoolsmodule.c diff --git a/Modules/_functoolsmodule.c b/Modules/_functoolsmodule.c index b121ec7d14113..1fcaf299e67bc 100644 --- a/Modules/_functoolsmodule.c +++ b/Modules/_functoolsmodule.c @@ -1436,6 +1436,7 @@ _functools_exec(PyObject *module) Py_DECREF(lru_cache_type); return -1; } + Py_DECREF(lru_cache_type); state->keyobject_type = (PyTypeObject *)PyType_FromModuleAndSpec(module, &keyobject_type_spec, NULL);

1 0

bpo-40052: Fix alignment issue in PyVectorcall_Function() (GH-23999)
by miss-islington Dec. 29, 2020

Dec. 29, 2020

https://github.com/python/cpython/commit/056c08211b402b4dbc1530a9de9d00ad53… commit: 056c08211b402b4dbc1530a9de9d00ad5309909f branch: master author: Petr Viktorin <encukou(a)gmail.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2020-12-29T15:32:07-08:00 summary: bpo-40052: Fix alignment issue in PyVectorcall_Function() (GH-23999) ``` In file included from /usr/include/python3.8/Python.h:147: In file included from /usr/include/python3.8/abstract.h:837: /usr/include/python3.8/cpython/abstract.h:91:11: error: cast from 'char *' to 'vectorcallfunc *' (aka 'struct _object *(**)(struct _object *, struct _object *const *, unsigned long, struct _object *)') increases required alignment from 1 to 8 [-Werror,-Wcast-align] ptr = (vectorcallfunc*)(((char *)callable) + offset); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated. ``` Co-Authored-By: Andreas Schneider <asn(a)cryptomilk.org> Co-Authored-By: Antoine Pitrou <antoine(a)python.org> files: A Misc/NEWS.d/next/C API/2020-03-24-09-27-10.bpo-40052.27P2KG.rst M Include/cpython/abstract.h M Objects/call.c diff --git a/Include/cpython/abstract.h b/Include/cpython/abstract.h index b5b6e4819788c..1083942c14929 100644 --- a/Include/cpython/abstract.h +++ b/Include/cpython/abstract.h @@ -63,7 +63,7 @@ PyVectorcall_Function(PyObject *callable) { PyTypeObject *tp; Py_ssize_t offset; - vectorcallfunc *ptr; + vectorcallfunc ptr; assert(callable != NULL); tp = Py_TYPE(callable); @@ -73,8 +73,8 @@ PyVectorcall_Function(PyObject *callable) assert(PyCallable_Check(callable)); offset = tp->tp_vectorcall_offset; assert(offset > 0); - ptr = (vectorcallfunc *)(((char *)callable) + offset); - return *ptr; + memcpy(&ptr, (char *) callable + offset, sizeof(ptr)); + return ptr; } /* Call the callable object 'callable' with the "vectorcall" calling diff --git a/Misc/NEWS.d/next/C API/2020-03-24-09-27-10.bpo-40052.27P2KG.rst b/Misc/NEWS.d/next/C API/2020-03-24-09-27-10.bpo-40052.27P2KG.rst new file mode 100644 index 0000000000000..538488e2fbacc --- /dev/null +++ b/Misc/NEWS.d/next/C API/2020-03-24-09-27-10.bpo-40052.27P2KG.rst @@ -0,0 +1,2 @@ +Fix an alignment build warning/error in function ``PyVectorcall_Function()``. +Patch by Andreas Schneider, Antoine Pitrou and Petr Viktorin. diff --git a/Objects/call.c b/Objects/call.c index 30fa14ccfd765..35b06a9b9c4d7 100644 --- a/Objects/call.c +++ b/Objects/call.c @@ -205,6 +205,7 @@ PyObject * PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *kwargs) { PyThreadState *tstate = _PyThreadState_GET(); + vectorcallfunc func; /* get vectorcallfunc as in PyVectorcall_Function, but without * the Py_TPFLAGS_HAVE_VECTORCALL check */ @@ -215,7 +216,7 @@ PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *kwargs) Py_TYPE(callable)->tp_name); return NULL; } - vectorcallfunc func = *(vectorcallfunc *)(((char *)callable) + offset); + memcpy(&func, (char *) callable + offset, sizeof(func)); if (func == NULL) { _PyErr_Format(tstate, PyExc_TypeError, "'%.200s' object does not support vectorcall",

1 0

bpo-41224: Add versionadded for Symbol.is_annotated (GH-23861)
by serhiy-storchaka Dec. 29, 2020

Dec. 29, 2020

https://github.com/python/cpython/commit/2edfc86f69d8a74f4821974678f664ff94… commit: 2edfc86f69d8a74f4821974678f664ff94a9dc22 branch: master author: Andre Delfino <adelfino(a)gmail.com> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2020-12-29T15:32:10+02:00 summary: bpo-41224: Add versionadded for Symbol.is_annotated (GH-23861) files: M Doc/library/symtable.rst diff --git a/Doc/library/symtable.rst b/Doc/library/symtable.rst index 40c1795b083a2..e364232247c20 100644 --- a/Doc/library/symtable.rst +++ b/Doc/library/symtable.rst @@ -160,6 +160,8 @@ Examining Symbol Tables Return ``True`` if the symbol is annotated. + .. versionadded:: 3.6 + .. method:: is_free() Return ``True`` if the symbol is referenced in its block, but not assigned

1 0

bpo-40956: Convert _sqlite3.Row to Argument Clinic (GH-23964)
by serhiy-storchaka Dec. 29, 2020

Dec. 29, 2020

https://github.com/python/cpython/commit/84d79cfda947f6bc28a5aa11db8055aa40… commit: 84d79cfda947f6bc28a5aa11db8055aa40a6b03a branch: master author: Erlend Egeberg Aasland <erlend.aasland(a)innova.no> committer: serhiy-storchaka <storchaka(a)gmail.com> date: 2020-12-29T15:22:55+02:00 summary: bpo-40956: Convert _sqlite3.Row to Argument Clinic (GH-23964) files: A Modules/_sqlite/clinic/row.c.h M Modules/_sqlite/row.c diff --git a/Modules/_sqlite/clinic/row.c.h b/Modules/_sqlite/clinic/row.c.h new file mode 100644 index 0000000000000..7ff110940d0b6 --- /dev/null +++ b/Modules/_sqlite/clinic/row.c.h @@ -0,0 +1,56 @@ +/*[clinic input] +preserve +[clinic start generated code]*/ + +static PyObject * +pysqlite_row_new_impl(PyTypeObject *type, pysqlite_Cursor *cursor, + PyObject *data); + +static PyObject * +pysqlite_row_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + PyObject *return_value = NULL; + pysqlite_Cursor *cursor; + PyObject *data; + + if ((type == pysqlite_RowType) && + !_PyArg_NoKeywords("Row", kwargs)) { + goto exit; + } + if (!_PyArg_CheckPositional("Row", PyTuple_GET_SIZE(args), 2, 2)) { + goto exit; + } + if (!PyObject_TypeCheck(PyTuple_GET_ITEM(args, 0), pysqlite_CursorType)) { + _PyArg_BadArgument("Row", "argument 1", (pysqlite_CursorType)->tp_name, PyTuple_GET_ITEM(args, 0)); + goto exit; + } + cursor = (pysqlite_Cursor *)PyTuple_GET_ITEM(args, 0); + if (!PyTuple_Check(PyTuple_GET_ITEM(args, 1))) { + _PyArg_BadArgument("Row", "argument 2", "tuple", PyTuple_GET_ITEM(args, 1)); + goto exit; + } + data = PyTuple_GET_ITEM(args, 1); + return_value = pysqlite_row_new_impl(type, cursor, data); + +exit: + return return_value; +} + +PyDoc_STRVAR(pysqlite_row_keys__doc__, +"keys($self, /)\n" +"--\n" +"\n" +"Returns the keys of the row."); + +#define PYSQLITE_ROW_KEYS_METHODDEF \ + {"keys", (PyCFunction)pysqlite_row_keys, METH_NOARGS, pysqlite_row_keys__doc__}, + +static PyObject * +pysqlite_row_keys_impl(pysqlite_Row *self); + +static PyObject * +pysqlite_row_keys(pysqlite_Row *self, PyObject *Py_UNUSED(ignored)) +{ + return pysqlite_row_keys_impl(self); +} +/*[clinic end generated code: output=8d29220b9cde035d input=a9049054013a1b77]*/ diff --git a/Modules/_sqlite/row.c b/Modules/_sqlite/row.c index 9086c19af73cc..04e308fa1c350 100644 --- a/Modules/_sqlite/row.c +++ b/Modules/_sqlite/row.c @@ -23,6 +23,13 @@ #include "row.h" #include "cursor.h" +#include "clinic/row.c.h" + +/*[clinic input] +module _sqlite3 +class _sqlite3.Row "pysqlite_Row *" "pysqlite_RowType" +[clinic start generated code]*/ +/*[clinic end generated code: output=da39a3ee5e6b4b0d input=384227da65f250fd]*/ void pysqlite_row_dealloc(pysqlite_Row* self) { @@ -35,30 +42,25 @@ void pysqlite_row_dealloc(pysqlite_Row* self) Py_DECREF(tp); } +/*[clinic input] +@classmethod +_sqlite3.Row.__new__ as pysqlite_row_new + + cursor: object(type='pysqlite_Cursor *', subclass_of='pysqlite_CursorType') + data: object(subclass_of='&PyTuple_Type') + / + +[clinic start generated code]*/ + static PyObject * -pysqlite_row_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) +pysqlite_row_new_impl(PyTypeObject *type, pysqlite_Cursor *cursor, + PyObject *data) +/*[clinic end generated code: output=10d58b09a819a4c1 input=f6cd7e6e0935828d]*/ { pysqlite_Row *self; - PyObject* data; - pysqlite_Cursor* cursor; assert(type != NULL && type->tp_alloc != NULL); - if (!_PyArg_NoKeywords("Row", kwargs)) - return NULL; - if (!PyArg_ParseTuple(args, "OO", &cursor, &data)) - return NULL; - - if (!PyObject_TypeCheck((PyObject*)cursor, pysqlite_CursorType)) { - PyErr_SetString(PyExc_TypeError, "instance of cursor required for first argument"); - return NULL; - } - - if (!PyTuple_Check(data)) { - PyErr_SetString(PyExc_TypeError, "tuple required for second argument"); - return NULL; - } - self = (pysqlite_Row *) type->tp_alloc(type, 0); if (self == NULL) return NULL; @@ -153,7 +155,15 @@ pysqlite_row_length(pysqlite_Row* self) return PyTuple_GET_SIZE(self->data); } -PyObject* pysqlite_row_keys(pysqlite_Row* self, PyObject *Py_UNUSED(ignored)) +/*[clinic input] +_sqlite3.Row.keys as pysqlite_row_keys + +Returns the keys of the row. +[clinic start generated code]*/ + +static PyObject * +pysqlite_row_keys_impl(pysqlite_Row *self) +/*[clinic end generated code: output=efe3dfb3af6edc07 input=7549a122827c5563]*/ { PyObject* list; Py_ssize_t nitems, i; @@ -204,8 +214,7 @@ static PyObject* pysqlite_row_richcompare(pysqlite_Row *self, PyObject *_other, } static PyMethodDef row_methods[] = { - {"keys", (PyCFunction)pysqlite_row_keys, METH_NOARGS, - PyDoc_STR("Returns the keys of the row.")}, + PYSQLITE_ROW_KEYS_METHODDEF {NULL, NULL} };

1 0

bpo-42655: Fix subprocess extra_groups gid conversion (GH-23762)
by miss-islington Dec. 29, 2020

Dec. 29, 2020

https://github.com/python/cpython/commit/3966e2ea412a5f190dc8655d9aa7a15c08… commit: 3966e2ea412a5f190dc8655d9aa7a15c08c4e280 branch: 3.9 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2020-12-29T05:22:13-08:00 summary: bpo-42655: Fix subprocess extra_groups gid conversion (GH-23762) (cherry picked from commit 0159e5efeebd12b3cf365c8569ca000eac7cb03e) Co-authored-by: Jakub Kulík <Kulikjak(a)gmail.com> files: A Misc/NEWS.d/next/Library/2020-12-25-12-32-47.bpo-42655.W5ytpV.rst M Modules/_posixsubprocess.c M Modules/posixmodule.c M Modules/posixmodule.h diff --git a/Misc/NEWS.d/next/Library/2020-12-25-12-32-47.bpo-42655.W5ytpV.rst b/Misc/NEWS.d/next/Library/2020-12-25-12-32-47.bpo-42655.W5ytpV.rst new file mode 100644 index 0000000000000..57c9a666e395a --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-12-25-12-32-47.bpo-42655.W5ytpV.rst @@ -0,0 +1,2 @@ +:mod:`subprocess` *extra_groups* is now correctly passed into setgroups() +system call. diff --git a/Modules/_posixsubprocess.c b/Modules/_posixsubprocess.c index c266fd1ec1756..d64e0a1cfa06d 100644 --- a/Modules/_posixsubprocess.c +++ b/Modules/_posixsubprocess.c @@ -753,7 +753,7 @@ subprocess_fork_exec(PyObject* self, PyObject *args) if (groups_list != Py_None) { #ifdef HAVE_SETGROUPS Py_ssize_t i; - unsigned long gid; + gid_t gid; if (!PyList_Check(groups_list)) { PyErr_SetString(PyExc_TypeError, @@ -787,10 +787,6 @@ subprocess_fork_exec(PyObject* self, PyObject *args) Py_DECREF(elem); goto cleanup; } else { - /* In posixmodule.c UnsignedLong is used as a fallback value - * if the value provided does not fit in a Long. Since we are - * already doing the bounds checking on the Python side, we - * can go directly to an UnsignedLong here. */ if (!_Py_Gid_Converter(elem, &gid)) { Py_DECREF(elem); PyErr_SetString(PyExc_ValueError, "invalid group id"); diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 12f72f525f7ae..d2ce8339e61f0 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -632,7 +632,7 @@ _PyLong_FromGid(gid_t gid) } int -_Py_Uid_Converter(PyObject *obj, void *p) +_Py_Uid_Converter(PyObject *obj, uid_t *p) { uid_t uid; PyObject *index; @@ -719,7 +719,7 @@ _Py_Uid_Converter(PyObject *obj, void *p) success: Py_DECREF(index); - *(uid_t *)p = uid; + *p = uid; return 1; underflow: @@ -738,7 +738,7 @@ _Py_Uid_Converter(PyObject *obj, void *p) } int -_Py_Gid_Converter(PyObject *obj, void *p) +_Py_Gid_Converter(PyObject *obj, gid_t *p) { gid_t gid; PyObject *index; @@ -826,7 +826,7 @@ _Py_Gid_Converter(PyObject *obj, void *p) success: Py_DECREF(index); - *(gid_t *)p = gid; + *p = gid; return 1; underflow: diff --git a/Modules/posixmodule.h b/Modules/posixmodule.h index 1e00562abc337..711ac686934b0 100644 --- a/Modules/posixmodule.h +++ b/Modules/posixmodule.h @@ -14,8 +14,8 @@ extern "C" { #ifndef MS_WINDOWS PyAPI_FUNC(PyObject *) _PyLong_FromUid(uid_t); PyAPI_FUNC(PyObject *) _PyLong_FromGid(gid_t); -PyAPI_FUNC(int) _Py_Uid_Converter(PyObject *, void *); -PyAPI_FUNC(int) _Py_Gid_Converter(PyObject *, void *); +PyAPI_FUNC(int) _Py_Uid_Converter(PyObject *, uid_t *); +PyAPI_FUNC(int) _Py_Gid_Converter(PyObject *, gid_t *); #endif /* MS_WINDOWS */ #if defined(PYPTHREAD_SIGMASK) || defined(HAVE_SIGWAIT) || \

1 0

Allow / character in username, password fields in _PROXY envvars. (GH-23973) (#23992)
by orsenthil Dec. 29, 2020

Dec. 29, 2020

https://github.com/python/cpython/commit/741f22df24ca61db38b5a7a2a58b5939b7… commit: 741f22df24ca61db38b5a7a2a58b5939b7154a01 branch: 3.8 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: orsenthil <skumaran(a)gatech.edu> date: 2020-12-29T05:15:14-08:00 summary: Allow / character in username,password fields in _PROXY envvars. (GH-23973) (#23992) (cherry picked from commit 030a713183084594659aefd77b76fe30178e23c8) Co-authored-by: Senthil Kumaran <senthil(a)uthcode.com> files: A Misc/NEWS.d/next/Library/2020-12-27-18-47-01.bpo-23328._xqepZ.rst M Lib/test/test_urllib2.py M Lib/urllib/request.py diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index 0132059fdb0ad..160315394094f 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1846,9 +1846,17 @@ def test_parse_proxy(self): ('ftp', 'joe', 'password', 'proxy.example.com')), # Test for no trailing '/' case ('http://joe:password@proxy.example.com', - ('http', 'joe', 'password', 'proxy.example.com')) + ('http', 'joe', 'password', 'proxy.example.com')), + # Testcases with '/' character in username, password + ('http://user/name:password@localhost:22', + ('http', 'user/name', 'password', 'localhost:22')), + ('http://username:pass/word@localhost:22', + ('http', 'username', 'pass/word', 'localhost:22')), + ('http://user/name:pass/word@localhost:22', + ('http', 'user/name', 'pass/word', 'localhost:22')), ] + for tc, expected in parse_proxy_test_cases: self.assertEqual(_parse_proxy(tc), expected) diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py index afd634159fa82..550a2732697d9 100644 --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -779,7 +779,11 @@ def _parse_proxy(proxy): raise ValueError("proxy URL with no authority: %r" % proxy) # We have an authority, so for RFC 3986-compliant URLs (by ss 3. # and 3.3.), path is empty or starts with '/' - end = r_scheme.find("/", 2) + if '@' in r_scheme: + host_separator = r_scheme.find('@') + end = r_scheme.find("/", host_separator) + else: + end = r_scheme.find("/", 2) if end == -1: end = None authority = r_scheme[2:end] diff --git a/Misc/NEWS.d/next/Library/2020-12-27-18-47-01.bpo-23328._xqepZ.rst b/Misc/NEWS.d/next/Library/2020-12-27-18-47-01.bpo-23328._xqepZ.rst new file mode 100644 index 0000000000000..07b15d34e8d56 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-12-27-18-47-01.bpo-23328._xqepZ.rst @@ -0,0 +1 @@ +Allow / character in username, password fields on _PROXY envars.

1 0