Mailman 3 December 2021 - Python-checkins

Fix a typo in the message from make_ssl_certs. (GH-30152)
by miss-islington Dec. 17, 2021

Dec. 17, 2021

https://github.com/python/cpython/commit/69ef1b59983065ddb0b712dac3b04107c5… commit: 69ef1b59983065ddb0b712dac3b04107c5059735 branch: main author: Yilei "Dolee" Yang <yileiyang9(a)gmail.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2021-12-16T21:08:18-08:00 summary: Fix a typo in the message from make_ssl_certs. (GH-30152) The file is utils.py not util.py. Automerge-Triggered-By: GH:gpshead files: M Lib/test/make_ssl_certs.py diff --git a/Lib/test/make_ssl_certs.py b/Lib/test/make_ssl_certs.py index c62896f861dab..94a35a64ab1ab 100644 --- a/Lib/test/make_ssl_certs.py +++ b/Lib/test/make_ssl_certs.py @@ -307,6 +307,6 @@ def print_cert(path): f.write(cert) unmake_ca() - print("update Lib/test/test_ssl.py and Lib/test/test_asyncio/util.py") + print("update Lib/test/test_ssl.py and Lib/test/test_asyncio/utils.py") print_cert('keycert.pem') print_cert('keycert3.pem')

1 0

[3.9] bpo-46105: Honor spec when generating requirement specs with urls and extras. (GH-30151). (GH-30157)
by jaraco Dec. 17, 2021

Dec. 17, 2021

https://github.com/python/cpython/commit/864ec170e14b663f999eb415a4f1a0067e… commit: 864ec170e14b663f999eb415a4f1a0067ec6833a branch: 3.9 author: Jason R. Coombs <jaraco(a)jaraco.com> committer: jaraco <jaraco(a)jaraco.com> date: 2021-12-16T19:58:19-05:00 summary: [3.9] bpo-46105: Honor spec when generating requirement specs with urls and extras. (GH-30151). (GH-30157) (cherry picked from commit 109d96602199a91e94eb14b8cb3720841f22ded7) Co-authored-by: Jason R. Coombs <jaraco(a)jaraco.com> files: A Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst M Lib/importlib/metadata.py M Lib/test/test_importlib/test_metadata_api.py diff --git a/Lib/importlib/metadata.py b/Lib/importlib/metadata.py index c6c7d31aa5d8a..594986ce23efa 100644 --- a/Lib/importlib/metadata.py +++ b/Lib/importlib/metadata.py @@ -344,17 +344,26 @@ def _convert_egg_info_reqs_to_simple_reqs(sections): def make_condition(name): return name and 'extra == "{name}"'.format(name=name) - def parse_condition(section): + def quoted_marker(section): section = section or '' extra, sep, markers = section.partition(':') if extra and markers: - markers = '({markers})'.format(markers=markers) + markers = f'({markers})' conditions = list(filter(None, [markers, make_condition(extra)])) return '; ' + ' and '.join(conditions) if conditions else '' + def url_req_space(req): + """ + PEP 508 requires a space between the url_spec and the quoted_marker. + Ref python/importlib_metadata#357. + """ + # '@' is uniquely indicative of a url_req. + return ' ' * ('@' in req) + for section, deps in sections.items(): for dep in deps: - yield dep + parse_condition(section) + space = url_req_space(dep) + yield dep + space + quoted_marker(section) class DistributionFinder(MetaPathFinder): diff --git a/Lib/test/test_importlib/test_metadata_api.py b/Lib/test/test_importlib/test_metadata_api.py index 1d7b29ae05fd1..420fbfeb3a704 100644 --- a/Lib/test/test_importlib/test_metadata_api.py +++ b/Lib/test/test_importlib/test_metadata_api.py @@ -121,6 +121,7 @@ def test_more_complex_deps_requires_text(self): [extra1] dep4 + dep6@ git+https://example.com/python/dep.git@v1.0.0 [extra2:python_version < "3"] dep5 @@ -132,6 +133,7 @@ def test_more_complex_deps_requires_text(self): 'dep3; python_version < "3"', 'dep4; extra == "extra1"', 'dep5; (python_version < "3") and extra == "extra2"', + 'dep6@ git+https://example.com/python/dep.git@v1.0.0 ; extra == "extra1"', ] # It's important that the environment marker expression be # wrapped in parentheses to avoid the following 'and' binding more diff --git a/Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst b/Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst new file mode 100644 index 0000000000000..145edccb2e7a5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst @@ -0,0 +1,2 @@ +Honor spec when generating requirement specs with urls and extras +(importlib_metadata 4.8.3).

1 0

bpo-45635: Do not suppress errors in functions called from _PyErr_Display (GH-30073)
by iritkatriel Dec. 17, 2021

Dec. 17, 2021

https://github.com/python/cpython/commit/8d6155ee9d1b05946f951d0ba602b9f638… commit: 8d6155ee9d1b05946f951d0ba602b9f63810fe0f branch: main author: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2021-12-16T23:00:13Z summary: bpo-45635: Do not suppress errors in functions called from _PyErr_Display (GH-30073) Co-authored-by: Erlend Egeberg Aasland <erlend.aasland(a)innova.no> files: A Misc/NEWS.d/next/Core and Builtins/2021-12-12-15-52-41.bpo-45635.ADVaPT.rst M Python/pythonrun.c diff --git a/Misc/NEWS.d/next/Core and Builtins/2021-12-12-15-52-41.bpo-45635.ADVaPT.rst b/Misc/NEWS.d/next/Core and Builtins/2021-12-12-15-52-41.bpo-45635.ADVaPT.rst new file mode 100644 index 0000000000000..d2c97f564b2d2 --- /dev/null +++ b/Misc/NEWS.d/next/Core and Builtins/2021-12-12-15-52-41.bpo-45635.ADVaPT.rst @@ -0,0 +1 @@ +The code called from :c:func:`_PyErr_Display` was refactored to improve error handling. It now exits immediately upon an unrecoverable error. \ No newline at end of file diff --git a/Python/pythonrun.c b/Python/pythonrun.c index 4fff088fc3e10..6f170a4dd63eb 100644 --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -629,14 +629,18 @@ parse_syntax_error(PyObject *err, PyObject **message, PyObject **filename, return 0; } -static void -print_error_text(PyObject *f, Py_ssize_t offset, Py_ssize_t end_offset, PyObject *text_obj) +static int +print_error_text(PyObject *f, Py_ssize_t offset, Py_ssize_t end_offset, + PyObject *text_obj) { - size_t caret_repetitions = (end_offset > 0 && end_offset > offset) ? end_offset - offset : 1; + size_t caret_repetitions = (end_offset > 0 && end_offset > offset) ? + end_offset - offset : 1; + /* Convert text to a char pointer; return if error */ const char *text = PyUnicode_AsUTF8(text_obj); - if (text == NULL) - return; + if (text == NULL) { + return -1; + } /* Convert offset from 1-based to 0-based */ offset--; @@ -675,27 +679,43 @@ print_error_text(PyObject *f, Py_ssize_t offset, Py_ssize_t end_offset, PyObject } /* Print text */ - PyFile_WriteString(" ", f); - PyFile_WriteString(text, f); + if (PyFile_WriteString(" ", f) < 0) { + return -1; + } + if (PyFile_WriteString(text, f) < 0) { + return -1; + } /* Make sure there's a newline at the end */ if (text[len] != '\n') { - PyFile_WriteString("\n", f); + if (PyFile_WriteString("\n", f) < 0) { + return -1; + } } /* Don't print caret if it points to the left of the text */ - if (offset < 0) - return; + if (offset < 0) { + return 0; + } /* Write caret line */ - PyFile_WriteString(" ", f); + if (PyFile_WriteString(" ", f) < 0) { + return -1; + } while (--offset >= 0) { - PyFile_WriteString(" ", f); + if (PyFile_WriteString(" ", f) < 0) { + return -1; + } } for (size_t caret_iter=0; caret_iter < caret_repetitions ; caret_iter++) { - PyFile_WriteString("^", f); + if (PyFile_WriteString("^", f) < 0) { + return -1; + } + } + if (PyFile_WriteString("\n", f) < 0) { + return -1; } - PyFile_WriteString("\n", f); + return 0; } @@ -953,6 +973,77 @@ print_exception_traceback(struct exception_print_context *ctx, PyObject *value) return err; } +static int +print_exception_file_and_line(struct exception_print_context *ctx, + PyObject **value_p) +{ + PyObject *f = ctx->file; + + _Py_IDENTIFIER(print_file_and_line); + PyObject *tmp; + int res = _PyObject_LookupAttrId(*value_p, &PyId_print_file_and_line, &tmp); + if (res <= 0) { + if (res < 0) { + PyErr_Clear(); + } + return 0; + } + Py_DECREF(tmp); + + PyObject *message, *filename, *text; + Py_ssize_t lineno, offset, end_lineno, end_offset; + if (!parse_syntax_error(*value_p, &message, &filename, + &lineno, &offset, + &end_lineno, &end_offset, &text)) { + PyErr_Clear(); + return 0; + } + + Py_SETREF(*value_p, message); + + PyObject *line = PyUnicode_FromFormat(" File \"%S\", line %zd\n", + filename, lineno); + Py_DECREF(filename); + if (line == NULL) { + goto error; + } + if (write_indented_margin(ctx, f) < 0) { + goto error; + } + if (PyFile_WriteObject(line, f, Py_PRINT_RAW) < 0) { + goto error; + } + Py_CLEAR(line); + + if (text != NULL) { + Py_ssize_t line_size; + const char *error_line = PyUnicode_AsUTF8AndSize(text, &line_size); + // If the location of the error spawn multiple lines, we want + // to just print the first one and highlight everything until + // the end of that one since we don't support multi-line error + // messages. + if (end_lineno > lineno) { + end_offset = (error_line != NULL) ? line_size : -1; + } + // Limit the amount of '^' that we can display to + // the size of the text in the source line. + if (error_line != NULL && end_offset > line_size + 1) { + end_offset = line_size + 1; + } + if (print_error_text(f, offset, end_offset, text) < 0) { + goto error; + } + Py_DECREF(text); + } + assert(!PyErr_Occurred()); + return 0; + +error: + Py_XDECREF(line); + Py_XDECREF(text); + return -1; +} + /* Prints the message line: module.qualname[: str(exc)] */ static int print_exception_message(struct exception_print_context *ctx, PyObject *type, @@ -1072,6 +1163,11 @@ static int print_exception_note(struct exception_print_context *ctx, PyObject *value) { PyObject *f = ctx->file; + + if (!PyExceptionInstance_Check(value)) { + return 0; + } + _Py_IDENTIFIER(__note__); PyObject *note = _PyObject_GetAttrId(value, &PyId___note__); @@ -1112,98 +1208,47 @@ print_exception_note(struct exception_print_context *ctx, PyObject *value) return -1; } -static void +static int print_exception(struct exception_print_context *ctx, PyObject *value) { - int err = 0; - PyObject *tmp; PyObject *f = ctx->file; - _Py_IDENTIFIER(print_file_and_line); - if (!PyExceptionInstance_Check(value)) { - if (print_exception_invalid_type(ctx, value) < 0) { - PyErr_Clear(); /* TODO: change to return -1 */ - } - return; + return print_exception_invalid_type(ctx, value); } Py_INCREF(value); fflush(stdout); - PyObject *type = (PyObject *) Py_TYPE(value); - err = print_exception_traceback(ctx, value); - if (err == 0 && - (err = _PyObject_LookupAttrId(value, &PyId_print_file_and_line, &tmp)) > 0) - { - PyObject *message, *filename, *text; - Py_ssize_t lineno, offset, end_lineno, end_offset; - err = 0; - Py_DECREF(tmp); - if (!parse_syntax_error(value, &message, &filename, - &lineno, &offset, - &end_lineno, &end_offset, &text)) { - PyErr_Clear(); - } - else { - Py_DECREF(value); - value = message; - - PyObject *line = PyUnicode_FromFormat(" File \"%S\", line %zd\n", - filename, lineno); - Py_DECREF(filename); - if (line != NULL) { - err = write_indented_margin(ctx, f); - if (err == 0) { - err = PyFile_WriteObject(line, f, Py_PRINT_RAW); - } - Py_DECREF(line); - } + if (print_exception_traceback(ctx, value) < 0) { + goto error; + } - if (text != NULL) { - Py_ssize_t line_size; - const char* error_line = PyUnicode_AsUTF8AndSize(text, &line_size); - // If the location of the error spawn multiple lines, we want - // to just print the first one and highlight everything until - // the end of that one since we don't support multi-line error - // messages. - if (end_lineno > lineno) { - end_offset = (error_line != NULL) ? line_size : -1; - } - // Limit the amount of '^' that we can display to - // the size of the text in the source line. - if (error_line != NULL && end_offset > line_size + 1) { - end_offset = line_size + 1; - } - print_error_text(f, offset, end_offset, text); - Py_DECREF(text); - } + /* grab the type now because value can change below */ + PyObject *type = (PyObject *) Py_TYPE(value); - /* Can't be bothered to check all those - PyFile_WriteString() calls */ - if (PyErr_Occurred()) - err = -1; - } + if (print_exception_file_and_line(ctx, &value) < 0) { + goto error; } - - if (err == 0) { - err = print_exception_message(ctx, type, value); + if (print_exception_message(ctx, type, value) < 0) { + goto error; } - if (err == 0) { - err = print_exception_suggestions(ctx, value); + if (print_exception_suggestions(ctx, value) < 0) { + goto error; } - if (err == 0) { - err = PyFile_WriteString("\n", f); + if (PyFile_WriteString("\n", f) < 0) { + goto error; } - if (err == 0 && PyExceptionInstance_Check(value)) { - err = print_exception_note(ctx, value); + if (print_exception_note(ctx, value) < 0) { + goto error; } Py_DECREF(value); - /* If an error happened here, don't show it. - XXX This is wrong, but too many callers rely on this behavior. */ - if (err != 0) - PyErr_Clear(); + assert(!PyErr_Occurred()); + return 0; +error: + Py_DECREF(value); + return -1; } static const char cause_message[] = @@ -1214,7 +1259,7 @@ static const char context_message[] = "During handling of the above exception, " "another exception occurred:\n"; -static void +static int print_exception_recursive(struct exception_print_context*, PyObject*); static int @@ -1227,10 +1272,12 @@ print_chained(struct exception_print_context* ctx, PyObject *value, return -1; } bool need_close = ctx->need_close; - print_exception_recursive(ctx, value); + int res = print_exception_recursive(ctx, value); ctx->need_close = need_close; - Py_LeaveRecursiveCall(); + if (res < 0) { + return -1; + } if (write_indented_margin(ctx, f) < 0) { return -1; @@ -1398,11 +1445,14 @@ print_exception_group(struct exception_print_context *ctx, PyObject *value) PyObject *exc = PyTuple_GET_ITEM(excs, i); if (!truncated) { - if (Py_EnterRecursiveCall(" in print_exception_recursive") != 0) { + if (Py_EnterRecursiveCall(" in print_exception_group") != 0) { return -1; } - print_exception_recursive(ctx, exc); + int res = print_exception_recursive(ctx, exc); Py_LeaveRecursiveCall(); + if (res < 0) { + return -1; + } } else { Py_ssize_t excs_remaining = num_excs - ctx->max_group_width; @@ -1445,33 +1495,25 @@ print_exception_group(struct exception_print_context *ctx, PyObject *value) return 0; } -static void +static int print_exception_recursive(struct exception_print_context *ctx, PyObject *value) { - int err = 0; if (ctx->seen != NULL) { /* Exception chaining */ - err = print_exception_cause_and_context(ctx, value); - } - if (err) { - /* don't do anything else */ - } - else if (!_PyBaseExceptionGroup_Check(value)) { - print_exception(ctx, value); - } - else { - int prev_depth = ctx->exception_group_depth; - err = print_exception_group(ctx, value); - if (err < 0) { - /* restore the depth as long as we're ignoring errors */ - ctx->exception_group_depth = prev_depth; + if (print_exception_cause_and_context(ctx, value) < 0) { + return -1; } - else { - assert(prev_depth == ctx->exception_group_depth); + } + if (!_PyBaseExceptionGroup_Check(value)) { + if (print_exception(ctx, value) < 0) { + return -1; } } - if (err != 0) - PyErr_Clear(); + else if (print_exception_group(ctx, value) < 0) { + return -1; + } + assert(!PyErr_Occurred()); + return 0; } #define PyErr_MAX_GROUP_WIDTH 15 @@ -1505,7 +1547,9 @@ _PyErr_Display(PyObject *file, PyObject *exception, PyObject *value, PyObject *t if (ctx.seen == NULL) { PyErr_Clear(); } - print_exception_recursive(&ctx, value); + if (print_exception_recursive(&ctx, value) < 0) { + PyErr_Clear(); + } Py_XDECREF(ctx.seen); /* Call file.flush() */

1 0

bpo-46105: Honor spec when generating requirement specs with urls and extras. (GH-30151)
by miss-islington Dec. 16, 2021

Dec. 16, 2021

https://github.com/python/cpython/commit/09d7319bfe0006d9aa3fc14833b69c24cc… commit: 09d7319bfe0006d9aa3fc14833b69c24ccafdca6 branch: 3.10 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2021-12-16T13:19:14-08:00 summary: bpo-46105: Honor spec when generating requirement specs with urls and extras. (GH-30151) (cherry picked from commit 109d96602199a91e94eb14b8cb3720841f22ded7) Co-authored-by: Jason R. Coombs <jaraco(a)jaraco.com> files: A Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst M Lib/importlib/metadata/__init__.py M Lib/test/test_importlib/test_metadata_api.py diff --git a/Lib/importlib/metadata/__init__.py b/Lib/importlib/metadata/__init__.py index b3e8fb05f1d98..ec41ed39157a9 100644 --- a/Lib/importlib/metadata/__init__.py +++ b/Lib/importlib/metadata/__init__.py @@ -669,7 +669,7 @@ def _convert_egg_info_reqs_to_simple_reqs(sections): def make_condition(name): return name and f'extra == "{name}"' - def parse_condition(section): + def quoted_marker(section): section = section or '' extra, sep, markers = section.partition(':') if extra and markers: @@ -677,8 +677,17 @@ def parse_condition(section): conditions = list(filter(None, [markers, make_condition(extra)])) return '; ' + ' and '.join(conditions) if conditions else '' + def url_req_space(req): + """ + PEP 508 requires a space between the url_spec and the quoted_marker. + Ref python/importlib_metadata#357. + """ + # '@' is uniquely indicative of a url_req. + return ' ' * ('@' in req) + for section in sections: - yield section.value + parse_condition(section.name) + space = url_req_space(section.value) + yield section.value + space + quoted_marker(section.name) class DistributionFinder(MetaPathFinder): diff --git a/Lib/test/test_importlib/test_metadata_api.py b/Lib/test/test_importlib/test_metadata_api.py index dc9c234d15b78..4a45312e31cd5 100644 --- a/Lib/test/test_importlib/test_metadata_api.py +++ b/Lib/test/test_importlib/test_metadata_api.py @@ -235,6 +235,7 @@ def test_more_complex_deps_requires_text(self): [extra1] dep4 + dep6@ git+https://example.com/python/dep.git@v1.0.0 [extra2:python_version < "3"] dep5 @@ -247,6 +248,7 @@ def test_more_complex_deps_requires_text(self): 'dep3; python_version < "3"', 'dep4; extra == "extra1"', 'dep5; (python_version < "3") and extra == "extra2"', + 'dep6@ git+https://example.com/python/dep.git@v1.0.0 ; extra == "extra1"', ] # It's important that the environment marker expression be # wrapped in parentheses to avoid the following 'and' binding more diff --git a/Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst b/Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst new file mode 100644 index 0000000000000..145edccb2e7a5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst @@ -0,0 +1,2 @@ +Honor spec when generating requirement specs with urls and extras +(importlib_metadata 4.8.3).

1 0

bpo-44893: Implement EntryPoint as simple class with attributes. (GH-30150)
by jaraco Dec. 16, 2021

Dec. 16, 2021

https://github.com/python/cpython/commit/04deaee4c8d313717f3ea8f6a4fd70286d… commit: 04deaee4c8d313717f3ea8f6a4fd70286d510d6e branch: main author: Jason R. Coombs <jaraco(a)jaraco.com> committer: jaraco <jaraco(a)jaraco.com> date: 2021-12-16T15:49:42-05:00 summary: bpo-44893: Implement EntryPoint as simple class with attributes. (GH-30150) * bpo-44893: Implement EntryPoint as simple class and deprecate tuple access in favor of attribute access. Syncs with importlib_metadata 4.8.1. * Apply refactorings found in importlib_metadata 4.8.2. files: A Lib/test/test_importlib/data/example2-1.0.0-py3-none-any.whl A Misc/NEWS.d/next/Library/2021-12-16-13-54-55.bpo-44893.I7aLiW.rst M Lib/importlib/metadata/__init__.py M Lib/importlib/metadata/_functools.py M Lib/importlib/metadata/_itertools.py M Lib/importlib/metadata/_meta.py M Lib/importlib/metadata/_text.py M Lib/test/test_importlib/fixtures.py M Lib/test/test_importlib/test_main.py M Lib/test/test_importlib/test_metadata_api.py M Lib/test/test_importlib/test_zip.py diff --git a/Lib/importlib/metadata/__init__.py b/Lib/importlib/metadata/__init__.py index ec41ed39157a9..d44541fcbfbf4 100644 --- a/Lib/importlib/metadata/__init__.py +++ b/Lib/importlib/metadata/__init__.py @@ -15,10 +15,9 @@ import collections from . import _adapters, _meta -from ._meta import PackageMetadata from ._collections import FreezableDefaultDict, Pair -from ._functools import method_cache -from ._itertools import unique_everseen +from ._functools import method_cache, pass_none +from ._itertools import always_iterable, unique_everseen from ._meta import PackageMetadata, SimplePath from contextlib import suppress @@ -121,8 +120,33 @@ def valid(line): return line and not line.startswith('#') -class EntryPoint( - collections.namedtuple('EntryPointBase', 'name value group')): +class DeprecatedTuple: + """ + Provide subscript item access for backward compatibility. + + >>> recwarn = getfixture('recwarn') + >>> ep = EntryPoint(name='name', value='value', group='group') + >>> ep[:] + ('name', 'value', 'group') + >>> ep[0] + 'name' + >>> len(recwarn) + 1 + """ + + _warn = functools.partial( + warnings.warn, + "EntryPoint tuple interface is deprecated. Access members by name.", + DeprecationWarning, + stacklevel=2, + ) + + def __getitem__(self, item): + self._warn() + return self._key()[item] + + +class EntryPoint(DeprecatedTuple): """An entry point as defined by Python packaging conventions. See `the packaging docs on entry points @@ -153,6 +177,9 @@ class EntryPoint( dist: Optional['Distribution'] = None + def __init__(self, name, value, group): + vars(self).update(name=name, value=value, group=group) + def load(self): """Load the entry point from its definition. If only a module is indicated by the value, return that module. Otherwise, @@ -179,7 +206,7 @@ def extras(self): return list(re.finditer(r'\w+', match.group('extras') or '')) def _for(self, dist): - self.dist = dist + vars(self).update(dist=dist) return self def __iter__(self): @@ -193,16 +220,31 @@ def __iter__(self): warnings.warn(msg, DeprecationWarning) return iter((self.name, self)) - def __reduce__(self): - return ( - self.__class__, - (self.name, self.value, self.group), - ) - def matches(self, **params): attrs = (getattr(self, param) for param in params) return all(map(operator.eq, params.values(), attrs)) + def _key(self): + return self.name, self.value, self.group + + def __lt__(self, other): + return self._key() < other._key() + + def __eq__(self, other): + return self._key() == other._key() + + def __setattr__(self, name, value): + raise AttributeError("EntryPoint objects are immutable.") + + def __repr__(self): + return ( + f'EntryPoint(name={self.name!r}, value={self.value!r}, ' + f'group={self.group!r})' + ) + + def __hash__(self): + return hash(self._key()) + class DeprecatedList(list): """ @@ -243,37 +285,26 @@ class DeprecatedList(list): stacklevel=2, ) - def __setitem__(self, *args, **kwargs): - self._warn() - return super().__setitem__(*args, **kwargs) - - def __delitem__(self, *args, **kwargs): - self._warn() - return super().__delitem__(*args, **kwargs) - - def append(self, *args, **kwargs): - self._warn() - return super().append(*args, **kwargs) - - def reverse(self, *args, **kwargs): - self._warn() - return super().reverse(*args, **kwargs) - - def extend(self, *args, **kwargs): - self._warn() - return super().extend(*args, **kwargs) - - def pop(self, *args, **kwargs): - self._warn() - return super().pop(*args, **kwargs) - - def remove(self, *args, **kwargs): - self._warn() - return super().remove(*args, **kwargs) - - def __iadd__(self, *args, **kwargs): - self._warn() - return super().__iadd__(*args, **kwargs) + def _wrap_deprecated_method(method_name: str): # type: ignore + def wrapped(self, *args, **kwargs): + self._warn() + return getattr(super(), method_name)(*args, **kwargs) + + return wrapped + + for method_name in [ + '__setitem__', + '__delitem__', + 'append', + 'reverse', + 'extend', + 'pop', + 'remove', + '__iadd__', + 'insert', + 'sort', + ]: + locals()[method_name] = _wrap_deprecated_method(method_name) def __add__(self, other): if not isinstance(other, tuple): @@ -281,14 +312,6 @@ def __add__(self, other): other = tuple(other) return self.__class__(tuple(self) + other) - def insert(self, *args, **kwargs): - self._warn() - return super().insert(*args, **kwargs) - - def sort(self, *args, **kwargs): - self._warn() - return super().sort(*args, **kwargs) - def __eq__(self, other): if not isinstance(other, tuple): self._warn() @@ -333,7 +356,7 @@ def names(self): """ Return the set of all names of all entry points. """ - return set(ep.name for ep in self) + return {ep.name for ep in self} @property def groups(self): @@ -344,21 +367,17 @@ def groups(self): >>> EntryPoints().groups set() """ - return set(ep.group for ep in self) + return {ep.group for ep in self} @classmethod def _from_text_for(cls, text, dist): return cls(ep._for(dist) for ep in cls._from_text(text)) - @classmethod - def _from_text(cls, text): - return itertools.starmap(EntryPoint, cls._parse_groups(text or '')) - @staticmethod - def _parse_groups(text): + def _from_text(text): return ( - (item.value.name, item.value.value, item.name) - for item in Sectioned.section_pairs(text) + EntryPoint(name=item.value.name, value=item.value.value, group=item.name) + for item in Sectioned.section_pairs(text or '') ) @@ -611,7 +630,6 @@ def files(self): missing. Result may be empty if the metadata exists but is empty. """ - file_lines = self._read_files_distinfo() or self._read_files_egginfo() def make_file(name, hash=None, size_str=None): result = PackagePath(name) @@ -620,7 +638,11 @@ def make_file(name, hash=None, size_str=None): result.dist = self return result - return file_lines and list(starmap(make_file, csv.reader(file_lines))) + @pass_none + def make_files(lines): + return list(starmap(make_file, csv.reader(lines))) + + return make_files(self._read_files_distinfo() or self._read_files_egginfo()) def _read_files_distinfo(self): """ @@ -742,6 +764,9 @@ class FastPath: """ Micro-optimized class for searching a path for children. + + >>> FastPath('').children() + ['...'] """ @functools.lru_cache() # type: ignore @@ -1011,6 +1036,18 @@ def packages_distributions() -> Mapping[str, List[str]]: """ pkg_to_dist = collections.defaultdict(list) for dist in distributions(): - for pkg in (dist.read_text('top_level.txt') or '').split(): + for pkg in _top_level_declared(dist) or _top_level_inferred(dist): pkg_to_dist[pkg].append(dist.metadata['Name']) return dict(pkg_to_dist) + + +def _top_level_declared(dist): + return (dist.read_text('top_level.txt') or '').split() + + +def _top_level_inferred(dist): + return { + f.parts[0] if len(f.parts) > 1 else f.with_suffix('').name + for f in always_iterable(dist.files) + if f.suffix == ".py" + } diff --git a/Lib/importlib/metadata/_functools.py b/Lib/importlib/metadata/_functools.py index 73f50d00bc04c..71f66bd03cb71 100644 --- a/Lib/importlib/metadata/_functools.py +++ b/Lib/importlib/metadata/_functools.py @@ -83,3 +83,22 @@ def wrapper(self, *args, **kwargs): wrapper.cache_clear = lambda: None return wrapper + + +# From jaraco.functools 3.3 +def pass_none(func): + """ + Wrap func so it's not called if its first param is None + + >>> print_text = pass_none(print) + >>> print_text('text') + text + >>> print_text(None) + """ + + @functools.wraps(func) + def wrapper(param, *args, **kwargs): + if param is not None: + return func(param, *args, **kwargs) + + return wrapper diff --git a/Lib/importlib/metadata/_itertools.py b/Lib/importlib/metadata/_itertools.py index dd45f2f096630..d4ca9b9140e3f 100644 --- a/Lib/importlib/metadata/_itertools.py +++ b/Lib/importlib/metadata/_itertools.py @@ -17,3 +17,57 @@ def unique_everseen(iterable, key=None): if k not in seen: seen_add(k) yield element + + +# copied from more_itertools 8.8 +def always_iterable(obj, base_type=(str, bytes)): + """If *obj* is iterable, return an iterator over its items:: + + >>> obj = (1, 2, 3) + >>> list(always_iterable(obj)) + [1, 2, 3] + + If *obj* is not iterable, return a one-item iterable containing *obj*:: + + >>> obj = 1 + >>> list(always_iterable(obj)) + [1] + + If *obj* is ``None``, return an empty iterable: + + >>> obj = None + >>> list(always_iterable(None)) + [] + + By default, binary and text strings are not considered iterable:: + + >>> obj = 'foo' + >>> list(always_iterable(obj)) + ['foo'] + + If *base_type* is set, objects for which ``isinstance(obj, base_type)`` + returns ``True`` won't be considered iterable. + + >>> obj = {'a': 1} + >>> list(always_iterable(obj)) # Iterate over the dict's keys + ['a'] + >>> list(always_iterable(obj, base_type=dict)) # Treat dicts as a unit + [{'a': 1}] + + Set *base_type* to ``None`` to avoid any special handling and treat objects + Python considers iterable as iterable: + + >>> obj = 'foo' + >>> list(always_iterable(obj, base_type=None)) + ['f', 'o', 'o'] + """ + if obj is None: + return iter(()) + + if (base_type is not None) and isinstance(obj, base_type): + return iter((obj,)) + + try: + return iter(obj) + except TypeError: + return iter((obj,)) diff --git a/Lib/importlib/metadata/_meta.py b/Lib/importlib/metadata/_meta.py index 1a6edbf957d5a..d5c0576194ece 100644 --- a/Lib/importlib/metadata/_meta.py +++ b/Lib/importlib/metadata/_meta.py @@ -37,7 +37,7 @@ class SimplePath(Protocol): def joinpath(self) -> 'SimplePath': ... # pragma: no cover - def __div__(self) -> 'SimplePath': + def __truediv__(self) -> 'SimplePath': ... # pragma: no cover def parent(self) -> 'SimplePath': diff --git a/Lib/importlib/metadata/_text.py b/Lib/importlib/metadata/_text.py index 766979d93c169..c88cfbb2349c6 100644 --- a/Lib/importlib/metadata/_text.py +++ b/Lib/importlib/metadata/_text.py @@ -80,7 +80,7 @@ def __hash__(self): return hash(self.lower()) def __contains__(self, other): - return super(FoldedCase, self).lower().__contains__(other.lower()) + return super().lower().__contains__(other.lower()) def in_(self, other): "Does self appear in other?" @@ -89,7 +89,7 @@ def in_(self, other): # cache lower since it's likely to be called frequently. @method_cache def lower(self): - return super(FoldedCase, self).lower() + return super().lower() def index(self, sub): return self.lower().index(sub.lower()) diff --git a/Lib/test/test_importlib/data/example2-1.0.0-py3-none-any.whl b/Lib/test/test_importlib/data/example2-1.0.0-py3-none-any.whl new file mode 100644 index 0000000000000..5ca93657f8196 Binary files /dev/null and b/Lib/test/test_importlib/data/example2-1.0.0-py3-none-any.whl differ diff --git a/Lib/test/test_importlib/fixtures.py b/Lib/test/test_importlib/fixtures.py index 12ed07d33744a..d7ed4e9d56ff5 100644 --- a/Lib/test/test_importlib/fixtures.py +++ b/Lib/test/test_importlib/fixtures.py @@ -8,8 +8,17 @@ import contextlib from test.support.os_helper import FS_NONASCII +from test.support import requires_zlib from typing import Dict, Union +try: + from importlib import resources + + getattr(resources, 'files') + getattr(resources, 'as_file') +except (ImportError, AttributeError): + import importlib_resources as resources # type: ignore + @contextlib.contextmanager def tempdir(): @@ -54,7 +63,7 @@ def setUp(self): class SiteDir(Fixtures): def setUp(self): - super(SiteDir, self).setUp() + super().setUp() self.site_dir = self.fixtures.enter_context(tempdir()) @@ -69,7 +78,7 @@ def add_sys_path(dir): sys.path.remove(str(dir)) def setUp(self): - super(OnSysPath, self).setUp() + super().setUp() self.fixtures.enter_context(self.add_sys_path(self.site_dir)) @@ -106,7 +115,7 @@ def main(): } def setUp(self): - super(DistInfoPkg, self).setUp() + super().setUp() build_files(DistInfoPkg.files, self.site_dir) def make_uppercase(self): @@ -131,7 +140,7 @@ class DistInfoPkgWithDot(OnSysPath, SiteDir): } def setUp(self): - super(DistInfoPkgWithDot, self).setUp() + super().setUp() build_files(DistInfoPkgWithDot.files, self.site_dir) @@ -152,13 +161,13 @@ class DistInfoPkgWithDotLegacy(OnSysPath, SiteDir): } def setUp(self): - super(DistInfoPkgWithDotLegacy, self).setUp() + super().setUp() build_files(DistInfoPkgWithDotLegacy.files, self.site_dir) class DistInfoPkgOffPath(SiteDir): def setUp(self): - super(DistInfoPkgOffPath, self).setUp() + super().setUp() build_files(DistInfoPkg.files, self.site_dir) @@ -198,7 +207,7 @@ def main(): } def setUp(self): - super(EggInfoPkg, self).setUp() + super().setUp() build_files(EggInfoPkg.files, prefix=self.site_dir) @@ -219,7 +228,7 @@ class EggInfoFile(OnSysPath, SiteDir): } def setUp(self): - super(EggInfoFile, self).setUp() + super().setUp() build_files(EggInfoFile.files, prefix=self.site_dir) @@ -285,3 +294,20 @@ def DALS(str): class NullFinder: def find_module(self, name): pass + + +@requires_zlib() +class ZipFixtures: + root = 'test.test_importlib.data' + + def _fixture_on_path(self, filename): + pkg_file = resources.files(self.root).joinpath(filename) + file = self.resources.enter_context(resources.as_file(pkg_file)) + assert file.name.startswith('example'), file.name + sys.path.insert(0, str(file)) + self.resources.callback(sys.path.pop, 0) + + def setUp(self): + # Add self.zip_name to the front of sys.path. + self.resources = contextlib.ExitStack() + self.addCleanup(self.resources.close) diff --git a/Lib/test/test_importlib/test_main.py b/Lib/test/test_importlib/test_main.py index 52cb63712a5cb..2e120f7ac50ac 100644 --- a/Lib/test/test_importlib/test_main.py +++ b/Lib/test/test_importlib/test_main.py @@ -19,6 +19,7 @@ distributions, entry_points, metadata, + packages_distributions, version, ) @@ -203,7 +204,7 @@ class InaccessibleSysPath(fixtures.OnSysPath, ffs.TestCase): site_dir = '/access-denied' def setUp(self): - super(InaccessibleSysPath, self).setUp() + super().setUp() self.setUpPyfakefs() self.fs.create_dir(self.site_dir, perm_bits=000) @@ -217,13 +218,21 @@ def test_discovery(self): class TestEntryPoints(unittest.TestCase): def __init__(self, *args): - super(TestEntryPoints, self).__init__(*args) - self.ep = importlib.metadata.EntryPoint('name', 'value', 'group') + super().__init__(*args) + self.ep = importlib.metadata.EntryPoint( + name='name', value='value', group='group' + ) def test_entry_point_pickleable(self): revived = pickle.loads(pickle.dumps(self.ep)) assert revived == self.ep + def test_positional_args(self): + """ + Capture legacy (namedtuple) construction, discouraged. + """ + EntryPoint('name', 'value', 'group') + def test_immutable(self): """EntryPoints should be immutable""" with self.assertRaises(AttributeError): @@ -254,8 +263,8 @@ def test_sortable(self): # EntryPoint objects are sortable, but result is undefined. sorted( [ - EntryPoint('b', 'val', 'group'), - EntryPoint('a', 'val', 'group'), + EntryPoint(name='b', value='val', group='group'), + EntryPoint(name='a', value='val', group='group'), ] ) @@ -271,3 +280,38 @@ def test_unicode_dir_on_sys_path(self): prefix=self.site_dir, ) list(distributions()) + + +class PackagesDistributionsPrebuiltTest(fixtures.ZipFixtures, unittest.TestCase): + def test_packages_distributions_example(self): + self._fixture_on_path('example-21.12-py3-none-any.whl') + assert packages_distributions()['example'] == ['example'] + + def test_packages_distributions_example2(self): + """ + Test packages_distributions on a wheel built + by trampolim. + """ + self._fixture_on_path('example2-1.0.0-py3-none-any.whl') + assert packages_distributions()['example2'] == ['example2'] + + +class PackagesDistributionsTest( + fixtures.OnSysPath, fixtures.SiteDir, unittest.TestCase +): + def test_packages_distributions_neither_toplevel_nor_files(self): + """ + Test a package built without 'top-level.txt' or a file list. + """ + fixtures.build_files( + { + 'trim_example-1.0.0.dist-info': { + 'METADATA': """ + Name: trim_example + Version: 1.0.0 + """, + } + }, + prefix=self.site_dir, + ) + packages_distributions() diff --git a/Lib/test/test_importlib/test_metadata_api.py b/Lib/test/test_importlib/test_metadata_api.py index 4a45312e31cd5..e16773a7e87ef 100644 --- a/Lib/test/test_importlib/test_metadata_api.py +++ b/Lib/test/test_importlib/test_metadata_api.py @@ -21,7 +21,7 @@ @contextlib.contextmanager def suppress_known_deprecation(): with warnings.catch_warnings(record=True) as ctx: - warnings.simplefilter('default') + warnings.simplefilter('default', category=DeprecationWarning) yield ctx @@ -113,7 +113,7 @@ def test_entry_points_unique_packages(self): for ep in entries ) # ns:sub doesn't exist in alt_pkg - assert 'ns:sub' not in entries + assert 'ns:sub' not in entries.names def test_entry_points_missing_name(self): with self.assertRaises(KeyError): @@ -194,10 +194,8 @@ def _test_files(files): file.read_text() def test_file_hash_repr(self): - assertRegex = self.assertRegex - util = [p for p in files('distinfo-pkg') if p.name == 'mod.py'][0] - assertRegex(repr(util.hash), '<FileHash mode: sha256 value: .*>') + self.assertRegex(repr(util.hash), '<FileHash mode: sha256 value: .*>') def test_files_dist_info(self): self._test_files(files('distinfo-pkg')) diff --git a/Lib/test/test_importlib/test_zip.py b/Lib/test/test_importlib/test_zip.py index bf16a3b95e18c..276f6288c9159 100644 --- a/Lib/test/test_importlib/test_zip.py +++ b/Lib/test/test_importlib/test_zip.py @@ -1,7 +1,7 @@ import sys import unittest -from contextlib import ExitStack +from . import fixtures from importlib.metadata import ( PackageNotFoundError, distribution, @@ -10,27 +10,11 @@ files, version, ) -from importlib import resources -from test.support import requires_zlib - - -@requires_zlib() -class TestZip(unittest.TestCase): - root = 'test.test_importlib.data' - - def _fixture_on_path(self, filename): - pkg_file = resources.files(self.root).joinpath(filename) - file = self.resources.enter_context(resources.as_file(pkg_file)) - assert file.name.startswith('example-'), file.name - sys.path.insert(0, str(file)) - self.resources.callback(sys.path.pop, 0) +class TestZip(fixtures.ZipFixtures, unittest.TestCase): def setUp(self): - # Find the path to the example-*.whl so we can add it to the front of - # sys.path, where we'll then try to find the metadata thereof. - self.resources = ExitStack() - self.addCleanup(self.resources.close) + super().setUp() self._fixture_on_path('example-21.12-py3-none-any.whl') def test_zip_version(self): @@ -63,13 +47,9 @@ def test_one_distribution(self): assert len(dists) == 1 -@requires_zlib() class TestEgg(TestZip): def setUp(self): - # Find the path to the example-*.egg so we can add it to the front of - # sys.path, where we'll then try to find the metadata thereof. - self.resources = ExitStack() - self.addCleanup(self.resources.close) + super().setUp() self._fixture_on_path('example-21.12-py3.6.egg') def test_files(self): diff --git a/Misc/NEWS.d/next/Library/2021-12-16-13-54-55.bpo-44893.I7aLiW.rst b/Misc/NEWS.d/next/Library/2021-12-16-13-54-55.bpo-44893.I7aLiW.rst new file mode 100644 index 0000000000000..e77c6ad2a483e --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-12-16-13-54-55.bpo-44893.I7aLiW.rst @@ -0,0 +1,3 @@ +EntryPoint objects are no longer tuples. Recommended means to access is by +attribute ('.name', '.group') or accessor ('.load()'). Access by index is +deprecated and will raise deprecation warning.

1 0

bpo-46105: Honor spec when generating requirement specs with urls and extras. (GH-30151)
by jaraco Dec. 16, 2021

Dec. 16, 2021

https://github.com/python/cpython/commit/109d96602199a91e94eb14b8cb3720841f… commit: 109d96602199a91e94eb14b8cb3720841f22ded7 branch: main author: Jason R. Coombs <jaraco(a)jaraco.com> committer: jaraco <jaraco(a)jaraco.com> date: 2021-12-16T15:48:35-05:00 summary: bpo-46105: Honor spec when generating requirement specs with urls and extras. (GH-30151) files: A Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst M Lib/importlib/metadata/__init__.py M Lib/test/test_importlib/test_metadata_api.py diff --git a/Lib/importlib/metadata/__init__.py b/Lib/importlib/metadata/__init__.py index b3e8fb05f1d98..ec41ed39157a9 100644 --- a/Lib/importlib/metadata/__init__.py +++ b/Lib/importlib/metadata/__init__.py @@ -669,7 +669,7 @@ def _convert_egg_info_reqs_to_simple_reqs(sections): def make_condition(name): return name and f'extra == "{name}"' - def parse_condition(section): + def quoted_marker(section): section = section or '' extra, sep, markers = section.partition(':') if extra and markers: @@ -677,8 +677,17 @@ def parse_condition(section): conditions = list(filter(None, [markers, make_condition(extra)])) return '; ' + ' and '.join(conditions) if conditions else '' + def url_req_space(req): + """ + PEP 508 requires a space between the url_spec and the quoted_marker. + Ref python/importlib_metadata#357. + """ + # '@' is uniquely indicative of a url_req. + return ' ' * ('@' in req) + for section in sections: - yield section.value + parse_condition(section.name) + space = url_req_space(section.value) + yield section.value + space + quoted_marker(section.name) class DistributionFinder(MetaPathFinder): diff --git a/Lib/test/test_importlib/test_metadata_api.py b/Lib/test/test_importlib/test_metadata_api.py index dc9c234d15b78..4a45312e31cd5 100644 --- a/Lib/test/test_importlib/test_metadata_api.py +++ b/Lib/test/test_importlib/test_metadata_api.py @@ -235,6 +235,7 @@ def test_more_complex_deps_requires_text(self): [extra1] dep4 + dep6@ git+https://example.com/python/dep.git@v1.0.0 [extra2:python_version < "3"] dep5 @@ -247,6 +248,7 @@ def test_more_complex_deps_requires_text(self): 'dep3; python_version < "3"', 'dep4; extra == "extra1"', 'dep5; (python_version < "3") and extra == "extra2"', + 'dep6@ git+https://example.com/python/dep.git@v1.0.0 ; extra == "extra1"', ] # It's important that the environment marker expression be # wrapped in parentheses to avoid the following 'and' binding more diff --git a/Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst b/Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst new file mode 100644 index 0000000000000..145edccb2e7a5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-12-16-14-30-36.bpo-46105.pprB1K.rst @@ -0,0 +1,2 @@ +Honor spec when generating requirement specs with urls and extras +(importlib_metadata 4.8.3).

1 0

bpo-46044: Annotate deprecated sdists formats (GH-30043)
by miss-islington Dec. 16, 2021

Dec. 16, 2021

https://github.com/python/cpython/commit/ecdc0ccede5f9ac4042ff56f295d81df2f… commit: ecdc0ccede5f9ac4042ff56f295d81df2f428950 branch: main author: Matthias Bussonnier <bussonniermatthias(a)gmail.com> committer: miss-islington <31488909+miss-islington(a)users.noreply.github.com> date: 2021-12-16T11:57:39-08:00 summary: bpo-46044: Annotate deprecated sdists formats (GH-30043) While this page have deprecated informations it is still heavily index by Google. Discussed on twitter: https://twitter.com/brettsky/status/1469465729082662916 files: M Doc/distutils/sourcedist.rst diff --git a/Doc/distutils/sourcedist.rst b/Doc/distutils/sourcedist.rst index 0600663d00e9d..43fbb03547f78 100644 --- a/Doc/distutils/sourcedist.rst +++ b/Doc/distutils/sourcedist.rst @@ -23,25 +23,25 @@ option, for example:: to create a gzipped tarball and a zip file. The available formats are: -+-----------+-------------------------+---------+ -| Format | Description | Notes | -+===========+=========================+=========+ -| ``zip`` | zip file (:file:`.zip`) | (1),(3) | -+-----------+-------------------------+---------+ -| ``gztar`` | gzip'ed tar file | \(2) | -| | (:file:`.tar.gz`) | | -+-----------+-------------------------+---------+ -| ``bztar`` | bzip2'ed tar file | | -| | (:file:`.tar.bz2`) | | -+-----------+-------------------------+---------+ -| ``xztar`` | xz'ed tar file | | -| | (:file:`.tar.xz`) | | -+-----------+-------------------------+---------+ -| ``ztar`` | compressed tar file | \(4) | -| | (:file:`.tar.Z`) | | -+-----------+-------------------------+---------+ -| ``tar`` | tar file (:file:`.tar`) | | -+-----------+-------------------------+---------+ ++-----------+-------------------------+-------------+ +| Format | Description | Notes | ++===========+=========================+=============+ +| ``zip`` | zip file (:file:`.zip`) | (1),(3) | ++-----------+-------------------------+-------------+ +| ``gztar`` | gzip'ed tar file | \(2) | +| | (:file:`.tar.gz`) | | ++-----------+-------------------------+-------------+ +| ``bztar`` | bzip2'ed tar file | \(5) | +| | (:file:`.tar.bz2`) | | ++-----------+-------------------------+-------------+ +| ``xztar`` | xz'ed tar file | \(5) | +| | (:file:`.tar.xz`) | | ++-----------+-------------------------+-------------+ +| ``ztar`` | compressed tar file | \(5) | +| | (:file:`.tar.Z`) | | ++-----------+-------------------------+-------------+ +| ``tar`` | tar file (:file:`.tar`) | \(5) | ++-----------+-------------------------+-------------+ .. versionchanged:: 3.5 Added support for the ``xztar`` format. @@ -61,6 +61,9 @@ Notes: (4) requires the :program:`compress` program. Notice that this format is now pending for deprecation and will be removed in the future versions of Python. +(5) + deprecated by `PEP 527 <https://www.python.org/dev/peps/pep-0527/>`_; + `PyPI <https://pypi.org>`_ only accepts ``.zip`` and ``.tar.gz`` files. When using any ``tar`` format (``gztar``, ``bztar``, ``xztar``, ``ztar`` or ``tar``), under Unix you can specify the ``owner`` and ``group`` names

1 0

Better randomization of stats filenames. (GH-30145)
by markshannon Dec. 16, 2021

Dec. 16, 2021

https://github.com/python/cpython/commit/86f42851c050d756679ae7797f8720adae… commit: 86f42851c050d756679ae7797f8720adaef381c4 branch: main author: Mark Shannon <mark(a)hotpy.org> committer: markshannon <mark(a)hotpy.org> date: 2021-12-16T15:56:01Z summary: Better randomization of stats filenames. (GH-30145) files: M Python/specialize.c diff --git a/Python/specialize.c b/Python/specialize.c index 730e2f045ebe5..1f168e31e6d3a 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -167,8 +167,20 @@ _Py_PrintSpecializationStats(int to_file) # else const char *dirname = "/tmp/py_stats/"; # endif - char buf[48]; - sprintf(buf, "%s%u_%u.txt", dirname, (unsigned)clock(), (unsigned)rand()); + /* Use random 160 bit number as file name, + * to avoid both accidental collisions and + * symlink attacks. */ + unsigned char rand[20]; + char hex_name[41]; + _PyOS_URandomNonblock(rand, 20); + for (int i = 0; i < 20; i++) { + hex_name[2*i] = "0123456789abcdef"[rand[i]&15]; + hex_name[2*i+1] = "0123456789abcdef"[(rand[i]>>4)&15]; + } + hex_name[40] = '\0'; + char buf[64]; + assert(strlen(dirname) + 40 + strlen(".txt") < 64); + sprintf(buf, "%s%s.txt", dirname, hex_name); FILE *fout = fopen(buf, "w"); if (fout) { out = fout;

1 0

bpo-22047: [argparse] deprecate nested argument groups and mutually exclusive groups (GH-30098)
by iritkatriel Dec. 16, 2021

Dec. 16, 2021

https://github.com/python/cpython/commit/30322c497e0b8d978f7a0de95985aac9c5… commit: 30322c497e0b8d978f7a0de95985aac9c5daf1ac branch: main author: Irit Katriel <1055913+iritkatriel(a)users.noreply.github.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2021-12-16T15:31:08Z summary: bpo-22047: [argparse] deprecate nested argument groups and mutually exclusive groups (GH-30098) files: A Misc/NEWS.d/next/Library/2021-12-15-19-24-54.bpo-22047.gBV4vT.rst M Doc/library/argparse.rst M Lib/argparse.py M Lib/test/test_argparse.py diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst index 2edf84e5f0444..80c382a981b8d 100644 --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -1898,6 +1898,12 @@ Argument groups Note that any arguments not in your user-defined groups will end up back in the usual "positional arguments" and "optional arguments" sections. + .. versionchanged:: 3.11 + Calling :meth:`add_argument_group` on an argument group is deprecated. + This feature was never supported and does not always work correctly. + The function exists on the API by accident through inheritance and + will be removed in the future. + Mutual exclusion ^^^^^^^^^^^^^^^^ @@ -1936,6 +1942,12 @@ Mutual exclusion *title* and *description* arguments of :meth:`~ArgumentParser.add_argument_group`. + .. versionchanged:: 3.11 + Calling :meth:`add_argument_group` or :meth:`add_mutually_exclusive_group` + on a mutually exclusive group is deprecated. These features were never + supported and do not always work correctly. The functions exist on the + API by accident through inheritance and will be removed in the future. + Parser defaults ^^^^^^^^^^^^^^^ diff --git a/Lib/argparse.py b/Lib/argparse.py index 8a81801ba9236..de95eedbee0ee 100644 --- a/Lib/argparse.py +++ b/Lib/argparse.py @@ -89,6 +89,8 @@ import re as _re import sys as _sys +import warnings + from gettext import gettext as _, ngettext SUPPRESS = '==SUPPRESS==' @@ -1648,6 +1650,14 @@ def _remove_action(self, action): super(_ArgumentGroup, self)._remove_action(action) self._group_actions.remove(action) + def add_argument_group(self, *args, **kwargs): + warnings.warn( + "Nesting argument groups is deprecated.", + category=DeprecationWarning, + stacklevel=2 + ) + return super().add_argument_group(*args, **kwargs) + class _MutuallyExclusiveGroup(_ArgumentGroup): @@ -1668,6 +1678,14 @@ def _remove_action(self, action): self._container._remove_action(action) self._group_actions.remove(action) + def add_mutually_exclusive_group(self, *args, **kwargs): + warnings.warn( + "Nesting mutually exclusive groups is deprecated.", + category=DeprecationWarning, + stacklevel=2 + ) + return super().add_mutually_exclusive_group(*args, **kwargs) + class ArgumentParser(_AttributeHolder, _ActionsContainer): """Object for parsing command line strings into Python objects. diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py index eb37d4d365d24..4c23610b9e961 100644 --- a/Lib/test/test_argparse.py +++ b/Lib/test/test_argparse.py @@ -9,6 +9,7 @@ import tempfile import unittest import argparse +import warnings from io import StringIO @@ -2973,15 +2974,24 @@ def get_parser(self, required): class TestMutuallyExclusiveNested(MEMixin, TestCase): + # Nesting mutually exclusive groups is an undocumented feature + # that came about by accident through inheritance and has been + # the source of many bugs. It is deprecated and this test should + # eventually be removed along with it. + def get_parser(self, required): parser = ErrorRaisingArgumentParser(prog='PROG') group = parser.add_mutually_exclusive_group(required=required) group.add_argument('-a') group.add_argument('-b') - group2 = group.add_mutually_exclusive_group(required=required) + with warnings.catch_warnings(): + warnings.simplefilter('ignore', DeprecationWarning) + group2 = group.add_mutually_exclusive_group(required=required) group2.add_argument('-c') group2.add_argument('-d') - group3 = group2.add_mutually_exclusive_group(required=required) + with warnings.catch_warnings(): + warnings.simplefilter('ignore', DeprecationWarning) + group3 = group2.add_mutually_exclusive_group(required=required) group3.add_argument('-e') group3.add_argument('-f') return parser diff --git a/Misc/NEWS.d/next/Library/2021-12-15-19-24-54.bpo-22047.gBV4vT.rst b/Misc/NEWS.d/next/Library/2021-12-15-19-24-54.bpo-22047.gBV4vT.rst new file mode 100644 index 0000000000000..a381ad88af6c5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-12-15-19-24-54.bpo-22047.gBV4vT.rst @@ -0,0 +1,3 @@ +Calling :meth:`add_argument_group` on an argument group is deprecated. Calling :meth:`add_argument_group` or :meth:`add_mutually_exclusive_group` on a mutually exclusive group is deprecated. + +These features were never supported and do not always work correctly. The functions exist on the API by accident through inheritance and will be removed in the future. \ No newline at end of file

1 0

bpo-45755: [typing] Reveal class attributes of generic in generic aliases in `dir()` (GH-29962)
by Fidget-Spinner Dec. 16, 2021

Dec. 16, 2021

https://github.com/python/cpython/commit/d6e13747161d7b634b47d2d3d212ed3be4… commit: d6e13747161d7b634b47d2d3d212ed3be4a21fab branch: main author: Ken Jin <28750310+Fidget-Spinner(a)users.noreply.github.com> committer: Fidget-Spinner <28750310+Fidget-Spinner(a)users.noreply.github.com> date: 2021-12-16T22:29:59+08:00 summary: bpo-45755: [typing] Reveal class attributes of generic in generic aliases in `dir()` (GH-29962) files: A Misc/NEWS.d/next/Library/2021-12-07-21-55-22.bpo-45755.bRqKGa.rst M Lib/test/test_typing.py M Lib/typing.py diff --git a/Lib/test/test_typing.py b/Lib/test/test_typing.py index 90d6bea59f899..a94d77d4edf4b 100644 --- a/Lib/test/test_typing.py +++ b/Lib/test/test_typing.py @@ -5040,6 +5040,17 @@ def test_special_attrs2(self): loaded = pickle.loads(s) self.assertIs(SpecialAttrsP, loaded) + def test_genericalias_dir(self): + class Foo(Generic[T]): + def bar(self): + pass + baz = 3 + # The class attributes of the original class should be visible even + # in dir() of the GenericAlias. See bpo-45755. + self.assertIn('bar', dir(Foo[int])) + self.assertIn('baz', dir(Foo[int])) + + class AllTests(BaseTestCase): """Tests for __all__.""" diff --git a/Lib/typing.py b/Lib/typing.py index bdd19cadb471a..ae1dd5c2d7689 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -991,6 +991,9 @@ def __subclasscheck__(self, cls): raise TypeError("Subscripted generics cannot be used with" " class and instance checks") + def __dir__(self): + return list(set(super().__dir__() + + [attr for attr in dir(self.__origin__) if not _is_dunder(attr)])) # Special typing constructs Union, Optional, Generic, Callable and Tuple # use three special attributes for internal bookkeeping of generic types: diff --git a/Misc/NEWS.d/next/Library/2021-12-07-21-55-22.bpo-45755.bRqKGa.rst b/Misc/NEWS.d/next/Library/2021-12-07-21-55-22.bpo-45755.bRqKGa.rst new file mode 100644 index 0000000000000..e5201b0dfde2d --- /dev/null +++ b/Misc/NEWS.d/next/Library/2021-12-07-21-55-22.bpo-45755.bRqKGa.rst @@ -0,0 +1,3 @@ +:mod:`typing` generic aliases now reveal the class attributes of the +original generic class when passed to ``dir()``. This was the behavior up to +Python 3.6, but was changed in 3.7-3.9.

1 0