Mailman 3 March 2021 - docs

[issue36868] New behavior of OpenSSL hostname verification not exposed, incorrectly documented
by Josh Rosenberg 18 Apr '21

18 Apr '21

New submission from Josh Rosenberg <shadowranger+python(a)gmail.com>: The What's New in 3.7 docs mention the change from #31399 to use OpenSSL's built-in hostname verification ( https://docs.python.org/3/whatsnew/3.7.html#ssl ), but aside from that, information about the change is largely undiscoverable and/or wrong. Specific problems: 1. The What's New docs repeatedly mention SSLContext.host_flags as the means of modifying behavior. The actual property is underscore prefixed though, as SSLContext._host_flags. Since SSLContext supports the creation of arbitrary names via __dict__, assigning to context.host_flags silently "works", it just fails to *do* anything (nothing ever reads it). 2. None of the flags are documented anywhere; the only way to discover them is to import _ssl (they're not exposed on ssl itself, just the internal C extension), then scan through the exposed names (they're all prefixed with HOSTFLAG_ AFAICT) 3. All of the flags are raw numeric values, but it seems like they should be IntEnums, like the other flags exposed by SSL (among other things, it would make it much easier to interpret the default _host_flags (currently it's just 4, when it could display as <HostFlags.HOSTFLAG_NO_PARTIAL_WILDCARDS: 4>) 4. Nothing about this change, _host_flags/host_flags, or the values of the flags themselves is mentioned on the ssl docs at all. 5. This unintentionally made a behavioral change (one that bit me, and may bite other folks using docker swarm, NETBIOS hostnames, etc.). Python's match_hostname implementation was fine with host names containing underscores (e.g. if the cert was wildcarded to *.example.com, it would match a_b_c.example.com just fine); they're not technically legal by the strict reading of the specs for host names (they're apparently legal for domain names, but not host names, which differ in ways I don't fully understand), but stuff like docker swarm names their services that way automatically, most (all?) browsers support visiting them, etc. It looks like OpenSSL (at least the 1.1.0g my Python 3.7.2 was built against) treats underscores as unmatchable, so any attempt to connect to such a host name in Python 3.7.2 dies with a SSLCertVerificationError, claiming a "Hostname mismatch, certificate is not valid for 'name_with_underscores.example.com'." I discovered all this because 3.7 broke some scripts I use to connect to docker swarm services. Before I realized the issue was underscores, I was trying to figure out how to tweak the host name checks (assuming maybe something was broken with wildcard matching), and stumbled across all the other issues with the docs, the lack of flag definition exposure, etc. For the record, I think it's reasonable to require legal host names (it was easy enough to fix for my case; I just updated our docker DNS server to provide aliases using only hyphens and changed the script to use the alias host names), but it would be nice if it was explicitly documented, and ideally, that Python itself recognize that underscores won't work and explicitly raise an exception saying why, rather than letting OpenSSL perform the rejection with a (to someone who doesn't know about the underscore issue) confusing error message. ---------- assignee: docs@python components: Documentation, Extension Modules, Library (Lib), SSL keywords: 3.6regression messages: 341990 nosy: christian.heimes, docs@python, josh.r, vstinner priority: normal severity: normal status: open title: New behavior of OpenSSL hostname verification not exposed, incorrectly documented type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue36868> _______________________________________

5 8

[issue34078] Broken CRL functionality in ssl.py
by Joe N 17 Apr '21

17 Apr '21

New submission from Joe N <nettijoe(a)gmail.com>: CRLs in ssl.py or at the documentation is broken. Specifically I think the documentation here is wrong: https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_verify_locat… Here is a stackoverflow post: https://stackoverflow.com/questions/51196492/how-to-use-crls-in-pyopenssl?n… I made a very user friendly test suite of files to show how it is broken. Run the code in here (follow readme instructions) to see the bug. https://github.com/nettijoe96/bugInSSL ---------- assignee: christian.heimes components: SSL messages: 321343 nosy: Joe N, christian.heimes, docs@python priority: normal severity: normal status: open title: Broken CRL functionality in ssl.py type: behavior versions: Python 3.6 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue34078> _______________________________________

2 2

[issue22128] patch: steer people away from codecs.open
by Frank van Dijk 17 Apr '21

17 Apr '21

New submission from Frank van Dijk: stackoverflow.com has a zillion answers recommending the use of codecs.open() as a unicode capable drop in replacement for open(). This probably means that there is still a lot of code being written that uses codecs.open(). That's bad thing because of codecs.open()'s lack of newline conversion. A lot of that code will - have compatibility issues when it is moved between unix and windows - silently break text files on windows, leading to issues further downstream (confusing other tools, messing up revision control histories) The problem has been fixed with io.open() in 2.x and open() in 3.x. Unfortunately the 2.7 unicode HOWTO still recommends the use of codecs.open(). The 2.7 and the 3.x documentation of codecs.open() doesn't refer the reader to better alternatives. The attached patches fix that. The only downside I see is that newly written code that uses the better alternatives would be incompatible with 2.5 and older. However croaking on a small minority of systems is better than silently disrupting workflows, causing platform incompatibilities, and inviting flaky workarounds. The 2.7 patch makes the unicode HOWTO recommend io.open() instead of codecs.open(). Both patches change the codecs.open() documentation to refer to io.open() or (on 3.x) open(). Additionally I removed the "data loss" explanation from codecs.open()'s note about its lack of newline conversion. It is not particularly helpful information and it is not entirely correct (data loss could also have been avoided by doing newline conversion before encoding and after decoding) ---------- assignee: docs@python components: Documentation files: codecsopen2.patch keywords: patch messages: 224632 nosy: Frank.van.Dijk, docs@python priority: normal severity: normal status: open title: patch: steer people away from codecs.open type: behavior versions: Python 2.7, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file36234/codecsopen2.patch _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue22128> _______________________________________

7 11

[issue42037] Documentation confusion in CookieJar functions
by Markus Israelsson 16 Apr '21

16 Apr '21

New submission from Markus Israelsson <markus.israelsson(a)surgicalscience.com>: The documentation in https://docs.python.org/3.8/library/http.cookiejar.html#http.cookiejar.Cook… claims the following for functions add_cookie_header and extract_cookies. *** The request object (usually a urllib.request.Request instance) must support the methods get_full_url(), get_host(), get_type(), unverifiable(), has_header(), get_header(), header_items(), add_unredirected_header() and origin_req_host attribute as documented by urllib.request. *** When reading the documentation for Request Objects https://docs.python.org/3.8/library/urllib.request.html?highlight=requests#… there is this: *** Changed in version 3.4: The request methods add_data, has_data, get_data, get_type, get_host, get_selector, get_origin_req_host and is_unverifiable that were deprecated since 3.3 have been removed. *** So basically the documentation claims that if the request object does not support functions that are removed then the headers will not be added. The code itself seem to do the correct thing however and add the header. ---------- assignee: docs@python components: Documentation messages: 378624 nosy: docs@python, markus priority: normal severity: normal status: open title: Documentation confusion in CookieJar functions type: enhancement versions: Python 3.8 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue42037> _______________________________________

4 10

[issue40006] enum: Add documentation for _create_pseudo_member_ and composite members
by Ram Rachum 15 Apr '21

15 Apr '21

New submission from Ram Rachum <ram(a)rachum.com>: Looking at the enum source code, there's a method `_create_pseudo_member_` that's used in a bunch of places. Its docstring says "Create a composite member iff value contains only members", which would have been useful if I had any idea what "composite member" meant. It would be good if the documentation for the enum module would include more information about these two concepts. ---------- assignee: docs@python components: Documentation messages: 364561 nosy: cool-RR, docs@python priority: normal severity: normal status: open title: enum: Add documentation for _create_pseudo_member_ and composite members type: enhancement versions: Python 3.8, Python 3.9 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue40006> _______________________________________

3 4

[issue37741] importlib.metadata docs not showing up in the module index
by Brett Cannon 15 Apr '21

15 Apr '21

New submission from Brett Cannon <brett(a)python.org>: If you look at https://docs.python.org/3.9/py-modindex.html#cap-i you will see that importlib.metadata isn't listed (same goes for the 3.8 docs). Or are you leaving it out due to it being provisional? ---------- assignee: docs@python components: Documentation messages: 348872 nosy: barry, brett.cannon, docs@python, jaraco priority: normal severity: normal status: open title: importlib.metadata docs not showing up in the module index versions: Python 3.8, Python 3.9 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue37741> _______________________________________

7 19

exception not documented for popping from mutable sequences
by Alan G. Isaac 15 Apr '21

15 Apr '21

Should the documentation for Mutable Sequence Types https://docs.python.org/3/library/stdtypes.html#mutable-sequence-types specify for the `pop` operation that an `IndexError` will be raised if there is no item at that index? (E.g., this could be added to note (2).) Cheers, Alan Isaac

2 1

Docs clarification for subprocess.run
by Davis, Matthew 15 Apr '21

15 Apr '21

Hi, I think the documentation for subprocess.run is wrong, or at least unclear. https://docs.python.org/3/library/subprocess.html > The stdout and stderr arguments may not be supplied at the same time as capture_output. If you wish to capture and combine both streams into one, use stdout=PIPE and stderr=STDOUT instead of capture_output. Here’s what happens when I try that: >>> p = subprocess.run(['ls', '-l'], stdout=PIPE, stderr=STDOUT) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'PIPE' is not defined PIPE and STDOUT are not defined. Is the latter supposed to be quoted to become a string? Or should I use sys.stdout? If so, that’s what the docs should say. I have no idea what PIPE is. Is that supposed to be a generic file-like object? It’s not clear. Thanks, Matt

2 1

[issue28197] range.index mismatch with documentation
by Vedran Čačić 14 Apr '21

14 Apr '21

New submission from Vedran Čačić: Look at this: >>> from collections.abc import Sequence >>> help(Sequence.index) index(self, value, start=0, stop=None) S.index(value, [start, [stop]]) -> integer -- return first index of value. Raises ValueError if the value is not present. >>> issubclass(range, Sequence) True >>> help(range.index) index(...) rangeobject.index(value, [start, [stop]]) -> integer -- return index of value. Raise ValueError if the value is not present. So far, so good. But: >>> range(9).index(2, 1, 5) TypeError: index() takes exactly one argument (3 given) Of course it's not essential, but the docs shouldn't lie. And if range _is_ a Sequence, then it should have the complete interface of a Sequence. Including start and end arguments for .index: they are optional from the point of call, not from the point of implementation. :-) ---------- assignee: docs@python components: Documentation, Library (Lib) messages: 276908 nosy: docs@python, veky priority: normal severity: normal status: open title: range.index mismatch with documentation type: behavior versions: Python 3.5, Python 3.6 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue28197> _______________________________________

6 17

[issue20364] Rename & explain sqlite3.Cursor.execute 'parameters' param
by Terry J. Reedy 14 Apr '21

14 Apr '21

New submission from Terry J. Reedy: "execute(sql[, parameters]) Executes an SQL statement. The SQL statement may be parametrized (i. e. placeholders instead of SQL literals). The sqlite3 module supports two kinds of placeholders: question marks (qmark style) and named placeholders (named style)." Experimental facts based on experiments with the code example in the doc, using 3.4.b2: 'parameters' is a single subscriptable collection parameter, sequence or dict, that might be called seq_dict. It is positional only, so whatever name is used is a dummy. Only one placeholder style can be used in a given SQL statement string. If question marks are used, seq_dict must be a sequence. If names are used, seq_dict can be either a sequence or dict or subclass thereof. A UserDict is treated as a sequence and raises KeyError(0). Possible text that encompasses the above, replacing the last sentence: "A statement may use one of two kinds of placeholders: question marks (qmark style) or named placeholders (named style). For qmark style, seq_dict must be a sequence. For named style, it can be either a sequence or dict instance. Len(seq_dict) must match the number of placeholders." After cleaning up the test file, I will verify on 2.7 and upload. ---------- assignee: docs@python components: Documentation, Library (Lib) messages: 208908 nosy: docs@python, terry.reedy priority: normal severity: normal stage: patch review status: open title: Rename & explain sqlite3.Cursor.execute 'parameters' param type: behavior versions: Python 2.7, Python 3.3, Python 3.4 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue20364> _______________________________________

7 12