docs May 2020

docs@python.org

19 participants
418 discussions

[issue36868] New behavior of OpenSSL hostname verification not exposed, incorrectly documented

by Josh Rosenberg

New submission from Josh Rosenberg <shadowranger+python(a)gmail.com>: The What's New in 3.7 docs mention the change from #31399 to use OpenSSL's built-in hostname verification ( https://docs.python.org/3/whatsnew/3.7.html#ssl ), but aside from that, information about the change is largely undiscoverable and/or wrong. Specific problems: 1. The What's New docs repeatedly mention SSLContext.host_flags as the means of modifying behavior. The actual property is underscore prefixed though, as SSLContext._host_flags. Since SSLContext supports the creation of arbitrary names via __dict__, assigning to context.host_flags silently "works", it just fails to *do* anything (nothing ever reads it). 2. None of the flags are documented anywhere; the only way to discover them is to import _ssl (they're not exposed on ssl itself, just the internal C extension), then scan through the exposed names (they're all prefixed with HOSTFLAG_ AFAICT) 3. All of the flags are raw numeric values, but it seems like they should be IntEnums, like the other flags exposed by SSL (among other things, it would make it much easier to interpret the default _host_flags (currently it's just 4, when it could display as <HostFlags.HOSTFLAG_NO_PARTIAL_WILDCARDS: 4>) 4. Nothing about this change, _host_flags/host_flags, or the values of the flags themselves is mentioned on the ssl docs at all. 5. This unintentionally made a behavioral change (one that bit me, and may bite other folks using docker swarm, NETBIOS hostnames, etc.). Python's match_hostname implementation was fine with host names containing underscores (e.g. if the cert was wildcarded to *.example.com, it would match a_b_c.example.com just fine); they're not technically legal by the strict reading of the specs for host names (they're apparently legal for domain names, but not host names, which differ in ways I don't fully understand), but stuff like docker swarm names their services that way automatically, most (all?) browsers support visiting them, etc. It looks like OpenSSL (at least the 1.1.0g my Python 3.7.2 was built against) treats underscores as unmatchable, so any attempt to connect to such a host name in Python 3.7.2 dies with a SSLCertVerificationError, claiming a "Hostname mismatch, certificate is not valid for 'name_with_underscores.example.com'." I discovered all this because 3.7 broke some scripts I use to connect to docker swarm services. Before I realized the issue was underscores, I was trying to figure out how to tweak the host name checks (assuming maybe something was broken with wildcard matching), and stumbled across all the other issues with the docs, the lack of flag definition exposure, etc. For the record, I think it's reasonable to require legal host names (it was easy enough to fix for my case; I just updated our docker DNS server to provide aliases using only hyphens and changed the script to use the alias host names), but it would be nice if it was explicitly documented, and ideally, that Python itself recognize that underscores won't work and explicitly raise an exception saying why, rather than letting OpenSSL perform the rejection with a (to someone who doesn't know about the underscore issue) confusing error message. ---------- assignee: docs@python components: Documentation, Extension Modules, Library (Lib), SSL keywords: 3.6regression messages: 341990 nosy: christian.heimes, docs@python, josh.r, vstinner priority: normal severity: normal status: open title: New behavior of OpenSSL hostname verification not exposed, incorrectly documented type: behavior versions: Python 3.7, Python 3.8 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue36868> _______________________________________

5 months

[issue34078] Broken CRL functionality in ssl.py

by Joe N

New submission from Joe N <nettijoe(a)gmail.com>: CRLs in ssl.py or at the documentation is broken. Specifically I think the documentation here is wrong: https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_verify_locat… Here is a stackoverflow post: https://stackoverflow.com/questions/51196492/how-to-use-crls-in-pyopenssl?n… I made a very user friendly test suite of files to show how it is broken. Run the code in here (follow readme instructions) to see the bug. https://github.com/nettijoe96/bugInSSL ---------- assignee: christian.heimes components: SSL messages: 321343 nosy: Joe N, christian.heimes, docs@python priority: normal severity: normal status: open title: Broken CRL functionality in ssl.py type: behavior versions: Python 3.6 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue34078> _______________________________________

5 months

[issue22128] patch: steer people away from codecs.open

by Frank van Dijk

New submission from Frank van Dijk: stackoverflow.com has a zillion answers recommending the use of codecs.open() as a unicode capable drop in replacement for open(). This probably means that there is still a lot of code being written that uses codecs.open(). That's bad thing because of codecs.open()'s lack of newline conversion. A lot of that code will - have compatibility issues when it is moved between unix and windows - silently break text files on windows, leading to issues further downstream (confusing other tools, messing up revision control histories) The problem has been fixed with io.open() in 2.x and open() in 3.x. Unfortunately the 2.7 unicode HOWTO still recommends the use of codecs.open(). The 2.7 and the 3.x documentation of codecs.open() doesn't refer the reader to better alternatives. The attached patches fix that. The only downside I see is that newly written code that uses the better alternatives would be incompatible with 2.5 and older. However croaking on a small minority of systems is better than silently disrupting workflows, causing platform incompatibilities, and inviting flaky workarounds. The 2.7 patch makes the unicode HOWTO recommend io.open() instead of codecs.open(). Both patches change the codecs.open() documentation to refer to io.open() or (on 3.x) open(). Additionally I removed the "data loss" explanation from codecs.open()'s note about its lack of newline conversion. It is not particularly helpful information and it is not entirely correct (data loss could also have been avoided by doing newline conversion before encoding and after decoding) ---------- assignee: docs@python components: Documentation files: codecsopen2.patch keywords: patch messages: 224632 nosy: Frank.van.Dijk, docs@python priority: normal severity: normal status: open title: patch: steer people away from codecs.open type: behavior versions: Python 2.7, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file36234/codecsopen2.patch _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue22128> _______________________________________

5 months, 1 week

[issue40006] enum: Add documentation for _create_pseudo_member_ and composite members

by Ram Rachum

New submission from Ram Rachum <ram(a)rachum.com>: Looking at the enum source code, there's a method `_create_pseudo_member_` that's used in a bunch of places. Its docstring says "Create a composite member iff value contains only members", which would have been useful if I had any idea what "composite member" meant. It would be good if the documentation for the enum module would include more information about these two concepts. ---------- assignee: docs@python components: Documentation messages: 364561 nosy: cool-RR, docs@python priority: normal severity: normal status: open title: enum: Add documentation for _create_pseudo_member_ and composite members type: enhancement versions: Python 3.8, Python 3.9 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue40006> _______________________________________

5 months, 1 week

[issue37741] importlib.metadata docs not showing up in the module index

by Brett Cannon

New submission from Brett Cannon <brett(a)python.org>: If you look at https://docs.python.org/3.9/py-modindex.html#cap-i you will see that importlib.metadata isn't listed (same goes for the 3.8 docs). Or are you leaving it out due to it being provisional? ---------- assignee: docs@python components: Documentation messages: 348872 nosy: barry, brett.cannon, docs@python, jaraco priority: normal severity: normal status: open title: importlib.metadata docs not showing up in the module index versions: Python 3.8, Python 3.9 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue37741> _______________________________________

5 months, 1 week

[issue28197] range.index mismatch with documentation

by Vedran Čačić

New submission from Vedran Čačić: Look at this: >>> from collections.abc import Sequence >>> help(Sequence.index) index(self, value, start=0, stop=None) S.index(value, [start, [stop]]) -> integer -- return first index of value. Raises ValueError if the value is not present. >>> issubclass(range, Sequence) True >>> help(range.index) index(...) rangeobject.index(value, [start, [stop]]) -> integer -- return index of value. Raise ValueError if the value is not present. So far, so good. But: >>> range(9).index(2, 1, 5) TypeError: index() takes exactly one argument (3 given) Of course it's not essential, but the docs shouldn't lie. And if range _is_ a Sequence, then it should have the complete interface of a Sequence. Including start and end arguments for .index: they are optional from the point of call, not from the point of implementation. :-) ---------- assignee: docs@python components: Documentation, Library (Lib) messages: 276908 nosy: docs@python, veky priority: normal severity: normal status: open title: range.index mismatch with documentation type: behavior versions: Python 3.5, Python 3.6 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue28197> _______________________________________

5 months, 1 week

[issue20364] Rename & explain sqlite3.Cursor.execute 'parameters' param

by Terry J. Reedy

New submission from Terry J. Reedy: "execute(sql[, parameters]) Executes an SQL statement. The SQL statement may be parametrized (i. e. placeholders instead of SQL literals). The sqlite3 module supports two kinds of placeholders: question marks (qmark style) and named placeholders (named style)." Experimental facts based on experiments with the code example in the doc, using 3.4.b2: 'parameters' is a single subscriptable collection parameter, sequence or dict, that might be called seq_dict. It is positional only, so whatever name is used is a dummy. Only one placeholder style can be used in a given SQL statement string. If question marks are used, seq_dict must be a sequence. If names are used, seq_dict can be either a sequence or dict or subclass thereof. A UserDict is treated as a sequence and raises KeyError(0). Possible text that encompasses the above, replacing the last sentence: "A statement may use one of two kinds of placeholders: question marks (qmark style) or named placeholders (named style). For qmark style, seq_dict must be a sequence. For named style, it can be either a sequence or dict instance. Len(seq_dict) must match the number of placeholders." After cleaning up the test file, I will verify on 2.7 and upload. ---------- assignee: docs@python components: Documentation, Library (Lib) messages: 208908 nosy: docs@python, terry.reedy priority: normal severity: normal stage: patch review status: open title: Rename & explain sqlite3.Cursor.execute 'parameters' param type: behavior versions: Python 2.7, Python 3.3, Python 3.4 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue20364> _______________________________________

5 months, 1 week

[issue39774] Missing documentation on how to make package executable as script

by igo95862

New submission from igo95862 <igo9586(a)gmail.com>: This is package documentation: https://docs.python.org/3/tutorial/modules.html#packages To make package executable (python -m package) you need to create a file __main__.py in the package directory. This is pretty much not documented anyone aside of trying to run a package missing __main__.py This page already contains information on how to make module executable. (https://docs.python.org/3/tutorial/modules.html#executing-modules-as-scripts) ---------- assignee: docs@python components: Documentation messages: 362790 nosy: docs@python, igo95862 priority: normal severity: normal status: open title: Missing documentation on how to make package executable as script type: enhancement versions: Python 3.8 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue39774> _______________________________________

5 months, 1 week

[issue39125] Type signature of @property not shown in help()

by Nguyễn Gia Phong

New submission from Nguyễn Gia Phong <vn.mcsinyx(a)gmail.com>: Dear Maintainer, I want to request a feature on the generative documentation of type-hinting. As of December 2019, I believe there is no support for generating such information in help(). For demonstration, I have this tiny piece of code class Foo: @property def bar(self) -> int: return 42 @bar.setter def bar(self, value: int) -> None: pass def baz(self, arg: float) -> str: pass whose documentation on CPython 3.7.5 (on Debian testing amd64 if that matters) is generated as class Foo(builtins.object) | Methods defined here: | | baz(self, arg: float) -> str | | ---------------------------------------------------------------------- | Data descriptors defined here: | | __dict__ | dictionary for instance variables (if defined) | | __weakref__ | list of weak references to the object (if defined) | | bar I expect the documentation for bar to be as informative as bar, i.e. something similar to ``bar: int''. As pointed out by ChrisWarrick on freenode#python, the annotations are already present, yet help() is not making use of them: >>> Foo.bar.fget.__annotations__ {'return': <class 'int'>} >>> Foo.bar.fset.__annotations__ {'value': <class 'int'>, 'return': None} Have a Merry Christmas or other holiday of your choice, Nguyễn Gia Phong ---------- assignee: docs@python components: Documentation messages: 358823 nosy: McSinyx, docs@python priority: normal severity: normal status: open title: Type signature of @property not shown in help() type: enhancement versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue39125> _______________________________________

5 months, 1 week

[issue38963] multiprocessing processes seem to "bleed" user information (GID/UID/groups)

by Roman Joost

New submission from Roman Joost <roman(a)bromeco.de>: When running a process which changes UID/GID, some of the following processes will run as the user I change to per process. In order to reproduce (see the attached reproducer): 1. Change the 'USERNAME' to an unprivileged user on your system. 2. Run the reproducer as a user with elevated privileges (e.g. root or some secondary user you have on your system). Mind you, I don't think the user you run as needs elevated privileges, but that's the user I ran as when I observed this behaviour. 3. The reproducer iterates over a list (It stems from a test function which was checking permissions on log files). Observe the print out, which prints the process' GID, UID and secondary groups before we're changing to the users GID, UID and secondary groups. 4. You should observe that at some point the process prints the user information of the user we want to change to not the one which initially started the script. Example output when running locally as root: ('B', (0, 0, [0])) ('A', (0, 0, [0])) ('C', (0, 0, [0])) ('E', (0, 0, [0])) ('D', (0, 0, [0])) ('F', (1002, 1002, [10, 135, 1000, 1002])) ('H', (1002, 1002, [10, 135, 1000, 1002])) ('I', (1002, 1002, [10, 135, 1000, 1002])) ('J', (1002, 1002, [10, 135, 1000, 1002])) ('G', (1002, 1002, [10, 135, 1000, 1002])) ('K', (1002, 1002, [10, 135, 1000, 1002])) ('L', (1002, 1002, [10, 135, 1000, 1002])) ('M', (1002, 1002, [10, 135, 1000, 1002])) ('N', (1002, 1002, [10, 135, 1000, 1002])) I would have expected `0` all the way through. However, if I initialise the Pool with `maxtasksperchild=1` the isolation seems as expected. I don't know whether this is a bug or I'm foolish to invoke multiprocessing like this. I've run out of time to investigate this further. It's certainly strange behaviour to me and I thought I better report it, since reproducing seems fairly deterministic. ---------- assignee: docs@python components: Documentation, Library (Lib) files: reproducer.py messages: 357773 nosy: docs@python, romanofski priority: normal severity: normal status: open title: multiprocessing processes seem to "bleed" user information (GID/UID/groups) type: behavior versions: Python 3.6, Python 3.7 Added file: https://bugs.python.org/file48753/reproducer.py _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue38963> _______________________________________

5 months, 2 weeks

Jump to page:

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

docs May 2020