New submission from Josh Rosenberg <shadowranger+python(a)gmail.com>:
The What's New in 3.7 docs mention the change from #31399 to use OpenSSL's built-in hostname verification ( https://docs.python.org/3/whatsnew/3.7.html#ssl ), but aside from that, information about the change is largely undiscoverable and/or wrong.
Specific problems:
1. The What's New docs repeatedly mention SSLContext.host_flags as the means of modifying behavior. The actual property is underscore prefixed though, as SSLContext._host_flags. Since SSLContext supports the creation of arbitrary names via __dict__, assigning to context.host_flags silently "works", it just fails to *do* anything (nothing ever reads it).
2. None of the flags are documented anywhere; the only way to discover them is to import _ssl (they're not exposed on ssl itself, just the internal C extension), then scan through the exposed names (they're all prefixed with HOSTFLAG_ AFAICT)
3. All of the flags are raw numeric values, but it seems like they should be IntEnums, like the other flags exposed by SSL (among other things, it would make it much easier to interpret the default _host_flags (currently it's just 4, when it could display as <HostFlags.HOSTFLAG_NO_PARTIAL_WILDCARDS: 4>)
4. Nothing about this change, _host_flags/host_flags, or the values of the flags themselves is mentioned on the ssl docs at all.
5. This unintentionally made a behavioral change (one that bit me, and may bite other folks using docker swarm, NETBIOS hostnames, etc.). Python's match_hostname implementation was fine with host names containing underscores (e.g. if the cert was wildcarded to *.example.com, it would match a_b_c.example.com just fine); they're not technically legal by the strict reading of the specs for host names (they're apparently legal for domain names, but not host names, which differ in ways I don't fully understand), but stuff like docker swarm names their services that way automatically, most (all?) browsers support visiting them, etc. It looks like OpenSSL (at least the 1.1.0g my Python 3.7.2 was built against) treats underscores as unmatchable, so any attempt to connect to such a host name in Python 3.7.2 dies with a SSLCertVerificationError, claiming a "Hostname mismatch, certificate is not valid for 'name_with_underscores.example.com'."
I discovered all this because 3.7 broke some scripts I use to connect to docker swarm services. Before I realized the issue was underscores, I was trying to figure out how to tweak the host name checks (assuming maybe something was broken with wildcard matching), and stumbled across all the other issues with the docs, the lack of flag definition exposure, etc.
For the record, I think it's reasonable to require legal host names (it was easy enough to fix for my case; I just updated our docker DNS server to provide aliases using only hyphens and changed the script to use the alias host names), but it would be nice if it was explicitly documented, and ideally, that Python itself recognize that underscores won't work and explicitly raise an exception saying why, rather than letting OpenSSL perform the rejection with a (to someone who doesn't know about the underscore issue) confusing error message.
----------
assignee: docs@python
components: Documentation, Extension Modules, Library (Lib), SSL
keywords: 3.6regression
messages: 341990
nosy: christian.heimes, docs@python, josh.r, vstinner
priority: normal
severity: normal
status: open
title: New behavior of OpenSSL hostname verification not exposed, incorrectly documented
type: behavior
versions: Python 3.7, Python 3.8
_______________________________________
Python tracker <report(a)bugs.python.org>
<https://bugs.python.org/issue36868>
_______________________________________
New submission from Joe N <nettijoe(a)gmail.com>:
CRLs in ssl.py or at the documentation is broken. Specifically I think the documentation here is wrong: https://docs.python.org/3/library/ssl.html#ssl.SSLContext.load_verify_locat…
Here is a stackoverflow post: https://stackoverflow.com/questions/51196492/how-to-use-crls-in-pyopenssl?n…
I made a very user friendly test suite of files to show how it is broken.
Run the code in here (follow readme instructions) to see the bug.
https://github.com/nettijoe96/bugInSSL
----------
assignee: christian.heimes
components: SSL
messages: 321343
nosy: Joe N, christian.heimes, docs@python
priority: normal
severity: normal
status: open
title: Broken CRL functionality in ssl.py
type: behavior
versions: Python 3.6
_______________________________________
Python tracker <report(a)bugs.python.org>
<https://bugs.python.org/issue34078>
_______________________________________
New submission from Frank van Dijk:
stackoverflow.com has a zillion answers recommending the use of codecs.open() as a unicode capable drop in replacement for open(). This probably means that there is still a lot of code being written that uses codecs.open(). That's bad thing because of codecs.open()'s lack of newline conversion. A lot of that code will
- have compatibility issues when it is moved between unix and windows
- silently break text files on windows, leading to issues further downstream (confusing other tools, messing up revision control histories)
The problem has been fixed with io.open() in 2.x and open() in 3.x. Unfortunately the 2.7 unicode HOWTO still recommends the use of codecs.open(). The 2.7 and the 3.x documentation of codecs.open() doesn't refer the reader to better alternatives.
The attached patches fix that.
The only downside I see is that newly written code that uses the better alternatives would be incompatible with 2.5 and older. However croaking on a small minority of systems is better than silently disrupting workflows, causing platform incompatibilities, and inviting flaky workarounds.
The 2.7 patch makes the unicode HOWTO recommend io.open() instead of codecs.open(). Both patches change the codecs.open() documentation to refer to io.open() or (on 3.x) open().
Additionally I removed the "data loss" explanation from codecs.open()'s note about its lack of newline conversion. It is not particularly helpful information and it is not entirely correct (data loss could also have been avoided by doing newline conversion before encoding and after decoding)
----------
assignee: docs@python
components: Documentation
files: codecsopen2.patch
keywords: patch
messages: 224632
nosy: Frank.van.Dijk, docs@python
priority: normal
severity: normal
status: open
title: patch: steer people away from codecs.open
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file36234/codecsopen2.patch
_______________________________________
Python tracker <report(a)bugs.python.org>
<http://bugs.python.org/issue22128>
_______________________________________
New submission from Ram Rachum <ram(a)rachum.com>:
Looking at the enum source code, there's a method `_create_pseudo_member_` that's used in a bunch of places. Its docstring says "Create a composite member iff value contains only members", which would have been useful if I had any idea what "composite member" meant.
It would be good if the documentation for the enum module would include more information about these two concepts.
----------
assignee: docs@python
components: Documentation
messages: 364561
nosy: cool-RR, docs@python
priority: normal
severity: normal
status: open
title: enum: Add documentation for _create_pseudo_member_ and composite members
type: enhancement
versions: Python 3.8, Python 3.9
_______________________________________
Python tracker <report(a)bugs.python.org>
<https://bugs.python.org/issue40006>
_______________________________________
New submission from Brett Cannon <brett(a)python.org>:
If you look at https://docs.python.org/3.9/py-modindex.html#cap-i you will see that importlib.metadata isn't listed (same goes for the 3.8 docs).
Or are you leaving it out due to it being provisional?
----------
assignee: docs@python
components: Documentation
messages: 348872
nosy: barry, brett.cannon, docs@python, jaraco
priority: normal
severity: normal
status: open
title: importlib.metadata docs not showing up in the module index
versions: Python 3.8, Python 3.9
_______________________________________
Python tracker <report(a)bugs.python.org>
<https://bugs.python.org/issue37741>
_______________________________________
New submission from Vedran Čačić:
Look at this:
>>> from collections.abc import Sequence
>>> help(Sequence.index)
index(self, value, start=0, stop=None)
S.index(value, [start, [stop]]) -> integer -- return first index of value.
Raises ValueError if the value is not present.
>>> issubclass(range, Sequence)
True
>>> help(range.index)
index(...)
rangeobject.index(value, [start, [stop]]) -> integer -- return index of value.
Raise ValueError if the value is not present.
So far, so good. But:
>>> range(9).index(2, 1, 5)
TypeError: index() takes exactly one argument (3 given)
Of course it's not essential, but the docs shouldn't lie. And if range _is_ a Sequence, then it should have the complete interface of a Sequence. Including start and end arguments for .index: they are optional from the point of call, not from the point of implementation. :-)
----------
assignee: docs@python
components: Documentation, Library (Lib)
messages: 276908
nosy: docs@python, veky
priority: normal
severity: normal
status: open
title: range.index mismatch with documentation
type: behavior
versions: Python 3.5, Python 3.6
_______________________________________
Python tracker <report(a)bugs.python.org>
<http://bugs.python.org/issue28197>
_______________________________________
New submission from Terry J. Reedy:
"execute(sql[, parameters])
Executes an SQL statement. The SQL statement may be parametrized (i. e. placeholders instead of SQL literals). The sqlite3 module supports two kinds of placeholders: question marks (qmark style) and named placeholders (named style)."
Experimental facts based on experiments with the code example in the doc, using 3.4.b2: 'parameters' is a single subscriptable collection parameter, sequence or dict, that might be called seq_dict. It is positional only, so whatever name is used is a dummy. Only one placeholder style can be used in a given SQL statement string. If question marks are used, seq_dict must be a sequence. If names are used, seq_dict can be either a sequence or dict or subclass thereof. A UserDict is treated as a sequence and raises KeyError(0).
Possible text that encompasses the above, replacing the last sentence:
"A statement may use one of two kinds of placeholders: question marks (qmark style) or named placeholders (named style). For qmark style, seq_dict must be a sequence. For named style, it can be either a sequence or dict instance. Len(seq_dict) must match the number of placeholders."
After cleaning up the test file, I will verify on 2.7 and upload.
----------
assignee: docs@python
components: Documentation, Library (Lib)
messages: 208908
nosy: docs@python, terry.reedy
priority: normal
severity: normal
stage: patch review
status: open
title: Rename & explain sqlite3.Cursor.execute 'parameters' param
type: behavior
versions: Python 2.7, Python 3.3, Python 3.4
_______________________________________
Python tracker <report(a)bugs.python.org>
<http://bugs.python.org/issue20364>
_______________________________________
New submission from igo95862 <igo9586(a)gmail.com>:
This is package documentation: https://docs.python.org/3/tutorial/modules.html#packages
To make package executable (python -m package) you need to create a file __main__.py in the package directory.
This is pretty much not documented anyone aside of trying to run a package missing __main__.py
This page already contains information on how to make module executable. (https://docs.python.org/3/tutorial/modules.html#executing-modules-as-scripts)
----------
assignee: docs@python
components: Documentation
messages: 362790
nosy: docs@python, igo95862
priority: normal
severity: normal
status: open
title: Missing documentation on how to make package executable as script
type: enhancement
versions: Python 3.8
_______________________________________
Python tracker <report(a)bugs.python.org>
<https://bugs.python.org/issue39774>
_______________________________________
New submission from Nguyễn Gia Phong <vn.mcsinyx(a)gmail.com>:
Dear Maintainer,
I want to request a feature on the generative documentation of type-hinting.
As of December 2019, I believe there is no support for generating such
information in help(). For demonstration, I have this tiny piece of code
class Foo:
@property
def bar(self) -> int: return 42
@bar.setter
def bar(self, value: int) -> None: pass
def baz(self, arg: float) -> str: pass
whose documentation on CPython 3.7.5 (on Debian testing amd64 if that matters)
is generated as
class Foo(builtins.object)
| Methods defined here:
|
| baz(self, arg: float) -> str
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)
|
| bar
I expect the documentation for bar to be as informative as bar, i.e. something
similar to ``bar: int''. As pointed out by ChrisWarrick on freenode#python,
the annotations are already present, yet help() is not making use of them:
>>> Foo.bar.fget.__annotations__
{'return': <class 'int'>}
>>> Foo.bar.fset.__annotations__
{'value': <class 'int'>, 'return': None}
Have a Merry Christmas or other holiday of your choice,
Nguyễn Gia Phong
----------
assignee: docs@python
components: Documentation
messages: 358823
nosy: McSinyx, docs@python
priority: normal
severity: normal
status: open
title: Type signature of @property not shown in help()
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9
_______________________________________
Python tracker <report(a)bugs.python.org>
<https://bugs.python.org/issue39125>
_______________________________________
New submission from Roman Joost <roman(a)bromeco.de>:
When running a process which changes UID/GID, some of the following processes will run as the user I change to per process.
In order to reproduce (see the attached reproducer):
1. Change the 'USERNAME' to an unprivileged user on your system.
2. Run the reproducer as a user with elevated privileges (e.g. root or some secondary user you have on your system). Mind you, I don't think the user you run as needs elevated privileges, but that's the user I ran as when I observed this behaviour.
3. The reproducer iterates over a list (It stems from a test function which was checking permissions on log files). Observe the print out, which prints the process' GID, UID and secondary groups before we're changing to the users GID, UID and secondary groups.
4. You should observe that at some point the process prints the user information of the user we want to change to not the one which initially started the script.
Example output when running locally as root:
('B', (0, 0, [0]))
('A', (0, 0, [0]))
('C', (0, 0, [0]))
('E', (0, 0, [0]))
('D', (0, 0, [0]))
('F', (1002, 1002, [10, 135, 1000, 1002]))
('H', (1002, 1002, [10, 135, 1000, 1002]))
('I', (1002, 1002, [10, 135, 1000, 1002]))
('J', (1002, 1002, [10, 135, 1000, 1002]))
('G', (1002, 1002, [10, 135, 1000, 1002]))
('K', (1002, 1002, [10, 135, 1000, 1002]))
('L', (1002, 1002, [10, 135, 1000, 1002]))
('M', (1002, 1002, [10, 135, 1000, 1002]))
('N', (1002, 1002, [10, 135, 1000, 1002]))
I would have expected `0` all the way through.
However, if I initialise the Pool with `maxtasksperchild=1` the isolation seems as expected.
I don't know whether this is a bug or I'm foolish to invoke multiprocessing like this. I've run out of time to investigate this further. It's certainly strange behaviour to me and I thought I better report it, since reproducing seems fairly deterministic.
----------
assignee: docs@python
components: Documentation, Library (Lib)
files: reproducer.py
messages: 357773
nosy: docs@python, romanofski
priority: normal
severity: normal
status: open
title: multiprocessing processes seem to "bleed" user information (GID/UID/groups)
type: behavior
versions: Python 3.6, Python 3.7
Added file: https://bugs.python.org/file48753/reproducer.py
_______________________________________
Python tracker <report(a)bugs.python.org>
<https://bugs.python.org/issue38963>
_______________________________________