Mailman 3 June 2015 - docs

[issue20344] subprocess.check_output() docs misrepresent what shell=True does
by Tobias Klausmann June 19, 2015

June 19, 2015

New submission from Tobias Klausmann: The subprocess docs state that the first argument can be either a string or an iterable that contains the program and arguments to run. It also points out that using shell=True allows for shell constructs. It does not mention a subtlety that is introduced by Python's use of sh -c (on Unix): Using a string that contains the full command line with args and shell=False breaks as expected: >>> subprocess.check_output("ls foo", shell=False) Traceback … [View More](most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python3.3/subprocess.py", line 576, in check_output with Popen(*popenargs, stdout=PIPE, **kwargs) as process: File "/usr/lib64/python3.3/subprocess.py", line 824, in __init__ restore_signals, start_new_session) File "/usr/lib64/python3.3/subprocess.py", line 1448, in _execute_child raise child_exception_type(errno_num, err_msg) FileNotFoundError: [Errno 2] No such file or directory: 'ls foo' Using shell=True instead works as expected (since foo does not exist, ls exits with non-0 and a different Exception is raised. This, too is to spec and exactly what the docs describe. >>> subprocess.check_output("ls foo", shell=True) ls: cannot access foo: No such file or directory Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python3.3/subprocess.py", line 589, in check_output raise CalledProcessError(retcode, process.args, output=output) subprocess.CalledProcessError: Command 'ls foo' returned non-zero exit status 2 Using shell=False and making the command a list of command and args does the same thing: >>> subprocess.check_output(["ls", "foo"], shell=False) ls: cannot access foo: No such file or directory Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python3.3/subprocess.py", line 589, in check_output raise CalledProcessError(retcode, process.args, output=output) subprocess.CalledProcessError: Command '['ls', 'foo']' returned non-zero exit status 2 But using an iterable _and_ requesting a shell results in something very broken: >>> subprocess.check_output(["ls", "foo"], shell=True) [contents of my homedir are returned] >>> Note that the argument "foo" completely disappears, apparently. strace() reveals that "foo" is never added to the call to "ls". Instead, it becomes $0 in the shell that ls runs in. This is exactly to spec (i.e. bash is not broken). "bash -c foo bar baz" will start a shell that sets $0 to "bar", $1 to "baz" and then executes "foo". Whereas "bash -c 'foo bar baz'" will run 'foo bar baz'. I think there are three possible fixes here: 1- Make check_output(list, shell=True) run something like "bash -c '%s'" % " ".join(list) 2- Change the docs to warn of the unintended consequences of combining lists with shell=True 3- Make check_output() throw an Exception if the first argument is a list and shell=True The first option would make it easiest for people to "just use it", but I fear the string manipulation may become the source of bugs with security implications in the future. The second option keeps the status quo, but people tend to not read docs, so it may not be as effective as one would like, especially since shell-True already has warnings and people tend to go "yeah I know about unverified input, but it's not a problem for me" and then ignore both warnings. Option 3 has the disadvantage that existing scripts that _expect_ the behavior may break. ---------- assignee: docs@python components: Documentation, Library (Lib) messages: 208811 nosy: docs@python, klausman priority: normal severity: normal status: open title: subprocess.check_output() docs misrepresent what shell=True does type: behavior versions: Python 3.3 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue20344> _______________________________________ [View Less]

4 11

[issue24460] urlencode() of dictionary not as expected
by Martin Panter June 19, 2015

June 19, 2015

Martin Panter added the comment: 1. urlencode(): I agree the documentation is unclear. But David Rueter’s suggestion does not help much. I think doseq=True is meant to also work for a mapping query (as in original post), and is not required in the sequence-of-tuples mode if each tuple has a single parameter value. Perhaps something like this could work instead: “When a sequence of two-element tuples is used as *query*, the first element of each tuple is a key and the second specifies one or … [View More]

1 0

Incorrect (misleading) statement in the execution model documentation (issue 24129)
by ericsnowcurrently＠gmail.com June 19, 2015

June 19, 2015

http://bugs.python.org/review/24129/diff/14851/Doc/reference/executionmodel… File Doc/reference/executionmodel.rst (right): http://bugs.python.org/review/24129/diff/14851/Doc/reference/executionmodel… Doc/reference/executionmodel.rst:148: Free variables follow the normal rules for name resolution, It may be more clear to say: These references follow the normal rules for name resolution with one exception. Unbound local variables are looked up in the global namespace. ... http://bugs.python.… [View More]

1 0

[issue24466] extend_path explanation in documentation is ambiguous
by alanf June 18, 2015

June 18, 2015

New submission from alanf: The description of pkgutil.extend_path in the doc (e.g., https://docs.python.org/2/library/pkgutil.html , https://docs.python.org/3/library/pkgutil.html ) is so ambiguous that I had to run a test to understand its behavior. The doc says: This will add to the package’s __path__ all subdirectories of directories on sys.path named after the package. It wasn't clear to me how this should be parsed. Using parentheses to group the clauses, there's this possibility : (1)… [View More]

1 0

make_server host parameter in underdocumented
by anatoly techtonik June 18, 2015

June 18, 2015

Hi, In https://docs.python.org/2/library/wsgiref.html#wsgiref.simple_server.make_s… example mentions host='', but doesn't clarify if that will make server listen on localhost or on all interfaces. -- anatoly t.

1 0

[issue23020] New matmul operator crashes modules compiled with CPython3.4
by Tal Einat June 18, 2015

June 18, 2015

Changes by Tal Einat <taleinat(a)gmail.com>: ---------- assignee: -> docs@python components: +Documentation nosy: +docs@python _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue23020> _______________________________________

1 0

Python doc link broken
by Adrien Pouligny June 17, 2015

June 17, 2015

Hi, The links to download the documentation are broken and return a 404 error. Thanks, Adrien

1 0

(correction)Bug in ETree
by Noonan June 17, 2015

June 17, 2015

Sorry my last email sent prematurely, I'm not sure how. I was using a hotkey for pasing when it accidently sent. I'm running Python version: 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:16:31) [MSC v.1600 64 bit (AMD64)] When I do a .findall or .find on an Element object of an ElementTree from xml.etree, xpath queries (of the format './/<tagname>') aren't letting me retrieve children that aren't direct descendants of the Element object. Also, the documentation doesn't clearly show/when … [View More]

2 3

[issue24460] urlencode() of dictionary not as expected
by David Rueter June 17, 2015

June 17, 2015

David Rueter added the comment: Ah hah! Indeed, urlencode() does work on dictionaries as expected when doseq=True. Thank you for clarifying. FWIW I had read the documentation and the referenced examples multiple times. I would like to make a few documentation suggestions for clarity. 1 ) Update https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlencode Where documentation currently says: "When a sequence of two-element tuples is used as the query argument, the first element … [View More]of each tuple is a key and the second is a value. The value element in itself can be a sequence and in that case, if the optional parameter doseq is evaluates to True, individual key=value pairs separated by '&' are generated for each element of the value sequence for the key. The order of parameters in the encoded string will match the order of parameter tuples in the sequence." Perhaps instead the following would be more clear: "The query argument may be a sequence of two-element tuples where the first element of each tuple is a key and the second is a value. However the optional parameter doseq must then be set to True in order to reliably generate individual key=value pairs separated by '&' for each element of the value sequence for the key, and to preserve the sequence of the elements in the query parameter." 2) Update https://docs.python.org/3/library/urllib.request.html#urllib-examples The examples are referenced from the documentation: "Refer to urllib examples to find out how urlencode method can be used for generating query string for a URL or data for POST." However the example page provides contradictory information and examples for this specific use case. Currently the examples page says: "The urllib.parse.urlencode() function takes a mapping or sequence of 2-tuples and returns a string in this format. It should be encoded to bytes before being used as the data parameter. The charset parameter in Content-Type header may be used to specify the encoding. If charset parameter is not sent with the Content-Type header, the server following the HTTP 1.1 recommendation may assume that the data is encoded in ISO-8859-1 encoding. It is advisable to use charset parameter with encoding used in Content-Type header with the Request." Perhaps instead the following would be more clear: "The urllib.parse.urlencode() query parameter can accept a mapping or sequence of 2-tuples and return a string in this format if the optional parameter doseq is set to True. It should be encoded to bytes before being used as the query parameter. The charset parameter in Content-Type header may be used to specify the encoding. If charset parameter is not sent with the Content-Type header, the server following the HTTP 1.1 recommendation may assume that the data is encoded in ISO-8859-1 encoding. It is advisable to use charset parameter with encoding used in Content-Type header with the Request." 3) Also on the example page, there are examples of urlencode operating on dictionaries where doseq is not provided. This is confusing. It would be better to show doseq = True: Here is an example session that uses the GET method to retrieve a URL containing parameters: ... >>> data = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0}) ... The following example uses the POST method instead. Note that params output from urlencode is encoded to bytes before it is sent to urlopen as data: ... >>> data = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0}) I suggest that these examples read: >>> data = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0}, doseq=true) ---------- _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue24460> _______________________________________ [View Less]

1 0

[issue24460] urlencode() of dictionary not as expected
by R. David Murray June 17, 2015

June 17, 2015

R. David Murray added the comment: That behavior is complex enough that I think it would be worth adding an example of it to the examples section (and maybe linking directly from the doseq explanation to that specific example). ---------- assignee: -> docs@python components: +Documentation nosy: +docs@python, r.david.murray versions: +Python 3.5, Python 3.6 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue24460> _______________________________________

1 0