[issue40932] subprocess docs don't qualify the instruction to use shlex.quote by OS
New submission from Stephen Farris stephen.farris@jhuapl.edu:
The subprocess docs state: "When using shell=True, the shlex.quote() function can be used to properly escape whitespace and shell metacharacters in strings that are going to be used to construct shell commands." While this is true on Unix, it is not true on Windows. On Windows it is easy to create scenarios where shell injection still exists despite using shlex.quote properly (e.g. subprocess.run(shlex.quote("'&calc '"), shell=True) launches the Windows calculator, which it wouldn't do if shlex.quote was able to prevent shell injection on Windows). While the shlex docs state that shlex is for Unix, the subprocess docs imply that shlex.quote will work on Windows too, possibly leading some developers to erroneously use shlex.quote on Windows to try to prevent shell injection. Recommend: 1) qualifying the above section in the subprocess docs to make it clear that this only works on Unix, and 2) updating the shlex docs with warnings that shlex.quote in particular is not for use on Windows.
---------- assignee: docs@python components: Documentation messages: 371140 nosy: Stephen Farris, docs@python priority: normal severity: normal status: open title: subprocess docs don't qualify the instruction to use shlex.quote by OS versions: Python 3.8
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue40932 _______________________________________
Change by Stephen Farris stephen.farris@jhuapl.edu:
---------- type: -> security
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue40932 _______________________________________
Change by Ned Deily nad@python.org:
---------- components: +Windows nosy: +paul.moore, steve.dower, tim.golden, zach.ware title: subprocess docs don't qualify the instruction to use shlex.quote by OS -> subprocess docs should warn of shlex use on Windows versions: +Python 3.10, Python 3.9
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue40932 _______________________________________
Change by Chris Jerdonek chris.jerdonek@gmail.com:
---------- nosy: +chris.jerdonek
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue40932 _______________________________________
Change by Ammar Askar ammar@ammaraskar.com:
---------- keywords: +patch nosy: +ammar2 nosy_count: 7.0 -> 8.0 pull_requests: +20643 stage: -> patch review pull_request: https://github.com/python/cpython/pull/21502
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue40932 _______________________________________
Steve Dower steve.dower@python.org added the comment:
I wonder whether we should be more specific about the shells that shlex works for? Since WSL makes *sh (Bash, Dash, Sh, etc.) easily available on Windows, and I believe PowerShell on Linux keeps its own quoting rules.
----------
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue40932 _______________________________________
Ammar Askar ammar@ammaraskar.com added the comment:
Hmm, it'd be hard to enumerate them all. The module does say, "...simple syntaxes resembling that of the Unix shell" but that's it.
Distinguishing at the OS level for shlex does seem a bit weird given the existence of WSL and non-compliant shells on Linux like xonsh. I think it'd be nice if we could be a bit more specific on whats supported, maybe it covers all POSIX compliant shells?
For the subprocess warning I think it's fine to talk about the OS since it looks like the shell used are hard-coded in:
* https://github.com/python/cpython/blob/5241e189e77972d3a07acbbb3f0c0cbc2aeeb... * https://github.com/python/cpython/blob/5241e189e77972d3a07acbbb3f0c0cbc2aeeb...
----------
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue40932 _______________________________________
miss-islington mariatta.wijaya+miss-islington@gmail.com added the comment:
New changeset f9a8386e44a695551a1e54e709969e90e9b96bc4 by Ammar Askar in branch 'master': bpo-40932: Note security caveat of shlex.quote on Windows (GH-21502) https://github.com/python/cpython/commit/f9a8386e44a695551a1e54e709969e90e9b...
---------- nosy: +miss-islington
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue40932 _______________________________________
Ammar Askar ammar@ammaraskar.com added the comment:
Thank you Steve and Zachary for reviewing, this warning is in the docs now.
---------- resolution: -> fixed stage: patch review -> resolved status: open -> closed
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue40932 _______________________________________
participants (6)
-
Ammar Askar -
Chris Jerdonek
-
miss-islington
-
Ned Deily
-
Stephen Farris
-
Steve Dower