Archives (.tar or .zip) with absolute paths
Hi, I noticed that "python3 -m tarfile -x archive.tar" uses absolute paths by default, whereas the UNIX tar command doesn't by default. The UNIX tar command requires to add explicitly --absolute-paths (-P) option. The tarfile and zipfile modules (maybe also some others, I didn't check) contain warnings absolute paths and paths containing "..". Why not ignoring "/" at start of filenames *by default*? By backward compatibility? I suggest to add a boolean absolute_path option to tarfile and zipfile and disable it by default in the CLI. The question is what should be the default value for the Python API. I suggest to use absolute_path=False by default for safety. Example to create such archive. See that tar also removes "/" by default and requires to pass explicitly -P: $ cd $HOME # /home/haypo $ echo TEST > test $ tar -cf test.tar /home/haypo/test tar: Removing leading `/' from member names $ rm -f test.tar $ tar -P -cf test.tar /home/haypo/test $ rm -f test Extracting such archive using tar is safe *by default*: $ mkdir z $ cd z $ tar -xf ~/test.tar tar: Removing leading `/' from member names $ find . ./home ./home/haypo ./home/haypo/test Extracting such archive using Python is unsafe: $ python3 -m tarfile -e ~/test.tar $ cat ~/test TEST $ pwd /home/haypo/z Python creates files outside the current directory which is unsafe, wheras tar doesn't. Victor
CWE-22: Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal') https://cwe.mitre.org/top25/#CWE-22 http://cwe.mitre.org/data/definitions/22.html - [ ] BUG,SEC: -P/--absolute-names *13* *CWE-22 <http://cwe.mitre.org/data/definitions/22.html>: Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal')* CWE CATEGORY 21: Pathname Traversal and Equivalence Errors https://cwe.mitre.org/data/definitions/21.html https://cwe.mitre.org/data/definitions/21.html#Relationships ... CWE 59: "Improper Link Resolution Before File Access ('Link Following')" https://cwe.mitre.org/data/definitions/59.html - [ ] ? BUG,SEC: symlinks? Docs: https://docs.python.org/3/library/tarfile.html - https://github.com/python/cpython/blob/2.6/Doc/library/tarfile.rst - https://github.com/python/cpython/blob/3.6/Doc/library/tarfile.rst - https://github.com/python/cpython/blob/master/Doc/library/tarfile.rst Src: https://github.com/python/cpython/blob/master/Lib/tarfile.py - https://github.com/python/cpython/blob/2.6/Lib/tarfile.py - https://github.com/python/cpython/blob/3.6/Lib/tarfile.py https://www.python.org/news/security/#reporting-security-issues-in-python - https://docs.python.org/devguide/documenting.html#security-considerations-an... - https://cpython-devguide.readthedocs.io/documenting.html#security-considerat... - [o] email to: security@python.org - [ ] Create an issue: https://bugs.python.org/ - [ ] Create a pull request: https://docs.python.org/devguide/pullrequest.html - [ ] DOC: - [ ] BUG,ENH,SEC: `python -m tarfile -P/--absolute-names` - [ ] BUG,ENH,SEC: `python -m tarfile -h/--dereference --hard-dereference` - [ ] bandit test(s): https://github.com/openstack/bandit#writing-tests - [ ] Python API - [ ] - [ ] python -m tarfile - [ ] someone to lead on this because I am actually looking for a job ... https://python-security.readthedocs.io/ - Src: https://github.com/haypo/python-security/blob/master/index.rst - [ ] https://github.com/ebranca/owasp-pysec/wiki - [ ] https://github.com/ebranca/owasp-pysec/wiki/Security-Concerns-in-modules-and... On Thu, Mar 9, 2017 at 11:37 AM, Victor Stinner <victor.stinner@gmail.com> wrote:
Hi,
I noticed that "python3 -m tarfile -x archive.tar" uses absolute paths by default, whereas the UNIX tar command doesn't by default. The UNIX tar command requires to add explicitly --absolute-paths (-P) option.
The tarfile and zipfile modules (maybe also some others, I didn't check) contain warnings absolute paths and paths containing "..".
Why not ignoring "/" at start of filenames *by default*? By backward compatibility?
I suggest to add a boolean absolute_path option to tarfile and zipfile and disable it by default in the CLI. The question is what should be the default value for the Python API. I suggest to use absolute_path=False by default for safety.
Example to create such archive. See that tar also removes "/" by default and requires to pass explicitly -P:
$ cd $HOME # /home/haypo $ echo TEST > test $ tar -cf test.tar /home/haypo/test tar: Removing leading `/' from member names
$ rm -f test.tar $ tar -P -cf test.tar /home/haypo/test $ rm -f test
Extracting such archive using tar is safe *by default*:
$ mkdir z $ cd z $ tar -xf ~/test.tar tar: Removing leading `/' from member names $ find . ./home ./home/haypo ./home/haypo/test
Extracting such archive using Python is unsafe:
$ python3 -m tarfile -e ~/test.tar $ cat ~/test TEST $ pwd /home/haypo/z
Python creates files outside the current directory which is unsafe, wheras tar doesn't.
Victor _______________________________________________ Security-SIG mailing list Security-SIG@python.org https://mail.python.org/mailman/listinfo/security-sig
Hi, I'm sorry Wes, but I don't understand your long list of urls :-( Can you elaborate? I'm asking if there is a reason for allowing absolute paths by default. Maybe backward compatibility? 2017-03-09 20:33 GMT+01:00 Wes Turner <wes.turner@gmail.com>:
I didn't write a private email to security@ because as you pointed, the issue is known and *documented* in Python since 10 years.
I wrote this doc :-) I just added notes about tarfile and zipfile. Victor
On Thursday, March 9, 2017, Victor Stinner <victor.stinner@gmail.com> wrote:
Hi,
I'm sorry Wes, but I don't understand your long list of urls :-( Can you elaborate?
I thought that's what I was doing?
I'm asking if there is a reason for allowing absolute paths by default. Maybe backward compatibility?
I think secure by default would be good here.
2017-03-09 20:33 GMT+01:00 Wes Turner <wes.turner@gmail.com <javascript:;>
I didn't write a private email to security@ because as you pointed, the issue is known and *documented* in Python since 10 years.
Doesn't mean it's not broken
I wrote this doc :-) I just added notes about tarfile and zipfile.
The [ ] wiki links could also be useful
Victor
Le 10 mars 2017 1:52 AM, "Wes Turner" <wes.turner@gmail.com> a écrit :
https://python-security.readthedocs.io/
I wrote this doc :-) I just added notes about tarfile and zipfile.
The [ ] wiki links could also be useful My long term plan is to not maintain this doc alone. Would you mind to propose a pull request directly? Victor
On Thu, Mar 9, 2017 at 7:03 PM, Victor Stinner <victor.stinner@gmail.com> wrote:
Le 10 mars 2017 1:52 AM, "Wes Turner" <wes.turner@gmail.com> a écrit :
https://python-security.readthedocs.io/
I wrote this doc :-) I just added notes about tarfile and zipfile.
The [ ] wiki links could also be useful
My long term plan is to not maintain this doc alone. Would you mind to propose a pull request directly?
- [ ] Add (nested?) links to the given wiki pages to python-security.rtfd.org - [ ] Would this (see: thread) info be useful for the docs or the devguide? - [ ] Would this (or a similar) checklist be useful for the security guidelines?
Victor
On Thu, Mar 9, 2017 at 9:37 AM, Victor Stinner <victor.stinner@gmail.com> wrote:
Hi,
I noticed that "python3 -m tarfile -x archive.tar" uses absolute paths by default, whereas the UNIX tar command doesn't by default. The UNIX tar command requires to add explicitly --absolute-paths (-P) option.
The tarfile and zipfile modules (maybe also some others, I didn't check) contain warnings absolute paths and paths containing "..".
Why not ignoring "/" at start of filenames *by default*? By backward compatibility?
I suggest to add a boolean absolute_path option to tarfile and zipfile and disable it by default in the CLI. The question is what should be the default value for the Python API. I suggest to use absolute_path=False by default for safety.
This sounds like the right default to me. Technically there's some backwards compatibility risk, but tarfiles and zipfiles with absolute paths are really rare and this behavior is really dangerous (unpacking a file gives the person who created the file the ability to overwrite arbitrary files on your computer), so it's unlikely that much will break and what does break was likely a huge security hole in the first place. -n -- Nathaniel J. Smith -- https://vorpus.org
I opened two public bug reports: tarfile: http://bugs.python.org/issue29788 zipfile: http://bugs.python.org/issue29789 It's unclear to me if it's ok or not to backport the new absolute_path option to stable Python versions, to fix the vulnerability? Victor
participants (3)
-
Nathaniel Smith
-
Victor Stinner
-
Wes Turner