Re: [Python-Dev] Remove tempfile.mktemp()
Nathaniel J. Smith:
Historically, mktemp variants have caused *tons* of serious security vulnerabilities. It's not a theoretical issue.
All the more reason to have a standard library function that gets it right.
The choice of ENTROPY_BYTES is an interesting question. 16 (= 128 bits) would be a nice "obviously safe" number, and gives 22-byte filenames. We might be able to get away with fewer, if we had a plausible cost model for the attack. This is another point where a security specialist might be helpful :-).
I'm not a security specialist but I play one on TV. Here's my take on it. Any kind of brute force attack will require at least one syscall per try, to create a file or check if a file by a given name exists. It's a safe assumption that names have to be tried individually, because if the attacker has a faster way of enumerating existing file names, then the entropy of the filename is worthless anyway. That means even with only 41 bits of entry, the attacker will have make 2^40 tries on average. For an individual short-lived file, that could be enough; even with a billion syscalls per second, that's over a thousand seconds, leaving plenty of time to initiate whatever writes the file. However, there could be applications where the window of attack is very long, hours or days even, or that are constantly writing new temporary files, and where the attacker can keep trying at a rapid pace, and then 41 bits is definitely not secure. 128 bits seems like overkill: There's no birthday attack because no-one keeps 2^(ENTROPY_BITS/2) files around, and the attack is running on the attackee's system, so there's no using specialised accelerator hardware. I'd say 64 bits is enough under those circumstances, but I wouldn't be surprised if a better security specialist could make a case for more. So maybe go with 80 bits, that's puts it at 15 or 16 characters. Med venlig hilsen/Best regards Anders Munch Chief Security Architect T: +45 76266981 * M: +45 51856626 ajm@flonidan.dk * www.flonidan.com FLONIDAN A/S * Islandsvej 29 * DK-8700 Horsens * CVR: 89919916 Winner of the 2018 Frost & Sullivan Customer Leadership Award
Hi,
I'm not really convinced that mktemp() should be made "more secure".
To be clear: mktemp() is vulnerable by design. It's not a matter of
entropy. You can watch the /tmp directory using inotify and "discover"
immediately the "secret" filename, it doesn't depend on the amount of
entropy used to generate the filename. A function is either unsafe or
secure.
Why mktemp() only uses 8 characters? Well, I guess that humans like to
be able to copy manually (type) a filename :-)
Note: For the ones who didn't notice, "mktemp()" name comes from a
function with the same name in the libc.
http://man7.org/linux/man-pages/man3/mktemp.3.html
Victor
Le mer. 20 mars 2019 à 12:29, Anders Munch
Nathaniel J. Smith:
Historically, mktemp variants have caused *tons* of serious security vulnerabilities. It's not a theoretical issue.
All the more reason to have a standard library function that gets it right.
The choice of ENTROPY_BYTES is an interesting question. 16 (= 128 bits) would be a nice "obviously safe" number, and gives 22-byte filenames. We might be able to get away with fewer, if we had a plausible cost model for the attack. This is another point where a security specialist might be helpful :-).
I'm not a security specialist but I play one on TV. Here's my take on it.
Any kind of brute force attack will require at least one syscall per try, to create a file or check if a file by a given name exists. It's a safe assumption that names have to be tried individually, because if the attacker has a faster way of enumerating existing file names, then the entropy of the filename is worthless anyway.
That means even with only 41 bits of entry, the attacker will have make 2^40 tries on average. For an individual short-lived file, that could be enough; even with a billion syscalls per second, that's over a thousand seconds, leaving plenty of time to initiate whatever writes the file.
However, there could be applications where the window of attack is very long, hours or days even, or that are constantly writing new temporary files, and where the attacker can keep trying at a rapid pace, and then 41 bits is definitely not secure.
128 bits seems like overkill: There's no birthday attack because no-one keeps 2^(ENTROPY_BITS/2) files around, and the attack is running on the attackee's system, so there's no using specialised accelerator hardware. I'd say 64 bits is enough under those circumstances, but I wouldn't be surprised if a better security specialist could make a case for more. So maybe go with 80 bits, that's puts it at 15 or 16 characters.
Med venlig hilsen/Best regards
Anders Munch Chief Security Architect
T: +45 76266981 * M: +45 51856626 ajm@flonidan.dk * www.flonidan.com FLONIDAN A/S * Islandsvej 29 * DK-8700 Horsens * CVR: 89919916 Winner of the 2018 Frost & Sullivan Customer Leadership Award _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com
-- Night gathers, and now my watch begins. It shall not end until my death.
On 2019-03-20 12:45, Victor Stinner wrote:
You can watch the /tmp directory using inotify and "discover" immediately the "secret" filename, it doesn't depend on the amount of entropy used to generate the filename.
That's not the problem. The security issue here is guessing the filename *before* it's created and putting a different file or symlink in place. So I actually do think that mktemp() could be made secure by using a longer name generated by a secure random generator.
On 20Mar2019 12:53, Jeroen Demeyer
On 2019-03-20 12:45, Victor Stinner wrote:
You can watch the /tmp directory using inotify and "discover" immediately the "secret" filename, it doesn't depend on the amount of entropy used to generate the filename.
That's not the problem. The security issue here is guessing the filename *before* it's created and putting a different file or symlink in place.
So I actually do think that mktemp() could be made secure by using a longer name generated by a secure random generator.
I know it is days later, but to add a little nuance: the security issue
is guessing the filename before it is _used_. Consider:
path = tempfile.mktemp()
with open(path, "w"):
write some secret stuff ...
call_other_function(path)
If an attacker gets in _after_ the open (which creates the file) by
using something like inotify to _observe_ the pathname instead of
guessing and supplants the file then, call_other_function is then
subverted.
Also, the common examples are attackers who are not the user making the
tempfile, in which case the _default_ mktemp is sort of secure with the
above because it gets made in /tmp which on a modern POSIX system
prevents _other_ uses from removing/renaming a file. (And Eryk I think
described the Windows situation which is similarly protected).
However, mktemp somewhere else is not so protected.
And the attacker might be malware running as the orignal user (yes the
game may already be overin that case for other reasons).
However, I wanted to make the point that the security issue isn't around
creation but use - trusting the mktemp pathname to be the same state as
it was earlier.
Cheers,
Cameron Simpson
On 3/23/19, Cameron Simpson
Also, the common examples are attackers who are not the user making the tempfile, in which case the _default_ mktemp is sort of secure with the above because it gets made in /tmp which on a modern POSIX system prevents _other_ uses from removing/renaming a file. (And Eryk I think described the Windows situation which is similarly protected).
Using NamedTemporaryFile(delete=False) or mkstemp() ensures that the file is created and opened securely. in contrast, the filename from mktemp() might be used naively in POSIX, such as open(path, "w"). This file might grant read access to everyone depending on the file-mode creation mask (umask). Also, since it neglects to use exclusive mode ("x"), it might open an existing file that grants read-write permission to the world, or maybe it's a symlink. By default, even naive use of the mktemp() name in Windows remains secure, since every user has a separate temp directory that's only accessible by privileged users such as SYSTEM, Administrators, and Backup Operators (with SeBackupPrivilege and SeRestorePrivilege enabled). The primary issue with a short name is an accidental name collision with another program that's not as careful as Python's tempfile. Using a longer name decreases the chance of this to practically nothing.
On Wed, Mar 20, 2019 at 12:45:40PM +0100, Victor Stinner wrote:
Hi,
I'm not really convinced that mktemp() should be made "more secure". To be clear: mktemp() is vulnerable by design. It's not a matter of entropy. You can watch the /tmp directory using inotify and "discover" immediately the "secret" filename, it doesn't depend on the amount of entropy used to generate the filename. A function is either unsafe or secure.
Security is not a binary state, it is never either-or "unsafe" or "secure". Secure against what attacks? Unsafe under what circumstances? I can use the unsafe mktemp on a stand alone single-user computer, disconnected from the internet, guaranteed to have nothing but trusted software, and it will be secure in practice. Or I can use the "safe interfaces" and I'm still vulnerable to an Advanced Persistent Threat that has compromised the OS specifically to target my application. If the attacker controls the OS or the hardware, then effectively they've already won. -- Steven
On Wed, Mar 20, 2019 at 11:25:03AM +0000, Anders Munch wrote:
128 bits seems like overkill: There's no birthday attack because no-one keeps 2^(ENTROPY_BITS/2) files around,
You haven't seen my Downloads folder... :-) But seriously:
and the attack is running on the attackee's system, so there's no using specialised accelerator hardware. I'd say 64 bits is enough under those circumstances, but I wouldn't be surprised if a better security specialist could make a case for more. So maybe go with 80 bits, that's puts it at 15 or 16 characters.
Why be so miserly with entropy? This probably isn't a token that goes to a human, who may have to type it into a web browser, or send it by SMS. Its likely to be a name used only by the machine. Using 128 bits is just 22 characters using secrets.token_urlsafe(). The default entropy used by secrets is 32 bytes, which gives a 43 character token. I have plenty of files with names longer than that: "Funny video of cat playing piano while dog does backflips.mp4" Of course, if you have some specific need for the file name to be shorter (or longer!) then there ought to be a way to set the entropy used. But I think the default secrets entropy is fine, and its better to have longer names than shorter ones, within reason. I don't think 40-50 characters (plus any prefix or suffix) is excessive for a temporary file intended for use by an application rather than a human. -- Steven
participants (6)
-
Anders Munch -
Cameron Simpson -
eryk sun -
Jeroen Demeyer -
Steven D'Aprano -
Victor Stinner