[Python-ideas] shutil.symlink to allow non-race replacement of existing link targets
tom at hale.ee
Mon May 13 05:38:08 EDT 2019
As suggested by Toshio Kuratomi at https://bugs.python.org/issue36656, I
am raising this here for inclusion in the shutil module.
Mimicking POSIX, os.symlink() will raise FileExistsError if the link
name to be created already exists.
A common use case is overwriting an existing file (often a symlink) with
a symlink. Naively, one would delete the file named link_name file if it
exists, then call symlink(). This "solution" is already 3 lines of code,
and without exception handling it introduces the race condition of a
file named link_name being created between unlink and symlink.
Depending on the functionality required, I suggest:
* os.symlink() - the new link name is expected to NOT exist
* shutil.symlink() - the new symlink replaces an existing file
Handling all possible race conditions (some detailed in issue36656) is
non-trivial, however this is the best that I have come up with so far:
import os, tempfile
def symlink(target, link_name):
'''Create a symbolic link link_name pointing to target.
Overwrites link_name if it exists. '''
# os.replace() may fail if files are on different filesystems
link_dir = os.path.dirname(link_name)
# Link to a temporary filename that doesn't exist
temp_link_name = tempfile.mktemp(dir=link_dir)
# os.* functions mimic as closely as possible system functions
# The POSIX symlink() returns EEXIST if link_name already exists
# Replace link_name with temp_link_name
# Pre-empt os.replace on a directory with a nicer message
raise IsADirectoryError(f"Cannot symlink over existing
The documentation (https://docs.python.org/3/library/shutil.html) I
suggest for this is:
Create a symbolic link named link_name pointing to target, overwriting
target if it exists. If link_name is a directory, IsADirectoryError is
raised. To not overwrite target, use os.symlink()
It would be tempting to do:
But this has a race condition when replacing a symlink should should
*always* exist, eg:
/lib/critical.so -> /lib/critical.so.1.2
When upgrading by:
There is a point in time when /lib/critical.so doesn't exist.
One issue I see with my suggested code is that the file at
temp_link_name could be changed before target is replaced with it. This
is mitigated by the randomness introduced by mktemp().
While it is far less likely that a file is accessed with a random and
unknown name than with an existing known name, I seek input on a
solution if this is an unacceptable risk.
* https://bugs.python.org/issue36656 (already mentioned above)
More information about the Python-ideas