[Python-ideas] shutil.symlink to allow non-race replacement of existing link targets

Tom Hale tom at hale.ee
Mon May 13 05:38:08 EDT 2019

As suggested by Toshio Kuratomi at https://bugs.python.org/issue36656, I 
am raising this here for inclusion in the shutil module.

Mimicking POSIX, os.symlink() will raise FileExistsError if the link 
name to be created already exists.

A common use case is overwriting an existing file (often a symlink) with 
a symlink. Naively, one would delete the file named link_name file if it 
exists, then call symlink(). This "solution" is already 3 lines of code, 
and without exception handling it introduces the race condition of a 
file named link_name being created between unlink and symlink.

Depending on the functionality required, I suggest:

* os.symlink() - the new link name is expected to NOT exist
* shutil.symlink() - the new symlink replaces an existing file

Handling all possible race conditions (some detailed in issue36656) is 
non-trivial, however this is the best that I have come up with so far:


import os, tempfile

def symlink(target, link_name):
     '''Create a symbolic link link_name pointing to target.
     Overwrites link_name if it exists. '''

     # os.replace() may fail if files are on different filesystems
     link_dir = os.path.dirname(link_name)

     # Link to a temporary filename that doesn't exist
     while True:
         temp_link_name = tempfile.mktemp(dir=link_dir)

         # os.* functions mimic as closely as possible system functions
         # The POSIX symlink() returns EEXIST if link_name already exists
             os.symlink(target, temp_link_name)
         except FileExistsError:

     # Replace link_name with temp_link_name
         # Pre-empt os.replace on a directory with a nicer message
         if os.path.isdir(link_name):
             raise IsADirectoryError(f"Cannot symlink over existing 
directory: '{link_name}'")
         os.replace(temp_link_name, link_name)
         if os.path.islink(temp_link_name):


The documentation (https://docs.python.org/3/library/shutil.html) I 
suggest for this is:

shutil.symlink(target, link_name)
Create a symbolic link named link_name pointing to target, overwriting 
target if it exists. If link_name is a directory, IsADirectoryError is 
raised. To not overwrite target, use os.symlink()


It would be tempting to do:

while True:
         os.symlink(target, link_name)
     except FileExistsError:

But this has a race condition when replacing a symlink should should 
*always* exist, eg:

     /lib/critical.so -> /lib/critical.so.1.2

When upgrading by:

     symlink('/lib/critical.so.2.0', '/lib/critical.so')

There is a point in time when /lib/critical.so doesn't exist.


One issue I see with my suggested code is that the file at 
temp_link_name could be changed before target is replaced with it. This 
is mitigated by the randomness introduced by mktemp().

While it is far less likely that a file is accessed with a random and 
unknown name than with an existing known name, I seek input on a 
solution if this is an unacceptable risk.

Prior art:

* https://bugs.python.org/issue36656 (already mentioned above)
* https://stackoverflow.com/a/55742015/5353461
* https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/ln.c

Tom Hale

More information about the Python-ideas mailing list