How to safely maintain a status file

[Christian Heimes]

Am 08.07.2012 13:29, schrieb Richard Baron Penman:
> My initial solution was a thread that writes status to a tmp file
> first and then renames:
> 
> open(tmp_file, 'w').write(status)
> os.rename(tmp_file, status_file)

You algorithm may not write and flush all data to disk. You need to do
additional work. You must also store the tmpfile on the same partition
(better: same directory) as the status file

with open(tmp_file, "w") as f:
    f.write(status)
    # flush buffer and write data/metadata to disk
    f.flush()
    os.fsync(f.fileno())

# now rename the file
os.rename(tmp_file, status_file)

# finally flush metadata of directory to disk
dirfd = os.open(os.path.dirname(status_file), os.O_RDONLY)
try:
    os.fsync(dirfd)
finally:
    os.close(dirfd)


> This works well on Linux but Windows raises an error when status_file
> already exists.
> http://docs.python.org/library/os.html#os.rename

Windows doesn't suppport atomic renames if the right side exists.  I
suggest that you implement two code paths:

if os.name == "posix":
    rename = os.rename
else:
    def rename(a, b):
        try:
            os.rename(a, b)
        except OSError, e:
            if e.errno != 183:
                raise
            os.unlink(b)
            os.rename(a, b)

Christian