[Python-ideas] Implementation of shutil.move

[Christian Heimes]

Am 12.08.2011 15:53, schrieb Devin Jeanpierre:
> This doesn't completely solve the race condition:
> 
> `os.open(path, O_EXCL | O_CREAT)` opens an fd for a file location. If
> a file was already at that location, it fails, if not, it succeeds
> with an fd.
> So if os.move() used this fd to actually write the new data, there
> would be no race condition in terms of file creation/deletion. This
> is, as far as I can tell, what was suggested by your linked thread.
> 
> However, in your suggestion it does not do that: it does opens a new
> fd to a new file, and then does precisely what it did before. During
> the time in-between the os.open and the shutil.move(), somebody can
> delete the created file, and write a new one, or whatever. If they do
> that, then any such changes they make will be lost because shutil.move
> will steamroller them and overwrite the file.

I couldn't figure out how the open fd should fix the race condition
until I read your answer. You are right! The trick doesn't help at all.
Even a lock file won't help since POSIX flock()s are only advisory locks.

Contrary to shutil.copy(), os.move() doesn't write any data. It only
changes some metadata by changing the name of the file. This makes the
move op on a single file system an atomic operation. The fd doesn't help
here at all.

We could implement move with a combination of link() and unlink().
link() sets errno to EEXIST if the destination already exists.
shutil.move() will no longer be an atomic operation but that's fine with
me. We still have os.move().

Christian