[issue27886] Docs: the difference between rename and replace is not obvious
New submission from Andrew Svetlov: Hi. On reading the doc for pathlib I've stuck with `.rename()` and `.replace()` (https://docs.python.org/3/library/pathlib.html#pathlib.Path.rename). What's the difference? Going to pathlib's source code I've figured out that methods are use different functions from `os` module: `os.rename()` and `os.replace()`. But the documentation for `os` module is not obvious too: the docs for both functions are almost equal from my perspective, the only significant difference is that `os.rename()` suggests to use `os.replace()` for cross-compatibility. Could anybody explain the difference? Also, at least the doc for `pathlib.Path.rename` worth to have a sentence like borrowed from `os.rename`: "If you want cross-platform overwriting of the destination, use replace()." ---------- assignee: docs@python components: Documentation keywords: easy messages: 273836 nosy: asvetlov, docs@python, pitrou priority: low severity: normal status: open title: Docs: the difference between rename and replace is not obvious type: behavior versions: Python 3.4, Python 3.5, Python 3.6 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue27886> _______________________________________
R. David Murray added the comment: The existing docs are pretty clear on the difference: rename is only guaranteed to replace an existing file on unix (which I think means posix in this context), whereas replace always replaces the file, regardless of platform. I'm actually surprised that rename is even part of the pathlib API, since its cross platform behavior is not consistent. It also seems as though the replace docs are incorrect, since they imply directories are replaced, but the os docs say they are not. The "this is a posix requirement" note in os.replace also seems imprecise. Windows is not posix, so that note leaves me wondering if replace is atomic on Windows or not. That should be clarified one way or another. I think adding the cross platform note in the pathlib docs is reasonable, since it addresses the question of why there are two similar functions. ---------- nosy: +r.david.murray _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue27886> _______________________________________
Changes by R. David Murray <rdmurray@bitdance.com>: ---------- versions: -Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue27886> _______________________________________
Andrew Svetlov added the comment: Just keeping the reference to `os.rename()`: https://docs.python.org/3/library/os.html#os.rename Aha, `os.rename` says: "On Unix, if dst exists and is a file, it will be replaced silently if the user has permission." and "On Windows, if dst already exists, OSError will be raised even if it is a file." Sorry, I've missed this. Maybe it worth to add `.. warning:` section for both `os.rename` and `pathlib.Path.rename`? ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue27886> _______________________________________
R. David Murray added the comment: No, we don't like to overuse warnings. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue27886> _______________________________________
Eryk Sun added the comment: Having rename() in pathlib is fine as long as it links to the os docs. This probably needs a new issue, but I do see room for improvement in the latter. For Unix, the os.rename and os.replace docs should clarify that an empty destination directory can be replaced by another directory. For example: >>> os.mkdir('foo1') >>> os.mkdir('foo2') >>> os.rename('foo1', 'foo2') The specification of rename [1] states that "[i]f the *old* argument points to the pathname of a directory, the *new* argument shall not point to the pathname of a file that is not a directory". It further specifies that "[i]f *new* names an existing directory, it shall be required to be an empty directory". Windows, on the other hand, doesn't allow replacing a directory, even an empty one. The wording for os.rename and os.replace could be changed to something like: "[i]f dst is a directory, OSError will be raised, except not on Unix if src is a directory and dst is empty". Windows MoveFileEx calls NtSetInformationFile to set the FileRenameInformation [2]. The MSDN docs do not explicitly require that the operation is atomic, unlike POSIX rename. I think it's a reasonable expectation that a Windows filesystem should rename atomically, and that's probably the case. However, it should be clear that only Unix guarantees this, e.g. "[i]f successful, on Unix the renaming will be an atomic operation, as required by POSIX". MSDN claims that a rename will fail in the following cases: A file or directory can only be renamed within a volume. Even if ReplaceIfExists is set to TRUE, the rename operation will still fail if a file with the same name already exists and is a directory, a read-only file, or a currently executing file. A file cannot be renamed if it has any open handles, unless it is only open because of a batch opportunistic lock (oplock) and the batch oplock can be broken immediately. A file cannot be renamed if a file with the same name exists and has open handles (except in the batch-oplock case described earlier). A directory cannot be renamed if it or any of its subdirectories contains a file that has open handles (except in the batch-oplock case described earlier). For the third case, actually an open file can be renamed if delete/rename access is shared. For example: >>> with open('foo1', 'w') as f: f.write('foo1') ... 4 Open the file with shared delete access: >>> SHARE_ALL = 7 >>> _winapi.CreateFile('foo1', 0x80000000, SHARE_ALL, 0, 3, 0, 0) 224 and rename/replace succeeds: >>> os.replace('foo1', 'foo2') >>> open('foo2').read() 'foo1' Anyway, the os.replace docs could state in general that: "[o]n Windows, a PermissionError will be raised if dst is a read-only file, or if either src or dst is currently open, or if src is a directory with an open file". [1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html [2]: https://msdn.microsoft.com/en-us/library/ff540344 ---------- nosy: +eryksun _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue27886> _______________________________________
Change by Ezio Melotti <ezio.melotti@gmail.com>: ---------- nosy: +ezio.melotti _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue27886> _______________________________________
Change by Eryk Sun <eryksun@gmail.com>: ---------- Removed message: https://bugs.python.org/msg273845 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue27886> _______________________________________
Eryk Sun <eryksun@gmail.com> added the comment: The pathlib documentation of Path.rename() says "[o]n Unix, if target exists and is a file, it will be replaced silently if the user has permission". This leaves the behavior on Windows in question. The reader has to scroll down to the correspondence table to link to the documentation of os.rename() and presume that Path.rename() also raises FileExistsError for this case. The Path.rename() docs should just state upfront that "[o]n Windows, if target exists, FileExistsError will be raised". ---------- components: +Library (Lib) title: Docs: the difference between rename and replace is not obvious -> Docs: the difference between Path.rename() and Path.replace() is not obvious versions: +Python 3.10, Python 3.8, Python 3.9 -Python 3.5, Python 3.6 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue27886> _______________________________________
participants (4)
-
Andrew Svetlov
-
Eryk Sun
-
Ezio Melotti
-
R. David Murray