[issue12970] os.wlak() consider some symlinks as dirs instead of non-dirs
New submission from Марк Коренберг <socketpair@gmail.com>: Consider code: for (root, dirs, nondirs) in os.walk(path, followlinks=False): print (nondirs) This code will not print symlinks that refer to some dir. I think it is the bug. In other words: If followlinks is True, we should consider some symlinks as dirs. If not, any symlink is the non-dir. Patch included. Also, please fix documentation about this nuance. ---------- assignee: docs@python components: Documentation, Library (Lib) files: z.patch keywords: patch messages: 143965 nosy: docs@python, mmarkk priority: normal severity: normal status: open title: os.wlak() consider some symlinks as dirs instead of non-dirs type: behavior versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4 Added file: http://bugs.python.org/file23140/z.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12970> _______________________________________
Changes by STINNER Victor <victor.stinner@haypocalc.com>: ---------- nosy: +haypo _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12970> _______________________________________
Марк Коренберг <socketpair@gmail.com> added the comment: Also, there is some mis-optimisation for followlinks=False: stat() and then lstat() will be called. Instead of one lstat(). Code may be rewritten as (but I don't know about cross-platform issues): --------------------------------- if followlinks: mode = os.stat(path).st_mode else: mode = os.lstat(path).st_mode if stat.S_ISDIR(mode): dirs.append(path) else: nondir.append(path) --------------------------------- It will be much cleaner than current (or patched with my patch) implementation ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12970> _______________________________________
Changes by Марк Коренберг <socketpair@gmail.com>: ---------- title: os.wlak() consider some symlinks as dirs instead of non-dirs -> os.walk() consider some symlinks as dirs instead of non-dirs _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12970> _______________________________________
Changes by Alexey Smirnov <alexey.smirnov@gmx.com>: ---------- nosy: +alexey-smirnov _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12970> _______________________________________
Changes by Sung-Yu Chen <eesonyu@gmail.com>: ---------- nosy: +Sung-Yu.Chen _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12970> _______________________________________
Nick Coghlan <ncoghlan@gmail.com> added the comment: This behaviour came up recently when implementing os.fwalk() [1]. There are problems with all 3 possible approaches (list as dirs, list as files, don't list at all) when followlinks is False. Since all alternatives are potentially surprising, the current behaviour wins by default (as people will already have written their code to cope with that behaviour and there's no net gain in changing the default, since the desired treatment of such links will vary according to the task at hand). As a result, I'm converting this to a pure documentation issue - the os.walk() docs should definitely mention this subtlety. The behaviour won't be changing, though. [1] http://bugs.python.org/issue13734,#msg151077 ---------- components: -Library (Lib) nosy: +ncoghlan type: behavior -> enhancement versions: -Python 3.1, Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12970> _______________________________________
Uwe Kleine-König added the comment: I like the function as it is documented, i.e. "filenames is a list of the names of the non-directory files in dirpath.". This includes all symlinks (in the followlinks=False cast at least). I'd say not including symlinks to directories but symlinks to files is a magnitude more surprising than treating a symlink to a directory as a file. And if you consider this as a short comming of the documentation this isn't (IMHO) a subtlety. The (my?) intuition says: all entries of a root (apart from . and .. as documented) are included in either dirnames or filenames. Yes, changing behaviour here might break some code, but this applies to all changes. For some usecases it might be right to just skip over symlinks-to-dirs, but if it's not you have to opendir + read all root entries again in the loop to find all symlinks which effectively means reimplementing os.walk. ---------- nosy: +ukl _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12970> _______________________________________
Changes by STINNER Victor <victor.stinner@gmail.com>: ---------- nosy: +benhoyt _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12970> _______________________________________
Akira Li added the comment: I've updated os.walk() documentation to mention that *dirnames* list includes symlinks to directories. To imitate the other two cases: - treat the symlinks as files: for dirpath, dirnames, files in os.walk(top): dirs = [] for name in dirnames: (files if islink(join(dirpath, name)) else dirs).append(name) dirnames = dirs - don't include in either of the lists: for dirpath, dirnames, files in os.walk(top): dirnames[:] = [name for name in dirnames if not islink(join(dirpath, name))] where islink = os.path.islink and join = os.path.join. I've uploaded the documentation patch. Please, review. ---------- nosy: +akira Added file: http://bugs.python.org/file36138/docs-walk-issue12970.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12970> _______________________________________
participants (7)
-
Akira Li -
Alexey Smirnov
-
Nick Coghlan
-
STINNER Victor
-
Sung-Yu Chen
-
Uwe Kleine-König
-
Марк Коренберг