[Python-ideas] BetterWalk, a better and faster os.walk() for Python

[Andrew Barnert]

First, another thought on the whole thing:

Wrapping readdir is useful. Replacing os.walk is also useful. But they don't 
necessarily have to be tied together at all.

In particular, instead of trying to write an iterdir_stat that can properly 
support os.walk on all platforms, why not just implement os.walk differently on 
platforms where iterdir_stat can't support it? (In fact, I think an os.walk 
replacement based on the fts API, which never uses iterdir_stat, would be the 
best answer, but let me put more thought into that idea...)

Anyway, comments:

From: John Mulligan <phlogistonjohn at asynchrono.us>
Sent: Fri, November 23, 2012 8:13:22 AM


> I like returning the d_type directly because in  the unix style APIs the 
> dirent structure doesn't provide the same stuff as  the stat result and I 
> don't want to trick myself into thinking I have all  the information 
> available from the readdir call. I also like to have my  Python functions 
> map pretty closely to the C calls.

Of course that means that implementing the same interface on Windows means 
faking d_type from the stat result, and making the functions map less closely to 
the C calls…

> In addition I have a fditerdir call that supports a directory file 
> descriptor as the first argument. This is handy because I also have a 
> wrapper for fstatat (this was all created for Python 2 and before 3.3 
> was released).

This can only be implemented on platforms that support the *at functions. I 
believe that means just linux and OpenBSD right now, other *BSD (including OS X) 
at some unspecified point in the future. Putting something like that in the 
stdlib would probably require also adding another function like os_supports_at 
(similar to supports_fd, supports_dirfd, etc.), but that's not a big deal.