[Python-ideas] Speed up os.walk() 5x to 9x by using file attributes from FindFirst/NextFile() and readdir()
robertc at robertcollins.net
Wed Nov 14 19:52:12 CET 2012
On Wed, Nov 14, 2012 at 11:54 PM, Serhiy Storchaka <storchaka at gmail.com> wrote:
> On 14.11.12 11:53, Robert Collins wrote:
>> - chdir to the directory before you stat and use a relative path: it
>> turns out when working with many files that the overhead of absolute
>> paths is substantial.
> Look at fwalk(). It reaches the same benefits without changing of process-global cwd (i.e. it is thread-safe).
cwd is thread-safe on unix (well, Linux anyhow :P), and the native OS
dir walking on Windows is better itself.
The only definitions I can find for fwalk are on BSD, not on Linux, so
fwalk is also likely something that would need porting; The
definitions I did find take a vector of file pointers, and so are
totally irrelevant for the point I made.
More information about the Python-ideas