Big speed boost in os.walk in Python 2.5
fredrik at pythonware.com
Fri Oct 13 18:05:56 CEST 2006
> Results on Windows XP after some run to fill the disk cache (with
> ~59000 files and ~3500 folders):
> Python 2.4.3 : 45s
> Python 2.5 : 10s
> Very nice, but somewhat strange...
> Is Python 2.4.3 os.walk buggy ???
No. A few "os" function are now implemented in terms of Windows API:s,
instead of using Microsoft C's POSIX compatibility layer. This includes
os.stat(), which is what isdir() uses to check if something is a
directory. The code was rewritten to work around problems with
timestamps, so the speedup is purely a side effect.
> Is this results only valid in Windows or *nix system show the same
> difference ?
On Unix system, Python uses POSIX API:s, not Windows API:s.
> The profiler show that most of time is spend in ntpath.isdir and this
> function is *a lot* faster in Python 2.5.
Why are you asking if something's buggy when you've already figured out
what's been improved?
> Maybe this improvement could be backported in Python 2.4 branch for the
> next release ?
It's not really broken, so that's not very likely.
More information about the Python-list