[Python-ideas] Speed up os.walk() 5x to 9x by using file attributes from FindFirst/NextFile() and readdir()

Stefan Drees stefan at drees.name
Sat Nov 17 08:23:32 CET 2012


Greg Ewing suggested:
> Mike Meyer wrote:
>> I can see a number of alternatives to improve this situation:
>>
>> 1) wrap the return partial stat info in a proxy object
>> 2) Make iterdir_stat an os.walk internal tool, and don't export it.
>> 3) Add some kind of "we have a full stat" indicator,
>> 4) document one of the a stat values as a "we have a full stat"
>> indicator,
>> 5) Add a keyword argument to ... always do the full stat.
>> 6) Depreciate os.walk, and provide os.itertree
>
> 7) Provide an iterdir() with a way of specifying exactly
> which stat fields you're interested in. Then it can perform
> stat calls if and only if needed, and the user doesn't have
> to tediously test for presence of things in the result.
>

+1 for following that seventh path. It offers the additional benefit for 
the library code, that constraints of the backend functionality used are 
more clearer to handle: If requested and available allthough expensive, 
"yield nevertheless the attribute values" is then a valid strategy.

All the best,
Stefan.



More information about the Python-ideas mailing list