[Tutor] os.path.walk vs unix find command

[Chris Fuller]

On Friday 15 August 2008 17:17, Angela Yang wrote:
> Hi Python gurus:
>
>
>
> Is os.popen("find") faster or slower than os.path.walk to find file pattern
> in the
>
> directory tree?  I thought os.path.walk would be faster than unix find, but
> that doesn't
>
> seem to be the case?
>
>
>
I'd expect find to be faster, its written in C, and probably optimized pretty 
well for its task.  Less portable, however.

> What is the fastest way in python to search for a file with a given pattern
> in a directory
>
> tree?
>
>
>
> Here's the code using os.path.walk:
>
>  
>
> def find_src_mk_file(walk_lst_result, dirname, fnames):
>
>  
>
>     x = len(dirname)
>
>     if (dirname[x-4:x] == "/src"):
This is a lot more than you need.  You can index strings relative to the end 
with negative numbers, and if you leave the second half of the slice empty, 
it will go to the end.  Similarly with the first half and starting from the 
beginning.  You could have used dirname[-4:], but an even better way would be 
to just use the string method endswith(): dirname.endswith('/src').  You 
might also have used os..path.split()[1]=='src', which would have avoided 
portability issues.


>
>         result = glob.glob(os.path.join(dirname, "src.mk"))
You are duplicating effort here.  The filenames are already in fnames,  Search 
that list, rather than perform additional disk access.
>
>         if result:
>
>             walk_lst_result.append(result[0])
>
> def is_makefile_outofdate():
>
>     <code>
>
>             walk_lst_result = []
>
>            
> os.path.walk( component_dir, find_src_mk_file, walk_lst_result )
>
>    
>
>             # check each src.mk and remove from lst
>
>             for sub_file_src_mk in walk_lst_result:
>
>     <code>
>
>
>
> Anything wrong with this code?  Please advise.
>
>
>
> Thanks.
>
> Angela

That additional directory scan is your main problem.

Cheers!