List files recursively: use recursive function or os.path.walk()

Ben Park benpark22 at yahoo.com
Wed Apr 3 13:30:14 EST 2002


I want to list all files recursively in a a directory. I can either
use a recursive function all or use os.path.walk(). I tested both, and
found that os.path.walk() was actually slightly slower.

Here is the result:
==============
recursive call to list 24530 files,  dtime= 3.30
use os.path.walk() to list 24530 files,  dtime= 3.54

On a linux Pentium machine. Python version 2.1

conclusions:
* recursive call is slightly faster than os.path.walk() .
* code is shorter using os.path.walk() .


Here is the code:
===============
#!/usr/bin/env python
# vim: set syntax=python:

import os, time, string

print ' '

#=================
def listAllFile1(fileNames9,dir,files):
#=================
  def f1(a,dir=dir): return os.path.join(dir,a)
  files2 = map(f1, files)
  fileNames9.extend(files2)

#=================
def listAllFile(dir,fileNames9):
#=================
  files = os.listdir(dir)

  def f1(a,dir=dir): return os.path.join(dir,a)
  files2 = map(f1, files)
  fileNames9.extend(files2)

  def f1(a): return os.path.isdir(a) and not os.path.islink(a)
  dirs1 = filter(f1, files2)

  for dir1 in dirs1:
    listAllFile(dir1, fileNames9)


dir0 = '/home/benpark/tmp'
fileNames9 = []
t0=time.time()
listAllFile(dir0,fileNames9)
print 'recursive call to list %d files,  dtime=%5.2f'
%(len(fileNames9), time.time()-t0)

fileNames9 = []
t0=time.time()
os.path.walk(dir0,listAllFile1,fileNames9)
print 'use os.path.walk() to list %d files,  dtime=%5.2f'
%(len(fileNames9), time.time()-t0)



More information about the Python-list mailing list