[Tutor] why is os.path.walk so slow?
Garry Willgoose
garry.willgoose at newcastle.edu.au
Wed Nov 4 13:16:17 CET 2009
I need to synchronize the files on my home and office machine and have
been using someone else's code for this to date but have been
frustrated by how slow it is in getting the information on files for
the mounted drive from my office machine so I thought I'd experiment
with a python facility for this. The code I've experimented with is as
below
def visitfunc(arg,dirname,names):
global filelist
import os.path
for name in names:
if not os.path.isdir(dirname+'/'+name):
fullname=dirname+'/'+name
filelist.append([fullname,os.path.getmtime(fullname)])
return()
def check(dir):
global filelist
import os.path
filelist=[]
os.path.walk(dir,visitfunc,'')
print filelist
return()
This is very fast for a directory on my local machine but
significantly slower on the remote machine. Not surprising but I would
have expected that the run time for the remote directory would be
limited by my broadband speed but when I look at upload/download in
real time it's less than 10% of maximum. Is this just par for the
course or is there something I can do that better utilizes my
broadband bandwidth?
====================================================================
Prof Garry Willgoose,
Australian Professorial Fellow in Environmental Engineering,
Director, Centre for Climate Impact Management (C2IM),
School of Engineering, The University of Newcastle,
Callaghan, 2308
Australia.
Centre webpage: www.c2im.org.au
Phone: (International) +61 2 4921 6050 (Tues-Fri AM); +61 2 6545 9574
(Fri PM-Mon)
FAX: (International) +61 2 4921 6991 (Uni); +61 2 6545 9574 (personal
and Telluric)
Env. Engg. Secretary: (International) +61 2 4921 6042
email: garry.willgoose at newcastle.edu.au; g.willgoose at telluricresearch.com
email-for-life: garry.willgoose at alum.mit.edu
personal webpage: www.telluricresearch.com/garry
====================================================================
"Do not go where the path may lead, go instead where there is no path
and leave a trail"
Ralph Waldo Emerson
====================================================================
More information about the Tutor
mailing list