[Tutor] why is os.path.walk so slow?
Garry Willgoose
Garry.Willgoose at newcastle.edu.au
Thu Nov 5 00:36:15 CET 2009
> [snip]
>> I need to synchronize the files on my home and office machine and
>> have
>> been using someone else's code for this to date but have been
>> frustrated by how slow it is in getting the information on files for
>> the mounted drive from my office machine...
> [/snip]
>
> Not to cut your coding project short, and it may not even be
> applicable, but have you looked into rsync? They kind of wrote the
> book on efficiency in regards to synchronization of files.
>
> Just a thought.
> -Modulok-
It looks like rsync is ideal for what I want ... thanks for pointing
this out. One less project to work on ;-)
To the other replies:
1. Both local and remote systems for the testing are OSX 10.5 and CPU
was not limiting during the test. Python was the Enthought binary
distribution. Internet service is ADSL 1500/512.
2. I neglected to mention that I did comment out the code inside
visitfunc() with the same result so the issue is inside walk().
I guess now I know abut rsync the pressing need for this is gone (I
might still write a nice GUI for rsync ;-) but I'm still intrigued by
the problem and whether its a latency vs bandwidth problem. The idea
of writing a script to run on the remote end is good, and something I
might use for another project.
More information about the Tutor
mailing list