Python vs. Ruby (and os.path.walk)

[Peter Hansen]

Steven Atkinson wrote:
> 
> I changed both to do the compile outside the loop. The Ruby is just a tad
> quicker (something like 13 seconds to Pythons 15 seconds). The main
> difference in the code is that the Python one has to combine the dir and the
> filename back into a full path in the lister if there is a filename match.
> 
> Interesting enough, the Python one gained the most from do the regex compile
> outside the loop.  It picked up about 20-30% while Ruby only gained 5-10%. 

You shouldn't draw conclusions too quickly about this.  The speedup in 
Python probably comes just because you are saving a couple of function
calls, not because the compilation is actually occurring repeatedly.  If
you profile the code, you will note that the regex compilation occurs
only once in any case, because the regex is cached internally.

> Still can't explain the intial slowness of my intial 5-6 runs. The Python
> code was taking several order of magnitudes longer. The walk code seemed to
> be pausing on each directory recursion. You'd see a file listing for a dir
> and it'd be quick, and then it'd pause for the next. It was hard to tell if
> that was because it was. The test environment did not change.

Are you sure nothing else was running in the background during these tests?
Much of the time spent running the code is actually I/O related, so maybe
another process was doing a lot of disk access at the time and you didn't
notice?

> In any case, though that mystery is not solved, at least I know that Ruby is
> about the same speed wise as Ruby.                                   ^^^^
                               ^^^^---------note identity function------'

That much, at least, is true.  <wink>

-Peter