os.walk bug?

[Alex Martelli]

Adam DePrince <adam at cognitcorp.com> wrote:

> Each iteration of os.walk returns three parameters; one of those, dirs,
> is a list of directories within the current directory pointed at by
> root. 

Yes, it does.

> Am I correct to assume that you beleve that by changing the contents of
> dir you will affect how os.walk traverses the directories.  If so, that
> is your problem.

It's not a problem: it's a correct description of os.walk behavior.

> When looping in a for loop, it is generally good programming practice to
> assume that nothing you do within your loop will affect which
> root,dir,files tuples have been set aside to loop over.  It is likely
> that once os.walk was called, the items to be iterated over are
> predestined (i.e. assume you are iterating over a tuple.) 

"Likely"?  I think it's generally good programming practice to read the
docs.  help(os.walk) says:

When topdown is true, the caller can modify the dirnames list in-place
(e.g., via del or slice assignment), and walk will only recurse into the
subdirectories whose names remain in dirnames; this can be used to prune
the search, or to impose a specific order of visiting.  Modifying

and topdown is true by default.  The problem was just that the poster
was not using the right way to "modify the dirnames list in-place", as
he was looping on said list rather than a copy and yet modifying what he
was looping on (two suggestions i saw were "loop on a copy", once
obtained with the odd but popular Python idiom somelist[:], once with
the much more readable approach list(somelist)...).  The best
alternative is often to use a list comprehension, with somelist[:] on
the left of the assignment since you do need to modify in-place.  I.e.,
say that, generically speaking, you're trying (and failing) to do:

for item in alist:
    if isnasty(item):
        alist.remove(item)

changing the header to ``in alist[:]:'' or ``in list(alist):'' probably
makes this code work, but it still can't make it GOOD... good would be:

alist[:] = [item for item in alist if not isnasty(item)]


Alex