[Python-ideas] Please consider skipping hidden directories in os.walk, os.fwalk, etc.

[Steven D'Aprano]

On Mon, May 07, 2018 at 06:05:15AM +0000, Steve Barnes wrote:
> In a lot of uses of os.walk it is desirable to skip version control 
> directories, (which are usually hidden directories), to the point that 
> almost all of the examples given look like:
> 
> import os
> for root, dirs, files in os.walk(some_dir):
>      if 'CVS' in dirs:
>          dirs.remove('CVS')  # or .svn or .hg etc.
>      # do something...

I would write something like:

    for root, dirs, files in filter(ignorable, os.walk(some_dir)):
        ...

where ignorable() is a function that returns False for whatever you want 
to ignore.

I don't think we can possibly agree on a single definition of 
"ignorable". This could include any combination of:

- dot files;
- files with the invisible bit set, for file systems which support that;
- files within certain directories;
- files ending in ~ (backup files);
- files with certain extensions;

or more.

Possibly this is a good use-case for composible functions, so we could 
have a set of pre-built filters:

ignorable = invisible + dotfiles + directories('.git', '.hg') + extensions('~', '.pdf')

but that sounds like it ought to be a separate module, not built in.



-- 
Steve