[Python-ideas] Please consider skipping hidden directories in os.walk, os.fwalk, etc.

[Eric Fahlgren]

On Tue, May 8, 2018 at 2:31 AM, Oleg Broytman <phd at phdru.name> wrote:

>    So anyone who wants to filter os.walk() must reimplement os.walk()
> themselves instead of passing something like filter_dir and filter_file
> (or accept_dir/accept_file) to os.walk()? Kind of painful, no?
>

​Not really.  It's pretty simple code so you put it in your 'usual
suspects' module and just forget about it.  Here's our version, maybe 10
years old (reworked last whenever scandir came out)​:

def _compiled_patterns(patterns, globs=True, flags=0):
    """ $uuid:95a9b8e2-fb6a-59be-b9c2-da0e6e12f8d3$
        Compile a list of patterns into regex patterns.  If ``globs``
        is true, use ``fnmatch`` to convert the patterns into regular
        expressions prior to compilation.  ``flags`` is any of the ``re``
        module's regular expression flags.
    """
    if globs:
        patterns = list(_fnmatch.translate(glob) for glob in patterns)
    return list(_re.compile(regex, flags=flags) for regex in patterns)

def walk(root_path, ignore_directories=[], show_directories=False):
    """ $uuid:f77197cd-239b-5d93-9253-c3eb7439d720$
        Walk the directory tree and return all the file entries, trimming
        directories as we go.  ``ignore_directories`` is a list of Unix
        file globs.
    """
    ignore_directories = _compiled_patterns(ignore_directories)

    def do_walk(top):
        """ $uuid:e6a4f789-5b5f-56a2-8551-297c142c3e17$ """
        for entry in _scandir.scandir(top):
            if not entry.is_dir():
                yield entry
            elif entry.is_symlink():
                pass # Ignore links.
            elif not any(ignore.match(entry.name) for ignore in
ignore_directories):
                if show_directories:
                    yield entry
                for entry in do_walk(entry.path):
                    yield entry

    return do_walk(root_path)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180508/5424fe17/attachment-0001.html>