[Python-ideas] Please consider skipping hidden directories in os.walk, os.fwalk, etc.

[Wes Turner]

fnmatch.filter does Unix filename pattern matching.
https://docs.python.org/3/library/fnmatch.html#fnmatch.filter

grin and grind are like grep and find with options to filter hidden files
and VCS directories by default.
https://pypi.org/project/grin/

There's an example of using the Python API here:
https://github.com/rkern/grin/blob/master/examples/grinpython.py

- grin.get_regex(args)
- grin.get_filenames(args)

https://github.com/rkern/grin/blob/master/grin.py

On Wednesday, May 9, 2018, Steve Barnes <gadgetsteve at live.co.uk> wrote:

>
>
> On 08/05/2018 15:53, Giampaolo Rodola' wrote:
> >
> >
> > On Tue, May 8, 2018 at 2:00 PM, David Mertz <mertz at gnosis.cx
> > <mailto:mertz at gnosis.cx>> wrote:
> >
> >     I like the idea. I think an argument to os.walk() is the simplest
> >     option for most users. But per some comments, "hidden" is actually
> >     more subtle than the filesystem bit sometimes. I.e. dot-files, ~
> >     suffix, maybe .bak, etc.
> >
> >     I'd suggest meeting the ideas slightly and making the new argument
> >     'filter' or 'skip' that takes a callable. Default to None, but
> >     provide an os.is_hidden that users don't need to figure out how to
> >     implement. E.g.
> >
> >     os.walk(PATH, skip=os.is_hidden)
> >
> >     os.walk(PATH, skip=lambda entry: entry.name.endswith(('~', '.bak',
> >     '.tmp')))
> >
> >
> > I think this would be a good addition because it gives direct access to
> > the underlying os.scandir() objects which are currently inaccessible and
> > discarded (if os.walk() were to be written today it'd probably yield
> > (root, os.DirEntry) instead of (root, dirs, files)). As such one can
> > implement advanced filtering logic without having to call os.stat() for
> > each path string yielded by os.walk() (faster).
> >
> > IMO the callback should accept a (root, os.DirEntry) pair though,
> > because the "root" path can also be part of the filtering logic.
> >
> > --
> >
> > Giampaolo - http://grodola.blogspot.com
> >
> >
>
> I like the idea of extending the original idea to a filtered walk
> possibly with some predefined filters.
>
> As there does not seem to be a lot of strong opposition so far to the
> basic idea, (other than a some "why bother it is too easy to do
> yourself"), it seems like there is a choice now is between:
>
>    a) raising an enhancement request on the tracker (I am not sure if
> this is major enough to require a PEP) or
>    b) setting up a new library on PyPi and putting it out there to see
> if it sinks or swims.
>
> What is the general feeling between the two options?
>
> --
> Steve (Gadget) Barnes
> Any opinions in this message are my personal opinions and do not reflect
> those of my employer.
>
> ---
> This email has been checked for viruses by AVG.
> http://www.avg.com
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180509/73a5a6cc/attachment-0001.html>