[Python-ideas] fnmatch.filter_false

[tritium-list at sdamon.com]

> -----Original Message-----
> From: Python-ideas [mailto:python-ideas-bounces+tritium-
> list=sdamon.com at python.org] On Behalf Of Steven D'Aprano
> Sent: Thursday, May 18, 2017 9:01 AM
> To: python-ideas at python.org
> Subject: Re: [Python-ideas] fnmatch.filter_false
> 
> On Thu, May 18, 2017 at 12:07:05AM -0400, tritium-list at sdamon.com wrote:
> 
> > > At the cost of a slight inefficiency, you could use the pure Python
> > > equivalent given in the docs:
> > >
> > > https://docs.python.org/3/library/fnmatch.html#fnmatch.filter
> > >
> > >
> > > fnmatch.filter(names, pattern)
> > >
> > >     Return the subset of the list of names that match pattern. It is
> > >     the same as [n for n in names if fnmatch(n, pattern)], but
implemented
> > >     more efficiently.
> > >
> > > So your filter_false is:
> > >
> > >     [n for n in names if not fnmatch(n, pattern)]
> > >
> > > which avoids the need for the copy-and-paste anti-pattern.
> >
> > I ran a test on the same dataset using listcomps.  The modified version
of
> > filter is still 22.<some noise>, the unmodified version is still
19.<some
> > noise>.  However the listcomp method is 41.<some noise>.  That
> performance
> > is important, at least to me.
> 
> 41 what? Nanoseconds? Hours? For how many files? *wink*


41 "integer units of whatever timeit reports with".  I would have to look,
but iirc it's seconds for 10000 runs.  I am not on unix.


> In any case, are you running on Linux or Unix? If so, try replacing the
> fnmatch with fnmatchcase, since that avoids calling os.path.normcase (a
> no-op on Linux) twice for each file.
> 
> If you want to reduce the number of function calls even more, try this
> untested code:
> 
> # using an implementation detail is a bit naughty
> match = fnmatch._compile_pattern(pattern).match
> results = [n for n in names if match(n) is None]
> 
> 
> 
> 
> --
> Steve
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/