On Wed, Jan 6, 2016 at 3:05 PM Brendan Moloney <moloney@ohsu.edu> wrote:
Its important to keep in mind the main benefit of scandir is you don't have to do ANY stat call in many cases, because the directory listing provides some subset of this info. On Linux you can at least tell if a path is a file or directory.  On windows there is much more info provided by the directory listing. Avoiding subsequent stat calls is also nice, but not nearly as important due to OS level caching.

+1 - this was one of the two primary motivations behind scandir.  Anything trying to reimplement a filesystem tree walker without using scandir is going to have sub-standard performance.

If we ever offer anything with "find like functionality" related to pathlib, it needs to be based on scandir.  Anything else would just be repeating the convenient but untrue limiting assumptions of os.listdir: That the contents of a directory can be loaded into memory and that we don't mind re-querying the OS for stat information that it already gave us but we threw away as part of reading the directory.


Brendan Moloney
Research Associate
Advanced Imaging Research Center
Oregon Health Science University
From: Python-ideas [python-ideas-bounces+moloney=ohsu.edu@python.org] on behalf of Guido van Rossum [guido@python.org]
Sent: Wednesday, January 06, 2016 2:42 PM
To: Random832

Cc: Python-Ideas
Subject: Re: [Python-ideas] find-like functionality in pathlib
I couldn't help myself and coded up a prototype for the StatCache design I sketched. See http://bugs.python.org/issue26031. Feedback welcome! On my Mac it only seems to offer limited benefits though...

Python-ideas mailing list
Code of Conduct: http://python.org/psf/codeofconduct/