[Python-ideas] Speed up os.walk() 5x to 9x by using file attributes from FindFirst/NextFile() and readdir()

[Jim Jewett]

On 11/14/12, Mike Meyer <mwm at mired.org> wrote:
> On Wed, Nov 14, 2012 at 5:51 PM, Jim Jewett <jimjjewett at gmail.com> wrote:
>> On 11/12/12, Ben Hoyt <benhoyt at gmail.com> wrote:

>> (c)  Attributes will default to None, supporting the "if x is None:
>> x=stat()" pattern for the users who do care about attributes that were
>> not available quickly.  ...

> Two questions:

> 1) Is there some way to distinguish that your st_mode field is only
> partially there (i.e. - you get the Linux/BSD d_type value, but not
> the rest of st_mode)?

os.iterdir did not call stat; you have partial information.

Or are you saying that you want to distinguish between "This
filesystem doesn't track that information", "This process couldn't get
that information right now", and "That particular piece of information
requires a second call that hasn't been made yet"?

> 2) How about making these attributes properties, so that touching one
> that isn't there causes them all to be populated.

Part of the motivation was to minimize extra system calls; that
suggests making another one should be a function call instead of a
property.

That said, I can see value in making that optional call return
something with the same API.  Perhaps:

    if thefile.X is None:
        thefile.restat()    # namespace clash, but is "restat" really
a likely file attribute?

or

    if thefile.X is None:
        thefile=stat(thefile)

-jJ