[Python-Dev] Updates to PEP 471, the os.scandir() proposal

[Ethan Furman]

On 07/09/2014 06:22 AM, Ben Hoyt wrote:
>
> One issue with option #2 that I just realized -- does scandir yield the entry at all if there's a stat error? It
> can't really, because the caller will expect the .lstat attribute to be set (assuming he asked for type='lstat') but
> it won't be. Is effectively removing these entries just because the stat failed a problem? I kind of think it is. If
> so, is there a way to solve it with option #2?

Leave it up to the onerror handler.  If it returns None, skip yielding the entry, otherwise yield whatever it returned
-- which also means the error handler should be able to set fields on the DirEntry:

   def log_err(exc, entry):
       logger.warn("Cannot stat {}".format(exc.filename))
       entry.lstat.st_size = 0
       return True

   def get_tree_size(path):
       total = 0
       for entry in os.scandir(path, info='lstat', onerror=log_err):
           if entry.is_dir:
               total += get_tree_size(entry.full_name)
           else:
               total += entry.lstat.st_size
       return total

This particular example doesn't benefit much from the addition, but this way we don't have to guess what the programmer 
wants or needs to do in the case of failure.

--
~Ethan~