[Python-3000] Removal of os.path.walk

[Guido van Rossum]

On Wed, Apr 30, 2008 at 7:48 AM, Aahz <aahz at pythoncraft.com> wrote:
>
> On Tue, Apr 29, 2008, Guido van Rossum wrote:
>  > On Tue, Apr 29, 2008 at 8:10 PM, Tim Heaney <theaney at gmail.com> wrote:
>  >>
>  >> Speaking of this, is it too late to lobby for an iterator version of
>  >>  os.listdir? (Perhaps listdir would not be the best name. :)
>  >>
>  >>  There is one at
>  >>
>  >>   http://wxidle.sourceforge.net/projects/xlistdir/
>  >>
>  >>  but I think it ought to be in the standard library. Moreover, if we
>  >>  had such a thing, shouldn't os.walk use it instead of lists?
>  >
>  > I'm not sure I see the advantage of having it as an iterator; I doubt
>  > that there is ever not enough memory to hold the contents of a single
>  > directory. Do you have a compelling use case?
>
>  There's a big difference between "not enough memory" and "directory
>  consumes lots of memory".  My company has some directories with several
>  hundred thousand entries, so using an iterator would be appreciated
>  (although by the time we upgrade to Python 3.x, we probably will have
>  fixed that architecture).
>
>  But even then, we're talking tens of megabytes at worst, so it's not a
>  killer -- just painful.

Wow. And the filesystem isn't impossibly slow when accessing the last
file in such a directory?

Anyway, I'd be fine with a separate os.opendir() call that returns an
iterator. The iterator object should also have an optional close()
method which explicitly frees the underlying file descriptor (or
whatever is used on Windows).

But I don't think that changing os.listdir() is worth the pain it's
going to cause.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)