waling a directory with very many files

[Christian Heimes]

tom schrieb:
> i can traverse a directory using os.listdir() or os.walk(). but if a
> directory has a very large number of files, these methods produce very
> large objects talking a lot of memory.
> 
> in other languages one can avoid generating such an object by walking
> a directory as a liked list. for example, in c, perl or php one can
> use opendir() and then repeatedly readdir() until getting to the end
> of the file list. it seems this could be more efficient in some
> applications.
> 
> is there a way to do this in python? i'm relatively new to the
> language. i looked through the documentation and tried googling but
> came up empty.

Some time ago we had a discussion about turning os.listdir() into a
generator. No conclusion was agreed on. We also thought about exposing
the functions opendir(), readdir(), closedir() and friends but as far as
I know and as far as I've checked the C code in Modules/posixmodule.c
none of the functions as been added.

For now you are on your own to implement wrappers for the system calls.
For the distant future you may see the appropriate functions in the os
module. A mail to the python ideas list may increase your chances. ;)

Christian