waling a directory with very many files
mail at timgolden.me.uk
Sun Jun 21 06:06:23 EDT 2009
> I might be a little late with my comment here.
> David Beazley in his PyCon'2008 presentation "Generator Tricks
> For Systems Programmers" had this very elegant example of handling an
> unlimited numbers of files:
David Beazley's generator stuff is definitely worth recommending
on. I think the issue here is that: anything which ultimately uses
os.listdir (and os.walk does) is bound by the fact that it will
create a long list of every file before handing it back. Certainly
there are techniques (someone posted a ctypes wrapper for opendir;
I recommended FindFirst/NextFile on Windows) which could be applied,
but those are all outside the stdlib.
More information about the Python-list