waling a directory with very many files

Tim Golden mail at timgolden.me.uk
Sun Jun 21 06:06:23 EDT 2009

rkl wrote:
> I might be a little late with my comment here.
> David Beazley in his PyCon'2008 presentation "Generator Tricks
> For Systems Programmers" had this very elegant example of handling an
> unlimited numbers of files:

David Beazley's generator stuff is definitely worth recommending
on. I think the issue here is that: anything which ultimately uses
os.listdir (and os.walk does) is bound by the fact that it will
create a long list of every file before handing it back. Certainly
there are techniques (someone posted a ctypes wrapper for opendir;
I recommended FindFirst/NextFile on Windows) which could be applied,
but those are all outside the stdlib.


More information about the Python-list mailing list