[issue11406] There is no os.listdir() equivalent returning generator instead of list

[Martin v. Löwis]

Martin v. Löwis <martin at v.loewis.de> added the comment:

> a cron script which must process just a bunch of them at a time.
 > There's no need to gather them all.

Can you please be more explicit? What's the application in which you
have several millions of files in a directory? What's the task that
the processing script needs to perform?

> http://pastebin.com/NCGmfF49 - here's a kind of test (cached and uncached)

This isn't really convincing - the test looks at all files, so it isn't 
clear why xlistdir should do any better than listdir. And indeed, with
a cold cache, xlistdir is slower (IIUC).

> http://pastebin.com/tTKRTiNc - here's a testcase for batch processing of directory contenst (first is xlistdir(), second - listdir()) both uncached.

This is not a real-world application - there is no actual processing done.

BTW, can you publish your xlistdir implementation somewhere?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11406>
_______________________________________