On Sun, Jan 30, 2011 at 11:33 PM, "Martin v. Löwis"
Maybe also
* Read and cache the directory contents and search it ourselves instead of making a system call for every possible name.
I wouldn't do that - I would expect that this is actually slower than making the system calls, because the system might get away with not reading the entire directory (whereas it will have to when we explicitly ask for that).
Hm. Long (very long) ago I had to implement just that, and it was much faster. But this was over NFS. Still, I think the directory would have to be truly enormous before reading its contents (which doesn't access all the inodes) is slower than statting a few dozen of its entries. At least on most *nix filesystems. Another thing to consider: on App Engine (which despite of all its architectural weirdness uses a -- mostly -- standard Linux filesystem for the Python code of the app) someone measured that importing from a zipfile is much faster than importing from the filesystem. I would imagine this extends to other contexts too, and it makes sense because the zipfile directory gets cached in memory so no stat() calls are necessary. (Basically I am biased to believe that stat() is a pretty slow system call -- this may just be old NFS lore though.) -- --Guido van Rossum (python.org/~guido)