[Python-Dev] Importing .pyc in -O mode and vice versa
aahz at pythoncraft.com
Sun Nov 5 17:24:58 CET 2006
On Sun, Nov 05, 2006, "Martin v. L?wis" wrote:
> Greg Ewing schrieb:
>> Fredrik Lundh wrote:
>>> well, from a performance perspective, it would be nice if Python looked
>>> for *fewer* things, not more things.
>> Instead of searching for things by doing a stat call for each
>> possible file name, would it perhaps be faster to read the contents
>> of all the directories along sys.path into memory and then go
>> searching through that?
> That should never be better: the system will cache the directory
> blocks, also, and it will do a better job than Python will.
Maybe so, but I recently dealt with a painful bottleneck in Python code
caused by excessive stat() calls on a directory with thousands of files,
while the os.listdir() function was bogging things down hardly at all.
Granted, Python bytecode was almost certainly the cause of much of the
overhead, but I still suspect that a simple listing will be faster in C
code because of fewer system calls. It should be a matter of profiling
before this suggestion is rejected rather than making assertions about
what "should" be happening.
Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/
"In many ways, it's a dull language, borrowing solid old concepts from
many other languages & styles: boring syntax, unsurprising semantics,
few automatic coercions, etc etc. But that's one of the things I like
about it." --Tim Peters on Python, 16 Sep 1993
More information about the Python-Dev