[Python-Dev] Importing .pyc in -O mode and vice versa

[Aahz]

On Sun, Nov 05, 2006, "Martin v. L?wis" wrote:
> Greg Ewing schrieb:
>> Fredrik Lundh wrote:
>>> 
>>> well, from a performance perspective, it would be nice if Python looked 
>>> for *fewer* things, not more things.
>> 
>> Instead of searching for things by doing a stat call for each
>> possible file name, would it perhaps be faster to read the contents
>> of all the directories along sys.path into memory and then go
>> searching through that?
>
> That should never be better: the system will cache the directory
> blocks, also, and it will do a better job than Python will.

Maybe so, but I recently dealt with a painful bottleneck in Python code
caused by excessive stat() calls on a directory with thousands of files,
while the os.listdir() function was bogging things down hardly at all.
Granted, Python bytecode was almost certainly the cause of much of the
overhead, but I still suspect that a simple listing will be faster in C
code because of fewer system calls.  It should be a matter of profiling
before this suggestion is rejected rather than making assertions about
what "should" be happening.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"In many ways, it's a dull language, borrowing solid old concepts from
many other languages & styles:  boring syntax, unsurprising semantics,
few automatic coercions, etc etc.  But that's one of the things I like
about it."  --Tim Peters on Python, 16 Sep 1993