[Python-Dev] Caching directory files in import.c
James C. Ahlstrom
jim@interet.com
Tue, 13 Nov 2001 14:05:11 -0500
I now have benchmarks for the zip import code. The network drive
is faster than the local drive when using zip! I plan to update
the PEP to reflect the changes.
Case Original Using os.listdir Zip Uncomp. Zip Compr.
---- -------------------- --------------------- ---------- ----------
1 3.2, 2.5, 3.2 -> 1.02 2.3, 2.5, 2.3 -> 0.87 1.66->0.93 1.5->1.07
2 2.8, 3.9, 3.0 -> 1.32 Same as case 1.
3 5.7, 5.7, 5.7 -> 5.7 2.1, 2.1, 2.1 -> 1.8 1.25->0.99 1.19->1.13
4 9.4, 9.4, 9.3 -> 9.35 Same as case 3.
Case 1: Local drive C:, sys.path has its default value.
Case 2: Local drive C:, move the correct directory to the end of sys.path.
Case 3: Network drive, sys.path has its default value.
Case 4: Network drive, move the correct directory to the end of sys.path.
Benchmarks were performed on a Pentium 4 clone, 1.4 GHz, 256 Meg.
The machine was running Windows 2000 with a Linux/Samba network server.
Times are in seconds, and are the time to import about 100 modules from Lib.
"Uncomp" means uncompressed zip archive, "Compr" means compressed.
The Python version is 2.2a3.
Initial times are after a re-boot of the system; the time after "->" is the
time after repeated runs. Times to import from C: after a re-boot are
rather highly variable for the "Original" case, but are more realistic.
JimA