[Python-Dev] Caching directory files in import.c

James C. Ahlstrom jim@interet.com
Tue, 13 Nov 2001 14:05:11 -0500


I now have benchmarks for the zip import code.  The network drive
is faster than the local drive when using zip!  I plan to update
the PEP to reflect the changes.

Case        Original           Using os.listdir     Zip Uncomp. Zip Compr.
----   --------------------  ---------------------  ----------  ----------
  1   3.2, 2.5, 3.2 -> 1.02  2.3, 2.5, 2.3 -> 0.87  1.66->0.93  1.5->1.07
  2   2.8, 3.9, 3.0 -> 1.32  Same as case 1.
  3   5.7, 5.7, 5.7 -> 5.7   2.1, 2.1, 2.1 -> 1.8   1.25->0.99  1.19->1.13
  4   9.4, 9.4, 9.3 -> 9.35  Same as case 3.

Case 1: Local drive C:, sys.path has its default value.
Case 2: Local drive C:, move the correct directory to the end of sys.path.
Case 3: Network  drive, sys.path has its default value.
Case 4: Network  drive, move the correct directory to the end of sys.path.

Benchmarks were performed on a Pentium 4 clone, 1.4 GHz, 256 Meg.
The machine was running Windows 2000 with a Linux/Samba network server.
Times are in seconds, and are the time to import about 100 modules from Lib.
"Uncomp" means uncompressed zip archive, "Compr" means compressed.
The Python version is 2.2a3.

Initial times are after a re-boot of the system; the time after "->" is the
time after repeated runs.  Times to import from C: after a re-boot are
rather highly variable for the "Original" case, but are more realistic.

JimA