[Python-Dev] Startup time

[Jeremy Hylton]

I don't know if this thread came to any conclusion.  I did the same
strace that everyone else has reported on, and I've included a summary
of the results here.  I have one directory in my PYTHONPATH, which
affects the number of directories that are searched for each imported
module.

Comparing Python 2.3 current CVS with Python 2.2 CVS, I see the
following system call counts (limited to top 6 in 2.3).

         2.3   2.2
open     305   104
stat64   102    44
fstat64   74    34
read      71    30
rt_sig... 69    68
brk       62    74

When a single module is imported from the standard library, Python 2.2
looks in 10 different places.  Specifically, it looks for five different
files in two different directories -- PYTHONPATH and the std library
directory.  For files that aren't found (e.g. sitecustomize), it looks
in 25 places (5 files x 5 directories).  Interesting to note that
PYTHONPATH directory is not searched for sitecustomize.

In Python 2.3, the standard library module requires 15 lookups because
/usr/local/lib/python23.zip is added to the path before the std library
directory.  The failed lookup of sitecustomize takes 35 lookups, because
PYTHONPATH and python23.zip are now on the path.

The list of attempted imports is much larger in 2.3 than in 2.2.

 -- 2.3 --                     -- 2.2 --
                             __future__
codecs
copy_reg                     copy_reg
encodings
encodings/__init__
encodings/aliases
encodings/iso_8859_15
exceptions
linecache
os                           os
posixpath                    posixpath
re
site                         site
sitecustomize                sitecustomize
sre
sre_compile
sre_constants
sre_parse
stat                         stat
string
strop
types                        types
UserDict                     UserDict
warnings
  22 total                     7 total

The increase in open, stat64, and fstat64 all seem consistent with a 3x
increase in the number of modules searched for.

The use of re in the warnings module seems the primary culprit, since it
pulls in re, sre and friends, string, and strop.

Jeremy