Mysterious delay in Python module loading (fwd)

[Bob Kline]

I originally posted this to the ActiveState Python mailing list, and it
was suggested there that someone on this list might have seen a similar
problem and be able to help.  Since posting the message I have
downloaded the source code for Python and am doing my own hand
instrumentation of the relevant parts (the Python profiler is great for
Python code, but this is happening under the covers in the engine
itself).  So far it seems that the bottleneck is not in the actual
loading of the modules but in finding them (not 100% certain of this
yet, though).  I have confirmed that none of the locations in sys.path
are out on a network (they're all on the local hard disks).  Any
suggestions *very* gratefully received!

Thanks.

-- 
Bob Kline
mailto:bkline at rksystems.com
http://www.rksystems.com

---------- Forwarded message ----------
Date: Mon, 7 Jan 2002 16:55:31 -0500 (EST)
From: Bob Kline <bkline at rksystems.com>
To: activepython at listserv.ActiveState.com
Cc: Peter Zhang <peterz at mail.nih.gov>
Subject: Mysterious delay in Python module loading

I just noticed that on one of our servers Python (ActiveState version
2.1.1, latest build) is taking an order of magnitude longer loading
standard Python modules than is normal.  Most of our development is done
on a 4-way Intel 400MHz box running Windows 2000 Server, latest service
pack.  On this machine, a typical project module loading, say, 10
standard Python modules will finish the loading of those modules in
under a second.  The same project module on the machine we've started
using for some of our database builds is consistently taking more than 8
seconds to load these same standard Python modules.  The newer machine
was running the latest build of AS Python until a few minutes ago, when
I uninstalled that build and replaced it with the same build which was
on the development server (version 2.0.0 build 203, using the same MSI
download package); this didn't solve the problem.

The newer machine is running a much faster (1700 MHz) single Intel
processor.  There's no other load on the machine.  Plenty of memory.
Everything else runs (significantly) faster on this machine than on our
development machine, so it's difficult to believe that we're dealing
with file system problems or slow hard disks.  Everything's on the local
machine (we're not trying to load the modules across the network).  The
newer machine is running the same OS and service pack level as the
development machine.

I've looked at the captured output from loading the modules running
Python with the -v and -d switches, and it doesn't look like there are
any suspicious-looking differences there.

I tried breaking down the load times for the separate Python modules, to
see if one of them were doing something differently from the others, but
the extra load time seems to be spread out fairly evenly between them,
making it look more like a problem with the module loading or finding
process, rather than an anomaly with one of the individual modules.

PYTHONPATH is identical on both machines.  All of our own Python scripts
are byte for byte identical.  All of the .pyc files for our own scripts
have been recompiled on the machine using them.

Nothing relevant appears to be showing up in the Event Logs for the
newer machine.

So, my three questions are:

1. Does this description ring any bells, reminding you of similar
   problem reports for which you might have a solution?

2. What other tools or procedures can I use to delve further into
   this problem to track down its cause myself?

3. What other information can I provide about our configuration
   to provide relevant clues?

-- 
Bob Kline
mailto:bkline at rksystems.com
http://www.rksystems.com


_______________________________________________
ActivePython mailing list
ActivePython at listserv.ActiveState.com
http://listserv.ActiveState.com/mailman/listinfo/activepython