(More) Re: Is __import__ known to be slow in windows?

[John Machin]

On Dec 1, 2:12 pm, Joshua Kugler <jkug... at bigfoot.com> wrote:
> Ok, so we have this code:
>
> t = timeit.Timer(stmt='r()', setup='from __main__ import r')
>
> sys.path.insert(0,'/path/to/code')
>
> def r():
>     for m in ['three','module','names']:
>         try:
>             x = __import__(m)

Have you ever tried print m, x.__file__ here to check that the modules
are being found where you expect them to be found?

>         except ImportError, e:
>             if not e.message.startswith('No module named'):
>                 raise

Why are you blindly ignoring the possibly that the module is not
found? Note that loading a module after it is found is not a zero-cost
operation.

>     x = None
>
> Each of those three module names is a directory under /path/to/code with an
> empty __init_.py.

Is there anything else in the /path/to/code directory?

>
> On Linux, the run of that code for 1000 iterations takes 0.014 CPU seconds
> with 0.007 of that spent in __import__().
>
> On Windows (with on-access checking turned off), 1000 iterations takes 7.9
> seconds, with 7.851 seconds of that spent in __import__().
>
> Let's try something...let's modify the code a bit:
>
> sys.path = ['/path/to/code']
>
> def r():
>     for m in ['three','module','names']:
>         x = __import__(m)

Have you tried print m, x.__file__ here to check that the modules are
being found where you expect them to be found?

>     x = None
>
> cProfile.run('t.timeit(number=%s)' % number, 'stat_file')
> p = pstats.Stats('stat_file')
> p.sort_stats('time', 'cum').print_stats(.5)
>
> Now, with only my directory in sys.path, the run times are:
> Linux: 0.0013 (0.006 spent in __import__)
> Windows: 0.0012 (0.007 spent in __import__)
>
> So, it appears walking the directory trees is what is costing the time.

Call me crazy, but: First experiment, sys.path was ['/path/to/code',
'', etc etc]. Now it's only ['/path/to/code']. How can that still load
properly but run faster?? What directory tree walking?? Should be none
if the modules are found in /path/to/code. Are you sure the modules
are always found in /path/to/code? What is in the current directory
[matched by '' in sys.path]?