[Python-Dev] importlib quest

[Antoine Pitrou]

On Mon, 6 Feb 2012 20:49:48 +0100
Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Mon, 6 Feb 2012 09:57:56 -0500
> Brett Cannon <brett at python.org> wrote:
> > Thanks for any help people can provide me on this now 5 year quest to get
> > this work finished.
> 
> Do you have any plan to solve the performance issue?
> 
> $ ./python -m timeit -s "import sys; mod='struct'" \
>   "__import__(mod); del sys.modules[mod]"
> 10000 loops, best of 3: 75.3 usec per loop
> $ ./python -m timeit -s "import sys; mod='struct'; from importlib import __import__" \
>   "__import__(mod); del sys.modules[mod]"
> 1000 loops, best of 3: 421 usec per loop

The culprit for the overhead is likely to be PathFinder.find_module:

$ ./python -m timeit -s "import sys; mod='struct'; from importlib._bootstrap import _DefaultPathFinder; finder=_DefaultPathFinder" "finder.find_module('struct')"
1000 loops, best of 3: 355 usec per loop
$ ./python -S -m timeit -s "import sys; mod='struct'; from importlib._bootstrap import _DefaultPathFinder; finder=_DefaultPathFinder" "finder.find_module('struct')"
10000 loops, best of 3: 176 usec per loop

Note how it's dependent on sys.path length. On an installed Python with
many additional sys.path entries (e.g. because of distribute-based
module installs), import times will be much worse.

Regards

Antoine.