[Python-Dev] C extension import time

[Stefan Krah]

Antoine Pitrou <solipsis at pitrou.net> wrote:
> Try the following:
> 
> $ ./python -m timeit -s "modname='decimal'; import sys" \
>   "__import__(modname); del sys.modules[modname]"
> 1000 loops, best of 3: 2.21 msec per loop
> 
> $ ./python -m timeit -s "modname='_decimal'; import sys" \
>   "__import__(modname); del sys.modules[modname]"
> 10000 loops, best of 3: 112 usec per loop
> 
> 
> Now compare with the time taken to simply find the module loader:
> 
> $ ./python -m timeit -s "modname='_decimal'; import importlib" \
>   "importlib.find_loader(modname)"
> 10000 loops, best of 3: 87.2 usec per loop
> 
> 
> So find_loader() is the bigger contributor here.

I'm getting about the same values as above. I may be misunderstanding
something, but I wanted to reduce the difference between the 2.21 msec
and the 112 usec.


So the first step I tried is something horrible (typing from memory):

try:
    import _decimal
except ImportError:
    <whole of decimal.py here !!!>
else:
    from _decimal import *

That way the 2.21 msec are reduced to 912 usec, but the indentation is
an abomination.


Another radical way would be to have importlib detect that the C version
is present and load it in the first place. For _decimal this should work,
since it's self-contained. For other modules probably not, so there would
need to be some way of distiguishing between modules that are self-contained
and modules that are not.


Stefan Krah