[Numpy-discussion] "import numpy" is slow

[Robert Kern]

On Mon, Jun 30, 2008 at 18:32, Andrew Dalke <dalke at dalkescientific.com> wrote:
> (Trying again now that I'm subscribed.  BTW, there's no link to the
> subscription page from numpy.scipy.org .)
>
>
> The initial 'import numpy' loads a huge number of modules, even when
> I don't need them.
>
> Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
> [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import sys
>  >>> len(sys.modules)
> 28
>  >>> import numpy
>  >>> len(sys.modules)
> 256
>  >>> len([s for s in sorted(sys.modules) if 'numpy' in s])
> 127
>  >>> numpy.__version__
> '1.1.0'
>  >>>
>
> As a result, I assume that's the reason my program's startup cost is
> quite high.
>
> [josiah:~/src/fp] dalke% time python -c 'a=4'
> 0.014u 0.038s 0:00.05 80.0%     0+0k 0+1io 0pf+0w
> [josiah:~/src/fp] dalke% time python -c 'import numpy'
> 0.161u 0.279s 0:00.44 97.7%     0+0k 0+9io 0pf+0w
>
> My total runtime is something like 1.4 seconds, and the only thing
> I'm using NumPy for is to make an array of doubles that I can pass to
> a C extension.  (I could use the array module or ctypes, but figured
> numpy is more useful for downstream code.)
>
> Why does numpy/__init__.py need to import all of these other modules
> and submodules?

Strictly speaking, there is no *need* for any of it. It was a judgment
call trading off import time for the convenience in fairly typical use
cases which do use functions across the breadth of the library. Your
use case isn't so typical and so suffers on the import time end of the
balance.

> Any chance of cutting down on the number, in order
> to improve startup costs?

Not at this point in time, no. That would break too much code.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco