[Numpy-discussion] "import numpy" performance

[Andrew Dalke]

On Jul 2, 2012, at 10:33 PM, David Cournapeau wrote:
> On Mon, Jul 2, 2012 at 8:17 PM, Andrew Dalke <dalke at dalkescientific.com> wrote:
>> In July of 2008 I started a thread about how "import numpy"
>> was noticeably slow for one of my customers.
   ...
>> I managed to get the import time down from 0.21 seconds to
>> 0.08 seconds.
> 
> I will answer to your other remarks later, but 0.21 sec to import
> numpy is very slow, especially on a recent computer. It is 0.095 sec
> on my mac, and 0.075 sec on a linux VM on the same computer (both hot
> cache of course).

That quote was historical review from 4 years ago. I described
the problems I had then, the work-around solution I implemented,
and my additional work to see if I could identify ways which
would have kept me from needing to find a work-around solution.

I then described why I have not worked on this problem for the
last four years, and what has changed to make me interested
in it again. That included current details, such as how "import numpy"
with a warm cache takes 0.083 seconds on my Mac.

> importing multiarray.so only is negligible for me (i.e. difference
> between python -c "import multiarray" and python -c "" is
> statistically insignificant).

The NumPy initialization is being done in C++ code through
"import_array()". That C function does (among other things)

  PyObject *numpy = PyImport_ImportModule("numpy.core.multiarray");

so the relevant timing test is more likely:

% time python -c 'import numpy.core.multiarray'
0.086u 0.031s 0:00.12 91.6%	0+0k 0+0io 0pf+0w
% time python -c 'import numpy.core.multiarray'
0.083u 0.031s 0:00.11 100.0%	0+0k 0+0io 0pf+0w
% time python -c 'import numpy.core.multiarray'
0.083u 0.030s 0:00.12 91.6%	0+0k 0+0io 0pf+0w


I do not know how to run the timing test you did, as I get:

% python -c "import multiarray"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named multi array


> I would check external factors, like the size of your sys.path as well.

I have checked that, and inspected the output of python -v -v.


				Andrew
				dalke at dalkescientific.com