[Tutor] improving speed using and recalling C functions

[Danny Yoo]

On Fri, Apr 11, 2014 at 1:01 PM, Gabriele Brambilla
<gb.gabrielebrambilla at gmail.com> wrote:
> Yes,
> but I want to make a C extension to run faster a function from
> scipy.interpolate (interp1d)
>
> It woulldn't change anything?


This is precisely why you want to drive your optimization based on
what the profiler is telling you.  Look at the profiler's output
again, closely:

---
         31594963 function calls in 103.708 seconds

   Ordered by: internal time
   List reduced from 47 to 20 due to restriction <20>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   57.133   57.133  103.692  103.692 skymapsI.py:44(mymain)
   176832    9.898    0.000   11.710    0.000 interpolate.py:394(_call_linear)
 18101000    7.808    0.000    7.808    0.000 {method 'write' of 'file' objects}

  1237824    3.794    0.000    3.794    0.000 {numpy.core.multiarray.array}
  1001000    3.610    0.000   38.383    0.000 instruments.py:10(kappa)
   353664    3.314    0.000    3.314    0.000 {method 'reduce' of 'numpy.ufunc'

[cutting some content]
---

About 8% of the time in your program is being spent in interpolate.py.
 But this is in SciPy code, so it is likely difficult to rewrite.
Also note that the amount of time being spent on merely writing the
output is about that much time too!  That's what the profile is
saying, qualitatively.

And on the other hand, the code in skymapsI.mymain is still a target
worthy of your attention.  Compare how much time it was taking before
we started investigating it.  Before, it took 75% of the total runtime
of your program.  We improved upon that a lot with a few small
changes.  But it's still taking 55% of the total time of your
program's running.  If you look at the rest of the profiler's output,
we know that everything else is fairly inconsequential.

That's why we're pushing you to look at the data.  Trust the profiler.
 Work on the thing that is contributing most to the cost of your
program: continue trying to improve the code in skymapsI.mymain.  I am
fairly certain there is still some low-hanging fruit there.

The profile you want to see, eventually, is one where the computation
is being done mostly in numpy code.  But as we can see now, the numpy
code is barely contributing to the runtime.  That's a situation that
needs improvement.

Peter Otten's offer to help you use NumPy more effectively is one you
should take seriously.


Good luck!