[Tutor] improving speed using and recalling C functions

Gabriele Brambilla gb.gabrielebrambilla at gmail.com
Fri Apr 11 00:51:48 CEST 2014


Hi,

I get this result:
Thu Apr 10 17:35:53 2014    restats

         21071736 function calls in 199.883 seconds

   Ordered by: internal time
   List reduced from 188 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1  149.479  149.479  199.851  199.851 skymaps5.py:16(mymain)
 18101000   28.682    0.000   28.682    0.000 {method 'write' of 'file'
objects}

    33044    5.470    0.000    6.444    0.000
interpolate.py:394(_call_linear)
   230000    2.272    0.000   21.279    0.000 instruments.py:10(kappa)
   231328    2.120    0.000    2.120    0.000 {numpy.core.multiarray.array}
    33044    1.719    0.000    3.836    0.000
interpolate.py:454(_check_bounds)
    66088    1.611    0.000    1.611    0.000 {method 'reduce' of
'numpy.ufunc'
objects}
    33044    1.146    0.000   11.623    0.000 interpolate.py:443(_evaluate)
    33044    1.120    0.000    5.542    0.000 interpolate.py:330(__init__)
    33044    0.659    0.000    2.329    0.000 polyint.py:82(_set_yi)

the major time is required by mymain that is the whole program.
the write on file is an operation that I do in the end but if I increase
the number of data it doesn't increase (I tried doubing the sample of
values and I know why it's behaving in this way)
the third are interpolate and kappa: the two functions (one called inside
the other one)

So they are the ones that are taking time.

Now what can I do?

thanks

Gabriele



2014-04-10 17:14 GMT-04:00 Danny Yoo <dyoo at hashcollision.org>:

> Hi Gabriele,
>
>
> I should probably have pointed you to:
>
>     https://docs.python.org/2/library/profile.html#instant-user-s-manual
>
> instead.
>
>
> Here is an example that uses the cProfile module.  Let's say that I'm
> trying to pinpoint where something is going slow in some_program():
>
> #########################################################
> import cProfile
>
> def slow_string_mul(w, n):
>   s = ""
>   for x in xrange(n):
>     s = slow_append(s, w)
>   return s
>
> def slow_append(s, w):
>   return s + w
>
> def fast_string_mul(w, n):
>   return w * n
>
> def some_program(w, n):
>   print slow_string_mul(w, n) == fast_string_mul(w, n)
>
> ## Try running the operation, and let cProfile report stats."
> cProfile.run('some_program("testing", 50000)', None, 'time')
> #########################################################
>
>
> We tell cProfile.run to execute some_program(), sorting its
> measurements by time.  You might save the collected statistics to a
> file, but here I have not, and have cProfile.run() just report the
> results after the program finishes.
>
>
>
> Here's what comes back from cProfile's report:
>
> #########################################################
> $ python test_profile.py
> True
>          50005 function calls in 1.422 seconds
>
>    Ordered by: internal time
>
>    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>     50000    1.225    0.000    1.225    0.000
> test_profile.py:9(slow_append)
>         1    0.197    0.197    1.422    1.422
> test_profile.py:3(slow_string_mul)
>         1    0.000    0.000    1.422    1.422
> test_profile.py:15(some_program)
>         1    0.000    0.000    0.000    0.000
> test_profile.py:12(fast_string_mul)
>         1    0.000    0.000    1.422    1.422 <string>:1(<module>)
>         1    0.000    0.000    0.000    0.000 {method 'disable' of
> '_lsprof.Profiler' objects}
> #########################################################
>
> This is telling me that slow_append is being called 50000 times, which
> in hindsight sounds right.  It's taking the majority of the time of
> this program, which is intuitively true, as I wrote it in a way to do
> a very naive string-appending operation, which gets more expensive the
> longer the string grows.
>
>
>
> The point of examples is to show simple uses of the libraries.  But
> you will have to adapt the examples to work for your context.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20140410/74577292/attachment-0001.html>


More information about the Tutor mailing list