[Tutor] improving speed using and recalling C functions
Gabriele Brambilla
gb.gabrielebrambilla at gmail.com
Fri Apr 11 00:51:48 CEST 2014
Hi,
I get this result:
Thu Apr 10 17:35:53 2014 restats
21071736 function calls in 199.883 seconds
Ordered by: internal time
List reduced from 188 to 10 due to restriction <10>
ncalls tottime percall cumtime percall filename:lineno(function)
1 149.479 149.479 199.851 199.851 skymaps5.py:16(mymain)
18101000 28.682 0.000 28.682 0.000 {method 'write' of 'file'
objects}
33044 5.470 0.000 6.444 0.000
interpolate.py:394(_call_linear)
230000 2.272 0.000 21.279 0.000 instruments.py:10(kappa)
231328 2.120 0.000 2.120 0.000 {numpy.core.multiarray.array}
33044 1.719 0.000 3.836 0.000
interpolate.py:454(_check_bounds)
66088 1.611 0.000 1.611 0.000 {method 'reduce' of
'numpy.ufunc'
objects}
33044 1.146 0.000 11.623 0.000 interpolate.py:443(_evaluate)
33044 1.120 0.000 5.542 0.000 interpolate.py:330(__init__)
33044 0.659 0.000 2.329 0.000 polyint.py:82(_set_yi)
the major time is required by mymain that is the whole program.
the write on file is an operation that I do in the end but if I increase
the number of data it doesn't increase (I tried doubing the sample of
values and I know why it's behaving in this way)
the third are interpolate and kappa: the two functions (one called inside
the other one)
So they are the ones that are taking time.
Now what can I do?
thanks
Gabriele
2014-04-10 17:14 GMT-04:00 Danny Yoo <dyoo at hashcollision.org>:
> Hi Gabriele,
>
>
> I should probably have pointed you to:
>
> https://docs.python.org/2/library/profile.html#instant-user-s-manual
>
> instead.
>
>
> Here is an example that uses the cProfile module. Let's say that I'm
> trying to pinpoint where something is going slow in some_program():
>
> #########################################################
> import cProfile
>
> def slow_string_mul(w, n):
> s = ""
> for x in xrange(n):
> s = slow_append(s, w)
> return s
>
> def slow_append(s, w):
> return s + w
>
> def fast_string_mul(w, n):
> return w * n
>
> def some_program(w, n):
> print slow_string_mul(w, n) == fast_string_mul(w, n)
>
> ## Try running the operation, and let cProfile report stats."
> cProfile.run('some_program("testing", 50000)', None, 'time')
> #########################################################
>
>
> We tell cProfile.run to execute some_program(), sorting its
> measurements by time. You might save the collected statistics to a
> file, but here I have not, and have cProfile.run() just report the
> results after the program finishes.
>
>
>
> Here's what comes back from cProfile's report:
>
> #########################################################
> $ python test_profile.py
> True
> 50005 function calls in 1.422 seconds
>
> Ordered by: internal time
>
> ncalls tottime percall cumtime percall filename:lineno(function)
> 50000 1.225 0.000 1.225 0.000
> test_profile.py:9(slow_append)
> 1 0.197 0.197 1.422 1.422
> test_profile.py:3(slow_string_mul)
> 1 0.000 0.000 1.422 1.422
> test_profile.py:15(some_program)
> 1 0.000 0.000 0.000 0.000
> test_profile.py:12(fast_string_mul)
> 1 0.000 0.000 1.422 1.422 <string>:1(<module>)
> 1 0.000 0.000 0.000 0.000 {method 'disable' of
> '_lsprof.Profiler' objects}
> #########################################################
>
> This is telling me that slow_append is being called 50000 times, which
> in hindsight sounds right. It's taking the majority of the time of
> this program, which is intuitively true, as I wrote it in a way to do
> a very naive string-appending operation, which gets more expensive the
> longer the string grows.
>
>
>
> The point of examples is to show simple uses of the libraries. But
> you will have to adapt the examples to work for your context.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20140410/74577292/attachment-0001.html>
More information about the Tutor
mailing list