[Tutor] improving speed using and recalling C functions

Danny Yoo dyoo at hashcollision.org
Thu Apr 10 23:14:25 CEST 2014


Hi Gabriele,


I should probably have pointed you to:

    https://docs.python.org/2/library/profile.html#instant-user-s-manual

instead.


Here is an example that uses the cProfile module.  Let's say that I'm
trying to pinpoint where something is going slow in some_program():

#########################################################
import cProfile

def slow_string_mul(w, n):
  s = ""
  for x in xrange(n):
    s = slow_append(s, w)
  return s

def slow_append(s, w):
  return s + w

def fast_string_mul(w, n):
  return w * n

def some_program(w, n):
  print slow_string_mul(w, n) == fast_string_mul(w, n)

## Try running the operation, and let cProfile report stats."
cProfile.run('some_program("testing", 50000)', None, 'time')
#########################################################


We tell cProfile.run to execute some_program(), sorting its
measurements by time.  You might save the collected statistics to a
file, but here I have not, and have cProfile.run() just report the
results after the program finishes.



Here's what comes back from cProfile's report:

#########################################################
$ python test_profile.py
True
         50005 function calls in 1.422 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    50000    1.225    0.000    1.225    0.000 test_profile.py:9(slow_append)
        1    0.197    0.197    1.422    1.422 test_profile.py:3(slow_string_mul)
        1    0.000    0.000    1.422    1.422 test_profile.py:15(some_program)
        1    0.000    0.000    0.000    0.000
test_profile.py:12(fast_string_mul)
        1    0.000    0.000    1.422    1.422 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 {method 'disable' of
'_lsprof.Profiler' objects}
#########################################################

This is telling me that slow_append is being called 50000 times, which
in hindsight sounds right.  It's taking the majority of the time of
this program, which is intuitively true, as I wrote it in a way to do
a very naive string-appending operation, which gets more expensive the
longer the string grows.



The point of examples is to show simple uses of the libraries.  But
you will have to adapt the examples to work for your context.


More information about the Tutor mailing list