[Tutor] script too slow

[Magnus Lycka]

At Mon, 24 Feb 2003 18:01:58 -0500, Paul Tremblay wrote:
>Where to I put the 'profile.run statment? Do I include it as part of the
>regular script, or do I write a completely different script to profile
>things? I have looked at the library documentation with no luck.

I've just successfully used the profiler to reduce runtime
for a use case from 426 to 55 seconds, so I guess I can
afford to provide a hint. I've done like this:

if __name__=='__main__':
     import sys
     if len(sys.argv) > 2:
         import profile
         profile.run('MainFrame.main("%s")' % sys.argv[1], sys.argv[2])
     elif len(sys.argv) == 2:
         MainFrame.main(sys.argv[1])
     else:
         MainFrame.main()


MainFrame.main() is a function in a module--whatever you would use to
start your code. In this case, I use more command line parameters.
sys.argv[0] is the name of the script you just launched, so I don't
use that. sys.argv[1] is another optional parameter which is not
related to profiling. But if I supply one more argument, sys.argv[2],
I will use this as the name for a profiling file, and run profiling.

(Of course you can do this in different ways, with a flag that you
read with the getopt module, with another script or whatever. Or you
can start python interactively, import your module (if it uses the
'if __name__ ==  "__main__":' guard so that it won't run at once
if it is imported.)

Anyway, the block after "if len(sys.argv) > 2:" is for profiling in
my code. After the program finishes, I'll open an interactive python
session and do something like

import pstats
p = pstats.Stats('filename.dat')
p.strip_dirs()
p.sort_stats('cumulative').print_stats(20)

But now I've changed it to using the new hotshot profiler, which is
much faster. The problem is that for me it takes ages to read the
stats file. Then it looks like this:

...
     if len(sys.argv) > 2:
         import hotshot
         profiler = hotshot.Profile(sys.argv[2])
         profiler.run('MainFrame.main("%s")' % sys.argv[1])
         profiler.close()
...


import hotshot.stats
p = hotshot.stats.load('filename.dat')
p.strip_dirs()
p.sort_stats('cumulative').print_stats(20)


Good luck!


-- 
Magnus Lycka, Thinkware AB
Alvans vag 99, SE-907 50 UMEA, SWEDEN
phone: int+46 70 582 80 65, fax: int+46 70 612 80 65
http://www.thinkware.se/  mailto:magnus@thinkware.se