ncoghlan at gmail.com
Mon Nov 21 23:02:38 CET 2005
Jim Jewett wrote:
> Jeremy Hylton jeremy at alum.mit.edu
>> Should lsprof be added to the standard distribution?
>> Should hotshot be removed from the standard distribution?
>> These two aren't at all related, unless you believe that two is the
>> maximum number of profiles allowed per Python distribution.
> One is a better number.
> ("There should be one-- and preferably only one --obvious way to do it.")
> Adding a second (let alone third) module to the stdlib to do
> the same thing just makes the documentation bulkier,
> and makes the "where do I start" problem harder for beginners.
> And yes, I think beginners are the most important audience
> here; anyone sufficiently comfortable with python to make an
> intelligent choice between different code profilers is probably
> also able to install 3rd-party modules anyway.
Chiming in as a user of 'profile', that has also attempted to use hotshot. . .
I used profile heavily when we working on the implementation of the decimal
module, trying to figure out where the bottlenecks were (e.g., profile showed
that converting to integers to do arithmetic and back to sequences to do
rounding was a net win, despite the conversion costs in switching back and
forth between the two formats).
I tried using hotshot to do the same thing (profiled runs of the arithmetic
tests took a *long* time), and found the results to be well-nigh useless (I
seem to recall it was related to the fact that profile separated out C calls,
while hotshot didn't).
So my experience of hotshot has been "sure it's slightly less invasive, but it
doesn't actually work". If hotshot can be replaced with something that
actually works as intended, or if lsprof can be added in a way that is more
closely coupled with profile (so that there is a clear choice between "less
invasive but less detailed results" and "more detailed results but more
invasive during execution"), I'd be quite happy.
If a statistical profiler was later added to round out the minimally invasive
end, that actually makes for a decent profiling toolset:
1. Use the statistical profiler to identify potential problem areas
2. Use hotshot/lsprof to further analyse the potential problem areas
3. Use profile to get detailed results on the bottlenecks
Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia
More information about the Python-Dev