[Python-Dev] Hotshot

[Walter Dörwald]

Fred L. Drake, Jr. wrote:

> Skip Montanaro writes:
>  > It seems to me it might be simpler to just write the profile file through
>  > the gzip module and teach the hotshot.stats.load() function to recognize
>  > such files and uncompress accordingly.
> 
> The problem with this is that the low-level log reader and writer are
> in C rather than Python (using fopen(), fwrite(), etc.).  We could
> probably work out a reader that gets input buffers from an arbitrary
> Python file object, and maybe we could handle writing that way, but
> that does change the cost of each write.  HotShot tries to compensate
> for that, but it's unclear how successful that is.

OK, I've checked both the old profile.py and the new hotshotmain.py
(Thanks for checking it in, Skip) with one of my scripts. The script
runs for:
    real   0m6.192s
    user   0m6.010s
    sys    0m0.160s
when running standalone.

Using the old profile.py I get the following:
    real   0m46.892s
    user   0m43.430s
    sys    0m0.970s

Running with the new hotshotmain.py gives the following run times:
    real   1m6.873s
    user   1m5.220s
    sys    0m0.840s

The size of hotshop.prof is 5104590 bytes. After gzipping it with
"gzip -9" the size of hotshop.prof drops to 562467 bytes. So gzipping
might help, but dropping filesize from 1 GB to 100 MB still doesn't
sound so convincing. And I wonder what would happen to run time.

Bye,
    Walter Dörwald