[Python-checkins] r46505 -python/trunk/Tools/pybench/systimes.py
Fredrik Lundh
fredrik at pythonware.com
Tue Jun 6 19:25:23 CEST 2006
M.-A. Lemburg wrote:
>>> Using these APIs, we can get an accuracy of +/- 15.6ms on each
>>> measurement, e.g. if a test tun takes 1 second, we have an accuracy
>>> of +/- 1.56%.
>>
>> here we go again.
>>
>> instead of posting more unqualified guesses, can you *PLEASE* read the
>> posts in the "python benchmarks" thread.
>
> I just did, but couldn't find anything that would allow you to
> call my posting unqualified:
you stated that using rusage gives you a measurement accuracy of +/-
1.56%. assuming that we agree that "accuracy" means "difference between
*actual* value (the number of cycles required to run the benchmark) and
measured value", that's not necessarily true. and as long as you see
varying totals, it's definitely not true. read on.
> If you are running a benchmark on a quiet system, the benchmark
> process *is* the process running most of the time, so your
> comments about the jiffies possibly not getting accounted to
> the benchmark process are not really all that realistic.
nope. consider a second task running with a priority equal to or higher
than your benchmark task. that task can "steal" nearly a full jiffy
every time it is run, but you'll still get a process time that's exactly
the same as the wall time (because your process will be assigned all
jiffies, and will thus appear to run at full speed throughout the entire
benchmark).
however, both the resulting process time and the resulting wall time can
be off by 2x, in a (highly hypothetical) worst case.
> In fact, if you run time.time() vs. resource.getrusage() on
> a Linux box, you'll find that both more or less show the same
> flux in timings - with an error interval of around 15ms.
which is easily explained by "cycle stealers", and is destroying the
benchmark's precision. and as usual, if you don't have precision, you
don't really have accuracy (unless you have a good statistical model,
and enough data to use it; see Andrew's posts for more on that).
</F>
More information about the Python-checkins
mailing list