On Fri, Jun 10, 2016 at 11:22:42PM +0200, Victor Stinner wrote:
2016-06-10 20:47 GMT+02:00 Meador Inge <meadori@gmail.com>:
Apologies in advance if this is answered in one of the links you posted, but out of curiosity was geometric mean considered?
In the compiler world this is a very common way of aggregating performance results.
FYI I chose to store all timings in the JSON file. So later, you are free to recompute the average differently, compute other statistics, etc.
I saw that the CPython benchmark suite has an *option* to compute the geometric mean. I don't understand well the difference with the arithmeric mean.
Is the geometric mean recommended to aggregate results of different (unrelated) benchmarks, or also even for multuple runs of a single benchmark?
The Wikipedia article discusses this, but sits on the fence and can't decide whether using the gmean for performance results is a good or bad idea: https://en.wikipedia.org/wiki/Geometric_mean#Properties Geometric mean is usually used in finance for averaging rates of growth: https://www.math.toronto.edu/mathnet/questionCorner/geomean.html If you express your performances as speeds (as "calculations per second") then the harmonic mean is the right way to average them. -- Steve