[pypy-dev] Change to the frontpage of speed.pypy.org
Janzert
janzert at janzert.com
Tue Mar 8 23:11:17 CET 2011
On 3/8/2011 12:17 PM, Miquel Torres wrote:
>
> I see a problem with the approach you suggest. Entering an arbitrary
> maximum yaxis number is not a good thing. I think the onus is there on
> the benchmark infrastructure to not send results that aren't
> statistically significant. See Javastats
> (http://www.elis.ugent.be/en/JavaStats), or ReBench
> (https://github.com/smarr/ReBench).
>
I think the problem is that even a statistically significant result can
be arbitrarily bad (e.g. if a major regression is checked in). It does
seem like it would be really nice to have the option of viewing with a
scale such that the maximum displayed is something like say 2 standard
deviations above the mean of displayed results.
Janzert
More information about the Pypy-dev
mailing list