[pypy-dev] Change to the frontpage of speed.pypy.org

Janzert janzert at janzert.com
Tue Mar 8 23:11:17 CET 2011


On 3/8/2011 12:17 PM, Miquel Torres wrote:
>
> I see a problem with the approach you suggest. Entering an arbitrary
> maximum yaxis number is not a good thing. I think the onus is there on
> the benchmark infrastructure to not send results that aren't
> statistically significant. See Javastats
> (http://www.elis.ugent.be/en/JavaStats), or ReBench
> (https://github.com/smarr/ReBench).
>

I think the problem is that even a statistically significant result can 
be arbitrarily bad (e.g. if a major regression is checked in). It does 
seem like it would be really nice to have the option of viewing with a 
scale such that the maximum displayed is something like say 2 standard 
deviations above the mean of displayed results.

Janzert




More information about the Pypy-dev mailing list