Re: [pypy-dev] Change to the frontpage of speed.pypy.org
In a message of Tue, 08 Mar 2011 18:17:17 +0100, Miquel Torres writes:
you mean this timeline, right?: http://speed.pypy.org/timeline/?ben=3Dspectral-norm
Because the December 22 result is so high, the yaxis maximum goes up to 2.5, thus having less space for the more interesting < 1 range, right?
yes
Regarding mozilla, do you mean this site?: http://arewefastyet.com/ I can see their timelines have some holes, probably failed runs...
I was seeing something else, and I don't have a url. I think that what I was seeing is what they use to make the arewefastyet.com site.
I see a problem with the approach you suggest. Entering an arbitrary maximum yaxis number is not a good thing. I think the onus is there on the benchmark infrastructure to not send results that aren't statistically significant. See Javastats (http://www.elis.ugent.be/en/JavaStats), or ReBench (https://github.com/smarr/ReBench).
I don't think you understand what I want. Sorry if I was unclear. I am fine with the way that the benchmarks are displayed right now, but I want a way to dynamically do there and say, I want to throw away all data that is higher than a certain figure, or lower than a certain one, because right now I am onoy interested in results in a certain range. I'm not looking to change what the benchmark says for everybody who looks at it, or change how it is presented in general. I just want a way to zoom in and only see results in the range that interests me. You and anybody else might have a different range that interests you, and you should be free to get this as well.
Something that can be done on the Codespeed side is to treat differently points that have a too high stddev. In the aforementioned spectral-norm timeline, the stddev "floor" is around 0.0050, while the spike has a 0.30 stddev, much higher. A "strict" mode could be implemented that invalidates or hides statistically unsound data.
The problem is that I want to throw away arbitrary amounts of data regardless of whether they are statistically significant or not, on the basis of I know what I want to see, and this other stuff is getting in the way or being distractingÃ.
Btw., I had written to the arewefastyet guys about the possibility of configuring a Codespeed instance for them. We may yet see collaboration there ;-)
Miquel
Laura
I see. There is an easy solution for that, at least for the moment: enabling zooming. I just did that, and you can now use zooming in a timeline plot to select a narrower yaxis range or just view a particular area in detail. A single click resets the zoom level. If that is not enough, we can discuss a better solution when you have more time. 2011/3/9 Laura Creighton <lac@openend.se>:
In a message of Tue, 08 Mar 2011 18:17:17 +0100, Miquel Torres writes:
you mean this timeline, right?: http://speed.pypy.org/timeline/?ben=3Dspectral-norm
Because the December 22 result is so high, the yaxis maximum goes up to 2.5, thus having less space for the more interesting < 1 range, right?
yes
Regarding mozilla, do you mean this site?: http://arewefastyet.com/ I can see their timelines have some holes, probably failed runs...
I was seeing something else, and I don't have a url. I think that what I was seeing is what they use to make the arewefastyet.com site.
I see a problem with the approach you suggest. Entering an arbitrary maximum yaxis number is not a good thing. I think the onus is there on the benchmark infrastructure to not send results that aren't statistically significant. See Javastats (http://www.elis.ugent.be/en/JavaStats), or ReBench (https://github.com/smarr/ReBench).
I don't think you understand what I want. Sorry if I was unclear. I am fine with the way that the benchmarks are displayed right now, but I want a way to dynamically do there and say, I want to throw away all data that is higher than a certain figure, or lower than a certain one, because right now I am onoy interested in results in a certain range.
I'm not looking to change what the benchmark says for everybody who looks at it, or change how it is presented in general. I just want a way to zoom in and only see results in the range that interests me. You and anybody else might have a different range that interests you, and you should be free to get this as well.
Something that can be done on the Codespeed side is to treat differently points that have a too high stddev. In the aforementioned spectral-norm timeline, the stddev "floor" is around 0.0050, while the spike has a 0.30 stddev, much higher. A "strict" mode could be implemented that invalidates or hides statistically unsound data.
The problem is that I want to throw away arbitrary amounts of data regardless of whether they are statistically significant or not, on the basis of I know what I want to see, and this other stuff is getting in the way or being distractingÃ.
Btw., I had written to the arewefastyet guys about the possibility of configuring a Codespeed instance for them. We may yet see collaboration there ;-)
Miquel
Laura
participants (2)
-
Laura Creighton -
Miquel Torres