Yeah, that can be seen in the fact that chaos's std dev for pypy-c is not as large as for pypy-c-jit, in fact it is perfectly normal.<div><div><br></div><div>Btw. for easily spotting big std dev values maybe they should be highlighted in red. What would a maximum reasonable value for std dev would be (compared to total time)?</div>
<div><br><br><div class="gmail_quote">2010/4/20 Maciej Fijalkowski <span dir="ltr"><<a href="mailto:fijall@gmail.com">fijall@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi Carl, Hi Miquel.<br>
<br>
Cool job!<br>
<div class="im"><br>
On Tue, Apr 20, 2010 at 12:29 PM, Carl Friedrich Bolz <<a href="mailto:cfbolz@gmx.de">cfbolz@gmx.de</a>> wrote:<br>
> Hi Miquel,<br>
><br>
> On 04/20/2010 07:14 PM, Miquel Torres wrote:<br>
>> Changes you may notice are geared towards easing the identification of<br>
>> the cause of performance changes:<br>
>> - Std deviation was added to the DB, overview table and timeline<br>
>> tooltips. This way we can rule out fluctuations in the measuring as a<br>
>> cause for a big change. It has to be pointed out though, that this is<br>
>> only as useful as the way the std dev is being computed. (there you go,<br>
>> Carl Friedrich ;-)<br>
><br>
> Yay, that's incredibly cool! Let's hope that eventually the graph<br>
> library supports errors too, so we can add them graphically. Anyway, I<br>
> already found some fun things about the benchmarks, so thanks a lot!<br>
> (e.g. the std dev of chaos is very large, which is not really a good thing)<br>
<br>
</div>Of course it's large, because as mentioned above the way we compute it<br>
doesn't make sense. We have an average over consecutive runs which<br>
include warmup (less and less so over the course).<br>
<br>
Cheers,<br>
fijal<br>
<div><div></div><div class="h5">_______________________________________________<br>
<a href="mailto:pypy-dev@codespeak.net">pypy-dev@codespeak.net</a><br>
<a href="http://codespeak.net/mailman/listinfo/pypy-dev" target="_blank">http://codespeak.net/mailman/listinfo/pypy-dev</a><br>
</div></div></blockquote></div><br></div></div>