[pypy-dev] speed.pypy.org launched

[Stefan Behnel]

Carl Friedrich Bolz, 25.02.2010 18:38:
> On 02/25/2010 04:10 PM, Miquel Torres wrote:
>> As some of you already know, there is a new performance site online:
>> speed.pypy.org.
> [...]
> 
> I'm quite impressed, this is very cool work and a good improvement over 
> the current plots. Thanks for doing this!
> 
> One thing that I would really find useful are error bars in the timeline 
> view. This would help us judge whether up-and-down movements are within 
> the margin of randomness, or whether it is a real effect. I don't know 
> how annoying they are to implement though, no clue whether the plot lib 
> supports that. There should be enough information about errors in the 
> json files, as far as I remember.

The standard deviation is commonly considered meaningless for benchmark
results. All that really matters is the fastest run, everything else is
just fog.

Read the docs on timeit, for example.

Stefan