[Tutor] Astonishing timing result

[Kent Johnson]

On Wed, Jun 25, 2008 at 2:06 PM, Lie Ryan <lie.1296 at gmail.com> wrote:
> On Wed, 2008-06-25 at 12:56 -0400, Kent Johnson wrote:
>> On Wed, Jun 25, 2008 at 12:05 PM, Lie Ryan <lie.1296 at gmail.com> wrote:
>>
>> >    t_a = min(t_A, t_a)
>> >    t_b = min(t_A, t_b)
>> >    t_c = min(t_A, t_c)
>> >    t_d = min(t_A, t_d)
>>
>> What is this for? It should at least be t_B, t_C, t_D.
>
> A common pitfall in benchmarking is averaging the benchmark result. That
> is WRONG, FLAT WRONG.

Yes, I agree. I missed the outer loop that this is in. But your code
is still WRONG, FLAT WRONG!
 t_b = min( *** t_A ***, t_b) // should be t_B, etc.

>> > ## OUTPUT
>> > # 1.02956604958
>> > # 1.02956604958
>> > # 1.02956604958
>> > # 1.02956604958
>>
>> It's *very easy* to write bogus timing tests, as this thread
>> demonstrates. Some protections:
>> - when comparing different implementations of a function, make sure
>> each implementation returns the correct result by checking the return
>> value.

> Actually the timing is all equal because of the timer's resolution. I
> don't have a high-precision timer on hand.

Or maybe they are all equal because they are all t_A...

Kent