Testing for performance regressions
Paddy
paddy3118 at googlemail.com
Tue Apr 5 00:17:18 EDT 2011
In an extended case when you try and capture how a function works over a range of inputs, you might want to not assume some relationship between input size and time, as this mnight limit your ability to change algorithms and still have acceptable performance.
I.e. instead of this:
input_range = (MIN, AVERAGE, MAX)
for i in inpute_range:
..baseline = Timer(simple_func(i)).timeit()
..time_taken = Timer(my_func(i)).timeit()
..assert time_taken <= simple_relation(i) * baseline
It might be better to do this:
input_range = (MIN_R, AVERAGE_R, MAX_R)
time_ranges = (MIN_T, AVERAGE_T, MAX_T)
for i,t in zip(inpute_range, time_ranges):
..baseline = Timer(simple_func(i)).timeit()
..time_taken = Timer(my_func(i)).timeit()
..assert time_taken <= t * baseline
This comes from electronic circuit design where designs must be proven to work over a range for different values for example voltage ranges and temperature ranges.
The action of the function being timed might not be simple, for example if my_func swapped algorithms depending on its input to favour the average case.
More information about the Python-list
mailing list