Antoine Pitrou wrote:
[snip]
By the way, the Mako benchmark shows a worrying regression (3x slower) on your new dict implementation.
Take a look at the timeline graph, is it very noisy?
There is a flaw in the benchmarking code in that the runs are not interleaved, so other processes tend to introduce systematic errors.
For example, here is a run of mako (comparing my dict with tip):
### mako ### Min: 0.805583 -> 0.839515: 1.04x slower Avg: 0.831936 -> 0.910184: 1.09x slower Significant (t=-3.25) Stddev: 0.01302 -> 0.11953: 9.1820x larger
It is 9% slower, right? Wrong. Take a look at the timeline: http://tinyurl.com/82l9jna its 1-2% slower, but another process grabs the CPU for some of the iterations.
This should not be a problem for speed.python.org as it will have a dedicated machine, but you need to be careful when benchmarking on your desktop machine.
As an experiment, try benchmarking a python build against itself and see what you get.
Cheers, Mark.