Antoine Pitrou wrote:

[snip]

By the way, the Mako benchmark shows a worrying regression (3x slower) on your new dict implementation.

Take a look at the timeline graph, is it very noisy?

There is a flaw in the benchmarking code in that the runs are not interleaved, so other processes tend to introduce systematic errors.

For example, here is a run of mako (comparing my dict with tip):

### mako ### Min: 0.805583 -> 0.839515: 1.04x slower Avg: 0.831936 -> 0.910184: 1.09x slower Significant (t=-3.25) Stddev: 0.01302 -> 0.11953: 9.1820x larger

It is 9% slower, right? Wrong. Take a look at the timeline: http://tinyurl.com/82l9jna its 1-2% slower, but another process grabs the CPU for some of the iterations.

This should not be a problem for speed.python.org as it will have a dedicated machine, but you need to be careful when benchmarking on your desktop machine.

As an experiment, try benchmarking a python build against itself and see what you get.

Cheers, Mark.