To see if the bad iterative_count and threaded_count results were consistently bad, I ran the benchmark suite on my MacBook Pro to see how &quot;reliable&quot; the benchmarks were. The output is below.<div><br></div><div>Basically 6 benchmarks (regex_effbot, queens, startup_nosite, iterative_count, threaded_count, and telco) had a variance of more than 15% performance between my 2 computers, although queens, iterative_count, and threaded_count were the only ones that swung between neutral/good to bad depending on the machine (the rest either want from bad to very bad, or very good to more very good).</div>

<div><br></div><div>And before Antoine asks, I added a ``sys.modules[&#39;markupsafe&#39;] = None` line to the mako_v2 benchmark locally. =) Still need to either explicitly block it or emit a warning in the code in the repo.</div>

<div><br></div><div><br></div><div>#########################################</div><div><br></div><div><br></div><div><div>Report on Darwin Darwin Kernel Version 12.2.0: Sat Aug 25 00:48:52 PDT 2012; root:xnu-2050.18.24~1/RELEASE_X86_64 x86_64 i386</div>

<div>Total CPU cores: 8</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">2to3</span> ###</div><div>10.321463 -&gt; 9.525119: 1.08x faster</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">call_method</span> ###</div>

<div>Min: 0.466812 -&gt; 0.417812: 1.12x faster</div><div>Avg: 0.483324 -&gt; 0.427158: 1.13x faster</div><div>Significant (t=28.77)</div><div>Stddev: 0.01876 -&gt; 0.01483: 1.2644x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8al5lmm">http://tinyurl.com/8al5lmm</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(102,255,153)">call_method_slots</span> ###</div><div>Min: 0.484923 -&gt; 0.409452: 1.18x faster</div><div>Avg: 0.487877 -&gt; 0.413054: 1.18x faster</div><div>Significant (t=131.11)</div>

<div>Stddev: 0.00395 -&gt; 0.00577: 1.4589x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9zhpg6z">http://tinyurl.com/9zhpg6z</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">call_method_unknown</span> ###</div>

<div>Min: 0.547050 -&gt; 0.406866: 1.34x faster</div><div>Avg: 0.550721 -&gt; 0.409359: 1.35x faster</div><div>Significant (t=328.32)</div><div>Stddev: 0.00415 -&gt; 0.00325: 1.2795x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9wxoddz">http://tinyurl.com/9wxoddz</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(102,255,153)">call_simple</span> ###</div><div>Min: 0.391213 -&gt; 0.332055: 1.18x faster</div><div>Avg: 0.393563 -&gt; 0.335362: 1.17x faster</div><div>Significant (t=127.15)</div>

<div>Stddev: 0.00363 -&gt; 0.00427: 1.1764x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8mmepzw">http://tinyurl.com/8mmepzw</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">chameleon</span> ###</div>

<div>Min: 0.078505 -&gt; 0.070175: 1.12x faster</div><div>Avg: 0.083754 -&gt; 0.071500: 1.17x faster</div><div>Significant (t=2.95)</div><div>Stddev: 0.05086 -&gt; 0.00119: 42.8425x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8bz9hpl">http://tinyurl.com/8bz9hpl</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(255,102,102)">chaos</span> ###</div><div>Min: 0.353739 -&gt; 0.423587: 1.20x slower</div><div>Avg: 0.356297 -&gt; 0.428197: 1.20x slower</div><div>Significant (t=-108.44)</div>

<div>Stddev: 0.00200 -&gt; 0.00424: 2.1147x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/98e56le">http://tinyurl.com/98e56le</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(255,102,102)">django</span> ###</div>

<div>Min: 0.824149 -&gt; 0.862750: 1.05x slower</div><div>Avg: 0.831614 -&gt; 0.869112: 1.05x slower</div><div>Significant (t=-21.47)</div><div>Stddev: 0.01020 -&gt; 0.00697: 1.4634x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8kz8owv">http://tinyurl.com/8kz8owv</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(255,102,102)">fannkuch</span> ###</div><div>Min: 1.776913 -&gt; 1.832973: 1.03x slower</div><div>Avg: 1.793116 -&gt; 1.915348: 1.07x slower</div><div>Significant (t=-11.57)</div>

<div>Stddev: 0.01436 -&gt; 0.07329: 5.1030x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9ptae4z">http://tinyurl.com/9ptae4z</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">fastpickle</span> ###</div>

<div>Min: 0.810968 -&gt; 0.739322: 1.10x faster</div><div>Avg: 0.818099 -&gt; 0.745148: 1.10x faster</div><div>Significant (t=58.02)</div><div>Stddev: 0.00577 -&gt; 0.00677: 1.1731x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8l769dd">http://tinyurl.com/8l769dd</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(255,102,102)">fastunpickle</span> ###</div><div>Min: 0.644198 -&gt; 0.659345: 1.02x slower</div><div>Avg: 0.647976 -&gt; 0.666154: 1.03x slower</div><div>Significant (t=-18.96)</div>

<div>Stddev: 0.00343 -&gt; 0.00584: 1.7020x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/93xn7el">http://tinyurl.com/93xn7el</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">float</span> ###</div>

<div>Min: 0.420888 -&gt; 0.363410: 1.16x faster</div><div>Avg: 0.432285 -&gt; 0.376179: 1.15x faster</div><div>Significant (t=38.14)</div><div>Stddev: 0.00762 -&gt; 0.00708: 1.0766x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8bjwka9">http://tinyurl.com/8bjwka9</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(255,102,102)">formatted_logging</span> ###</div><div>Min: 0.325707 -&gt; 0.413196: 1.27x slower</div><div>Avg: 0.329846 -&gt; 0.418099: 1.27x slower</div><div>Significant (t=-119.89)</div>

<div>Stddev: 0.00397 -&gt; 0.00337: 1.1787x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8ktbs49">http://tinyurl.com/8ktbs49</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(255,102,102)">genshi</span> ###</div>

<div>Min: 0.254604 -&gt; 0.269696: 1.06x slower</div><div>Avg: 0.258585 -&gt; 0.275615: 1.07x slower</div><div>Significant (t=-33.39)</div><div>Stddev: 0.00283 -&gt; 0.00557: 1.9704x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8bqvcwl">http://tinyurl.com/8bqvcwl</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(255,102,102)">go</span> ###</div><div>Min: 0.676453 -&gt; 0.745504: 1.10x slower</div><div>Avg: 0.681833 -&gt; 0.752170: 1.10x slower</div><div>Significant (t=-48.67)</div>

<div>Stddev: 0.00520 -&gt; 0.00880: 1.6917x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9d6qj3y">http://tinyurl.com/9d6qj3y</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">hexiom2</span> ###</div>

<div>Min: 186.378727 -&gt; 172.939507: 1.08x faster</div><div>Avg: 186.679821 -&gt; 173.103242: 1.08x faster</div><div>Significant (t=39.61)</div><div>Stddev: 0.42581 -&gt; 0.23156: 1.8389x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9mc3pmg">http://tinyurl.com/9mc3pmg</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(102,255,153)">html5lib</span> ###</div><div>Min: 11.827770 -&gt; 11.239556: 1.05x faster</div><div>Avg: 11.858253 -&gt; 11.370960: 1.04x faster</div><div>Significant (t=6.93)</div>

<div>Stddev: 0.02825 -&gt; 0.15466: 5.4746x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8vl952y">http://tinyurl.com/8vl952y</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">iterative_count</span> ###</div>

<div>Min: 0.168182 -&gt; 0.154105: 1.09x faster</div><div>Avg: 0.169512 -&gt; 0.155952: 1.09x faster</div><div>Significant (t=50.77)</div><div>Stddev: 0.00139 -&gt; 0.00128: 1.0899x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9eymjtf">http://tinyurl.com/9eymjtf</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(255,102,102)">json_dump_v2</span> ###</div><div>Min: 3.350528 -&gt; 3.795307: 1.13x slower</div><div>Avg: 3.369661 -&gt; 3.825400: 1.14x slower</div><div>Significant (t=-125.93)</div>

<div>Stddev: 0.01470 -&gt; 0.02095: 1.4250x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8wyn9qa">http://tinyurl.com/8wyn9qa</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">json_load</span> ###</div>

<div>Min: 0.999717 -&gt; 0.607549: 1.65x faster</div><div>Avg: 1.007319 -&gt; 0.613016: 1.64x faster</div><div>Significant (t=289.24)</div><div>Stddev: 0.00673 -&gt; 0.00690: 1.0240x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8qxakdw">http://tinyurl.com/8qxakdw</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(255,102,102)">mako_v2</span> ###</div><div>Min: 0.094817 -&gt; 0.279593: 2.95x slower</div><div>Avg: 0.096962 -&gt; 0.286479: 2.95x slower</div><div>Significant (t=-866.63)</div>

<div>Stddev: 0.00182 -&gt; 0.00454: 2.4945x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9lufgwz">http://tinyurl.com/9lufgwz</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">meteor_contest</span> ###</div>

<div>Min: 0.276138 -&gt; 0.243228: 1.14x faster</div><div>Avg: 0.279559 -&gt; 0.246018: 1.14x faster</div><div>Significant (t=72.30)</div><div>Stddev: 0.00298 -&gt; 0.00136: 2.1943x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8pj9dnc">http://tinyurl.com/8pj9dnc</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(102,255,153)">nbody</span> ###</div><div>Min: 0.421698 -&gt; 0.320496: 1.32x faster</div><div>Avg: 0.425878 -&gt; 0.323483: 1.32x faster</div><div>Significant (t=158.15)</div>

<div>Stddev: 0.00386 -&gt; 0.00247: 1.5638x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9fy8dfg">http://tinyurl.com/9fy8dfg</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(255,102,102)">normal_startup</span> ###</div>

<div>Min: 0.612120 -&gt; 0.876470: 1.43x slower</div><div>Avg: 0.618945 -&gt; 0.885492: 1.43x slower</div><div>Significant (t=-280.36)</div><div>Stddev: 0.00422 -&gt; 0.00523: 1.2397x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/98ap93d">http://tinyurl.com/98ap93d</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(255,102,102)">nqueens</span> ###</div><div>Min: 0.402125 -&gt; 0.410580: 1.02x slower</div><div>Avg: 0.406403 -&gt; 0.414676: 1.02x slower</div><div>Significant (t=-12.06)</div>

<div>Stddev: 0.00442 -&gt; 0.00199: 2.2189x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8wd3lez">http://tinyurl.com/8wd3lez</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(255,102,102)">pathlib</span> ###</div>

<div>Min: 0.132423 -&gt; 0.164525: 1.24x slower</div><div>Avg: 0.136298 -&gt; 0.168843: 1.24x slower</div><div>Significant (t=-49.05)</div><div>Stddev: 0.00763 -&gt; 0.00720: 1.0586x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9o86dc5">http://tinyurl.com/9o86dc5</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(102,255,153)">pidigits</span> ###</div><div>Min: 0.387690 -&gt; 0.367871: 1.05x faster</div><div>Avg: 0.391308 -&gt; 0.371194: 1.05x faster</div><div>Significant (t=32.69)</div>

<div>Stddev: 0.00369 -&gt; 0.00230: 1.6066x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9med7ko">http://tinyurl.com/9med7ko</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(255,102,102)">raytrace</span> ###</div>

<div>Min: 1.650066 -&gt; 1.808829: 1.10x slower</div><div>Avg: 1.660110 -&gt; 1.832654: 1.10x slower</div><div>Significant (t=-25.26)</div><div>Stddev: 0.01165 -&gt; 0.04687: 4.0224x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8fmyhex">http://tinyurl.com/8fmyhex</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(255,102,102)">regex_compile</span> ###</div><div>Min: 0.559449 -&gt; 0.571906: 1.02x slower</div><div>Avg: 0.563738 -&gt; 0.580054: 1.03x slower</div><div>Significant (t=-8.38)</div>

<div>Stddev: 0.00434 -&gt; 0.01306: 3.0087x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8g6xcmd">http://tinyurl.com/8g6xcmd</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(255,102,102)">regex_effbot</span> ###</div>

<div>Min: 0.074999 -&gt; 0.097456: 1.30x slower</div><div>Avg: 0.076343 -&gt; 0.099435: 1.30x slower</div><div>Significant (t=-39.79)</div><div>Stddev: 0.00147 -&gt; 0.00383: 2.5994x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9vfaeux">http://tinyurl.com/9vfaeux</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(255,102,102)">regex_v8</span> ###</div><div>Min: 0.087433 -&gt; 0.104053: 1.19x slower</div><div>Avg: 0.088804 -&gt; 0.105520: 1.19x slower</div><div>Significant (t=-39.48)</div>

<div>Stddev: 0.00115 -&gt; 0.00277: 2.4122x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8un7vfr">http://tinyurl.com/8un7vfr</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">richards</span> ###</div>

<div>Min: 0.247208 -&gt; 0.222483: 1.11x faster</div><div>Avg: 0.251661 -&gt; 0.225276: 1.12x faster</div><div>Significant (t=44.04)</div><div>Stddev: 0.00392 -&gt; 0.00161: 2.4275x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8b2zv34">http://tinyurl.com/8b2zv34</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(102,255,153)">silent_logging</span> ###</div><div>Min: 0.099170 -&gt; 0.095099: 1.04x faster</div><div>Avg: 0.099713 -&gt; 0.095892: 1.04x faster</div><div>Significant (t=33.32)</div>

<div>Stddev: 0.00045 -&gt; 0.00068: 1.5062x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9arurw6">http://tinyurl.com/9arurw6</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(255,102,102)">simple_logging</span> ###</div>

<div>Min: 0.316639 -&gt; 0.392833: 1.24x slower</div><div>Avg: 0.320059 -&gt; 0.396853: 1.24x slower</div><div>Significant (t=-120.31)</div><div>Stddev: 0.00224 -&gt; 0.00392: 1.7450x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/95bfxu7">http://tinyurl.com/95bfxu7</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(102,255,153)">spectral_norm</span> ###</div><div>Min: 0.434691 -&gt; 0.379294: 1.15x faster</div><div>Avg: 0.437958 -&gt; 0.383761: 1.14x faster</div><div>Significant (t=67.75)</div>

<div>Stddev: 0.00410 -&gt; 0.00390: 1.0502x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/98s9c56">http://tinyurl.com/98s9c56</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(255,102,102)">startup_nosite</span> ###</div>

<div>Min: 0.209685 -&gt; 0.660867: 3.15x slower</div><div>Avg: 0.218654 -&gt; 0.673249: 3.08x slower</div><div>Significant (t=-458.50)</div><div>Stddev: 0.00646 -&gt; 0.00752: 1.1645x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9zyerhn">http://tinyurl.com/9zyerhn</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(102,255,153)">telco</span> ###</div><div>Min: 0.840453 -&gt; 0.018312: 45.90x faster</div><div>Avg: 0.844250 -&gt; 0.019255: 43.85x faster</div><div>Significant (t=1088.45)</div>

<div>Stddev: 0.00521 -&gt; 0.00127: 4.0959x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/924mje7">http://tinyurl.com/924mje7</a>&#39;</div><div><br></div><div>### <span style="background-color:rgb(102,255,153)">threaded_count</span> ###</div>

<div>Min: 0.197525 -&gt; 0.151649: 1.30x faster</div><div>Avg: 0.213657 -&gt; 0.153572: 1.39x faster</div><div>Significant (t=52.58)</div><div>Stddev: 0.00779 -&gt; 0.00214: 3.6451x smaller</div><div>Timeline: b&#39;<a href="http://tinyurl.com/8mrrqla">http://tinyurl.com/8mrrqla</a>&#39;</div>

<div><br></div><div>### <span style="background-color:rgb(102,255,153)">unpack_sequence</span> ###</div><div>Min: 0.000060 -&gt; 0.000052: 1.16x faster</div><div>Avg: 0.000088 -&gt; 0.000069: 1.29x faster</div><div>Significant (t=1118.61)</div>

<div>Stddev: 0.00000 -&gt; 0.00000: 1.0022x larger</div><div>Timeline: b&#39;<a href="http://tinyurl.com/9ejrega">http://tinyurl.com/9ejrega</a>&#39;</div></div>