[Python-Dev] 2.5a1 Performance

[Raymond Hettinger]

> Benchmarking is hard, let's go shopping!

Quick reminder:  pystone is mostly useful for predicting Python's relative 
performance across various machines and operating systems.  For benchmarking 
Python itself, pystone is a seriously impaired tool.  For one, it exercises only 
a tiny subset of the language.  For another, it times an empty loop and 
subtracts that from the result of loops with bodies -- that means that 
improvements/impairments to the eval-loop get netted-out of the result.  Let's 
stop talking about pystone in this thread and focus on meaningful metrics 
instead.

If you want some good measurements of the eval-loop speed and a few simple 
instructions, use timeit.py.  The results should be directly comparable between 
Py2.4 and Py2.5a.

If you want good measurements that specifically exercise a wide gamut of 
commonly used functions, then use pybench.py.

If you want to thoroughly exercise the language, use the parrot benchmark in the 
sandbox.

Of course, the only truly useful benchmark is how Python performs on your own 
apps.


Raymond