[Python-Dev] Information about how cpython in benchmarked

[Nick Coghlan]

On Tue, Mar 29, 2011 at 8:01 PM, Tennessee Leeuwenburg
<tleeuwenburg at gmail.com> wrote:
> PyPy maintains http://speed.pypy.org/, which provides very clear information
> about the relative performance of PyPy trunk against some version of cpython
> (presumably 2.6 or 2.7). I'm not aware of a similar site for cpython, but
> that could easily just be my ignorance speaking.
> My interest is that I'm looking at building a benchmarking solution at work.
> and I can't think of a better way to build something good and general than
> to try and write something that could potentially be released as open source
> and be useful to others. As such I thought that benchmarking cpython would
> be a great use case, but I want to find out as much as I can about how
> people currently go about benchmarking Python. Initially I'm just looking at
> CPU profiling since it's easiest.

One of the points coming out of the VM summit at Pycon is actually
that we want to create a shared benchmarking site for CPython, PyPy,
Jython, IronPython (and possibly Stackless) under the python.org
banner (either speed.python.org, or possibly performance.python.org,
since we want to do memory profiling as well).

speed.pypy.org will be the reference site for this, but Maciej
indicated at the VM summit that the code that runs that site needs
some improvements before it will really be up to the task of
effectively benchmarking multiple targets.

So, according to http://speed.pypy.org/about/, the place to start with
your benchmarking system would probably be
https://github.com/tobami/codespeed.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia