Hi Jesse, hi all,

On 31/08/11 02:37, Jesse Noller wrote:

Yes, I should have looped Noah in sooner.

I have all the keys / passwords. Right now we need:

1. codespeed
2. benchmark runners

The second item is divided into two sub-items: the runner itself, and something that triggers a run nightly/on request.

In PyPy, we currently use a modified version of the Unladen Swallow benchmark runner (look at runner.py): https://bitbucket.org/pypy/benchmarks/src

in its current state, it is sub-optimal, because it was designed to compare two pythons side-by-side, and not to save the results and then upload them to a website. As a consequence, it always runs the benchmarks twice. With PyPy, it's lesser of a problem because for each binary we always run the benchmarks with the default options and with "--jit off" anyway, but for CPython running them twice would result in a waste of time.

I suppose that the current runner is good enough for now and to get started, however at some point it will need some care.

Then, we need something which triggers the actual benchmark run. In PyPy, we use buildbot, and each benchmark run first do a full translation, then executes the benchmarks.

I can setup a buildbot instance on speed.pypy.org, if you give me access to the machine. I propose that as a very first step, we just make speed.pypy.org a buildslave which depends on pypy's own buildmaster. This makes it very easy and fast to setup it, so we can have something running soon.

ciao, Anto