[pypy-dev] benchmarking input
fijall at gmail.com
Thu Sep 24 16:38:14 CEST 2009
I, personally, would start with a more modest goal than fully
running infrastructure. I would like to be able to run it myself,
provided I have downloaded and compiled all necessary interpreters.
So say, I want to run benchmarks a, b, c using python, pypy-a, pypy-b
And I say something like:
./run_benchmarks --benchmarks="a b c" --interpreters="pypy-a pypy-b
And get some sort of results, to start with in a text form.
Next step would be to have a backend that stores informations between
runs, but I would really really like to go with incremental approach,
where I have something to start with and later on improve and add features.
PS. Sorry for late reply, was on holiday.
On Mon, Sep 21, 2009 at 9:27 AM, Anders Hammarquist <iko at openend.se> wrote:
> Hi all,
> I need some input for the benchmarking infrastructure. I'm nearly
> at the point where I need to have some place to run it before
> continuing (i.e. I need to try and use it, not just speculate).
> Anyway, what I was thinking about, and need input on, is how to get
> at the interpreters to run the benchmark. When we were talking just
> benchmarks, and not profiling, my thought was to just use whatever
> python the machine has, and fetch the pypy from the last buildbot
> run, but for profiling that will not work (and anyway, running the
> profiling on the standard python is quite pointless). So benchmarks
> will obviously have to specify what interpreter(s) they should be run
> by somehow.
> The bigger question is how to get those interpreters. Should running
> the benchmarks also trigger building one (or more) pypy interpreters
> according to specs in the benchmarking framework? (but then if you
> only want it to run one benchmark, you may have to wait for all the
> interpreters to build) Perhaps each benchmark should build its own
> interpreter (though this seems slow, given that most benchmarks
> can probably run on an identically built interpreter).
> Or maybe the installed infrastructure should only care about history,
> and if you want to run a single benchmark, you do that on your own.
> Thoughts please!
> pypy-dev at codespeak.net
More information about the Pypy-dev