[pypy-dev] benchmarking input

fijall fijall at gmail.com
Thu Sep 24 16:38:14 CEST 2009


Hi Anders.

I, personally, would start with a more modest goal than fully
running infrastructure. I would like to be able to run it myself,
provided I have downloaded and compiled all necessary interpreters.

So say, I want to run benchmarks a, b, c using python, pypy-a, pypy-b
and pypy-c.

And I say something like:
./run_benchmarks --benchmarks="a b c" --interpreters="pypy-a pypy-b
python pypy-c"

And get some sort of results, to start with in a text form.

Next step would be to have a backend that stores informations between
runs, but I would really really like to go with incremental approach,
where I have something to start with and later on improve and add features.

PS. Sorry for late reply, was on holiday.

Cheers,
fijal


On Mon, Sep 21, 2009 at 9:27 AM, Anders Hammarquist <iko at openend.se> wrote:
> Hi all,
>
> I need some input for the benchmarking infrastructure. I'm nearly
> at the point where I need to have some place to run it before
> continuing (i.e. I need to try and use it, not just speculate).
>
> Anyway, what I was thinking about, and need input on, is how to get
> at the interpreters to run the benchmark. When we were talking just
> benchmarks, and not profiling, my thought was to just use whatever
> python the machine has, and fetch the pypy from the last buildbot
> run, but for profiling that will not work (and anyway, running the
> profiling on the standard python is quite pointless). So benchmarks
> will obviously have to specify what interpreter(s) they should be run
> by somehow.
>
> The bigger question is how to get those interpreters. Should running
> the benchmarks also trigger building one (or more) pypy interpreters
> according to specs in the benchmarking framework? (but then if you
> only want it to run one benchmark, you may have to wait for all the
> interpreters to build) Perhaps each benchmark should build its own
> interpreter (though this seems slow, given that most benchmarks
> can probably run on an identically built interpreter).
>
> Or maybe the installed infrastructure should only care about history,
> and if you want to run a single benchmark, you do that on your own.
>
> Thoughts please!
>
> /Anders
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
>
>
>



More information about the Pypy-dev mailing list