Hi all, Fijal may comment on this more, but note that he tweaked the number of runs of the benchmarks displayed by speed.pypy.org, which explains the jumps we see in some of them. The positive-for-us jumps, I should add :-) I think that we should remove the history up to today, as it makes little sense to compare... A bientot, Armin.
Hey. Yes, sorry about not saying that upfront. I've removed --fast from run, so our numbers looks better (because warmup is more spread over). It makes sense to remove history IMO. However, I plan to work a bit more on that, to completely remove warmup time from some benchmark and to include warmup for each run for others (like richards, html5lib), so that would be yet another removal of history. Right now the "average" is not that meaningful (and stddev even less, although it's not displayed on speed), since it averages warmup across all the run. I would like to have warmup either completely included for each run (for benchmarks that it makes sense) or completely separated and reported as some other variable. Cheers, fijal On Sat, Mar 6, 2010 at 2:27 PM, Armin Rigo <arigo@tunes.org> wrote:
Hi all,
Fijal may comment on this more, but note that he tweaked the number of runs of the benchmarks displayed by speed.pypy.org, which explains the jumps we see in some of them. The positive-for-us jumps, I should add :-)
I think that we should remove the history up to today, as it makes little sense to compare...
A bientot,
Armin. _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
Yeah, history is now not very meaningful. While we are at it, I think we should complete the job and define a standard set of benchmarks together with the python.org and unladen guys. So that we can keep the data this time :-) How were the chats about that developing, fijal? 2010/3/7 Maciej Fijalkowski <fijall@gmail.com>:
Hey.
Yes, sorry about not saying that upfront. I've removed --fast from run, so our numbers looks better (because warmup is more spread over). It makes sense to remove history IMO.
However, I plan to work a bit more on that, to completely remove warmup time from some benchmark and to include warmup for each run for others (like richards, html5lib), so that would be yet another removal of history.
Right now the "average" is not that meaningful (and stddev even less, although it's not displayed on speed), since it averages warmup across all the run. I would like to have warmup either completely included for each run (for benchmarks that it makes sense) or completely separated and reported as some other variable.
Cheers, fijal
On Sat, Mar 6, 2010 at 2:27 PM, Armin Rigo <arigo@tunes.org> wrote:
Hi all,
Fijal may comment on this more, but note that he tweaked the number of runs of the benchmarks displayed by speed.pypy.org, which explains the jumps we see in some of them. The positive-for-us jumps, I should add :-)
I think that we should remove the history up to today, as it makes little sense to compare...
A bientot,
Armin. _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
On Mon, Mar 8, 2010 at 3:29 AM, Miquel Torres <tobami@googlemail.com> wrote:
Yeah, history is now not very meaningful.
While we are at it, I think we should complete the job and define a standard set of benchmarks together with the python.org and unladen guys. So that we can keep the data this time :-)
How were the chats about that developing, fijal?
Progressing, but slowly. We all have slightly different goals I believe. For example I think pypy is aiming also at people who are doing stuff in C and would like to do python instead. US is aiming exclusively at python programmers. (I think) Cheers, fijal
2010/3/7 Maciej Fijalkowski <fijall@gmail.com>:
Hey.
Yes, sorry about not saying that upfront. I've removed --fast from run, so our numbers looks better (because warmup is more spread over). It makes sense to remove history IMO.
However, I plan to work a bit more on that, to completely remove warmup time from some benchmark and to include warmup for each run for others (like richards, html5lib), so that would be yet another removal of history.
Right now the "average" is not that meaningful (and stddev even less, although it's not displayed on speed), since it averages warmup across all the run. I would like to have warmup either completely included for each run (for benchmarks that it makes sense) or completely separated and reported as some other variable.
Cheers, fijal
On Sat, Mar 6, 2010 at 2:27 PM, Armin Rigo <arigo@tunes.org> wrote:
Hi all,
Fijal may comment on this more, but note that he tweaked the number of runs of the benchmarks displayed by speed.pypy.org, which explains the jumps we see in some of them. The positive-for-us jumps, I should add :-)
I think that we should remove the history up to today, as it makes little sense to compare...
A bientot,
Armin. _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
participants (3)
-
Armin Rigo -
Maciej Fijalkowski -
Miquel Torres