[pypy-issue] [issue1229] PyPy Benchmarking Bias?
tracker at bugs.pypy.org
Mon Aug 6 07:16:50 CEST 2012
New submission from 3TATUK <ztatik.light at gmail.com>:
I had a suggestion for PyPy: Considering as how part of it's primary goal is
speed... I think the performance benchmarks could use a revamp.. Reason being,
the could potentially be biased towards showing better results over time by
incorporating the use of special 'hacks'.. Just like OPEC companies report
higher outputs than in reality.
I recommend having two versions of each benchmark, both having the same exact
functional output, but one optimized for greatest CPython/PyPy speed, and one
optimized for lowest CPython/PyPy speed, and then either value could be
compared, or an average of the two.
nosy: 3TATUK, fijal, pypy-issue
title: PyPy Benchmarking Bias?
PyPy bug tracker <tracker at bugs.pypy.org>
More information about the pypy-issue