[pypy-issue] [issue1229] PyPy Benchmarking Bias?

[3TATUK]

New submission from 3TATUK <ztatik.light at gmail.com>:

I had a suggestion for PyPy: Considering as how part of it's primary goal is
speed... I think the performance benchmarks could use a revamp..  Reason being,
the could potentially be biased towards showing better results over time by
incorporating the use of special 'hacks'.. Just like OPEC companies report
higher outputs than in reality.

I recommend having two versions of each benchmark, both having the same exact
functional output, but one optimized for greatest CPython/PyPy speed, and one
optimized for lowest CPython/PyPy speed, and then either value could be
compared, or an average of the two.

----------
assignedto: fijal
messages: 4636
nosy: 3TATUK, fijal, pypy-issue
priority: wish
release: 2.0
status: in-progress
title: PyPy Benchmarking Bias?

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1229>
________________________________________