[pypy-dev] PyPy in the benchmarks game - yes or no?

[Maciej Fijalkowski]

On Wed, Apr 6, 2011 at 3:08 PM, Isaac Gouy <igouy2 at yahoo.com> wrote:
>
>
> --- On Wed, 4/6/11, Maciej Fijalkowski <fijall at gmail.com> wrote:
>
> -snip-
>> We don't get that information now at least, since those
>> benchmarks are badly skewed towards CPython. I know how
>> hard is to find out a reasonable set of benchmarks and how to
>> keep them balanced.
>
>
> Do you mean the program you contributed is badly skewed towards CPython?
>
> http://shootout.alioth.debian.org/u32/program.php?test=nbody&lang=pypy&id=1
>

No

>
> Do you mean that the n-body problem is badly skewed towards CPython?

No, that would be nonsense. I would never discuss whether those
benchmarks does represent typical workflow in language X because it's
impossible to find such a set that's true for every X. I never did
discuss the choice of problems.

>
> Your PyPy program is shown as so much faster - how is that "badly skewed towards CPython"?
>

That's true, but that's one that got through. For example reverse
complement (the current version) is skewed towards CPython. I'm fine
with saying that ctypes (or numpy) are not allowed, with a good
explanation (and maybe an explanation why custom malloc library is
allowed for C and gcbench).

Another question which was raised - are programs that only work on
PyPy allowed? (Due to pypy's extensions or cpython bugs). Since
programs that only compile on GCC clearly are.

Cheers,
fijal