[pypy-dev] PyPy in the benchmarks game - yes or no?

[Dima Tisnek]

Overall one language implementation per language is a good idea for
shootout. Consequently, using the latest version of most widely used
implementation (cpython 3.2) is indeed the way to go, even in the face
of all the 2.6 recalcitrants, myself included :-)

As for pypy, I think it is in the pypy's best interest to be
represented as widely as possible, I am surprised to see negative
attitude here... Though I understand pypy implementation of these
benchmarks requires work; btw that would be a great project for, say,
summer of code.

Coming back to Isaac's question, perhaps you want to a tiered
representation for languages, e.g. compiled vs dynamic, then inside
dynamic perl vs python, then inside that cpython 2 vs cpython 3 vs
pypy. This is really a matter of presentation. Alternative is to list
"supported" implementations  separate from that
untested/contributed/questionable/etc. Coming back to presentation,
shootout is pretty old by now, compare it to speed.pypy.org which is
outright sexy! Also you need serious QA of the benchmarks, rules,
implementations and even the test environment itself.

Dima Tisnek

On 8 April 2011 12:19, Piotr Skamruk <piotr.skamruk at gmail.com> wrote:
> 2011/4/8 Isaac Gouy <igouy2 at yahoo.com>:
>> [...]
>>> I think it's super stupid to remove Tracemonkey, LuaJIT and
>>> PyPy from it, but that's as you pointed out *your* website.
>>> On the other hand it's good, because people won't cite the
>>> computer language shootout anymore and those benchmarks are
>>> more silly than they have to be.
>>
>> You express both of the contrary wishes that I've heard here this week - you seem to want yes and no :-)
>>
>> fwiw someone did write - "While a comparison between languages may be interesting, maybe having 1 implementation per language in the shootout would work better." - let's hope at least they are happy now.
> sorry, but now it's not a comparsion of languages, but comparison of
> several randomly selected programs.
> remember that You are not comparing languages, but their
> implementations (still that looks like randomly choosen - cpython 3.2
> have much, much less usage than 2.6/2.7).
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>