[pypy-dev] PyPy in the benchmarks game - yes or no?

[Piotr Skamruk]

2011/4/8 Dima Tisnek <dimaqq at gmail.com>:
> Overall one language implementation per language is a good idea for
> shootout. Consequently, using the latest version of most widely used
> implementation (cpython 3.2) is indeed the way to go, even in the face
> of all the 2.6 recalcitrants, myself included :-)
So last (3.2) or most used (2.6)?
Sorry, but 3.x is not widely used for now...

> As for pypy, I think it is in the pypy's best interest to be
> represented as widely as possible, I am surprised to see negative
> attitude here... Though I understand pypy implementation of these
> benchmarks requires work; btw that would be a great project for, say,
> summer of code.
pypy is probably fastest implementation for now, but also it could not
be described as "widely used"

> Coming back to Isaac's question, perhaps you want to a tiered
> representation for languages, e.g. compiled vs dynamic, then inside
> dynamic perl vs python, then inside that cpython 2 vs cpython 3 vs
> pypy. This is really a matter of presentation. Alternative is to list
> "supported" implementations  separate from that
> untested/contributed/questionable/etc. Coming back to presentation,
> shootout is pretty old by now, compare it to speed.pypy.org which is
> outright sexy! Also you need serious QA of the benchmarks, rules,
> implementations and even the test environment itself.
If someone is comparing some implemetations of languages, than that
person should probably choose most common used version of
compiler/interpreter - not "most recent release/revision".

shootout.alioth.debian.org is runned under debian domain, so it's for
me hard to understand why py3.2 was choosen as python representation
when it has minor usage even in debian... (still in debian 2.6 is more
mainline, even 2.7 is not so supported outside of ubuntu)

speed.pypy.org have resonable value probably mostly for developers,
with strong focus on python developers.

shootout.alioth.debian.org, as more general site, is focused on
informing users less with less knowledge about programming.
in current state comparsion in shootout.alioth.debian.org introduces
more confusion than usefull information, what probably had in mind
Maciej.