[pypy-dev] PyPy in the benchmarks game - yes or no?

[Jacob Hallén]

Friday 08 April 2011 you wrote:
> --- On Fri, 4/8/11, Maciej Fijalkowski <fijall at gmail.com> wrote:
> 
> -snip-
> 
> > I guess a lot of discussions are about getting some sort of
> > consensus. I see this one is so you can know what we think
> > and that's it. Well, that comes as a bit of surprise.
> 
> I don't know why that would come as any surprise - "Of course, I'll make up
> my own mind but at least I'll be able to take your wishes into account."
> 
> http://permalink.gmane.org/gmane.comp.python.pypy/7303
> 
> > I think it's super stupid to remove Tracemonkey, LuaJIT and
> > PyPy from it, but that's as you pointed out *your* website.
> > On the other hand it's good, because people won't cite the
> > computer language shootout anymore and those benchmarks are
> > more silly than they have to be.
> 
> You express both of the contrary wishes that I've heard here this week -
> you seem to want yes and no :-)
> 
> fwiw someone did write - "While a comparison between languages may be
> interesting, maybe having 1 implementation per language in the shootout
> would work better." - let's hope at least they are happy now.


I think you should consider your own wishes a little more. If you want the 
language shootout to be relevant to people, you can't ignore multiple 
implementations. Especially it seems to b excessively excentric to ignore the 
fastest implementations of some languages while not doing so for others. I 
assume you are not measuring C speed by the old AT&T reference implementation.

I think you have to make up your mind. Do you want to provide a service to 
other people, in which case you should work on making fair and reasonable 
comparisons, mislead people, in which case you should continue on the current 
course. Perhaps you don't care either way; perhaps you are just in it for the 
ego boost. Then we want to find out, so we can stop wasting our time trying to 
get a reasonable outcome.

I think you need to stop and think about what you really want, instead of 
emmotionally reacting to evolving events. When you have given it a good 
thought, you tell people what you have come up with and go off implementing 
it. Tose who like it will gather around, those who dislike it will go away.

If you don't know what you want out of the language shootout, you might ask 
people in different camps what they would like. You will be surprised by the 
number of sane and reasonable answers you would get.

Jacob Hallén
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20110409/22054499/attachment.pgp>