[pypy-dev] PyPy in the benchmarks game - yes or no?

Maciej Fijalkowski fijall at gmail.com
Wed Apr 6 12:14:22 CEST 2011


On Wed, Apr 6, 2011 at 1:22 AM, Aaron DeVore <aaron.devore at gmail.com> wrote:
> On Tue, Apr 5, 2011 at 2:49 PM, Joe <qbproger at gmail.com> wrote:
>> While I spent my Saturday trying to make PyPy look better in the
>> language shootout, I'm leaning towards taking it out.  While a
>> comparison between languages may be interesting, maybe having 1
>> implementation per language in the shootout would work better.  Then
>> there is one target to optimize for.  PyPy and CPython have very
>> different performance characteristics.
>>
>> I feel as though speed.python.org may be a better venue for comparing
>> python implementations.  Since the pypy developers have closer ties to
>> the Python Core developers and it's been stated they'll have
>> influence.  It can be made to be fair for all parties involved.  Since
>> all parties will likely be python implementations they can all agree
>> on one implementation and use that.
>>
>> Joe
>
> I heavily recommend keeping PyPy in the Shootout in stome form. Even
> with the Shootout's flaws, it is nice to have a general idea of how
> PyPy compares to both CPython and other languages.

We don't get that information now at least, since those benchmarks are
badly skewed towards CPython. I know how hard is to find out a
reasonable set of benchmarks and how to keep them balanced.

I have another issue with ctypes & numpy. This is that C
implementations are allowed to use gcc-specific hacks and non-standard
libraries (apache malloc for gcbench). Why wouldn't we be allowed to
do the same then?

I would like to have PyPy included, but I would also like the
benchmark game to be "fair" or as close to "fair" as we can get.
Having benchmarks with different rules for different languages doesn't
seem to be quite fair in my opinion. Note that this is up to
discussions - I'm fine with saying "numpy & ctypes are disallowed" or
having a separate category "python + numpy" with both CPython and PyPy
(once we get numpy).

>
> -Aaron DeVore
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



More information about the Pypy-dev mailing list