[pypy-dev] PyPy in the benchmarks game - yes or no?

[Isaac Gouy]

--- On Wed, 4/6/11, Maciej Fijalkowski <fijall at gmail.com> wrote:

-snip-
> > Do you mean the program you contributed is badly
> skewed towards CPython?
> >
> > http://shootout.alioth.debian.org/u32/program.php?test=nbody〈=pypy&id=1
> >
> 
> No
> 
> >
> > Do you mean that the n-body problem is badly skewed
> towards CPython?
> 
> No, that would be nonsense. I would never discuss whether
> those
> benchmarks does represent typical workflow in language X
> because it's
> impossible to find such a set that's true for every X. I
> never did
> discuss the choice of problems.
> 
> >
> > Your PyPy program is shown as so much faster - how is
> that "badly skewed towards CPython"?
> >
> 
> That's true, but that's one that got through. 


I don't see how the program you contributed could be described as "one that got through".

Here's what happened - I noticed the n-body program failed with PyPy, I asked you guys about the problem and was told "we have nbody_modified in our benchmarks" and then I asked you guys to contribute your modified program.

http://morepypy.blogspot.com/2010/03/introducing-pypy-12-release.html 

Once you contributed the program modified for PyPy it was displayed on the website within 3 hours.



> For example reverse complement (the current version) is
> skewed towards CPython.

If only someone could manage to write a reverse complement skewed towards PyPy without using libc.write ;-)



> I'm fine with saying that ctypes (or numpy) are not allowed, 
> with a good explanation (and maybe an explanation why custom malloc
> library is allowed for C and gcbench).
> 
> Another question which was raised - are programs that only
> work on PyPy allowed? (Due to pypy's extensions or cpython bugs).

PyPy extensions - No.

CPython bugs - How strange that the CPython bug was never mentioned! - maybe.


> Since programs that only compile on GCC clearly are.

How many C language implementations are shown?

How many Python language implementations are shown?

If only one Python language implementation was shown do you think it would be PyPy ?