Re: [Speed] Adapting pypy benchmark suite
On Wed, Apr 13, 2016 at 1:57 PM, Maciej Fijalkowski fijall@gmail.com wrote:
Hi
I have a radical idea: to take a pypy benchmark suite, update the libraries to newer ones and replace python benchmarks with that. The main reason being that pypy has a much better coverage of things that are not microbenchmarks, the list (in json):
http://paste.pound-python.org/show/4YVq0fv6pv8rVOSmCTag/
Which is much more extensive than this:
https://hg.python.org/benchmarks/file/tip/performance
I'm willing to put *some* effort, what do people think?
I'm in favor. My support has two conditions, though:
at least a majority of the benchmarks must be Python3 compatible. Preferably 2/3 compatible, but I assume all of the PyPy benchmarks are 2 compatible anyway.
and there should be an easy way to run the benchmarks against exactly 1 interpreter (for use with speed.python.org). I initially tried to set up speed.python.org using the PyPy benchmarks, but quickly ran into issues with trying to use 'nullpython.py' as the baseline Python. When I switched to using h.p.o/benchmarks, I added the '--raw' flag to perf.py which allows the benchmarks to be run on one interpreter instead of two. It was just a quick hack, though; I have no problems with that feature completely changing (even invoking it a different way is ok), so long as it exists.
This project could probably start its life as github.com/python/benchmarks and save us from having to migrate h.p.o/benchmarks to GitHub.
-- Zach
participants (1)
-
Zachary Ware