As some of you already know, there is a new performance site online:
speed.pypy.org.
In Autumn last year I offered my help with a benchmarking website, and
eventually for a new main site for the project. What you can see now
is a work in progress, which I hope to improve in the next months. I
called it Codespeed, and in time it could become a framework for other
open source projects to use (for example javascript implementation
development, connected to a forge, etc...).
But right now, speed has two goals:
- To have a powerful tool for the devs to analyse performance and
detect regressions in the different pypy interpreters.
- To improve the visibility of the project by letting python
developers(users) have a look themselves at current performance and to
compare pypy to other implementations, not having to wait for some dev
to manually put together the graphs and post them on the blog.
To that end, there are currently two views.
The Overview does what it's name says, and allows a developer to have
a broad view of how last revisions are affecting performance, and
immediately spot regressions. If the user wants to closer examine the
changes in the performance of a given benchmark, just clicking on it
takes you to the the timeline for the selected benchmark.
Both views will get some more features, and I have further plans down
the pipeline:
- A Comparison tab: graph bars directly comparing performance of
different releases (pypy 1.2, trunk, cpython 2.x, unladen swallow,
Jython, etc.)
- A statistics tab: Go through regression test literature to see if we
can have good metrics shown.
- RSS feed/email alerts (for big regressions)
- svn info integration
- etc...
In the end it is a site for the pypy devs, so I hope to get a lot of
feedback from you so that it caters exactly to your needs.
Have a nice day!
Miquel
PD: When it is a bit more complete I'll write a post for the PyPy
status blog so it gets wider known.
