tl;dr I'm trying to run benchmarks with PGO, but I get new errors, so speed.python.org is currently broken.
Last december, I upgraded the speed-python server (server used to run benchmarks for CPython) from Ubuntu 14.04 (LTS) to 16.04 (LTS) to be able to compile Python using PGO compilation. On Ubuntu 14.04, GCC failed with an internal error. Sadly, I lost my sudo permission during the upgrade: it took almost one month to retrieve the permission.
I removed all results from speed.python.org to start to publish new benchmark results of LTO+PGO compilation. Sadly, I got new not funny errors.
The first benchmark on the 2.7 branch failed because of a recent regression specific to the 2.7 branch (now fixed): http://bugs.python.org/issue28871#msg284145
I fixed my configuration file to only test a list of revisions instead of starting by testing all branches. One more time, benchmarks failed: this time, the "python3 -m performance venv recreate" failed on the creation of the virtual environment:
performance has multiple functions to create the venv, but all failed. I'm really unlucky.
I modified my scripts to run benchmarks (scripts/ directory of the performance project) to handle errors more nicely. For example, don't fail immediately, but log errors.
I also modified performance to flush stdout+stderr before running subprocess to not mess logs (when stdout and/or stderr are redirected).
I'm now trying to run benchmarks one more time... ;-)