[pypy-dev] [Speed] adding a pypy runner for speed.python.org

[Victor Stinner]

Hi,

I tried but failed to find someone in PyPy to adjust performance
benchmarks for PyPy. Currently, the JIT is not properly warmed up, and
the results can be dishonnest or not reliable.

My latest attempt to support is PyPy is:
http://vstinner.readthedocs.io/pypy_warmups.html

IMHO we need to develop a statistical methodology in perf to compute
when values become stable. That's hard to define and it was proven
that "performance stability" doesn't exist (see " Virtual Machine
Warmup Blows Hot and Cold" paper).

Fijal from PyPy would like to use hardcoded configuration for the
number of warmup values. I like the idea, but nobody implemented it
yet.

Victor

2018-01-30 21:37 GMT+01:00 Matti Picus <matti.picus at gmail.com>:
> I am cross-posting to both speed and pypy-dev to ask what needs to be done
> to get pypy2 and pypy3 benchmark runners onto speed.python.org
> I am willing to be the contact person from the PyPy side, who do we need to
> talk to on the speed maintainers' side?
> Instead of spamming the lists again, we could discuss this off-line, and
> report back with a plan, required resources, and a timeline.
>
> Thanks,
> Matti Picus
> _______________________________________________
> Speed mailing list
> Speed at python.org
> https://mail.python.org/mailman/listinfo/speed