Hi Pablo,
thanks for pointing this out.
Would it be possible to get the data for older runs back, so that
it's easier to find the changes which caused the slowdown ?
Going to the timeline, it seems that the system only has data
for Oct 14 (today):
https://speed.python.org/timeline/#/?exe=12&ben=regex_dna&env=1&revs=1000&equid=off&quarts=on&extr=on&base=none
In addition to unpack_sequence, the regex_dna test has slowed
down a lot compared to Py3.8.
https://github.com/python/pyperformance/blob/master/pyperformance/benchmarks/bm_unpack_sequence.py
https://github.com/python/pyperformance/blob/master/pyperformance/benchmarks/bm_regex_dna.py
Thanks.
On 14.10.2020 15:16, Pablo Galindo Salgado wrote:
> Hi!
>
> I have updated the branch benchmarks in the pyperformance server and now they
> include 3.9. There are
> some benchmarks that are faster but on the other hand some benchmarks are
> substantially slower, pointing
> at a possible performance regression in 3.9 in some aspects. In particular some
> tests like "unpack sequence" are
> almost 20% slower. As there are some other tests were 3.9 is faster, is not fair
> to conclude that 3.9 is slower, but
> this is something we should look into in my opinion.
>
> You can check these benchmarks I am talking about by:
>
> * Go here: https://speed.python.org/comparison/
> * In the left bar, select "lto-pgo latest in branch '3.9'" and "lto-pgo latest
> in branch '3.8'"
> * To better read the plot, I would recommend to select a "Normalization" to the
> 3.8 branch (this is in the top part of the page)
> and to check the "horizontal" checkbox.
>
> These benchmarks are very stable: I have executed them several times over the
> weekend yielding the same results and,
> more importantly, they are being executed on a server specially prepared to
> running reproducible benchmarks: CPU affinity,
> CPU isolation, CPU pinning for NUMA nodes, CPU frequency is fixed, CPU governor
> set to performance mode, IRQ affinity is
> disabled for the benchmarking CPU nodes...etc so you can trust these numbers.
>
> I kindly suggest for everyone interested in trying to improve the 3.9 (and
> master) performance, to review these benchmarks
> and try to identify the problems and fix them or to find what changes introduced
> the regressions in the first place. All benchmarks
> are the ones being executed by the pyperformance suite
> (https://github.com/python/pyperformance) so you can execute them
> locally if you need to.
>
> ---
>
> On a related note, I am also working on the speed.python.org
> <http://speed.python.org> server to provide more automation and
> ideally some integrations with GitHub to detect performance regressions. For
> now, I have done the following:
>
> * Recompute benchmarks for all branches using the same version of
> pyperformance (except master) so they can
> be compared with each other. This can only be seen in the "Comparison"
> tab: https://speed.python.org/comparison/
> * I am setting daily builds of the master branch so we can detect performance
> regressions with daily granularity. These
> daily builds will be located in the "Changes" and "Timeline" tabs
> (https://speed.python.org/timeline/).
> * Once the daily builds are working as expected, I plan to work on trying to
> automatically comment or PRs or on bpo if
> we detect that a commit has introduced some notable performance regression.
>
> Regards from sunny London,
> Pablo Galindo Salgado.
>
> _______________________________________________
> python-committers mailing list -- python-committers@python.org
> To unsubscribe send an email to python-committers-leave@python.org
> https://mail.python.org/mailman3/lists/python-committers.python.org/
> Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/G3LB4BCAY7T7WG22YQJNQ64XA4BXBCT4/
> Code of Conduct: https://www.python.org/psf/codeofconduct/
>
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Experts (#1, Oct 14 2020)
>>> Python Projects, Coaching and Support ... https://www.egenix.com/
>>> Python Product Development ... https://consulting.egenix.com/
________________________________________________________________________
::: We implement business ideas - efficiently in both time and costs :::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
https://www.egenix.com/company/contact/
https://www.malemburg.com/