Speed January 2016

speed@python.org

3 participants
3 discussions

Re: [Speed] People interested in reworking the benchmark suite in 2016?

by Antoine Pitrou

On Sun, 03 Jan 2016 19:57:50 +0000 Brett Cannon <brett(a)python.org> wrote: > With the planned move to GitHub, there is an opportunity to try and rework > the set of benchmarks -- and anything else -- in 2016 by starting a new > benchmark repo from scratch. E.g., modern numeric benchmarks, long-running > benchmarks that warm up JITs, using pip with pegged bugfix versions so we > stop shipping library code with the benchmarks, etc. That feature list should be discussed :-) For example I don't think downloading dependencies with pip is desirable compared to the ease of cloning a single repo (also what happens if the files become unavailable?). > We could also > standardize results output -- e.g. should we just make everything run under > codespeed? -- so that the benchmarks are easy to run locally for one-off > results as well as continuous benchmarking for trend details with a common > benchmark driver? I don't know who uses codespeed apart from PyPy, but asv is a possible benchmark runner: https://asv.readthedocs.org/en/latest/index.html > Would people be interested and motivated enough in getting representatives > from the various Python implementations together at PyCon and have a BoF to > discuss what we want from a proper, unified, baseline benchmark suite and > see if we can pull one together -- or at least start one -- in 2016? I probably won't be at PyCon. Regards Antoine.

6 years

Re: [Speed] People interested in reworking the benchmark suite in 2016?

by Maciej Fijalkowski

count me in (even if I don't make it to pycon) On Sun, Jan 3, 2016 at 9:57 PM, Brett Cannon <brett(a)python.org> wrote: > With the planned move to GitHub, there is an opportunity to try and rework > the set of benchmarks -- and anything else -- in 2016 by starting a new > benchmark repo from scratch. E.g., modern numeric benchmarks, long-running > benchmarks that warm up JITs, using pip with pegged bugfix versions so we > stop shipping library code with the benchmarks, etc. We could also > standardize results output -- e.g. should we just make everything run under > codespeed? -- so that the benchmarks are easy to run locally for one-off > results as well as continuous benchmarking for trend details with a common > benchmark driver? > > Would people be interested and motivated enough in getting representatives > from the various Python implementations together at PyCon and have a BoF to > discuss what we want from a proper, unified, baseline benchmark suite and > see if we can pull one together -- or at least start one -- in 2016? > > _______________________________________________ > Speed mailing list > Speed(a)python.org > https://mail.python.org/mailman/listinfo/speed >

6 years

People interested in reworking the benchmark suite in 2016?

by Brett Cannon

With the planned move to GitHub, there is an opportunity to try and rework the set of benchmarks -- and anything else -- in 2016 by starting a new benchmark repo from scratch. E.g., modern numeric benchmarks, long-running benchmarks that warm up JITs, using pip with pegged bugfix versions so we stop shipping library code with the benchmarks, etc. We could also standardize results output -- e.g. should we just make everything run under codespeed? -- so that the benchmarks are easy to run locally for one-off results as well as continuous benchmarking for trend details with a common benchmark driver? Would people be interested and motivated enough in getting representatives from the various Python implementations together at PyCon and have a BoF to discuss what we want from a proper, unified, baseline benchmark suite and see if we can pull one together -- or at least start one -- in 2016?

6 years

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Speed January 2016