2016-06-02 10:38 GMT+02:00 Armin Rigo email@example.com:
Looks good. IMHO the important bit is that
timeit is simple to use,
readily available, and gives just a number, which makes it very
attractive for people.
By default min & max are hidden. You can show them using -v option.
To make the output even simpler, maybe the standard deviation can be displayed "in english". Something like:
Usually, timeit it used to compare two versions of Python. Maybe we should focus on this use case, and check if the difference is significant, as perf.py does? By default, perf.py does not display any number if the difference is not significant. I like this behaviour, even if it can be surprising for the first time.
For the CLI, we can extend timeit CLI to accept the path/name of two python binaries. Or we can use something like pybench to store result into files and then load & compare two files.