Hi,

I released perf 0.9.6 with many changes. First, "Mean +- std dev" is now displayed, instead of "Median +- std dev", as a result of the previous thread on this list. The median is still accessible via the stats command. By the way, the "stats" command now displays "Median +- MAD" instead of "Median +- std dev".

I broke the API to fix an old mistake. I used the term "sample" for a single value, whereas a "sample" in statistics is a set of values (one or more), and so the term is misused. I replace "sample" with "value" and "samples" with "values" everywhere in perf.

http://perf.readthedocs.io/en/latest/changelog.html#version-0-9-6-2017-03-15

Version 0.9.6 (2017-03-15)

Major change:

• Display Mean +- std dev instead of Median +- std dev

Enhancements:

• Add a new Runner.bench_command() method to measure the execution time of a command.
• Add mean(), median_abs_dev() and stdev() methods to Benchmark
• check command: test also minimum and maximum compared to the mean

Major API change, rename "sample" to "value":

• Rename attributes and methods:

  • Benchmark.bench_sample_func() => Benchmark.bench_time_func().
  • Run.samples => Run.values
  • Benchmark.get_samples() => Benchmark.get_values()
  • get_nsample() => get_nvalue()
  • Benchmark.format_sample() => Benchmark.format_value()
  • Benchmark.format_samples() => Benchmark.format_values()

• Rename Runner command line options:

  • --samples => --values
  • --debug-single-sample => --debug-single-value

Changes:

• convert: Remove --remove-outliers option
• check command now tests stdev/mean, instead of testing stdev/median
• setup.py: statistics dependency is now installed using extras_require to support setuptools 18 and newer
• Add setup.cfg to enable universal builds: same wheel package for Python 2 and Python 3
• Add perf.VERSION constant: tuple of int
• JSON version 6: write metadata common to all benchmarks (common to all runs of all benchmarks) at the root; rename 'samples' to 'values' in runs.

Victor