[pypy-dev] hexiom2 benchmark

[Michael Hudson-Doyle]

Hi all,

Long time no see!  Apologies if this is not the right place to ask this
question.

I'm trying to run the benchmarks from speed.pypy.org on my system (just
with cpython for now -- there is a chance that Linaro will be doing some
work on the performance of Python on ARM at some point), but when I check
out https://bitbucket.org/pypy/benchmarks/overview and run "./runner.py
--fast" I get this:

Running hexiom2...
INFO:root:Running /usr/bin/python
/home/mwhudson/src/benchmarks/own/hexiom2.py -n 5
INFO:root:Running /usr/bin/python
/home/mwhudson/src/benchmarks/own/hexiom2.py -n 5
Traceback (most recent call last):
  File "./runner.py", line 302, in <module>
    main(sys.argv[1:])
  File "./runner.py", line 283, in main
    full_store=full_store, branch=branch)
  File "./runner.py", line 41, in run_and_store
    results = perf.main(opts, funcs)
  File "/home/mwhudson/src/benchmarks/unladen_swallow/perf.py", line 1617,
in main
    bench_result = func(base_cmd_prefix, changed_cmd_prefix, options)
  File "/home/mwhudson/src/benchmarks/benchmarks.py", line 17, in BM
    return SimpleBenchmark(Measure, *args, **kwds)
  File "/home/mwhudson/src/benchmarks/unladen_swallow/perf.py", line 462,
in SimpleBenchmark
    return CompareBenchmarkData(base_data, changed_data, options)
  File "/home/mwhudson/src/benchmarks/unladen_swallow/perf.py", line 736,
in CompareBenchmarkData
    return CompareMultipleRuns(base_times, changed_times, options)
  File "/home/mwhudson/src/benchmarks/unladen_swallow/perf.py", line 704,
in CompareMultipleRuns
    significant, t_score = IsSignificant(base_times, changed_times)
  File "/home/mwhudson/src/benchmarks/unladen_swallow/perf.py", line 189,
in IsSignificant
    t_score = TScore(sample1, sample2)
  File "/home/mwhudson/src/benchmarks/unladen_swallow/perf.py", line 170,
in TScore
    return (avg(sample1) - avg(sample2)) / math.sqrt(error * 2)
ZeroDivisionError: float division by zero

is this just an artifact of running --fast?  It seems the code could
probably do with being a bit more robust?

Cheers,
mwh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130501/cfdc8c9b/attachment.html>