[Numpy-discussion] Really cruel draft of vbench setup for NumPy (.add.reduce benchmarks since 2011)

[Yaroslav Halchenko]

On Fri, 06 Sep 2013, josef.pktd at gmail.com wrote:

> On Fri, Sep 6, 2013 at 3:21 PM, Yaroslav Halchenko <lists at onerussian.com> wrote:
> > FWIW -- updated runs of the benchmarks are available at
> > http://yarikoptic.github.io/numpy-vbench which now include also
> > maintenance/1.8.x branch (no divergences were detected yet).  There are
> > only recent improvements as I see and no new (but some old ones are
> > still there, some might be specific to my CPU here) performance
> > regressions.

> You would have enough data to add some quality control bands to the
> charts (like cusum chart for example).
> Then it would be possible to send a congratulation note or ring an
> alarm bell without looking at all the plots.

well -- I did cook up some basic "detector" but I believe I haven't
adjusted it for multiple branches yet:
http://yarikoptic.github.io/numpy-vbench/#benchmarks-performance-analysis
you are welcome to introduce additional (or replacement) detection goodness
http://github.com/yarikoptic/vbench/blob/HEAD/vbench/analysis.py
and plotting is done here I believe:
https://github.com/yarikoptic/vbench/blob/HEAD/vbench/benchmark.py#L155

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik