On Tue, 15 Oct 2013, Nathaniel Smith wrote:
and I think it might be worth letting people know about my little project. I would really appreciate your sincere feedback (e.g. "not worth it" would be valuable too). Here is the title/abstract
numpy-vbench -- speed benchmarks for NumPy
http://yarikoptic.github.io/numpy-vbench provides collection of speed performance benchmarks for NumPy. Benchmarking of multiple maintenance and current development branches allows not only to timely react to new performance regressions, but also to compare NumPy performance across releases. Your contributions would help to guarantee that your code does not become slower with a new NumPy release.
What do you have to lose?
time? ;)
btw -- fresh results are here http://yarikoptic.github.io/numpy-vbench/ .
I have tuned benchmarking so it now reflects the best performance across multiple executions of the whole battery, thus eliminating spurious variance if estimate is provided from a single point in time. Eventually I expect many of those curves to become even "cleaner".
On another note, what do you think of moving the vbench benchmarks into the main numpy tree? We already require everyone who submits a bug fix to add a test; there are a bunch of speed enhancements coming in these days and it would be nice if we had some way to ask people to submit a benchmark along with each one so that we know that the enhancement stays enhanced...
I would be thrilled to do so, but #1 I would need time I currently do not have to harmonize my setup for such inclusion #2 I am still not sure if my tune ups on top of vbench should get merged as is https://github.com/pydata/vbench/pull/33 #3 vbench itself might go through a considerable RF if someone finds some spare time: https://github.com/pydata/vbench/issues/34 If merge for the PR #2 would happen, and test_perf.py from pandas would get integrated within vbench -- then it might be logical to think about #1 regardless of the ultimate resolution to #3 ;) Whenever I have some time for it I would try to push in that direction, and I do not think it is unfeasible to see all of that done actually by PyCon 2014 (so the talk will have a different url ;) ), I just can't promise ATM. -- Yaroslav O. Halchenko, Ph.D. http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Senior Research Associate, Psychological and Brain Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik