[Numpy-discussion] Really cruel draft of vbench setup for NumPy (.add.reduce benchmarks since 2011)
lists at onerussian.com
Tue Jul 9 12:10:07 EDT 2013
Julian Taylor contributed some benchmarks he was "concerned" about, so
now the collection is even better.
I will keep updating tests on the same url:
[it is now running and later I will upload with more commits for higher temporal fidelity]
of particular interest for you might be:
some minor consistent recent losses in
seems have lost more than 25% of performance throughout the timeline
"fast" calls to all/any seemed to be hurt twice in their life time now running
*3 times slower* than in 2011 -- inflection points correspond to regressions
and/or their fixes in those functions to bring back performance on "slow"
cases (when array traversal is needed, e.g. on arrays of zeros for any)
On Mon, 01 Jul 2013, Yaroslav Halchenko wrote:
> FWIW -- updated plots with contribution from Julian Taylor
> On Mon, 01 Jul 2013, Yaroslav Halchenko wrote:
> > Hi Guys,
> > not quite the recommendations you expressed, but here is my ugly
> > attempt to improve benchmarks coverage:
> > http://www.onerussian.com/tmp/numpy-vbench-20130701/index.html
> > initially I also ran those ufunc benchmarks per each dtype separately,
> > but then resulting webpage is loong which brings my laptop on its knees
> > by firefox. So I commented those out for now, and left only "summary"
> > ones across multiple datatypes.
> > There is a bug in sphinx which forbids embedding some figures for
> > vb_random "as is", so pardon that for now...
> > I have not set cpu affinity of the process (but ran it at nice -10), so may be
> > that also contributed to variance of benchmark estimates. And there probably
> > could be more of goodies (e.g. gc control etc) to borrow from
> > https://github.com/pydata/pandas/blob/master/vb_suite/test_perf.py which I have
> > just discovered to minimize variance.
> > nothing really interesting was pin-pointed so far, besides that
> > - svd became a bit faster since few months back ;-)
> > http://www.onerussian.com/tmp/numpy-vbench-20130701/vb_vb_linalg.html
> > - isnan (and isinf, isfinite) got improved
> > http://www.onerussian.com/tmp/numpy-vbench-20130701/vb_vb_ufunc.html#numpy-isnan-a-10types
> > - right_shift got a miniscule slowdown from what it used to be?
> > http://www.onerussian.com/tmp/numpy-vbench-20130701/vb_vb_ufunc.html#numpy-right-shift-a-a-3types
> > As before -- current code of those benchmarks collection is available
> > at http://github.com/yarikoptic/numpy-vbench/pull/new/master
> > if you have specific snippets you would like to benchmark -- just state them
> > here or send a PR -- I will add them in.
> > Cheers,
> > On Tue, 07 May 2013, Daπid wrote:
> > > On 7 May 2013 13:47, Sebastian Berg <sebastian at sipsolutions.net> wrote:
> > > > Indexing/assignment was the first thing I thought of too (also because
> > > > fancy indexing/assignment really could use some speedups...). Other then
> > > > that maybe some timings for small arrays/scalar math, but that might be
> > > > nice for that GSoC project.
> > > Why not going bigger? Ufunc operations on big arrays, CPU and memory bound.
> > > Also, what about interfacing with other packages? It may increase the
> > > compiling overhead, but I would like to see Cython in action (say,
> > > only last version, maybe it can be fixed).
> > > _______________________________________________
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion at scipy.org
> > > http://mail.scipy.org/mailman/listinfo/numpy-discussion
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
More information about the NumPy-Discussion