[Numpy-discussion] Really cruel draft of vbench setup for NumPy (.add.reduce benchmarks since 2011)

Yaroslav Halchenko lists at onerussian.com
Tue Jul 9 12:10:07 EDT 2013


Julian Taylor contributed some benchmarks he was "concerned" about, so
now the collection is even better.

I will keep updating tests on the same url:
http://www.onerussian.com/tmp/numpy-vbench/
[it is now running and later I will upload with more commits for higher temporal fidelity]

of particular interest for you might be:
some minor consistent recent losses in
http://www.onerussian.com/tmp/numpy-vbench/vb_vb_io.html#strided-assign-float64
http://www.onerussian.com/tmp/numpy-vbench/vb_vb_io.html#strided-assign-float32
http://www.onerussian.com/tmp/numpy-vbench/vb_vb_io.html#strided-assign-int16
http://www.onerussian.com/tmp/numpy-vbench/vb_vb_io.html#strided-assign-int8

seems have lost more than 25% of performance throughout the timeline
http://www.onerussian.com/tmp/numpy-vbench/vb_vb_io.html#memcpy-int8 

"fast" calls to all/any seemed to be hurt twice in their life time now running
*3 times slower* than in 2011 -- inflection points correspond to regressions
and/or their fixes in those functions to bring back performance on "slow"
cases (when array traversal is needed, e.g. on arrays of zeros for any)

http://www.onerussian.com/tmp/numpy-vbench/vb_vb_reduce.html#numpy-all-fast
http://www.onerussian.com/tmp/numpy-vbench/vb_vb_reduce.html#numpy-any-fast

Enjoy

On Mon, 01 Jul 2013, Yaroslav Halchenko wrote:

> FWIW -- updated plots with contribution from Julian Taylor
> http://www.onerussian.com/tmp/numpy-vbench-20130701/vb_vb_indexing.html#mmap-slicing
> ;-)

> On Mon, 01 Jul 2013, Yaroslav Halchenko wrote:

> > Hi Guys,

> > not quite the recommendations you expressed,  but here is my ugly
> > attempt to improve benchmarks coverage:

> > http://www.onerussian.com/tmp/numpy-vbench-20130701/index.html

> > initially I also ran those ufunc benchmarks per each dtype separately,
> > but then resulting webpage is loong which brings my laptop on its knees
> > by firefox.  So I commented those out for now, and left only "summary"
> > ones across multiple datatypes.

> > There is a bug in sphinx which forbids embedding some figures for
> > vb_random "as is", so pardon that for now...

> > I have not set cpu affinity of the process (but ran it at nice -10), so  may be
> > that also contributed to variance of benchmark estimates.  And there probably
> > could be more of goodies (e.g. gc control etc) to borrow from
> > https://github.com/pydata/pandas/blob/master/vb_suite/test_perf.py which I have
> > just discovered to minimize variance.

> > nothing really interesting was pin-pointed so far, besides that 

> > - svd became a bit faster since few months back ;-)

> > http://www.onerussian.com/tmp/numpy-vbench-20130701/vb_vb_linalg.html

> > - isnan (and isinf, isfinite) got improved

> > http://www.onerussian.com/tmp/numpy-vbench-20130701/vb_vb_ufunc.html#numpy-isnan-a-10types

> > - right_shift got a miniscule slowdown from what it used to be?

> > http://www.onerussian.com/tmp/numpy-vbench-20130701/vb_vb_ufunc.html#numpy-right-shift-a-a-3types

> > As before -- current code of those benchmarks collection is available
> > at http://github.com/yarikoptic/numpy-vbench/pull/new/master

> > if you have specific snippets you would like to benchmark -- just state them
> > here or send a PR -- I will add them in.

> > Cheers,

> > On Tue, 07 May 2013, Daπid wrote:

> > > On 7 May 2013 13:47, Sebastian Berg <sebastian at sipsolutions.net> wrote:
> > > > Indexing/assignment was the first thing I thought of too (also because
> > > > fancy indexing/assignment really could use some speedups...). Other then
> > > > that maybe some timings for small arrays/scalar math, but that might be
> > > > nice for that GSoC project.

> > > Why not going bigger? Ufunc operations on big arrays, CPU and memory bound.

> > > Also, what about interfacing with other packages? It may increase the
> > > compiling overhead, but I would like to see Cython in action (say,
> > > only last version, maybe it can be fixed).
> > > _______________________________________________
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion at scipy.org
> > > http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate,     Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        



More information about the NumPy-Discussion mailing list