[Numpy-discussion] Really cruel draft of vbench setup for NumPy (.add.reduce benchmarks since 2011)

[Sebastian Berg]

On Mon, 2013-05-06 at 12:11 -0400, Yaroslav Halchenko wrote:
> On Mon, 06 May 2013, Sebastian Berg wrote:
> 
> > > if you care to tune it up/extend and then I could fire it up again on
> > > that box (which doesn't do anything else ATM AFAIK).   Since majority of
> > > time is spent actually building it (did it with ccache though) it would
> > > be neat if you come up with more of benchmarks to run which you might
> > > think could be interesting/important.
> 
> > I think this is pretty cool! Probably would be a while until there are
> > many tests, but if you or someone could set such thing up it could
> > slowly grow when larger code changes are done?
> 
> that is the idea but it would be nice to gather such simple
> benchmark-tests.  if you could hint on the numpy functionality you think
> especially worth benchmarking (I know -- there is a lot of things
> which could be set to be benchmarked) -- that would be a nice starting
> point: just list functionality/functions  you consider of primary
> interest. and either it is worth testing for different types or just a
> gross estimate (e.g. for the selection of types in a loop)
> 
> As for myself -- I guess I will add fancy indexing and slicing tests.
> 

Indexing/assignment was the first thing I thought of too (also because
fancy indexing/assignment really could use some speedups...). Other then
that maybe some timings for small arrays/scalar math, but that might be
nice for that GSoC project.

Maybe array creation functions, just to see if performance bugs should
sneak into something that central. But can't think of something else
that isn't specific functionality.

- Sebastian

> Adding them is quite easy: have a look at
> https://github.com/yarikoptic/numpy-vbench/blob/master/vb_reduce.py
> which is actually a bit more cumbersome because of running them for
> different types.
> This one is more obvious:
> https://github.com/yarikoptic/numpy-vbench/blob/master/vb_io.py
>