On Sat, Aug 6, 2016 at 9:20 AM, Benjamin Root <ben.v.root@gmail.com> wrote:

Don't know if it is what you are looking for, but NumPy has a built-in suite of benchmarks: http://docs.scipy.org/doc/numpy/reference/generated/numpy.testing.Tester.bench.html

That's the very old (now unused) benchmark runner. Numpy has had an ASV test suite for a while, see https://github.com/numpy/numpy/tree/master/benchmarks for how to run it.

Also, some projects have taken to utilizing the "airspeed velocity" utility to track benchmarking stats for their projects. I know astropy utilizes it. So, maybe their benchmarks might be a good starting point since they utilize numpy heavily?

Cheers!
Ben Root

On Fri, Aug 5, 2016 at 3:42 AM, Papa, Florin <florin.papa@intel.com> wrote:

Hi,

This is Florin Papa from the Dynamic Scripting Languages Optimizations team in Intel Corporation.

Our team is working on optimizing the PyPy interpreter and part of this work is to find and fix incompatibilities between NumPy and PyPy. Does anyone have knowledge of real life workloads that use NumPy and cannot be run using PyPy?

We are also interested in creating a repository with relevant benchmarks for real world usage of NumPy, like GUPB for CPython, but we have not found such workloads for NumPy.

The approach of GUPB is interesting (the whole application part that is, the rest looks much more cumbersome than ASV benchmarks), but of course easier to create for Python than for Numpy. You'd need to find whole applications that spend most of their time in numpy but not in too small a set of numpy functions. Maybe benchmark suites of other projects aren't such a bad idea for that. Or spend a bit of time collecting relevant published ipython notebooks.

Ralf