Hi,
On Tue, May 26, 2015 at 12:53 PM, Julian Taylor
On 05/26/2015 04:56 PM, Matthew Brett wrote:
Hi,
This morning I was wondering whether we ought to plan to devote some resources to collaborating with the OpenBLAS team.
It is relatively easy to add tests using Python / numpy. We like tests. Why don't we propose a collaboration with OpenBLAS where we build and test numpy with every / most / some commits of OpenBLAS, and try to make it easy for the OpenBLAS team to add tests. Maybe we can use and add to the list of machines on which OpenBLAS is tested [1]? We Berkeley Pythonistas can certainly add the machines at our buildbot farm [2]. Maybe the Julia / R developers would be interested to help too?
Technically we only need a single machine with the newest instruction set available. All other cases could then be tested via a virtual machine that only exposes specific instruction sets (e.g. qemu which could technically also emulate stuff the host does not have).
Concerning test generation there is a huge parameter space that needs testing due with openblas, at least some of it would need to be automated/fuzzed. We also need specific preconditioning of memory to test failure cases openblas had in the past, E.g. filling memory around the matrices with nans and also somehow filling openblas own temporary buffers with some signaling values (might require special built openblas if _MALLOC_PERTURB does not work).
Maybe it would be feasible to write a hypothesis [0] strategy for some of the blas stuff to automate the parameter exploration.
And then we'd need to run everything under valgrind as due to the assembler implementation of openblas we can't use the faster address sanitizers gcc and clang now provide.
All this sounds extremely useful. What do you think we should do next? How feasible is it to start to set this kind of thing up for our own use, and then offer to integrate with OpenBLAS? Is there anyone out there who knows the Julia and / or R community well enough to know if they would be interested to help? What kind of help do you think we need? Money for a machine? Cheers, Matthew