[Numpy-discussion] Accelerate or OpenBLAS for numpy / scipy wheels?

[Matthew Brett]

Hi,

On Tue, Jun 28, 2016 at 7:33 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
> On Tue, Jun 28, 2016 at 2:55 PM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>>
>> Hi,
>>
>> On Tue, Jun 28, 2016 at 5:25 AM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> >
>> > On Mon, Jun 27, 2016 at 9:46 PM, Matthew Brett <matthew.brett at gmail.com>
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> I just succeeded in getting an automated dual arch build of numpy and
>> >> scipy, using OpenBLAS.  See the last three build jobs in these two
>> >> build matrices:
>> >>
>> >> https://travis-ci.org/matthew-brett/numpy-wheels/builds/140388119
>> >> https://travis-ci.org/matthew-brett/scipy-wheels/builds/140684673
>> >>
>> >> Tests are passing on 32 and 64-bit.
>> >>
>> >> I didn't upload these to the usual Rackspace container at
>> >> wheels.scipy.org to avoid confusion.
>> >>
>> >> So, I guess the question now is - should we switch to shipping
>> >> OpenBLAS wheels for the next release of numpy and scipy?  Or should we
>> >> stick with the Accelerate framework that comes with OSX?
>> >>
>> >> In favor of the Accelerate build : faster to build, it's what we've
>> >> been doing thus far.
>
>
> Faster to build isn't really an argument right? Should be the same build
> time except for building OpenBLAS itself once per OpenBLAS version. And only
> applies to building wheels for releases - nothing changes for source builds
> done by users on OS X. If build time ever becomes a real issue, then
> dropping the dual arch stuff is probably the way to go - the 32-bit builds
> make very little sense these days.

Yes, that's true, but as you know, the OSX system and Python.org
Pythons are still dual arch, so technically a matching wheel should
also be dual arch.   I agree that we're near the point where there's
near zero likelihood that the 32-bit arch will ever get exercised.

> What we've been doing thus far - that is the more important argument.
> There's a risk in switching, we may encounter new bugs or lose some
> performance in particular functions.
>
>>
>> >>
>> >> In favor of OpenBLAS build : allows us to commit to one BLAS / LAPACK
>> >> library cross platform,
>
>
> This doesn't really matter too much imho, we have to support Accelerate
> either way.
>
>>
>> when we have the Windows builds working.
>> >> Faster to fix bugs with good support from main developer.  No
>> >> multiprocessing crashes for Python 2.7.
>
>
> This is probably the main reason to make the switch, if we decide to do
> that.
>
>>
>>   I'm still a bit nervous about OpenBLAS, see
>> > https://github.com/scipy/scipy/issues/6286. That was with version
>> > 0.2.18,
>> > which is pretty recent.
>>
>> Well - we are committed to OpenBLAS already for the Linux wheels, so
>> if that failure was due to an error in OpenBLAS, we'll have to report
>> it and get it fixed / fix it ourselves upstream.
>
>
> Indeed. And those wheels have been downloaded a lot already, without any
> issues being reported.
>
> I'm +0 on the proposal - the risk seems acceptable, but the reasons to make
> the switch are also not super compelling.

I guess I'm about +0.5 (multiprocessing, simplifying mainstream blas /
lapack support) - I'm floating it now because I hadn't got the build
machinery working before.

Cheers,

Matthew