[Numpy-discussion] The BLAS problem (was: Re: Wiki page for building numerical stuff on Windows)
njs at pobox.com
Fri Apr 11 18:39:39 EDT 2014
On Fri, Apr 11, 2014 at 11:26 PM, Sturla Molden <sturla.molden at gmail.com> wrote:
> On 11/04/14 20:47, Nathaniel Smith wrote:
>> Also, while Windows is maybe in the worst shape, all platforms would
>> seriously benefit from the existence of a reliable speed-competitive
>> binary-distribution-compatible BLAS that doesn't break fork().
> Windows is worst off, yes.
> I don't think fork breakage by Accelerate is a big problem on Mac OS X.
> Apple has made clear that only POSIX APIs are fork safe. And actually
> this is now recognized as an error in multiprocessing and fixed in
> Python 3.4:
I don't really care whether it's *documented* that BLAS and fork are
incompatible. I care whether it *works*, because it is useful
The spawn mode is fine and all, but (a) the presence of something in
3.4 helps only a minority of users, (b) "spawn" is not a full
replacement for fork; with large read-mostly data sets it can be a
*huge* win to load them into the parent process and then let them be
COW-inherited by forked children. ATM the only other way to work with
a data set that's larger than memory-divided-by-numcpus is to
explicitly set up shared memory, and this is *really* hard for
anything more complicated than a single flat array.
> On Linux the distributions will usually ship with prebuilt ATLAS.
And it's generally recommended that everyone rebuild their own ATLAS
anyway. I can do it, but I'd much rather be able to install a BLAS
library that just worked. (Presumably this is a large part of why
scipy-stack distributors prefer MKL over ATLAS.)
If it comes down to it then of course I'd rather have a Windows-only
BLAS than no BLAS at all. I just don't think we should be setting our
sights so low at this point. The marginal cost of portability doesn't
Besides, even Windows users will benefit more from having a standard
cross-platform BLAS that everyone uses -- it would mean lots more
people familiar with the library's quirks, better testing, etc.
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
More information about the NumPy-Discussion