On Fri, Apr 11, 2014 at 11:26 PM, Sturla Molden <sturla.molden@gmail.com> wrote:
On 11/04/14 20:47, Nathaniel Smith wrote:
Also, while Windows is maybe in the worst shape, all platforms would seriously benefit from the existence of a reliable speed-competitive binary-distribution-compatible BLAS that doesn't break fork().
Windows is worst off, yes.
I don't think fork breakage by Accelerate is a big problem on Mac OS X. Apple has made clear that only POSIX APIs are fork safe. And actually this is now recognized as an error in multiprocessing and fixed in Python 3.4:
multiprocessing.set_start_method('spawn')
I don't really care whether it's *documented* that BLAS and fork are incompatible. I care whether it *works*, because it is useful functionality :-). The spawn mode is fine and all, but (a) the presence of something in 3.4 helps only a minority of users, (b) "spawn" is not a full replacement for fork; with large read-mostly data sets it can be a *huge* win to load them into the parent process and then let them be COW-inherited by forked children. ATM the only other way to work with a data set that's larger than memory-divided-by-numcpus is to explicitly set up shared memory, and this is *really* hard for anything more complicated than a single flat array.
On Linux the distributions will usually ship with prebuilt ATLAS.
And it's generally recommended that everyone rebuild their own ATLAS anyway. I can do it, but I'd much rather be able to install a BLAS library that just worked. (Presumably this is a large part of why scipy-stack distributors prefer MKL over ATLAS.) If it comes down to it then of course I'd rather have a Windows-only BLAS than no BLAS at all. I just don't think we should be setting our sights so low at this point. The marginal cost of portability doesn't seem high. Besides, even Windows users will benefit more from having a standard cross-platform BLAS that everyone uses -- it would mean lots more people familiar with the library's quirks, better testing, etc. -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org