[Numpy-discussion] Windows wheels, built, but should we deploy?
chris.barker at noaa.gov
Fri Mar 4 15:29:52 EST 2016
+1 -- thanks for doing all this work.
There is a HUGE amount you can do with numpy that doesn't give a whit about
how fast .dot() et all are. If you really do need that to be fast as
possible, you can pug in a faster build later.
This is great.
Just as one example -- I teach a general python class every year --I do
only one session on numpy/scipy. If I can expect my students to be able to
simply pip install the core scipy stack, this will be SO much easier.
On Thu, Mar 3, 2016 at 8:42 PM, Matthew Brett <matthew.brett at gmail.com>
> I propose that we upload Windows wheels to pypi. The wheels are
> likely to be stable and relatively easy to maintain, but will have
> slower performance than other versions of numpy linked against faster
> BLAS / LAPACK libraries.
> There's a long discussion going on at issue github #5479 , where
> the old problem of Windows wheels for numpy came up.
> For those of you not following this issue, the current situation for
> community-built numpy Windows binaries is dire:
> * We have not so far provided windows wheels on pypi, so `pip install
> numpy` on Windows will bring you a world of pain;
> * Until recently we did provide .exe "superpack" installers on
> sourceforge, but these became increasingly difficult to build and we
> gave up building them as of the latest (1.10.4) release.
> Despite this, popularity of Windows wheels on pypi is high. A few
> weeks ago, Donald Stufft ran a query for the binary wheels most often
> downloaded from pypi, for any platform  . The top five most
> downloaded were (n_downloads, name):
> 5445, cryptography-1.2.1-cp27-none-win_amd64.whl
> 5243, matplotlib-1.4.0-cp34-none-win32.whl
> 5241, scikit_learn-0.15.1-cp34-none-win32.whl
> 4573, pandas-0.17.1-cp27-none-win_amd64.whl
> So a) the OSX numpy wheel is very popular and b) despite the fact that
> we don't provide a numpy wheel for Windows, matplotlib, sckit_learn
> and pandas, that depend on numpy, are the 3rd, 4th and 5th most
> downloaded wheels as of a few weeks ago.
> So, there seems to be a large appetite for numpy wheels.
> Current proposal:
> I have now built numpy wheels, using the ATLAS blas / lapack library -
> the build is automatic and reproducible .
> I chose ATLAS to build against, rather than, say OpenBLAS, because
> we've had some significant worries in the past about the reliability
> of OpenBLAS, and I thought it better to err on the side of
> However, these builds are relatively slow for matrix multiply and
> other linear algebra routines compared numpy built against OpenBLAS or
> MKL (which we cannot use because of its license) . In my very
> crude array test of a dot product and matrix inversion, the ATLAS
> wheels were 2-3 times slower than MKL. Other benchmarks on Julia
> found about the same result for ATLAS vs OpenBLAS on 32-bit bit, but a
> much bigger difference on 64-bit (for an earlier version of ATLAS than
> we are currently using) .
> So, our numpy wheels likely to be stable and give correct results, but
> will be somewhat slow for linear algebra.
> I propose that we upload these ATLAS wheels to pypi. The upside is
> that this gives our Windows users a much better experience with pip,
> and allows other developers to build Windows wheels that depend on
> numpy. The downside is that these will not be optimized for
> performance on modern processors. In order to signal that, I propose
> adding the following text to the numpy pypi front page:
> All numpy wheels distributed from pypi are BSD licensed.
> Windows wheels are linked against the ATLAS BLAS / LAPACK library,
> restricted to SSE2 instructions, so may not give optimal linear
> algebra performance for your machine. See
> http://docs.scipy.org/doc/numpy/user/install.html for alternatives.
> In a way this is very similar to our previous situation, in that the
> superpack installers also used ATLAS - in fact an older version of
> Once we are up and running with numpy wheels, we can consider whether
> we should switch to other BLAS libraries, such as OpenBLAS or BLIS
> (see ).
> I'm posting here hoping for your feedback...
>  https://github.com/numpy/numpy/issues/5479
>  https://gist.github.com/dstufft/1dda9a9f87ee7121e0ee
>  https://ci.appveyor.com/project/matthew-brett/np-wheel-builder
>  http://mingwpy.github.io/blas_lapack.html#intel-math-kernel-library
>  https://github.com/numpy/numpy/issues/5479#issuecomment-185033668
>  https://github.com/numpy/numpy/issues/7372
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
Christopher Barker, Ph.D.
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion