Re: [SciPy-Dev] [Numpy-discussion] NumPy 1.12.0 release
Hi, On Tue, Jan 17, 2017 at 5:56 AM, Neal Becker <ndbecker2@gmail.com> wrote:
Charles R Harris wrote:
Hi All,
I'm pleased to announce the NumPy 1.12.0 release. This release supports Python 2.7 and 3.4-3.6. Wheels for all supported Python versions may be downloaded from PiPY <https://pypi.python.org/pypi?%3Aaction=pkg_edit&name=numpy>, the tarball and zip files may be downloaded from Github <https://github.com/numpy/numpy/releases/tag/v1.12.0>. The release notes and files hashes may also be found at Github <https://github.com/numpy/numpy/releases/tag/v1.12.0> .
NumPy 1.12.0rc 2 is the result of 418 pull requests submitted by 139 contributors and comprises a large number of fixes and improvements. Among the many improvements it is difficult to pick out just a few as standing above the others, but the following may be of particular interest or indicate areas likely to have future consequences.
* Order of operations in ``np.einsum`` can now be optimized for large speed improvements. * New ``signature`` argument to ``np.vectorize`` for vectorizing with core dimensions. * The ``keepdims`` argument was added to many functions. * New context manager for testing warnings * Support for BLIS in numpy.distutils * Much improved support for PyPy (not yet finished)
Enjoy,
Chuck
I've installed via pip3 on linux x86_64, which gives me a wheel. My question is, am I loosing significant performance choosing this pre-built binary vs. compiling myself? For example, my processor might have some more features than the base version used to build wheels.
I guess you are thinking about using this built wheel on some other machine? You'd have to be lucky for that to work; the wheel depends on the symbols it found at build time, which may not exist in the same places on your other machine. If it does work, the speed will primarily depend on your BLAS library. The pypi wheels should be pretty fast; they are built with OpenBLAS, which is at or near top of range for speed, across a range of platforms. Cheers, Matthew
Matthew Brett wrote:
Hi,
On Tue, Jan 17, 2017 at 5:56 AM, Neal Becker <ndbecker2@gmail.com> wrote:
Charles R Harris wrote:
Hi All,
I'm pleased to announce the NumPy 1.12.0 release. This release supports Python 2.7 and 3.4-3.6. Wheels for all supported Python versions may be downloaded from PiPY <https://pypi.python.org/pypi?%3Aaction=pkg_edit&name=numpy>, the tarball and zip files may be downloaded from Github <https://github.com/numpy/numpy/releases/tag/v1.12.0>. The release notes and files hashes may also be found at Github <https://github.com/numpy/numpy/releases/tag/v1.12.0> .
NumPy 1.12.0rc 2 is the result of 418 pull requests submitted by 139 contributors and comprises a large number of fixes and improvements. Among the many improvements it is difficult to pick out just a few as standing above the others, but the following may be of particular interest or indicate areas likely to have future consequences.
* Order of operations in ``np.einsum`` can now be optimized for large speed improvements. * New ``signature`` argument to ``np.vectorize`` for vectorizing with core dimensions. * The ``keepdims`` argument was added to many functions. * New context manager for testing warnings * Support for BLIS in numpy.distutils * Much improved support for PyPy (not yet finished)
Enjoy,
Chuck
I've installed via pip3 on linux x86_64, which gives me a wheel. My question is, am I loosing significant performance choosing this pre-built binary vs. compiling myself? For example, my processor might have some more features than the base version used to build wheels.
I guess you are thinking about using this built wheel on some other machine? You'd have to be lucky for that to work; the wheel depends on the symbols it found at build time, which may not exist in the same places on your other machine.
If it does work, the speed will primarily depend on your BLAS library.
The pypi wheels should be pretty fast; they are built with OpenBLAS, which is at or near top of range for speed, across a range of platforms.
Cheers,
Matthew
I installed using pip3 install, and it installed a wheel package. I did not build it - aren't wheels already compiled packages? So isn't it built for the common denominator architecture, not necessarily as fast as one I built myself on my own machine? My question is, on x86_64, is this potential difference large enough to bother with not using precompiled wheel packages?
On Tue, Jan 17, 2017 at 3:47 PM, Neal Becker <ndbecker2@gmail.com> wrote:
Matthew Brett wrote:
Hi,
On Tue, Jan 17, 2017 at 5:56 AM, Neal Becker <ndbecker2@gmail.com> wrote:
Charles R Harris wrote:
Hi All,
I'm pleased to announce the NumPy 1.12.0 release. This release supports Python 2.7 and 3.4-3.6. Wheels for all supported Python versions may be downloaded from PiPY <https://pypi.python.org/pypi?%3Aaction=pkg_edit&name=numpy>, the tarball and zip files may be downloaded from Github <https://github.com/numpy/numpy/releases/tag/v1.12.0>. The release notes and files hashes may also be found at Github <https://github.com/numpy/numpy/releases/tag/v1.12.0> .
NumPy 1.12.0rc 2 is the result of 418 pull requests submitted by 139 contributors and comprises a large number of fixes and improvements. Among the many improvements it is difficult to pick out just a few as standing above the others, but the following may be of particular interest or indicate areas likely to have future consequences.
* Order of operations in ``np.einsum`` can now be optimized for large speed improvements. * New ``signature`` argument to ``np.vectorize`` for vectorizing with core dimensions. * The ``keepdims`` argument was added to many functions. * New context manager for testing warnings * Support for BLIS in numpy.distutils * Much improved support for PyPy (not yet finished)
Enjoy,
Chuck
I've installed via pip3 on linux x86_64, which gives me a wheel. My question is, am I loosing significant performance choosing this pre-built binary vs. compiling myself? For example, my processor might have some more features than the base version used to build wheels.
I guess you are thinking about using this built wheel on some other machine? You'd have to be lucky for that to work; the wheel depends on the symbols it found at build time, which may not exist in the same places on your other machine.
If it does work, the speed will primarily depend on your BLAS library.
The pypi wheels should be pretty fast; they are built with OpenBLAS, which is at or near top of range for speed, across a range of platforms.
Cheers,
Matthew
I installed using pip3 install, and it installed a wheel package. I did not build it - aren't wheels already compiled packages? So isn't it built for the common denominator architecture, not necessarily as fast as one I built myself on my own machine? My question is, on x86_64, is this potential difference large enough to bother with not using precompiled wheel packages?
Ah - my guess is that you'd be hard pressed to make a numpy that is as fast as the precompiled wheel. The OpenBLAS library included in numpy selects the routines for your CPU at run-time, so they will generally be fast on your CPU. You might be able to get equivalent or even better performance with a ATLAS BLAS library recompiled on your exact machine, but that's quite a serious investment of time to get working, and you'd have to benchmark to find if you were really doing any better. Cheers, Matthew
On Tue, Jan 17, 2017 at 3:47 PM, Neal Becker <ndbecker2@gmail.com> wrote:
Matthew Brett wrote:
Hi,
On Tue, Jan 17, 2017 at 5:56 AM, Neal Becker <ndbecker2@gmail.com> wrote:
Charles R Harris wrote:
Hi All,
I'm pleased to announce the NumPy 1.12.0 release. This release supports Python 2.7 and 3.4-3.6. Wheels for all supported Python versions may be downloaded from PiPY <https://pypi.python.org/pypi?%3Aaction=pkg_edit&name=numpy>, the tarball and zip files may be downloaded from Github <https://github.com/numpy/numpy/releases/tag/v1.12.0>. The release notes and files hashes may also be found at Github <https://github.com/numpy/numpy/releases/tag/v1.12.0> .
NumPy 1.12.0rc 2 is the result of 418 pull requests submitted by 139 contributors and comprises a large number of fixes and improvements. Among the many improvements it is difficult to pick out just a few as standing above the others, but the following may be of particular interest or indicate areas likely to have future consequences.
* Order of operations in ``np.einsum`` can now be optimized for large speed improvements. * New ``signature`` argument to ``np.vectorize`` for vectorizing with core dimensions. * The ``keepdims`` argument was added to many functions. * New context manager for testing warnings * Support for BLIS in numpy.distutils * Much improved support for PyPy (not yet finished)
Enjoy,
Chuck
I've installed via pip3 on linux x86_64, which gives me a wheel. My question is, am I loosing significant performance choosing this pre-built binary vs. compiling myself? For example, my processor might have some more features than the base version used to build wheels.
I guess you are thinking about using this built wheel on some other machine? You'd have to be lucky for that to work; the wheel depends on the symbols it found at build time, which may not exist in the same places on your other machine.
If it does work, the speed will primarily depend on your BLAS library.
The pypi wheels should be pretty fast; they are built with OpenBLAS, which is at or near top of range for speed, across a range of platforms.
Cheers,
Matthew
I installed using pip3 install, and it installed a wheel package. I did not build it - aren't wheels already compiled packages? So isn't it built for the common denominator architecture, not necessarily as fast as one I built myself on my own machine? My question is, on x86_64, is this potential difference large enough to bother with not using precompiled wheel packages?
Ultimately, it's going to depend on all sorts of things, including most importantly your actual code. Like most speed questions, the only real way to know is to try it and measure the difference. The wheels do ship with a fast BLAS (OpenBLAS configured to automatically adapt to your CPU at runtime), so the performance will at least be reasonable. Possible improvements would include using a different and somehow better BLAS (MKL might be faster in some cases), tweaking your compiler options to take advantage of whatever SIMD ISAs your particular CPU supports (numpy's build system doesn't do this automatically but in principle you could do it by hand -- were you bothering before? does it even make a difference in practice? I dunno), and using a new compiler (the linux wheels use a somewhat ancient version of gcc for Reasons; newer compilers are better at optimizing -- how much does it matter? again I dunno). Basically: if you want to experiment and report back then I think we'd all be interested to hear; OTOH if you aren't feeling particularly curious/ambitious then I wouldn't worry about it :-). -n -- Nathaniel J. Smith -- https://vorpus.org
Matthew Brett wrote:
On Tue, Jan 17, 2017 at 3:47 PM, Neal Becker <ndbecker2@gmail.com> wrote:
Matthew Brett wrote:
Hi,
On Tue, Jan 17, 2017 at 5:56 AM, Neal Becker <ndbecker2@gmail.com> wrote:
Charles R Harris wrote:
Hi All,
I'm pleased to announce the NumPy 1.12.0 release. This release supports Python 2.7 and 3.4-3.6. Wheels for all supported Python versions may be downloaded from PiPY <https://pypi.python.org/pypi?%3Aaction=pkg_edit&name=numpy>, the tarball and zip files may be downloaded from Github <https://github.com/numpy/numpy/releases/tag/v1.12.0>. The release notes and files hashes may also be found at Github <https://github.com/numpy/numpy/releases/tag/v1.12.0> .
NumPy 1.12.0rc 2 is the result of 418 pull requests submitted by 139 contributors and comprises a large number of fixes and improvements. Among the many improvements it is difficult to pick out just a few as standing above the others, but the following may be of particular interest or indicate areas likely to have future consequences.
* Order of operations in ``np.einsum`` can now be optimized for large speed improvements. * New ``signature`` argument to ``np.vectorize`` for vectorizing with core dimensions. * The ``keepdims`` argument was added to many functions. * New context manager for testing warnings * Support for BLIS in numpy.distutils * Much improved support for PyPy (not yet finished)
Enjoy,
Chuck
I've installed via pip3 on linux x86_64, which gives me a wheel. My question is, am I loosing significant performance choosing this pre-built binary vs. compiling myself? For example, my processor might have some more features than the base version used to build wheels.
I guess you are thinking about using this built wheel on some other machine? You'd have to be lucky for that to work; the wheel depends on the symbols it found at build time, which may not exist in the same places on your other machine.
If it does work, the speed will primarily depend on your BLAS library.
The pypi wheels should be pretty fast; they are built with OpenBLAS, which is at or near top of range for speed, across a range of platforms.
Cheers,
Matthew
I installed using pip3 install, and it installed a wheel package. I did not build it - aren't wheels already compiled packages? So isn't it built for the common denominator architecture, not necessarily as fast as one I built myself on my own machine? My question is, on x86_64, is this potential difference large enough to bother with not using precompiled wheel packages?
Ah - my guess is that you'd be hard pressed to make a numpy that is as fast as the precompiled wheel. The OpenBLAS library included in numpy selects the routines for your CPU at run-time, so they will generally be fast on your CPU. You might be able to get equivalent or even better performance with a ATLAS BLAS library recompiled on your exact machine, but that's quite a serious investment of time to get working, and you'd have to benchmark to find if you were really doing any better.
Cheers,
Matthew
OK, so at least for BLAS things should be pretty well optimized.
participants (3)
-
Matthew Brett -
Nathaniel Smith -
Neal Becker