Install Scipy with Anaconda's MKL libraries
Anaconda now comes with free MKL libraries. I am trying to install the development branch of Scipy into an Anaconda environment, which means running the full Scipy install, not just the wheel. Scipy needs BLAS/LAPACK in order to compile, so the built in MKL seems to make sense. However, it seems that Scipy can't find the MKL libraries because it gives the "no lapack/blas resources found" error during setup.py when running a pip install. Is there an environment variable I need to set in order to tell Scipy's setup where my LAPACK library lives? Here is exactly what I am running: conda create --name scipy_master python=3.6 numpy cython activate scipy_master pip install https://github.com/scipy/scipy/tarball/master I asked this on Stack Overflow <https://stackoverflow.com/questions/43559959/compiling-scipy-with-anacondas-...> last week but didn't get an answer.
Welcome to the world of pain with building scientific packages from source on Windows! You need Fortran and C/C++ compilers on Windows to build scipy from source (and also enough disk space): https://www.scipy.org/scipylib/building/windows.html The mingwpy project mentioned in the wiki does not support Python 3.5+ due to UCRT. mkl binaries linked with numpy and scipy and provided in Anaconda are not enough for building scipy. If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy. But I had no luck with this, looks like this still requires dealing with include paths for blas and lapack. On Sat, Apr 29, 2017 at 8:56 PM, David Hagen <david@drhagen.com> wrote:
Anaconda now comes with free MKL libraries. I am trying to install the development branch of Scipy into an Anaconda environment, which means running the full Scipy install, not just the wheel. Scipy needs BLAS/LAPACK in order to compile, so the built in MKL seems to make sense. However, it seems that Scipy can't find the MKL libraries because it gives the "no lapack/blas resources found" error during setup.py when running a pip install. Is there an environment variable I need to set in order to tell Scipy's setup where my LAPACK library lives?
Here is exactly what I am running:
conda create --name scipy_master python=3.6 numpy cython activate scipy_master pip install https://github.com/scipy/scipy/tarball/master
I asked this on Stack Overflow <https://stackoverflow.com/questions/43559959/compiling-scipy-with-anacondas-...> last week but didn't get an answer.
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
I’m pretty sure that anaconda does not come with the development files for MKL, only the runtime files. I just looked under the mac couldn’t find any of the header files. That would make sense if Intel has not changed the licensing for MKL in the last few years. This means that you cannot link new code to the MKL shipped with Anaconda but would need to get your own development license for MKL.
On Apr 29, 2017, at 21:56, David Hagen <david@drhagen.com> wrote:
Anaconda now comes with free MKL libraries. I am trying to install the development branch of Scipy into an Anaconda environment, which means running the full Scipy install, not just the wheel. Scipy needs BLAS/LAPACK in order to compile, so the built in MKL seems to make sense. However, it seems that Scipy can't find the MKL libraries because it gives the "no lapack/blas resources found" error during setup.py when running a pip install. Is there an environment variable I need to set in order to tell Scipy's setup where my LAPACK library lives?
Here is exactly what I am running:
conda create --name scipy_master python=3.6 numpy cython activate scipy_master pip install https://github.com/scipy/scipy/tarball/master <https://github.com/scipy/scipy/tarball/master>
I asked this on Stack Overflow <https://stackoverflow.com/questions/43559959/compiling-scipy-with-anacondas-...> last week but didn't get an answer. _______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
Welcome to the world of pain with building scientific packages from source on Windows!
I am beginning to feel it.
You need Fortran and C/C++ compilers on Windows to build scipy from source
I have MinGW-w64 installed, which seems to be the recommended method.
I’m pretty sure that anaconda does not come with the development files for MKL, only the runtime files.
I understand now. It looks like MKL is not the way to go unless I want to pay Intel.
If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy.
My only goal is to install and use Scipy master somewhere where it won't break my stable installation. I thought Anaconda would be a good place to start because once I activate an Anaconda environment, I should be able to treat like a normal Python installation and follow the normal install-from-source instructions. I went ahead and installed that m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I should change my question to: how do I install Scipy on Windows from source? Though when I search for this specifically on the web, the answer seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by Scipy. Do you know of a conda/pip package that provides ATLAS for building Scipy or another more suitable BLAS/LAPACK?
Why do you want to pay Intel? You can install the MKL and develop with it, no sweat. 2017-04-30 22:41 GMT+01:00 David Hagen <david@drhagen.com>:
Welcome to the world of pain with building scientific packages from source on Windows!
I am beginning to feel it.
You need Fortran and C/C++ compilers on Windows to build scipy from source
I have MinGW-w64 installed, which seems to be the recommended method.
I’m pretty sure that anaconda does not come with the development files for MKL, only the runtime files.
I understand now. It looks like MKL is not the way to go unless I want to pay Intel.
If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy.
My only goal is to install and use Scipy master somewhere where it won't break my stable installation. I thought Anaconda would be a good place to start because once I activate an Anaconda environment, I should be able to treat like a normal Python installation and follow the normal install-from-source instructions. I went ahead and installed that m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I should change my question to: how do I install Scipy on Windows from source? Though when I search for this specifically on the web, the answer seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by Scipy. Do you know of a conda/pip package that provides ATLAS for building Scipy or another more suitable BLAS/LAPACK?
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
-- Information System Engineer, Ph.D. Blog: http://blog.audio-tk.com/ LinkedIn: http://www.linkedin.com/in/matthieubrucher
I still suggest Intel+MSVC compilers, since you can use trial version or request license for open-source projects from Intel. This is what Anaconda team is using. Also this is what Christoph Gohlke wheels are based on: http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy If you end up with m2w64, here is lapack for conda, you may still have to modify paths: https://anaconda.org/conda-forge/lapack And blas: https://anaconda.org/search?q=Blas On Sun, Apr 30, 2017, 5:22 PM Matthieu Brucher <matthieu.brucher@gmail.com> wrote:
Why do you want to pay Intel? You can install the MKL and develop with it, no sweat.
2017-04-30 22:41 GMT+01:00 David Hagen <david@drhagen.com>:
Welcome to the world of pain with building scientific packages from source on Windows!
I am beginning to feel it.
You need Fortran and C/C++ compilers on Windows to build scipy from source
I have MinGW-w64 installed, which seems to be the recommended method.
I’m pretty sure that anaconda does not come with the development files for MKL, only the runtime files.
I understand now. It looks like MKL is not the way to go unless I want to pay Intel.
If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy.
My only goal is to install and use Scipy master somewhere where it won't break my stable installation. I thought Anaconda would be a good place to start because once I activate an Anaconda environment, I should be able to treat like a normal Python installation and follow the normal install-from-source instructions. I went ahead and installed that m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I should change my question to: how do I install Scipy on Windows from source? Though when I search for this specifically on the web, the answer seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by Scipy. Do you know of a conda/pip package that provides ATLAS for building Scipy or another more suitable BLAS/LAPACK?
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
-- Information System Engineer, Ph.D. Blog: http://blog.audio-tk.com/ LinkedIn: http://www.linkedin.com/in/matthieubrucher _______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
I'll try to stick with MinGW-w64 for now, but I don't even get to the compilation phase. If I install lapack and blas from conda-forge, it still says that lapack/blas are not found, but you indicated that I need to set some paths. Are there instructions for this? I have no idea what environment variables to set in order to tell Scipy to use these packages. On Mon, May 1, 2017 at 3:44 AM, Denis Akhiyarov <denis.akhiyarov@gmail.com> wrote:
I still suggest Intel+MSVC compilers, since you can use trial version or request license for open-source projects from Intel. This is what Anaconda team is using. Also this is what Christoph Gohlke wheels are based on:
http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy
If you end up with m2w64, here is lapack for conda, you may still have to modify paths:
https://anaconda.org/conda-forge/lapack
And blas:
https://anaconda.org/search?q=Blas
On Sun, Apr 30, 2017, 5:22 PM Matthieu Brucher <matthieu.brucher@gmail.com> wrote:
Why do you want to pay Intel? You can install the MKL and develop with it, no sweat.
2017-04-30 22:41 GMT+01:00 David Hagen <david@drhagen.com>:
Welcome to the world of pain with building scientific packages from source on Windows!
I am beginning to feel it.
You need Fortran and C/C++ compilers on Windows to build scipy from source
I have MinGW-w64 installed, which seems to be the recommended method.
I’m pretty sure that anaconda does not come with the development files for MKL, only the runtime files.
I understand now. It looks like MKL is not the way to go unless I want to pay Intel.
If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy.
My only goal is to install and use Scipy master somewhere where it won't break my stable installation. I thought Anaconda would be a good place to start because once I activate an Anaconda environment, I should be able to treat like a normal Python installation and follow the normal install-from-source instructions. I went ahead and installed that m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I should change my question to: how do I install Scipy on Windows from source? Though when I search for this specifically on the web, the answer seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by Scipy. Do you know of a conda/pip package that provides ATLAS for building Scipy or another more suitable BLAS/LAPACK?
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
-- Information System Engineer, Ph.D. Blog: http://blog.audio-tk.com/ LinkedIn: http://www.linkedin.com/in/matthieubrucher _______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
Intel has made MKL, TBB, and a few other things completely free to use, even in a commercial project. Visual Studio is also free unless you are a very large company. https://software.intel.com/en-us/articles/free-mkl I have been using that for other projects without any problems and compiling with Visual Studio has been very easy. On Tue, May 2, 2017 at 1:40 AM, David Hagen <david@drhagen.com> wrote:
I'll try to stick with MinGW-w64 for now, but I don't even get to the compilation phase. If I install lapack and blas from conda-forge, it still says that lapack/blas are not found, but you indicated that I need to set some paths. Are there instructions for this? I have no idea what environment variables to set in order to tell Scipy to use these packages.
On Mon, May 1, 2017 at 3:44 AM, Denis Akhiyarov <denis.akhiyarov@gmail.com
wrote:
I still suggest Intel+MSVC compilers, since you can use trial version or request license for open-source projects from Intel. This is what Anaconda team is using. Also this is what Christoph Gohlke wheels are based on:
http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy
If you end up with m2w64, here is lapack for conda, you may still have to modify paths:
https://anaconda.org/conda-forge/lapack
And blas:
https://anaconda.org/search?q=Blas
On Sun, Apr 30, 2017, 5:22 PM Matthieu Brucher < matthieu.brucher@gmail.com> wrote:
Why do you want to pay Intel? You can install the MKL and develop with it, no sweat.
2017-04-30 22:41 GMT+01:00 David Hagen <david@drhagen.com>:
Welcome to the world of pain with building scientific packages from source on Windows!
I am beginning to feel it.
You need Fortran and C/C++ compilers on Windows to build scipy from source
I have MinGW-w64 installed, which seems to be the recommended method.
I’m pretty sure that anaconda does not come with the development files for MKL, only the runtime files.
I understand now. It looks like MKL is not the way to go unless I want to pay Intel.
If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy.
My only goal is to install and use Scipy master somewhere where it won't break my stable installation. I thought Anaconda would be a good place to start because once I activate an Anaconda environment, I should be able to treat like a normal Python installation and follow the normal install-from-source instructions. I went ahead and installed that m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I should change my question to: how do I install Scipy on Windows from source? Though when I search for this specifically on the web, the answer seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by Scipy. Do you know of a conda/pip package that provides ATLAS for building Scipy or another more suitable BLAS/LAPACK?
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
-- Information System Engineer, Ph.D. Blog: http://blog.audio-tk.com/ LinkedIn: http://www.linkedin.com/in/matthieubrucher _______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
Is there a recipe for this combination? I installed MKL from that link and the latest Visual Studio. Scipy did not find MKL on its own. I'm sure there's some environment variables that need to be set, but I don't know what they are. On Tue, May 2, 2017 at 4:00 AM, William Heymann <immudzen@gmail.com> wrote:
Intel has made MKL, TBB, and a few other things completely free to use, even in a commercial project. Visual Studio is also free unless you are a very large company.
https://software.intel.com/en-us/articles/free-mkl
I have been using that for other projects without any problems and compiling with Visual Studio has been very easy.
On Tue, May 2, 2017 at 1:40 AM, David Hagen <david@drhagen.com> wrote:
I'll try to stick with MinGW-w64 for now, but I don't even get to the compilation phase. If I install lapack and blas from conda-forge, it still says that lapack/blas are not found, but you indicated that I need to set some paths. Are there instructions for this? I have no idea what environment variables to set in order to tell Scipy to use these packages.
On Mon, May 1, 2017 at 3:44 AM, Denis Akhiyarov < denis.akhiyarov@gmail.com> wrote:
I still suggest Intel+MSVC compilers, since you can use trial version or request license for open-source projects from Intel. This is what Anaconda team is using. Also this is what Christoph Gohlke wheels are based on:
http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy
If you end up with m2w64, here is lapack for conda, you may still have to modify paths:
https://anaconda.org/conda-forge/lapack
And blas:
https://anaconda.org/search?q=Blas
On Sun, Apr 30, 2017, 5:22 PM Matthieu Brucher < matthieu.brucher@gmail.com> wrote:
Why do you want to pay Intel? You can install the MKL and develop with it, no sweat.
2017-04-30 22:41 GMT+01:00 David Hagen <david@drhagen.com>:
Welcome to the world of pain with building scientific packages from source on Windows!
I am beginning to feel it.
You need Fortran and C/C++ compilers on Windows to build scipy from source
I have MinGW-w64 installed, which seems to be the recommended method.
I’m pretty sure that anaconda does not come with the development files for MKL, only the runtime files.
I understand now. It looks like MKL is not the way to go unless I want to pay Intel.
If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy.
My only goal is to install and use Scipy master somewhere where it won't break my stable installation. I thought Anaconda would be a good place to start because once I activate an Anaconda environment, I should be able to treat like a normal Python installation and follow the normal install-from-source instructions. I went ahead and installed that m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I should change my question to: how do I install Scipy on Windows from source? Though when I search for this specifically on the web, the answer seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by Scipy. Do you know of a conda/pip package that provides ATLAS for building Scipy or another more suitable BLAS/LAPACK?
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
-- Information System Engineer, Ph.D. Blog: http://blog.audio-tk.com/ LinkedIn: http://www.linkedin.com/in/matthieubrucher _______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
Did you follow these instructions? https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl On Tue, May 2, 2017, 10:00 PM David Hagen <david@drhagen.com> wrote:
Is there a recipe for this combination? I installed MKL from that link and the latest Visual Studio. Scipy did not find MKL on its own. I'm sure there's some environment variables that need to be set, but I don't know what they are.
On Tue, May 2, 2017 at 4:00 AM, William Heymann <immudzen@gmail.com> wrote:
Intel has made MKL, TBB, and a few other things completely free to use, even in a commercial project. Visual Studio is also free unless you are a very large company.
https://software.intel.com/en-us/articles/free-mkl
I have been using that for other projects without any problems and compiling with Visual Studio has been very easy.
On Tue, May 2, 2017 at 1:40 AM, David Hagen <david@drhagen.com> wrote:
I'll try to stick with MinGW-w64 for now, but I don't even get to the compilation phase. If I install lapack and blas from conda-forge, it still says that lapack/blas are not found, but you indicated that I need to set some paths. Are there instructions for this? I have no idea what environment variables to set in order to tell Scipy to use these packages.
On Mon, May 1, 2017 at 3:44 AM, Denis Akhiyarov < denis.akhiyarov@gmail.com> wrote:
I still suggest Intel+MSVC compilers, since you can use trial version or request license for open-source projects from Intel. This is what Anaconda team is using. Also this is what Christoph Gohlke wheels are based on:
http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy
If you end up with m2w64, here is lapack for conda, you may still have to modify paths:
https://anaconda.org/conda-forge/lapack
And blas:
https://anaconda.org/search?q=Blas
On Sun, Apr 30, 2017, 5:22 PM Matthieu Brucher < matthieu.brucher@gmail.com> wrote:
Why do you want to pay Intel? You can install the MKL and develop with it, no sweat.
2017-04-30 22:41 GMT+01:00 David Hagen <david@drhagen.com>:
> Welcome to the world of pain with building scientific packages from source on Windows!
I am beginning to feel it.
> You need Fortran and C/C++ compilers on Windows to build scipy from source
I have MinGW-w64 installed, which seems to be the recommended method.
> I’m pretty sure that anaconda does not come with the development files for MKL, only the runtime files.
I understand now. It looks like MKL is not the way to go unless I want to pay Intel.
> If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy.
My only goal is to install and use Scipy master somewhere where it won't break my stable installation. I thought Anaconda would be a good place to start because once I activate an Anaconda environment, I should be able to treat like a normal Python installation and follow the normal install-from-source instructions. I went ahead and installed that m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I should change my question to: how do I install Scipy on Windows from source? Though when I search for this specifically on the web, the answer seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by Scipy. Do you know of a conda/pip package that provides ATLAS for building Scipy or another more suitable BLAS/LAPACK?
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
-- Information System Engineer, Ph.D. Blog: http://blog.audio-tk.com/ LinkedIn: http://www.linkedin.com/in/matthieubrucher _______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
So those instructions get me farther than I have gotten before. Is this actually necessary?: Note: You will need to rebuild Python from source files. This is due to the downloaded Python binary on Windows might be incompatible with the Visual Studio version you used. Otherwise you will encounter runtime crash when run numpy or script tests. This is what I have done so far: 1) Download Scipy zip file from Github mater branch 2) Unzip file 3) Add a site.cfg file with these contents: [mkl] library_dirs = C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\mkl\lib\intel64_win;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\compiler\lib\intel64_win include_dirs = C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\mkl\include mkl_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md lapack_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md 4) cd into that directory 5) run: python setup.py config --compiler=intelemw --fcompiler=intelvem build_clib --compiler=intelemw --fcompiler=intelvem build_ext --compiler=intelemw --fcompiler=intelvem install It gets really far into the build but this is the error that results: building extension "scipy.spatial.qhull" sources creating build\src.win-amd64-3.6\scipy\spatial Could not locate executable icc Could not locate executable ecc Traceback (most recent call last): File "setup.py", line 417, in <module> setup_package() File "setup.py", line 413, in setup_package setup(**metadata) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\core.py ", line 166, in setup return old_setup(**new_attr) File "C:\Anaconda3\envs\scipy_master\lib\distutils\core.py", line 148, in setu p dist.run_commands() File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 955, in run_ commands self.run_command(cmd) File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974, in run_ command cmd_obj.run() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \build_clib.py", line 74, in run self.run_command('build_src') File "C:\Anaconda3\envs\scipy_master\lib\distutils\cmd.py", line 313, in run_c ommand self.distribution.run_command(command) File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974, in run_ command cmd_obj.run() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \build_src.py", line 148, in run self.build_sources() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \build_src.py", line 165, in build_sources self.build_extension_sources(ext) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \build_src.py", line 324, in build_extension_sources sources = self.generate_sources(sources, ext) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \build_src.py", line 377, in generate_sources source = func(extension, build_dir) File "scipy\spatial\setup.py", line 35, in get_qhull_misc_config if config_cmd.check_func('open_memstream', decl=True, call=True): File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \config.py", line 312, in check_func self._check_compiler() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \config.py", line 51, in _check_compiler self.compiler.initialize() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\intelcc ompiler.py", line 86, in initialize MSVCCompiler.initialize(self, plat_name) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\msvc9co mpiler.py", line 53, in initialize os.environ['lib'] = _merge(environ_lib, os.environ['lib']) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\msvc9co mpiler.py", line 32, in _merge if new in old: TypeError: argument of type 'NoneType' is not iterable I notice that this is calling into Numpy. Do I have modify Numpy in order to compile Scipy? I skipped the Numpy specific steps in the instructions because I don't need the development version of Numpy. On Tue, May 2, 2017 at 11:15 PM, Denis Akhiyarov <denis.akhiyarov@gmail.com> wrote:
Did you follow these instructions?
https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl
On Tue, May 2, 2017, 10:00 PM David Hagen <david@drhagen.com> wrote:
Is there a recipe for this combination? I installed MKL from that link and the latest Visual Studio. Scipy did not find MKL on its own. I'm sure there's some environment variables that need to be set, but I don't know what they are.
On Tue, May 2, 2017 at 4:00 AM, William Heymann <immudzen@gmail.com> wrote:
Intel has made MKL, TBB, and a few other things completely free to use, even in a commercial project. Visual Studio is also free unless you are a very large company.
https://software.intel.com/en-us/articles/free-mkl
I have been using that for other projects without any problems and compiling with Visual Studio has been very easy.
On Tue, May 2, 2017 at 1:40 AM, David Hagen <david@drhagen.com> wrote:
I'll try to stick with MinGW-w64 for now, but I don't even get to the compilation phase. If I install lapack and blas from conda-forge, it still says that lapack/blas are not found, but you indicated that I need to set some paths. Are there instructions for this? I have no idea what environment variables to set in order to tell Scipy to use these packages.
On Mon, May 1, 2017 at 3:44 AM, Denis Akhiyarov < denis.akhiyarov@gmail.com> wrote:
I still suggest Intel+MSVC compilers, since you can use trial version or request license for open-source projects from Intel. This is what Anaconda team is using. Also this is what Christoph Gohlke wheels are based on:
http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy
If you end up with m2w64, here is lapack for conda, you may still have to modify paths:
https://anaconda.org/conda-forge/lapack
And blas:
https://anaconda.org/search?q=Blas
On Sun, Apr 30, 2017, 5:22 PM Matthieu Brucher < matthieu.brucher@gmail.com> wrote:
Why do you want to pay Intel? You can install the MKL and develop with it, no sweat.
2017-04-30 22:41 GMT+01:00 David Hagen <david@drhagen.com>:
> > Welcome to the world of pain with building scientific packages > from source on Windows! > > I am beginning to feel it. > > > You need Fortran and C/C++ compilers on Windows to build scipy > from source > > I have MinGW-w64 installed, which seems to be the recommended method. > > > I’m pretty sure that anaconda does not come with the development > files for MKL, only the runtime files. > > I understand now. It looks like MKL is not the way to go unless I > want to pay Intel. > > > If you don't need mkl and lapack/blas is good enough, then > m2w64-toolchain from conda should have all necessary dependencies for > building scipy. > > My only goal is to install and use Scipy master somewhere where it > won't break my stable installation. I thought Anaconda would be a good > place to start because once I activate an Anaconda environment, I should be > able to treat like a normal Python installation and follow the normal > install-from-source instructions. I went ahead and installed that > m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I > should change my question to: how do I install Scipy on Windows from > source? Though when I search for this specifically on the web, the answer > seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by > Scipy. Do you know of a conda/pip package that provides ATLAS for building > Scipy or another more suitable BLAS/LAPACK? > > _______________________________________________ > SciPy-User mailing list > SciPy-User@python.org > https://mail.python.org/mailman/listinfo/scipy-user > >
-- Information System Engineer, Ph.D. Blog: http://blog.audio-tk.com/ LinkedIn: http://www.linkedin.com/in/matthieubrucher _______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
Ok, I followed instructions on this page: https://software.intel.com/en-us/articles/building-numpyscipy-with-intel-mkl... Installed parallel_studio_xe_2016_update4 with mkl, ifort, and icc. 1. I compiled numpy (python setup.py install) from source with this site.cfg file: [mkl] include_dirs = C:\Program Files (x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\mkl\include library_dirs = C:\Program Files (x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\mkl\lib\intel64;C:\Program Files (x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\compiler\lib\intel64 mkl_libs = mkl_core_dll, mkl_intel_lp64_dll, mkl_intel_thread_dll lapack_libs = mkl_lapack95_lp64 2. I replaced site.cfg file in ..\Lib\site-packages\numpy\distutils with the one above, since this is used by scipy in step 3. 3. I compiled scipy (python setup.py install) from source. Compilation went without errors, both packages import without errors in python. On Wed, May 3, 2017 at 8:26 PM, David Hagen <david@drhagen.com> wrote:
So those instructions get me farther than I have gotten before.
Is this actually necessary?:
Note: You will need to rebuild Python from source files. This is due to the downloaded Python binary on Windows might be incompatible with the Visual Studio version you used. Otherwise you will encounter runtime crash when run numpy or script tests.
This is what I have done so far:
1) Download Scipy zip file from Github mater branch 2) Unzip file 3) Add a site.cfg file with these contents: [mkl] library_dirs = C:\Program Files (x86)\IntelSWTools\compilers_ and_libraries_2017.2.187\windows\mkl\lib\intel64_win;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\ windows\compiler\lib\intel64_win include_dirs = C:\Program Files (x86)\IntelSWTools\compilers_ and_libraries_2017.2.187\windows\mkl\include mkl_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_ thread,mkl_core,libiomp5md lapack_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_ thread,mkl_core,libiomp5md
4) cd into that directory 5) run: python setup.py config --compiler=intelemw --fcompiler=intelvem build_clib --compiler=intelemw --fcompiler=intelvem build_ext --compiler=intelemw --fcompiler=intelvem install
It gets really far into the build but this is the error that results:
building extension "scipy.spatial.qhull" sources creating build\src.win-amd64-3.6\scipy\spatial Could not locate executable icc Could not locate executable ecc Traceback (most recent call last): File "setup.py", line 417, in <module> setup_package() File "setup.py", line 413, in setup_package setup(**metadata) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\ numpy\distutils\core.py ", line 166, in setup return old_setup(**new_attr) File "C:\Anaconda3\envs\scipy_master\lib\distutils\core.py", line 148, in setu p dist.run_commands() File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 955, in run_ commands self.run_command(cmd) File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974, in run_ command cmd_obj.run() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\ numpy\distutils\command \build_clib.py", line 74, in run self.run_command('build_src') File "C:\Anaconda3\envs\scipy_master\lib\distutils\cmd.py", line 313, in run_c ommand self.distribution.run_command(command) File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974, in run_ command cmd_obj.run() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\ numpy\distutils\command \build_src.py", line 148, in run self.build_sources() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\ numpy\distutils\command \build_src.py", line 165, in build_sources self.build_extension_sources(ext) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\ numpy\distutils\command \build_src.py", line 324, in build_extension_sources sources = self.generate_sources(sources, ext) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\ numpy\distutils\command \build_src.py", line 377, in generate_sources source = func(extension, build_dir) File "scipy\spatial\setup.py", line 35, in get_qhull_misc_config if config_cmd.check_func('open_memstream', decl=True, call=True): File "C:\Anaconda3\envs\scipy_master\lib\site-packages\ numpy\distutils\command \config.py", line 312, in check_func self._check_compiler() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\ numpy\distutils\command \config.py", line 51, in _check_compiler self.compiler.initialize() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\ numpy\distutils\intelcc ompiler.py", line 86, in initialize MSVCCompiler.initialize(self, plat_name) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\ numpy\distutils\msvc9co mpiler.py", line 53, in initialize os.environ['lib'] = _merge(environ_lib, os.environ['lib']) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\ numpy\distutils\msvc9co mpiler.py", line 32, in _merge if new in old: TypeError: argument of type 'NoneType' is not iterable
I notice that this is calling into Numpy. Do I have modify Numpy in order to compile Scipy? I skipped the Numpy specific steps in the instructions because I don't need the development version of Numpy.
On Tue, May 2, 2017 at 11:15 PM, Denis Akhiyarov < denis.akhiyarov@gmail.com> wrote:
Did you follow these instructions?
https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl
On Tue, May 2, 2017, 10:00 PM David Hagen <david@drhagen.com> wrote:
Is there a recipe for this combination? I installed MKL from that link and the latest Visual Studio. Scipy did not find MKL on its own. I'm sure there's some environment variables that need to be set, but I don't know what they are.
On Tue, May 2, 2017 at 4:00 AM, William Heymann <immudzen@gmail.com> wrote:
Intel has made MKL, TBB, and a few other things completely free to use, even in a commercial project. Visual Studio is also free unless you are a very large company.
https://software.intel.com/en-us/articles/free-mkl
I have been using that for other projects without any problems and compiling with Visual Studio has been very easy.
On Tue, May 2, 2017 at 1:40 AM, David Hagen <david@drhagen.com> wrote:
I'll try to stick with MinGW-w64 for now, but I don't even get to the compilation phase. If I install lapack and blas from conda-forge, it still says that lapack/blas are not found, but you indicated that I need to set some paths. Are there instructions for this? I have no idea what environment variables to set in order to tell Scipy to use these packages.
On Mon, May 1, 2017 at 3:44 AM, Denis Akhiyarov < denis.akhiyarov@gmail.com> wrote:
I still suggest Intel+MSVC compilers, since you can use trial version or request license for open-source projects from Intel. This is what Anaconda team is using. Also this is what Christoph Gohlke wheels are based on:
http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy
If you end up with m2w64, here is lapack for conda, you may still have to modify paths:
https://anaconda.org/conda-forge/lapack
And blas:
https://anaconda.org/search?q=Blas
On Sun, Apr 30, 2017, 5:22 PM Matthieu Brucher < matthieu.brucher@gmail.com> wrote:
> Why do you want to pay Intel? You can install the MKL and develop > with it, no sweat. > > 2017-04-30 22:41 GMT+01:00 David Hagen <david@drhagen.com>: > >> > Welcome to the world of pain with building scientific packages >> from source on Windows! >> >> I am beginning to feel it. >> >> > You need Fortran and C/C++ compilers on Windows to build scipy >> from source >> >> I have MinGW-w64 installed, which seems to be the recommended >> method. >> >> > I’m pretty sure that anaconda does not come with the development >> files for MKL, only the runtime files. >> >> I understand now. It looks like MKL is not the way to go unless I >> want to pay Intel. >> >> > If you don't need mkl and lapack/blas is good enough, then >> m2w64-toolchain from conda should have all necessary dependencies for >> building scipy. >> >> My only goal is to install and use Scipy master somewhere where it >> won't break my stable installation. I thought Anaconda would be a good >> place to start because once I activate an Anaconda environment, I should be >> able to treat like a normal Python installation and follow the normal >> install-from-source instructions. I went ahead and installed that >> m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I >> should change my question to: how do I install Scipy on Windows from >> source? Though when I search for this specifically on the web, the answer >> seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by >> Scipy. Do you know of a conda/pip package that provides ATLAS for building >> Scipy or another more suitable BLAS/LAPACK? >> >> _______________________________________________ >> SciPy-User mailing list >> SciPy-User@python.org >> https://mail.python.org/mailman/listinfo/scipy-user >> >> > > > -- > Information System Engineer, Ph.D. > Blog: http://blog.audio-tk.com/ > LinkedIn: http://www.linkedin.com/in/matthieubrucher > _______________________________________________ > SciPy-User mailing list > SciPy-User@python.org > https://mail.python.org/mailman/listinfo/scipy-user >
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
Am I correct that there is no free version of ifort and icc? Will this break after the 30-trial ends? On Thu, May 4, 2017 at 2:58 AM, Denis Akhiyarov <denis.akhiyarov@gmail.com> wrote:
Ok, I followed instructions on this page:
https://software.intel.com/en-us/articles/building- numpyscipy-with-intel-mkl-and-intel-fortran-on-windows
Installed parallel_studio_xe_2016_update4 with mkl, ifort, and icc.
1. I compiled numpy (python setup.py install) from source with this site.cfg file:
[mkl] include_dirs = C:\Program Files (x86)\IntelSWTools\parallel_ studio_xe_2016.4.062\compilers_and_libraries_2016\windows\mkl\include library_dirs = C:\Program Files (x86)\IntelSWTools\parallel_ studio_xe_2016.4.062\compilers_and_libraries_2016\ windows\mkl\lib\intel64;C:\Program Files (x86)\IntelSWTools\parallel_ studio_xe_2016.4.062\compilers_and_libraries_2016\ windows\compiler\lib\intel64 mkl_libs = mkl_core_dll, mkl_intel_lp64_dll, mkl_intel_thread_dll lapack_libs = mkl_lapack95_lp64
2. I replaced site.cfg file in ..\Lib\site-packages\numpy\distutils with the one above, since this is used by scipy in step 3.
3. I compiled scipy (python setup.py install) from source.
Compilation went without errors, both packages import without errors in python.
On Wed, May 3, 2017 at 8:26 PM, David Hagen <david@drhagen.com> wrote:
So those instructions get me farther than I have gotten before.
Is this actually necessary?:
Note: You will need to rebuild Python from source files. This is due to the downloaded Python binary on Windows might be incompatible with the Visual Studio version you used. Otherwise you will encounter runtime crash when run numpy or script tests.
This is what I have done so far:
1) Download Scipy zip file from Github mater branch 2) Unzip file 3) Add a site.cfg file with these contents: [mkl] library_dirs = C:\Program Files (x86)\IntelSWTools\compilers_a nd_libraries_2017.2.187\windows\mkl\lib\intel64_win;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\window s\compiler\lib\intel64_win include_dirs = C:\Program Files (x86)\IntelSWTools\compilers_a nd_libraries_2017.2.187\windows\mkl\include mkl_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_t hread,mkl_core,libiomp5md lapack_libs = mkl_lapack95_lp64,mkl_blas95_l p64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md
4) cd into that directory 5) run: python setup.py config --compiler=intelemw --fcompiler=intelvem build_clib --compiler=intelemw --fcompiler=intelvem build_ext --compiler=intelemw --fcompiler=intelvem install
It gets really far into the build but this is the error that results:
building extension "scipy.spatial.qhull" sources creating build\src.win-amd64-3.6\scipy\spatial Could not locate executable icc Could not locate executable ecc Traceback (most recent call last): File "setup.py", line 417, in <module> setup_package() File "setup.py", line 413, in setup_package setup(**metadata) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\ distutils\core.py ", line 166, in setup return old_setup(**new_attr) File "C:\Anaconda3\envs\scipy_master\lib\distutils\core.py", line 148, in setu p dist.run_commands() File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 955, in run_ commands self.run_command(cmd) File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974, in run_ command cmd_obj.run() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\ distutils\command \build_clib.py", line 74, in run self.run_command('build_src') File "C:\Anaconda3\envs\scipy_master\lib\distutils\cmd.py", line 313, in run_c ommand self.distribution.run_command(command) File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974, in run_ command cmd_obj.run() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\ distutils\command \build_src.py", line 148, in run self.build_sources() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\ distutils\command \build_src.py", line 165, in build_sources self.build_extension_sources(ext) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\ distutils\command \build_src.py", line 324, in build_extension_sources sources = self.generate_sources(sources, ext) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\ distutils\command \build_src.py", line 377, in generate_sources source = func(extension, build_dir) File "scipy\spatial\setup.py", line 35, in get_qhull_misc_config if config_cmd.check_func('open_memstream', decl=True, call=True): File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\ distutils\command \config.py", line 312, in check_func self._check_compiler() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\ distutils\command \config.py", line 51, in _check_compiler self.compiler.initialize() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\ distutils\intelcc ompiler.py", line 86, in initialize MSVCCompiler.initialize(self, plat_name) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\ distutils\msvc9co mpiler.py", line 53, in initialize os.environ['lib'] = _merge(environ_lib, os.environ['lib']) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\ distutils\msvc9co mpiler.py", line 32, in _merge if new in old: TypeError: argument of type 'NoneType' is not iterable
I notice that this is calling into Numpy. Do I have modify Numpy in order to compile Scipy? I skipped the Numpy specific steps in the instructions because I don't need the development version of Numpy.
On Tue, May 2, 2017 at 11:15 PM, Denis Akhiyarov < denis.akhiyarov@gmail.com> wrote:
Did you follow these instructions?
https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl
On Tue, May 2, 2017, 10:00 PM David Hagen <david@drhagen.com> wrote:
Is there a recipe for this combination? I installed MKL from that link and the latest Visual Studio. Scipy did not find MKL on its own. I'm sure there's some environment variables that need to be set, but I don't know what they are.
On Tue, May 2, 2017 at 4:00 AM, William Heymann <immudzen@gmail.com> wrote:
Intel has made MKL, TBB, and a few other things completely free to use, even in a commercial project. Visual Studio is also free unless you are a very large company.
https://software.intel.com/en-us/articles/free-mkl
I have been using that for other projects without any problems and compiling with Visual Studio has been very easy.
On Tue, May 2, 2017 at 1:40 AM, David Hagen <david@drhagen.com> wrote:
I'll try to stick with MinGW-w64 for now, but I don't even get to the compilation phase. If I install lapack and blas from conda-forge, it still says that lapack/blas are not found, but you indicated that I need to set some paths. Are there instructions for this? I have no idea what environment variables to set in order to tell Scipy to use these packages.
On Mon, May 1, 2017 at 3:44 AM, Denis Akhiyarov < denis.akhiyarov@gmail.com> wrote:
> I still suggest Intel+MSVC compilers, since you can use trial > version or request license for open-source projects from Intel. This is > what Anaconda team is using. Also this is what Christoph Gohlke wheels are > based on: > > http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy > > If you end up with m2w64, here is lapack for conda, you may still > have to modify paths: > > https://anaconda.org/conda-forge/lapack > > And blas: > > https://anaconda.org/search?q=Blas > > On Sun, Apr 30, 2017, 5:22 PM Matthieu Brucher < > matthieu.brucher@gmail.com> wrote: > >> Why do you want to pay Intel? You can install the MKL and develop >> with it, no sweat. >> >> 2017-04-30 22:41 GMT+01:00 David Hagen <david@drhagen.com>: >> >>> > Welcome to the world of pain with building scientific packages >>> from source on Windows! >>> >>> I am beginning to feel it. >>> >>> > You need Fortran and C/C++ compilers on Windows to build scipy >>> from source >>> >>> I have MinGW-w64 installed, which seems to be the recommended >>> method. >>> >>> > I’m pretty sure that anaconda does not come with the development >>> files for MKL, only the runtime files. >>> >>> I understand now. It looks like MKL is not the way to go unless I >>> want to pay Intel. >>> >>> > If you don't need mkl and lapack/blas is good enough, then >>> m2w64-toolchain from conda should have all necessary dependencies for >>> building scipy. >>> >>> My only goal is to install and use Scipy master somewhere where it >>> won't break my stable installation. I thought Anaconda would be a good >>> place to start because once I activate an Anaconda environment, I should be >>> able to treat like a normal Python installation and follow the normal >>> install-from-source instructions. I went ahead and installed that >>> m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I >>> should change my question to: how do I install Scipy on Windows from >>> source? Though when I search for this specifically on the web, the answer >>> seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by >>> Scipy. Do you know of a conda/pip package that provides ATLAS for building >>> Scipy or another more suitable BLAS/LAPACK? >>> >>> _______________________________________________ >>> SciPy-User mailing list >>> SciPy-User@python.org >>> https://mail.python.org/mailman/listinfo/scipy-user >>> >>> >> >> >> -- >> Information System Engineer, Ph.D. >> Blog: http://blog.audio-tk.com/ >> LinkedIn: http://www.linkedin.com/in/matthieubrucher >> _______________________________________________ >> SciPy-User mailing list >> SciPy-User@python.org >> https://mail.python.org/mailman/listinfo/scipy-user >> > > _______________________________________________ > SciPy-User mailing list > SciPy-User@python.org > https://mail.python.org/mailman/listinfo/scipy-user > >
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
On 05/04/2017 08:16 AM, David Hagen wrote:
Am I correct that there is no free version of ifort and icc? Will this break after the 30-trial ends?
On Thu, May 4, 2017 at 2:58 AM, Denis Akhiyarov <denis.akhiyarov@gmail.com <mailto:denis.akhiyarov@gmail.com>> wrote:
Ok, I followed instructions on this page:
https://software.intel.com/en-us/articles/building-numpyscipy-with-intel-mkl... <https://software.intel.com/en-us/articles/building-numpyscipy-with-intel-mkl...>
Installed parallel_studio_xe_2016_update4 with mkl, ifort, and icc.
1. I compiled numpy (python setup.py install) from source with this site.cfg file:
[mkl] include_dirs = C:\Program Files (x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\mkl\include library_dirs = C:\Program Files (x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\mkl\lib\intel64;C:\Program Files (x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\compiler\lib\intel64 mkl_libs = mkl_core_dll, mkl_intel_lp64_dll, mkl_intel_thread_dll lapack_libs = mkl_lapack95_lp64
2. I replaced site.cfg file in ..\Lib\site-packages\numpy\distutils with the one above, since this is used by scipy in step 3.
3. I compiled scipy (python setup.py install) from source.
Compilation went without errors, both packages import without errors in python.
On Wed, May 3, 2017 at 8:26 PM, David Hagen <david@drhagen.com <mailto:david@drhagen.com>> wrote:
So those instructions get me farther than I have gotten before.
Is this actually necessary?:
Note: You will need to rebuild Python from source files. This is due to the downloaded Python binary on Windows might be incompatible with the Visual Studio version you used. Otherwise you will encounter runtime crash when run numpy or script tests.
This is what I have done so far:
1) Download Scipy zip file from Github mater branch 2) Unzip file 3) Add a site.cfg file with these contents: [mkl] library_dirs = C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\mkl\lib\intel64_win;C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\compiler\lib\intel64_win include_dirs = C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\mkl\include mkl_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md lapack_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md
4) cd into that directory 5) run: python setup.py config --compiler=intelemw --fcompiler=intelvem build_clib --compiler=intelemw --fcompiler=intelvem build_ext --compiler=intelemw --fcompiler=intelvem install
It gets really far into the build but this is the error that results:
building extension "scipy.spatial.qhull" sources creating build\src.win-amd64-3.6\scipy\spatial Could not locate executable icc Could not locate executable ecc Traceback (most recent call last): File "setup.py", line 417, in <module> setup_package() File "setup.py", line 413, in setup_package setup(**metadata) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\core.py ", line 166, in setup return old_setup(**new_attr) File "C:\Anaconda3\envs\scipy_master\lib\distutils\core.py", line 148, in setu p dist.run_commands() File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 955, in run_ commands self.run_command(cmd) File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974, in run_ command cmd_obj.run() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \build_clib.py", line 74, in run self.run_command('build_src') File "C:\Anaconda3\envs\scipy_master\lib\distutils\cmd.py", line 313, in run_c ommand self.distribution.run_command(command) File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974, in run_ command cmd_obj.run() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \build_src.py", line 148, in run self.build_sources() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \build_src.py", line 165, in build_sources self.build_extension_sources(ext) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \build_src.py", line 324, in build_extension_sources sources = self.generate_sources(sources, ext) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \build_src.py", line 377, in generate_sources source = func(extension, build_dir) File "scipy\spatial\setup.py", line 35, in get_qhull_misc_config if config_cmd.check_func('open_memstream', decl=True, call=True): File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \config.py", line 312, in check_func self._check_compiler() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command \config.py", line 51, in _check_compiler self.compiler.initialize() File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\intelcc ompiler.py", line 86, in initialize MSVCCompiler.initialize(self, plat_name) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\msvc9co mpiler.py", line 53, in initialize os.environ['lib'] = _merge(environ_lib, os.environ['lib']) File "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\msvc9co mpiler.py", line 32, in _merge if new in old: TypeError: argument of type 'NoneType' is not iterable
I notice that this is calling into Numpy. Do I have modify Numpy in order to compile Scipy? I skipped the Numpy specific steps in the instructions because I don't need the development version of Numpy.
On Tue, May 2, 2017 at 11:15 PM, Denis Akhiyarov <denis.akhiyarov@gmail.com <mailto:denis.akhiyarov@gmail.com>> wrote:
Did you follow these instructions?
https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl <https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl>
On Tue, May 2, 2017, 10:00 PM David Hagen <david@drhagen.com <mailto:david@drhagen.com>> wrote:
Is there a recipe for this combination? I installed MKL from that link and the latest Visual Studio. Scipy did not find MKL on its own. I'm sure there's some environment variables that need to be set, but I don't know what they are.
On Tue, May 2, 2017 at 4:00 AM, William Heymann <immudzen@gmail.com <mailto:immudzen@gmail.com>> wrote:
Intel has made MKL, TBB, and a few other things completely free to use, even in a commercial project. Visual Studio is also free unless you are a very large company.
https://software.intel.com/en-us/articles/free-mkl <https://software.intel.com/en-us/articles/free-mkl>
I have been using that for other projects without any problems and compiling with Visual Studio has been very easy.
On Tue, May 2, 2017 at 1:40 AM, David Hagen <david@drhagen.com <mailto:david@drhagen.com>> wrote:
I'll try to stick with MinGW-w64 for now, but I don't even get to the compilation phase. If I install lapack and blas from conda-forge, it still says that lapack/blas are not found, but you indicated that I need to set some paths. Are there instructions for this? I have no idea what environment variables to set in order to tell Scipy to use these packages.
On Mon, May 1, 2017 at 3:44 AM, Denis Akhiyarov <denis.akhiyarov@gmail.com <mailto:denis.akhiyarov@gmail.com>> wrote:
I still suggest Intel+MSVC compilers, since you can use trial version or request license for open-source projects from Intel. This is what Anaconda team is using. Also this is what Christoph Gohlke wheels are based on:
http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy <http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy>
If you end up with m2w64, here is lapack for conda, you may still have to modify paths:
https://anaconda.org/conda-forge/lapack <https://anaconda.org/conda-forge/lapack>
And blas:
https://anaconda.org/search?q=Blas <https://anaconda.org/search?q=Blas>
On Sun, Apr 30, 2017, 5:22 PM Matthieu Brucher <matthieu.brucher@gmail.com <mailto:matthieu.brucher@gmail.com>> wrote:
Why do you want to pay Intel? You can install the MKL and develop with it, no sweat.
2017-04-30 22:41 GMT+01:00 David Hagen <david@drhagen.com <mailto:david@drhagen.com>>:
> Welcome to the world of pain with building scientific packages from source on Windows!
I am beginning to feel it.
> You need Fortran and C/C++ compilers on Windows to build scipy from source
I have MinGW-w64 installed, which seems to be the recommended method.
> I’m pretty sure that anaconda does not come with the development files for MKL, only the runtime files.
I understand now. It looks like MKL is not the way to go unless I want to pay Intel.
> If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy.
My only goal is to install and use Scipy master somewhere where it won't break my stable installation. I thought Anaconda would be a good place to start because once I activate an Anaconda environment, I should be able to treat like a normal Python installation and follow the normal install-from-source instructions. I went ahead and installed that m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I should change my question to: how do I install Scipy on Windows from source? Though when I search for this specifically on the web, the answer seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by Scipy. Do you know of a conda/pip package that provides ATLAS for building Scipy or another more suitable BLAS/LAPACK?
_______________________________________________ SciPy-User mailing list SciPy-User@python.org <mailto:SciPy-User@python.org> https://mail.python.org/mailman/listinfo/scipy-user <https://mail.python.org/mailman/listinfo/scipy-user>
-- Information System Engineer, Ph.D. Blog: http://blog.audio-tk.com/ LinkedIn: http://www.linkedin.com/in/matthieubrucher <http://www.linkedin.com/in/matthieubrucher> _______________________________________________ SciPy-User mailing list SciPy-User@python.org <mailto:SciPy-User@python.org> https://mail.python.org/mailman/listinfo/scipy-user <https://mail.python.org/mailman/listinfo/scipy-user>
_______________________________________________ SciPy-User mailing list SciPy-User@python.org <mailto:SciPy-User@python.org> https://mail.python.org/mailman/listinfo/scipy-user <https://mail.python.org/mailman/listinfo/scipy-user>
_______________________________________________ SciPy-User mailing list SciPy-User@python.org <mailto:SciPy-User@python.org> https://mail.python.org/mailman/listinfo/scipy-user <https://mail.python.org/mailman/listinfo/scipy-user>
_______________________________________________ SciPy-User mailing list SciPy-User@python.org <mailto:SciPy-User@python.org> https://mail.python.org/mailman/listinfo/scipy-user <https://mail.python.org/mailman/listinfo/scipy-user>
_______________________________________________ SciPy-User mailing list SciPy-User@python.org <mailto:SciPy-User@python.org> https://mail.python.org/mailman/listinfo/scipy-user <https://mail.python.org/mailman/listinfo/scipy-user>
_______________________________________________ SciPy-User mailing list SciPy-User@python.org <mailto:SciPy-User@python.org> https://mail.python.org/mailman/listinfo/scipy-user <https://mail.python.org/mailman/listinfo/scipy-user>
_______________________________________________ SciPy-User mailing list SciPy-User@python.org <mailto:SciPy-User@python.org> https://mail.python.org/mailman/listinfo/scipy-user <https://mail.python.org/mailman/listinfo/scipy-user>
_______________________________________________ SciPy-User mailing list SciPy-User@python.org <mailto:SciPy-User@python.org> https://mail.python.org/mailman/listinfo/scipy-user <https://mail.python.org/mailman/listinfo/scipy-user>
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-us No, you are not correct about ifort. It's wstill available. ayou just won't receive support.
-- Stephen P. Molnar, Ph.D. Life is a fuzzy set www.molecular-modeling.net Stochastic and multivariate (614)312-7528 (c) Skype: smolnar1
No, you are not correct about ifort. It's wstill available. ayou just won't receive support.
I must be unable to Google correctly. I can find free MKL for everyone. I can find free ifort and icc for students. I can find free ifort and icc for Linux for open source developers. But I can't find free ifort and icc for Windows for open source developers. This is what I am looking at: https://software.intel.com/en-us/articles/free-ipsxe-tools-and-libraries
You can sign-up for beta program which includes Windows ifort compiler. https://software.intel.com/en-us/articles/intel-parallel-studio-xe-2018-beta "The beta program officially ends *June 22nd, 2017*. The beta license provided will expire October 12, 2017." On Thu, May 4, 2017 at 9:54 AM, David Hagen <david@drhagen.com> wrote:
No, you are not correct about ifort. It's wstill available. ayou just won't receive support.
I must be unable to Google correctly. I can find free MKL for everyone. I can find free ifort and icc for students. I can find free ifort and icc for Linux for open source developers. But I can't find free ifort and icc for Windows for open source developers. This is what I am looking at: https://software.intel.com/en-us/articles/free-ipsxe-tools-and-libraries
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
On 05/04/2017 10:54 AM, David Hagen wrote:
No, you are not correct about ifort. It's wstill available. ayou just won't receive support.
I must be unable to Google correctly. I can find free MKL for everyone. I can find free ifort and icc for students. I can find free ifort and icc for Linux for open source developers. But I can't find free ifort and icc for Windows for open source developers. This is what I am looking at: https://software.intel.com/en-us/articles/free-ipsxe-tools-and-libraries
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
Check out: https://software.intel.com/en-us/free-software -- Stephen P. Molnar, Ph.D. Life is a fuzzy set www.molecular-modeling.net Stochastic and multivariate (614)312-7528 (c) Skype: smolnar1
I am following Stephen's instructions with the Beta distribution of Intel Parallel Studio. The compilation of Numpy fails with this error: LINK : fatal error LNK1158: cannot run 'rc.exe' It also has this message during setup: customize IntelVisualFCompiler Could not locate executable ifort Could not locate executable ifl and this: Found executable C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe Do I need to configure Numpy to use the Parallel Studio compilers rather than whatever else I may have installed?
I noticed that I had two versions of Visual Studio installed: Microsoft Visual C++ 14.0 and Microsoft Visual Studio 2017. I thought that this might be confusing the installer, so I uninstalled the older Microsoft Visual C++ 14.0. However, the Numpy installation now gives a new error: error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools This seems to make it clear that uninstalling was a mistake, but the Numpy build instructions <https://www.scipy.org/scipylib/building/windows.html> say to specifically get the latest version of Visual Studio.
I reinstalled Microsoft Visual C++ 14.0. Then following various advice on the web, I copied rc.exe and rcdll.exe from C:\Program Files (x86)\Windows Kits\8.1\bin\x64 into C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64 It seems that MSVC++ 14.0 and that weird file copying is necessary for compiling Numpy. I can now install Numpy from source using python setup.py install. I then copied the site.cfg into every numpy/distutils directory I could find. However, compiling Scipy gives a LNK2001 linking error with some _gfortran symbols: mach.lib(d1mach.o) : error LNK2001: unresolved external symbol _gfortran_st_write mach.lib(d1mach.o) : error LNK2001: unresolved external symbol _gfortran_st_write_done mach.lib(d1mach.o) : error LNK2001: unresolved external symbol _gfortran_stop_numeric_f08 mach.lib(d1mach.o) : error LNK2001: unresolved external symbol _gfortran_transfer_character_write mach.lib(d1mach.o) : error LNK2001: unresolved external symbol _gfortran_transfer_integer_write mach.lib(d1mach.o) : error LNK2001: unresolved external symbol _gfortran_stop_string
Hi, David. You (re-?)learned a frustrating lesson: when testing the hypothesis that an installation is "in the way," before/instead of uninstalling, rename/move. Now that i've moralized, with evidence of a "bug in the doc", my belief is that you will be asked to file a ticket (doc bugs are considered just as legitimate bugs as code bugs). The only exception to this i can see is if someone else chimes in with a reasonable explanation as to why that's a "feature," not a bug, but in this instance, given what you say, that doesn't seem likely. DLG On Sat, May 6, 2017 at 7:50 AM David Hagen <david@drhagen.com> wrote:
I noticed that I had two versions of Visual Studio installed: Microsoft Visual C++ 14.0 and Microsoft Visual Studio 2017. I thought that this might be confusing the installer, so I uninstalled the older Microsoft Visual C++ 14.0. However, the Numpy installation now gives a new error:
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools
This seems to make it clear that uninstalling was a mistake, but the Numpy build instructions <https://www.scipy.org/scipylib/building/windows.html> say to specifically get the latest version of Visual Studio. _______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
1. The documentation definitely needs improvements. The difference here is that CPython requirements for MSVC compilers from Visual Studio are not the same as ifort requirements for Visual Studio. For example MSVC compiler for Python 2.7 (aka.ms/vcpython27) is likely not enough to install ifort. So 2.7 - MSVC 2008, 3.4 - MSVC 2010, 3.5-3.6 MSVC 2015/2017: https://wiki.python.org/moin/WindowsCompilers Here are ifort requirements for VS: https://software.intel.com /en-us/articles/troubleshooting-fortran-integration-issues-w ith-visual-studio 2. VS 2017 is supported by setuptools 34.4.0+ 3. Anyway I followed Intel article about scipy and ifort using VS 2015 with C++ compiler installed. In python 3.5, 64-bit both scipy and numpy compile without any issues even with beta 2018 ifort compiler. Don't forget to use the command prompt provided by Intel, Step 3 - Configuration: https://software.intel.com/en-us/articles/building-numpyscipy-with-intel-mkl... On Sat, May 6, 2017 at 10:25 AM, David Goldsmith < eulergaussriemann@gmail.com> wrote:
Hi, David. You (re-?)learned a frustrating lesson: when testing the hypothesis that an installation is "in the way," before/instead of uninstalling, rename/move.
Now that i've moralized, with evidence of a "bug in the doc", my belief is that you will be asked to file a ticket (doc bugs are considered just as legitimate bugs as code bugs). The only exception to this i can see is if someone else chimes in with a reasonable explanation as to why that's a "feature," not a bug, but in this instance, given what you say, that doesn't seem likely.
DLG
On Sat, May 6, 2017 at 7:50 AM David Hagen <david@drhagen.com> wrote:
I noticed that I had two versions of Visual Studio installed: Microsoft Visual C++ 14.0 and Microsoft Visual Studio 2017. I thought that this might be confusing the installer, so I uninstalled the older Microsoft Visual C++ 14.0. However, the Numpy installation now gives a new error:
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools
This seems to make it clear that uninstalling was a mistake, but the Numpy build instructions <https://www.scipy.org/scipylib/building/windows.html> say to specifically get the latest version of Visual Studio. _______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
The difference here is that CPython requirements for MSVC compilers from Visual Studio are not the same as ifort requirements for Visual Studio.
I currently have Python 3.6.1. I have both Visual Studio C++ 14.0 (I think this is the same thing as VS2015) and Visual Studio 2017 installed. I have Parallel Studio XE 2018 Beta installed. I am not sure if that is all compatible because 2018 Beta isn't on there. Should I uninstall VS 2017?
VS 2017 is supported by setuptools 34.4.0+
Should I upgrade from 27.2.0, which for whatever reason is the version that conda installs with Python 3.6.1?
Anyway I followed Intel article about scipy and ifort using VS 2015 with C++ compiler installed.
I am able to run plain python setup.py install on the latest version of Numpy and it installs fine. (Doing the same with Scipy errors out with missing symbols described previously.) However, running the install command given for Intel64 in the link it fails. I followed everything except for recompile Python. It gets to here: running config running build_clib running build_src build_src building py_modules sources building library "npymath" sources Could not locate executable icc Could not locate executable ecc And then throws a NoneType is not iterable exception: ... File "c:\numpy-1.12.1\numpy\distutils\msvc9compiler.py", line 53, in initialize os.environ['lib'] = _merge(environ_lib, os.environ['lib']) File "c:\numpy-1.12.1\numpy\distutils\msvc9compiler.py", line 32, in _merge if new in old: TypeError: argument of type 'NoneType' is not iterable This happen both with and without modifying the compile options. It looks like it can't find icc, but without a real error, it is difficult to tell if that was a notify, warning, or error. It installs with just python setup.py install. But if I do that is that the reason I get symbol errors trying to install Scipy? Is it just managing to muddle through with one of my other compilers but not be linkable by Scipy? I have Intel C++ Compiler installed as part of the Parallel Studio install.
I uninstalled all versions of MSVS and installed Visual Studio 2015 update 3. This did not fix anything, but at least I ruled out conflicts there. To get past the NoneType error from my previous message I had to fix a bug in Numpy; namely, in the file numpy\distutils\msvc9compiler.py the _merge function fails if old is None. I merely swap the two `if` statements and it got past. I will submit a pull request. Then the installer couldn't find icl.exe. I also had to add this to my PATH environment variable: C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2018.0.065\windows\bin\intel64 Then I got this error LINK : fatal error LNK1104: cannot open file 'libiomp5md.lib' which I tracked down to MPI. The instructions here <https://software.intel.com/en-us/articles/building-numpyscipy-with-intel-mkl...> had told me to add several compiler options in numpy/distutils/intelccompiler.py, one of which was openmp. Once I reverted the compiler options to their original state, this error went away. Then I got this error LINK : fatal error LNK1104: cannot open file 'libmmd.lib' I tracked this down to the compiler libraries not be on the library path so I added this to the LIB environment variable: C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2018.0.065\windows\compiler\lib\intel64 Finally Numpy installed and imports. But while it was nice to learn how to install Numpy from source, the main goal was to install Scipy from source. I thought that installing Numpy from source would fix my earlier problems with unresolved external _gfortran symbols. Alas, I still get this error while installing Scipy: building 'scipy.integrate._quadpack' extension compiling C sources C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2018.0.065\windows\bin\intel64\icl.exe /c /nologo /O3 /MD /W3 /Qstd=c99 -DSCIPY_MKL_H -DHAVE_CBLAS - Iscipy\_lib\src -I"C:\Program Files (x86)\IntelSWTools\parallel_studio_xe_2018.0.019\compilers_and_libraries_2018\windows\mkl\include" -IC:\Anaconda3\envs\numpy _source\lib\site-packages\numpy-1.12.1-py3.6-win-amd64.egg\numpy\core\include -IC:\Anaconda3\envs\numpy_source\include -IC:\Anaconda3\envs\numpy_source\include /Tcscipy\integrate\_quadpackmodule.c /Fobuild\temp.win-amd64-3.6\Release\scipy\integrate\_quadpackmodule.obj could not find library 'quadpack' in directories ['C:\\Program Files (x86)\\Inte lSWTools\\parallel_studio_xe_2018.0.019\\compilers_and_libraries_2018\\windows\\ mkl\\lib\\intel64'] ... Creating library build\temp.win-amd64-3.6\Release\scipy\integrate\_quadpack.c p36-win_amd64.lib and object build\temp.win-amd64-3.6\Release\scipy\integrate\_quadpack.cp36-win_amd64.exp mach.lib(d1mach.o) : error LNK2019: unresolved external symbol _gfortran_st_write referenced in function d1mach_ mach.lib(d1mach.o) : error LNK2019: unresolved external symbol _gfortran_st_write_done referenced in function d1mach_ mach.lib(d1mach.o) : error LNK2019: unresolved external symbol _gfortran_stop_numeric_f08 referenced in function d1mach_ mach.lib(d1mach.o) : error LNK2019: unresolved external symbol _gfortran_transfer_character_write referenced in function d1mach_ mach.lib(d1mach.o) : error LNK2019: unresolved external symbol _gfortran_transfer_integer_write referenced in function d1mach_ mach.lib(d1mach.o) : error LNK2019: unresolved external symbol _gfortran_stop_string referenced in function d1mach_ build\lib.win-amd64-3.6\scipy\integrate\_quadpack.cp36-win_amd64.pyd : fatal error LNK1120: 6 unresolved externals *What does this even mean?*
Ok, I solved everything. Here's the solution to the unresolved external symbols error: *DELETE THE BUILD FOLDER*. If Scipy starts compiling with one compiler, it will used the cached binaries when linking later even if it is now using a different compiler. Deleting the build folder forces it to start over.
Ah yes, the old "start the build fresh each time" problem--been there, done that. Even if something says i shouldn't have to start over, i no longer trust that: always start with a fresh build. DLG On Sat, May 13, 2017 at 6:01 PM David Hagen <david@drhagen.com> wrote:
Ok, I solved everything. Here's the solution to the unresolved external symbols error: *DELETE THE BUILD FOLDER*. If Scipy starts compiling with one compiler, it will used the cached binaries when linking later even if it is now using a different compiler. Deleting the build folder forces it to start over. _______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
I spoke too soon. I "successfully" compiled Numpy and Scipy from master and both can be imported. However, some Scipy items don't import. It doesn't really matter what exactly I import. If I run scipy.test() everything dies. Can someone interpret this error message?
from scipy.special import roots_legendre Traceback (most recent call last): File "__init__.pxd", line 987, in numpy.import_array (scipy\special\_ufuncs_cx x.cxx:3729) RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Anaconda3\envs\numpy_scipy\lib\site-packages\scipy- 1.0.0.dev0+unknown -py3.6-win-amd64.egg\scipy\special\__init__.py", line 640, in <module> from ._ufuncs import * File "scipy\special\_ufuncs.pyx", line 1, in init scipy.special._ufuncs (scipy \special\_ufuncs.c:39403) File "scipy\special\_ufuncs_extra_code_common.pxi", line 34, in init scipy.spe cial._ufuncs_cxx (scipy\special\_ufuncs_cxx.cxx:4376) File "__init__.pxd", line 989, in numpy.import_array (scipy\special\_ufuncs_cx x.cxx:3772) ImportError: numpy.core.multiarray failed to import
It looks like you have multiple numpy versions installed. Did you uninstall numpy, scipy with both pip and conda before building and installing numpy and scipy from source? I presume that you also built numpy from clean source without cache? Since you are using conda environments, make sure that you activate it before the build process and before launching python interpreter. On Sun, May 14, 2017 at 4:37 PM, David Hagen <david@drhagen.com> wrote:
I spoke too soon. I "successfully" compiled Numpy and Scipy from master and both can be imported. However, some Scipy items don't import. It doesn't really matter what exactly I import. If I run scipy.test() everything dies. Can someone interpret this error message?
from scipy.special import roots_legendre Traceback (most recent call last): File "__init__.pxd", line 987, in numpy.import_array (scipy\special\_ufuncs_cx x.cxx:3729) RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Anaconda3\envs\numpy_scipy\lib\site-packages\scipy-1.0. 0.dev0+unknown -py3.6-win-amd64.egg\scipy\special\__init__.py", line 640, in <module> from ._ufuncs import * File "scipy\special\_ufuncs.pyx", line 1, in init scipy.special._ufuncs (scipy \special\_ufuncs.c:39403) File "scipy\special\_ufuncs_extra_code_common.pxi", line 34, in init scipy.spe cial._ufuncs_cxx (scipy\special\_ufuncs_cxx.cxx:4376) File "__init__.pxd", line 989, in numpy.import_array (scipy\special\_ufuncs_cx x.cxx:3772) ImportError: numpy.core.multiarray failed to import
_______________________________________________ SciPy-User mailing list SciPy-User@python.org https://mail.python.org/mailman/listinfo/scipy-user
It looks like you have multiple numpy versions installed. Did you uninstall numpy, scipy with both pip and conda before building and installing numpy and scipy from source? I presume that you also built numpy from clean source without cache?
I have numpy installed in a separate conda environment because I don't want to lose my production environment while playing with Numpy and Scipy master. I got this error after creating a fresh conda environment, activating it, conda installing python and cython in it, and then running setup.py on freshly downloaded numpy and scipy master folders. How can I check if my other Numpy copies are visible in the search path for modules? Is there a command I can run to diagnose the version compatibility of the numpy and scipy modules that are imported and where they are located?
It looks like you have multiple numpy versions installed. Did you uninstall numpy, scipy with both pip and conda before building and installing numpy and scipy from source? I presume that you also built numpy from clean source without cache?
I can confirm it was another version of Numpy installed alongside. Anaconda had pulled it in when I installed matplotlib. Once I removed it and installed matplotlib from pip, then everything worked fine.
participants (7)
-
David Goldsmith -
David Hagen -
Denis Akhiyarov -
Matthieu Brucher -
Micha F -
Stephen P. Molnar -
William Heymann