[SciPy-User] Install Scipy with Anaconda's MKL libraries
Stephen P. Molnar
s.molnar at sbcglobal.net
Thu May 4 09:20:53 EDT 2017
On 05/04/2017 08:16 AM, David Hagen wrote:
> Am I correct that there is no free version of ifort and icc? Will this
> break after the 30-trial ends?
>
> On Thu, May 4, 2017 at 2:58 AM, Denis Akhiyarov
> <denis.akhiyarov at gmail.com <mailto:denis.akhiyarov at gmail.com>> wrote:
>
> Ok, I followed instructions on this page:
>
> https://software.intel.com/en-us/articles/building-numpyscipy-with-intel-mkl-and-intel-fortran-on-windows
> <https://software.intel.com/en-us/articles/building-numpyscipy-with-intel-mkl-and-intel-fortran-on-windows>
>
> Installed parallel_studio_xe_2016_update4 with mkl, ifort, and icc.
>
> 1. I compiled numpy (python setup.py install) from source with this
> site.cfg file:
>
> [mkl]
> include_dirs = C:\Program Files
> (x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\mkl\include
> library_dirs = C:\Program Files
> (x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\mkl\lib\intel64;C:\Program
> Files
> (x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\compiler\lib\intel64
> mkl_libs = mkl_core_dll, mkl_intel_lp64_dll, mkl_intel_thread_dll
> lapack_libs = mkl_lapack95_lp64
>
>
> 2. I replaced site.cfg file in ..\Lib\site-packages\numpy\distutils
> with the one above, since this is used by scipy in step 3.
>
> 3. I compiled scipy (python setup.py install) from source.
>
> Compilation went without errors, both packages import without errors
> in python.
>
>
> On Wed, May 3, 2017 at 8:26 PM, David Hagen <david at drhagen.com
> <mailto:david at drhagen.com>> wrote:
>
> So those instructions get me farther than I have gotten before.
>
> Is this actually necessary?:
>
> Note: You will need to rebuild Python from source files. This is
> due to the downloaded Python binary on Windows might
> be incompatible with the Visual Studio version you used.
> Otherwise you will encounter runtime crash when run numpy or
> script tests.
>
> This is what I have done so far:
>
> 1) Download Scipy zip file from Github mater branch
> 2) Unzip file
> 3) Add a site.cfg file with these contents:
> [mkl]
> library_dirs = C:\Program Files
> (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\mkl\lib\intel64_win;C:\Program
> Files
> (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\compiler\lib\intel64_win
> include_dirs = C:\Program Files
> (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\mkl\include
> mkl_libs =
> mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md
> lapack_libs =
> mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md
>
> 4) cd into that directory
> 5) run: python setup.py config --compiler=intelemw
> --fcompiler=intelvem build_clib --compiler=intelemw
> --fcompiler=intelvem build_ext --compiler=intelemw
> --fcompiler=intelvem install
>
> It gets really far into the build but this is the error that
> results:
>
> building extension "scipy.spatial.qhull" sources
> creating build\src.win-amd64-3.6\scipy\spatial
> Could not locate executable icc
> Could not locate executable ecc
> Traceback (most recent call last):
> File "setup.py", line 417, in <module>
> setup_package()
> File "setup.py", line 413, in setup_package
> setup(**metadata)
> File
> "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\core.py
> ", line 166, in setup
> return old_setup(**new_attr)
> File "C:\Anaconda3\envs\scipy_master\lib\distutils\core.py",
> line 148, in setu
> p
> dist.run_commands()
> File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py",
> line 955, in run_
> commands
> self.run_command(cmd)
> File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py",
> line 974, in run_
> command
> cmd_obj.run()
> File
> "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
> \build_clib.py", line 74, in run
> self.run_command('build_src')
> File "C:\Anaconda3\envs\scipy_master\lib\distutils\cmd.py",
> line 313, in run_c
> ommand
> self.distribution.run_command(command)
> File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py",
> line 974, in run_
> command
> cmd_obj.run()
> File
> "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
> \build_src.py", line 148, in run
> self.build_sources()
> File
> "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
> \build_src.py", line 165, in build_sources
> self.build_extension_sources(ext)
> File
> "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
> \build_src.py", line 324, in build_extension_sources
> sources = self.generate_sources(sources, ext)
> File
> "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
> \build_src.py", line 377, in generate_sources
> source = func(extension, build_dir)
> File "scipy\spatial\setup.py", line 35, in get_qhull_misc_config
> if config_cmd.check_func('open_memstream', decl=True,
> call=True):
> File
> "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
> \config.py", line 312, in check_func
> self._check_compiler()
> File
> "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
> \config.py", line 51, in _check_compiler
> self.compiler.initialize()
> File
> "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\intelcc
> ompiler.py", line 86, in initialize
> MSVCCompiler.initialize(self, plat_name)
> File
> "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\msvc9co
> mpiler.py", line 53, in initialize
> os.environ['lib'] = _merge(environ_lib, os.environ['lib'])
> File
> "C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\msvc9co
> mpiler.py", line 32, in _merge
> if new in old:
> TypeError: argument of type 'NoneType' is not iterable
>
> I notice that this is calling into Numpy. Do I have modify Numpy
> in order to compile Scipy? I skipped the Numpy specific steps in
> the instructions because I don't need the development version of
> Numpy.
>
> On Tue, May 2, 2017 at 11:15 PM, Denis Akhiyarov
> <denis.akhiyarov at gmail.com <mailto:denis.akhiyarov at gmail.com>>
> wrote:
>
> Did you follow these instructions?
>
> https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl
> <https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl>
>
>
> On Tue, May 2, 2017, 10:00 PM David Hagen <david at drhagen.com
> <mailto:david at drhagen.com>> wrote:
>
> Is there a recipe for this combination? I installed MKL
> from that link and the latest Visual Studio. Scipy did
> not find MKL on its own. I'm sure there's some
> environment variables that need to be set, but I don't
> know what they are.
>
> On Tue, May 2, 2017 at 4:00 AM, William Heymann
> <immudzen at gmail.com <mailto:immudzen at gmail.com>> wrote:
>
> Intel has made MKL, TBB, and a few other things
> completely free to use, even in a commercial
> project. Visual Studio is also free unless you are a
> very large company.
>
> https://software.intel.com/en-us/articles/free-mkl
> <https://software.intel.com/en-us/articles/free-mkl>
>
> I have been using that for other projects without
> any problems and compiling with Visual Studio has
> been very easy.
>
> On Tue, May 2, 2017 at 1:40 AM, David Hagen
> <david at drhagen.com <mailto:david at drhagen.com>> wrote:
>
> I'll try to stick with MinGW-w64 for now, but I
> don't even get to the compilation phase. If I
> install lapack and blas from conda-forge, it
> still says that lapack/blas are not found, but
> you indicated that I need to set some paths. Are
> there instructions for this? I have no idea what
> environment variables to set in order to tell
> Scipy to use these packages.
>
> On Mon, May 1, 2017 at 3:44 AM, Denis Akhiyarov
> <denis.akhiyarov at gmail.com
> <mailto:denis.akhiyarov at gmail.com>> wrote:
>
> I still suggest Intel+MSVC compilers, since
> you can use trial version or request license
> for open-source projects from Intel. This is
> what Anaconda team is using. Also this is
> what Christoph Gohlke wheels are based on:
>
> http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy
> <http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy>
>
> If you end up with m2w64, here is lapack for
> conda, you may still have to modify paths:
>
> https://anaconda.org/conda-forge/lapack
> <https://anaconda.org/conda-forge/lapack>
>
> And blas:
>
> https://anaconda.org/search?q=Blas
> <https://anaconda.org/search?q=Blas>
>
>
> On Sun, Apr 30, 2017, 5:22 PM Matthieu
> Brucher <matthieu.brucher at gmail.com
> <mailto:matthieu.brucher at gmail.com>> wrote:
>
> Why do you want to pay Intel? You can
> install the MKL and develop with it, no
> sweat.
>
> 2017-04-30 22:41 GMT+01:00 David Hagen
> <david at drhagen.com
> <mailto:david at drhagen.com>>:
>
> > Welcome to the world of pain with building scientific packages from source on Windows!
>
> I am beginning to feel it.
>
> > You need Fortran and C/C++ compilers on Windows to build scipy from source
>
> I have MinGW-w64 installed, which
> seems to be the recommended method.
>
> > I’m pretty sure that anaconda does not come with the development files for MKL, only the runtime files.
>
> I understand now. It looks like MKL
> is not the way to go unless I want
> to pay Intel.
>
> > If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy.
>
> My only goal is to install and use
> Scipy master somewhere where it
> won't break my stable installation.
> I thought Anaconda would be a good
> place to start because once I
> activate an Anaconda environment, I
> should be able to treat like a
> normal Python installation and
> follow the normal
> install-from-source instructions. I
> went ahead and installed that
> m2w64-toolchain package, but it
> still doesn't find any BLAS/LAPACK.
> Maybe I should change my question
> to: how do I install Scipy on
> Windows from source? Though when I
> search for this specifically on the
> web, the answer seems to be
> "Don't.". It seems that MinGW-w64
> and ATLAS are recommended by Scipy.
> Do you know of a conda/pip package
> that provides ATLAS for building
> Scipy or another more suitable
> BLAS/LAPACK?
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org
> <mailto:SciPy-User at python.org>
> https://mail.python.org/mailman/listinfo/scipy-user
> <https://mail.python.org/mailman/listinfo/scipy-user>
>
>
>
>
> --
> Information System Engineer, Ph.D.
> Blog: http://blog.audio-tk.com/
> LinkedIn:
> http://www.linkedin.com/in/matthieubrucher
> <http://www.linkedin.com/in/matthieubrucher>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org
> <mailto:SciPy-User at python.org>
> https://mail.python.org/mailman/listinfo/scipy-user
> <https://mail.python.org/mailman/listinfo/scipy-user>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org
> <mailto:SciPy-User at python.org>
> https://mail.python.org/mailman/listinfo/scipy-user
> <https://mail.python.org/mailman/listinfo/scipy-user>
>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org <mailto:SciPy-User at python.org>
> https://mail.python.org/mailman/listinfo/scipy-user
> <https://mail.python.org/mailman/listinfo/scipy-user>
>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org <mailto:SciPy-User at python.org>
> https://mail.python.org/mailman/listinfo/scipy-user
> <https://mail.python.org/mailman/listinfo/scipy-user>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org <mailto:SciPy-User at python.org>
> https://mail.python.org/mailman/listinfo/scipy-user
> <https://mail.python.org/mailman/listinfo/scipy-user>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org <mailto:SciPy-User at python.org>
> https://mail.python.org/mailman/listinfo/scipy-user
> <https://mail.python.org/mailman/listinfo/scipy-user>
>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org <mailto:SciPy-User at python.org>
> https://mail.python.org/mailman/listinfo/scipy-user
> <https://mail.python.org/mailman/listinfo/scipy-user>
>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org <mailto:SciPy-User at python.org>
> https://mail.python.org/mailman/listinfo/scipy-user
> <https://mail.python.org/mailman/listinfo/scipy-user>
>
>
>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org
> https://mail.python.org/mailman/listinfo/scipy-us
No, you are not correct about ifort. It's wstill available. ayou just
won't receive support.
--
Stephen P. Molnar, Ph.D. Life is a fuzzy set
www.molecular-modeling.net Stochastic and multivariate
(614)312-7528 (c)
Skype: smolnar1
More information about the SciPy-User
mailing list