[SciPy-User] Install Scipy with Anaconda's MKL libraries

Denis Akhiyarov denis.akhiyarov at gmail.com
Thu May 4 02:58:34 EDT 2017 

Ok, I followed instructions on this page:

https://software.intel.com/en-us/articles/building-numpyscipy-with-intel-mkl-and-intel-fortran-on-windows

Installed parallel_studio_xe_2016_update4 with mkl, ifort, and icc.

1. I compiled numpy (python setup.py install) from source with this
site.cfg file:

[mkl]
include_dirs = C:\Program Files
(x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\mkl\include
library_dirs = C:\Program Files
(x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\mkl\lib\intel64;C:\Program
Files
(x86)\IntelSWTools\parallel_studio_xe_2016.4.062\compilers_and_libraries_2016\windows\compiler\lib\intel64
mkl_libs = mkl_core_dll, mkl_intel_lp64_dll, mkl_intel_thread_dll
lapack_libs = mkl_lapack95_lp64


2. I replaced site.cfg file in ..\Lib\site-packages\numpy\distutils with
the one above, since this is used by scipy in step 3.

3. I compiled scipy (python setup.py install) from source.

Compilation went without errors, both packages import without errors in
python.


On Wed, May 3, 2017 at 8:26 PM, David Hagen <david at drhagen.com> wrote:

> So those instructions get me farther than I have gotten before.
>
> Is this actually necessary?:
>
> Note: You will need to rebuild Python from source files. This is due to
> the downloaded Python binary on Windows might be incompatible with the
> Visual Studio version you used. Otherwise you will encounter runtime crash
> when run numpy or script tests.
>
> This is what I have done so far:
>
> 1) Download Scipy zip file from Github mater branch
> 2) Unzip file
> 3) Add a site.cfg file with these contents:
> [mkl]
> library_dirs = C:\Program Files (x86)\IntelSWTools\compilers_
> and_libraries_2017.2.187\windows\mkl\lib\intel64_win;C:\Program Files
> (x86)\IntelSWTools\compilers_and_libraries_2017.2.187\
> windows\compiler\lib\intel64_win
> include_dirs = C:\Program Files (x86)\IntelSWTools\compilers_
> and_libraries_2017.2.187\windows\mkl\include
> mkl_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_
> thread,mkl_core,libiomp5md
> lapack_libs = mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_
> thread,mkl_core,libiomp5md
>
> 4) cd into that directory
> 5) run: python setup.py config --compiler=intelemw --fcompiler=intelvem
> build_clib --compiler=intelemw --fcompiler=intelvem build_ext
> --compiler=intelemw --fcompiler=intelvem install
>
> It gets really far into the build but this is the error that results:
>
> building extension "scipy.spatial.qhull" sources
> creating build\src.win-amd64-3.6\scipy\spatial
> Could not locate executable icc
> Could not locate executable ecc
> Traceback (most recent call last):
>   File "setup.py", line 417, in <module>
>     setup_package()
>   File "setup.py", line 413, in setup_package
>     setup(**metadata)
>   File "C:\Anaconda3\envs\scipy_master\lib\site-packages\
> numpy\distutils\core.py
> ", line 166, in setup
>     return old_setup(**new_attr)
>   File "C:\Anaconda3\envs\scipy_master\lib\distutils\core.py", line 148,
> in setu
> p
>     dist.run_commands()
>   File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 955,
> in run_
> commands
>     self.run_command(cmd)
>   File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974,
> in run_
> command
>     cmd_obj.run()
>   File "C:\Anaconda3\envs\scipy_master\lib\site-packages\
> numpy\distutils\command
> \build_clib.py", line 74, in run
>     self.run_command('build_src')
>   File "C:\Anaconda3\envs\scipy_master\lib\distutils\cmd.py", line 313,
> in run_c
> ommand
>     self.distribution.run_command(command)
>   File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974,
> in run_
> command
>     cmd_obj.run()
>   File "C:\Anaconda3\envs\scipy_master\lib\site-packages\
> numpy\distutils\command
> \build_src.py", line 148, in run
>     self.build_sources()
>   File "C:\Anaconda3\envs\scipy_master\lib\site-packages\
> numpy\distutils\command
> \build_src.py", line 165, in build_sources
>     self.build_extension_sources(ext)
>   File "C:\Anaconda3\envs\scipy_master\lib\site-packages\
> numpy\distutils\command
> \build_src.py", line 324, in build_extension_sources
>     sources = self.generate_sources(sources, ext)
>   File "C:\Anaconda3\envs\scipy_master\lib\site-packages\
> numpy\distutils\command
> \build_src.py", line 377, in generate_sources
>     source = func(extension, build_dir)
>   File "scipy\spatial\setup.py", line 35, in get_qhull_misc_config
>     if config_cmd.check_func('open_memstream', decl=True, call=True):
>   File "C:\Anaconda3\envs\scipy_master\lib\site-packages\
> numpy\distutils\command
> \config.py", line 312, in check_func
>     self._check_compiler()
>   File "C:\Anaconda3\envs\scipy_master\lib\site-packages\
> numpy\distutils\command
> \config.py", line 51, in _check_compiler
>     self.compiler.initialize()
>   File "C:\Anaconda3\envs\scipy_master\lib\site-packages\
> numpy\distutils\intelcc
> ompiler.py", line 86, in initialize
>     MSVCCompiler.initialize(self, plat_name)
>   File "C:\Anaconda3\envs\scipy_master\lib\site-packages\
> numpy\distutils\msvc9co
> mpiler.py", line 53, in initialize
>     os.environ['lib'] = _merge(environ_lib, os.environ['lib'])
>   File "C:\Anaconda3\envs\scipy_master\lib\site-packages\
> numpy\distutils\msvc9co
> mpiler.py", line 32, in _merge
>     if new in old:
> TypeError: argument of type 'NoneType' is not iterable
>
> I notice that this is calling into Numpy. Do I have modify Numpy in order
> to compile Scipy? I skipped the Numpy specific steps in the instructions
> because I don't need the development version of Numpy.
>
> On Tue, May 2, 2017 at 11:15 PM, Denis Akhiyarov <
> denis.akhiyarov at gmail.com> wrote:
>
>> Did you follow these instructions?
>>
>> https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl
>>
>> On Tue, May 2, 2017, 10:00 PM David Hagen <david at drhagen.com> wrote:
>>
>>> Is there a recipe for this combination? I installed MKL from that link
>>> and the latest Visual Studio. Scipy did not find MKL on its own. I'm sure
>>> there's some environment variables that need to be set, but I don't know
>>> what they are.
>>>
>>> On Tue, May 2, 2017 at 4:00 AM, William Heymann <immudzen at gmail.com>
>>> wrote:
>>>
>>>> Intel has made MKL, TBB, and a few other things completely free to use,
>>>> even in a commercial project. Visual Studio is also free unless you are a
>>>> very large company.
>>>>
>>>> https://software.intel.com/en-us/articles/free-mkl
>>>>
>>>> I have been using that for other projects without any problems and
>>>> compiling with Visual Studio has been very easy.
>>>>
>>>> On Tue, May 2, 2017 at 1:40 AM, David Hagen <david at drhagen.com> wrote:
>>>>
>>>>> I'll try to stick with MinGW-w64 for now, but I don't even get to the
>>>>> compilation phase. If I install lapack and blas from conda-forge, it still
>>>>> says that lapack/blas are not found, but you indicated that I need to set
>>>>> some paths. Are there instructions for this? I have no idea what
>>>>> environment variables to set in order to tell Scipy to use these packages.
>>>>>
>>>>> On Mon, May 1, 2017 at 3:44 AM, Denis Akhiyarov <
>>>>> denis.akhiyarov at gmail.com> wrote:
>>>>>
>>>>>> I still suggest Intel+MSVC compilers, since you can use trial version
>>>>>> or request license for open-source projects from Intel. This is what
>>>>>> Anaconda team is using. Also this is what Christoph Gohlke wheels are based
>>>>>> on:
>>>>>>
>>>>>> http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy
>>>>>>
>>>>>> If you end up with m2w64, here is lapack for conda, you may still
>>>>>> have to modify paths:
>>>>>>
>>>>>> https://anaconda.org/conda-forge/lapack
>>>>>>
>>>>>> And blas:
>>>>>>
>>>>>> https://anaconda.org/search?q=Blas
>>>>>>
>>>>>> On Sun, Apr 30, 2017, 5:22 PM Matthieu Brucher <
>>>>>> matthieu.brucher at gmail.com> wrote:
>>>>>>
>>>>>>> Why do you want to pay Intel? You can install the MKL and develop
>>>>>>> with it, no sweat.
>>>>>>>
>>>>>>> 2017-04-30 22:41 GMT+01:00 David Hagen <david at drhagen.com>:
>>>>>>>
>>>>>>>> > Welcome to the world of pain with building scientific packages
>>>>>>>> from source on Windows!
>>>>>>>>
>>>>>>>> I am beginning to feel it.
>>>>>>>>
>>>>>>>> > You need Fortran and C/C++ compilers on Windows to build scipy
>>>>>>>> from source
>>>>>>>>
>>>>>>>> I have MinGW-w64 installed, which seems to be the recommended
>>>>>>>> method.
>>>>>>>>
>>>>>>>> > I’m pretty sure that anaconda does not come with the development
>>>>>>>> files for MKL, only the runtime files.
>>>>>>>>
>>>>>>>> I understand now. It looks like MKL is not the way to go unless I
>>>>>>>> want to pay Intel.
>>>>>>>>
>>>>>>>> > If you don't need mkl and lapack/blas is good enough, then
>>>>>>>> m2w64-toolchain from conda should have all necessary dependencies for
>>>>>>>> building scipy.
>>>>>>>>
>>>>>>>> My only goal is to install and use Scipy master somewhere where it
>>>>>>>> won't break my stable installation. I thought Anaconda would be a good
>>>>>>>> place to start because once I activate an Anaconda environment, I should be
>>>>>>>> able to treat like a normal Python installation and follow the normal
>>>>>>>> install-from-source instructions. I went ahead and installed that
>>>>>>>> m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I
>>>>>>>> should change my question to: how do I install Scipy on Windows from
>>>>>>>> source? Though when I search for this specifically on the web, the answer
>>>>>>>> seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by
>>>>>>>> Scipy. Do you know of a conda/pip package that provides ATLAS for building
>>>>>>>> Scipy or another more suitable BLAS/LAPACK?
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> SciPy-User mailing list
>>>>>>>> SciPy-User at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Information System Engineer, Ph.D.
>>>>>>> Blog: http://blog.audio-tk.com/
>>>>>>> LinkedIn: http://www.linkedin.com/in/matthieubrucher
>>>>>>> _______________________________________________
>>>>>>> SciPy-User mailing list
>>>>>>> SciPy-User at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> SciPy-User mailing list
>>>>>> SciPy-User at python.org
>>>>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> SciPy-User mailing list
>>>>> SciPy-User at python.org
>>>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> SciPy-User mailing list
>>>> SciPy-User at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>>
>>>>
>>> _______________________________________________
>>> SciPy-User mailing list
>>> SciPy-User at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>
>>
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at python.org
>> https://mail.python.org/mailman/listinfo/scipy-user
>>
>>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org
> https://mail.python.org/mailman/listinfo/scipy-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20170504/00cbc834/attachment-0001.html>

More information about the SciPy-User mailing list