[SciPy-User] Install Scipy with Anaconda's MKL libraries
David Hagen
david at drhagen.com
Wed May 3 21:26:42 EDT 2017
So those instructions get me farther than I have gotten before.
Is this actually necessary?:
Note: You will need to rebuild Python from source files. This is due to the
downloaded Python binary on Windows might be incompatible with the Visual
Studio version you used. Otherwise you will encounter runtime crash when
run numpy or script tests.
This is what I have done so far:
1) Download Scipy zip file from Github mater branch
2) Unzip file
3) Add a site.cfg file with these contents:
[mkl]
library_dirs = C:\Program Files
(x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\mkl\lib\intel64_win;C:\Program
Files
(x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\compiler\lib\intel64_win
include_dirs = C:\Program Files
(x86)\IntelSWTools\compilers_and_libraries_2017.2.187\windows\mkl\include
mkl_libs =
mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md
lapack_libs =
mkl_lapack95_lp64,mkl_blas95_lp64,mkl_intel_lp64,mkl_intel_thread,mkl_core,libiomp5md
4) cd into that directory
5) run: python setup.py config --compiler=intelemw --fcompiler=intelvem
build_clib --compiler=intelemw --fcompiler=intelvem build_ext
--compiler=intelemw --fcompiler=intelvem install
It gets really far into the build but this is the error that results:
building extension "scipy.spatial.qhull" sources
creating build\src.win-amd64-3.6\scipy\spatial
Could not locate executable icc
Could not locate executable ecc
Traceback (most recent call last):
File "setup.py", line 417, in <module>
setup_package()
File "setup.py", line 413, in setup_package
setup(**metadata)
File
"C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\core.py
", line 166, in setup
return old_setup(**new_attr)
File "C:\Anaconda3\envs\scipy_master\lib\distutils\core.py", line 148, in
setu
p
dist.run_commands()
File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 955, in
run_
commands
self.run_command(cmd)
File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974, in
run_
command
cmd_obj.run()
File
"C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
\build_clib.py", line 74, in run
self.run_command('build_src')
File "C:\Anaconda3\envs\scipy_master\lib\distutils\cmd.py", line 313, in
run_c
ommand
self.distribution.run_command(command)
File "C:\Anaconda3\envs\scipy_master\lib\distutils\dist.py", line 974, in
run_
command
cmd_obj.run()
File
"C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
\build_src.py", line 148, in run
self.build_sources()
File
"C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
\build_src.py", line 165, in build_sources
self.build_extension_sources(ext)
File
"C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
\build_src.py", line 324, in build_extension_sources
sources = self.generate_sources(sources, ext)
File
"C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
\build_src.py", line 377, in generate_sources
source = func(extension, build_dir)
File "scipy\spatial\setup.py", line 35, in get_qhull_misc_config
if config_cmd.check_func('open_memstream', decl=True, call=True):
File
"C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
\config.py", line 312, in check_func
self._check_compiler()
File
"C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\command
\config.py", line 51, in _check_compiler
self.compiler.initialize()
File
"C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\intelcc
ompiler.py", line 86, in initialize
MSVCCompiler.initialize(self, plat_name)
File
"C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\msvc9co
mpiler.py", line 53, in initialize
os.environ['lib'] = _merge(environ_lib, os.environ['lib'])
File
"C:\Anaconda3\envs\scipy_master\lib\site-packages\numpy\distutils\msvc9co
mpiler.py", line 32, in _merge
if new in old:
TypeError: argument of type 'NoneType' is not iterable
I notice that this is calling into Numpy. Do I have modify Numpy in order
to compile Scipy? I skipped the Numpy specific steps in the instructions
because I don't need the development version of Numpy.
On Tue, May 2, 2017 at 11:15 PM, Denis Akhiyarov <denis.akhiyarov at gmail.com>
wrote:
> Did you follow these instructions?
>
> https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl
>
> On Tue, May 2, 2017, 10:00 PM David Hagen <david at drhagen.com> wrote:
>
>> Is there a recipe for this combination? I installed MKL from that link
>> and the latest Visual Studio. Scipy did not find MKL on its own. I'm sure
>> there's some environment variables that need to be set, but I don't know
>> what they are.
>>
>> On Tue, May 2, 2017 at 4:00 AM, William Heymann <immudzen at gmail.com>
>> wrote:
>>
>>> Intel has made MKL, TBB, and a few other things completely free to use,
>>> even in a commercial project. Visual Studio is also free unless you are a
>>> very large company.
>>>
>>> https://software.intel.com/en-us/articles/free-mkl
>>>
>>> I have been using that for other projects without any problems and
>>> compiling with Visual Studio has been very easy.
>>>
>>> On Tue, May 2, 2017 at 1:40 AM, David Hagen <david at drhagen.com> wrote:
>>>
>>>> I'll try to stick with MinGW-w64 for now, but I don't even get to the
>>>> compilation phase. If I install lapack and blas from conda-forge, it still
>>>> says that lapack/blas are not found, but you indicated that I need to set
>>>> some paths. Are there instructions for this? I have no idea what
>>>> environment variables to set in order to tell Scipy to use these packages.
>>>>
>>>> On Mon, May 1, 2017 at 3:44 AM, Denis Akhiyarov <
>>>> denis.akhiyarov at gmail.com> wrote:
>>>>
>>>>> I still suggest Intel+MSVC compilers, since you can use trial version
>>>>> or request license for open-source projects from Intel. This is what
>>>>> Anaconda team is using. Also this is what Christoph Gohlke wheels are based
>>>>> on:
>>>>>
>>>>> http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy
>>>>>
>>>>> If you end up with m2w64, here is lapack for conda, you may still have
>>>>> to modify paths:
>>>>>
>>>>> https://anaconda.org/conda-forge/lapack
>>>>>
>>>>> And blas:
>>>>>
>>>>> https://anaconda.org/search?q=Blas
>>>>>
>>>>> On Sun, Apr 30, 2017, 5:22 PM Matthieu Brucher <
>>>>> matthieu.brucher at gmail.com> wrote:
>>>>>
>>>>>> Why do you want to pay Intel? You can install the MKL and develop
>>>>>> with it, no sweat.
>>>>>>
>>>>>> 2017-04-30 22:41 GMT+01:00 David Hagen <david at drhagen.com>:
>>>>>>
>>>>>>> > Welcome to the world of pain with building scientific packages
>>>>>>> from source on Windows!
>>>>>>>
>>>>>>> I am beginning to feel it.
>>>>>>>
>>>>>>> > You need Fortran and C/C++ compilers on Windows to build scipy
>>>>>>> from source
>>>>>>>
>>>>>>> I have MinGW-w64 installed, which seems to be the recommended method.
>>>>>>>
>>>>>>> > I’m pretty sure that anaconda does not come with the development
>>>>>>> files for MKL, only the runtime files.
>>>>>>>
>>>>>>> I understand now. It looks like MKL is not the way to go unless I
>>>>>>> want to pay Intel.
>>>>>>>
>>>>>>> > If you don't need mkl and lapack/blas is good enough, then
>>>>>>> m2w64-toolchain from conda should have all necessary dependencies for
>>>>>>> building scipy.
>>>>>>>
>>>>>>> My only goal is to install and use Scipy master somewhere where it
>>>>>>> won't break my stable installation. I thought Anaconda would be a good
>>>>>>> place to start because once I activate an Anaconda environment, I should be
>>>>>>> able to treat like a normal Python installation and follow the normal
>>>>>>> install-from-source instructions. I went ahead and installed that
>>>>>>> m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I
>>>>>>> should change my question to: how do I install Scipy on Windows from
>>>>>>> source? Though when I search for this specifically on the web, the answer
>>>>>>> seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by
>>>>>>> Scipy. Do you know of a conda/pip package that provides ATLAS for building
>>>>>>> Scipy or another more suitable BLAS/LAPACK?
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> SciPy-User mailing list
>>>>>>> SciPy-User at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Information System Engineer, Ph.D.
>>>>>> Blog: http://blog.audio-tk.com/
>>>>>> LinkedIn: http://www.linkedin.com/in/matthieubrucher
>>>>>> _______________________________________________
>>>>>> SciPy-User mailing list
>>>>>> SciPy-User at python.org
>>>>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> SciPy-User mailing list
>>>>> SciPy-User at python.org
>>>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> SciPy-User mailing list
>>>> SciPy-User at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>>
>>>>
>>>
>>> _______________________________________________
>>> SciPy-User mailing list
>>> SciPy-User at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>
>>>
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at python.org
>> https://mail.python.org/mailman/listinfo/scipy-user
>>
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org
> https://mail.python.org/mailman/listinfo/scipy-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20170503/658af4d0/attachment-0001.html>
More information about the SciPy-User
mailing list