[SciPy-dev] Compiling scipy with Intel ifort & MKL
rex
rex at nosyntax.com
Mon Jun 11 23:31:06 EDT 2007
rex <rex at nosyntax.com> [2007-06-11 16:12]:
> Pearu Peterson <pearu at cens.ioc.ee> [2007-06-11 15:12]:
> > It is due to a bug in numpy introduced, I think, just before releasing
> > 1.0.3. It has been fixed in numpy svn.
>
> Thank you! Building numpy from svn did eliminate the "error: file
> 'dfftpack/*.f' does not exist" message when building scipy, and the
> scipy compilation proceeded much further. However, it still produces an
> error which appears to be due to a failure to recognize the ID of ifort
> 10.
>
> I've already had to fix one problem in distutils due to Intel changing
> 'mkl_lapack32' in MKL8.1 to 'mkl_lapack' in MKL9.1. It appears another
> change is needed, perhaps in numpy/distutils/fcompiler/intel.py
I don't understand the distutils/* code well enough to fix it, but it
appears that the problem is triggered by packages that cause:
library 'mach' defined more than once, overwriting build_info {'sources': ['Lib/integrate/mach/i1mach.f', 'Lib/integrate/mach/d1mach.f', 'Lib/integrate/mach/r1mach.f', 'Lib/integrate/mach/xerror.f'], 'config_fc': {'noopt': ('Lib/integrate/setup.pyc', 1)}, 'source_languages': ['f77']} with {'sources': ['Lib/special/mach/i1mach.f', 'Lib/special/mach/d1mach.f', 'Lib/special/mach/r1mach.f', 'Lib/special/mach/xerror.f'], 'config_fc': {'noopt': ('Lib/special/setup.pyc', 1)}, 'source_languages': ['f77']}.
extending extension 'scipy.linsolve._zsuperlu' defined_macros with [('USE_VENDOR_BLAS', 1)]
This results in an error:
[...]
customize IntelCCompiler
customize IntelCCompiler using build_ext
Traceback (most recent call last):
File "setup.py", line 55, in <module>
setup_package()
File "setup.py", line 47, in setup_package
configuration=configuration )
File "/usr/local/lib/python2.5/site-packages/numpy/distutils/core.py", line 176, in setup
return old_setup(**new_attr)
File "/usr/lib/python2.5/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.5/distutils/dist.py", line 974, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
cmd_obj.run()
File "/usr/lib/python2.5/distutils/command/build.py", line 112, in run
self.run_command(cmd_name)
File "/usr/lib/python2.5/distutils/cmd.py", line 333, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.5/site-packages/numpy/distutils/command/build_ext.py", line 181, in run
if fcompiler and fcompiler.get_version():
File "/usr/local/lib/python2.5/site-packages/numpy/distutils/ccompiler.py", line 265, in CCompiler_get_version
cmd = ' '.join(version_cmd)
TypeError: sequence item 1: expected string, NoneType found
I determined this by the brute force method of deleting the current .f file
that triggered the problem. The result was another .f file causing the
same problem, leading to my conclusion that something in distutils/* is
broken for the v10 Intel compilers. (I've already verified and fixed a
problem in distutils/* re v9.1 MKL).
Here's a shortend (by ~5000 lines) session illustrating the error:
/usr/local/src/scipy/python setup.py config --compiler=intel --fcompiler=intel build >& build_noansari.log
mkl_info:
FOUND:
libraries = ['mkl', 'vml', 'guide', 'pthread']
library_dirs = ['/opt/intel/mkl/9.1/lib/32']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/intel/mkl/9.1/include']
blas_opt_info:
blas_mkl_info:
FOUND:
libraries = ['mkl', 'vml', 'guide', 'pthread']
library_dirs = ['/opt/intel/mkl/9.1/lib/32']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/intel/mkl/9.1/include']
FOUND:
libraries = ['mkl', 'vml', 'guide', 'pthread']
library_dirs = ['/opt/intel/mkl/9.1/lib/32']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/intel/mkl/9.1/include']
lapack_opt_info:
lapack_mkl_info:
FOUND:
libraries = ['mkl_lapack', 'mkl', 'vml', 'guide', 'pthread']
library_dirs = ['/opt/intel/mkl/9.1/lib/32']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/intel/mkl/9.1/include']
FOUND:
libraries = ['mkl_lapack', 'mkl', 'vml', 'guide', 'pthread']
library_dirs = ['/opt/intel/mkl/9.1/lib/32']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/intel/mkl/9.1/include']
non-existing path in 'Lib/linsolve': 'tests'
umfpack_info:
libraries umfpack not found in /opt/intel/mkl/9.1/lib/32
/usr/local/lib/python2.5/site-packages/numpy/distutils/system_info.py:403: UserWarning:
UMFPACK sparse solver (http://www.cise.ufl.edu/research/sparse/umfpack/)
not found. Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [umfpack]) or by setting
the UMFPACK environment variable.
warnings.warn(self.notfounderror.__doc__)
NOT AVAILABLE
Warning: Subpackage 'Lib' configuration returned as 'scipy'
non-existing path in 'Lib/maxentropy': 'doc'
running config
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
building py_modules sources
creating build
creating build/src.linux-i686-2.5
creating build/src.linux-i686-2.5/scipy
building library "dfftpack" sources
building library "linpack_lite" sources
building library "mach" sources
building library "quadpack" sources
building library "odepack" sources
building library "fitpack" sources
building library "superlu_src" sources
building library "odrpack" sources
building library "minpack" sources
building library "rootfind" sources
building library "c_misc" sources
building library "cephes" sources
building library "mach" sources
building library "toms" sources
building library "amos" sources
building library "cdf" sources
building library "specfun" sources
building library "statlib" sources
building extension "scipy.cluster._vq" sources
building extension "scipy.fftpack._fftpack" sources
creating build/src.linux-i686-2.5/Lib
creating build/src.linux-i686-2.5/Lib/fftpack
f2py options: []
f2py: Lib/fftpack/fftpack.pyf
Reading fortran codes...
Reading file 'Lib/fftpack/fftpack.pyf' (format:free)
Post-processing...
Block: _fftpack
Block: zfft
Block: drfft
Block: zrfft
Block: zfftnd
Block: destroy_zfft_cache
Block: destroy_zfftnd_cache
Block: destroy_drfft_cache
Post-processing (stage 2)...
Building modules...
Building module "_fftpack"...
Constructing wrapper function "zfft"...
getarrdims:warning: assumed shape array, using 0 instead of '*'
[snip]
Lib/fftpack/dfftpack/dfftb1.f(129): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(142): (col. 16) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(187): (col. 10) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(214): (col. 16) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(384): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(272): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(237): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftb1.f(315): (col. 13) remark: LOOP WAS VECTORIZED.
ifort:f77: Lib/fftpack/dfftpack/dfftf1.f
ifort: command line remark #10148: option '-K' not supported
Lib/fftpack/dfftpack/dfftf1.f(56): (col. 10) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(135): (col. 10) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(76): (col. 10) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(103): (col. 16) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(153): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(166): (col. 16) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(185): (col. 10) remark: PERMUTED LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(180): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(204): (col. 16) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(379): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(277): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(244): (col. 13) remark: LOOP WAS VECTORIZED.
Lib/fftpack/dfftpack/dfftf1.f(318): (col. 13) remark: LOOP WAS VECTORIZED.
ifort:f77: Lib/fftpack/dfftpack/dffti1.f
ifort: command line remark #10148: option '-K' not supported
[at least hundreds of yummy "LOOP WAS VECTORIZED" lines (and others) snipped]
customize IntelCCompiler
customize IntelCCompiler using build_ext
library 'mach' defined more than once, overwriting build_info {'sources': ['Lib/integrate/mach/i1mach.f', 'Lib/integrate/mach/d1mach.f', 'Lib/integrate/mach/r1mach.f', 'Lib/integrate/mach/xerror.f'], 'config_fc': {'noopt': ('Lib/integrate/setup.pyc', 1)}, 'source_languages': ['f77']} with {'sources': ['Lib/special/mach/i1mach.f', 'Lib/special/mach/d1mach.f', 'Lib/special/mach/r1mach.f', 'Lib/special/mach/xerror.f'], 'config_fc': {'noopt': ('Lib/special/setup.pyc', 1)}, 'source_languages': ['f77']}.
extending extension 'scipy.linsolve._zsuperlu' defined_macros with [('USE_VENDOR_BLAS', 1)]
extending extension 'scipy.linsolve._dsuperlu' defined_macros with [('USE_VENDOR_BLAS', 1)]
extending extension 'scipy.linsolve._csuperlu' defined_macros with [('USE_VENDOR_BLAS', 1)]
extending extension 'scipy.linsolve._ssuperlu' defined_macros with [('USE_VENDOR_BLAS', 1)]
customize IntelCCompiler
customize IntelCCompiler using build_ext
Traceback (most recent call last):
File "setup.py", line 55, in <module>
setup_package()
File "setup.py", line 47, in setup_package
configuration=configuration )
File "/usr/local/lib/python2.5/site-packages/numpy/distutils/core.py", line 176, in setup
return old_setup(**new_attr)
File "/usr/lib/python2.5/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.5/distutils/dist.py", line 974, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
cmd_obj.run()
File "/usr/lib/python2.5/distutils/command/build.py", line 112, in run
self.run_command(cmd_name)
File "/usr/lib/python2.5/distutils/cmd.py", line 333, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.5/site-packages/numpy/distutils/command/build_ext.py", line 181, in run
if fcompiler and fcompiler.get_version():
File "/usr/local/lib/python2.5/site-packages/numpy/distutils/ccompiler.py", line 265, in CCompiler_get_version
cmd = ' '.join(version_cmd)
TypeError: sequence item 1: expected string, NoneType found
More information about the SciPy-Dev
mailing list