Hi,
Gregor and I are trying to give support for Intel's VML (Vector Mathematical Library) in numexpr. For this, we are trying to make use of the weaponery in NumPy's distutils so as to be able to discover where the MKL (the package that contains VML) is located. The libraries that we need to link with are: mkl_gf_lp64, mkl_gnu_thread, mkl_core *and* iomp5 (Intel's OpenMP library).
The problem is that I have installed MKL as part as the Intel compiler for Unix. In this setup, most of the libraries are in one place, namely:
/opt/intel/Compiler/11.0/074/mkl/lib/em64t/
However, the OpenMP library is in another directory:
/opt/intel/Compiler/11.0/074/lib/intel64
So, I need to specify *two* directories to get the complete set of libraries. My first attempt was setting a site.cfg like:
[DEFAULT] #libraries = gfortran
[mkl] library_dirs= /opt/intel/Compiler/11.0/074/mkl/lib/em64t/:/opt/intel/Compiler/11.0/074/lib/intel64 include_dirs = /opt/intel/Compiler/11.0/074/mkl/include/ mkl_libs = mkl_gf_lp64, mkl_gnu_thread, mkl_core, iomp5
Unfortunately, distutils complains and says that it cannot find the complete set of libraries:
mkl_info: libraries mkl_gf_lp64,mkl_gnu_thread,mkl_core,iomp5 not found in /opt/intel/Compiler/11.0/074/mkl/lib/em64t/ libraries mkl_gf_lp64,mkl_gnu_thread,mkl_core,iomp5 not found in /opt/intel/Compiler/11.0/074/lib/intel64 NOT AVAILABLE
After some debugging of the problem, it seems that distutils needs to find *all* the required libraries in *one* single directory. As iomp5 is on a different directory, distutils thinks that the requisites are not fulfilled.
I've solved this by requering the iomp5 to be find in the DEFAULT section. Something like:
[DEFAULT] library_dirs = /opt/intel/Compiler/11.0/074/lib/intel64 #libraries = gfortran, iomp5 libraries = iomp5
[mkl] library_dirs = /opt/intel/Compiler/11.0/074/mkl/lib/em64t/ include_dirs = /opt/intel/Compiler/11.0/074/mkl/include/ mkl_libs = mkl_gf_lp64, mkl_gnu_thread, mkl_core
However, in case one would need to specify several other directories in the DEFAULT section for finding other hypothetical necessary libraries (like gfortran or others), we may run into the same problem than above.
My question is, is there an elegant way to handle this problem, or it is a limitation of the current distutils? If the later, it would be nice it that could be solved in a future version, and several libraries can be found in *several* directories.
Thanks,
On Fri, Jan 30, 2009 at 14:30, Francesc Alted faltet@pytables.org wrote:
My question is, is there an elegant way to handle this problem, or it is a limitation of the current distutils?
Probably the latter.
If the later, it would be nice it that could be solved in a future version, and several libraries can be found in *several* directories.
Patches are welcome.
On Sat, Jan 31, 2009 at 5:30 AM, Francesc Alted faltet@pytables.org wrote:
Hi,
Gregor and I are trying to give support for Intel's VML (Vector Mathematical Library) in numexpr. For this, we are trying to make use of the weaponery in NumPy's distutils so as to be able to discover where the MKL (the package that contains VML) is located. The libraries that we need to link with are: mkl_gf_lp64, mkl_gnu_thread, mkl_core *and* iomp5 (Intel's OpenMP library).
The problem is that I have installed MKL as part as the Intel compiler for Unix. In this setup, most of the libraries are in one place, namely:
/opt/intel/Compiler/11.0/074/mkl/lib/em64t/
However, the OpenMP library is in another directory:
/opt/intel/Compiler/11.0/074/lib/intel64
So, I need to specify *two* directories to get the complete set of libraries. My first attempt was setting a site.cfg like:
[DEFAULT] #libraries = gfortran
[mkl] library_dirs= /opt/intel/Compiler/11.0/074/mkl/lib/em64t/:/opt/intel/Compiler/11.0/074/lib/intel64 include_dirs = /opt/intel/Compiler/11.0/074/mkl/include/ mkl_libs = mkl_gf_lp64, mkl_gnu_thread, mkl_core, iomp5
Unfortunately, distutils complains and says that it cannot find the complete set of libraries:
mkl_info: libraries mkl_gf_lp64,mkl_gnu_thread,mkl_core,iomp5 not found in /opt/intel/Compiler/11.0/074/mkl/lib/em64t/ libraries mkl_gf_lp64,mkl_gnu_thread,mkl_core,iomp5 not found in /opt/intel/Compiler/11.0/074/lib/intel64 NOT AVAILABLE
After some debugging of the problem, it seems that distutils needs to find *all* the required libraries in *one* single directory.
Yes
My question is, is there an elegant way to handle this problem, or it is a limitation of the current distutils?
The "elegant" solution is to make softlink on unix.
If the later, it would be nice it that could be solved in a future version, and several libraries can be found in *several* directories.
Unfortunately, this would mean rewriting system_info, because this assumption is deeply ingrained in the whole design. Personally, I think the really idea of looking for files for libraries is not the right one - all other tools I know (autoconf, scons, jam, cmake) link code snippet instead. But doing it in distutils won't be fun.
David