[Numpy-discussion] NumPy distutils limitation?

[David Cournapeau]

On Sat, Jan 31, 2009 at 5:30 AM, Francesc Alted <faltet at pytables.org> wrote:
> Hi,
>
> Gregor and I are trying to give support for Intel's VML (Vector
> Mathematical Library) in numexpr.  For this, we are trying to make use
> of the weaponery in NumPy's distutils so as to be able to discover
> where the MKL (the package that contains VML) is located.  The
> libraries that we need to link with are: mkl_gf_lp64, mkl_gnu_thread,
> mkl_core *and* iomp5 (Intel's OpenMP library).
>
> The problem is that I have installed MKL as part as the Intel compiler
> for Unix.  In this setup, most of the libraries are in one place,
> namely:
>
> /opt/intel/Compiler/11.0/074/mkl/lib/em64t/
>
> However, the OpenMP library is in another directory:
>
> /opt/intel/Compiler/11.0/074/lib/intel64
>
> So, I need to specify *two* directories to get the complete set of
> libraries.  My first attempt was setting a site.cfg like:
>
> [DEFAULT]
> #libraries = gfortran
>
> [mkl]
> library_dirs= /opt/intel/Compiler/11.0/074/mkl/lib/em64t/:/opt/intel/Compiler/11.0/074/lib/intel64
> include_dirs =  /opt/intel/Compiler/11.0/074/mkl/include/
> mkl_libs = mkl_gf_lp64, mkl_gnu_thread, mkl_core, iomp5
>
>
> Unfortunately, distutils complains and says that it cannot find the
> complete set of libraries:
>
> mkl_info:
>  libraries mkl_gf_lp64,mkl_gnu_thread,mkl_core,iomp5 not found
> in /opt/intel/Compiler/11.0/074/mkl/lib/em64t/
>  libraries mkl_gf_lp64,mkl_gnu_thread,mkl_core,iomp5 not found
> in /opt/intel/Compiler/11.0/074/lib/intel64
>  NOT AVAILABLE
>
> After some debugging of the problem, it seems that distutils needs to
> find *all* the required libraries in *one* single directory.

Yes

>
> My question is, is there an elegant way to handle this problem, or it
> is a limitation of the current distutils?

The "elegant" solution is to make softlink on unix.

> If the later, it would be
> nice it that could be solved in a future version, and several libraries
> can be found in *several* directories.

Unfortunately, this would mean rewriting system_info, because this
assumption is deeply ingrained in the whole design. Personally, I
think the really idea of looking for files for libraries is not the
right one - all other tools I know (autoconf, scons, jam, cmake) link
code snippet instead. But doing it in distutils won't be fun.

David