[Numpy-discussion] Need compilations with compilers other than gcc

[Arnd Baecker]

On Tue, 7 Feb 2006, Pearu Peterson wrote:


[... /numpy/distutils/exec_command.py ...]

>    from numpy.distutils.misc_util import is_sequence, is_string
>
> should be changed to
>
>    from misc_util import is_sequence, is_string
>
> to fix this.

Making the same type of change in
  numpy/distutils/system_info.py
worked if ATLAS is not used (`export ATLAS=None`).

Otherwise I get:
python numpy/distutils/system_info.py lapack_opt
lapack_opt_info:
lapack_mkl_info:
mkl_info:
  NOT AVAILABLE

  NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
system_info.atlas_threads_info
Setting PTATLAS=ATLAS
Setting PTATLAS=ATLAS
  FOUND:
    libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
    library_dirs = ['/home/baecker/python2/lib/atlas']
    language = f77
    include_dirs = ['/usr/include']

Traceback (most recent call last):
  File "numpy/distutils/system_info.py", line 1693, in ?
    show_all()
  File "numpy/distutils/system_info.py", line 1689, in show_all
    r = c.get_info()
  File
"/work/home/baecker/INSTALL_PYTHON5_icc/CompileDir/numpy/numpy/distutils/system_info.py",
line 338, in get_info
    self.calc_info()
  File
"/work/home/baecker/INSTALL_PYTHON5_icc/CompileDir/numpy/numpy/distutils/system_info.py",
line 1123, in calc_info
    atlas_version = get_atlas_version(**version_info)
  File
"/work/home/baecker/INSTALL_PYTHON5_icc/CompileDir/numpy/numpy/distutils/system_info.py",
line 1028, in get_atlas_version
    from core import Extension, setup
  File
"/work/home/baecker/INSTALL_PYTHON5_icc/CompileDir/numpy/numpy/distutils/core.py",
line 12, in ?
    from numpy.distutils.extension import Extension
ImportError: No module named numpy.distutils.extension

numpy/distutils/core.py is full of
`from numpy.distutils.command import ...`.



> >>> Concerning icc compilation I used:
> >>>
> >>>  export FC_VENDOR=Intel
> >>
> >> This has no effect anymore. Use --fcompiler=intel instead.
> >
> > OK - I have to confess that I am really confused about
> > which options might work and which not.
> > Is there a document which describes this?
>
> FC_VENDOR env. variable was used in old f2py long time ago. When Fortran
> compiler support was moved to scipy_distutils, --fcompiler option was
> introduced to config, config_fc, build_ext,.. setup.py commands.
> One should use any of these commands to specify a Fortran compiler and
> config_fc to change various Fortran compiler flags. See
>    python setup.py config_fc --help
> for more information.
>
> How to enhance C compiler options, see standard Distutils documentation.
>
> >>>  export F77=ifort
> >>>  export CC=icc
> >>>  export CXX=icc
> >
> > But these are still needed?
>
> No for F77, using --fcompiler=.. should be enough. I am not sure about CC,
> CXX, must try it out..
>
> >> When Python is compiled with a different compiler than numpy (or any
> >> extension module) is going to be installed then proper libraries must be
> >> specified manually. Which libraries and flags are needed exactly, this is
> >> described in compilers manual.
> >>
> >> So, a recommended fix would be to build Python with icc and as a
> >> result correct libraries will be used for building 3rd party extension
> >> modules.
> >
> > This would also mean that all dependent packages will have
> > to be installed again, right?
> > I am sorry but then I won't be able to help with icc at the moment
> > as I am completely swamped with other stuff...
> >
> >> Otherwise one has to read compilers manual, sections like
> >> about gcc-compatibility and  linking might be useful. See also
> >>   http://www.scipy.org/Wiki/FAQ#head-8371c35ef08b877875217aaac5489fc747b4aceb
> >
> > I thought that supplying ``--libraries="irc"``
> > might cure the problem, but
> > (quoting from
> > http://aspn.activestate.com/ASPN/Mail/Message/scipy-dev/2983903
> > )
> > """
> > However, in the build log I only found -lirc for
> > the config_tests but nowhere else.
> > What should I do instead of the above?
> > """
>
> Try:
>
>    export CC=icc
>    python setup.py build build_ext -lirc
		>
> This will probably use gcc for linking

Yes, it does use gcc for linking. I also had to specify
the location of `libirc`,

  export CC=icc
  python setup.py build build_ext -L/opt/intel/cc_90/lib/ -lirc
followed by
  python setup.py config --fcompiler=intel   install
worked.

On import I get another error

import core -> failed:
/home/baecker/python2/scipy_icc5_lintst_n_N3/lib/python2.4/site-packages/numpy/core/umath.so:
undefined symbol: __libm_sincos
import random -> failed: 'module' object has no attribute 'dtype'
import lib -> failed:
/home/baecker/python2/scipy_icc5_lintst_n_N3/lib/python2.4/site-packages/numpy/core/umath.so:
undefined symbol: __libm_sincos
import linalg -> failed: /opt/intel/fc_90/lib/libunwind.so.6: undefined
symbol: ?1__serial_memmove
import dft -> failed:
/home/baecker/python2/scipy_icc5_lintst_n_N3/lib/python2.4/site-packages/numpy/core/umath.so:
undefined symbol: __libm_sincos


So it seems I will have to specify more libraries,

Would this be the correct syntax:
  python setup.py build build_ext -L/opt/intel/cc_90/lib/:SomeOtherPath
-lirc:someotherlibrary
?

>From ``python setup.py build build_ext --help``

  --libraries (-l)     external C libraries to link with
  --library-dirs (-L)  directories to search for external C libraries
                       (separated by ':')

it is not clear how to specify several libraries with "-l"?

But that did not work (neither did -lirc -lm)

> but might fix undefined symbol problems.

Many thanks,

Arnd