Probs with numpy/scipy on Itanium 64Bit with Intel Compilers with SCSL libs.
Hello, We are working hard to compile scipy on our box. We have already sent a post on this list but so far we haven't managed to install scipy yet, see : http://projects.scipy.org/pipermail/scipy-dev/2007-November/007998.html here is the problem : we are trying to compile numpy/scipy with intel compilers on a box of itanium processors with the SCSL library. the problem is that both numpy and scipy seem to loose track of the fortran compiler configuration during their build_ext phase even though this configuration is recognized during the build_clib phase, giving this log message. customize IntelItaniumFCompiler warning: build_ext: f77_compiler=intele is not available. the prob is the same when python has been compiled with gcc or with intel c compiler. the prob doesn't seems to be related to scsl lib at all but only to the handling of intel compiler for Itanium . some help would be much appreciated. In particular, is there a Intel/Itanium implementation of scipy/numpy somewhere that we could get some information about ? thanks for your help -- Julien Le Sommer, LEGI/MEOM, BP53, 38041 Grenoble cedex 9, France Tel : +33 4 76 82 50 65 Fax : +33 4 76 82 52 71 http://www.legi.hmg.inpg.fr/~lesommer/<http://www.legi.hmg.inpg.fr/%7Elesommer/>
Julien Le Sommer wrote:
Hello,
We are working hard to compile scipy on our box. We have already sent a post on this list but so far we haven't managed to install scipy yet, see : http://projects.scipy.org/pipermail/scipy-dev/2007-November/007998.html <http://projects.scipy.org/pipermail/scipy-dev/2007-November/007998.html>
here is the problem : we are trying to compile numpy/scipy with intel compilers on a box of itanium processors with the SCSL library.
the problem is that both numpy and scipy seem to loose track of the fortran compiler configuration during their build_ext phase even though this configuration is recognized during the build_clib phase, giving this log message.
customize IntelItaniumFCompiler
warning: build_ext: f77_compiler=intele is not available.
the prob is the same when python has been compiled with gcc or with intel c compiler.
the prob doesn't seems to be related to scsl lib at all but only to the handling of intel compiler for Itanium .
some help would be much appreciated. In particular, is there a Intel/Itanium implementation of scipy/numpy somewhere that we could get some information about ?
If you could give one of us a shell access to a computer where everything is installed to build numpy/scipy on an itanium, it would be much easier to solve the problem, cheers, David
Hello, David Cournapeau a écrit
If you could give one of us a shell access to a computer where everything is installed to build numpy/scipy on an itanium, it would be much easier to solve the problem,
I give you, one access to our computer. I am not in my office these two following days but you can contact me since monday. If you have a system problem, you can send a mail to Bruno.Bzeznik@imag.fr login: cournapd pass: IfTSMHfo 1. We have two versions of python installed to test scipy installation: * *one compiled with gcc* (usr/local/stow/Python2.4.4) * t*he other with the intel compiler* "icc" (usr/local/stow/Python2.4.4_INTEL) You have acces on these two directories. 2. I put in your *.bashrc the environment to use the Intel compilers * 3. The *scsl library *(scientific library optimized on our cluster with intel compilers) is in */usr/lib (compiled with Intel Fortran compiler) *(On Itanium, the install procedure of ATLAS want to use Intel compilers, we can forced it to use Gnu compilers but we obtain bad optimizations)* * 4. I attached on my mail the file *'site.cfg*' used to install scipy and numpy 4. I attached on my mail a file named *"whatWeHaveDone.txt" *in which I give you again what we have done to install numpy and scipy. remarks: It seem that it's a bug in the part of program which extract the intel version compiler. *An additional test is made in numpy/distutils/command/build_ext.py to test this version *(ifort -V), there is no version test in numpy/distutils/command/build_clib.py. I tried to change this test in /usr/local/stow/Python-2.4.4/lib/python2.4/site-packages/numpy/distutils/fcompiler/intel.py...but it doesn't run. thanks for your help Laurence Viry Julien Le Sommer wrote:
Hello,
We are working hard to compile scipy on our box. We have already sent a post on this list but so far we haven't managed to install scipy yet, see : http://projects.scipy.org/pipermail/scipy-dev/2007-November/007998.html <http://projects.scipy.org/pipermail/scipy-dev/2007-November/007998.html>
here is the problem : we are trying to compile numpy/scipy with intel compilers on a box of itanium processors with the SCSL library.
the problem is that both numpy and scipy seem to loose track of the fortran compiler configuration during their build_ext phase even though this configuration is recognized during the build_clib phase, giving this log message.
customize IntelItaniumFCompiler
warning: build_ext: f77_compiler=intele is not available.
the prob is the same when python has been compiled with gcc or with intel c compiler.
the prob doesn't seems to be related to scsl lib at all but only to the handling of intel compiler for Itanium .
some help would be much appreciated. In particular, is there a Intel/Itanium implementation of scipy/numpy somewhere that we could get some information about ?
cheers,
David _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
-- Laurence Viry CRIP UJF - Projet MIRAGE ( http://mirage.imag.fr ) Laboratoire de Modélisation et Calcul - IMAG tel: 04 76 63 56 34 fax: 04 76 63 12 63 e-mail: Laurence.Viry@imag.fr What we have done: ----------------- I try to install numpy 1.0.4 and scipy 0.6.0 with INTEL C/C++ and Fortran compilers and SCSL scientific library with python2.4.4 on Itanium with linux OS. (Altix SGI with SUSE linux). SCSL is the SGI scientific library optimized on our cluster, it is compiled with INTEL Fortran Compiler. It is highly recommended that all libraries that scipy uses (e.g. blas and atlas libraries) are built with the same Fortran compiler. It's the reason for installing scipy with INTEL Fortran Compiler ---------------------------------- I didn't success to configure the construct of build_ext for scipy with the fortran INTEL compiler (--fcompiler=intele). We have the same problem for numpy for the fortran configuration , we have the warning message (It's ok for build_clib): "warning: build_ext: f77_compiler=intele is not available". * First, I set LD_LIBRARY_PATH and LD_RUN_PATH with directories which contains c/c++, fortran and SCSL libraries export LD_LIBRARY_PATH=/opt/intel/intel_cc/9.0.024/lib:/opt/intel/intel_fc/9.0.021/lib:/usr/lib Export LD_RUN_PATH=/opt/intel/intel_cc/9.0.024/lib:/opt/intel/intel_fc/9.0.021/lib:/usr/lib * I put in the file "site.cfg" : [DEFAULT] library_dirs = /usr/lib:/usr/local/lib/:/usr/local/stow/fftw-3.1.2/lib include_dirs = /usr/include:/usr/local/include:/usr/local/stow/fftw-3.1.2/include [blas] library_dirs = /usr/lib include_dirs=/usr/include blas_libs= scs [lapack] library_dirs = /usr/lib include_dirs=/usr/include lapack_libs = scs [fftw] libraries = fftw3 * Compile and install numpy with: python setup.py config --compiler=intel build_clib --compiler=intel build_ext --compiler=intel install It's Ok for numpy, it doesn'tt need fortran compiler but for scipy python setup.py -v config_cc --compiler=intele config_fc --fcompiler=intele build_clib --compiler=intele --fcompiler=intele build_ext --compiler=intele --fcompiler=intele install It's Ok for build_clib, the python script find c and fortran compiler and scsl library. it doesn't find fortran compiler for build_ext, first I had a warning warning: build_ext: f77_compiler=intele is not available, it was available for the construct of build_clib? ???? then warning: build_ext: extension 'scipy.fftpack._fftpack' has Fortran libraries but no Fortran linker found, using default linker for the construct of build_ext, the python script use the default fortran compiler g77 and the construct stoped with the message error: "extension 'scipy.interpolate.dfitpack' has Fortran sources but no Fortran compiler found" why --fcompiler=intele option of build_ext doesn't operate????????? I show this option of build_ext command with the python command Remarks: * I verify which Fortran compiler to use by using the following install command:: python setup.py config_fc --fcompiler=<Vendor> install To see a valid list of <Vendor> names: python setup.py config_fc --help-fcompiler # This file provides configuration information about non-Python dependencies for # numpy.distutils-using packages. Create a file like this called "site.cfg" next # to your package's setup.py file and fill in the appropriate sections. Not all # packages will use all sections so you should leave out sections that your # package does not use. # To assist automatic installation like easy_install, the user's home directory # will also be checked for the file ~/.numpy-site.cfg . # The format of the file is that of the standard library's ConfigParser module. # # http://www.python.org/doc/current/lib/module-ConfigParser.html # # Each section defines settings that apply to one particular dependency. Some of # the settings are general and apply to nearly any section and are defined here. # Settings specific to a particular section will be defined near their section. # # libraries # Comma-separated list of library names to add to compile the extension # with. Note that these should be just the names, not the filenames. For # example, the file "libfoo.so" would become simply "foo". # libraries = lapack,f77blas,cblas,atlas # # library_dirs # List of directories to add to the library search path when compiling # extensions with this dependency. Use the character given by os.pathsep # to separate the items in the list. On UN*X-type systems (Linux, FreeBSD, # OS X): # library_dirs = /usr/lib:/usr/local/lib # On Windows: # library_dirs = c:\mingw\lib,c:\atlas\lib # # include_dirs # List of directories to add to the header file earch path. # include_dirs = /usr/include:/usr/local/include # # src_dirs # List of directories that contain extracted source code for the # dependency. For some dependencies, numpy.distutils will be able to build # them from source if binaries cannot be found. The FORTRAN BLAS and # LAPACK libraries are one example. However, most dependencies are more # complicated and require actual installation that you need to do # yourself. # src_dirs = /home/rkern/src/BLAS_SRC:/home/rkern/src/LAPACK_SRC # # search_static_first # Boolean (one of (0, false, no, off) for False or (1, true, yes, on) for # True) to tell numpy.distutils to prefer static libraries (.a) over # shared libraries (.so). It is turned off by default. # search_static_first = false # Defaults # ======== # The settings given here will apply to all other sections if not overridden. # This is a good place to add general library and include directories like # /usr/local/{lib,include} # [DEFAULT] library_dirs = /usr/lib:/usr/local/lib/:/usr/local/stow/fftw-3.1.2/lib include_dirs = /usr/include:/usr/local/include:/usr/local/stow/fftw-3.1.2/include # Optimized BLAS and LAPACK # ------------------------- # Use the blas_opt and lapack_opt sections to give any settings that are # required to link against your chosen BLAS and LAPACK, including the regular # FORTRAN reference BLAS and also ATLAS. Some other sections still exist for # linking against certain optimized libraries (e.g. [atlas], [lapack_atlas]), # however, they are now deprecated and should not be used. # # These are typical configurations for ATLAS (assuming that the library and # include directories have already been set in [DEFAULT]; the include directory # is important for the BLAS C interface): # # Utilise les blas et lapack issus d'ATLAS #[atlas] #library_dirs = /usr/lib #include_dirs=/usr/include #atlas_libs= scs [blas] library_dirs = /usr/lib include_dirs=/usr/include blas_libs= scs [lapack] library_dirs = /usr/lib include_dirs=/usr/include lapack_libs = scs [config_fc] fcompiler=ifort f77exec=ifort f90exec=ifort # UMFPACK # ------- # The UMFPACK library is used to factor large sparse matrices. It, in turn, # depends on the AMD library for reordering the matrices for better performance. # Note that the AMD library has nothing to do with AMD (Advanced Micro Devices), # the CPU company. # # http://www.cise.ufl.edu/research/sparse/umfpack/ # http://www.cise.ufl.edu/research/sparse/amd/ # #[amd] #amd_libs = amd # #[umfpack] #umfpack_libs = umfpack # FFT libraries # ------------- # There are two FFT libraries that we can configure here: FFTW (2 and 3) and djbfft. # # http://fftw.org/ # http://cr.yp.to/djbfft.html # # Given only this section, numpy.distutils will try to figure out which version # of FFTW you are using. [fftw] libraries = fftw3 # For djbfft, numpy.distutils will look for either djbfft.a or libdjbfft.a . #[djbfft] #include_dirs = /usr/local/stow/fftw-3.1.2/include #library_dirs = /usr/local/stow/fftw-3.1.2/lib
participants (3)
-
David Cournapeau -
Julien Le Sommer -
Laurence Viry