[Numpy-discussion] The --compilers flag is ignored for build_ext

Andres Guzman-Ballen agballen189 at aol.com
Thu Nov 17 11:49:29 EST 2016

Hello Numpy community!

First time poster here. I was recommended by Charles Harris to present a question I have to you guys regarding using --compiler=intelem being ignored when I use build_ext.

I am trying to compile Numpy v1.11.2 that I am getting from https://pypi.python.org/packages/16/f5/b432f028134dd30cfbf6f21b8264a9938e5e0f75204e72453af08d67eb0b/numpy-1.11.2.tar.gz, using Python 2.7.12 on my Ubuntu 10.04 LTS Machine. I specifically want to compile this with the Intel compiler. I have sourced the compilervars.sh so the Intel compiler is in the path. This is the command that is failing:

'python setup.py build_ext --inplace --debug --force --compiler=intelem --fcompiler=intelem'. 

The reason it fails is because I have added compiler flags to umath's configuration extension code in numpy/core/setup.py as extra_compiler_arg and they are specifically meant for Intel. Since the compiler setup is ignored (unlike in setup.py build, where config_cc is correctly called and takes care of this), it defaults to GCC. This is a snippet of what you get for 'build_ext':
running build_clib
running build_src
building py_modules sources
building library "npymath" sources
creating build
creating build/src.linux-x86_64-2.7
customize Gnu95FCompiler
Found executable /usr/bin/gfortran
customize Gnu95FCompiler
customize Gnu95FCompiler using config

This is what you get for using 'build':
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
building py_modules sources
creating build
creating build/src.linux-x86_64-2.7
creating build/src.linux-x86_64-2.7/numpy
creating build/src.linux-x86_64-2.7/numpy/distutils
building library "npymath" sources
Found executable /localdisk/psxe_16/compilers_and_libraries_2016.3.210/linux/bin/intel64/icc
Could not locate executable ecc
customize IntelEM64TFCompiler
Found executable /localdisk/psxe_16/compilers_and_libraries_2016.3.210/linux/bin/intel64/ifort
customize IntelEM64TFCompiler using config

I do not experience this problem with 'python setup.py build --debug --force --compiler=intelem --fcompiler=intelem'.  because the configuration function is correctly called here: https://github.intel.com/SAT/numpy/blob/develop/numpy/distutils/command/build.py#L11 (that is where I got the output from above). However, I do not see anything like this in build_ext: https://github.intel.com/SAT/numpy/blob/develop/numpy/distutils/command/build_ext.py#L34. This of course could be because build_ext is doing something else below in the run command on line 109.

This is most likely not the right answer, but what harm does this do?

diff --git a/numpy/distutils/command/build_ext.py b/numpy/distutils/command/build_ext.py
index 0fa52a2..ffa88ae 100644
--- a/numpy/distutils/command/build_ext.py
+++ b/numpy/distutils/command/build_ext.py
@@ -79,6 +79,8 @@ class build_ext (old_build_ext):

         # Make sure that extension sources are complete.
+        self.run_command('config_cc')
+        self.run_command('config_fc')

         if self.distribution.has_c_libraries():

I am able to get build_ext to behave properly by adding this snippet, but I'm pretty sure this is the wrong approach because there seems to be code below line 109 in numpy/distutils/command/build_ext.py that sets compiler-related things up.

This is the ticket I created under Numpy's issues page: https://github.com/numpy/numpy/issues/8283. Any feedback is very well appreciated and I'd be more than happy to help contribute and get this fixed for the community. I don't want to just throw a problem at you guys and expect someone else to fix it for me. Thanks!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20161117/6f36745f/attachment.html>

More information about the NumPy-Discussion mailing list