Hi, I need to reinstall numpy because the cluster I am using was recently overhauled. I am wondering if numpy works with Python 2.7 now. Also, I would like numpy to run as fast as possible. The last time I did this, I was advised to install ATLAS by hand, as the one that comes with RHEL is not suitable. The first time I tried this, I kept running into problems that I think were due to mismatched fortran compilers. Is there a good resource for how to do this? I am fairly new to Linux. Thanks, Jonathan Tu
Hello Jonathan, yes, numpy work fine under Python 2.7 now. I don't see why building numpy against the system ATLAS should not work, as long as you install the developer version with the header files, and make sure that you edit the site.cfg file correct. - Ilan On Tue, Jan 18, 2011 at 10:39 AM, Jonathan Tu <jhtu@princeton.edu> wrote:
Hi,
I need to reinstall numpy because the cluster I am using was recently overhauled. I am wondering if numpy works with Python 2.7 now.
Also, I would like numpy to run as fast as possible. The last time I did this, I was advised to install ATLAS by hand, as the one that comes with RHEL is not suitable. The first time I tried this, I kept running into problems that I think were due to mismatched fortran compilers. Is there a good resource for how to do this? I am fairly new to Linux.
Thanks,
Jonathan Tu _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi, I realized that my cluster has MKL installed. I've been trying to install against MKL, but am having trouble getting this to work. After it finishes, I do import numpy numpy.show_config() and nothing about the MKL libraries shows up. I have edited site.cfg to read like this: [mkl] library_dirs = /opt/intel/mkl/10.2.4.032/lib/em64t include_dirs = /opt/intel/mkl/10.2.4.032/include lapack_libs = mkl_lapack mkl_libs = mkl, guide My cluster is using Intel Xeon processors, and I edited cc_exe as follows cc_exe = 'icc -O2 -fPIC' Then I did python setup.py config --compiler=intel build_clib --compiler=intel build_ext --compiler=intel install --prefix=/home/jhtu/local Jonathan Tu On Jan 18, 2011, at 3:28 PM, Ilan Schnell wrote:
Hello Jonathan,
yes, numpy work fine under Python 2.7 now. I don't see why building numpy against the system ATLAS should not work, as long as you install the developer version with the header files, and make sure that you edit the site.cfg file correct.
- Ilan
On Tue, Jan 18, 2011 at 10:39 AM, Jonathan Tu <jhtu@princeton.edu> wrote:
Hi,
I need to reinstall numpy because the cluster I am using was recently overhauled. I am wondering if numpy works with Python 2.7 now.
Also, I would like numpy to run as fast as possible. The last time I did this, I was advised to install ATLAS by hand, as the one that comes with RHEL is not suitable. The first time I tried this, I kept running into problems that I think were due to mismatched fortran compilers. Is there a good resource for how to do this? I am fairly new to Linux.
Thanks,
Jonathan Tu _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
The MKL configuration looks right, except that I had to use: mkl_libs = mkl_intel_lp64, mkl_intel_thread, mkl_core, iomp5 During the build process, it should tell you what it is linking aginast. Look at the compiler options passed to icc. - Ilan On Tue, Jan 18, 2011 at 2:31 PM, Jonathan Tu <jhtu@princeton.edu> wrote:
Hi,
I realized that my cluster has MKL installed. I've been trying to install against MKL, but am having trouble getting this to work. After it finishes, I do
import numpy numpy.show_config()
and nothing about the MKL libraries shows up. I have edited site.cfg to read like this:
[mkl] library_dirs = /opt/intel/mkl/10.2.4.032/lib/em64t include_dirs = /opt/intel/mkl/10.2.4.032/include lapack_libs = mkl_lapack mkl_libs = mkl, guide
My cluster is using Intel Xeon processors, and I edited cc_exe as follows
cc_exe = 'icc -O2 -fPIC'
Then I did
python setup.py config --compiler=intel build_clib --compiler=intel build_ext --compiler=intel install --prefix=/home/jhtu/local
Jonathan Tu
On Jan 18, 2011, at 3:28 PM, Ilan Schnell wrote:
Hello Jonathan,
yes, numpy work fine under Python 2.7 now. I don't see why building numpy against the system ATLAS should not work, as long as you install the developer version with the header files, and make sure that you edit the site.cfg file correct.
- Ilan
On Tue, Jan 18, 2011 at 10:39 AM, Jonathan Tu <jhtu@princeton.edu> wrote:
Hi,
I need to reinstall numpy because the cluster I am using was recently overhauled. I am wondering if numpy works with Python 2.7 now.
Also, I would like numpy to run as fast as possible. The last time I did this, I was advised to install ATLAS by hand, as the one that comes with RHEL is not suitable. The first time I tried this, I kept running into problems that I think were due to mismatched fortran compilers. Is there a good resource for how to do this? I am fairly new to Linux.
Thanks,
Jonathan Tu _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi, I have installed numpy but the unit tests fail. When I ran them, I got Traceback (most recent call last): File "/home/jhtu/local/lib/python2.7/site-packages/numpy/testing/ decorators.py", line 215, in knownfailer return f(*args, **kwargs) File "/home/jhtu/local/lib/python2.7/site-packages/numpy/core/tests/ test_umath_complex.py", line 312, in test_special_values assert_almost_equal(np.log(np.conj(xa[i])), np.conj(np.log(xa[i]))) File "/home/jhtu/local/lib/python2.7/site-packages/numpy/testing/ utils.py", line 443, in assert_almost_equal raise AssertionError(msg) AssertionError: Arrays are not almost equal ACTUAL: array([-inf+3.14159265j]) DESIRED: array([-inf-3.14159265j]) This was with numpy built against MKL. To install I modified site.cfg to read [mkl] library_dirs = /opt/intel/mkl/10.2.4.032/lib/em64t include_dirs = /opt/intel/mkl/10.2.4.032/include lapack_libs = mkl_lapack mkl_libs = mkl_intel_lp64, mkl_intel_thread, mkl_core My cluster is using Intel Xeon processors, and I edited cc_exe as follows cc_exe = 'icc -O2 -fPIC' I installed using python setup.py config --compiler=intel build_clib --compiler=intel build_ext --compiler=intel install --prefix=/home/jhtu/local Jonathan Tu On Jan 18, 2011, at 3:39 PM, Ilan Schnell wrote:
The MKL configuration looks right, except that I had to use: mkl_libs = mkl_intel_lp64, mkl_intel_thread, mkl_core, iomp5
During the build process, it should tell you what it is linking aginast. Look at the compiler options passed to icc.
- Ilan
On Tue, Jan 18, 2011 at 2:31 PM, Jonathan Tu <jhtu@princeton.edu> wrote:
Hi,
I realized that my cluster has MKL installed. I've been trying to install against MKL, but am having trouble getting this to work. After it finishes, I do
import numpy numpy.show_config()
and nothing about the MKL libraries shows up. I have edited site.cfg to read like this:
[mkl] library_dirs = /opt/intel/mkl/10.2.4.032/lib/em64t include_dirs = /opt/intel/mkl/10.2.4.032/include lapack_libs = mkl_lapack mkl_libs = mkl, guide
My cluster is using Intel Xeon processors, and I edited cc_exe as follows
cc_exe = 'icc -O2 -fPIC'
Then I did
python setup.py config --compiler=intel build_clib --compiler=intel build_ext --compiler=intel install --prefix=/home/jhtu/local
Jonathan Tu
On Jan 18, 2011, at 3:28 PM, Ilan Schnell wrote:
Hello Jonathan,
yes, numpy work fine under Python 2.7 now. I don't see why building numpy against the system ATLAS should not work, as long as you install the developer version with the header files, and make sure that you edit the site.cfg file correct.
- Ilan
On Tue, Jan 18, 2011 at 10:39 AM, Jonathan Tu <jhtu@princeton.edu> wrote:
Hi,
I need to reinstall numpy because the cluster I am using was recently overhauled. I am wondering if numpy works with Python 2.7 now.
Also, I would like numpy to run as fast as possible. The last time I did this, I was advised to install ATLAS by hand, as the one that comes with RHEL is not suitable. The first time I tried this, I kept running into problems that I think were due to mismatched fortran compilers. Is there a good resource for how to do this? I am fairly new to Linux.
Thanks,
Jonathan Tu _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi, I am trying to install numpy with the MKL libraries available on my cluster. Most of the libraries are available in one directory, but the iomp5 library is in another. /opt/intel/Compiler/11.1/072/mkl/lib/em64t ---> mkl_intel_lp64, mkl_intel_thread, mkl_core, mkl_def, mkl_mc /opt/intel/Compiler/11.1/072/lib/intel64 ---> iomp5 Using an older MKL library that was available, I found that when all libraries are in one directory, the install went through fine. But in this case it says the libraries cannot be found, even if I list both under the library_dirs in site.cfg [mkl] library_dirs = /opt/intel/Compiler/11.1/072/mkl/lib/em64t:/opt/intel/Compiler/11.1/072/lib/intel64 include_dirs = /opt/intel/Compiler/11.1/072/mkl/include lapack_libs = mkl_lapack mkl_libs = mkl_intel_lp64, mkl_intel_thread, mkl_core, mkl_def, mkl_mc, iomp5 If I try to install without iomp5, then when I import numpy I get the following error /opt/intel/Compiler/11.1/072/mkl/lib/em64t/libmkl_intel_thread.so: undefined symbol: omp_in_parallel Any ideas? I tried to put symbolic links to both library directories in one place, but that didn't work either. I'm trying to avoid creating a directory of symbolic links to every necessary library. Jonathan Tu
Hey Jonathan, Not sure if it will help, but here is how I got my MKL install working: cd ~/install virtualenv numpy_mkl cd numpy_mkl source bin/activate wget http://downloads.sourceforge.net/project/numpy/NumPy/1.5.1/numpy-1.5.1.tar.g... tar xzvf numpy-1.5.1.tar.gz cd numpy-1.5.1 cat << EOF > site.cfg [DEFAULT] library_dirs = /usr/lib include_dirs = /usr/include# numpy's configuration on resonance [DEFAULT] library_dirs = /usr/lib include_dirs = /usr/include [fftw] libraries = fftw3 [mkl] library_dirs = /opt/intel/Compiler/11.1/073/mkl/lib/em64t:/opt/intel/Compiler/11.1/073/lib/intel64 include_dirs = /opt/intel/Compiler/11.1/073/mkl/include:/opt/intel/Compiler/11.1/073/include lapack_libs = mkl_lapack mkl_libs = mkl_gf_lp64, mkl_gnu_thread, mkl_core, iomp5, guide, irc, mkl_mc, mkl_def EOF wget http://projects.scipy.org/numpy/raw-attachment/ticket/993/system_info.patch patch numpy/distutils/system_info.py < system_info.patch cat > intelccompiler.patch << EOF --- numpy/distutils/intelccompiler.py 2010-11-08 17:58:24.000000000 -0600 +++ numpy/distutils/intelccompiler.py.patched 2011-02-13 02:09:24.247386568 -0600 @@ -8,7 +8,7 @@ """ compiler_type = 'intel' - cc_exe = 'icc' + cc_exe = 'icc -fPIC -O3 -mmse4 -fast' def __init__ (self, verbose=0, dry_run=0, force=0): UnixCCompiler.__init__ (self, verbose,dry_run, force) EOF patch -p0 < intelccompiler.patch python setup.py config python setup.py build_ext --compiler=intel --fcompiler=intel install --prefix=~/local HTH Best, N On Wed, Jan 19, 2011 at 5:24 PM, Jonathan Tu <jhtu@princeton.edu> wrote:
Hi,
I am trying to install numpy with the MKL libraries available on my cluster. Most of the libraries are available in one directory, but the iomp5 library is in another.
/opt/intel/Compiler/11.1/072/mkl/lib/em64t ---> mkl_intel_lp64, mkl_intel_thread, mkl_core, mkl_def, mkl_mc /opt/intel/Compiler/11.1/072/lib/intel64 ---> iomp5
Using an older MKL library that was available, I found that when all libraries are in one directory, the install went through fine. But in this case it says the libraries cannot be found, even if I list both under the library_dirs in site.cfg
[mkl] library_dirs = /opt/intel/Compiler/11.1/072/mkl/lib/em64t:/opt/intel/Compiler/11.1/072/lib/intel64 include_dirs = /opt/intel/Compiler/11.1/072/mkl/include lapack_libs = mkl_lapack mkl_libs = mkl_intel_lp64, mkl_intel_thread, mkl_core, mkl_def, mkl_mc, iomp5
If I try to install without iomp5, then when I import numpy I get the following error
/opt/intel/Compiler/11.1/072/mkl/lib/em64t/libmkl_intel_thread.so: undefined symbol: omp_in_parallel
Any ideas? I tried to put symbolic links to both library directories in one place, but that didn't work either. I'm trying to avoid creating a directory of symbolic links to every necessary library.
Jonathan Tu _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Nicolas Pinto http://web.mit.edu/pinto
Hi, I installed numpy with MKL and found that the unit tests fail. In particular, I get the error message FAIL: test_special_values (test_umath_complex.TestClog) It says that this is a "known failure," specifically KNOWNFAIL=4. Is this ok? I saw from Googling that "The test failure indicates that your platform has a non-C99 compliant implementation of clog. Not fatal, but the test should be marked as a known failure on the platform." I'm not sure what this means. Would it be safer for my work to use a package w/o MKL that passes the tests? I'm currently benchmarking to see what the slowdown would really be. Jonathan Tu
On Wed, Jan 19, 2011 at 4:29 PM, Jonathan Tu <jhtu@princeton.edu> wrote:
Hi,
I installed numpy with MKL and found that the unit tests fail. In particular, I get the error message
FAIL: test_special_values (test_umath_complex.TestClog)
It says that this is a "known failure," specifically KNOWNFAIL=4. Is this ok? I saw from Googling that "The test failure indicates that your platform has a non-C99 compliant implementation of clog. Not fatal, but the test should be marked as a known failure on the platform."
I'm not sure what this means. Would it be safer for my work to use a package w/o MKL that passes the tests? I'm currently benchmarking to see what the slowdown would really be.
Don't worry about it, the test that failed is a corner case. Few, if any, libraries are fully c99 compliant for corner cases. Chuck
participants (5)
-
Charles R Harris -
Ilan Schnell -
Jonathan Tu -
Jonathan Tu
-
Nicolas Pinto