Intel MKL on Core2 system
Hi, I noticed the following in numpy/distutils/system_info.py while trying to get numpy to build against MKL: if cpu.is_Itanium(): plt = '64' #l = 'mkl_ipf' elif cpu.is_Xeon(): plt = 'em64t' #l = 'mkl_em64t' else: plt = '32' #l = 'mkl_ia32' So in the autodetection for MKL, the only way to get plt (platform) set to 'em64t' is to test true for a Xeon. This function returns false on my Core2 Duo system, even though the platform is very much 'em64t'. I think that check should instead read: elif cpu.is_Xeon() or cpu.is_Core2(): Thoughts? Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma Sent from: Norman Oklahoma United States.
A Wednesday 11 March 2009, Ryan May escrigué:
Hi,
I noticed the following in numpy/distutils/system_info.py while trying to get numpy to build against MKL:
if cpu.is_Itanium(): plt = '64' #l = 'mkl_ipf' elif cpu.is_Xeon(): plt = 'em64t' #l = 'mkl_em64t' else: plt = '32' #l = 'mkl_ia32'
So in the autodetection for MKL, the only way to get plt (platform) set to 'em64t' is to test true for a Xeon. This function returns false on my Core2 Duo system, even though the platform is very much 'em64t'. I think that check should instead read:
elif cpu.is_Xeon() or cpu.is_Core2():
Thoughts?
This may help you to see the developer's view on this subject: http://projects.scipy.org/numpy/ticket/994 Cheers, -- Francesc Alted
On Wed, Mar 11, 2009 at 1:34 PM, Francesc Alted <faltet@pytables.org> wrote:
A Wednesday 11 March 2009, Ryan May escrigué:
Hi,
I noticed the following in numpy/distutils/system_info.py while trying to get numpy to build against MKL:
if cpu.is_Itanium(): plt = '64' #l = 'mkl_ipf' elif cpu.is_Xeon(): plt = 'em64t' #l = 'mkl_em64t' else: plt = '32' #l = 'mkl_ia32'
So in the autodetection for MKL, the only way to get plt (platform) set to 'em64t' is to test true for a Xeon. This function returns false on my Core2 Duo system, even though the platform is very much 'em64t'. I think that check should instead read:
elif cpu.is_Xeon() or cpu.is_Core2():
Thoughts?
This may help you to see the developer's view on this subject:
http://projects.scipy.org/numpy/ticket/994
Cheers,
-- Francesc Alted _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
You know, I knew this sounded familiar. If you regularly build against MKL, can you send me your site.cfg. I've had a lot more success getting the build to work using the autodetection than the blas_opt and lapack_opt sections. Since the autodetection doesn't seem like the accepted way, I'd love to see how to get the accepted way to actually work. :) Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma Sent from: Norman Oklahoma United States.
A Wednesday 11 March 2009, Ryan May escrigué:
You know, I knew this sounded familiar. If you regularly build against MKL, can you send me your site.cfg. I've had a lot more success getting the build to work using the autodetection than the blas_opt and lapack_opt sections. Since the autodetection doesn't seem like the accepted way, I'd love to see how to get the accepted way to actually work. :)
Not that I'm an expert in that sort of black magic, but the next worked fine for me and numexpr: [mkl] # Example for using MKL 10.0 #library_dirs = /opt/intel/mkl/10.0.2.018/lib/em64t #include_dirs = /opt/intel/mkl/10.0.2.018/include # Example for the MKL included in Intel C 11.0 compiler library_dirs = /opt/intel/Compiler/11.0/074/mkl/lib/em64t/ include_dirs = /opt/intel/Compiler/11.0/074/mkl/include/ ##the following set of libraries is suited for compilation ##with the GNU C compiler (gcc). Refer to the MKL documentation ##if you use other compilers (e.g., Intel C compiler) mkl_libs = mkl_gf_lp64, mkl_gnu_thread, mkl_core HTH, -- Francesc Alted
On Wed, Mar 11, 2009 at 2:20 PM, Francesc Alted <faltet@pytables.org> wrote:
A Wednesday 11 March 2009, Ryan May escrigué:
You know, I knew this sounded familiar. If you regularly build against MKL, can you send me your site.cfg. I've had a lot more success getting the build to work using the autodetection than the blas_opt and lapack_opt sections. Since the autodetection doesn't seem like the accepted way, I'd love to see how to get the accepted way to actually work. :)
Not that I'm an expert in that sort of black magic, but the next worked fine for me and numexpr:
[mkl]
# Example for using MKL 10.0 #library_dirs = /opt/intel/mkl/10.0.2.018/lib/em64t #include_dirs <http://10.0.2.018/lib/em64t%0A#include_dirs> = /opt/intel/mkl/10.0.2.018/include
# Example for the MKL included in Intel C 11.0 compiler library_dirs = /opt/intel/Compiler/11.0/074/mkl/lib/em64t/ include_dirs = /opt/intel/Compiler/11.0/074/mkl/include/
##the following set of libraries is suited for compilation ##with the GNU C compiler (gcc). Refer to the MKL documentation ##if you use other compilers (e.g., Intel C compiler) mkl_libs = mkl_gf_lp64, mkl_gnu_thread, mkl_core
Thanks. That's actually pretty close to what I had. I was actually thinking that you were using only blas_opt and lapack_opt, since supposedly the [mkl] style section is deprecated. Thus far, I cannot get these to work with MKL. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma Sent from: Norman Oklahoma United States.
A Wednesday 11 March 2009, Ryan May escrigué:
Thanks. That's actually pretty close to what I had. I was actually thinking that you were using only blas_opt and lapack_opt, since supposedly the [mkl] style section is deprecated. Thus far, I cannot get these to work with MKL.
Well, my configuration was thought to link with the VML integrated in the MKL, but I'd say that it would be similar for blas and lapack. What's you configuration? What's the error you are running into? Cheers, -- Francesc Alted
On Thu, Mar 12, 2009 at 3:05 AM, Francesc Alted <faltet@pytables.org> wrote:
A Wednesday 11 March 2009, Ryan May escrigué:
Thanks. That's actually pretty close to what I had. I was actually thinking that you were using only blas_opt and lapack_opt, since supposedly the [mkl] style section is deprecated. Thus far, I cannot get these to work with MKL.
Well, my configuration was thought to link with the VML integrated in the MKL, but I'd say that it would be similar for blas and lapack. What's you configuration? What's the error you are running into?
I can get it working now with either the [mkl] section like your config or the following config: [DEFAULT] include_dirs = /opt/intel/mkl/10.0.2.018/include/ library_dirs = /opt/intel/mkl/10.0.2.018/lib/em64t/:/usr/lib [blas] libraries = mkl_gf_lp64, mkl_gnu_thread, mkl_core, iomp5 [lapack] libraries = mkl_lapack, mkl_gf_lp64, mkl_gnu_thread, mkl_core, iomp5 It's just confusing I guess because if I change blas and lapack to blas_opt and lapack_opt, I cannot get it to work. The only reason I even care is that site.cfg.example leads me to believe that the *_opt sections are the way you're supposed to add them. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma
Ryan May wrote:
[DEFAULT] include_dirs = /opt/intel/mkl/10.0.2.018/include/ <http://10.0.2.018/include/> library_dirs = /opt/intel/mkl/10.0.2.018/lib/em64t/:/usr/lib <http://10.0.2.018/lib/em64t/:/usr/lib>
[blas] libraries = mkl_gf_lp64, mkl_gnu_thread, mkl_core, iomp5
[lapack] libraries = mkl_lapack, mkl_gf_lp64, mkl_gnu_thread, mkl_core, iomp5
It's just confusing I guess because if I change blas and lapack to blas_opt and lapack_opt, I cannot get it to work.
Yes, the whole thing is very confusing; trying to understand it when I try to be compatible with it in numscons drove me crazy (the changes with default section handling in python 2.6 did no help). IMHO, we should get rid of all this at some point, and use something much simpler (one file, no sections, just straight LIBPATH + LIBS + CPPATH options), because the current code has gone much beyond the madness point. But it will break some configurations for sure. cheers, David
On Thu, Mar 12, 2009 at 8:30 AM, David Cournapeau < david@ar.media.kyoto-u.ac.jp> wrote:
Ryan May wrote:
[DEFAULT] include_dirs = /opt/intel/mkl/10.0.2.018/include/ <http://10.0.2.018/include/> library_dirs = /opt/intel/mkl/10.0.2.018/lib/em64t/:/usr/lib <http://10.0.2.018/lib/em64t/:/usr/lib>
[blas] libraries = mkl_gf_lp64, mkl_gnu_thread, mkl_core, iomp5
[lapack] libraries = mkl_lapack, mkl_gf_lp64, mkl_gnu_thread, mkl_core, iomp5
It's just confusing I guess because if I change blas and lapack to blas_opt and lapack_opt, I cannot get it to work.
Yes, the whole thing is very confusing; trying to understand it when I try to be compatible with it in numscons drove me crazy (the changes with default section handling in python 2.6 did no help). IMHO, we should get rid of all this at some point, and use something much simpler (one file, no sections, just straight LIBPATH + LIBS + CPPATH options), because the current code has gone much beyond the madness point. But it will break some configurations for sure.
Glad to hear it's not just me. I was beginning to think I was being thick headed.... Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma Sent from: Norman Oklahoma United States.
A Thursday 12 March 2009, Ryan May escrigué:
I can get it working now with either the [mkl] section like your config or the following config:
[DEFAULT] include_dirs = /opt/intel/mkl/10.0.2.018/include/ library_dirs = /opt/intel/mkl/10.0.2.018/lib/em64t/:/usr/lib ^^^^^^^^^ I see that you are using a multi-directory path here. My understanding was that this is not supported by numpy.distutils, but apparently it worked for you (?), or if you get rid of the ':/usr/lib' trailing part of library_dirs it works ok too?
-- Francesc Alted
On Thu, Mar 12, 2009 at 10:11 AM, Francesc Alted <faltet@pytables.org>wrote:
A Thursday 12 March 2009, Ryan May escrigué:
I can get it working now with either the [mkl] section like your config or the following config:
[DEFAULT] include_dirs = /opt/intel/mkl/10.0.2.018/include/ library_dirs = /opt/intel/mkl/10.0.2.018/lib/em64t/:/usr/lib ^^^^^^^^^ I see that you are using a multi-directory path here. My understanding was that this is not supported by numpy.distutils, but apparently it worked for you (?), or if you get rid of the ':/usr/lib' trailing part of library_dirs it works ok too?
Well, if by multi-directory you mean the colon-separated list, this is what is documented in site.cfg.example and used by the gentoo ebuild on my system. I need the /usr/lib part so that it can pick up libblas.so and liblapack.so. Otherwise, it won't link in MKL. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma Sent from: Norman Oklahoma United States.
On Thu, Mar 12, 2009 at 3:15 AM, Ryan May <rmay31@gmail.com> wrote:
Hi,
I noticed the following in numpy/distutils/system_info.py while trying to get numpy to build against MKL:
if cpu.is_Itanium(): plt = '64' #l = 'mkl_ipf' elif cpu.is_Xeon(): plt = 'em64t' #l = 'mkl_em64t' else: plt = '32' #l = 'mkl_ia32'
So in the autodetection for MKL, the only way to get plt (platform) set to 'em64t' is to test true for a Xeon. This function returns false on my Core2 Duo system, even though the platform is very much 'em64t'. I think that check should instead read:
elif cpu.is_Xeon() or cpu.is_Core2():
Thoughts?
I think this whole code is inherently fragile. A much better solution is to make the build process customization easier and more straightforward. Auto-detection will never work well. David
On Wed, Mar 11, 2009 at 1:41 PM, David Cournapeau <cournape@gmail.com>wrote:
On Thu, Mar 12, 2009 at 3:15 AM, Ryan May <rmay31@gmail.com> wrote:
Hi,
I noticed the following in numpy/distutils/system_info.py while trying to get numpy to build against MKL:
if cpu.is_Itanium(): plt = '64' #l = 'mkl_ipf' elif cpu.is_Xeon(): plt = 'em64t' #l = 'mkl_em64t' else: plt = '32' #l = 'mkl_ia32'
So in the autodetection for MKL, the only way to get plt (platform) set to 'em64t' is to test true for a Xeon. This function returns false on my Core2 Duo system, even though the platform is very much 'em64t'. I think that check should instead read:
elif cpu.is_Xeon() or cpu.is_Core2():
Thoughts?
I think this whole code is inherently fragile. A much better solution is to make the build process customization easier and more straightforward. Auto-detection will never work well.
David _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Fair enough. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma
participants (4)
-
David Cournapeau -
David Cournapeau -
Francesc Alted -
Ryan May