site.cfg: Custom BLAS / LAPACK configuration
I recently installed NumPy 1.6.2 on a new computer and wanted to use ACML as the BLAS/LAPACK library. [I'm aware that ACML doesn't provide CBLAS, but that is easy to work around by compiling it yourself to produce libcblas.a or libcblas.so].
I experienced a great bit of difficulty in getting NumPy to use ACML (-lcblas -lacml), primarily stemming from the fact that there was a working ATLAS installation already in /usr/lib64.
As far as I can tell, it's IMPOSSIBLE to create a site.cfg which will link to ACML when a system installed ATLAS is present.
The detection routine for blas_opt (and similarly for lapack_opt) seem to operate as: * Is MKL present? If so, use it. * Is ATLAS present? If so, use it. * Use [blas] section from site.cfg.
Instead I would have expected the detection routine to be more like: * Is [blas_opt] present in site.cfg? If so, use it. * Is MKL present? ... * Is ATLAS present? ... * Use [blas] section from site.cfg.
This is not just a problem with ACML. I've also experienced this when using NumPy on some cray supercomputers where the default C compiler automatically links a preferred BLAS/LAPACK.
I created a GitHub issue for this: https://github.com/numpy/numpy/issues/2728. In addition, I created a pull request with a "works for me" solution, but which should have needs some wider visibility https://github.com/numpy/numpy/pull/2751.
I'd appreciate any reviews, workarounds, or other general feedback. If you want to test our the library detection mechanism you can run the following from within the NumPy source directory::
import __builtin__ as builtins builtins.__NUMPY_SETUP__ = True import numpy.distutils.system_info as si print si.get_info('blas_opt')
Thanks, Brad
On Fri, 30 Nov 2012 12:13:58 -0800 "Bradley M. Froehle" brad.froehle@gmail.com wrote:
As far as I can tell, it's IMPOSSIBLE to create a site.cfg which will link to ACML when a system installed ATLAS is present.
setup.py respects environment variables. You can set ATLAS to None and force the setup to use $LAPACK and $BLAS. See also this link:
http://www.der-schnorz.de/2012/06/optimized-linear-algebra-and-numpyscipy/
Greetings
Alex
Thanks Alexander, that was quite helpful, but unfortunately does not actually work. The recommendations there are akin to a site.cfg file:
[atlas] atlas_libs = library_dirs =
[blas] blas_libs = cblas,acml library_dirs = /opt/acml5.2.0/gfortan64_fma4/lib
[lapack] blas_libs = cblas,acml library_dirs = /opt/acml5.2.0/gfortan64_fma4/lib $ python setup.py build
However this makes numpy think that there is no optimized blas available and prevents the numpy.core._dotblas module from being built.
-Brad
On Thu, Dec 6, 2012 at 4:29 AM, Alexander Eberspächer < alex.eberspaecher@gmail.com> wrote:
On Fri, 30 Nov 2012 12:13:58 -0800 "Bradley M. Froehle" brad.froehle@gmail.com wrote:
As far as I can tell, it's IMPOSSIBLE to create a site.cfg which will link to ACML when a system installed ATLAS is present.
setup.py respects environment variables. You can set ATLAS to None and force the setup to use $LAPACK and $BLAS. See also this link:
http://www.der-schnorz.de/2012/06/optimized-linear-algebra-and-numpyscipy/
Greetings
Alex _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Thu, Dec 6, 2012 at 7:13 PM, Bradley M. Froehle brad.froehle@gmail.com wrote:
Thanks Alexander, that was quite helpful, but unfortunately does not actually work. The recommendations there are akin to a site.cfg file:
[atlas] atlas_libs = library_dirs =
[blas] blas_libs = cblas,acml library_dirs = /opt/acml5.2.0/gfortan64_fma4/lib
[lapack] blas_libs = cblas,acml library_dirs = /opt/acml5.2.0/gfortan64_fma4/lib $ python setup.py build
However this makes numpy think that there is no optimized blas available and prevents the numpy.core._dotblas module from being built.
_dotblas is only built if *C*blas is available (atlas, accelerate and mkl only are supported ATM).
David
Right, but if I link to libcblas, cblas would be available, no?
On Thu, Dec 6, 2012 at 10:34 AM, David Cournapeau cournape@gmail.comwrote:
On Thu, Dec 6, 2012 at 7:13 PM, Bradley M. Froehle brad.froehle@gmail.com wrote:
Thanks Alexander, that was quite helpful, but unfortunately does not actually work. The recommendations there are akin to a site.cfg file:
[atlas] atlas_libs = library_dirs =
[blas] blas_libs = cblas,acml library_dirs = /opt/acml5.2.0/gfortan64_fma4/lib
[lapack] blas_libs = cblas,acml library_dirs = /opt/acml5.2.0/gfortan64_fma4/lib $ python setup.py build
However this makes numpy think that there is no optimized blas available
and
prevents the numpy.core._dotblas module from being built.
_dotblas is only built if *C*blas is available (atlas, accelerate and mkl only are supported ATM).
David _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Thu, Dec 6, 2012 at 7:35 PM, Bradley M. Froehle brad.froehle@gmail.com wrote:
Right, but if I link to libcblas, cblas would be available, no?
No, because we don't explicitly check for CBLAS. We assume it is there if Atlas, Accelerate or MKL is found.
cheers, David
On Thu, Dec 6, 2012 at 10:34 AM, David Cournapeau cournape@gmail.com wrote:
On Thu, Dec 6, 2012 at 7:13 PM, Bradley M. Froehle brad.froehle@gmail.com wrote:
Thanks Alexander, that was quite helpful, but unfortunately does not actually work. The recommendations there are akin to a site.cfg file:
[atlas] atlas_libs = library_dirs =
[blas] blas_libs = cblas,acml library_dirs = /opt/acml5.2.0/gfortan64_fma4/lib
[lapack] blas_libs = cblas,acml library_dirs = /opt/acml5.2.0/gfortan64_fma4/lib $ python setup.py build
However this makes numpy think that there is no optimized blas available and prevents the numpy.core._dotblas module from being built.
_dotblas is only built if *C*blas is available (atlas, accelerate and mkl only are supported ATM).
David _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Aha, thanks for the clarification. I've always been surpassed that NumPy doesn't ship with a copy of CBLAS. It's easy to compile --- just a thin wrapper over BLAS, if I remember correctly.
-Brad
On Friday, December 7, 2012 at 4:01 AM, David Cournapeau wrote:
On Thu, Dec 6, 2012 at 7:35 PM, Bradley M. Froehle <brad.froehle@gmail.com (mailto:brad.froehle@gmail.com)> wrote:
Right, but if I link to libcblas, cblas would be available, no?
No, because we don't explicitly check for CBLAS. We assume it is there if Atlas, Accelerate or MKL is found.
cheers, David
One way of fixing this I'm sort of itching to do is to create a "pylapack" project which can iterate quickly on these build issues, run-time selection of LAPACK backend and so on. (With some templates generating some Cython code it shouldn't be more than a few days for an MVP.)
Then patch NumPy to attempt to import pylapack (and if it's there, get a table of function pointers from it).
The idea would be that powerusers could more easily build pylapack the way they wanted, in isolation, and then have other things switch to that without a rebuild (note again that it would need to export a table of function pointers).
But I'm itching to do too many things, we'll see.
Dag Sverre
On 12/07/2012 07:09 PM, Bradley M. Froehle wrote:
Aha, thanks for the clarification. I've always been surpassed that NumPy doesn't ship with a copy of CBLAS. It's easy to compile --- just a thin wrapper over BLAS, if I remember correctly.
-Brad
On Friday, December 7, 2012 at 4:01 AM, David Cournapeau wrote:
On Thu, Dec 6, 2012 at 7:35 PM, Bradley M. Froehle <brad.froehle@gmail.com (mailto:brad.froehle@gmail.com)> wrote:
Right, but if I link to libcblas, cblas would be available, no?
No, because we don't explicitly check for CBLAS. We assume it is there if Atlas, Accelerate or MKL is found.
cheers, David
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 12/07/2012 07:58 PM, Dag Sverre Seljebotn wrote:
One way of fixing this I'm sort of itching to do is to create a "pylapack" project which can iterate quickly on these build issues, run-time selection of LAPACK backend and so on. (With some templates generating some Cython code it shouldn't be more than a few days for an MVP.)
Then patch NumPy to attempt to import pylapack (and if it's there, get a table of function pointers from it).
The idea would be that powerusers could more easily build pylapack the way they wanted, in isolation, and then have other things switch to that without a rebuild (note again that it would need to export a table of function pointers).
But I'm itching to do too many things, we'll see.
Update: This can be tackled as part of Hashdist funding, so I'm hoping that I can 3-4 days on improving the LAPACK situation in January; as an option for power-users at first, hopefully something nice for everybody eventually. Opinions on the plan above welcome.
Dag Sverre
Dag Sverre
On 12/07/2012 07:09 PM, Bradley M. Froehle wrote:
Aha, thanks for the clarification. I've always been surpassed that NumPy doesn't ship with a copy of CBLAS. It's easy to compile --- just a thin wrapper over BLAS, if I remember correctly.
-Brad
On Friday, December 7, 2012 at 4:01 AM, David Cournapeau wrote:
On Thu, Dec 6, 2012 at 7:35 PM, Bradley M. Froehle <brad.froehle@gmail.com (mailto:brad.froehle@gmail.com)> wrote:
Right, but if I link to libcblas, cblas would be available, no?
No, because we don't explicitly check for CBLAS. We assume it is there if Atlas, Accelerate or MKL is found.
cheers, David
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Fri, Dec 7, 2012 at 7:58 PM, Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
One way of fixing this I'm sort of itching to do is to create a "pylapack" project which can iterate quickly on these build issues, run-time selection of LAPACK backend and so on. (With some templates generating some Cython code it shouldn't be more than a few days for an MVP.)
Then patch NumPy to attempt to import pylapack (and if it's there, get a table of function pointers from it).
The idea would be that powerusers could more easily build pylapack the way they wanted, in isolation, and then have other things switch to that without a rebuild (note again that it would need to export a table of function pointers).
But I'm itching to do too many things, we'll see.
It would be hard to support in a cross platform way I think (windows being the elephant in the room). I would be more than happy to be proven wrong, though :)
cheers, David
On 12/13/2012 07:59 PM, David Cournapeau wrote:
On Fri, Dec 7, 2012 at 7:58 PM, Dag Sverre Seljebotn d.s.seljebotn@astro.uio.no wrote:
One way of fixing this I'm sort of itching to do is to create a "pylapack" project which can iterate quickly on these build issues, run-time selection of LAPACK backend and so on. (With some templates generating some Cython code it shouldn't be more than a few days for an MVP.)
Then patch NumPy to attempt to import pylapack (and if it's there, get a table of function pointers from it).
The idea would be that powerusers could more easily build pylapack the way they wanted, in isolation, and then have other things switch to that without a rebuild (note again that it would need to export a table of function pointers).
But I'm itching to do too many things, we'll see.
It would be hard to support in a cross platform way I think (windows being the elephant in the room). I would be more than happy to be proven wrong, though :)
Right, perhaps I should have said "Linux users" rather than "power users" :-)
But seriously, I'd like to have a few more details here.
Is it something about calling conventions and "libc" and so on that makes the function-pointer-table approach difficult? (But the NumPy C API is exported in the same way?)
Or is it about the build? Can't one just lift whatever numpy.distutils or the NumPy Bento build does for detection? Though what I'll aim for at first is the complete opposite, no auto-detection, but a Windows user should also be able to specify the right flags manually...
Dag Sverre
participants (4)
-
Alexander Eberspächer -
Bradley M. Froehle -
Dag Sverre Seljebotn -
David Cournapeau