Correct way to query NumPy for linktime BLAS and LAPACK

Dear NumPy Developers, In the Clawpack/* repositories [1], we use a mixture of Fortran and Python source, currently glued together using f2py. Occasionally, we'll need to link the Fortran code directly against LAPACK. In particular, we're using dgeev and dgesv to solve several different Riemann problems [2,3]. In the past, we've relied on either the operating system or the user to provide these link commands for us, but it would be ideal in the future if we could query NumPy for how it is linked. Currently, the only information I can find is in the hidden __config__ module of NumPy's distutils module: numpy.distutils.__config__.blas_opt_info['extra_link_args'] numpy.distutils.__config__.lapack_opt_info['extra_link_args'] This seems to suggest that we shouldn't be relying on this information being available in future versions of NumPy (or at least, not in this location). That said, we'd still probably like to use this to avoid the possibility of multiple BLAS/LAPACK libraries being linked in to our builds. Any comments? Thanks, Aron [1] https://github.com/clawpack/clawpack [2] https://github.com/clawpack/riemann/blob/master/src/rp1_layered_shallow_wate... [3] https://github.com/clawpack/riemann/blob/master/src/rpn2_layered_shallow_wat...

Hi, In Theano, we use the information in this dictionnary: numpy.distutils.__config__.blas_opt_info. We do this for a few years already, so I don't know how much future proof it is, but I would expect that they aren't going to change this shortly. We use this dict for the default configuration, but still we allow the user to provide its own library and it work well. In case you don't know Theano, it is a compiler that generate dynamically c code, compile them as python module and load them in the python interpreter. So it happen that numpy and Theano module use different version of BLAS. Up to now, I never heard a problem about this. Don't forget that many different BLAS version use different internal symbol for the BLAS function and just provide an official function for the interface. So if we mix different BLAS, it work. But I'm not sure if what will happen if we link with different version of the same BLAS project, like different MKL version. Maybe just on the them will get imported if the library name is the same. HTH Fred On Mon, Aug 5, 2013 at 5:06 PM, Aron Ahmadia <aron@ahmadia.net> wrote:
Dear NumPy Developers,
In the Clawpack/* repositories [1], we use a mixture of Fortran and Python source, currently glued together using f2py.
Occasionally, we'll need to link the Fortran code directly against LAPACK. In particular, we're using dgeev and dgesv to solve several different Riemann problems [2,3].
In the past, we've relied on either the operating system or the user to provide these link commands for us, but it would be ideal in the future if we could query NumPy for how it is linked. Currently, the only information I can find is in the hidden __config__ module of NumPy's distutils module:
numpy.distutils.__config__.blas_opt_info['extra_link_args'] numpy.distutils.__config__.lapack_opt_info['extra_link_args']
This seems to suggest that we shouldn't be relying on this information being available in future versions of NumPy (or at least, not in this location). That said, we'd still probably like to use this to avoid the possibility of multiple BLAS/LAPACK libraries being linked in to our builds.
Any comments?
Thanks, Aron
[1] https://github.com/clawpack/clawpack [2] https://github.com/clawpack/riemann/blob/master/src/rp1_layered_shallow_wate... [3] https://github.com/clawpack/riemann/blob/master/src/rpn2_layered_shallow_wat...
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Hi Fred, Thanks for the feedback. We'll try this out in Clawpack moving forward. Regards, Aron On Tue, Aug 6, 2013 at 7:57 AM, Frédéric Bastien <nouiz@nouiz.org> wrote:
Hi,
In Theano, we use the information in this dictionnary: numpy.distutils.__config__.blas_opt_info. We do this for a few years already, so I don't know how much future proof it is, but I would expect that they aren't going to change this shortly.
We use this dict for the default configuration, but still we allow the user to provide its own library and it work well. In case you don't know Theano, it is a compiler that generate dynamically c code, compile them as python module and load them in the python interpreter. So it happen that numpy and Theano module use different version of BLAS. Up to now, I never heard a problem about this.
Don't forget that many different BLAS version use different internal symbol for the BLAS function and just provide an official function for the interface. So if we mix different BLAS, it work. But I'm not sure if what will happen if we link with different version of the same BLAS project, like different MKL version. Maybe just on the them will get imported if the library name is the same.
HTH
Fred
On Mon, Aug 5, 2013 at 5:06 PM, Aron Ahmadia <aron@ahmadia.net> wrote:
Dear NumPy Developers,
In the Clawpack/* repositories [1], we use a mixture of Fortran and Python source, currently glued together using f2py.
Occasionally, we'll need to link the Fortran code directly against LAPACK. In particular, we're using dgeev and dgesv to solve several different Riemann problems [2,3].
In the past, we've relied on either the operating system or the user to provide these link commands for us, but it would be ideal in the future if we could query NumPy for how it is linked. Currently, the only information I can find is in the hidden __config__ module of NumPy's distutils module:
numpy.distutils.__config__.blas_opt_info['extra_link_args'] numpy.distutils.__config__.lapack_opt_info['extra_link_args']
This seems to suggest that we shouldn't be relying on this information being available in future versions of NumPy (or at least, not in this location). That said, we'd still probably like to use this to avoid the possibility of multiple BLAS/LAPACK libraries being linked in to our builds.
Any comments?
Thanks, Aron
[1] https://github.com/clawpack/clawpack [2] https://github.com/clawpack/riemann/blob/master/src/rp1_layered_shallow_wate... [3] https://github.com/clawpack/riemann/blob/master/src/rpn2_layered_shallow_wat...
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (2)
-
Aron Ahmadia
-
Frédéric Bastien