poatajoseph@gmail.com wrote:

After looking through old threads I came across [1], which discusses the effect LAPACK-level parallelization has on preformance. For the user that started that thread, they saw improvements by changing their environmental variables. This I understand, what is confusing me is how you figure out which library a Kwant distribution is linked to.

E.g. without setting these independently and observing what effect each has on performance, how can I tell which of the following environmental variables need to be changed? OPENBLAS_NUM_THREADS OMP_NUM_THREADS MKL_DYNAMIC MKL_NUM_THREADS

Short answer: Experiment! For OpenBLAS the environment variable OPENBLAS_NUM_THREADS is specific to OpenBLAS, while OMP_NUM_THREADS applies to everything that uses OpenMP (including OpenBLAS). Use top/htop or some similar program to monitor CPU usage as you run your computation. You want to use all cores, but avoid oversubscription. Slight oversubscription is OK, but running 16*16 threads on 16 cores will not work well. Long answer: Kwant potentially uses two different LAPACKs: • The module kwant.linalg.lapack exposes the LAPACK that is used by SciPy. This LAPACK is used whenever Kwant itself needs to call some LAPACK functionality, for example when computing modes. • The module kwant.linalg.mumps exposes MUMPS (a library for sparse linear algebra). MUMPS requires LAPACK, and so Kwant’s mumps module must be linked to some (potentially different) LAPACK. To check which LAPACK is used by Kwant with MUMPS, you can use a tool like ldd (it exists on Linux, no idea about MacOS). For example: $ cd $(python3 -c 'import kwant; print(kwant.linalg.__file__)' | xargs dirname) $ ldd *.so lapack.cpython-311-x86_64-linux-gnu.so: linux-vdso.so.1 (0x00007ffc0866e000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f045035e000) /lib64/ld-linux-x86-64.so.2 (0x00007f0450657000) _mumps.cpython-311-x86_64-linux-gnu.so: linux-vdso.so.1 (0x00007fff1a57a000) libzmumps_scotch-5.5.so => /lib/x86_64-linux-gnu/libzmumps_scotch-5.5.so (0x00007fb2d3400000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb2d321f000) liblapack.so.3 => /lib/x86_64-linux-gnu/liblapack.so.3 (0x00007fb2d2a00000) libmumps_common_scotch-5.5.so => /lib/x86_64-linux-gnu/libmumps_common_scotch-5.5.so (0x00007fb2d371f000) libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007fb2d2600000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fb2d3140000) /lib64/ld-linux-x86-64.so.2 (0x00007fb2d37d9000) libopenblas.so.0 => /lib/x86_64-linux-gnu/libopenblas.so.0 (0x00007fb2d0318000) libesmumps-7.0.so => /lib/x86_64-linux-gnu/libesmumps-7.0.so (0x00007fb2d3714000) libscotch-7.0.so => /lib/x86_64-linux-gnu/libscotch-7.0.so (0x00007fb2d3683000) libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007fb2d29b9000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fb2d3661000) libscotcherr-7.0.so => /lib/x86_64-linux-gnu/libscotcherr-7.0.so (0x00007fb2d365c000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fb2d3121000) libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007fb2d310e000) liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fb2d298a000) We can see that my Kwant/MUMPS is linked with OpenBLAS. One can do the same exercise to find out the LAPACK with which SciPy has been linked: $ cd $(python3 -c 'import scipy; print(scipy.linalg.lapack.__file__)' | xargs dirname) $ ldd *.so Cheers Christoph