On Mon, Nov 27, 2023 at 6:51 PM Dr. Thomas Orgis < thomas.orgis@uni-hamburg.de> wrote:
Am Mon, 27 Nov 2023 14:58:45 +0100 schrieb Ralf Gommers <ralf.gommers@gmail.com>:
The NumPy build does not know anything about this. It will just build, and it will simply call the OpenBLAS functionality
Great!
Yes, that is possible. You should be building with a build frontend (pip or pypa/build) and then the invocation will include `-C-Dblas=<your .pc name> -C-Dlapack=<your .pc name>`.
I'm confused about these frontends, I must say. I imagined that if you're using meson, one could just call meson setup/build? That being said: I am not sure now myself how the pkgsrc build actually works right now. There's common machinery to 'build python stuff' and the part about meson-based packages is rather fresh and not documented yet.
You have to go through a "build frontend" to produce a wheel, which then gets installed/repackaged for your distro. If you call meson/ninja directly, you will not get the Python package metadata that meson-python produces. And you do need that, others there are some things that will break (e.g., using `importlib` APIs for introspecting Python packages). So what your machinery should be doing is building with `pip install . --no-build-isolation` or `python -m build --no-isolation`.
The build output starts with
* Building wheel... + /data/pkg/bin/python3.11 /data/projekte/pkgsrc/work/math/py-numpy/work/numpy-1.26.2/vendored-meson/meson/meson.py setup /data/projekte/pkgsrc/work/math/py-numpy/work/numpy-1.26.2 /data/projekte/pkgsrc/work/math/py-numpy/work/numpy-1.26.2/.mesonpy-_lv
… so some wrapped call to a vendored copy of meson that NumPy ships.
Yes, we need that for extra BLAS/LAPACK and SIMD related functionality that is still in the process of being upstreamed into Meson.
Adding -Dblas=$CBLAS_PC to that command should do the trick, no? (however that is effected)
Sounds like it, assuming CBLAS_PC is the name of a library.
And: NumPy needs CBLAS … does it also need LAPACKE instead of LAPACK?
No need for LAPACKE.
Good, if also somewhat weird;-) I'm curious, though: When you need the CBLAS API, why is the dependency called blas and not cblas? In practice, most accelerated libraries offer all APIs in one binary and -Dlapack is already redundant, but when we use the netlib reference, blas, cblas, lapack, and lapacke are distinct entities. Calling cblas just blas where lapack _does_ mean the Fortran one, is rather confusing.
Partly a matter of history since we always did it like this, but I think there's more to it. The two libraries are called BLAS and LAPACK, those offer distinct functionality. CBLAS and LAPACKE are basically much less important implementation details, and typically shipped in the same library because they're interfaces to the exact same functionality. We're not "calling CBLAS just BLAS" here, but rather: BLAS is the main name and has the functionality you want. CBLAS is an optional interface to that, and if you want it you have to ask for it with (in Meson): dependency('blas', modules: ['cblas']) It doesn't make much sense for us to expose CBLAS (or LAPACKE) as a separate thing in our own build interface.
You have to opt in to ILP64, via a `-Duse-ilp64` flag. It will not work to craft a blas.pc which points at a 64-bit BLAS.
So -Dblas=openblas64 -Dlapack=openblas64 -Duse-ilp64 would do it, right?
Exactly.
Alrighty then,
Thomas
PS: You might want to fix that one:
../../numpy/meson.build:124: WARNING: Project targets '>=1.2.99' but uses feature introduced in '1.3.0': dep 'blas' custom lookup.
Yeah, that'll go away when we update the vendored copy, will be done in the next few days. Cheers, Ralf
-- Dr. Thomas Orgis HPC @ Universität Hamburg _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: ralf.gommers@gmail.com