On Mon, Nov 27, 2023 at 6:51 PM Dr. Thomas Orgis <thomas.orgis@uni-hamburg.de> wrote:
Am Mon, 27 Nov 2023 14:58:45 +0100
schrieb Ralf Gommers <ralf.gommers@gmail.com>:

> The NumPy build does not know anything about this. It will just build, and
> it will simply call the OpenBLAS functionality


Great!

> Yes, that is possible. You should be building with a build frontend (pip or
> pypa/build) and then the invocation will include `-C-Dblas=<your .pc name>
> -C-Dlapack=<your .pc name>`.

I'm confused about these frontends, I must say. I imagined that if
you're using meson, one could just call meson setup/build? That being
said: I am not sure now myself how the pkgsrc build actually works
right now. There's common machinery to 'build python stuff' and the
part about meson-based packages is rather fresh and not documented yet.

You have to go through a "build frontend" to produce a wheel, which then gets installed/repackaged for your distro. If you call meson/ninja directly, you will not get the Python package metadata that meson-python produces. And you do need that, others there are some things that will break (e.g., using `importlib` APIs for introspecting Python packages). So what your machinery should be doing is building with `pip install . --no-build-isolation` or `python -m build --no-isolation`.
 

The build output starts with

* Building wheel...
+ /data/pkg/bin/python3.11 /data/projekte/pkgsrc/work/math/py-numpy/work/numpy-1.26.2/vendored-meson/meson/meson.py setup /data/projekte/pkgsrc/work/math/py-numpy/work/numpy-1.26.2 /data/projekte/pkgsrc/work/math/py-numpy/work/numpy-1.26.2/.mesonpy-_lv

… so some wrapped call to a vendored copy of meson that NumPy ships.

Yes, we need that for extra BLAS/LAPACK and SIMD related functionality that is still in the process of being upstreamed into Meson.
 
Adding -Dblas=$CBLAS_PC to that command should do the trick, no?
(however that is effected)

Sounds like it, assuming CBLAS_PC is the name of a library.
 

> > And: NumPy needs CBLAS … does it also need LAPACKE instead of LAPACK?
> > 
>
> No need for LAPACKE.

Good, if also somewhat weird;-) I'm curious, though: When you need the
CBLAS API, why is the dependency called blas and not cblas? In
practice, most accelerated libraries offer all APIs in one binary and
-Dlapack is already redundant, but when we use the netlib reference,
blas, cblas, lapack, and lapacke are distinct entities. Calling cblas
just blas where lapack _does_ mean the Fortran one, is rather confusing.

Partly a matter of history since we always did it like this, but I think there's more to it. The two libraries are called BLAS and LAPACK, those offer distinct functionality. CBLAS and LAPACKE are basically much less important implementation details, and typically shipped in the same library because they're interfaces to the exact same functionality. We're not "calling CBLAS just BLAS" here, but rather: BLAS is the main name and has the functionality you want. CBLAS is an optional interface to that, and if you want it you have to ask for it with (in Meson):

    dependency('blas', modules: ['cblas'])

It doesn't make much sense for us to expose CBLAS (or LAPACKE) as a separate thing in our own build interface.


> You have to opt in to ILP64, via a `-Duse-ilp64` flag. It will not work to
> craft a blas.pc which points at a 64-bit BLAS.

So -Dblas=openblas64 -Dlapack=openblas64 -Duse-ilp64 would do it, right?

Exactly.

 


Alrighty then,

Thomas

PS: You might want to fix that one:

../../numpy/meson.build:124: WARNING: Project targets '>=1.2.99' but uses feature introduced in '1.3.0': dep 'blas' custom lookup.

Yeah, that'll go away when we update the vendored copy, will be done in the next few days.

Cheers,
Ralf


--
Dr. Thomas Orgis
HPC @ Universität Hamburg
_______________________________________________
NumPy-Discussion mailing list -- numpy-discussion@python.org
To unsubscribe send an email to numpy-discussion-leave@python.org
https://mail.python.org/mailman3/lists/numpy-discussion.python.org/
Member address: ralf.gommers@gmail.com