![](https://secure.gravatar.com/avatar/1eb8b802c83c7b774c0481a9981ddf59.jpg?s=120&d=mm&r=g)
Hi, I am writing on behalf of my team from Endava company. Our task is to work with opensource community, and setup multiple applications for use on win11 arm64 machines. One of the tasks is to understand the problems and if possible help NumPy team in setting up win11 arm64 compatible wheel on the PyPi. I saw that there were some earlier discussions about this subject, and as I understood problem was that at that moment this configuration was not so common and there were no appropriate CI slaves to create the wheel. Today there are more devices in this configuration available on the market and additional are announced, so I wanted to check if there are plans for creating this wheel, and if we can somehow help with that work? Regards, Slobodan
![](https://secure.gravatar.com/avatar/9bf244c487ed59f016d93ccf42929914.jpg?s=120&d=mm&r=g)
On Thu, Jul 11, 2024, 22:08 slobodan.miletic--- via NumPy-Discussion < numpy-discussion@python.org> wrote:
This is discussed in detail in https://github.com/numpy/numpy/issues/22530. TLDR - we need CI providers to offer win arm before it's sensible to make wheels. Firstly because we need to make them (less secure to upload wheels not made on CI infra), secondly because we need to test them. If less tested wheels give rise to lots of issues reported by the community, then this generates large maintainer workload.
![](https://secure.gravatar.com/avatar/b4929294417e9ac44c17967baae75a36.jpg?s=120&d=mm&r=g)
Hi, On Thu, Jul 11, 2024 at 1:16 PM Andrew Nelson <andyfaff@gmail.com> wrote:
Would it make sense for a company to set up and maintain some self-hosted workflow runners? https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-ho... Cheers, Matthew
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Thu, Jul 11, 2024 at 2:20 PM Andrew Nelson <andyfaff@gmail.com> wrote:
Thanks for offering help Slobodan! Contributions here are welcome - and it's hopefully not too difficult if you have a dev setup for Windows on Arm locally.
A good start to get this rolling could be: 1. Address open issues for Windows on Arm. There's only one right now ( https://github.com/numpy/numpy/issues/25626) - we'd be happy to accept a PR for that, we don't need CI to validate it. 2. Setting up a single CI job that builds an arm64 wheel on x86-64 (build only, no test - unless emulation works). gh-22530 has comments saying it should work. Once those things are done, we can consider uploading wheels to https://anaconda.org/scientific-python-nightly-wheels/numpy/. Cheers, Ralf
![](https://secure.gravatar.com/avatar/1eb8b802c83c7b774c0481a9981ddf59.jpg?s=120&d=mm&r=g)
As the bug fix is merged we are starting the investigation on the CI job. I have a few questions about this: 1) Are there some additional instructions for making and running the numpy CI jobs and cross compilation available in the documentation? 2) Do we need to have arm64 scipy-openblas released to build numpy wheel?
![](https://secure.gravatar.com/avatar/72f994ca072df3a3d2c3db8a137790fd.jpg?s=120&d=mm&r=g)
On 12/08/2024 11:55, slobodan.miletic--- via NumPy-Discussion wrote:
1. You can look at .github/workflows/linux_qemu.yml that use qemu/docker to run. AFAIK there is no documentation about cross-compilation. 2. It would be preferable to build a scipy-openblas wheel but we could use a "vanilla" build of OpenBLAS as well. We do expect to use 64-bit interfaces, but scipy does not do so yet, so we would need two builds. I don't know how @gholke compiles OpenBLAS/NumPy and where there is a gfortran for windows arm64, maybe worthwhile asking? Matti
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Mon, Aug 12, 2024 at 12:23 PM Matti Picus <matti.picus@gmail.com> wrote:
There is: https://numpy.org/devdocs/building/cross_compilation.html https://cibuildwheel.pypa.io/en/stable/faq/#windows-arm64 says that cibuildwheel only supports `setuptools` and `setuptools-rust` so far, so that's not particularly helpful yet. Neither is the `linux_qemu.yml` example I think, that's the furthest away from what we need I think (unless I missed something and QEMU or Docker actually supports Windows on Arm). I'm not sure if this should go the `crossenv` way (see https://github.com/benfogle/crossenv), or a regular cross compile with a cross file and build/host envs. I think the latter won't quite work until PEP 739 (https://peps.python.org/pep-0739/) lands. 2. It would be preferable to build a scipy-openblas wheel but we could
I recommend getting step (1) to work first, without any BLAS library. That may be challenging enough - if there's a blocker, then there is no point looking into cross-compiling OpenBLAS before native arm64 CI runners become available. Cheers, Ralf
![](https://secure.gravatar.com/avatar/1eb8b802c83c7b774c0481a9981ddf59.jpg?s=120&d=mm&r=g)
Hi, I finished the first step and created the workflow for cross-compiling wheel without OpenBlas. PR for this can be found at: https://github.com/numpy/numpy/pull/27330 Output wheel is successfully installed and tested on the arm machine. After this gets merged I will continue with adding the steps for cross-compiling OpenBlas and linking the Numpy with it. Thanks, Slobodan
![](https://secure.gravatar.com/avatar/1eb8b802c83c7b774c0481a9981ddf59.jpg?s=120&d=mm&r=g)
I accidentally created new thread without subject with message that is belonging here: Hi, I made the second PR for full wheel with openblas. It is reviewed, but rejected as solution with linaro built openblas is not good enough, so I am investigating alternate options for cross compiling openblas. While I am working on this I have one question about the wheel without OpenBLAS. As the job is now created on the main it can be triggered manually and the produced wheel can be used for publishing to PyPi. I'm wondering what the next steps are with this build. Is it planned for the wheel to be built/published without OpenBLAS and what would be the conditions for that.
![](https://secure.gravatar.com/avatar/9bf244c487ed59f016d93ccf42929914.jpg?s=120&d=mm&r=g)
I don't think we would make an official release to PyPI without OpenBLAS functionality, and we would want any official wheel to be built on CI infrastructure that we can configure ourselves. I guess it might be possible to upload an 'unofficial' wheel to https://anaconda.org/scientific-python-nightly-wheels/numpy that doesn't have OpenBLAS. However, we would want to be building that via one of our CI workflows.
![](https://secure.gravatar.com/avatar/9bf244c487ed59f016d93ccf42929914.jpg?s=120&d=mm&r=g)
On Thu, Jul 11, 2024, 22:08 slobodan.miletic--- via NumPy-Discussion < numpy-discussion@python.org> wrote:
This is discussed in detail in https://github.com/numpy/numpy/issues/22530. TLDR - we need CI providers to offer win arm before it's sensible to make wheels. Firstly because we need to make them (less secure to upload wheels not made on CI infra), secondly because we need to test them. If less tested wheels give rise to lots of issues reported by the community, then this generates large maintainer workload.
![](https://secure.gravatar.com/avatar/b4929294417e9ac44c17967baae75a36.jpg?s=120&d=mm&r=g)
Hi, On Thu, Jul 11, 2024 at 1:16 PM Andrew Nelson <andyfaff@gmail.com> wrote:
Would it make sense for a company to set up and maintain some self-hosted workflow runners? https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-ho... Cheers, Matthew
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Thu, Jul 11, 2024 at 2:20 PM Andrew Nelson <andyfaff@gmail.com> wrote:
Thanks for offering help Slobodan! Contributions here are welcome - and it's hopefully not too difficult if you have a dev setup for Windows on Arm locally.
A good start to get this rolling could be: 1. Address open issues for Windows on Arm. There's only one right now ( https://github.com/numpy/numpy/issues/25626) - we'd be happy to accept a PR for that, we don't need CI to validate it. 2. Setting up a single CI job that builds an arm64 wheel on x86-64 (build only, no test - unless emulation works). gh-22530 has comments saying it should work. Once those things are done, we can consider uploading wheels to https://anaconda.org/scientific-python-nightly-wheels/numpy/. Cheers, Ralf
![](https://secure.gravatar.com/avatar/1eb8b802c83c7b774c0481a9981ddf59.jpg?s=120&d=mm&r=g)
As the bug fix is merged we are starting the investigation on the CI job. I have a few questions about this: 1) Are there some additional instructions for making and running the numpy CI jobs and cross compilation available in the documentation? 2) Do we need to have arm64 scipy-openblas released to build numpy wheel?
![](https://secure.gravatar.com/avatar/72f994ca072df3a3d2c3db8a137790fd.jpg?s=120&d=mm&r=g)
On 12/08/2024 11:55, slobodan.miletic--- via NumPy-Discussion wrote:
1. You can look at .github/workflows/linux_qemu.yml that use qemu/docker to run. AFAIK there is no documentation about cross-compilation. 2. It would be preferable to build a scipy-openblas wheel but we could use a "vanilla" build of OpenBLAS as well. We do expect to use 64-bit interfaces, but scipy does not do so yet, so we would need two builds. I don't know how @gholke compiles OpenBLAS/NumPy and where there is a gfortran for windows arm64, maybe worthwhile asking? Matti
![](https://secure.gravatar.com/avatar/5f88830d19f9c83e2ddfd913496c5025.jpg?s=120&d=mm&r=g)
On Mon, Aug 12, 2024 at 12:23 PM Matti Picus <matti.picus@gmail.com> wrote:
There is: https://numpy.org/devdocs/building/cross_compilation.html https://cibuildwheel.pypa.io/en/stable/faq/#windows-arm64 says that cibuildwheel only supports `setuptools` and `setuptools-rust` so far, so that's not particularly helpful yet. Neither is the `linux_qemu.yml` example I think, that's the furthest away from what we need I think (unless I missed something and QEMU or Docker actually supports Windows on Arm). I'm not sure if this should go the `crossenv` way (see https://github.com/benfogle/crossenv), or a regular cross compile with a cross file and build/host envs. I think the latter won't quite work until PEP 739 (https://peps.python.org/pep-0739/) lands. 2. It would be preferable to build a scipy-openblas wheel but we could
I recommend getting step (1) to work first, without any BLAS library. That may be challenging enough - if there's a blocker, then there is no point looking into cross-compiling OpenBLAS before native arm64 CI runners become available. Cheers, Ralf
![](https://secure.gravatar.com/avatar/1eb8b802c83c7b774c0481a9981ddf59.jpg?s=120&d=mm&r=g)
Hi, I finished the first step and created the workflow for cross-compiling wheel without OpenBlas. PR for this can be found at: https://github.com/numpy/numpy/pull/27330 Output wheel is successfully installed and tested on the arm machine. After this gets merged I will continue with adding the steps for cross-compiling OpenBlas and linking the Numpy with it. Thanks, Slobodan
![](https://secure.gravatar.com/avatar/1eb8b802c83c7b774c0481a9981ddf59.jpg?s=120&d=mm&r=g)
I accidentally created new thread without subject with message that is belonging here: Hi, I made the second PR for full wheel with openblas. It is reviewed, but rejected as solution with linaro built openblas is not good enough, so I am investigating alternate options for cross compiling openblas. While I am working on this I have one question about the wheel without OpenBLAS. As the job is now created on the main it can be triggered manually and the produced wheel can be used for publishing to PyPi. I'm wondering what the next steps are with this build. Is it planned for the wheel to be built/published without OpenBLAS and what would be the conditions for that.
![](https://secure.gravatar.com/avatar/9bf244c487ed59f016d93ccf42929914.jpg?s=120&d=mm&r=g)
I don't think we would make an official release to PyPI without OpenBLAS functionality, and we would want any official wheel to be built on CI infrastructure that we can configure ourselves. I guess it might be possible to upload an 'unofficial' wheel to https://anaconda.org/scientific-python-nightly-wheels/numpy that doesn't have OpenBLAS. However, we would want to be building that via one of our CI workflows.
participants (5)
-
Andrew Nelson
-
Matthew Brett
-
Matti Picus
-
Ralf Gommers
-
slobodan.miletic@endava.com