On 21/12/22 06:41, Andrew Nelson wrote:

On Wed, 21 Dec 2022 at 07:29, Andrew Nelson wrote:

In https://github.com/scipy/scipy/issues/17630 we're seeing some issues (amongst others) with the scipy test suite that uses numpy==1.24.0 on musllinux.

I was wondering if numpy would like to add a musllinux CI entry? This would help prepare for a musllinux wheel.

From local testing in a docker container there are currently several test fails and at least one segfault on numpy main with `musllinux_x86_64`.

Maybe we should have a scientific-python wide discussion of what platforms we wish to support, like NEP 29 for python versions. The NEP should include some mechanism for adding new platforms. It doesn't make sense to me to be testing something on SciPy's CI if we are not testing it in NumPy. As for musllinux: if we decided to go ahead I think we should proceed cautiously with CirrusCI. We are still learning its limitations. Maybe we should add the CI run to github actions instead? Matti

On Wed, Dec 21, 2022 at 12:04 PM Andrew Nelson wrote:

On Wed, 21 Dec 2022 at 20:40, Matti Picus wrote:

...

Maybe we should have a scientific-python wide discussion of what platforms we wish to support, like NEP 29 for python versions. The NEP should include some mechanism for adding new platforms. It doesn't make sense to me to be testing something on SciPy's CI if we are not testing it in NumPy.

Over time there have been multiple requests for musllinux wheels for scipy. We created a CI job to check that the package would be somewhat useable on distros using musl. scipy (and probably other downstream projects) are waiting until numpy releases a musllinux_* wheel to make accompanying wheels. A side effect is that testing on other platforms helps uncover portability bugs, etc. One might argue that musllinux wheels would probably be more popular than platforms like s390x. musllinux is also straightforward to test on CI. In https://github.com/numpy/numpy/issues/20089 there's a reasonable level of support for making musl wheels.

I agree with what Andrew says here regarding popularity. Let me also link the SciPy mailing list discussion on this: https://mail.python.org/archives/list/scipy-dev@python.org/message/QPLZNSK7S.... What I would like to do is separate the questions of whether to run it in CI vs. whether to support it with wheels on PyPI. The former is clearly useful, probably low overhead (once it works), guards against bugs that need fixing either way, and is a decision that can be reversed. The latter is a much higher-impact decision, less easy to reverse, and that is indeed worth thinking about across projects as Matti suggests. Of all the more niche platforms, I'd say testing with Musl is one of the most interesting ones at the moment.

As for musllinux: if we decided to go ahead I think we should proceed

...

cautiously with CirrusCI. We are still learning its limitations. Maybe we should add the CI run to github actions instead?

I suggested cirrus-ci because it's relatively straightforward to run tests inside docker containers (plus I know how to do it). From https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container it should be possible to do something similar on github actions.

+1 for Cirrus, because it remains common with SciPy and Andrew is happy to do the work. This is easiest for now and it's fast. If really needed (I hope not), it can always be moved later. Cheers, Ralf