Re: [Numpy-discussion] reducing effort spent on wheel builds?

However, I think reducing the CI dedication (+maintainer effort towards that) to those minority platforms would inevitably increase the risk of numpy/scipy failing to work on them, and possibly risk not being installable on them. Would we want to drop them without having a positive assurance that the minority platform/package maintainers would keep checking for numpy/scipy health

Nothing blocks a platform champion from running the CI on their scipy fork. This way, the burden is on them, not on individual contributors (including first-time contributors who only deal with common platforms). чт, 15 июл. 2021 г., 15:22 Andrew Nelson :

I'd be +1 on reducing the number of wheels to reduce maintainer effort (esp. the 32 bit builds). However, I think reducing the CI dedication (+maintainer effort towards that) to those minority platforms would inevitably increase the risk of numpy/scipy failing to work on them, and possibly risk not being installable on them. Would we want to drop them without having a positive assurance that the minority platform/package maintainers would keep checking for numpy/scipy health; I guess one can make that choice when you know just how many times those exotic wheels are downloaded (modulo CI runs).

I'd like to see sdists being kept on PyPI. Those minority platforms may keep patching scipy/numpy source to keep it installable from source, and pip knows where to find it. I suspect the majority platforms will have all the wheels they need, so only the small set of exotic platforms will use sdist, with those users being more capable of dealing with the aftermath.

On Thu, 15 Jul 2021 at 20:53, Evgeni Burovski wrote:

...

FWIW, here's a big fat +1 from me for spreading the load. I'd even advocate for trimming the CI and going for "We officially support this (small) list of platforms and gladly accept patches for anything else as long as they do not break the officially supported ones". ISTM the list of supported platforms (both wheels and CI) has grown too large and seems to only grow in time.

The user requests are perfectly understandable, everyone wants to be in the press-the-button-and-it-works world, but the maintainer/RM effort seems to be crossing over to being too much.

A perfect picture in this regard is probably what we had a couple of years ago with Gohlke Windows wheels. Christoph was doing a great job maintaining the wheels and sending patches. For more exotic platforms, there simply has to be a champion. Or if some entity wants to fund the work for some platform, great --- this should also live somewhere outside of the main repo/CI/pypi.

Whether to upload sdists to PYPI --- this is a bit orthogonal, but indeed maybe it's best to only upload the small set of wheels indeed. sdists are just stored in the GH releases and it's not a big deal to grab them from GH (or even pip install from the release tag directly). This would probably improve user experience too (no more cryptic errors from attempting to compile from source on an unsuspecting user).

My 2p,

Evgeni

On Thu, Jul 15, 2021 at 1:22 PM Ralf Gommers wrote:

...

Hey all,

This whole thread is quite interesting:

https://twitter.com/zooba/status/1415440484181417998. Given how much effort we are spending on really niche wheel builds, I’m wondering if we should just draw a line somewhere:

...

we do what we do now for the main platforms: Windows, Linux (x86,

...

no wheels for ppc64le no wheels for Alpine Linux no wheels for PyPy no wheels for Raspberry Pi, AIX or whatever other niche thing comes next. drop 32-bit Linux in case it is becoming an annoyance.

This is not an actual proposal (yet) and I should sleep on this some more, but I've seen Chuck and Matti burn a lot of time on the numpy-wheels repo again recently, and I've done the same for SciPy. The situation is not very sustainable and needs a rethink.

The current recipe is "someone who cares about a platform writes a PEP,

aarch64), macOS, *but*: then pip/wheel add a platform tag for it (very little work), and then the maintainers of each Python package are now responsible for wheel builds (a ton of work)". Most of these platforms have package managers, which are all more capable than pip et al., and if they don't then wheels can be hosted elsewhere (example: https://www.piwheels.org/). And then there's Conda, Nix, Spack, etc. too of course.

...

Drawing a line somewhere distributes the workload, where packagers who

care about some platform and have better tools at hand can do the packaging, and maintainers can go do something with more impact like write new code or review PRs.

...

<end of brainwave>

Cheers, Ralf

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion

-- _____________________________________ Dr. Andrew Nelson

_____________________________________ _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion