I'd be +1 on reducing the number of wheels to reduce maintainer effort (esp. the 32 bit builds). However, I think reducing the CI dedication (+maintainer effort towards that) to those minority platforms would inevitably increase the risk of numpy/scipy failing to work on them, and possibly risk not being installable on them. Would we want to drop them without having a positive assurance that the minority platform/package maintainers would keep checking for numpy/scipy health; I guess one can make that choice when you know just how many times those exotic wheels are downloaded (modulo CI runs).

I'd like to see sdists being kept on PyPI. Those minority platforms may keep patching scipy/numpy source to keep it installable from source, and pip knows where to find it. I suspect the majority platforms will have all the wheels they need, so only the small set of exotic platforms will use sdist, with those users being more capable of dealing with the aftermath.

On Thu, 15 Jul 2021 at 20:53, Evgeni Burovski <evgeny.burovskiy@gmail.com> wrote:

FWIW, here's a big fat +1 from me for spreading the load. I'd even
advocate for trimming the CI and going for "We officially support this
(small) list of platforms and gladly accept patches for anything else
as long as they do not break the officially supported ones". ISTM the
list of supported platforms (both wheels and CI) has grown too large
and seems to only grow in time.

The user requests are perfectly understandable, everyone wants to be
in the press-the-button-and-it-works world, but the maintainer/RM
effort seems to be crossing over to being too much.

A perfect picture in this regard is probably what we had a couple of
years ago with Gohlke Windows wheels. Christoph was doing a great job
maintaining the wheels and sending patches. For more exotic platforms,
there simply has to be a champion. Or if some entity wants to fund the
work for some platform, great --- this should also live somewhere
outside of the main repo/CI/pypi.

Whether to upload sdists to PYPI --- this is a bit orthogonal, but
indeed maybe it's best to only upload the small set of wheels indeed.
sdists are just stored in the GH releases and it's not a big deal to
grab them from GH (or even pip install from the release tag directly).
This would probably improve user experience too (no more cryptic
errors from attempting to compile from source on an unsuspecting
user).

My 2p,

Evgeni

On Thu, Jul 15, 2021 at 1:22 PM Ralf Gommers <ralf.gommers@gmail.com> wrote:
>
> Hey all,
>
> This whole thread is quite interesting: https://twitter.com/zooba/status/1415440484181417998. Given how much effort we are spending on really niche wheel builds, I’m wondering if we should just draw a line somewhere:
>
> we do what we do now for the main platforms: Windows, Linux (x86, aarch64), macOS, *but*:
> no wheels for ppc64le
> no wheels for Alpine Linux
> no wheels for PyPy
> no wheels for Raspberry Pi, AIX or whatever other niche thing comes next.
> drop 32-bit Linux in case it is becoming an annoyance.
>
> This is not an actual proposal (yet) and I should sleep on this some more, but I've seen Chuck and Matti burn a lot of time on the numpy-wheels repo again recently, and I've done the same for SciPy. The situation is not very sustainable and needs a rethink.
>
> The current recipe is "someone who cares about a platform writes a PEP, then pip/wheel add a platform tag for it (very little work), and then the maintainers of each Python package are now responsible for wheel builds (a ton of work)". Most of these platforms have package managers, which are all more capable than pip et al., and if they don't then wheels can be hosted elsewhere (example: https://www.piwheels.org/). And then there's Conda, Nix, Spack, etc. too of course.
>
> Drawing a line somewhere distributes the workload, where packagers who care about some platform and have better tools at hand can do the packaging, and maintainers can go do something with more impact like write new code or review PRs.
>
> <end of brainwave>
>
> Cheers,
> Ralf
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@python.org
https://mail.python.org/mailman/listinfo/numpy-discussion