[Numpy-discussion] reducing effort spent on wheel builds?

[Charles R Harris]

On Thu, Jul 15, 2021 at 8:02 AM Charles R Harris <charlesr.harris at gmail.com>
wrote:

> I spent so much time updating the wheels builds to 64 bit BLAS mostly
> because
>
>
>    - I needed to actually understand how multibuild worked (and docs are
>    minimal).
>    - I don't know powershell (and docs are hard to find).
>    - I don't know azure works at a lower level (and docs are hard to
>    find).
>
>
> And I thought it would all be drop in easy :) What that indicates to me is
> that we could use a build expert and simpler infrastructure. Anaconda no
> doubt has people far more knowledgeable about such matters than I am.
>
> As to the twitter thread, what is the proposal(s)? I'm perfectly happy to
> leave support for more exotic platforms to people who have the hardware and
> make use of it, but no doubt such people have the opposite problem of not
> knowing much about NumPy.
>
> Chuck
>
> On Thu, Jul 15, 2021 at 4:22 AM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>> Hey all,
>>
>> This whole thread is quite interesting:
>> https://twitter.com/zooba/status/1415440484181417998. Given how much
>> effort we are spending on really niche wheel builds, I’m wondering if we
>> should just draw a line somewhere:
>>
>>    - we do what we do now for the main platforms: Windows, Linux (x86,
>>    aarch64), macOS, *but*:
>>    - no wheels for ppc64le
>>    - no wheels for Alpine Linux
>>    - no wheels for PyPy
>>    - no wheels for Raspberry Pi, AIX or whatever other niche thing comes
>>    next.
>>    - drop 32-bit Linux in case it is becoming an annoyance.
>>
>> This is not an actual proposal (yet) and I should sleep on this some
>> more, but I've seen Chuck and Matti burn a lot of time on the numpy-wheels
>> repo again recently, and I've done the same for SciPy. The situation is not
>> very sustainable and needs a rethink.
>>
>> The current recipe is "someone who cares about a platform writes a PEP,
>> then pip/wheel add a platform tag for it (very little work), and then the
>> maintainers of each Python package are now responsible for wheel builds (a
>> ton of work)". Most of these platforms have package managers, which are all
>> more capable than pip et al., and if they don't then wheels can be hosted
>> elsewhere (example: https://www.piwheels.org/). And then there's Conda,
>> Nix, Spack, etc. too of course.
>>
>> Drawing a line somewhere distributes the workload, where packagers who
>> care about some platform and have better tools at hand can do the
>> packaging, and maintainers can go do something with more impact like write
>> new code or review PRs.
>>
>> <end of brainwave>
>>
>> Cheers,
>> Ralf
>>
>>
Let me add that distutils brings it's own set of problems. A better build
system could help make building wheels simpler on various platforms. But
then, someone would also need to become expert in that build system.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/numpy-discussion/attachments/20210715/17c486c6/attachment.html>