[Distutils] PEP425 - Wheel Binary Package Compatibility

[Paul Moore]

On 28 October 2014 10:13, Matthias Hafner <hafner87 at gmail.com> wrote:

(in reverse order, easiest to hardest :-))

> Why does that work on MacOS, btw? Are all library versions fixed there for
> one version of OSX?

I believe (I'm not 100% sure as I'm a Windows user not an OSX user)
that PyPI only supports binaries compatible with the official
python.org binaries of Python. So library version decisions are moot.
This means that homebrew or "the other ones" (sorry :-)) users can't
avail themselves of PyPI-hosted binaries yet, until the wider library
compatibility issues are resolved.

> So should the wheel be shipped with GLIBC, or the GLIBC version be specified
> in the wheel name (means to build a new wheel for each GLIBC version?).
> Also, maybe this is not the only library it is dynamically linked against?

Whatever solution is devised will need to be encapsulated in a set of
compatibility tags (see PEP 425) that capture what it means to be
"compatible". The wheels can then be uploaded with the correct tags,
and pip and other tools will automatically select the right one (or
report an error if no suitable binary exists and source is not
available).

> How should we deal with incompatibility of dynamically linked libraries?
> Doing "pip wheel numpy" on a Linux 64bit machine results in , which is
> linked dynamically against the GLIBC version installed on the build machine.

This is the big question. On Windows, we only really have to deal with
32-bit or 64-bit binaries, using the C runtime that the python.org
binaries use. So the architecture is sufficient and the ABI flag is
unused.

On OSX, as I say, PyPI has in effect artificially set up a similar
situation, by not considering anything other than python.org build
compatible binaries.

On Linux, nobody has really looked at the problem (beyond
acknowledging that it exists). We need a group of Linux experts with
an understanding of how all the various aspects of distribution,
kernel, libc, etc, work together to form a "compatible environment"
for binaries. As Linux has a strong culture of only building from
source or using distro packages, there doesn't seem to me (as an
outsider) to be much momentum around solving this.

Sorry I can't offer any more help or clarification than this.
Hopefully someone from the Linux side of things can offer some views
on how this can be moved forwards.

Paul