[Distutils] 400 Client Error: Binary wheel for an unsupported platform

[David Cournapeau]

On Tue, Jul 7, 2015 at 7:02 PM, Donald Stufft <donald at stufft.io> wrote:

> On July 7, 2015 at 10:22:55 AM, Nick Coghlan (ncoghlan at gmail.com) wrote:
> > On 8 July 2015 at 00:07, Antoine Pitrou wrote:
> > > On Tue, 7 Jul 2015 23:53:59 +1000
> > > Nick Coghlan wrote:
> > >> Unfortunately, the compatibility tagging for Linux wheels is currently
> > >> so thoroughly inadequate that even in tightly controlled environments
> > >> having a wheel file escape from its "intended" target platforms can
> > >> cause hard to debug problems.
> > >
> > > I'm not sure what you're pointing to, could you elaborate a bit?
> >
> > That was a reference to a case of someone building for Debian (I
> > think), and then having one of their wheel files end up installed on a
> > CentOS system and wondering why things weren't working.
> >
> > > For the record, building against a well-known, old glibc + gcc has
> > > served the Anaconda platform well.
> >
> > The key problem is that there's no straightforward way for us to
> > verify that folks are actually building against a suitably limited set
> > of platform APIs that all Linux distros in widespread use provide.
> >
> > And when it inevitably fails (which it will, Python and PyPI are too
> > popular for it not to), the folks running into the problem are
> > unlikely to be able to diagnose what has happened, and even once we
> > figure out what has gone wrong, we'd be left having to explain how to
> > blacklist wheel files, and the UX would just generally be terrible
> > (and the burden of dealing with that would fall on the pip and PyPI
> > maintainers, not on the individual projects publishing insufficiently
> > conservative Linux wheel files).
> >
> > That's why the platform override tags are such a nice idea, as it
> > becomes possible to start iterating on possible solutions to the
> > problem without affecting the default installation UX in the near
> > term.
> >
> >
>
> pip 7+ actually has the UI for blacklisting binary packages now, primarily
> to
> ask "no don't build a wheel for X", but it also functions to ask *not* to
> accept wheels for a particular project from PyPI.
>
> In my mind, the biggest reason to not just open up the ability to upload
> even
> generic linux wheels right now is the lack of a safe-ish default. I think
> if
> we added a few things:
>
> * Default to per platform tags (e.g. ubuntu_14_04), but allow this to be
>   customized and also accept "Generic" Linux wheels as well.
> * Put the libc into the file name as well since it's reasonable to build a
>   "generic" linux wheel that statically links all dependencies (afaik),
> however
>   it does not really work to statically link glibc. This means that even if
>   you build against an old version of glibc if you're running on a Linux
> that
>   *doesnt* use glibc (like Alpine which uses MUSL) you'll run into
> problems.
>
> I think that it is entirely possible to build a generic linux wheel that
> will
> work on any Linux built the same libc* of the same or newer version,
> however I
> think that you have to be careful if you do it. You have to ensure all your
> dependencies are statically linked (if you have any) and you have to ensure
> that you build against an old enough Linux (likely some form of CentOS).
>
> * Side question, since I don't actually know how a computer works: Is it
> even
>   possible to have a CPython extension link against a different libc than
>   CPython itself is linked against? What if static linking is involved
> since
>   there are non glibc libcs which actually do support static linking?
>

You can use versioned symbols to manage some of those issues (
https://www.kernel.org/pub/software/libs/glibc/hjl/compat/). Some softwares
even include their own libc, but this is getting hair fast.

The common solution is to do what Antoine mentioned: build on the lowest
common denominator, and reduce the dependencies on the system as much as
possible. To be honest, glibc is rarely your problem: the kernel is
actually more problematic (some common python packages don't build on
2.6.18 anymore), and C++ even more so. E.g. llvm 3.6 will not build on gcc
4.1 (the version of centos 5), so you need a new g++ which means a new
libtstdc++.

I am biased, but that's the kind of things where you may want to work with
"professional" providers with people paid to work on those boring but
critical issues.

David

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150708/cd29d798/attachment.html>