[Wheel-builders] C++ ABI v5

[Nathaniel Smith]

Hi Michael,

On Wed, Mar 9, 2016 at 3:49 PM, Michael Sarahan <msarahan at gmail.com> wrote:
> Howdy,
>
> Are you all aware of the looming move to the newer C++ ABI for GCC?
> http://developers.redhat.com/blog/2015/02/05/gcc5-and-the-c11-abi/
>
> This won't have humongous impact on Python as a whole, as it affects only
> C++ packages.  However, for those packages, I think we'll see breakage on
> new platforms.  Importantly, the upcoming Ubuntu 16.04 release will use that
> newer ABI (15.10 already does, but 16.04 is LTS).  Many distros are ending
> up recompiling their entire package libraries.

To check if I'm understanding the problem correctly... it's not that
C++ packages will be broken in general, right? The specific situation
where things break is:

1) you distribute a C++ library that's built using the old ABI, AND
2) your C++ library has a public API/ABI that involves some of parts
of the stdlib whose ABI changed (e.g. std::string), AND
3) your users running Ubuntu 15.10 (for example) want to compile their
own code against your C++ library's API/ABI

If all these things happen, then the users' builds will fail by
default (because the users' compiler will default to using the new
ABI, but your package is built using the old ABI), and your users will
have to add -D_GLIBCXX_USE_CXX11_ABI=0 to their compile lines.

And then AFAICT there are fundamentally two strategies one could use
to deal with this:
- modify your C++ library so that it exports both old- and
new-versions of its ABI from the same .so simultaneously. This is what
libstdc++ does, and what that blog post describes how to do. OTOH this
requires non-trivial modifications to the package source, and it's
unlikely that any distributor is going to try and go through and hack
up a giant C++ library to do this (since the cases we're worried about
here are like... Qt and LLVM, right? And now that I think about it I'm
not even sure whether Qt's ABI is affected, since they go to some
effort to avoid using standard library classes in their public ABI --
QString instead of std::string, etc.)

- make a build of your C++ library that just uses the new ABI, and
figure out some way to manage the resulting hassles involved in keep
track of the two builds -- can they be installed simultaneously? how
do you make sure that only systems with a new libstdc++ get the new
version of the library? or do you switch to distributing libstdc++
yourself and push everyone to the new ABI, even if they're on an old
distro? if you do, then can RHEL5 system gcc build against a newer
version of libstdc++, or would this mean that you need to start
shipping gcc as well?)

> Just curious - how (and when) will the manylinux docker image tackle this?
> We (Continuum) don't have a solid answer yet.  We are looking into a docker
> image similar to manylinux that uses a newer GCC that can output either ABI:
> https://github.com/ContinuumIO/docker-images/pull/20

The manylinux image doesn't really give much consideration to the
problem of distributing non-Python C++ libraries, at least so far... I
guess at some point there may be wheels that just contain LLVM or Qt,
but we're not there yet :-). In any case, the hard problems to me all
seem to be at the distro/package management level, not at the docker
image level -- I think once one has decided how one wants to solve
those problems, then the docker image part should follow?

-n

-- 
Nathaniel J. Smith -- https://vorpus.org